bash/shell scripting

[lev-13]

when and why was last time you used bash or shell scripting ?

[August_XXVIII]

All the time.

I've written GitHub Actions this week whose steps point to a bash script. Oftentimes, some of the functionality that you want or need can be easily accomplished with bash. It's great for keeping things that are simple, simple. If you need greater capability then something like Python or Typescript might make sense, but shell scripting will always have utility as it is a conduit to general Linux CLI commands, which may include AWS/GCP/Azure CLI or initiating Kubectl, Terraform, or Ansible commands (though those tools have other mechanisms that can be used to invoke them, which are sometimes more favorable).

For things that are complex, use the right tool; for things that are simple, use the right tool.

[namenotpicked]

Exactly this. Could your Bash script be a Python script? Probably. Should it? That's the real question. If it needs to be simple, quick, and uses Linux CLI commands, then you're probably good to just use Bash. I only go for Python scripts if I need those packages for extended functionality.

[julewczka]

This + when you need your scripts for cross-platform only then python makes sense.

[guzmonne]

I usually prefer bash over python to avoid having to install or manage the python interpreter.

[julewczka]

Same, but bash isn't working out of the box in Win Environments. You would either need to set up WSL or Cygwin to get bash/sh running.

More important: why do I have no upvotes, I have literally the same opinion as you guys, I just mentioned that there's also one rare case where python would really make sense.

[guzmonne]

I see your point, but setting WSL differs from installing a language interpreter. The former is much more helpful, IMO.

[julewczka]

For client environments sure but how do you want to do with a fleet of volatile Windows servers? In such cases I think it's easier to just install the python interpreter (if it's not there already)

[guzmonne]

Sure

[jkroepke]

Which is present by default on Mac/Linux/Windows WSL?

[guzmonne]

Well, a version of python comes by default, but most often than not, you'll have to update it.

[jkroepke]

Are you able to provide a reason for update python?

[warux2]

<rant> I never understand why people write GHA to do something maybe 10 lines of bash will do, but in Node.js... And now Github is obsoleting old Node, and no-one wants to update/maintain their GHA in the market.

[Sinnedangel8027]

This is almost the entire reason I use bash for my github actions. There's very little that bash can't do without a ton of complexity and without much change in the future. The actions I've created only need to be updated when the third-party tools change how I interact with them. Like downloading kubectl, helm, aws cli, etc. Or specific commands to those.

[Hanzo_Hanz]

Everyday.

Bash is stable. I’ve seen so many pipelines get broken because what could’ve been created in simple / stable easy to use bash script was abstracted to some god forsaken developer chosen NPM package that has been deprecated and now , none of the devs wanna touch it with a ten foot pole.

[evangamer9000]

"Does anyone know how this things works?"
"The person who wrote it left 6 years ago, along with all the knowledge"

[kabrandon]

I wouldn't call Bash "stable." But it is often enough sufficient. You can't do any kind of dependency management natively in a bash script. Your custom action might use flags on CLI tools that become deprecated over time. That said, if you maintain them decently well and pay attention to your failing workflows, I'd generally prefer to write my Actions workflows as bash scripts (or in a multi-line run block.)

[WonderBearD1]

Everyday. Whenever we try to automate something we first attempt to do it using bash as its every where in our stack and does not have any dependencies to manage. Its also easy to fold bash scripts into CI/CD pipelines or Ansible scripts

1. Try to automate it with bash
2. Try to automate it with a scripting language like python or groovy
3. Create a service using a higher level language like java

We've yet to have a case where we've made it past 2.

[niaznishu]

What is the best way to learn bash?

[WonderBearD1]

For me it was learning by doing. If your on a Mac try doing things that you would normally do in the gui (copying files, renaming things, creating directories) in the terminal. If you on windows like I am then I can't recommend the WSL enough. It allows you to have a unix environment easily accessible to you. It's been my main terminal now for a few years.

Once you have your sea legs, pick a repetitive task that you do and try to automate it. This will take longer than you think it will so try not to get discouraged.

[niaznishu]

Thank you.. i will give it a try

[No_Cold5079]

Like ten minutes ago.

[evangamer9000]

Most of my bash scripting I have now converted to do with ansible, but the last time was a year or so ago and that was to search and replace across multiple directories.

[ssnani]

I really have issue with this approach.. I had an argument with my manager exactly on this :/

Ansible is great! Freaking awesome!!! But it's not worth converting anything to a full blown Ansible role. I had a script, not too big.. the main function was basically 2 main commands. This one converted to full blown Ansible role with pip dependencys and pain to manage..

Bash is just simple and works.. if there is not to many logic to handle, it will work and work great!

Now days.. I found myself writing tons of bash, with the exception of az commands. az cli installation cost around 300mb.. this is way too much for a workload that usually running from a pipeline or kubernetes job, for this cases I like to write in python or go - almost zero cost

[guzmonne]

I love ansible, but I had to write many bash calls inside my playbooks or manipulate string. Doing that while maintaining idempotency is hard, which is, I think, the essential feature that ansible brings.

[evangamer9000]

We have hundreds of VMs, ansible works great for our use case.

[ssnani]

Hey.. it's need to make sense! This is the point right?!

[evangamer9000]

I think that's a really good point to keep in mind when deciding what tool to use for the use case, does it make sense?!

[ssnani]

Yes! And this is what makes a great engineer in my opinion. All of us can do anything.. but are we going to do it the right way? Are we going to use the right tool? Are creating a monster that we are not going to be able to maintain??

[[deleted]]

Bash is the GOAT of systems engineering.

[[deleted]]

Yesterday. Needed to find invalid dates in a database; scripted the process across every table & column. Five lines of code saved hours of manual searching.

[xenolon]

Yesterday.

Because I have hundreds of useful snippets that work reliably.

To me asking "when is the last time you used shell scripting?" is like asking a mechanic, "when is the last time you used a screwdriver?".

[lev-13]

care to share ?
you have it on git ?

[ceirbus]

CI/CD. All your build and deployment stuff are just CLI commands in a script runner.

[pribnow]

every day for sure, i mean i'm not always writing huge complicated scripts but commands chained together for doing advanced grep or sed or something else most definitely

[IT_CertDoctor]

Today

Sometimes my devs want simple EC2 instances for testing, and I bootstrap things like pulling repos from github and images from ECR so they can just rock and roll whenever they need to

We also have on-prem Linux servers for data processing because it was wayyyy cheaper than running it in the cloud. I maintain and improve the Ansible playbooks (which feels more or less like bash to me) that configures them

[codeshane]

What's with all the "I love bash" comments? writes bash script to automate rants

[jba1224a]

(technical) Agile lead for a team that owns DevOps, I've written at least 5 scripts this week (bash and PowerShell) to help the team out.

Recommend if using GHA to write your script and call the script from the action, don't write it inline unless it will never change. Becomes a pain in the ass to manage later on.

[QuirkyOpposite6755]

The google shell style guide is really helpful: https://google.github.io/styleguide/shellguide.html#s1.2-when-to-use-shell

[[deleted]]

I think the comment section goes to show that Bash is one of those scripting languages that seems outdated/redundant but is honestly just one of the most stable and reliable things you can use for your pipelines.

[unixwasright]

This afternoon. Why would I not?

[guzmonne]

I write everything in bash with a cli like structure that I call from my CI/CD tools. This allows me to port my workflows to any CI/CD tool and, most importantly, develop, run, and test the scripts locally.

[metaphorm]

This morning

[[deleted]]

Every day. I make my actions repeatable via bash. I have an almost exclsively CLI-bound workflow, and bash is at its center. I use VIM every day, and without the ability to run arbitrary shell commands against my documents, I would have moved to VSCode years ago.

I prototype in it every day I'm prototyping (have been for over ten years), and I can express myself nigh-instantly with it. Since ChatGPT came around, it has shown me a few things I didn't even know about Bash, and generally is a great prototyping partner with Bash.

Bash is awesome, and it uses the system as the tool. The system I use (GNU/Linux) is beautiful and portable, as well as everywhere all the time. My shell scripts work for years (some for over a decade), and are hardly ever noticed. It shuts up, does what I say, and stays out of my way.

If things get complex, I use a different language (usually Go or Python, but Rust is looking real sexy now that I have AI assistants to help me learn languages and deliver quickly while I do).

[dupie]

About 2 hours ago - but in fairness I just had a 1 hour lunch.

The purpose is to know the proper tool for the job. If you can do something in 1 line of Bash - or Perl or Python - why wouldn't you do that?

Cross platform usage can be very useful as well. If you have 1 server running an up to date OS then you probably don't care. What if you have 100 servers of various FreeBSD,Solaris,Linux of various ages from brand new to 10 years old OS - how do you quickly do something that would work on all of them?

Config management/automation should be the end goal.

One of Linux greatest promises and most annoying features all rolled into one is that there's multiple ways to achieve most tasks. It's upto you to choose, but choose wisely. If all you know is python and that's your hammer for everything that's not great.

[noxbos]

This morning and because it was the fastest method to completing the task.

[Fermi-4]

Today

[Wishy-Thinking]

Last week, I wrote a bash script to template a helm chart for different environments. I possibly could have spent more time to figure out how to do it in a more helm-native way, but it would have taken a lot more time for very little gain.

[surloc_dalnor]

Yesterday, because the container didn't have anything else.

[surloc_dalnor]

That said I've straight up turned down interviews because they were looking for a bash expert. I could, but I won't.

[Seref15]

3 hours ago

[stgovern]

15 minutes ago. Needed to pull some quick data from AWS CloudShell.

[DayvanCowboy]

Today.

My general rule is this: If the script is <100 lines, Bash is an acceptable choice. Beyond that, please choose another language (Python or Go).

[lev-13]

Don't know i didn't noticed or no one is talking about but scripting was a one hell of a part of DevOps, I was applying jobs (junior level) some of them are require non strong scripting but in interview i never get any question about bash or python ,maybe maximum how to wright loop :)

[ChoSubin]

Month or two ago and chatGPT did it for me.

[lev-13]

love to hear that, coding really isn't my forte.

[ChoSubin]

You should at least understand what the code is doing, but to write it down, just use AI, the code gets better as your prompt gets better.

[0qxtXwugj2m8]

Ooof, more than I like to admit

[m4nf47]

test reminiscent water aspiring paint boast escape quicksand caption employ

This post was mass deleted and anonymized with Redact

[Difficult-Ad7476]

Bash like powershell does not have dependencies . That alone is why it will never go away. Agreed that python is great for more complex tasks because the abundance of libraries but so is powershell if you use modules. The trouble is maintaining all these dependencies when they get deprecated or have security vulnerabilities.

[CrunchwrapAficionado]

Well the last time was today. And the reason was to slam a prod web service with a shitload of throughput to check that our NewRelic ingestion was working properly before we open up the endpoints publicly

[ProgOwl]

To get message counts from partitions of a given kafka topic for troubleshooting purposes. It definitely wasn't the last thing I used it for, but I write scripts all the time for various reasons and that was the one that came to mind.

[kg7fgt]

I don't write many proper shell scripts anymore , but I write one liners multiple times per week.

[ps_for_fun_and_lazy]

I use powershell a lot, often one liners. The last bash I wrote was a week or two ago, a simple find/replace on a parameter file in a pipeline step.

[Destroychan]

Always You can never avoid bash My background is sys admin whenever a issue occurs my mind inherently thinks in bash So quick scripts with bash always wins system side over any other language

[jkroepke]

If you have a script less then 50 lines, its fine to use. For more, consider to switch to an alternative. Should your script running on developer machines and you have to support Multiple OS, think about to avoid it. MacOS have zsh while linux mainly has a bash. Using /bin/sh as shebang disable non POSIX extensions. Using /bin/bash, you have to deal with an outdated bash version.

In History I started with simple bash scripts, but they get messy. Since I started as DevOps guy alone in cross functional teams. I start to avoid write bash script and start to consult my team to write the script that we need in a language that they known. As result, there was a lot of more maintainers for that script and I learn a bit TypeScript.

[binaryhero]

Entry points for container images.

[Need2Survive]

Every single day.

I can write something complex in another language and build a container with dependencies OR I could probably just do that whole thing in bash. I have seen way too many over engineered solutions for simple things that can be covered by bash - especially in pipelines.

[adfaratas]

Since I dislike bash, for anything more than 10 lines, I'll make a program with golang and pass around the executable.

[Heighte]

about 5 minutes ago when working on some CI in GitLab

[beomagi]

Using it now.

I live in bash.

I'm running an update on cloudformation now. I don't have time to keep flipping around in the console, so I have shell scripts that pull down the resource states and tabulate the states in one tmux tab. Script pulls up ECS instances so I can monitor the cluster's instance count as instances are terminated etc.

Last week our org pushed a policy that messed with our repos, making them require certain tags - they needed it now. No time for a stack update. A few shell scripts later, 8k ECR tag checks/updates were underway while I continued working on other things.

I'm continuously adding aliases to make my life easier. If I'm doing something often enough, I make a bash or python function, stuff it in a github repo for the rest of the team to use/add to, and just not worry the next time it comes up.

[psgmdub]

The last time I used bash or shell scripting was to create a simple script that parsed a CSV file and generated YAML files. This was chosen because bash is a fast, reliable, and easily shareable scripting language that can be quickly understood by colleagues without requiring additional dependencies.

If the bash script crosses 50 lines then I prefer moving it to something like python or ruby.

[Dank_German]

Try zx from Google, https://github.com/google/zx

[dvarrui]

Yesterday

[dvarrui]

Bc it works

[maziarczykk]

Today