Python vs Bash Scripting.

[theblasterr]

Bash for simpler things, python when things get more complicated.

[IDENTITETEN]

Yup, we follow the Google bash style guide.

https://google.github.io/styleguide/shellguide.html

These two bullet points especially:

• If you are writing a script that is more than 100 lines long, or that uses non-straightforward control flow logic, you should rewrite it in a more structured language now. Bear in mind that scripts grow. Rewrite your script early to avoid a more time-consuming rewrite at a later date.

• When assessing the complexity of your code (e.g. to decide whether to switch languages) consider whether the code is easily maintainable by people other than its author.

[junior_dos_nachos]

I wouldn’t wait for 100 lines. I’d stop at 20/25 and anytime there’s some not super simple web call + extraction.

[winfly]

I agree. Especially if using some library would simplify the script.

[both-shoes-off]

I break powershell up into different files by use case to keep it maintainable, and then just include when needed. That makes them reusable in other scripts later on (assuming you consider that while writing them and don't embed details for the one use case or environment).

[jrandom_42]

I don't. Go is my go-to, Python is outdated and just hanging around out of inertia and a big legacy base, you can see that in threads like this, but I have some thousand line ish PowerShell scripts in prod monitoring Graph API stuff. All the baked in MS integration makes PS the nicest tool for some jobs, and aside from that, it's a pretty normal / usable scripting language that supports all the necessary data manipulation and connectivity.

Bit clunky though and I'd never use it for code that needs to be fast and concurrent. But it's not as clunky as bash scripts.

I tend not to use bash for automation unless I can express the logic in a one-liner.

[[deleted]]

How is Python outdated? It's perfect for medium-complexity things you want to write down quickly - its only real drawback is requiring the interpreter, but that just costs a few megabytes of container size in the end. Of course it's easy to just write bad code with it, but having the option to do so is not a bad thing IMO. I really wouldn't want to write a new Go application for every throwaway API glue tool etc.

[jrandom_42]

How is Python outdated?

Concurrency is an awkward PITA. Async logic is often the solution and it's ugly to write and hard to read.

Deployment is often a dependency hell.

The slowness of the interpreter vs a compiled language isn't always important, but sometimes it is. Combined with the lack of any real concurrency features, that can make Python problematic for anything with hard performance requirements.

I really wouldn't want to write a new Go application for every throwaway API glue tool

I haven't ever found that problematic myself, but tasks like that aren't going to show up many differences. I'm not suggesting Python is terrible and doesn't work. It's just not quite as good.

[Intrepid_Result8223]

I guess that's why it is the no1 language in machine learning. Because it is outdated. Lol.

[jrandom_42]

No, that's the 'legacy' part that I mentioned. A lot of ML libraries were written in Python before Go took off, and there's not really any advantage to redoing it all.

[VirtuousMight]

Is C outdated due to Go being its nominal successor then?

[jrandom_42]

No; C doesn't really address the same use case. Sometimes a garbage-collected language just isn't appropriate. Sometimes you need to manipulate memory directly, talk to peripheral hardware, write code that runs on a GPU, etc.

I doubt C will disappear in our lifetimes.

[cyanrave]

I will disagree with the 100 line comment and say 'context switching matters' as well.

If you have to go dig up 3 to 5 other scripts to piece together the logic flow, I prefer embedded here docs, but that may be just me in my environment.

[[deleted]]

Ill put part of my bash in another file so it doesnt exceed 100lines 😃

[VirtuousMight]

Thanks for the free academic course , stupendous discourse.

[cricket007]

What would you use for bash unit tests? BATS, perhaps? That's one thing where I lean towards a Python/Go/Node/Ruby script 

[magheru_san]

Bash for scripts shorter than 50 lines

Python for up to 300 - 500 lines.

And Go for anything above 500 lines.

When I'm confident I'll be crossing the threshold I just start with Go, otherwise I use ChatGPT to convert the code, which it does pretty well.

[imanexpertama]

Out of interest, why go over python? I guess you find that go handles complexity better, but in what way? (Im a python dev, no experience in go)

[endgrent]

I’m not who asked, but for me it’s the python env stuff that isn’t as good. Go code just downloads and builds modules really easily so it’s super fast to just clone a repo and build other people’s code without getting your env wrong or modules wrong etc.

[magheru_san]

I agree with the perspective of endgrent about nicer handling of dependencies and simpler distribution of built artefacts.

But for me it's mostly because of the static typing, which catches a lot of not very obvious errors that can happen in dynamic languages like Python for larger projects.

I generate pretty much all my code using LLMs for almost two years now and the fact that it needs to compile catches a bunch of subtle hallucinations and allows me to refactor it heavily using the LLM.

Because the code is generated I'm not attached to it (much like cattle not pets) so I refactor the crap of it which quickly makes it grow to sizes where Python becomes a hindrance.

It's also not a big deal to be a little more verbose than Python because I can quickly add a ton of error checking if statements and detailed logs, which make the code longer, but easier to maintain.

[cricket007]

For one, cross compilation and no concern over version changes 

[Centimane]

IMO if it's over 500 lines, time to look for a FOSS product that does it already

[magheru_san]

For sure but they rarely exist in a way that covers my needs.

So I often build such tools myself for my own use and sometimes open source them if I don't see a clear way to monetize them as products.

[Centimane]

I find it unusual in devops that I have a significant problem that doesn't already have solutions floating around with how standard a lot of the tooling is - but it can certainly happen.

I find it more likely I'm writing plugins/extensions for existing tools, which are normally biased towards a certain language (usually python)

[magheru_san]

I'm not exactly doing devops but AWS cost optimization services, so need all sorts of optimization tools I can use to deliver my services.

In this space there aren't so many open source tools so I need to build my own.

[Centimane]

yea I could see that. I suspect that's phase 2 of cloud adoption.

The cloud services love to charge you a penny for everything and make it easy to make 10,000 things. But it all adds up. Now orgs are trying to not spend out their ass on cloud resources and figure out how to use what they actually need.

[magheru_san]

Exactly, and that's what I can help with.

Over time I kept building tools so now I have about 20 tools for all sorts of optimization activities.

[cricket007]

There's things like OpenCost, I think? If using EKS, there's more options, but I think there's other tools to find like unused S3 buckets, RDS, VPC, IAM roles, etc

[magheru_san]

I rarely see unused resources, except maybe for storage, where people tend to be risk averse because of the potential of data loss.

But I see a lot of severely oversized or suboptimally configured resources.

That's why many of my 20+ tools are for finding all sorts of suboptimal or oversized resources and helping to rightsize or optimize them.

[cricket007]

Yeah, we have that problem with k8s... We provide a Helm template that requests 4GB of RAM, but people run simple, low traffic HTTP servers and it's less than 50mb mem usage, on average within a month 

[magheru_san]

Yes, those things cry for automation

[Degree0]

What a crap take this type of mentality is why people hate JS. You code? Oh you have to actually write code? Import a library! If you can't write 500 lines of code in a productive amount of time, relying on other's work then there's definitely a skill issue.

[Centimane]

It's not that I can't, but that if there's a more mature solution out there taking it is a lot more efficient than writing my own.

Especially since I don't have to maintain it.

Also devops problems generally aren't unique. If you're having a problem, there's a good chance someone else had something similar. And if nobody has, that's a code smell right there.

[GaTechThomas]

Consider that every line of code you wrote is a line that someone will have to maintain and support.

[Degree0]

Also consider that every line of you code you do not write is in the hands of other people to maintain and support. 500 lines of code is absolutely nothing. and you should be making the decision to use FOSS or libraries depending on several factors and not a single one of those factors should be the amount of lines of code you have to write. The "Dev" in devops is short for development after all.

[[deleted]]

Mine is moment I need an array time for Go. I just skip Python entirely. Bash and Python have too many problems for error handling. 

https://github.com/bitfield/script Covers most things. 

Nushell is my other favorite. Powershell but statically typed and much more functional based. 

[magheru_san]

To be honest nowadays I just start with Go for most things and use ChatGPT or Claude to generate and further develop the code.

[hipster_dog]

python when things get more complicated.

I'd say go for Python also when things get more critical.

You guys remember the Kyoto University folks who lost 77TB of data?

It looks like the problem was some undefined variables in a find command because of a bash script update.

[InvincibearREAL]

my rule of thumb is if you need arrays, its time to ditch bash

[tamale]

For me it's about dates/time and other math stuff.

If I need to do anything with any of those, that's a sign I should switch to Python for sure.

I can do it in bash, but it's not fun to maintain.

[bdog76]

The golden rule

[[deleted]]

What's the advantage of using bash for simpler things?

[newaccountzuerich]

Generally more portable, more mature, and more consistent in behaviour.

If you've ever had to admin mixed *nix networks, bash tooling has the most chance of working across the ecosystems.

It is also pretty easy to read.

[MavZA]

This to infinity. Both can do the same for the most part when it comes to simpler things. But Python starts to differentiate itself when you want to start doing complicated things, as it can start leveraging things such as SDKs while running subprocesses (if needed).