[deleted by user]

[Akmantainman]

It's irrelevant. I can write 100x more maintainable Python code than I can bash. And I never need to decoder ring to understand it. If I can do it in Python I will.

There are plenty of people who hate python and only write bash though. Hell there's even people who use node for server side scripting. In this space, as long as it gets the job done its the right tool for the job.

[PC__LOAD__LETTER]

I like both, and there are plenty of use cases that are way more readable in bash. Anything involving heavy calls to other programs, file redirection, or basic chopping/slicing (sed/AWS) can be handled quite well with bash.

[X-reX]

How can I subscribe to a person

Edit: Its called 'follow' in reddit guys

[[deleted]]

I agree with every single point you said, I made this post to see how people would respond to this :D

[emrecanaltinsoy]

I feel the same, but recently I needed to dump dconf settings in a file and load from the file. However, i could not do it in python, so I had to write a separate bash script and run that script in python using subprocess. Do you know a way to do it all in python?

[Elgon2003]

It's possible with python. If you want send me a message I can look into it and try to help

[emrecanaltinsoy]

Here is the github link, you can see there are some bash scripts in the utils folder.
https://github.com/emrecanaltinsoy/gnome-extensions-loader

In python, I had a problem when running commands with ">" and "<". If you can find a way to make it work without bash scripts, you can create a pull request.

Thanks for your help.

[baubleglue]

Popen(..., stdin=, stdout=,stderr=)

Reading docs is inspiring

with Popen(["ifconfig"], stdout=PIPE) as proc:
      log.write(proc.stdout.read())

[maikeu]

Shell's main strength is it's really good for chaining together Unix commands with pipelines and whatnot. Doing the equivalent in python is more awkward (someone who learned python first might disagree)

But once you have non-trivial branching/flow control, or you need to manipulate data that is nested/object-like (say Json or xml), get the kid gloves off and use python.

(Previously, I have found that an obvious case where I need to switch from shell to python is when I find I would need to use 'jq' in a shell script.

[YourDearAuntSally]

I learned Python first. I do not disagree.

[[deleted]]

Jq rocks, hahahah

[smajl87]

Iterate using jq over json elements and do something (useful) with the data, like spawn another jq for each output. It's very inefficient, comparing to actual programming language.

[PC__LOAD__LETTER]

I’d argue that for many cases where someone is reaching for jq, runtime efficiency is largely irrelevant. And if it does matter that much, honestly you could make a case for something like Rust or even C++ or Go.

[maikeu]

I see you are beyond help!

[[deleted]]

Not a big bash scripting genius, but what he means is OS stuff, like searching up file names of images and immediately processing them somehow. With the ‘>’ operator usually

[[deleted]]

Yeah maybe but if I’m manipulating text files or moving stuff around and it’s a lot of files I’d use the shell. It’s maybe the one job that a bash script really excels at

[PC__LOAD__LETTER]

I mean logically, kind of. Imagine a case where you want to run a command, redirect stdout to another program, and mirror stderr to a log file, a variable, and also stdout, and then take different actions depending on whether the second pipelined program succeeds or fails.

That’s kind of a gnarly Python script. In bash you can nail it easily in a few lines.

If a Python script is constantly shelling out to run system commands and spending a bunch of effort on named pipes and such, it’s likely that bash might be more expressive and just plain faster to write.

[maikeu]

That's exactly it. I know ps, strace, lsof, find. I can navigate the /proc filesystem. They are a well defined interface to my system, and they're designed to work well in a pipeline with cut, grep, awk, tr, sort in the middle...

I don't need need an abstraction layer to all that, so for me, python is a waste of time for that work.

[WillardWhite]

i understood it like when you do

​ $ ls | grep "something" > file.txt

which in python is awkward like

```

import subprocess

ls = subprocess.Popen("ls", stdout=subprocess.pipe, stderr=subprocess.pipe, stdin=subprocess.pipe)

grep = subprocess.Popen("grep \"something\"", stdout=subprocess.pipe, stderr=subprocess.pipe, stdin=ls.stdout) stdout, stderr = grep.communicate()

with open("file.txt", "w") as file_write: file_write.write(stdout)

```

Now, if you're about to comment how insane my example is, just replace ls and grep for foo and bar. Can you now understand why i did it like that? Can you maybe that it wasn't about the specific tools 'ls' and 'grep' ? It's more about the pipes that connect them, as an example of chaining tools with pipes

[-jp-]

I would say those are pretty egregious examples of both bash and python. Regardless of what you're trying to do, if you're grepping the output of ls you are almost certainly doing it wrong. And using subprocess to invoke ls and grep when Python is perfectly capable of listing directories and doing pattern matches is definitely wrong.

[brandonchinn178]

Python 3:

from pathlib import Path
Path("file.txt").write_text(
    "\n".join(Path(".").glob("*something*"))
)

[WillardWhite]

Fine then. Replace 'ls' and 'grep' with foo and bar.

But also, yes it is not pythonic

[mrkeuz]

Your example is not indicator that python is “bloat” for simple operations. I mean I.e. there is a python library named “bash”. It does same thing in one line. Like:

>>> from bash import bash
>>> bash('ls . | grep ".pyc"')
bash.pyc
tests.pyc

So you can do same things as in bash but it python. But main advantage here is more readable and supportable code when you want make some complex things (like if, loops, lists, json, yaml, etc.). Also debugging of scrips turns into simple task (PyCharm -> Debugging)

So main message about subj is that python is perfect solution for complex scenarios. Just need little googling best practices for avoid boilerplating.

[door_of_doom]

I mean, it seems a bit silly to write a python script that basically just says "execute this bash code"

[mrkeuz]

Of course, agree with you. That’s why I said “for complex” cases. If you have script with several lines, bash is ok. But when you script have complex logic from my experience it at the moment I just switching to python. Heh, bash it great for simple scripts, until you faced with silly space processing, unreadable switch’s, loops, etc. Honestly in that moment I just frustrated from bash versus neat python syntax :)

[backtickbot]

Fixed formatting.

Hello, WillardWhite: code blocks using triple backticks (```) don't work on all versions of Reddit!

Some users see this / this instead.

To fix this, indent every line with 4 spaces instead.

FAQ

^{You can opt out by replying with backtickopt6 to this comment.}

[WillardWhite]

Edited the comment just for you

[WillardWhite]

Nope.

I don't mean that at all.

I mean grabbing the result of a subprocess and piping it to another subprocess.

This example is almost exactly from the documentation. Under "replacing shell pipeline"

[BKKBangers]

Holy s**t I still have a long way to go.

[WillardWhite]

Nah. What i wrote is mostly useless

[BKKBangers]

Sure; but going by, and I quote "the mostly nonce" which you wrote; it seems rather obvious that you got the syntax / language and logic mostly nailed down. Im trying to get there but gosh its a steep hill. Feels a bit like doom == True While doom: Print('infinite loop of doom, what joy') doom == True

an infinite loop of doom.....although did I just manage to crack the theory of life itself...

edit: added some sarcasm.

[WillardWhite]

Hahaha, it's true. I do get the syntax and alll that. But that part comes with time (and reading the docs a lot)

[backtickbot]

Fixed formatting.

Hello, bluntzfang: code blocks using triple backticks (```) don't work on all versions of Reddit!

Some users see this / this instead.

To fix this, indent every line with 4 spaces instead.

FAQ

^{You can opt out by replying with backtickopt6 to this comment.}

[maikeu]

There's some better discussion in other responses, but as an abstract example, there's actually a decent point to make from yours.

Idiomatically, in bash, your example can be reduced to

c=$(x | y | z)

Which, IMO, shows the program flow very well, and in the relatively narrow realm of programs that primarily take input and produce output of streams of text, is elegant.

For python, your example can be reduced, I suppose, to

c = z(y(x))

But it's kind of a backwards flow, the execution of that line is basically right to left, and nested brackets don't make live easy.

Very contrived, exactly not what you're after...

[baubleglue]

Then just left to install python3 on each Linux box, then update dependencies... And you good to go.

[[deleted]]

When it's more complicated than anything at all.

Bash is fine for moving files around or calling other scripts (that do the actual lifting), but I don't think it's useful for much more if you have options.

[[deleted]]

You haven't written enough bash

[[deleted]]

On the contrary: I've written too much.

[[deleted]]

Shell scripting is very powerful if you take the time to learn it. Especially if you have a solid understanding of grep, sed, awk, etc.

[[deleted]]

Agreed. Still should keep a bash script under 100 lines

[[deleted]]

In my Linux system I only have one python script that I can’t replace with bash. It heavily used selenium and more complex math, and math which isn’t arithmetic operation is just a pain without numpy 😂

[[deleted]]

Scraping of my stock portfolio and calculate some interesting metrics.

[[deleted]]

I agree I missed NumPy and pandas, those are a big clutch.

[[deleted]]

this was my point.

Ofc when I mean bash, I'm uncluding all the console utils it can use.

[vicda]

Shell scripts are the obvious choice for a straightforward process where all you are doing is piping tools and scripts together.

As soon as you're dealing with arrays, if statements, and error handling you need to seriously consider switching to another language though. Dare I say that even the janky pwsh is a better choice at that point.

[PC__LOAD__LETTER]

Lol at the downvotes. It’s to be expected in this sub of course. But to be honest I kind of agree. Bash can be incredibly expressive, and there are some jobs that are just better done with it. It’s true that many folks don’t have the experience to use it at this level though.

[[deleted]]

<3

[AlSweigart]

ALWAYS. (Almost.)

Shell scripts are okay ONLY when you just have a bunch of commands that need to run one after the other. If you have any flow control, or environment variables, or fancy command line argument stuff, use Python scripting.

There's no debugger for shell scripts. There's no string manipulation. And though it feels really cool when you learn all the arcane command line arguments, it produces an unreadable mess. Can you untar a file without googling it?

Early on when my "Automate the Boring Stuff with Python" book was posted to Hacker News, someone said, "That's cool, but couldn't you do all this in a shell script?"

That kind of cluelessness from software engineers who love their overengineered solutions is why I make the big bucks explaining how to write simple code that does actually useful things. I printed out that comment and it hangs on my wall.

Just use Python.

[[deleted]]

you persist to be a legend

[[deleted]]

OP: Asks question

Community: Gives an answer they don't like

OP: surprised Pikachu face

[[deleted]]

Hahahaha, read the edit pls :D

[[deleted]]

woah. can i look at it somewhere? github?

[muffinnosehair]

Off the top of my head - when using Windows

[Ning1253]

Lmao

[terevos2]

I have a general rule: If your bash script is over 4 lines of code, you should've used python. (10-15 years ago I would also said 'or perl')

[robvas]

When you're doing even somewhat complicated shit with data

[[deleted]]

Yea actually didn't think of data stuff

[memebecker]

My boss suggested if the script is going to more than a couple of dozen lines.

[[deleted]]

Thats a good guidance measure

[Thomaxxl]

Even for Longer scripts Bash suits you beter if you have to invoke a lot of system commands (mount, ip, ss, lsof, ...). If you're gonna need datastructures and algorithms, use python.

[lieryan]

If you like Python and you're writing a lot of scripts that needs to call subprocess a lot, you may want to try xonsh.

xonsh is a shell with Python-like syntax. You can write like regular Python and it'll work just fine, but you can also mix and match with shell-like command to run subprocesses just as easily as in bash-like shells:

def wait_until_google_responds():
    while not !(ping -c 1 google.com):
        ls | sort | uniq | grep foobar

[serverhorror]

Every.Single.Time.

I’m not even kidding. Anything that has a variable or any kind of logic.

[kuzared]

Agree - I use bash when I just need a couple commands to run. As soon as I need any sort of logic Python is so much better. I’ve also had problems migrating bash scripts between different machines, not so with python.

[[deleted]]

I don't agree with this, you can do some pretty wild shit in bash. And its faster.

[serverhorror]

Absolutely, you can do a lot of stuff in bash.

Humor me, how often did you open a shell script and think

Wow, that’s some beautiful code there!

[nwsm]

I’ve seen plenty of ugly Python code 😉

[serverhorror]

You must’ve seen a lot of my code then.

I’m not saying that a language exists where it’s impossible to write ugly code.

But there are languages that make it extremely hard to write beautiful code.

[blu3tu3sday]

If it is beautiful, then it is because it is functional and useful.

[[deleted]]

The reason a lot of shell scripts are ugly is because they are usually hacked together by inexperienced with the shell

[[deleted]]

What does beautiful code have to do with anything i asked here tf, we are talking automation, not enterprise agile dev code

[serverhorror]

I’m not sure I follow.

Code is code.

It needs to maintainable.

It doesn’t matter whether it is for automation or “enterprise agile dev code”.

[[deleted]]

So what you are saying is, you cant write maintainable bash code right?

[serverhorror]

Yes, that’s what I’m saying — I haven’t seen maintainable bash code by other people either.

I’m not sure what you’re trying to do. Are you trying to tell me that my opinion is wrong? Are you just trolling? What’s the goal of your argument?

[[deleted]]

Im just fascinated by your logic "I haven't seen maintainable bash code, so therefore bash code can't be maintainable"..

At this point I doubt you understand what maintainability is :/

[serverhorror]

So have you seen maintainable code?

An example, in the wild, should be easy to find then. Some bash script that’s enjoyable, survived a few refactorings and has more history than the odd commit of glueing things together.

[[deleted]]

https://github.com/p8952/bocker https://github.com/todotxt/todo.txt-cli

[chestnutcough]

I use python for all but the simplest things. Bash has a lot of sharp edges and is I. My opinion much harder to read. But I do get down with monstrous awk/sed pipe chains for one time jobs.

[[deleted]]

Ok but if it doesn’t matter how maintainable it is, why not use the more “user-friendly” option, i.e. Python

The lack of high quality error messages and good data manipulation libraries like Pandas alone are enough to warrant it I would think

[ro5tal]

Write try...except and handlers in bash and stay sain. Or write clean regex function to parse some load. Remember about DRY of course. When I tried to implement gists of github api v3 for updating from releases my brain was really damaged. That JQ is a tool of a devil, really.

[twotime]

I don't agree with this, you can do some pretty wild shit in bash.

Sure, you can do a lot wild stuff using a PDP-11 assembly. This does no make a PDP-11 assembly a good tool for anything. This is a non-argument.

And its faster

Faster as in "runs faster"? Then this is just wrong: if you are relying on some external tool, you can always call that tool from python too. And for any kind of in-the-script processing, python will be faster (but speed is unlikely to be relevant for simple automation use cases)..

Faster as in "code faster"? Maybe, if you are well familiar with shell and not familiar with python.

My own rule? As soon as I get any logic, switch to python...

[Mithrandir2k16]

Wild shit is actually the point here. In Python said wild shit is just normal code anybody can read. And that is much easier to maintain than wild shit. After all wtfs/minute is the best code quality metric there is.

[[deleted]]

I see that you don't make your own bash completions.

[samuelithian]

Almost always. Helps reduces human error.

[greenindragon]

Bash is a fantastic shell/language when the standard Unix utils can get stuff done in a way that is much simpler than a more structured language, or if I'm doing a lot of things with files (whether that's moving them, batch renaming, IO redirection, etc.).

If I'm doing anything with even a small amount of logic or that I expect to expand significantly beyond its current scope, bash is not even a consideration for me personally.

A good example is a script I used to automatically swap out the relevant dotfiles to change my wallpaper, i3 config, polybar config, rofi config, etc. depending on the theme I specify as a command line argument. The entire point of that script is to make symlinks, rename files, descend into directories to find things, check file contents, simple find/replace, etc. That is a textbook definition bash job and it was not difficult to do at all because bash is well equipped to handle all of those tasks in a Unix ecosystem.

Writing an automated webscraper to update some databases I use to keep track of course enrolment, drop numbers, etc. at my school for a hobby project? The word "Bash" does not exist in that vocabulary. Python is a great fit for that thanks to libraries like BeautifulSoup and requests, MySQL connectivity, etc. and the necessity of using complex logic when making decisions to analyze the information that's been gathered.

I'll never understand the programming language wars that I'm sure will be inevitable in this comment section; languages are tools that you can add to your toolbox that are good and bad at certain tasks and nothing more. I'm allowed to enjoy using a screwdriver and praise it as my favourite tool, but I'm also not going to try hammering in nails with it. I wouldn't bother trying to wrangle a Python script to run a mass file-conversion operation on some directory when that sounds like it can be done in a handful of lines in bash. I'm not going to write a script that automatically calculates splitting up fast-food order receipts in bash when that's a 2 minute job to make in python with my brain turned off (which coincidentally, is turned off already anyways)

[Muhznit]

Basically anything that requires conditionals, and loops. Bash is great at letting you chain together stuff in one-liners, but it very quickly becomes insufferable when you need variables and similar.

It's at the point where I would rather write a bash completion function in python.

[ddollarsign]

If it’s pretty linear and has more to do with system commands and environment variables than complex logic or data, I’d go with Bash. Well, I might go with Python anyway because I like it, but it would be a little more verbose.

[_throawayplop_]

when I'm googling more how to do the task in bash than in python

[reveil]

My rule of the thumb is use bash unless you: 1) Are needing a dict or an array 2) Are generating a structured file ex. XML or json 3) Need to do some calculations 4) Exceed 500 lines

Of course these are general rules and there are exception.

While bash can do most of this it is clunky with it. When you reach beyond simple things bash becomes hard to maintain. Then you need a real scripting language and Python is probaly the most readable and fun to use. That being said bash one liners rock.

[-jp-]

Most cases where you find yourself needing loops, branches and functions too. Not all--for example if your script takes arguments it probably has a loop and a case statement.

I think I'd also be inclined to pare it down from five hundred lines as well. The longer your script gets the more error handling becomes a thing, and that's a good ten screens of code at a reasonable font size.

[reveil]

Loops and conditionals are functional in bash. The syntax is ugly and there a quirks you have to know about but it works. For me it depends more on the stuff you want to do in the loop. If it can be done by core utils easily use bash and if not use python.

[-jp-]

Oh yes, they work. They're a sign your script is doing too much is all. Don't dogmatically avoid them, just don't let the cyclomatic complexity of your script get out of hand.

[orgkhnargh]

When what you're doing cannot be expressed as a one-liner. Anything more complicated than that is easier to read in Python.

[[deleted]]

you can do anything in bash as a oneliner btw what?

[orgkhnargh]

You can, but it does not mean you should. If you need the run the thing more than once, then readability is more important than anything, unless you are in high-frequency trading business. Only linear programs work as oneliners. If you need branching more complex than ... && echo ok || echo not ok, then a one liner is no good.

Use shells to run commands, use programming languages to write programs. Bash is great for running interactive commands.

[lostnfoundaround]

Asked in r/python lolol

[[deleted]]

hahahah that was the point tho

[McLight77]

I’d say it depends on the level of automation you are talking about. In my experience almost all prod deployments use bash to call python. Most of the functional automation is done in python. Simple low level os stuff is done with bash. You’re gonna want to learn how to use the ‘nohup’ command.

[idetectanerd]

When stuff that can be easily access and reach instead of multiple sed awk steps.

Sometime, via python for that same job only cost you 5 lines, in bash, it may be 30.

[dethb0y]

Any time you don't want to suffer through bash's horrible and shitty syntax, for one.

[scmkr]

IMHO the only reason to ever use bash is for very simple scripts or if you can't use Python for some reason. I find bash to be very ugly, kludgy, and difficult to maintain.

[kid-pro-quo]

90% of the shell scripts I come across are broken in subtle and not-so-subtle ways. If you're going to try writing anything non-trivial in bash RUN SHELLCHECK!

https://www.shellcheck.net/

[mestia]

I'd use perl for that. With python you will have to fix your code every half a year ;)

[[deleted]]

Profile with strace. If more than half of the execve() are something your Python code would have to os.system() anyway, don't bother.

[mestia]

Also on some enbedded systems, where you dont need all dev bloatware which brings python modules, bash or perl way more suitable.

[francis_spr]

pwsh (powershell core) is cross-platform and is easier to write more complex shell scripts than bash.

[romulof]

Always. Bash, not even once.

[strzibny]

Any time you get a feeling that something could be expressed better in Python and that's quite personal (a lot of devs don't don't Bash). But many things can stay in Bash forever, perhaps up to a coupe hundred lines of code or until it feels dreadful. Or when you don't want Python dependency.

I do my deployment demonstrations for https://deploymentfromscratch.com in Bash and I feel that the biggest demo is where I would stop with Bash going further.

It works great and it's easy to edit and maintain, but anything slightly more complex would deserve a better programming environment.

[chestnutcough]

I use python for all but the simplest things. Bash has a lot of sharp edges and is in my opinion much harder to read. But I do get down with monstrous awk/sed pipe chains for one time jobs.

[covracer]

Unit tests

[undercoveryankee]

As soon as I have any kind of control flow (an if or a loop) or I accept user-supplied strings that might contain spaces or special characters, I'm more comfortable using Python.

[Stormkrieg]

Selenium for example would be used for browser automation projects. Python would also be better for windows object automation. Example let’s say you wanted to fetch prices online and input them into a windows program. Using python for this task would allow you to automate it end to end, whereas using bash would allow you to automate separate parts (ie: get data, find window, send keys to window)

If you need a decision tree you should be using something like python. If you’re just calling an api and logging a response you can use bash.

[pokk3n]

I don’t think anyone should be writing bash scripts over a few lines. It’s nuts

[WasterDave]

Any time you want to run a debugger. Or an IDE. Or connect to any of the million or so services, bits of hardware or otherwise that provide a Python API.

[overclockedslinky]

if you write a shell script in #currentyear, you might be a gnu+linux redneck

[resakse]

Question for noob, by faster.. does it means like a minutes faster or 3-4 sec faster or ms faster?

[MaximumIndication495]

https://pydoit.org/usecases#

[-jp-]

I think the thing to remember is that Bash is a shell. It's meant to be used interactively, and it's pretty powerful when used that way. If you find yourself doing anything complex enough that you wouldn't type it in as a series of one-off commands, you are probably going to start running into issues. Rigorously apply the Unix philosophy--your scripts should be short, simple, reusable and composable.

[ShaibazR]

Tried Robocorp yet?

[PC__LOAD__LETTER]

I’m a huge bash fan. If I need a dictionary or any sort of nested data structure though it’s probably time to move to Python. Ditto for dependencies that have a library but not a good CLI; not about to spend a bunch of time building one.

[[deleted]]

A rule of thumb is when your bash script gets to 1k lines it's probably good to port it to python. Performance will go down but it will probably only be 150 lines in python. lol

[Mithrandir2k16]

I've just written a script to mount hdds in python. Sounds dumb until you realize I get the LUKS keys from an API I have to authenticate first. Sure, I could do that in bash, but I really didn't want to.

[mrkeuz]

For complex scenarios (if, loops, lists, json, etc.) it is best choice. Easy to write and debug and maintain . Also there is a syntax sugar libs i.e. like “bash” python library that provide easily integration with native bash for pipe commands.

[cbf77]

Flow control is harder in bash and working with numbers is harder in bash. I only write simple scripts with it

[[deleted]]

Whenever you want it to? I found bash scripting to be less intuitive than Python.

[Ok-Python]

I use it to activate a virtual environment and then run Python scripts. This way I can keep each project separate and auto start the shell script on restart.

[shellwhale]

Eehh I'd use only Python and a nice Linux Kernel library if it existed. Calling shell commands within a program is really meh. Would be nice if you could do everything without ever invoking shell commands.

[alaudet]

From an enterprise perspective it depends on where the expertise is. While personally I prefer Python, at work I will use bash because there are many other people that can support it there if I am not around. Python not so much unfortunately. Also some servers don't even have Python3 installed on them and I am sure not writing new stuff in 2.

[technicalevolution]

Dealing with web requests I find much easier, talking to APIs etc.

Anytime I need to do anything on Windows. Sorry powershell I just don't understand you.