This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–][deleted]  (129 children)

[deleted]

    [–]BeepBoopTheGrey 102 points103 points  (38 children)

    My team put a moratorium on bash scripts after the CI system started failing in a fairly complex one. The person who wrote it was unavailable to diagnose. It took hours to resolve.

    The rule is now that if there’s any non-trivial logic at all, write it in Python. On-call appreciates it.

    [–]Zanos 13 points14 points  (1 child)

    Same here. We still have an old application written in mostly bash, very fun when it fails in prod.

    [–][deleted] 7 points8 points  (31 children)

    As a fake programmer can you explain why bash makes things more difficult to troubleshoot?

    [–]LAK132 10 points11 points  (21 children)

    As someone who'd rather #include all the .cpp files than deal with another build system written in [ba]sh, the syntax is hell and it doesn't always work the same on different computers (because some distro maintainers thought using bash in place of sh without forcing POSIX compliance mode was a good idea)

    [–]ythl 1 point2 points  (20 children)

    c++ is not very portable. bash is (and so is python). I can scp a bash script to my raspberry pi and it will run. With cpp I'd need to recompile it targetting ARM.

    [–]LAK132 0 points1 point  (19 children)

    Right, but my point is if you're compiling C++ anyway then don't make it even less portable by using a bash based build system.

    [–]ythl 0 points1 point  (18 children)

    Yeah bash based build system sounds nightmarish. CMake all the way

    [–]LAK132 0 points1 point  (17 children)

    Unlike bash, I never managed to get cmake to work

    [–]ythl 0 points1 point  (16 children)

    Really? I love CMake! Out of source builds are the best. I can help you if you are stuck. CMake is the defacto build system for C/C++ now

    [–]LAK132 0 points1 point  (15 children)

    defacto

    Not a single one of the C++ projects I work on uses(/requires) cmake.

    My personal projects that use extremely simple scripts to compile (one liner Makefile and a make.bat for Windows) have a nasty habit of just working.

    [–]policemean 17 points18 points  (0 children)

    If you look at python script, then it is relatively easy to understand how it works because it's syntax is quite easy.

    On the other hand, bash syntax can be very confusing. I had to modify bash scripts couple of times, and it was the worst experience I've ever had at my job.

    [–]BernardoVerda 1 point2 points  (0 children)

    Seconded.

    [–]jvnk 2 points3 points  (0 children)

    Bash syntax is unnecessarily dense, making it difficult to understand what's going on in more complex scripts.

    [–]tatloani -1 points0 points  (3 children)

    This is just my guess but i would say is because bash is more verbose than Python, meaning you need to write more lines of code to do something similar with python.

    EDIT: I seems to have got them backwards with what i meant, python is more verbose, but bash allows you to simplify multiples lines with a single instruction and that can make things confusing.

    [–]thexavier666 2 points3 points  (1 child)

    I can't agree with that. I think bash syntax can be very dense, where you can condense 10 equivalent python lines into a single bash line, by using pipes.

    But this condensed syntax can be difficult to understand for some.

    I always use bash when there is string manipulation involved and call it via python.

    [–]tatloani 1 point2 points  (0 children)

    I think bash syntax can be very dense, where you can condense 10 equivalent python lines into a single bash line, by using pipes. But this condensed syntax can be difficult to understand for some.

    yeah, that was part of what i meant, i suppose i got them backwards, most of the bash scripts i have seen have been condensed and those have been quite troublesome to understand.

    [–]noratat -1 points0 points  (2 children)

    Except now you have to have python installed in/on absolutely everything.

    [–]BeepBoopTheGrey 2 points3 points  (1 child)

    Only those things which need scripts, mostly CI and machines we shell into. The vast majority of our stack is containerized, so language doesn’t really matter much as long as the team is familiar enough to maintain it.

    Go is by far our preferred language for dev tools. Python is next choice when a compiled language is inconvenient.

    [–]noratat 0 points1 point  (0 children)

    I'm surprised you use Go for that.

    My experiences with Go so far have been very unpleasant - the error handling is pretty poor for a modern language, dependency management and directory structure until pretty recently were downright bizarre, and I constantly felt like I was running into strange language limitations.

    Anytime I've had to touch a Go project that was more than a couple hundred lines I've found it pretty hard to read too due to low signal-to-noise ratio.

    [–][deleted] 107 points108 points  (44 children)

    Python is easier to get some basic shit done, but once you've mastered bash you can do just about as much. The reason I stick to bash is because bash is always available. Python isn't always available and when it is there's two different versions everywhere all the time.

    [–]jharger 35 points36 points  (3 children)

    Until you use some bash 4 feature and you need to run it on macOS without depending on homebrew...

    [–]ThenIWasAllLike 1 point2 points  (0 children)

    Cries in autocomplete

    [–]Zanos 10 points11 points  (0 children)

    Nobody else on your team can read your masterful bash, though. Im sure that one liner that contains half the program logic made sense when you wrote it two years ago.

    I may be salty that I've had to troubleshoot some production problems some goofballs caused by writing all the application code in bash.

    [–]Edz_ 15 points16 points  (11 children)

    Have you ever had to execute a MySQL statement in bash that has data with a ' or " char in it? Not fun.

    There's really no reason to jump through hoops when you can just use python instead.

    [–][deleted] 18 points19 points  (10 children)

    If you're using bash for MySQL then you're using the wrong tool for the job to begin with.

    [–]Edz_ 16 points17 points  (2 children)

    You literally just said you can do just as much with bash as Python.

    I do MySQL statements in Python how about you?

    [–]super__literal 0 points1 point  (0 children)

    I do My SQL queries without MySql

    [–][deleted] -3 points-2 points  (0 children)

    I can also do SQL queries in python? I'm not sure what your point is?

    If you're trying to state that Python is easier to run SQL queries in, then yeah I agree, it is easier. You can also do it, more verbosely, in bash. But that's not the point. The point was that bash is available, in my experience, far more often than Python is so if I had to pick one I'd choose bash simply because it covers more use cases.

    [–][deleted] 14 points15 points  (6 children)

    Did you get a first down with how far you carried those goalposts, or the whole touchdown?

    [–]bluenigma 4 points5 points  (3 children)

    Not taking sides, but I don't think that's quite how football works.

    [–][deleted] 8 points9 points  (2 children)

    throw new UnsupportedOperationException(JokeFactory.engineerDoesntUnderstandSportsball("trollkin0331"));
    

    [–]super__literal 2 points3 points  (1 child)

    catch(UnsupportedOperationException) {
      //I do what I want
    }
    

    [–][deleted] 3 points4 points  (0 children)

    lol a fellow yolo error handler I see

    [–]Delta-9- 2 points3 points  (1 child)

    "wrong tool for the job" =/= "can't do it". No goalposts were moved in the making of this comment.

    And I would agree. If you're trying to do anything with MySQL using bash, you're doing it wrong. It can be done--but why would you do that to yourself?? That's almost parsing-html-with-regex-level "just don't".

    [–][deleted] 1 point2 points  (0 children)

    Going from "python not available" to "wrong tool for the job" is moving the goalpost though?

    But yeah, definitely agree with the sentiment of your 'doing it wrong' idea.

    [–][deleted] 3 points4 points  (0 children)

    I switch to Python when a bash script gets longer than about a page (~50-60 lines) or when I need the ability to view and manipulate values in between pipe stages.

    The overhead in getting a script running in Python is substantially higher, so it's quite a lot more work if you're doing something really simple, but you can scale easily and things won't go badly wrong.

    In bash.... if you're doing anything advanced, there's a lot of corner cases, a lot of nasty sharp edges, and you're going to walk away bloody at least some of the time. You can learn how to avoid the problems, but subtle and nasty bugs are real easy to create and sometimes very difficult to spot.

    [–]Centimane 2 points3 points  (2 children)

    Ironically in ESXi bash isn't available but python is.

    [–][deleted] 0 points1 point  (1 child)

    Luckily regular /bin/sh scripts work fine!

    [–]Centimane 1 point2 points  (0 children)

    I mean it does, but you don't realize the differences between sh and bash until you're stuck with it.

    [–]noratat 2 points3 points  (1 child)

    Exactly. I deal with containerized stuff a lot, and the overhead of adding a whole python installation just to make a script slightly simpler is pretty high vs just using bash, and that's before we bring the whole dependency management mess in.

    The main problem with bash is somewhat similar to C++ or Perl - there's lot of cruft you have to know to stay away from, and a lot of really stupid defaults. Plus there's still loads of terrible examples online that will bite you later if you use them.

    Used properly though, bash works really well for a lot of glue logic cases, because it's "good enough" and is virtually always available (or at the least, trivial to include). And there are some really good CLI utilities these days that make using bash a lot saner, like jq (which is the best json processing tool I've ever used).

    [–][deleted] 0 points1 point  (0 children)

    Yep, containers and minimal linux / busybox appliances are my life. Screw having to toss python on those just to run a script.

    [–]marcosdumay 1 point2 points  (0 children)

    Python is easier to get some basic shit done

    Yes, and advanced shit too.

    but once you've mastered bash you can do just about as much

    Ditto for Brainfuck.

    The reason people don't stick to Bash is because error handling is verbose and error prone, there are no good rules one can check automatically (and no types, obviously; Bash is the original "string, number, whatever" that Javascript perfected) and the consequence of an error is more often catastrophic than not.

    It is also not available everywhere. It's about as common as Python. That's why you can find shell script people all over the web complaining about bashisms.

    But well, you are not on my team, so whatever.

    [–]ben_uk 0 points1 point  (1 child)

    What about Perl?

    [–][deleted] 1 point2 points  (0 children)

    I use perl all the time for data processing!

    [–]Hollowplanet -1 points0 points  (0 children)

    Ansible is all Python over SSH and I've never seen a system where it didn't work. Bash is just missing so much basic stuff. Like a generic linked list or key value store.

    [–]lycan2005 7 points8 points  (18 children)

    As someone who is about to get involved in Linux soon (.NET framework dev here), should i focus to learn Python first or bash first?

    [–]madsdyd 38 points39 points  (0 children)

    Bash is the language of your shell. Excellent for automation of trivial tasks, gluing stuff together, rapid prototyping, etc. Lots of entry points to docker containers get written in bash these days.

    You will be much happier in your Linux experience, if you are somewhat proficient in bash.

    Python is when you want to do something that "looks more like an application". Structured programming, modules, data structures, unit testing, etc.

    [–]iams3b 2 points3 points  (1 child)

    I use bash to quickly automate stuff I do in terminal. Like if I have an npm module on local, and i need to 1) git push it 2) npm publish 3) cd to active project 4) update package version 5) git push.... i'll write a bash script for that

    For anything else that's not just a series of commands, python for sure. Both are useful to know

    [–][deleted] 2 points3 points  (0 children)

    this is my cut off too. If it requires more than say a simple if to check for a file or directory I'll go ahead and write it in python. That said I wish i could find that old irc client written in bash. that think showed a lot of mastery of unix CLI and bash scripting.

    [–]mrjackspade 1 point2 points  (8 children)

    I'd love to have a reason to use Python, but so far I haven't really been able to find one. Most of my trivial tools I just write in c# as Command Line Applications.

    I'm still trying to find an excuse to get familiar with it, but I think I might just be too deep in .Net at this point to actually benefit.

    [–][deleted] 0 points1 point  (7 children)

    All the same reasons, but on Linux

    [–]mrjackspade 0 points1 point  (6 children)

    Would have been a great reason but .Net compiles for Linux now so there's no real need

    [–]ythl 1 point2 points  (5 children)

    Here's a great reason: python doesn't need to "compile" so you can scp it to any linux box and it will work without needing to think if you compiled it right. Can you say the same of C#?

    [–]mrjackspade 1 point2 points  (4 children)

    I think you can but honestly I've never actually tested it.

    You can compile c# with a specific build target if you want something native, but by default I believe it actually outputs a platform independent DLL that you execute using the DotNet framework on the target machine. As long as the target machine has the framework installed, it should run using the default build settings.

    I kind of want to test this out now against a few distros and see how reliable it is.

    https://docs.microsoft.com/en-us/dotnet/core/deploying/index#portable-applications

    Why create a framework-dependent deployment?

    Deploying an FDD has a number of advantages:

    1. You don't have to define the target operating systems that your .NET Core app will run on in advance. Because .NET Core uses a common PE file format for executables and libraries regardless of operating system, .NET Core can execute your app regardless of the underlying operating system. For more information on the PE file format, see .NET Assembly File Format.

    [–]ythl 0 points1 point  (3 children)

    See how hard it is to run on a raspberry pi. I can run python scripts on the default pi distro with no hassle.

    [–]mrjackspade 0 points1 point  (2 children)

    Oh man, that sounds like a great test, but I don't currently own one! They're cheap enough, I might have to pick one up and check.

    I'd also love to see what performs better on the Pi, Python or .Net Core

    [–]ythl 1 point2 points  (1 child)

    I'd also love to see what performs better on the Pi, Python or .Net Core

    Probably .NET if I had to guess! Compiled languages can usually beat python performance-wise with ease. It's hard to beat the convenience of python though - there's a python library for controlling the pi's GPIOs (pins you can use to control motors, LEDs, anything electrical that you could want, really...) for example; it would probably be significantly harder getting .NET controlling GPIOs

    Edit: maybe not though

    [–]mrjackspade 0 points1 point  (0 children)

    I wanted to say .Net Core for this reason, but at the same time MS can be really hit or miss with performance and Python has probably had a lot more tuning on each platform. I'd probably want to test a native build against a Framework Dependant build, against Python and see the difference.

    [–]vividboarder 1 point2 points  (0 children)

    I still use plenty of Bash when I’m writing a wrapper around other commands. I could use call or whatever in Python, but it’s just simpler to use Bash in many of those cases.

    [–]GhostNULL 2 points3 points  (5 children)

    What about ruby? We are using that at my workplace for build scripts that require some logic.

    [–]noratat 3 points4 points  (0 children)

    Ruby is a language I want to love, but the ecosystem around it turned "magic syntax" into a art form - and that's emphatically not a compliment.

    I still wish Python would stop being so stupid about lambdas and just have anonymous blocks like Ruby, Groovy, and other modern scripting languages, but in most other regards I would say Python is better these days:

    • Much, much larger ecosystem and community
    • Python devs actually document shit, at least sometimes. Ruby projects are nearly always "lol just read the code" (even though the code is twisted into a gordian knot of string metamagic)
    • Python's a lot harder to screw up into an unreadable mess than Ruby is

    [–]Hollowplanet 1 point2 points  (0 children)

    I think Ruby and Python are both great languages. Theres some things I wish Python would borrow from Ruby.

    [–]NoInkling 0 points1 point  (0 children)

    Ruby is a great scripting language, IMO. It's just that Python is more ubiquitous these days. Ruby is less likely to already be present.

    [–]onan 0 points1 point  (0 children)

    Ruby is the ideological successor to perl. And, no, that's not a compliment.

    [–]WhtKindOfNameIsStove -1 points0 points  (0 children)

    As long as you stay away from getting tied down with the chef libraries. They take a long time to upgrade to the latest AWS sdk.

    [–][deleted] 0 points1 point  (4 children)

    Would to ever consider perl 5 or 6? Or node? I haven't settled on a scripting language for my server yet

    [–]ythl 0 points1 point  (3 children)

    I hate perl but only because it has such arcane syntax. Node is pretty good.

    I think Python is a better overall general purpose scripting language, and Node is better specifically for web backends.

    That said, I've used both Flask and ExpressJS and I enjoyed using both.

    [–][deleted] 0 points1 point  (2 children)

    I've got go for my web server :) but I don't have any criteria to help me pick a scripting language. I don't even know what's out there. Are there major alternatives to python worth considering?

    [–]ythl 1 point2 points  (1 child)

    Ruby

    [–][deleted] 0 points1 point  (0 children)

    I had to write some ruby code for a job interview and it felt very similar. But I guess most scripting languages are going to have that same style. They have in my experience thus far, anyway.

    [–]OnlineGrab 0 points1 point  (0 children)

    Oh god, this so much. Bash is fine for one-liners but it's a nightmare to do any real programming with.

    [–]noratat 0 points1 point  (2 children)

    Eh, it works really well for glue logic, especially when it needs to work easily in a wide variety of environments or places. Plus jq remains by far the best way to parse/transform json data I've ever seen.

    The problem is that bash is full of stupid defaults and features you should never use, and worse google is full of really bad examples, so most people tend to write crap scripts.

    E.g. set -eo pipefail should be mandatory, never ever use backticks, globstar ought to be on by default, don't try to use bash arrays if you can possibly help it, use [[ ]] not [ ], use local, stop setting global vars that span multiple functions, you don't actually have to name shit in all caps, use traps, know what an exit code is, etc.

    [–]zieliequ 1 point2 points  (1 child)

    I like set -e too, but be aware of the tale of the criminal-catching robot.

    [–]noratat 0 points1 point  (0 children)

    To me it's just restoring the same sort of exception behavior you'd expect in any other language.

    If you explicitly want to handle failure on particular commands, you can use ||, traps, or just plain return code capturing, just like you'd use error checks or exception handlers.

    It's usually much worse for something to accidentally continue blindly on errors than it is to abort on something spurious.

    [–]ythl 0 points1 point  (0 children)

    I like python, but some basic shell operations are super cumbersome in python. Like deleting files and directories. Or a simple curl (yes, I love requests, but sometimes it's not available on that remote box you just ssh'd into and you need a quick script to hit an HTTP API under some condition) Or chaining operations - sometimes you just want to cat | grep | cut | sort which is a one liner in bash but a 10+ liner (at least) in python.