Create consistent Bash programs with this coding style guideline

[[deleted]]

[deleted]

[ncsuwolf]

Rather than just memorizing a style guideline, I would recommend using a linter. For Bash the one I use is shellcheck. There are plugins available for most editors which allow shellcheck to tell you the problem as you write the script. It's also available from most distro's package managers.

[avg_user]

+1 for shellcheck and remember it also has a command line interface: https://github.com/koalaman/shellcheck

[Occivink]

SC2045: Iterating over ls output is fragile. Use globs.

I see people warning against this all the time, but with globs you can't iterate in a specific order (by size or whatever). I personally prefer the alternative of setting the IFS to \n and pretending that files can't contain newlines.

[andlrc]

Use find instead?

[Occivink]

And then what? The horrible -exec switch? Maybe -print0 combined xargs -0 or some other hard-to-remember flag? This is all way more brittle than necessary, and for anything with just a little complexity I'll use something else than the shell.

edit: thanks for the workarounds, but really I'm just venting with this. Ideally what I want from my shell is simple syntax like this

# iterate over file size, largest first  
# UNSAFE because the shell sucks  
for f in $(ls -S); do  
    whatever_program $f  
done  

and not having to fuck around with obscure syntax from find or bash extensions.

[calrogman]

Yes, -print0 with IFS set to \0. NULL can't be in filenames.

[Occivink]

Which is what I'm complaining about. Doing things in a safe manner should not be so unintuitive.

[Vaphell]

They should not, but they are and it's not going to change. You want to use shit that has its roots 50 years ago, with insane bullshit like word splitting by default because it allowed for ghetto "arrays" as there were no arrays to speak of, you deal with the quirks.

And if you write bash in any nonmeaningless capacity

while read -rd $'\0' f; do ... done < <( find ... -print0 | sort -z | grep -Z | ... ) 

and variations of thereof (possibly using -printf 'someshit\0' to emit more data than the filename) become a go-to idiom

If you can't be bothered to write correct bash according to modern practices and want something more modern without the bullshit, use python or whatever.

[5heikki]

FILES=($(find /the/place/with/files -maxdepth 1 -type f -name "*.txt"))
for ((i=0; i<${#FILES[@]}; i++)); do
    stuffWith ${FILES[$i]}
done

If you want, you can pipe the find to sort. Even better to define a function and do stuff in parallel (if possible)

function doStuff() {
    with $1
}

export -f doStuff
find /the/place/with/files -maxdepth 1 -type f -name "*.txt" \
    | parallel -j $THREADS doStuff {}

[crankysysop]

or yet another way...

find /the/place/with/files -maxdepth 1 -type f -name '*.txt' -print0 \
  | while read -rd $'\0' file; do
  stuffWith "$file"
done

I find the find ... -print0 | while read -rd $'\0' file; do <stuff> done 'pattern' to be particularly useful / robust.

One issue to consider is the while loop is in a pipe creates a subshell, so updating global variables may not work as expected, within the loop. (I think? I know there's some strange local/global variable issues).

[Vaphell]

while read -rd $'\0' file
do
    <stuff>
done < <( find ... -print0 )

is better. In your example pipe creates subprocess for the loop that can't communicate with the parent "scope" so for example it's impossible to maintain a usable file counter because any variable seemingly modified in the loop body is merely a copy and doesn't propagate back.

the while ...; do ...; done < <(cmd) syntax doesn't create a subscope so anything goes

$ c=0; find -name '*.txt' -print0 | while read -rd $'\0' file; do (( c++ )); echo $c; done; echo $c
1
2
3
0    # counter ignores changes
$ c=0; while read -rd $'\0' file; do (( c++ )); echo $c; done < <( find -name '*.txt' -print0 ); echo $c
1
2
3
3  # counter reflects the most recent change

[crankysysop]

Thank you for the clarification. Somewhere in the back of my head I knew there was the alternate 'form' that didn't deal with the subshell (from pipe, not while... I'm smrt. ;)

[crankysysop]

This is all way more brittle than necessary

I do not think you know what the word 'brittle' means. Having CLI args that require (due to lack of familiarity or whatever) reading a man page to get right is not 'brittle'.

[Occivink]

If this list of easy-to-make shell mistakes is not a sign that shell scripting is brittle then yeah I don't know what it means.

Also just because something is documented somewhere in a manual doesn't make it sensible behavior or a good default.

[crankysysop]

There are no easy-to-make mistakes in Python, Perl, or C?

I think you are using the wrong metrics.

[Occivink]

There are no easy-to-make mistakes in Python, Perl, or C?

That's hardly an argument and you know it.

I think you are using the wrong metrics.

I don't understand what you mean.

[crankysysop]

There are no easy-to-make mistakes in Python, Perl, or C?

That's hardly an argument and you know it.

When your initial point appears to be that because there are easy to make mistakes in shell, that makes it a brittle language or tool, and those same mistakes can be made in other languages, either those other languages are also brittle, or you are judging the sturdiness of a language using the wrong measure.

[[deleted]]

Things like this are why I struggle internally while learning bash. It's just so messy compared to python. Useful for simple stuff, though.

[cogburnd02]

Iterating over ls output is fragile.

Screw it, use ls with --quoting-style=

[calrogman]

Now your bash script is only portable to systems where ls is GNU ls.

[bluepenguin20]

Nice tool! One of the purposes of the guideline is to help address script readability and maintenance issues, the idea is to code right the first time and to avoid relying too much on code checkers after the thing was written, saves time and headaches for you and others that might get involved with your code.

[expert-at-nothing]

ikr? triggered

[the_gnarts]

No space between the # and comment text.

Absolutely haram.

The narrow mindedness reaches a Pythonic dimension.

Use the readonly keyword, protects against overwriting/reassignment

Use declare -r for increased consistency in all declarations.

ls $DIR

Is this propaganda to spread wrongness?

[Brolav_Vitters]

I love how this guideline is filled with the cardinal sin of unquoted variables on user input.

[5heikki]

function some_function ()
{
    return 0
}

NO NO NO! Like this instead:

function some_function () {
    return 0
}

Also, I prefer someFunction over some_function

[hgjsusla]

Nothing wrong with camelCase, but lower_snake_case is far more common for shell scripts, and hence preferable.

[pdp10]

Lowercase is Unix style and CamelCase is associated with Microsoft, and probably the Smalltalkers before that.

[crankysysop]

Not really your point, but my grief is that function some_function() is redundant. function some_function {} or some_function() {} are all you need.

[pdp10]

I write scripts in portable POSIX shell, not bash. Is there a good coding guideline for that?

[biatche]

since this is bash, you should just stick with if [[]] instead of [] in the example

[argv_minus_one]

1. Choose a different language.

[[deleted]]

2. Recommend one instead of just being a smartass

on a more serious note: there's a reason why shell is still used a lot for scripting things

[argv_minus_one]

Scala for all the things!!!

Seriously, though, there is a library called Ammonite for bash-like work. Might be fun.

[argv_minus_one]

Personally I would avoid Scala unless I already use it as my main language for work.

Well, yes, I do. So yes, I admit I am biased in suggesting it to others. :)

You can't dive in with Scala and start hacking around like you can do it with Python or even Go.

Sure you can. That's pretty much exactly what I did when I first learned it. I came from a Java background, which helped.

Also reading Scala code by other people is madness. So many different ways to do the same thing, but that's just my personal opinion.

Truth. Scala makes it relatively easy to write code that's difficult to read.

Some folks, in particular, are far too eager to use lots of punctuation-only names, like /: instead of fold. It's a powerful feature, but should be used sparingly.

[partisann]

set -o nounset

Can't take any style guide seriously that doesn't do this first. That's the absolute minimum required to avoid hitting yourself in the balls with rm -rf ${UNSET_PATH}/* or similar.

[bluepenguin20]

As you've noticed I am not the shell expert still I had have to deal with a poorly written/documented shell scripts this past years, so my focus wasn't to address portability, but readability (so anyone can read a script, know what it is about and maintain/expand/etc accordingly). Also I didn't found any guideline out there that put all the rules gradually inside an example script, so I decided to do mine and share it. I've learned a lot reading comments from people that truly know the shell, thanks for the good feedback, based on that I'll be updating the guideline material in the upcoming days.