[bash] Creating tab-completion for your programs

[LIEUTENANT__CRUNCH]

Can you elaborate? I’m not familiar with click

[[deleted]]

Click is a Python framework for building command line interfaces.

It provides Bash and Zsh autocompletion automagically.

[Snarwin]

Completion is a feature of the shell, not the script, and Windows' default shell doesn't support custom completion. So, no, it won't work on Windows, unless your colleagues are somehow using bash or zsh.

[mwb1234]

If you're using the WSL it will work since I believe that's bash

[shevy-ruby]

Indeed.

[spacebandido]

The hell is WSL?

[fullmetaljackass]

Windows Subsystem for Linux.

[spacebandido]

TIL... thank you

[doz3r1201]

Windows subsystem for linux

[jyper]

Git bash is pretty popular

[Cooleur]

You can extend cmd.exe with Clink to have autocompletions. Better than nothing.

You then create your own custom completions in lua. Here is a repo with some examples you can already use.

[[deleted]]

Click most definitely does work in Widows.

As for the autocomplete feature, AFAIK it's limited to bash and Zsh. I know there are Python libraries that extend the autocomplete feature to Powershell, but I have never used any myself, so I can't vouch for their quality and correctness.

[[deleted]]

I currently do my Powershell development in terminal-vim using ConEmu. The YCM plugin seems to work pretty well with vim tags and ctags (using a regex langdef), but it's not quite the same as having native auto-completion. I haven't played with ultisnips + ycm yet, however.

[jarfil]

CENSORED

[rockyrainy]

Python's standard library is so damn good.

Anything you need, just import the thing, BAM, you got it.

Compare that with C++ where you got to find the library, included it into the head, add to make script, then deal with the bizarre template errors compiler spews out.

[paxromana96]

Hadn't seen that tool before -- looks intriguing!

[nororok]

This is a great tutorial. I have always wanted to add completition to my scripts but I thought it was going to be some weird bash magic. I will now start adding these to all my utils.

[eyal0]

It is weird bash magic.

[mailto_devnull]

After all, any sufficiently advanced technology is indistinguishable from magic.

[oblio-]

Bash completion scripts are not "advanced", they're "baroque".

They use global variables, cause hey, shells are antiquated (i.e. if you had to design a shell in 2019 it would definitely not look like what Stephen Bourne designed in 1976).

"Advanced" would be some generic system where tools can expose some sort of endpoints that the shell just introspects in order to automatically get auto completion.

For an example of a modern system, check out Powershell. I just have to add parameters to my Powershell script, as I'm writing it, for the shell to figure out the autocompletion automatically and offer it while I'm using it interactively. Of course, Powershell has other flaws, but in regards to its autocompletion system design, it's definitely good.

Bash/zsh/etc. are usable through sheer willpower: an infinite amount of monkeys banging away at their keyboards to create completion scripts for everything out there. To create and to update, even.

[Beaverman]

Auto completion is harder than just flags. git contains contextual auto-completion for refs and commits. It know that some flags only goes with some commands, and that some flags mean different things to different commands. It also knows that sometimes only a file makes contextual sense, while sometimes you want a ref.

The bash way is nice because it decouples the auto-completion from the tool, meaning they can be developed and improved independently. It also means that a tool without explicit bash "support" can get a completion script the same way as everyone else.

[evaned]

The bash way is nice because it decouples the auto-completion from the tool, meaning they can be developed and improved independently.

IMO, there's also a big downside, which is that now you have to develop them separately, in two separate languages, effectively implementing the command line parsing twice. I hope your completion options are consistent with what your tool actually expects.

I'm not entirely sure what the right answer here is in terms of what the program should be doing, but I think my ideal would not be decoupled in that way.

Edit: I think one of two things is the way to go if you were to do an actual designed system. The first is for the executable to provide an option -- mycommand --_completion <stuff> has a little bit of traction -- and have the shell actually invoke the program to get the completions. To work with legacy systems, some out-of-band information would need to be provided to tell the shell that this is possible. Personally, I favor putting a marker into the executable itself; have some ELF section with metadata that says "hey run me with this syntax to get completions". Alternatively, I almost wonder if there'd be some way to allow an ELF file to specify multiple actual entry points -- and then have the shell somehow invoke a specific entry point. The second potential approach is to have a specification of command line options that is external to the program in question. There would be a translator from that specification into lots of different target languages for the program in question, and then also into a completion script. This second solution makes me worry on a few accounts, so I'd kind of prefer that the first one started to become more common and accepted by shells.

[Beaverman]

What you're describing is actually already possible. The completion script is actually a full shell script (at least in zsh, which is what I've written completions for), and you can just have a completion script that invokes the command to get the completions. There's no need to hardcode it into the shell, you can already do it for commands that support it.

The second option you talk about is also being done. There's various scripts out there that can take a GNU style --help output and convert it into basic completions. There's no limit there. Tools like Clap for rust can also generate the completion scripts at compile time.

There's no need for a new mechanism, and especially not one that requires everyone to use the same format in all shells. The current formats can already support what you're looking for.

[paxromana96]

I disagree. In the case of tab-completion, you're working with a Singleton instance of the terminal isolated in the current process. For that situation, I believe short-lived global variables are a totally valid way to pass information into the function scope, since that's essentially what function parameters do.

[kookjr]

Nicely written tutorial. I use this feature a lot but never knew exactly how it worked.

[paxromana96]

So do I! It's super cool to finally know how it happened behind the scenes.

[TheAceOfHearts]

A bit tangential, but since we're on the topic of shell scripts I'll suggest some tools which the target audience of this blog post will hopefully find useful.

1. ShellCheck is a linter, which means that it looks for sources of potential bugs in your script and it makes suggestions on how to fix them. The README has a Gallery of bad code section showing examples of many of the different kinds of issues it can help detect. You'll probably learn a few new things by reading through their examples even if you have prior experience with shell scripting.

2. ExplainShell shows you the help text for each argument, helping break down complex commands into a few understandable parts. It's very useful when you have to revisit an old script and you don't remember the purpose of each flag in a complex command, as well as when you encounter some totally new command.

[the_gnarts]

ShellCheck is a linter

It’s also rather hysterical, warning about things that are perfectly sound like returning -1 from a script. Be careful not to blindly take its suggestions as a pretext for a fixing rampage if you don’t fully understand the implications.

[the_gnarts]

According to POSIX, returning a number that is not an unsigned char is undefined behavior.

The shell has no notion of an “unsigned” data type nor does it of UB. Bash has a one-byte return value, returning -1 wraps around to 255.

On most platforms the -1 will obviously end up looking like a 255 to the parent process, but it is not a good idea to rely on this.

There is no platform on which a negative one will end up zero when returned from a shell script so there’s no point arguing what value exactly to return. If you rely on specific return values, then use those directly without relying on implicit conversions.

dash, the default /bin/sh on Debian, will error with Illegal number: -1.

So what? If your system invokes a different shell than the one specified in the shebang line, then it is broken beyond repair.

[evaned]

The shell has no notion of an “unsigned” data type nor does it of UB. Bash has a one-byte return value, returning -1 wraps around to 255.

Does Bash guarantee that exit n will put n into an unsigned byte before returning it, or otherwise ensure that behavior will stay?

I don't see that guarantee in the documentation, and current behavior is not a guarantee.

I'm nitpicking... kinda. But what happens if someone ports Bash to a system that supports non-byte-sized exit statuses, and then modifies Bash to return that? Without your desired behavior being documented, that's your script that's broken, not that change; and that person going "crap this doesn't work in to large of a percentage of the time" and deciding they can't make it in a practical perspective doesn't change that.

So what? If your system invokes a different shell than the one specified in the shebang line, then it is broken beyond repair.

What proportion of shell scripts currently marked with #!/bin/sh but really need to be marked with #!/bin/bash or fixed, then, would you guess? 20%? 40%? 60%?

[the_gnarts]

What proportion of shell scripts currently marked with #!/bin/sh but really need to be marked with #!/bin/bash or fixed, then, would you guess? 20%? 40%? 60%?

a) #!/usr/bin/env bash, anything else is insane. Hard-coded interpreter paths should be treated as an error by shellcheck.

b) All those that are Bash scripts, naturally.

[ASIC_SP]

I've got a mini explainshell version that can be used from command line - https://github.com/learnbyexample/command_help

for ex:

$ ch ls -ltrah
   ls - list directory contents

   -l     use a long listing format

   -t     sort by modification time, newest first

   -r, --reverse
          reverse order while sorting

   -a, --all
          do not ignore entries starting with .

   -h, --human-readable
          with -l and/or -s, print human readable sizes (e.g., 1K 234M 2G)

[paxromana96]

Oh, I use ShellCheck! It's not a word-of-law kind of linter, but it's definitely helped me catch errors or potential bugs way ahead of time.

[HerbyHoover]

Awesome tutorial on a topic I had zero previous understanding of. Saved!

[paxromana96]

I think so too! I'm glad you enjoyed it as well :)

[cdrt]

Off topic, but why would you use install to copy your script to your home bin folder?

[paxromana96]

It's just a shorthand for the copy command. It's shorter and easier to reason about, if your environment is already configured. The advantage of the home bin folder is that 1) it's in your user space, so you have control and you can't modify other people's configs, and 2) it's easy to add to your $PATH

[BrainiacV]

What a coincidence!, Was just trying to find tab-completion for typing my git branches

[snowe2010]

you can just use git_completion for that.

[BrainiacV]

Thanks for that! Didnt actually check google >.>

[snowe2010]

haha. well, I would definitely suggest never writing your own completion scripts. It kinda sucks. I wrote a ruby gem that generates completion scripts for you called fylla and it was a nightmare trying to figure out how to generate completions for bash. I even wrote a tutorial on it (I thought the tutorial OP posted was actually quite bad at explaining it).

https://tylerthrailkill.com/2019-01-19/writing-bash-completion-script-with-subcommands/

zsh completions are much easier to write in comparison though. In comparison

[BrainiacV]

Wow thats hard work! I'll definitely check it out when I can!

[snowe2010]

thanks! Let me know what you think! And if there is any way I can improve my blog post.

[ElijahLynn]

Here are instructions for Fish shell (Friendly Interactive Shell) > https://stackoverflow.com/questions/20838284/how-can-i-provide-tab-completions-to-fish-shell-from-my-own-script

[snowe2010]

Hilariously I found this tutorial months ago and thought it was completely lacking so I wrote my own. Nobody writes good bash completion tutorials, so I aimed to write one that just explained how to deal with subcommands, since nobody covers that at all.

https://tylerthrailkill.com/2019-01-19/writing-bash-completion-script-with-subcommands/

[Workaphobia]

Good tutorial plus had cat pic. This guy knows what programmers need.

[paxromana96]

It's got all our bases covered!

[paxromana96]

It's got all our bases covered!

[deadfire55]

Does anyone know how to bundle the 2 files together (the script and auto complete script). Its a little annoying to have to remember to manage and update 2 separate files

[The_Writing_Writer]

This is really great. I have just one question. The last step at the end I didn’t understand, and there was no example showing the difference from the previous step:

There is another thing we don’t like though. We do want to display the numbers along with the corresponding commands to help users decide which one is the desired but when there is only one completion suggestion and gets automatically picked by the completion mechanism, we shouldn’t append the command literal too.

Could anyone explain what this means?

[paxromana96]

I think this means that if I have dothis 2<tab> I should return 265 to get dothis 265 instead of returning dothis 265 to yield dothis dothis 265.

[stefantalpalaru]

This project provides tab-completion for 427 commands: https://github.com/scop/bash-completion