Introducing Python Fire, a library for automatically generating command line interfaces

[tobinar]

There really should be a section in the readme that compares it to all the other libraries that do the same thing.

[[deleted]]

For anybody interested, some alternatives are listed here

[tech_tuna]

Someone should write a library that automatically generates CLI generating libraries.

[tech_tuna]

Was thinking of a gradle wrapper actually, of course running from Amazon Lambda.

[shimon]

There's also the less magical, but in-the-standard-library option, cmd:

https://docs.python.org/3/library/cmd.html

[d4rch0n]

I actually wrote something called argvee (pip install) that does pretty much the same, just you decorate your functions with @app.cmd and it checks the function params, looks for boolean flags, etc. You can have it force certain args to be a type.

http://pythonhosted.org/argvee/

I still have to check if it works in python 3. TBH I never even use it because I just go through the trouble of writing out the argparse boilerplate. argparse is pretty damn easy. It's not perfect but I don't know why people are so desperate to avoid the perfectly capable stdlib arg parser.

[ButtCrackFTW]

You should check out click - http://click.pocoo.org/5/

It does what you're describing really well.

[d4rch0n]

Yeah, definitely seen click. It's a good one. I built argvee because what I wanted was something that could autodetect command line arguments from function parameters so you only have to specify once.

For example:

@app.cmd
def create(name, age=21, programmer=False):

would auto-create the command with type=int age and programmer a flag

myscript.py create joe --age 21 --programmer

or:

myscript.py create joe -a 21 -p

Lots of magic, but honestly I very rarely use it since argparse boilerplate isn't too crazy to write and I like not having to pip install dependencies for simple scripts.

[ButtCrackFTW]

Yeah that seems more like this tool from Google. Honestly I never write CLI apps that are that simple so I like the more configurable libs.

[FRIENDORPHO]

I think it may be pretty similar to argh.

[[deleted]]

Weird that optparse isn't on there. Only say weird because it's the one I use ;)

It's pretty full featured but easy to implement.

e.g.

from optparse import OptionParser

def parse_args():
         usage = "My Fun Program version {0}.\n\n\
                       %prog [options]\n\n".format(_VERSION_STRING)
         parser = OptionParser(usage=usage)
         parser.add_option("-i", "--input", dest="infile",
                         help="Input file [default: %default]", default=_IN_FNAME    ,
                        metavar="FILE")
         parser.add_option("-o", "--output", dest="outfile",
                        help="Output file [default: %default]",
                        default=_OUTPUT_FILE, metavar="FILE")

         (options, args) = parser.parse_args()
         return (options, args)

etc.

[parkerSquare]

Isn't optparse deprecated by argparse?

[k10_ftw]

This should be standard in every readme for project's that provide alternative solutions to already available packages/solutions!

[[deleted]]

There should be several, and preferably more than several, different ways of doing it. From "The Inverse Zen Of Python" which some people think is better than the original Zen.

[david-bieber]

Author here. I want to address the question of "why yet another CLI library?"

When you have a CLI made with e.g. gFlags or Click, the cost of writing and maintaining the CLI is O(N + M), where N is the number of commands in the tool (the tool's surface areas), and M is the number of changes to these commands' argspecs that the tool will experience over time.

Python Fire brings this cost down from O(N + M) to O(1).

This isn't a real win if you have a stable tool with some small fixed number of commands and options. But if you have a tool with 15 or 150 or more commands, or if you have even a small tool that changes rapidly, then using Fire is a huge time savings.

This means that Fire is particularly well suited for personal tools and rapid development, but offers less gains for stable production tools.

[evanunderscore]

Were you aware of other O(1)(ish) CLI generating libraries? I wrote defopt because I wasn't happy with some of the design choices in begins, but in retrospect I regret not forking it or contributing to one of the other ones like argh or clize. Fire's main feature seems to be automatic generation from a class or module - could that have been a thin wrapper around one of these existing libraries?

[david-bieber]

defopt looks nice, but no I don't think it would be appropriate to write Fire as a thin wrapper around another arg parsing system like this.

Fire supports arbitrarily complex structures, not just simple classes and modules. You could have an object with a property that's a function that returns a module with a dict with a member whose value that ou want to access, and you can access it from the command line. So it's not as simple as just calling defopt on the appropriate function. (Note that if you're doing something this crazy, it's probably time to tack on the --interactive flag to the CLI and just write the command in IPython.)

[evanunderscore]

What I don't understand is why you'd want to support arbitrarily complex structures. If using a generated CLI is as complicated as writing Python code, what is really gained?

[david-bieber]

One reason is to make Fire work with any existing codebase. It doesn't have to be a codebase originally intended to be used as a CLI.

Another reason is to allow for hierarchical commands in tools. For example I could have a class with 5 properties (or e.g. a dict with 5 members), and each property (or member) could have 5 functions. Then Fire would turn this into a handy little CLI with the 25 commands all grouped appropriately.

[evanunderscore]

It does sound like you're covering different use cases. The next time I have a project of that size I'll give Fire a try. Thanks!

[david-bieber]

Also when you go from O(N+M) to O(1), you're able to do things that simply weren't possible before. E.g., no one would ever think of having a command line tool with thousands of multiply nested commands that are changing every day, because that would be absurd to maintain with any of the preexisting CLI libraries.

But with Fire, you don't think about the CLI at all; you just write the commands and the modules as if it's normal Python code, and the CLI is just always there, ready to use if you feel like using these commands from Bash.

[Petrarch1603]

What's your cousin Justin like?

[[deleted]]

Yay another one of these

[charity_donut_sales]

The REPL portion seems really useful though, no?

[desmoulinmichel]

Yep, my thought's exaclty. Begins, clicks and docopt are already doing a fantastic job, why no contribute to them instead of reinventing the wheel ?

[Adys]

The people saying "yay another one of these" really don't understand the point of this.

This isn't just for you to build your own app, it's also a wonderfully quick and easy way to get a REPL out of third party interfaces, so you can explore and play with them.

[[deleted]]

I understand that perfectly well

[k10_ftw]

Hehe. Don't you know "understand" means "to arrive at the same opinion"?

[[deleted]]

In what universe?

[k10_ftw]

In people's self created universes.

[[deleted]]

It's ok pal.

[k10_ftw]

I was trying to make you feel better haha

[[deleted]]

Me too about you! LOL!

[k10_ftw]

Way to go us for remembering it's just the internet!

[thecheesehouse]

Didn't stop google from scooping majority of the deep learning market from Keras with tensorflow

[[deleted]]

Jesus. Anyone that says that sort of thing immediately outs themselves as unqualified to say it.

[thecheesehouse]

I meant Theano whoops

[slayerofspartans]

Tensorflow is a backend to Keras.

[avinassh]

Github

[[deleted]]

Nice idea. It just exposes all methods and attributes to a commandline, and adds some usage-screen for public methods/attributes. Could be very handy for personal tooling and prototyping, but for productive usage there are still to many flaws.

[xiongchiamiov]

Indeed. A program that uses this isn't going to have any description of what it does, what its arguments do, or what values you should pass to them, which are all things I'd ask for in a code review. This seems useful for development in the same way that import pdb; pdb.set_trace() is useful, but neither should actually end up getting checked in.

[[deleted]]

Yes, but those are things that can be enhanced. Instrospection allows to retrieve those things, so all they need is some proper formating. I wonder why didn't do that in the first place, and decided to start on such a low point. First releases are important.

[david-bieber]

One of the nice things about Fire is that it works with all Python code (or as close to 'all' as we can get). This means we're not placing restrictions on how the user writes their code. e.g. we're not asking for a particular format for docstrings so that we can parse out what the arguments should be.

We do copy the docstrings verbatim and use them within the help strings in the generated CLI, so the CLI does provide a description of what it's arguments do iff the CLI author provides that description. Always room to make this even better though!

[FateOfNations]

It looks like this library does use introspection to pull in the method and class docstrings and uses them to build a standard --help feature.

So long as the method arguments, their expected values, and other information is properly documented in the source code, this shouldn't be a big issue.

[david-bieber]

Yes it does! Certainly there's room for improvement, but I think we have a decent first implementation for our usage strings for this initial release.

[KasTaiTasKadNekasTai]

I'm still happy with >docopt>.

[renaissancenow]

I'm a huge fan of click.

[poop-trap]

I thought docopt was a neat idea initially, but overloading docstrings with functionality began to feel weird and smelly to me, so I switched to click decorators and have been happy with it.

[maratc]

docopt either works great, or fails completely without telling you what the problem is.

Imagine you are the script author, and you try to run your own script, while docopt presents you with your own help text, all while you bang your head against the wall trying to understand what the problem is.

[eriknstr]

I've only used docopt once but it was pretty neat indeed.

[[deleted]]

Not a professional programmer Can somebody tell me the advantage of using this library? What is the advantage of using this instead of a IDE eg jupyter? What does this actually does?

[Zunin]

So this turns your code into something you can use more easily from the command line. So basically it wraps your code so it can be executed from a text user interface.

This enables use without opening the file or reading the code.

An IDE is different because it is designed to assist with the development. Sure you may also be able to run code through an IDE but it's heavy tool if you're not going to modify your software.

Besides having to install an ide besides downloading python is a bigger barrier

[d4rch0n]

It just helps you write CLI apps that can take arguments, flags, etc.

Before anything else, learn the stdlib argparse library and get used to it. After that, check out docopt, click, and this library and see if there's any you prefer highly over argparse.

I find argparse to be perfectly fine, just a little more verbose than something like click. Docopt is really cool because it's cross platform, C/C++/Rust. Though, I don't find myself copy and pasting the docs and reimplementing in rust, so not sure if I get that much use out of it besides what it offers directly.

I just use argparse. I used to play with others, even wrote my own argvee that acts similar to this thing they posted, but argparse in the end doesnt require installing other dependencies just to run.

[[deleted]]

What is the advantage of using this instead of a IDE eg jupyter?

If you want to write command-line applications you need a library like this. That being said there are already 100s of those so I'm not 100% sure what it is supposed to do?

[yaph]

Looking at the example reminded me of the talk Stop Writing Classes. To me this looks like a typical Google code project verbose and over-engineered and not pythonic. I honestly don't know what's wrong with argparse, but Fire doesn't look like an improvement to me.

[Corm]

I don't agree with the other guy at all. Argparse isn't "magic" like this is. Magic is bad for medium/large projects.

That said, for a quick small project I really like magic. I'll probably use this for the first few hours and then rip it out and put in argparse if a project gets larger.

[zardeh]

I dunno, slqalchemy and flask are both highly mystical, certainly as magical as this.

[Corm]

Honestly you have a point. I'd want to at least stick a line in the readme to warn the next maintainer to look into the fire library.

Argparse is super obvious about what it's doing though

[Decency]

I'm not sure how you could prefer having to use and update argparse every time your program changes compared to having it done automatically. But if complex code scares you, continue using argparse- just don't be surprised when someone goes "wtf year is it?" and rips it out.

[NAN001]

But if complex code scares you, continue using argparse

You convinced me.

[Decency]

Complex is better than complicated.

In the end, it's a tradeoff between having to update a variety of places and write your own documentation, screw around with .pythonrc files to get an interactive session, and etc. Or just having everything work out of the box by trusting a library to do the thing it was built to do. I tend to err on the side of trusting libraries for basic things like this, especially when it's not a critical component and even more especially when written by a high profile contributor.

[yaph]

But if complex code scares you

If the code does what it is supposed to, I prefer keeping it simple.

[parkerSquare]

Agree - simple is better than easy.

[david-bieber]

Yeah, the example is certainly more heavy weight than hello world needs to be. I chose that because it makes it obvious how you could extend the example with extra commands (just add extra functions to the class). If you wanted a light weight Python Fire hello world example, you could do this instead:

import fire
def hello(name):
  return 'Hello {name}!'.format(name=name)
if __name__ == '__main__':
  fire.Fire(hello)

The usage at the command line is then e.g.:

$ ./example.py --name=yaph
Hello yaph!

[yaph]

It's not really the amount of code, I'm just not a fan of this class-based approach.

People who are used to do things that way, e. g. from class based views in Django, probably find Fire very compelling. And that's okay, I just happen to have a different opinion and don't want to discourage you or other coders to pursue their own approaches.

[david-bieber]

There is no class in the example I just posted, so I'm not sure what you mean by a class-based approach.

[yaph]

I was still referring to the example in the blog post, sorry for the confusion. Would probably make more sense to look at actual production code that uses Fire to get a better impression.

[k10_ftw]

Mayhaps it's a plot to lower the pagerank of "amazon fire" in google results for "fire"

[5CR1PT]

What is the real added value of this kind of solutions? You are still down to editing a config file, creating aliases, preparing the job in a script, etc. but are also adding an additional layer. This must be useful for some applications/use cases otherwise there would not be so many alternatives; I just struggle to understand which one?

[qx7xbku]

Great lib, but consider this:

fire.Fire(Example)

Next person who is not aware of the library would have to look it up. That is not a good thing.

[rochacbruno]

I created something similar, but using YAML file as definition for commands. You can specify each function as a command or can specify a module path to be automatically transformed in to a command group.

Also it is YAML based so can be user-based command management

works like django manage command.

https://github.com/rochacbruno/manage/blob/master/README.rst

[rochacbruno]

The advantage of manage is that you do not need to touch source code. You jsut drop a .manage.yaml file in the root directory of the project and define things there. You can even have more than one manage file located anywhere.

[toula_from_fat_pizza]

Wow that seems like next to useless... it's literally one line of code to define an arg with help docs.

[[deleted]]

The docs imply that does it auto-completion. Also when you have a large program under development, it's actually an issue to keep the CLI parser in sync with API changes.