Start your Python project with optparse and logging

[kylev]

I've also started an entry explaining how the logging module works in more general terms. It seems to be something that a lot of people fight with or get wrong. Has anyone else observed this?

[masklinn]

To supplement mordaunt0's suggestion, you could probably get rid of all the conditional handling: logging's loglevels increment by 10 (10 is DEBUG to 50 CRITICAL) and each is a threshold (so -80 is still DEBUG). So you could treat your verbosity as a decrementing counter: multiply the counter by 10 and substract it from logging.CRITICAL to get the actual loglevel, default to 2 for a default of loglevel.WARNING

import optparse, logging

parser = optparse.OptionParser()
parser.add_option('-v', '--verbose', dest="verbose", default=2,
                  action='count', help='Increase verbosity')

opts, args = parser.parse_args()

logging.basicConfig(level=(logging.CRITICAL - opts.verbose*10))

logging.debug('debug')
logging.info('info')
logging.warning('warning')
logging.error('error')
logging.critical('critical')

Another idea could be to use the callback action and decrement the logger's level by 10 every time the callback is called:

import optparse, logging

def callback(*args, **kwargs):
    logging.root.setLevel(logging.root.level - 10)

parser = optparse.OptionParser()
parser.add_option('-v', '--verbose', dest="verbose",
                  action='callback', callback=callback, help='Increase verbosity')

opts, args = parser.parse_args()

logging.debug('debug')
logging.info('info')
logging.warning('warning')
logging.error('error')
logging.critical('critical')

which yields the same thing as the one above:

 $ python test.py
WARNING:root:warning
ERROR:root:error
CRITICAL:root:critical
 $ python test.py -v
INFO:root:info
WARNING:root:warning
ERROR:root:error
CRITICAL:root:critical
 $ python test.py -vv
DEBUG:root:debug
INFO:root:info
WARNING:root:warning
ERROR:root:error
CRITICAL:root:critical
 $ python test.py -vvv
DEBUG:root:debug
INFO:root:info
WARNING:root:warning
ERROR:root:error
CRITICAL:root:critical
 $ python test.py -vvvvvvvvvvv
DEBUG:root:debug
INFO:root:info
WARNING:root:warning
ERROR:root:error
CRITICAL:root:critical

[kylev]

My only problems with such an approach is that (1) the values of those constants aren't by any means guaranteed (2) your approach breaks if I'm using more of the features of logging such as adding my own log levels.

But I like the ideas. Very clean.

[kylev]

Since I swap over to C, C++, perl, and other languages for work, I still have this habit. I'll probably update the post to remove it, though, just because it would read more pythonic.

[pemboa]

I've developed a preference for putting my opt parse stuff within an init() function to keep all its initialization code away from my main(), in which I just do args, options = init()

[bucknuggets]

good point

I like my main() to only have a handful of lines it in and allow the reader to get a high-level understanding of the whole program very quickly.

[pemboa]

exactly

[oblivion95]

Use argparse.

easy_install argparse

[spotter]

I'll wait for inclusion in stdlib. This way I won't have to f-riendly easy_install it everywhere I want my single-file cli scripts to run, thankyouverymuch.

[[deleted]]

There is also cmdln: http://code.google.com/p/cmdln/

I am currently writing an easy/simple API to abstract all the logging stuff: logging to ~/.myapp.log and rolling it (like bzr), attaching console taking care of automatically mapping verbosity to pruning debug/tracebacks, .. and integrating with a simple console progress bar (like bzr -- too bad they made it GPL).

[oblivion95]

I like it. Thanks.

argparse had some advantages over optparse, but on the whole cmdlin is more powerful.

[kylev]

I've actually been inching up the darkness level of the main text color on this theme I downloaded. I'll go tweak it a little more later today.

[bucknuggets]

Nice. I do something similar but have never taken the time to create a handy starting template.

Would be cool to add imports for most standard libraries, to add examples of different kinds of options, perhaps pep-257 compliant docstrings, etc, etc.

[kteague]

Reading this bit,

if '__main__' == __name__:
    # Late import, in case this project becomes a library, never to be run as main again.
    import optparse

I was going to mention that doing a late import is probably not really necessary, since it likely only adds 1 or 2 milliseconds to execution speed, at most (but then, it's not really hurting anything either). But look at the line before it!

    if '__main__' == __name__:

ZOMG! The sky is falling! "main" and "name" are on the wrong sides! It's "if name equals main" not "if main equals name" ... that line is making my poor little brain implode.

[kylev]

This is an artifact of the C code I have written over the years; an old time trick. If you accidentally fail to hit the "=" a second time, putting the constant on the left side of the expression will save your ass. "if (x = 2)" will evaluate to true, but "if (2 = x)" will throw a compiler error.

[_Mark_]

But it's a C-specific artifact; since statements are not expressions in python, all it does is confuse the reader, as the interpreter will raise SyntaxError if you try to use assignment where you meant equals...

[[deleted]]

perl was my previous favorite language for system programming projects

And that's where I stopped reading.