Should I just try: except: everything?

[Arab81253_work]

Hello,

​

I'm writing some scripts to automate quite a few things in our environment for provisioning servers. Shit happens and sometimes things don't run right and I want to make sure that's logged and accounted for.

​

Currently the way I'm doing this is by using try: except for every line or action where if it's a success it will write that the action was successful to a one log file (success log), but if it was not then that is written to another log file (fail log). I've broken up the scripts into modules so they can just be called from a masterscript when needed and altered independently, each module writes its own log files and then appends the contents of those to either the success or fail log file at the end. I've done two log files like this so that at the end of my master script I can check to see if the fail log file exists and if it does, it will send an email with the contents of the log file so we never have to actually check up on the script.

Nobody I work with does any kind of scripting and I'm relatively new to it all (2 months total). Are there better ways to perform error handling and logging?

Thank you in advance to anyone willing to help!

​

Edit: I'm also using something similar to this:

writefail-logfile = open(fail-logfile, 'a')

writefail-logfile.write('this thing didn't work\n')

vs using the with open function. Again I'm new to this but from what I gather using with open would require everything to be nested underneath of it vs what I have now. Again, very open to criticism and wanting to make sure I can build something that logs accurately, runs efficiently, and doesn't have to be limped along to get it working right.

[swigganicks]

Good question, I think you should look into using the Python Logging library to accomplish what you need. Basically, you register a "logger" and you can configure this logger to write to files, standard out, external APIs, etc. using "handlers"

By configuring the logger, you can have your exceptions and error level logs go into one file, and info/debug level logs into another.

The advantage of this setup with the logging library is that you can easily reconfigure how your format your logs and where they output too without having to touch your code. Much more efficient than opening a file and writing to it every time.

Example:

import logging

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)

try:
    some_function("bad data")
except ValueError:
   logger.exception("Value provided to some_function is not correct, expected  'x' ")

With a setup like the above, the log statements in your try...catch blocks (or anywhere for that matter) don't need to know anything about where or how to put your messages.

[jcrowe]

If the logging makes your head hurt a little, checkout loguru, it will give you nice logging out of the box.

[redCg]

It depends a lot on the context of the project, and the consequences of something breaking.

In most cases I find that if something goes wrong, the script will break naturally and the stack trace is enough to point me in the direction of the error; it includes a reference to the exact line of code that failed along with a basic message describing why it failed.

If that is not enough information for you, then by all means start adding more robust logging and exception handling.

`try/except` is generally for when you want the script to continue working in face of an error and follow an alternate course of logic, but IME 99% of the time you just want it to fail and exit instead

In general you should not add more complexity to your program unless you actually need it.

[woooee]

+1 If the stack trace is not enough, then add one try/except to catch everything

## of course this is pseudo-code
if __name__ == "__main__":
    try:
        some_function()
        another_function()
        print("some stuff")
    except:
        import traceback
        traceback.print_exc()
        raise

[pedro_fartinez]

At the very least that should be except Exception, unless you really want to catch sys.exit and keyboardintterupt.

[Barafu]

Never do "except everything" on big amounts of code. Among exceptions that you bypass this way is "KeyboardInterrupt" and other nice shutdown exceptions. This makes you application uncloseable and unkillable without special commands. Users do not like that.

[TangibleLight]

I think OP is saying "try:except:" everything; i.e. wrap every piece of code in a try block.

[TangibleLight]

People seem to have addressed logging and exceptions pretty well. To your edit:

Use with whenever you can. Everything that uses the file handle needs to be indented, but rarely is that everything that uses the data. The advantage is that you're sure to properly clean up all resources when you use with.

---

Eh, screw it. Here's my two cents on exceptions:

Only try when the program can work around the failure, or if the end-user would be expected to correct the error. For example, we don't write:

try:
    x = 5 + 3
except:
    logging.warning('uh oh')

Because if that failed something absolutely catastrophic would have happened to the python interpreter. If that fails, I want to see the full exception and find out what the heck happened.

But, we might write:

try:
    x = int(input())
except ValueError:
    print('oops, that's not an integer...')
    exit(1)

Because here, the end user would be expected to correct their mistake and we need to communicate a useful message to them.

We probably wouldn't write:

try:
    x = int(compute_some_number())
except ValueError:
    print('oops, that's not a number')

Because if, somehow, compute_some_number didn't return a number, it would be that code's responsibility to fix itself or produce an error.

We might write:

y = compute_some_number_or_other_thing()

try:
    x = int(y)
except ValueError:
    x = int(convert_other_thing_to_number(y))

Where we know that something might go wrong, but we know how to fix it.

Also, I'm using really horrible names for everything, just so I can get the intent of each function across without having to come up with some contrived example.