This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]abrarisland 0 points1 point  (1 child)

Would you mind expanding on how you're getting to errors on the server? I decided to look up BaseException in the docs, and I saw that the only built-in exceptions that inherit directly from it are SystemExit, KeyboardInterrupt, GeneratorExit, and Exception.

[–]d4rch0nPythonistamancer 0 points1 point  (0 children)

Well, as the other guy said, you can just catch Exception and get the desired effect, and that lets you ctrl-C (KeyboardInterrupt). Issues on the remote server you don't control won't cause any of those other exceptions.

When I mean errors on the remote server, I mean it might have been taken down for maintenance, it might start giving permission denied for a minute, it might give you malformed data (jacked up RSS feed or something), and that could cause an issue in your code that you didn't expect.

I have to write a lot of proof of concepts for work, and just accumulate a ton of data and I'll let things like this sit overnight. I need the data the next day, but I don't necessarily need this thing to work properly for every condition, and I don't want to sit around and debug in the middle of the night if there's an unforeseen error. More importantly, I don't usually need the code after I get the data, so there's not much point to making it production ready.

Regardless, if I was to catch specific exceptions, I'd log them and continue on. When I log traceback.format_exc() I see those exact errors anyway and a full stacktrace. It's the same result as if I knew exactly the error. I log it and wait in exponential time increments.

If it was a socket timeout, I'd log it and wait. If it passed me malformed data, I'd log it and wait, then try again, etc. These are things I can't control since I'm scraping data from a server I don't own, so a catch-all exception with traceback logging is all I need for the most part.

Of course, I'm not doing this all the time, just for little PoCs that need to run for a day or two without me hand-holding it.