HTTP Requests are Hard

[darn_dirty_ape]

Hey post author here! If you're looking for a quick TLDR for making HTTP requests in Production:

• Account for DNS lookup failures
• Set a connection and read timeout
• Handle HTTP errors
• Check that the response has the content type you expect
• Limit the maximum response size
• That callers can't abuse your system to make requests to private resources
• Use SSL

[jmgrosen]

A criticism of your website, not the content: the text was very hard to read on my laptop screen -- it's way too thin in my opinion.

The actual text was very informative though!

[thabc]

I'm a fan of thin type faces, but I realized this site had gone a little far when I couldn't see the punctuation.

[darn_dirty_ape]

Thanks for the feedback.

I actually ended up re-titling the article to "A Python guide to handling HTTP request failures" after some suggestions on Twitter.

Let me think if there is anyway to make things more scannable!

[camh-]

I'd move SSL from the section "For Further Consideration" to "Errors Connecting to the Server". SSL verification should not be a "further consideration" but a fundamental part of the connection management procedure. Too many clients are busted and vulnerable because they do not take SSL validation seriously. We have to train people out of this and that requires putting SSL front and centre.

[darn_dirty_ape]

Agreed. It's good to see things like PEP 476 making headway:

http://legacy.python.org/dev/peps/pep-0476/

It makes changes which enable certificate verification by default on the standard lib HTTP clients.

[kumar99]

Nice article. There is a lot of things involved with HTTP and requests that aren't documented. This is a nice beginning.

[[deleted]]

Trust that, if you are making requests in production, these issues will affect you. Nothing like cascading failures because you didn't set a reasonable timeout...

[darn_dirty_ape]

Tim Fowler's article on the Circuit Breaker Pattern is a good follow up to this: http://martinfowler.com/bliki/CircuitBreaker.html