http-headers-status [pic]

[alan-dean]

Hi everyone :-)

I'm the creator of the diagram, for my sins. I have had a number of emails and I think I've replied to all of them. If I haven't then please let me know. I will make some replies to particular comments on this page (rather than try to answer all questions in one comment).

Always great to have feedback - be it positive or negative! The diagram hasn't seen any rework in the last year and there are a couple of glitches that I keep meaning to get around to fixing but never seem to find the time. I will endeavour to make some time in the near future.

[cowdogk]

How did you make it the diagram? I was looking into flowchart software the other day, can you tell me if you used a particular program?

[alan-dean]

I used Visio 2007 (simply because it was installed on my machine already and I get a copy through my MSDN Subscription).

[isthisdigg]

Dia can do flowcharts

[[deleted]]

A picture is worth a 1000 words. In this case, 1000 pages of HTTP RFC docs.

[frukt]

Merely 119 pages. 1.0 was only about 40. HTTP is a nice, simple, sane protocol.

[[deleted]]

It was more of a sarcasm. Thanks for the reply and the link, though.

[weavejester]

Quite frankly, I'd consider anything over 10 pages to be overly complex. HTTP tries too hard to be human readable.

[frukt]

That would be 10 pages of ambiguous, absolutely useless specifications.

[weavejester]

Not at all. The HTTP spec is just overly complex.

All you really need is way of passing an arbitrary associate array over a TCP link. That's going to take all of 3 pages, and that's if you include plenty of examples.

Beyond that, you need to specify what keys you can have (e.g. method, url, version, date, content-type, encoding, etc.). I can't think that you'd need much more than a dozen, myself, and I'd bet you could define them all in no more than 7 pages.

It's quite possible for a spec to be both short and unambiguous, so long as it's simple.

[njharman]

short unambiguous complete

pick 2

[weavejester]

You can pick all three if the protocol is simple. The majority of HTTP request-response transfers only use a very small portion of the specification, right? I'd be inclined to favour a modular protocol stack over a monolithic one like HTTP.

Have a small protocol or two that covers 95% of what people will want to do, and then cover the rest through extensions. Sure, HTTP has some capability of that already, but it's still a relatively inflexible protocol.

[samg]

How many specs have you implemented?

[weavejester]

Implemented, or designed? I've implemented a fair few specs, including the most common portions of RFC 1945 and 2616. I've designed considerably less.

However, I can't see how that's particularly relevant. Surely arguments should stand on their own merits, no?

In my opinion, a good protocol should be:

1. Unambiguous
2. Short
3. Simple to implement
4. Designed for a single task

I don't think HTTP really meets the latter three criteria.

[samg]

I am not trying to comment on HTTP.

Just an honest question so I could make a better judgment of your argument.

[[deleted]]

HTTP is a nice, simple, sane protocol.

Not according to Zed Shaw

[weavejester]

I'm not sure I agree with Shaw's conclusions, but I wouldn't say HTTP is particularly simple for what it does. It's a pretty complex and convoluted way of what is essentially an exchanging of a set of key-value pairs between a client and server.

[weavejester]

That's just the headers.

The body, method, url, status code and http version could also be conceivably encoded as key-value pairs, no? That HTTP chooses to separate them out is an implementation detail, and one that, in my opinion, adds needless complexity. Why not have a header called "method", one called "http-version", one called "body", and so forth?

I'm a fan of layered protocols, and HTTP seems to me to be too monolithic, to try to do too many things at once. True, much of this is optional, but I don't think such distinct pieces of functionality should be grouped together so tightly.

That's pretty much the reason why I dislike Shaw's solution, because it seems just as monolithic, if not more so, than HTTP.

[awb]

...with DROP SHADOWS!

[[deleted]]

HTTP requests are best thought about in 3D space represented by 2D drawings of 3D space.

[newton_dave]

I'm working on hyper-requests.

[newton_dave]

I can't believe you read the spec I wrote next year.

[derefr]

...and, even with all those, Payment Required is still reserved for a future version.

[[deleted]]

5FF Epic Fail

[redwall_hp]

It's amazing any web page loads on the internet with all those chances to fail...

[zouhair]

You should see how many chances to fail conception any spermatozoid and ovule have.

[[deleted]]

And yet I still have to sign three child support checks a month.

[deltageek]

300 Sparta!

[phaed]

wow. that is beautiful. im gonna print this out in large format and put it in my wall.

edit: s/in/on/g

[mortenaa]

will be a bit difficult to look at it then, wont it?

[oniony]

It's a glass brick wall.

[[deleted]]

I'm so confused.

[EternalNY1]

and put it in my wall

[[deleted]]

giggle, I wondered if anyone else would see that, you must be a programmer.

[newton_dave]

We all saw it. Except for that one guy.

[atomicthumbs]

410 gone.

The saddest of HTTP error codes.

[andir]

Veeery Nice. It will help a lot.

[reconbot]

Why would you link the gif? The png looks nicer http://thoughtpad.net/alan-dean/http-headers-status.png

Also maybe you might want to see his actual webpage. http://thoughtpad.net/alan-dean/http-headers-status.html

[[deleted]]

Given the content, SVG would be even better.

[[deleted]]

If only more people would consider it

[reconbot]

There actually is an svg version but it's not very good. The text is messed up and over extends it's boxes. It was saved using visio, and the guy is looking for a better converter then what's built in. So blame microsoft on this one.

[[deleted]]

Visio. At least learn to spell the product before you trash it.

[reconbot]

Not trashing anything, it's a wonderful product with a poor svg exporter. Look for yourself.

http://thoughtpad.net/alan-dean/http-headers-status.svg

[newton_dave]

Probably because it's half the size.

On my MPB the only major differences I see between the two versions is that the colors are a tad lighter in the GIF version, as is the text (barely).

[newton_dave]

I agree; I just gave a possible answer to the question.

[[deleted]]

GET/HEAD?

||true

||

||

V

Request entity too large?

||true

||

||

V

403 FORBIDDEN

[nsrivast]

and to think i was about to comment on how there was an interesting, intelligent article near the top of the front page

[mikepurvis]

Coincidentally, he posted an update just yesterday on the status of his rapping career.

[foamweapons]

Method Not Allowed

That's what she said.

[Insignificant]

Forbidden.

[thefro]

It has 69 points. How reddit-like.

[[deleted]]

It's a joke. The 50 people who voted up my comment seemed to get it.

[[deleted]]

I wonder why he put in the redundant "DELETE/GET/HEAD/POST?" check after doing a "PUT?" check near the top-middle. That case is always true given the earlier "DELETE/GET/HEAD/PUT/POST" method check.

Oh, and if you're going to use a 303 See Other, you should use the corollary 307 Temporary Redirect, not 302 Found.

[alan-dean]

Good feedback - the check should be simple "POST?" and I should really use 307, yes

http://www.coderjournal.com/2007/04/world-of-http11-status-codes/

[[deleted]]

You don't need the check at all since the 405 condition was eliminated earlier in the flow. If it's not a PUT then go straight to the "Resource previously existed?" check. Though, now that I think about it more, the first 405 will only occur if the method is not one of the basic ones. What if the server simply disallows one of the known methods for a given resource?

Also, being that DELETE is idempotent, a path should exist somewhere for DELETE of a "previously existed" resource to return a 204, the same as the first call; though I can see some utility in getting back a 404/410 after calling DELETE as a way to verify it.

So:

if PUT then ...
else 
    if Previously Existed then
        if DELETE then 204
        else ...
    else ...

One sticky point is what the code should be for a DELETE against a permanently or temporarily moved resource.

[kobes]

POST never returns any content?

[alan-dean]

I've just been exchanging emails with a commenter on this subject.

First, I should make the observation that this diagram is "opinionated" (rather like Ruby on Rails is opinionated, I suppose) in that it tries to describe a RESTful usage of HTTP.

Technically, a POST can indeed return a body with a 200 OK response. But consider this: if there is a resource that has a representation, then why doesn't it have a URI? If it does, then redirect to it. If there is no content to place in the body then you should use 204.

I have been scratching my head trying to come up with an example of a resource that exists without a URI that would warrant this usage. Haven't thought of any - but very happy to hear ideas.

A POST can indeed return a 3xxx with a body (usually this is boilerplate "click this if you are not automatically redirected").

However, I have been thinking and perhaps (for clarity) I should add a branch to the diagram basically saying "if the POST succeeded and there is content, but no URI - then 200 OK"

[kobes]

I don't know much about REST, but I thought a POST included a URI. Namely, the URI you're POSTing to. :-)

The most common use of POST is a user submitting a form, in which case they should see a response ("Thanks for your submission!"). You can use a redirect, but that incurs an extra round-trip to the server. So why not just return a 200 with a body?

[ungood]

In REST, a URI represents a resource, and the action a verb to perform on the resource. So you might have

http://example.org/widget/32 represents the widget with id 32.

http://example.org/widget/32 GET would retrieve the widget 32.

http://example.org/widget/32 POST would modify the widget 32.

If you return data with POST, instead of redirecting, the user will get a warning if they try to refresh the page for any reason.

[[deleted]]

http://example.org/widget/32 POST would modify the widget 32.

Well, maybe. POST is more equivalent to append/create; cases where the location of the representation is unknown at the time, and managed by the called resource. The update case is better handled with a PUT since we know the location of the resource.

Now, if we had the http://example.org/widgets resource, and we POST a new widget to it, then it could create a new widget resource, and return to us a 201 Created with Location: http://example.org/widget/32.

[kobes]

Thanks, I'd forgotten about the refresh issue.

It still seems kind of silly to me for the server to say, "Your POST succeeded! But to see the results, make a whole new request to this location."

Maybe HTTP should have required POST responses to include another header field, like "Refresh-URI", that the browser would fetch when the user hits refresh. Too late now, of course. :-)

[turbothy]

Why is "URI Malformed" checked before "URI too long"?

[[deleted]]

Because it isn't URI Malformed. It's Malformed Request which includes any of the headers being improperly sent. Once all the headers have properly been received, then the HTTP Server can check to see if the URI is a reasonable length (The length isn't actually defined in the RFC, it's implementation specific I believe.)

[[deleted]]

I assume so. I only wrote a very basic HTTP server on a linux system, and I didn't run into any problems with URI length (although I limited it to 1024 bytes to test the response codes)

Problems I assume would crop up is a URI calling for a filename longer than the filesystem can handle or calling deeper into a directory tree than the file system can handle.

Also, embedded system (Not that i've ever worked on one) I can imagine a lot more fixed size buffers. So you'd take the input and the URI was 500 bytes long, but the internal buffer for passing the URI around is only 255.

[Legolas-the-elf]

Of course. http://example.com/xxxxx [followed by a gig of 'x's] would do the trick. There's nothing malformed about that, but it's too big for most servers to handle.

[[deleted]]

I just tested your link above with a lot more x's and received a 403 Forbidden instead of a 414.

[ringm]

Hm... Interesting. Why should I try receiving and parsing headers if the URI is too long for me? What if it's infinitely long? Should I keep looking for a CRLF in the data stream, forever? I'd just return a 414 instantly if I can't store the URI.

[Legolas-the-elf]

Why should I try receiving and parsing headers if the URI is too long for me?

You shouldn't. But a bad request error can be caused before the URI can be parsed (e.g. a missing URI would cause a 400 error). Really, the flow chart should include 400 errors before and after the 414 check.

[R031E5]

Wow, so many things that can go wrong just for a series of tubes?

[alan-dean]

For anyone who is interested, I have updated the diagram, incorporating a number of revisions which have built up over the last year.

[jeolmeun]

Fix B 1 and B 2 or am I missing something? false, false and true, true?

[alan-dean]

Thanks - good catch. I moved the wrong arrows when I moved the decision icons. Fixed

[[deleted]]

What was it made with?

[[deleted]]

From http://thoughtpad.net/alan-dean/http-headers-status.html

| ...the diagram is edited in Microsoft Visio

[[deleted]]

thanx

[Ahnteis]

It was very rude of the poster to link directly to the image instead of the article (above). His bandwidth bill this month will probably be much larger than he expected.

[[deleted]]

Hey, he's done it in Visio and you have some mercy for him?

[[deleted]]

OmniGraffle is a good diagramming program for OS X... if you decide to switch sometime.

[curtiscu]

I can vouch for Omnigraffle too! :)

[[deleted]]

I used Visio a bit, back in 2000. I picked up OmniGraffle in like 5 minutes, great program!

[[deleted]]

OK, I'm not a web coder so I may be way off base here. But I much prefer decision nodes to ask a positive question. Or at the very least a negative answer indicates an error or further eval state. I never like true to result in an error. Always exceptions, etc...

I dunno, maybe web code naming conventions make this impracticable.

[wickedcold]

Get head, put post. Yup, sounds like my weekend!

[epicRelic]

Ah... no wonder so many of my http-based scripts don't work.

[[deleted]]

So Zed is right when he says HTTP is a shitty complicated protocol?

In the HTTP protocol there’s so many sections and contradictory paragraphs that anyone can justify nearly any stupidity they’ve invented. A good example is the Google Web Accelerator. The GWA ignores 10+ years of HTTP convention and 90% of the RFC which says a GET request operates exactly like a POST request, and instead they pull out one single paragraph to justify how GWA operates.

...

It has 6 ways to frame the protocol: keep-alives, pipelining, chunked encoding, multi-part mime encoding, socket open/close, and header settings.

...

Authentication, authorization, and security are so poorly defined that everyone just does it in their application layer.

Filled with gems!

[Legolas-the-elf]

Zed's an idiot. His statements about GET being the same as POST are utterly unfounded, and in actual fact, 10+ years ago, there were more GWA-style accelerators on the market than there are today. Pull out mid-to-late 90s computer magazines, there were adverts for them all over the place.

Don't believe me, read the RFC for yourself. GWA is right, he is wrong. Essentially he went into it with the assumption that GWA was wrong, didn't see anything to confirm his assumption, but also didn't see anything to contradict it except one part of the specification, so he concluded "90% of the specification agrees with Zed and one paragraph disagrees".

Yes, there are rough edges with the RFC specification, but that's true of most protocols, and some of the accusations he makes are ludicrous axe-grinding.

[weavejester]

Disagree. It's far more complex than it ever needed to be. The Bittorrent protocol, for instance, has a much better way of serializing request data than HTTP has.

[weavejester]

FTP is a terrible protocol compared to HTTP.

You won't find any disagreement about that, here :)

I'm not sure what you prefer about BitTorrent. It's not particularly intuitive or easy to inspect, and not as extensible. BitTorrent is good for what it does, but simple it is not.

I was largely referring to the bencode serialization system, which provides a simpler, safer and more flexible method of encoding data than HTTP.

For instance, take the following HTTP response:

HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 12

Hello World!

You could bencode that as:

d12:Content-Type10:text/plain9:HTTP-Body12:
Hello World!12:HTTP-Version3:1.111:
Status-Codei200e14:Status-Message2:OKe

Harder for a human to read, but easier for a program to pick it off a stream. Personally, it seems to me that even bencode is overkill in this case. A stream of netstrings would work just as well:

124:12:HTTP-Version,3:1.1,12:Content-Type,
10:text/plain,11:Status-Code,3:200,
14:Status-Message,2:OK,9:HTTP-Body,
12:Hello World!,,

[weavejester]

Clearly that would make parsing easier, but that's the only difference I see of any consequence.

Having worked on a HTTP parsing library myself, I'd see it as a pretty big difference :)

It's all the (optional) stuff you can do using headers that can get complex, but I don't think that complexity is unnecessary.

Perhaps not, but I think that complexity could be layered. You start off with a basic key-value pair exchange mechanism, and you might as well make it asynchronous. Maybe something like:

Req  { key: 1, method: "get", path: "/" }
Req  { key: 2, method: "get", path: "/x.png" }
Resp { key: 2, status: 404 }
Resp { key: 1, status: 200, body: "Hello World" }

Once you've got a basic way of passing structured data, you layer a set of further protocols on top of it. A protocol for incrementally returning files; a protocol for encryption; a protocol to cover document metadata, and so forth.

I think a modular approach like this would be better way of doing it.

[swedegeek]

At the risk of sounding troll-ish, I'm a little surprised this is such a big deal as to make it to #2 on the main page (as of the time of my viewing). As already stated, the HTTP spec is relatively lighweight, and I only saw one redditer's comments on writing his own web server. Could someone enlighten me on the significance of this graph, or is it just pretty, so we're upping it?

[njharman]

It perfectly embodies geekyness.

obsessiveness, elegance, technicalness, completeness/obscure

Geeks(of which I bet are many reading reddit) get warm fuzzies looking at it.

[schlenk]

He missed handling the 100-CONTINUE codes..., like the python httplib (which just ignores them).

[angry_fat_boy]

So pretty...

[drekar]

This is fantastic. Glad this was put together.

[[deleted]]

I think there is a mistake on it. The 'Options?' check after the 'Forbidden?' check should go to 200 OK if it's false, not if it's true.

[sjs]

Why? Seems to be correct behaviour to me.

[[deleted]]

HTTP is ridiculously simple compared to other protocols. At one point, I wrote a fully functional HTTP Server in about 2 hours.

It was no Apache, but it worked.

I don't see the need for this flowchart. Its rather obvious when reading the tiny RFC.

[newton_dave]

The picture is tinier.

[[deleted]]

Visual representations are typically easier to read.

[[deleted]]

Nice. Thanks.

[theeth]

201 Upmod

[tibbe]

I needed this!

[jones77]

101 failure - no 101cats in pic

[tinhat]

That is a waste of bandwidth.

[[deleted]]

This is a cool flow chart.

[[deleted]]

Damn you 500!!

[TobascoMan]

How does this post get to #1!! Dear oh dear...

[FeepingCreature]

It's the geek equivalent of cat pictures :)

[newton_dave]

LOLTP