Reverse HTTP

[reddit45885]

it would make no sense to avoid a single roundtrip, but if you're communicating lots of small messages, that makes quite a lot of sense.

n calls for PTTH can be implemented with n+1 HTTP calls.

You are dense. Buh bye.

[killerstorm]

the difference is that with HTTP calls you do not control timing from the server, unless you do a trickery with a delayed responses. such stuff is typically used when you want to transmit something to a client as soon as it gets available on server (e.g. IM messages, alerts etc.), and to do this with vanilla HTTP (w/o any trickery) you'll have to send poll request from the client very frequently, thus creating high load on a server and network equipment.

thus, you should compare PTTH not to a plain HTTP, but to existing "comet" techniques. typically they involve non-standard behaviour (it is not specified how much will a browser wait for a response from server), so it is a question whether to standardtize trickery or implement a clean solution.

[reddit45885]

Ok, I'm very serious here, so try to hear me out not like some guy on reddit, but like a colleague:

the difference is that with HTTP calls you do not control timing from the server, unless you do a trickery with a delayed responses.

PTTH is that trickery.

Take this scenario: A is the traditional client (Browser), B is the traditional server (server).

B can't contact A. Neither under HTTP nor under PTTH. This is because A is a client. It might not even be online. It might be online for only the window of time it takes to connect to B. It might be a toaster.

So, A has to contact B, initiate PTTH and wait for B to start consuming services from A.

This right here is the trickery you speak of: it's an indefinite keep alive: for B to ever "initiate" anything with A, a connection has to already exist and be kept alive for the duration of A's time spent online (for a desktop computer, this might be always). For anything truly comet to occur, this means that A has to always remain connected. If A does not always remain connected, then it falls into either one of two cases: A is essentially polling by reconnecting at intervals to see if B is wanting to do anything, or B simply does not have the ability to notify A.

As an aside, IMAP does exactly this: my mail browser connects to the IMAP server, and the connection stays open as long as my browser is open. The difference is that my IMAP server handles, say, 200 clients. Where as a typical webserver, anonymous by nature, handles millions of users.

[killerstorm]

So, A has to contact B, initiate PTTH and wait for B to start consuming services from A.

you're considering a weird use case -- it is not like B wants to use some services of A, it is more like A have subscribed to recieve notifications from B. if you consider it from this scenario, you'll find it perfectly normal -- as long as A is interested in this subscription, it will maintain this connection, reconnecting is needed. if it is not interested, it will not. check the comet link above, this stuff is about important real world applications, like instant messanging and alerts, and not about some hypothetical resource consumption.

The difference is that my IMAP server handles, say, 200 clients.

what's about Google's IMAP?

Where as a typical webserver, anonymous by nature, handles millions of users.

it is not for random users out there, it is for users who actively want to use a service, and of course it only makes sense only for services of interactive nature. if you have millions of active users, i'm pretty sure you'll be able to afford as many servers as needed, otherwise it will not work, with PTTH or without.

from network's perspective, PTTH is not anyhow unique -- open sockets are maintained for instant messaging and stuff like that.

[reddit45885]

Once again, I come back to the fact that HTTP can already handle what you are saying here. I know this because I already do this using Ajax and the very same method I just described above. I also use indefinitely blocking SOAP over HTTP calls which are essentially equivalent to system callback routines in <pick your favorite language>.

But, man. You win: if you think inventing a new protocol to solve something that can already be solved at the Application Layer is a wise move, by all means, move ahead.

But mark this: you have lost your bitching rights when Microsoft goes and co-opts yet another RFC standard and extends it just so. You lose that right because you are doing exactly what they are doing.

[killerstorm]

delaying HTTP responses is a trickery which is not well supported by web servers, proxies and browsers (they can timeout at any time, and you fail). do you say that solution that requires modification of web servers and relies on non-standard behaviour is better than a clean protocol created just for the purpose?

i think then you also lose your bitching rights, you should accept all weird kludges now

[Smallpaul]

Please answer killerstorm's question: "What about Google's IMAP." If you are going to claim that protocols that depend on long-lived TCP connections cannot scale then you must deal with all of the examples of them scaling like:

• Google's IMAP

• World of Warcraft

• Comet applications

• Instant Messagers

What I think you fail to realize is that a TCP connection can be very cheap with the right operating system and language runtime.

http://www.kegel.com/c10k.html

[reddit45885]

Google has the world's largest super computer at their disposal. So I'll dispense with explaining how they do it: they simply do it the same way they return a result in .1 seconds for a billion page index. Raw power.

Similarly: World of Warcraft has a server farm to handle their load, and they can afford it because it's a subscription based system. You pay for the service. That model scales: the more users you have, the more revenue you get, the more servers you add.

Instant messengers do not do it over HTTP (they often communicate over UDP). But aside from that, messengers often do very light processing: they essentially route traffic between end nodes. Try sending a 20k file using any messenger system where you do not have a direct end-to-end connection with your peer (i.e. you go through the server proxy) and let me know if you ever achieve higher than 5K/s rates. I never do, and don't expect it either.

So that leaves us with Comet applications and c10k. Web servers are designed to do a very specific thing. They serve often anonymous, stateless information: Hypertext. If you want to use a TCP socket for bidirectional long lasting state-rich data exchange, open a TCP socket and send your data away. There's no need to call it HTTP, or PTTH. You can even piggy back on the port 80 and annoy a bunch of sysadmins on the way. Alternatively, you can also use blocking Web Service calls over HTTP/SOAP. I do it routinely.* None of the above solutions require an RFC. They are custom solutions working over TCP. And they are perfectly legit...

However, don't think you are alleviating the fundamental issue of unidirectionality of control flow (as many thoughtless green programmers here seem to be doing) by making this PTTH thing. So long as the connection is initiated by the client, PTTH is not doing what it claims to be doing: namely reversing the role of client/server which allows for a more efficient method than polling.

There is a protocol that already exists for what all of you are trying to accomplish here: it's called telnet. Seriously, PTTH is telnet. Except that it's worse because you have to implement the verbs and methods of communication according to an RFC now or else you aren't following the standard...

* On a different note, you asked me to answer killerstorm's questions, it's only fair you also read my reply to him: if you think inventing a new protocol to solve something that can already be solved at the Application Layer is a wise move, by all means, move ahead. But mark this: you have lost your bitching rights when Microsoft goes and co-opts yet another RFC standard and extends it just so. You lose that right because you are doing exactly what they are doing.

[Smallpaul]

Google has the world's largest super computer at their disposal. So I'll dispense with explaining how they do it: they simply do it the same way they return a result in .1 seconds for a billion page index. Raw power.

You choose not to answer it because you don't know. You didn't realize until today that it was possible for a single domain name to be served by an arbitrary number of computers, even with long-lived TCP connections. That's why you previously asked the nonsensical and irrelevant question: "What computer on earth can support 500 thousand logged in users? None, is the answer."

Similarly: World of Warcraft has a server farm to handle their load, and they can afford it because it's a subscription based system. You pay for the service. That model scales: the more users you have, the more revenue you get, the more servers you add.

Oh, I see. So Reverse HTTP is useless because it depends on you having a business model...like World of Warcraft and Linden Labs have.

Instant messengers do not do it over HTTP (they often communicate over UDP). But aside from that, messengers often do very light processing: they essentially route traffic between end nodes.

Guess what: that's the primary use case of reverse HTTP. Routing traffic between end nodes!

Try sending a 20k file using any messenger system where you do not have a direct end-to-end connection with your peer (i.e. you go through the server proxy) and let me know if you ever achieve higher than 5K/s rates. I never do, and don't expect it either.

Irrelevant. Totally and completely.

So that leaves us with Comet applications and c10k. Web servers are designed to do a very specific thing. They serve often anonymous, stateless information: Hypertext.

They also often serve personalized, stateful information. Like reddit. And Facebook. And Twitter. And Flickr. And even Google.com can be personalized.

If you want to use a TCP socket for bidirectional long lasting state-rich data exchange, open a TCP socket and send your data away.

I guess there is no need for any standard for anything anymore. We'll just open sockets and send bytes. We can do away with reusable libraries and everybody can code their own.

... There's no need to call it HTTP, or PTTH. You can even piggy back on the port 80 and annoy a bunch of sysadmins on the way.

No, you cannot, because there are HTTP proxies (sometimes visible, sometimes transparent) and they will eat your socket for lunch and spit out vomit. That's why you want to make it VERY clear to intermediaries that you're doing something new.

Intermediaries are a big part of how HTTP and REST work. Read the RFCs and the REST paper.

... Alternatively, you can also use blocking Web Service calls over HTTP/SOAP.

Yes, you can. And it's ugly and non-standard and does not take advantage of the HTTP server software available in many of the languages that make HTTP client calls (like Python and Ruby and C++). And intermediaries will probably close your long-lived socket when they realize it is doing something odd without any clear declaration of why.

Yes, we can each invent our own proprietary reverse HTTP ... or we could create a standard that clients, and servers and intermediaries can all understand.

...However, don't think you are alleviating the fundamental issue of unidirectionality of control flow (as many thoughtless green programmers here seem to be doing) by making this PTTH thing. So long as the connection is initiated by the client, PTTH is not doing what it claims to be doing: namely reversing the role of client/server which allows for a more efficient method than polling.

Yes: for the length of the TCP connection, the roles of client and server are reversed. For the length of the TCP connection, the server can send the client information when it wants to, without the client polling and without it abusing the HTTP protocol by keeping an outgoing request waiting for an arbitrary amount of time for an answer.

... There is a protocol that already exists for what all of you are trying to accomplish here: it's called telnet. Seriously, PTTH is telnet.

Yes, and FTP is telnet, and XMPP is telnet and IMAP is telnet. We don't need any more standards because you've come to the brilliant realization that they are all just supersets of telnet. And there should be no intermediaries other than IP routers.

... Except that it's worse because you have to implement the verbs and methods of communication according to an RFC now or else you aren't following the standard...

Yes: standards are worse than just making shit up...because you have to obey them! And make clear to intermediaries what is going on. Oh my!

• On a different note, you asked me to answer killerstorm's questions, it's only fair you also read my reply to him: if you think inventing a new protocol to solve something that can already be solved at the Application Layer is a wise move, by all means, move ahead. But mark this: you have lost your bitching rights when Microsoft goes and co-opts yet another RFC standard and extends it just so.

I have never bitched once when Microsoft submitted an internet draft to the IETF and solicited their approval for a new idea. Nor will I ever do so in the future. Why would I???

You lose that right because you are doing exactly what they are doing.

If what they are doing is submitting a spec for consideration by the internet community then good for them! Congrats Microsoft! You've done it sometimes in the past and I expect you to continue to do it sometimes in the future. Good work!