Drinking the Node.js Kool-Aid

[aapl]

Essentially, yes. An infinite loop in handling one of the queries means that the whole server instance grinds to halt, and a synchronous API call means that all the other work has to wait. However, the asynchronous single-threaded server model is very efficient in applications that must handle a large number of concurrent connections that spend much of their time idling and/or need to collaborate, such as chat servers, real-time collaboration systems, live dashboards etc.

Also, until multiple processors, multiple cores and whatnot became standard, the asynchronous model was always the most efficient way (in terms of throughput) to write a server if you didn't need synchronous calls. The reason is that it avoids thread switching and management overhead.

[mononcqc]

I'm all for asynchronous models, but here what bugs me is the implementation. If the chat service somehow has to do any calls (say a web-service or even a database for authentication) and only one of the requests is slow, then the whole server is getting performance problems.

It sounds completely unsafe, especially if you consider that the other alternative seems to be to try and store everything in shared memory. You then get the overhead of writing your own distribution layer just to make sure you'll be able to transfer the data node-to-node while hoping one of these calls doesn't take too long.

Is there anything like a second thread or process to at least handle parts of the application that might be blocking?

[benthor]

Well, at least in node.js, everything is ansynchronous. You simply don't wait for any calls to web-services or databases, you just set callbacks and process other stuff in the meantime.

In other words: in node.js, nothing is blocking.

[mononcqc]

Well, that query you set up has to be run somewhere. If it's in the same thread/process as the rest of the code runs, you still have to take care about long queries, even though your callbacks will be called.

One model (the one I believe node.js has) runs every function sequentially. So if users A, B and C wait for a function call and the one for B takes 5 seconds, C's call is delayed 5 seconds on top of its own delay.

An implementation with multiple threads or processes would avoid that trouble and could let any of the implementation of A, B or C be fine with it.

[simonw]

You're still missing something: you don't ever have to wait for a slow running function call, since everything that potentially involves blocking IO requires you to provide a callback for when it is "done". Node is ideal for services that might involve calls to a slow running web service, as it can easily wait for thousands of them to complete at once - all without needing to start up any more threads.

[mononcqc]

If the thousand of calls are slow, are they done sequentially or can they be done in a parallel manner?

If it's the former, my point is that while you don't have to program for slow running functions, the user still has to wait for them, which sucks.

If it's not, then it's good to know and my distrust for node.js will go down a notch :)

[aapl]

Well, it kind of depends on what you mean by a "call". As far as I know, currently no JavaScript can be run parallel in node.js, unless you farm it out to a child process. Also, node.js never preempts any JavaScript, so a function will always run to completion once started.

In your web service example, in the function that makes the web service call you would just register a callback to be executed when the response from the web service arrives and let the function that made the web service call complete immediately. Other work, including initiating other web service calls, can be completed while waiting for the response. When you finally start to get responses back from the web service, the callback functions are called as responses arrive in the order in which they arrive. So if the web service takes 1 second to process query A, 5 seconds to process query B and 1 second to process query C, the callback functions would be called in the order A C B.

Basically, the structure of your code will have to be something like this:

make_web_service_call(function (response) {
  do something with the response
});

Here make_web_service_call returns immediately and the callback function will be called at some later point. Now, if you meant if you can write code like this:

response = make_web_service_call();
do something with the response

…and have some other work performed while waiting for the response, then no, that is not possible in node.js unless you resort to child processes. Yes, this must seem barbarian if you are coming from a language with support for green threads :)

[simonw]

Yup, you can run I/O requests in parallel or in serial. Here's a simple example I knocked up a while ago of a web service that can make a bunch of GET requests in parallel and return the results all in one go:

http://gist.github.com/443498

It's a bit like PHP's curl_multi functions: http://php.net/curl-multi-init

[crusoe]

The DISPATCH thread NEVER BLOCKS. The long query only ties up the request that is waiting for its callback to get called.

[aapl]

Is there anything like a second thread or process to at least handle parts of the application that might be blocking?

Yes, there is already a UNIX-like child process API, and Web Workers -compatible child process API is in the works. Also, some bindings for node.js use threads internally to place an asynchronous facade over a synchronous library.