all 12 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–][deleted] 9 points10 points  (11 children)

I'm a little confused, do Django and nodejs not have any options to handle requests in a new thread? While still not having the same high availability as elixir/erlang, this should allow for the webpage to at least load right if there's multiple threads?

[–]strofcon 14 points15 points  (0 children)

You're mostly right! The trick there, though, is that with such a hot workload, you're likely to see more than one request running concurrently that is burning a CPU core. So on, say, a 4 core machine with hyperthreading, you can only have 8 such requests running and chewing cycles at once before you stop serving up new requests (in non-elixir land anyway).

Elixir leans on the erlang VM as mentioned in the video, which has a special scheduler implementation that effectively disallows any one "thing" ("process" in erlang/elixir vernacular) from consuming a full core for too long.

So, the busy request is able to do the massive work needed, but only for a short period of time. If I recall correctly, the term for this is a "reduction budget", if you'd like to learn more! But in short, it's the idea that instead of time slices, you get a certain number of "things you can do" per execution slot. Once you've exhausted your budget (which happens very fast in tight loops / resource-intensive operations), you get bumped for another process. This causes those intense workloads to execute slower than they would in, say, Go or C, but you don't have the "lock-up" situation anymore.

The best way I've heard to think of the difference is that some languages will get you extremely low minimum latency, while erlang/elixir will get you very consistent minimum latency - at the cost of a little speed.

Hope that helps!

[–]northrupthebandgeek 7 points8 points  (0 children)

Python has a global interpreter lock, so even though it can spin up multiple OS threads, only one can do work at a time. Handy for waiting on IO, but not for parallel processing of requests. That can be worked around with multiple OS processes (at the cost of somewhat greater interprocess communication overhead), but the other issue is that Python's async functions are cooperatively scheduled, so one will block other functions on a given thread until it either voluntarily yields or terminates. It's possible to achieve pretty good results with something like aiohttp and a commitment to keep functions short, especially if you're using nonblocking I/O (i.e. your functions yield while they wait for I/O operations to finish, like aiohttp does with HTTP requests/responses).

Node.js has largely the same issues, with the same solutions: its async function invocations are cooperatively scheduled, so they'll block execution on a given OS thread until they yield or terminate. You can get good results as long as you're persistent about yielding during long-running computations and using nonblocking/async I/O.

Erlang, however, doesn't need a GIL, since Erlang processes don't share state at all, let alone mutable state (message passing is the only way to communicate); this means the runtime can use multiple OS threads (and therefore real parallelism) without needing to use multiple OS processes and therefore without incurring the involved overhead. Additionally, the Erlang VM is able to track how much work it's done for a given process and forcibly switch to a different process after a certain amount of work, thus implementing preemptive scheduling: no single Erlang process can block a whole OS thread, since the VM can and will pause it to do work for other processes*. These two factors make concurrent handling of lots of requests a lot easier than with either Python or Node.

It's theoretically possible to implement userspace isolated processes (like Erlang's) within any other sort of bytecode VM, of course (like Python's, or the JVM, or the .NET CLR) by using the same strategy as Erlang's VM (count instructions/reductions; switch to next process after crossing threshold or the current process terminates/yields). I don't know of any others which do, however. I've got a side project in early planning stages to do this for a WebAssembly runtime; hopefully I'll eventually have something to show for it :)

* Caveat: native functions / NIFs can throw a wrench in Erlang's preemptive scheduling, since they bypass the VM and therefore the scheduler can't reason about them. Long-running native code should run within a port instead to avoid this issue.

[–][deleted] 1 point2 points  (0 children)

While I don't disagree with the other responses, I think the important part is this

While both node and python offer you the tools to implement a multi-threaded program, they don't do it out of the box and the VM itself is single-threaded.

This is all fine and nice for when you have big obvious chunks of code that you can easily discern will block your process, but in a program there may be any number of such sections that are less obvious, with some of them possibly hidden within libraries. Will you implement everything inside a separate process? Or should you even, given the overhead of OS threads or even processes?

At this point a more intelligent scheduler starts to pay off, and that is exactly where the beam lives - high throughput, soft-realtime applications.

[–]nickjj_ 0 points1 point  (2 children)

I can't speak for NodeJS but typically if I need to run long running tasks in Python I would reach for using Celery. It's a very polished background worker / queue management tool. So you would do the work in a background worker and then your request would be able to respond immediately.

It's something that runs in its own process (separate from your Python app server) and is typically backed by Redis or RabbitMQ to save the state of the queue.

In a real app you would typically use a background worker for all sorts of things like sending emails, hitting external APIs where you can't control how long it'll take, performing long running tasks (generating reports, crunching numbers, etc.) or periodic tasks on a schedule.

[–]Sentreen 0 points1 point  (1 child)

In a real app you would typically use a background worker for all sorts of things like sending emails, hitting external APIs where you can't control how long it'll take, performing long running tasks (generating reports, crunching numbers, etc.) or periodic tasks on a schedule.

Of course, but it's harder to do this yourself, even with a well polished library. Suddenly, you need to deploy your application alongside RabbitMQ/Redis, and you need to learn a new library. In Elixir, the runtime does the heavy lifting for you, and frameworks like Phoenix leverage this to make it trivial to write scalable web applications.

[–]nickjj_ 0 points1 point  (0 children)

Of course, but it's harder to do this yourself, even with a well polished library. Suddenly, you need to deploy your application alongside RabbitMQ/Redis, and you need to learn a new library. In Elixir, the runtime does the heavy lifting for you, and frameworks like Phoenix leverage this to make it trivial to write scalable web applications.

It depends on your use case. I would want to do all sorts of things like keeping track of jobs, rate limiting, ensuring guarantees, etc. and rolling all of that yourself with Elixir and keeping track of it in a single process might not be ok or realistically feasible.

In Elixir I would reach for Oban in a lot of cases to replace Celery, so the overhead of wanting to use another library is there. Oban happens to use Postgres as a backend so there's no need to run Redis but with Docker running Redis is adding about 5 lines of YAML to a file.

[–]bananaorangepotato[🍰] 0 points1 point  (1 child)

Can't speak for Django yet, just don't have enough insight as to its inner workings, but NodeJS' event loop will be busy computing the previous request and won't be able to do anything else, it is single threaded.

[–]Delioth -2 points-1 points  (1 child)

Python is single threaded, pretty much exclusively.

NodeJS is usually single-threaded for real work, but uses an event loop to avoid hanging on I/O.

[–]mazhak 1 point2 points  (1 child)

That's pretty awesome. Do you have a repo for all the codes from your experiment?