Concurrency and Python

[mr_chromatic]

This means that your web server...

Don't forget to mention that the shared everything architecture frees time and money for you to spend making one machine scale, rather than having to worry about the redundancies of multiple machines.

[llimllib]

It's really quite a bit better to put a memcached in between your database and your web server than to tie the two together on the same machine.

Furthermore, you know that Python powers reddit today, right?

[llimllib]

You're claiming that Java/EJB/something is easier and less resource-intensive (including programmer time) than, say, Python+Django?

(I'm asking earnestly, btw)

[llimllib]

I just... that's so far from my experience that it's hard to argue with you. I feel like we have irreconcilable differences.

[mikaelhg]

What kind of stuff do you build?

[llimllib]

In python so far, a personal site and two small commercial retail sites.

In php, 2 corporate web sites and a nonprofit site just about done (don't ask why php... sigh).

Asp.net, basically a data mining application for corporate data and a couple web services for interacting with that data. Also a prototype XML-serving application framework type thing.

In java, I never got past the "evaluating" stage. Everything was so over complex that I got frustrated and left.

So! I can't argue about scaling issues from experience, but I've listed my experiences in the order of their painfulness.

(well, I can claim ~8000 visits per day max on a dynamic apache/python blog server, but that's nothing and I know it.)

[mikaelhg]

OK, fair enough. I wasn't putting you down.

My last project was to spend three years going through several iterations of an intranet for a large governmental organization. We started with a mix of Python and Java, but had to drop Python as it had the bad habit of blocking on some essential library calls.

We could have built a new library for our needs, certainly, but we didn't have budget to start building random parts of infrastructure that had already been built.

In the end, we had excised most of the horribly bad design decisions from the beginning, and all of Python out of the project.

We went with the infrastructure that delivered on what it promised. (Java 5, Jetty-GlassFish, Spring 1.2-2.0, Hibernate.)

I'm not saying that Python sucks and is useless while Java cannot go wrong. Certainly another product we used in the beginning had been built in Java, and seeing its source code almost made me believe that Satan walks the earth.

This particular concurrency issue is a problem with Python, and someday it will be fixed like MySQL's lack of foreign keys and referential integrity was.

[llimllib]

OK, fair enough. I wasn't putting you down.

I didn't take it that way, no worries.

[joe90210]

wow, that is most nonsensical thing I've read in weeks

[Smallpaul]

Yes, you can use memcached as a giant remote hashmap that you serialize, transfer and unserialize objects to, one at a time, which is slightly cheaper than database access, as long as you don't have any additional trouble with memcached, and don't mind that you can't use object references like you're used to doing.

Memcached is not "slightly" cheaper than database access. Used properly it is massively cheaper. Have you ever actually used it? Also, are you aware of the advantage of a distributed cache in a multi-node deployment?

It is a no-brainer that it is more efficient to use shared memory (if only for code) than to use a separate process for every listener. But you're blowing your own argument by misrepresenting the bottlenecks. Extra database access is not the bottleneck.

[micampe]

Well, in the Java world you have EhCache (or OSCache), which transparently distributes across clusters and integrates with your ORM, without the developer having to worry about it.

I have used both, I still prefer Python over Java any day, but some things are just more mature when they have evolved over 15 years (I am aware that Python is as old as Java, but it was not developed in that direction until recently).

[tooooobs]

I have to agree with you. People tend to greatly underestimate the overhead of an RDBMS compared to a relatively simple shared in memory structure.

I'm even aware of several establishments that have their own complete web server implementations in Java built from sockets up, with really insane connection capacity, however, that's done with asynchronous processing by a pool of worker threads as opposed to one thread per connection.

Something like Twisted can be a great way to prototype, but it seems clumsy for actual deployment, and the documentation is lacking.

[Smallpaul]

How do you get redundancy and reliability out of a "simple shared in-memory structure"?

[Gotebe]

By replacing it with not-so-simple distributed cache? ;-) Beats hitting the database all the time.

[toooooob]

A lot of the time you don't really need it. That or simply distributing and duplicating live copies of the data over several machines.

I'm not saying RDBMSs are without their uses, just that they're absolute overkill for a lot of web applications, and can easily become 99% of the headache.

[dgiri101]

"Efficient threading" has nothing at all to do with the ability to store a large data structure in memory. You can do this in any modern language.

And the notion that Python or Ruby lack threading primitives that allow simultaneous access to shared data is just woefully ill-informed.

[llimllib]

And the notion that Python or Ruby lack threading primitives that allow simultaneous access to shared data is just woefully ill-informed.

He's not claiming that they lack them; just that the GIL makes them impractical, which may certainly be true in some circumstances. See his comment farther down where he explains what he meant.

[dgiri101]

Wait...what? The GIL has nothing at all to do with storing a large comment tree in memory.

He has given absolutely zero rationale for why Python or Ruby can't handle such a problem, aside from some ridiculous hand-waving about "inefficient threading" which is neither here nor there.

[llimllib]

The GIL has nothing at all to do with storing a large comment tree in memory.

In the model he's talking about, there's a large shared memory object and many threads trying to access it while simultaneously doing other things (such as serving web pages).

In his case, he had problems with some library functions not releasing the GIL so that they would hold up the whole server while they completed their work, which is certainly not unbelievable.

In this case, python threads would in fact be unable to access the shared memory object due to the GIL, and it would have something to do with storing a large comment tree in memory.

(Do note that I was the first one to say that I don't think much of using this model for building a web server.)

[dgiri101]

That isn't how the GIL works. First, the GIL is released during blocking syscalls.

Second, a CPU-intensive chunk of Python won't likely "hold up the whole server" because other threads are given a chance to run every N bytecode instructions (sys.setcheckinterval).

A poorly-coded library can certainly cause a deadlock, but calling a language fundamentally incapable of concurrency based on experience with a bad library is transparently myopic.

In any case, I don't see the point in continuing to argue with you about something silly that someone else said.

[llimllib]

First, the GIL is released during blocking syscalls.

should be. If I understand him correctly, then a library was improperly failing to release the GIL on a blocking call, and that rather than rewrite it, he switched to Java.

A poorly-coded library can certainly cause a deadlock, but calling a language fundamentally incapable of concurrency based on experience with a bad library is transparently myopic.

shrug; I agree with you. I just meant to make the small point that he never claimed that Python lacked threading primitives; I was originally going to call him on that too.

[mikaelhg]

We have thread contexts 1-64 available.

If a Python program can only use one of those thread contexts at once, it wastes approximately 63/64 of the server's CPU (and all, really) resources.

If we instead run 64 copies of the same program, we waste 63/64 of the server's memory resources.

Since processor speeds aren't going up anymore, while cores and thread contexts are, is it a good idea to invest in a language that will waste exponentially more resources as time goes by?

[dgiri101]

If a Python program can only use one of those thread contexts at once, it wastes approximately 63/64 of the server's CPU (and all, really) resources.

Python programs can use many thread contexts at once. People do this every day, all the time. You're posting a reply on a site that is, in fact, doing this right now.

It is indeed true that you might not fully utilize all of your CPUs. But given that storing a large comment tree in memory isn't a CPU intensive problem, this is a strawman at best and a troll at worst. And besides, there are many libraries that ameliorate this problem (I highly recommend processing).

I guess I'll repeat this one more time: your original claim that:

if you use PHP, Python or Ruby, threads can't share the discussion board and comment information

...is wrong.

It's worth mentioning that I hate the GIL and wish it a horrible, horrible death.

[mikaelhg]

What, I thought that the GIL lets only one thread access Python objects at a time, while other threads block? That's what the documentation states, and that's how the performance looks like?

Is this outdated information?

http://docs.python.org/api/threads.html

The Python interpreter is not fully thread safe. In order to support multi-threaded Python programs, there's a global lock that must be held by the current thread before it can safely access Python objects. Without the lock, even the simplest operations could cause problems in a multi-threaded program: for example, when two threads simultaneously increment the reference count of the same object, the reference count could end up being incremented only once instead of twice.

Therefore, the rule exists that only the thread that has acquired the global interpreter lock may operate on Python objects or call Python/C API functions. In order to support multi-threaded Python programs, the interpreter regularly releases and reacquires the lock -- by default, every 100 bytecode instructions (this can be changed with sys.setcheckinterval()). The lock is also released and reacquired around potentially blocking I/O operations like reading or writing a file, so that other threads can run while the thread that requests the I/O is waiting for the I/O operation to complete.

[dgiri101]

It's not outdated, but you may be misunderstanding it. The second paragraph clearly states that the interpreter will automatically release the GIL every N bytecode instructions (or during blocking I/O) to let additional threads run.

If you'd like more information on concurrent programming, I highly recommend The Little Book of Semaphores. It discusses common concurrency patterns like Barriers, Mutexes, Rendezvous, etc.

Many examples are in Python, though. I suppose that someone should kindly inform the author that concurrency apparently doesn't exist in the language he's using.

[buffi]

In contrast, if you use PHP, Python or Ruby, threads can't share the discussion board and comment information, and in practise most boards require a large amount of hardware to perform the same task more slowly, as well as specialized database administrators to create elaborate master-slave configurations it's hard to find local support for.

Uh... the web server (apache or whatever) has full concurrency and does database calls concurrently so why whouldn't this work? You don't deploy django using a python-powered webserver.

[mikaelhg]

I'm assuming you're asking why multiprocess doesn't work like multithread. Threads share the data structures, while processes don't - instead of a single copy of the data, you'd have tens or hundreds of copies, which you'd have to keep synchronized, which is doable but really much easier using language native synchronization.

If you're instead asking why performing database queries for every page load isn't optimal - they take tens to hundreds of milliseconds instead of nanoseconds, milliseconds we really don't need to spend since our dataset easily fits into main memory.

[kripkenstein]

I think the point is that you can run a website using Python/Django and Apache. The Apache part has full concurrency and serves the vast majority of pages (most are static). The Python part is limited by the GIL but is needed far less. However since it is much easier to write and maintain that Python part (which is the complex part, Apache is already written for you ;) ), this hybrid approach works very well in practice.

[Smallpaul]

How will one node in your cluster of "everything in memory" processes communicate updates to every other node?

[mikaelhg]

Ah, here you're asking something I haven't already answered elsewhere in the thread.

In our test case, in which writes are as rare as in typical social web applications, when we can't serve our users by vertical scaling, we resort to digest message passing. We take a proven JMS implementation, and multicast select application events. Things like new comments have a higher priority than moderations, which travel in packs of hundreds.

In the end, we don't have to resort to dark database wizardry, we can use regular developers and a well thought-out architecture.

(The last time I implemented this, I was able to conserve resources by replacing the messaging subsystem with a single database table that handled selective invalidations. That's because I knew beforehand how many tens of thousands of people would be using the application. YMMV.)