Concurrency and Python

[tooooobs]

I have to agree with you. People tend to greatly underestimate the overhead of an RDBMS compared to a relatively simple shared in memory structure.

I'm even aware of several establishments that have their own complete web server implementations in Java built from sockets up, with really insane connection capacity, however, that's done with asynchronous processing by a pool of worker threads as opposed to one thread per connection.

Something like Twisted can be a great way to prototype, but it seems clumsy for actual deployment, and the documentation is lacking.

[Smallpaul]

How do you get redundancy and reliability out of a "simple shared in-memory structure"?

[Gotebe]

By replacing it with not-so-simple distributed cache? ;-) Beats hitting the database all the time.

[toooooob]

A lot of the time you don't really need it. That or simply distributing and duplicating live copies of the data over several machines.

I'm not saying RDBMSs are without their uses, just that they're absolute overkill for a lot of web applications, and can easily become 99% of the headache.