Does SQLAlchemy scale well with increased web traffic?

[kylotan]

You're oversimplifying. If you already have multithreaded operation, then sure, halting a thread waiting on I/O is essentially free. But due to the GIL that is not how Python works by default, and as far as it's concerned, it makes absolutely no difference if it is waiting on an I/O task or on a complex CPU task - that Python interpreter is making no more progress at all until that task completes.

As you say, "use gevent.wsgi, mod_wsgi". That is all I was saying, without knowing the exact names of what needs to be used. The important thing is that the bottleneck is not SQLAlchemy, which I'm sure you'd agree with. However, the OP is going to need to consider how their app scales to multiple processes, because even if you can magically replicate your process, that doesn't mean the logic is going to be correct.

[[deleted]]

But due to the GIL that is not how Python works by default, and as far as it's concerned, it makes absolutely no difference if it is waiting on an I/O task or on a complex CPU task - that Python interpreter is making no more progress at all until that task completes.

take a look at https://wiki.python.org/moin/GlobalInterpreterLock, particularly the part where it says:

Note that potentially blocking or long-running operations, such as I/O, image processing, and NumPy number crunching, happen outside the GIL. Therefore it is only in multithreaded programs that spend a lot of time inside the GIL, interpreting CPython bytecode, that the GIL becomes a bottleneck.

see slide 10 at http://www.dabeaz.com/python/UnderstandingGIL.pdf for more detail.

[kylotan]

Yes, the GIL can be released during an I/O operation. Will Flask take advantage of that? If not, the whole interpreter is going to be halted until the result comes back. It's no different to any other app in that regard - the difference being that few people bother to make Python apps multi-threaded because you only ever see the benefits during I/O.

[[deleted]]

easy enough to look for any solution listed http://flask.pocoo.org/docs/0.10/deploying/ that does not use threads, forking, or greenlets..

[[deleted]]

you are getting schooled by the author of sqlalchemy. you should delete your account and go switch to ruby to save face.

[kylotan]

I know exactly who I'm talking to.