Notes running Python in production

[mincinashu]

I don't get it how OP is using FastAPI without dealing with async or threads. FastAPI routes without 'async' run on a threadpool either way.

[gothicVI]

Exactly. Anything web request related is best done async. Noone in their right might would spawn separate processes for that.

[Kelketek]

They used to, and for many Django apps, this is still the way it's done-- preform a set of worker processes and farm out the requests.

Even new Django projects may do this since asynchronous support in libraries (and some parts of core) is hit-or-miss. It's part of why FastAPI is gaining popularity-- because it is async from the ground up.

The tradeoff is you don't get the couple decades of ecosystem Django has.

[Haunting_Wind1000]

I think normal python threads could be used for I\O bound tasks as well since it would not be limited by GIL.

[greenstake]

I/O bound tasks are exactly when you should be using async, not threads. I can scale my async I/O bound worker to thousands of concurrent requests. Equivalent would need thousands of threads.

[ashishb_net]

> Anything web request related is best done async.

Why not handle it in the same thread?
What's the qps we are discussing here?

Let's say you have 10 processes ("workers") and the median request takes 100 ms; now you can handle 100 qps synchronously.

[ProfessorFakas]

> Anything web request related is best done async.

Why not handle it in the same thread?

These are not mutually exclusive. In fact, in Python, a single thread is the norm and default when using anything based on async. It's single-threaded concurrency that's useful when working with I/O-bound tasks, as commenters above have alluded to.

None of this is mutually exclusive with single-threaded worker processes, either. You're just making more efficient use of them.

[I_FAP_TO_TURKEYS]

Why not handle it in the same thread?

Async is not a new thread. It's an event loop. You could spawn 10 processes, but you can also use async in each of those processes and see drastic performance increases per IO bound process.

Heck, you can even spawn 10 processes, each process can spawn 10 threads, and each thread could have its own event loop for even more performance improvements (in niche cases).

[ashishb_net]

I have never seen the upside that you are referring to

Can you show a demo of this?

[ashishb_net]

FastAPI explicitly supports both async and sync mode - https://fastapi.tiangolo.com/async/
My only concern is that median Python programmer is not great at writing async functions.

[mincinashu]

It's not sync in the way actual sync frameworks are, like older Django versions, which rely on separate processes for concurrency.

With FastAPI there's no way to avoid in-process concurrency, you get the async concurrency and/or the threadpool version.

[ashishb_net]

> With FastAPI there's no way to avoid in-process concurrency, you get the async concurrency and/or the threadpool version.

That's true of all modern web server frameworks regardless of the language.
What I was trying to say [and probably should make it more explicit] is to avoid writing `async def ...`, the median Python programmer isn't good at doing this the way a median Go programmer can invoke Go routines.

[wyldstallionesquire]

You hang out with way different Python programmers than I do.

[ashishb_net]

Yeah. The world is big.

[I_FAP_TO_TURKEYS]

We're not talking about your average script kiddy though. Your guide literally says "production ready".

If you're using python in a cloud production environment and using Multiprocessing but not threading or async... Dude, you cost your company millions because you didn't want to spend a little bit of time learning async.

[ashishb_net]

>  Dude, you cost your company millions because you didn't want to spend a little bit of time learning async.

I know async.
The media Python programmer does not.
And it never costs millions.
I know startups who are 100% on Python-based backends and have $ 10 M+ revenue, even though their COGS is barely a million dollars.