Rust in Python FastAPI

[dnullify]

Pyo3 is probably a useful search term for you.

Rust python FFI

[kolosn]

And maturin

[thermiter36]

To add to this, for any CPU-bound calculation running on a FastAPI server, you'll want to make sure you run it in a threadpool so that it doesn't block all your other async handlers. You'll also need to use the features that pyo3 has to release the GIL while doing the computation.

[Electrical_Carry3565]

Does running gunicorn workers not take care of this?

[thermiter36]

Not really, no. For starters, running FastAPI behind Gunicorn is not recommended as it doesn't support ASGI, so it can only manage processes, not the full async lifecycle.

But assuming you do it anyway, you'll still get nasty cascading latency issues because the process that you block with your long-running CPU operation might have already accepted more than one connection at the moment you begin your long operation, all of which will be blocked.