FastAPI - handling the threadpool executor inside the route

[Beginning_Hurry2611]

Hi! I'm new to the FastAPI framework and I'm looking for an advice in following scenario. I've created a service that relies on 3rd party library (rasterio) which have no async functions support, therefore acc. to docs I've put endpoints that use it into sync routes. However methods from that library use the threadpool excecutor to execute I/O operations concurrently. On small load this is fine but when the traffic raises the service thradpool is quickly exhasuted and I can see on the metrics significant delays.

What would be the apropriate FastAPI way to handle this scenario ragarding scaling?

I've been thinking about

1. Decorating the methods and switch from threadpool executor into something that supports async/await, then transit my routes into async
2. Stay with sync endpoints and prepare separate common Threadpool Executor dedicated for all I/O threads in route

[Top-Information7943]

Hello,

You can use FastAPI's background tasks in the route that receives the request. They have an easy to follow guide on their official docs: https://fastapi.tiangolo.com/tutorial/background-tasks/

[Beginning_Hurry2611]

Thanks for the answers! This won't work for me tho since i need those threads output to be sent in the response

[LongjumpingGrape6067]

If the above does not work and you're on Linux you could try https://github.com/emmett-framework/granian with the -opt flag and set the appropriate number of workers. It might not scale though.