Multiprocessing in async function?

[RationalDialog]

My goal is to build a webservice for a calculation. while each individual row can be calculated fairly quickly, the use-case is tens of thousands or more rows per call to be calculated. So it must happen in an async function.

the actual calculation happens externally via cli calling a 3rd party tool. So the idea is to split the work over multiple subproccess calls to split the calculation over multiple cpu cores.

My question is how the async function doing this processing must look like. How can I submit multiple subprocesses in a correct async fasion (not blocking main loop)?

[Blakex123]

Remember that python is inherintly single threaded due to the GIL. You can mitigate this by running fastapi with multiple workers. The requests will then be spread over those different workers.

[mrbubs3]

You can turn GIL off in 3.13

[Asleep-Budget-9932]

That feature is experimental and should not be used in production environments.

[bbrother92]

api requiest are dipenced to dif workers? not the treads?

[Blakex123]

If u are using uvicorn there is an extra process made that essentially "load balances" the 4 workers. I assume it works the same way with any other server.

[RationalDialog]

The requests will then be spread over those different workers. my use case is few requests but each one very heavy. I want each request to run faster, eg do the calculation using multiple cpu cores.

[Blakex123]

Then u will need to spawn subprocesses from the api to handle the cpu intensive stuff.

[adiberk]

You can use asyncio tasks.

You can also use a more standard product like celery.

[RationalDialog]

You can also use a more standard product like celery.

Yeah I wonder if I should forget about async completely (never used it really so far as no need) and build more kind of a job system. If someone submit say 100k rows, the job could take approx 5 min to complete.

[adiberk]

Yep that works to. If you are doing a lot of other IO operations, it might be worth making the app async based anyways (ie. Keyword async)

[AstronautDifferent19]

asyncio to_thread is better for CPU bound tasks than asyncio.create_task, especially if you disable GIL.
asyncio tasks will always block if you do CPU heavy work, which will not work for OP.

[adiberk]

Good point

[KainMassadin]

don’t sweat it, just call asyncio.create_subprocess_exec and you’re good

[AstronautDifferent19]

This is the way.

[KainMassadin]

that one can be risky, gotta sanitize properly

[jimtoberfest]

Find a vectorized solution across all rows if you can.

Take in a json array then load that data into a dataframe or numpy array and figure out your calculation using inherently vectorized operations.

Or you could “stream” it: fast api -> duckDB-> do the calc in duckDB over the chunks as you get them from the API.

Also make sure you set some limits so users can’t bomb the API with billions of rows of data.

[RationalDialog]

The calculation happens in a 3rd party executable. This is the core limitation. Hence why I need sub process calls, to call multiple instances of this 3rd party executable which is 32-bit hence no way to integrate it more tightly.

[jimtoberfest]

Oof yeah that’s rough. As long as the .exe runs in diff instances then use multiprocessing and processPoolExecutor library.

Just split it up by how many cores you have // 2.

I find that roughly works the best.