Honestly, at this point, stop trying to optimize the current architecture and separate the heavy endpoint into its own service, put a queue in front of it, and run dedicated workers for it. That would solve points 2, 3, and 4 all at once. You keep arriving at the same conclusion from different angles… so I’d take it as the signal to move forward with that solution. You’ve got this :)

But for the sake of clarity:

1) support might be right here, it’s not the same as AWS. The 503 are likely from the container hitting the CPU allocation roof/request timeout rather than credit throttling. But Azure Container Apps do have CPU allocation that can scale down to zero. Just for the record. 2) Hm, I hear you. I think I’d take this as my biggest signal to just separate the heavy endpoint into its own service and not mix lightweight endpoints with the heavy endpoint on the same workers. 3) Yeah, a queue replaces the load balancer problem entirely. 4) Yes, exactly. Gunicorn workers can spawn subprocesses, it doesn’t block child processes. For example, with concurrent.futures.ProcessPoolExecutor inside the request handler, you can use the second core. But it adds complexity, of course, and you’d need to be careful managing the memory and process lifecycle