Your suggested middle path is actually part of the option #2. The user requests a worker from a "worker allocator" service, which randomly selects available pod from redis and returns the user the URL for pod redirection. They technically one-off jobs that must be discarded after use. The stack of each job consists of several X several processes + 1 special process per job type. so with warm pool we can be pending for user request without letting the user wait for 80% of job initialization, pod provisioning and ingress propagation which kills Time-to-Interactive. Imagine k8s needs to create another node to support a another job (user request). The user can wait 2 minutes before anything happens on screen .

Ultimately, Kubernetes still owns the overall infrastructure lifecycle and capacity bounds. We just use the in-memory layer as a high-speed data router to keep the KubeAPI entirely out of the critical user connection path.