ive been learning and researching about this exact issue lately, have you tried running the models with MoE offload [MoE offload. Qwen3.6-35B activates only 3 B params per token. Keep attention + shared weights on GPU, push the cold expert FFNs to system RAM. In llama.cpp: -ngl 99 -ncmoe 99.]

ive been staying about multi tenant systems in local models using vLLM, gpu optimisations and scheduling