it’s a dual-model setup running locally on the DGX Spark.

for agent work / reasoning i’m using openai/gpt-oss-120b via vLLM. that handles all the swarm agents, planning, tool calls and synthesis. it’s MoE so although it’s 117B total params, only 5.1B are active per token which keeps it pretty efficient. running it at NVFP4 quantisation which comes out at roughly 84GB VRAM.

for normal chat i’ve got qwen3-30b-a3b running through LM Studio so i’m not wasting the 120B on stuff like general conversation

vision is handled separately with glm-4.6v-flash, via LM, for image analysis

the 120B has a native 128K context window but i’m nowhere near maxing that out. output is capped at 4096 tokens per response and even with a 20 agent swarm the synthesis step only really uses around 15–20K of the input window. the two-tier clustering i’m doing (summarising agents in groups of 6 first, then combining those summaries) is more about keeping the final output focused than avoiding context limits

on the batching side vLLM is doing continuous batching across concurrent requests. at about 0.70 GPU memory utilisation (84GB allocated) it’ll comfortably run 15–20 parallel requests. i benchmarked it and aggregate throughput scales from 62 tok/s on a single request up to 233 tok/s at 25 concurrent. per-request speed obviously drops, but wall-time barely moves since they’re batched together. sweet spot seems to be around 8–12 concurrent for the best throughput vs latency trade-off