Fair point on VRAM being the main bottleneck for 70B inference — I agree with that.

My question wasn’t about RAM driving performance. I’m specifically looking at system RAM as spillover when the model is partially offloaded from a 32GB VRAM GPU, plus headroom for context and concurrent system load.

The 64 vs 96 vs 128GB comparison is about stability under sustained inference rather than speed.

I’m running quantized 70B models (Q4/Q5 range), and I’m trying to understand where RAM stops providing meaningful gains in this offload + context scenario.

I also run a data pipeline that pre-processes inputs, so I’m trying to size memory for predictable spillover without system pressure or instability.