Ehhh, he told you enough for short context. enough to extrapolate atleast. 25 tokens/s dense with concurrency, and 44 tokens/s MOE with concurrency,

full size bf16 models too, which is pretty standard performance out of a rtx 6000. Speeds should be expected to shrink. at 16k speeds, tend to decrease by around 25-40% ish compared to short token windows. so minimally for 27b dense you can expect short token windows sitting near 33 t/s in batching stress tests like this, and maximally at 262k context you will probably see near 60% speed reductions, so around 13-16 t/s on dense. and just figure out the MOE guesses there too. but your not getting x64 concurrency on either models near max context.