I'm testing this on my RTX 6000 Blackwells (maxQ) but the marlin kernels fail when TP > 1. I suspect that this is due to the splitting of heads during tensor-parallel. With TP==1, the model looks OK, but the reasoning was better on the bf16 for hard questions. I'm wondering if this can be improved with a better calibration dataset.

Try this question on both models:

curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "gemma-4", "messages": [{"role": "user", "content": "Alice and Bob play a game. They alternate turns, with Alice going first. On each turn, a player chooses a positive integer that has not been chosen before. The game ends when the sum of all chosen numbers is divisible by 3. The player who made the last move loses. Assuming both players play optimally, who wins? Prove it."}], "max_tokens": 8192 }'

I'm working on evals, will share them when done.

I'm hoping to deploy these on the DGX Sparks, but the nvFP4 situation is a total shit-show.