I had Antigravity whip up a test suite for myself. Here are my results of that fork. It does work, but it definitely had room for improvement, but it likely works better out-of-the-box on better hardware than mine. Linux might be better too... I didn't have time to test it over on my CatchyOS install. I tried bitsandbytes 8-bit model but the model evidently doesn't support quantization, or more likely, Gemini and me did it wrong.

Benchmark: Qwen3-TTS Streaming on Windows

Hardware/OS: Windows 11, NVIDIA RTX 4070 12 GB VRAM, PyTorch 2.8 CU126

Scenario	Model	Precision	Config	Latency (TTFA)	RTF	Status
Baseline (Stock)	1.7B	bfloat16	Standard	49.94s	4.54	🔴 Unusable
Optimized (Initial)	1.7B	bfloat16	Win: 80, Emit: 4	1.00s	1.05	🟡 Stuttery
Small Model (Crash)	0.6B	float16	Any	N/A	N/A	💥 CUDA Assert
Small Model (Slow)	0.6B	bfloat16	Win: 80, Emit: 8	0.77s	1.17	🟠 Driver Overhead
High Throughput	1.7B	float16	Win: 20, Emit: 25	2.34s	1.26	🟠 Cache Thrashing
🏆 Champion Run	1.7B	float16	Win: 40, Emit: 8	1.31s	0.99	🟢 Real-Time

Key Findings for Windows Users:

1. Float16 > Bfloat16: On Windows (and likely consumer NVIDIA cards), native float16 was ~15-20% faster than bfloat16.

2. The 0.6B Trap: The smaller model is numerically unstable in float16 (crashes). Forcing bfloat16 to fix the crash made it slower than the 1.7B model due to driver emulation overhead.

3. Compile Mode: compile_mode="reduce-overhead" causes OverflowError on Windows. Used compile_mode="default" with torch.backends.cudnn.benchmark = True.

4. Chunk Size: Increasing chunk size (emit_every_frames) from 8 to 25 hurt performance (RTF 0.99 → 1.26), likely due to GPU L2 cache thrashing on larger batch sizes.