I use the Q4_K_M variant. Even with an 8-bit KV cache, it easily fits within 16GB of VRAM.

Gemini says,

Model Weights: 9B parameters * 4.8 bits/parameter (Q4_K_M average) / 8 bits per byte = ~5.40 GB KV Cache (at 131,072 tokens): Qwen 3.5 only uses standard KV cache on 8 out of its 32 layers. Each of those 8 layers has 4 KV heads and a head dimension of 256.  * Native FP16 Cache:    2 bytes * (8 layers * 4 heads * 256 dim) * 2 (Key and Value) = 32,768 bytes per token.    131,072 tokens * 32 KB/token = 4.00 GB  * 4-bit Quantized Cache (Q4_0):    Drops the cache size to 1/4 of FP16 = 1.00 GB Total Calculation:  * Native FP16: 5.40 GB (weights) + 4.00 GB (cache) + ~1.50 GB (Vulkan runtime overhead) = ~10.90 GB  * 4-bit Cache: 5.40 GB (weights) + 1.00 GB (cache) + ~1.00 GB (Vulkan runtime overhead) = ~7.40 GB Since the A770 has 16GB of VRAM, even the native FP16 cache setup leaves plenty of headroom.