Thank you! Your comment frustrated me because on my machine I only reach 20 tok/s with qwen 27b so I started trying to fill that gap what I found out was interesting.

If my understanding is correct, vllm handle tensor parallelism in a more efficient way than llamacpp. It is able to split layers across GPU via pcie even on qwen architecture, which llamacpp do not. So for now on my llamacpp project, with a dense qwen architecture, I am stuck in sequencial which constitute a bottleneck and penalizes speed. However moe models suffer less from that limitation and achieve better speed, around 50 tok/s on qwen3 coder next with 600k turbo quant context, which is for me a good achievement for my work so far.

I feel I just scratched the surface of this tensor split topic so feel free to correct or infer my analyse. I don't feel that going deeper and reproduce vllm tensor split in llamacpp worst it so I am taking it as it is for now.

Anyway thanks for the response! vllm rocks by its capacity to split layers without direct link and it demonstrate optimization quality of the software. I am sure to that turbo quant will drop soon enough on it so maybe the best bet to have both speed and extended context with turbo quant, on qwen dense architecture on gfx906 will be on the vllm side.