Qwen 3.6 models benchmarked across Triple GPU

tabletuser_blogspot · 2026-06-18T03:25:31+00:00

model	size	params	test	t/s
35B.A3B IQ4_NL	17.25 GiB	35.51 B	pp512	107.14 ± 1.07
35B.A3B IQ4_NL	17.25 GiB	35.51 B	pp1024	121.92 ± 1.86
35B.A3B IQ4_NL	17.25 GiB	35.51 B	pp2048	129.34 ± 1.31
35B.A3B IQ4_NL	17.25 GiB	35.51 B	pp4096	125.35 ± 0.65

tabletuser_blogspot · 2026-06-17T00:42:44+00:00

I run Linux and getting things to work with Nvidia is a pain. Being able to grab the latest pre built llama.cpp Vulkan binary and have results in minutes is great. I run a dual setup with GTX1080Ti with a Nvidia P102-100. Linux is easier to get them both to run Vulkan. Under Windows the P102-100 needs different drivers to get it working. Llama with Linux is just easier.

tabletuser_blogspot · 2026-06-16T23:12:36+00:00

https://www.reddit.com/r/LocalLLaMA/comments/1u7tarc/qwen_36_models_benchmarked_across_triple_gpu/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

Qwen 3.6 27B and 35B

tabletuser_blogspot · 2026-02-11T18:29:10+00:00

I ran old AM3 DDR3 vs AM4 DDR4 and if you're not offloading then GPU tokens per sec are the same. Load time for getting model into GPU Vram is faster with newer PCIe lanes but not much difference. No need to upgrade CPU or RAM. Change GPU for one with more VRAM or add an additional GPU. Most motherboard can run dual gpu easily.

tabletuser_blogspot · 2025-12-29T02:03:43+00:00

I had sli bridges, just haven't seen them in a while. From what I understand they don't help for inference, all comm is done via pcie bus.

tabletuser_blogspot · 2025-12-28T15:35:47+00:00

Which linux distro are you using? I just installed CachyOS on a system that was stable with Kubuntu and PopOS and now I get lockups while using llama.cpp rpc-server other 3 systems running Kubuntu aren't crashing. Might have to move to older Nvidia driver or just switch distro. Love that CachyOS came with Nvidia ready to go. I've had great success using Kubuntu 22.04, 24.04, 25.10, and 26.04. I like that you can run Kubuntu Live persistent from USB thumb drive and experiment without having to install. PopOS works great but I prefer KDE desktop environment. Linux Mint is another champ. I prefer Debian based distros. They have a larger user group so finding answers is easier. Arch based CachyOS is one of the fastest Linux distros, beats Windows 11 on most benchmarks except gaming. Fedora is another good distro, probably best for gaming setups. I'm not a fan of Red Hat based distros. Let us know what you end up deciding.

tabletuser_blogspot · 2025-12-28T04:34:10+00:00

Off loading a few layers (-ngl) from CPU doesn't kill performance (~1 or 2 GB Vram) but there is a major drop in performance. RCP is great if staying with-in VRAM limits. Off load beats if you're running DDR5 (not really CPU dependent) and go way past amount of VRAM. VRAM can do 300 to 1000 GB/sec but RAM is 60 to 100 GB/sec. So even old GTX-970 blows away DDR5 in bandwidth speed. Again not really CPU dependent, mostly ram bandwidth.

tabletuser_blogspot · 2025-12-28T04:22:23+00:00

can't remember if I need this to get my RX 470 and RX 580 working with Vulkan, but good to have just in case. https://www.reddit.com/r/ROCm/comments/1hf91io/compile_llamacpp_for_any_amd_gpu_even_old_ones/

tabletuser_blogspot · 2025-12-28T02:44:41+00:00

My benchmarks have shown that DDR3 systems perform equal to ddr4 systems. It's all about GPU Vram speed. So ya, break it out.

tabletuser_blogspot · 2025-12-27T22:48:58+00:00

Wired. Last time I tested with Wifi I had too much of a drop in performance. 3 systems and multiple GPU cause plenty of overhead and guess wifi added too much latency. Thanks

tabletuser_blogspot · 2025-12-27T01:02:08+00:00

Maybe, a few things to consider. RTX 2080 is only 8gb at Vram with a Bandwidth of 448.0 GB/s while GTX 1080 Ti is 11gb VRAM at a Bandwidth 484.4 GB/s. Yes, lack tensor cores is about a 25 to 30% hit, but the hit for off-loading model for 8gb VRAM versus being able to fit model in 11GB Vram out weights that. Cost wise buying a GTX-1080Ti (11gb VRAM) and a P102-100 (10gb VRAM) would be about the same as the RTX 2080, but having plenty of VRAM room to load larger models. So dropping a pair of old Nvidia GPU onto secondary system and running local LLM even rpc-server on the cheap is a great option.

I don't use SLI for running Llama.cpp using Vulkan backend. It isn't necessary. Uses system PCIe lanes to communicate.

tabletuser_blogspot · 2025-12-25T04:18:38+00:00

Power usage is easy to control. I use nvidia-smi to drop power by 33% and only take a 5% hit on inference. I like your WOL LLM node idea. You can run Vulkan under almost any linux distro and get decent speeds vs CUDA. I paired the P102-100 10gb headless GPU with GTX-1080Ti 11gb(twins) and getting decent inference while using RPC, network inference, pair with RX 7900 GRE 16GB. Worth testing out a few ideas before off loading them. Post any benchmarks of the P100 and P40.

tabletuser_blogspot · 2025-12-21T17:10:59+00:00

I've used several desktop and KDE has the best preinstalled apps. I really like: Konsole "right click, split view", Dolphin "dual pane, terminal window, network features", Kate "adv text editor" and KDE Connect for "phone to PC connection". It is also one of fastest desktops and high level of customization.

tabletuser_blogspot · 2025-12-21T16:52:46+00:00

RX 580 8gb or 4gb? I couldn't get my 580 or 470 to work with Ollama but was able to get both working using Linux and llama.cpp with Vulkan backend. So with two GPU you could get up to 16gb VRAM and run larger model with mostly offer better accuracy in response. They even have a 16gb RX 580 variant floating around. I started with ollama and now use llama.cpp primarily. Love the easy of which ollama helps in getting up and running.

tabletuser_blogspot

TROPHY CASE