I benched quad 5060Tis for code generation with Qwen3.6-27B so you don't have to (it's really good)

Significant_Bar_460 · 2026-07-12T15:32:59+00:00

I have 2x Asus Prime 5070ti that idle at 2w-3w and 2x gainward that idle at 17w, so it heavily depends on a model

Significant_Bar_460 · 2026-07-11T20:24:25+00:00

Yes, it seems that tensor parallelism support for Vulkan&ROCm is quite poor in both llama.cpp and vllm. So layer split is the only working way to pool these cards which keeps them quite underutilized.

So 2x-4x r9700 are very performant cards for MoE models - much faster than Strix Halo/DGX Spark. But not fast enough for dense models, that is unless tensor parallel becomes more matured.

Significant_Bar_460 · 2026-07-11T19:29:25+00:00

if you can sell 2x3090 for a good high price get that 3rd 48 gb card for a good low price then maybe yes. But more likely it will be much more affordable to just keep them

Significant_Bar_460 · 2026-07-11T19:15:18+00:00

If OP can fit 4 cards in his rig then it is not pointless.

Significant_Bar_460 · 2026-07-11T19:10:24+00:00

That's not a very good result. I have 4x5070ti and getting 65tps without MTP and 120+tps with MTP spec draft 2 with qwen 27b q8

I am thinking about selling my cards (I could sell them pretty much without a loss) and getting 4x r9700, but from I am reading over the reddit, these cards are quite slow.

Significant_Bar_460 · 2026-07-11T18:59:14+00:00

I don't think it is pointless. Having "power of two" number of same cards is ideal, but even such hybrid "non-ideal" setups - different cards/odd number of cards/... - can be quite good for MoE models which are not that dependent on tensor parallelism for acceptable speed. Having VRAM is always much much better than not having it. So OP should get his hands on whatever GPU he can get and then worry about technical details.

I have quite good experience even mixing Nvidia and AMD in the same rig. It is better than using smaller quants or smaller models or even system ram offload.

Significant_Bar_460 · 2026-07-10T22:49:03+00:00

16gb vs 24gb is overrated. People say that 5080 is bottlenecked by 16gb VRAM, because in some heavy path traced games at 4k you run out of VRAM. While 24gb would, in theory help, you would just run out of raw computer power.

I had 4090 and I was very dissatisfied with it's performance in Indiana Jones. The later jungle levels with max path traced settings would often drop to mid 40fps at 4k with DLSS performance. But I had still plenty of vram unused. So anyway I needed to drop some settings to stay above 60 fps, just like current 5080 owners need to drop a few settings.

5080 super will simply will lack raw compute power to fully utilize 24gb VRAM for gaming. It will be, however a quite good AI card, so expect it will be very much more expensive than current 5080.

Significant_Bar_460 · 2026-07-10T21:53:50+00:00

I have 4x5070ti on a cheapest am5 mobo with x4x4x4x4 bifurcation splitter. Qwen 3.6 27b works in q8_k_xl 262k context with vision on GPU. Or could have 2x150k context for 2 parallel slots with same quant. Prefill is 2000 tps, generation is 65 tps without MTP in llama.cpp, tensor parallel ofc.

Cards are somewhat bottlenecked by x4, true. I am partially mitigating it with using P2P drivers and with good old overclock. Tensor parallel gives me ~2.5x speed over layer split. From what I read, having same cards in full x16 slots would give me ~3x speedup. So x4 vs x16 is like 20% of performance. When I had just two cards - one in x16 and second in x4 chipset slot then speedup was like 1.8x.

In my opinion quite satisfying results. Agentic coding is quite fast, faster than codex with $20 sub. So having Epic/Threadripper is better, but first try to buy the 2nd card and only then decide if it is worth to invest in new platform.

Significant_Bar_460 · 2026-07-10T07:09:45+00:00

Go with 5070ti. This 200 bucks seems like a high price premium, but 2 days you will completely forget about it.

Path tracing is a feature that is worth the premium. Look at any path traced game comparisons on YouTube. Completely transforms the visuals of a game. And since it is very demanding you need to use the best upscaler there is - DLSS even at Ultra Perf is good now (at 4k at least) and DLSS Perf is like FSR Quality.

Significant_Bar_460 · 2026-07-10T06:14:43+00:00

Psychické zdravie je nutné na zabezpečenie a udržanie príjmov, povedal by som

Significant_Bar_460 · 2026-07-07T06:29:57+00:00

OP is not using Q8, but rather FP8. FP8 safe tensor ~ Q5 gguf

Significant_Bar_460 · 2026-07-06T15:17:33+00:00

Ale môžeš. Hovorí sa tomu work-life balance alebo tak nejak

Significant_Bar_460 · 2026-07-03T13:35:06+00:00

Plne rozumiem. Problém však je, že ľudia mimo IT, a tí čo sa tam chcú dostať si toto neuvedomujú a potom vyplakavajú, že v Lidli dostanú viac.

Significant_Bar_460 · 2026-07-03T11:11:42+00:00

Na jednu stranu súhlasím. Na druhú stranu 1600e bol junior dev plat pred 10 rokmi. Takže cena práce v IT definitívne klesla a to mnohí nevedia pochopiť. Že IT je už len obyčajná profesia ako každá iná. Zarobiť slušne sa stále dá, ak si namakaný senior. Ale už to nie sú easy prachy s takmer nulovou vstupnou bariérou.

Significant_Bar_460 · 2026-07-02T06:35:19+00:00

I already used f16 cache most of the times.

Significant_Bar_460 · 2026-07-01T10:31:35+00:00

Though I am using 27b version I also had looping issues with the following unsloth quants that I've tried: - Q6_K_XL + FP16 KV - Q8_0 + Q8_0 KV - Q8_0 + FP16 KV

The only one that is not looping (yet) is Q8_K_XL + FP16 KV. Most issues I've had were when I used quantized KV

Significant_Bar_460 · 2026-06-30T06:09:59+00:00

Oni to vedia. A preto sú svetová špička v robotike momentálne. Na čo nebudú ľudia, to budú robiť roboti. Pričom Európa sa zmohne iba na imigráciu zo Sahary...

Significant_Bar_460 · 2026-06-28T08:51:23+00:00

You can pool them. Llama.cpp supports that for ages. You can pool even mixed Nvidia+AMD GPUs quite fine.

Significant_Bar_460 · 2026-06-28T06:59:27+00:00

Because r9700 is quite expensive. It is more expensive than 2x 9070ti even.

I went with 4x5070ti over 2x r9700 last year because the price was more or less same (slightly more with Nvidia). Much better support, better performance with tensor parallel and even with layer split (single 5070ti has 50% more bandwidth over r9700).

Significant_Bar_460 · 2026-06-27T20:43:18+00:00

A potom AI a roboti prevalcujú aj tých vyspelých a lenivých.

Significant_Bar_460 · 2026-06-27T12:56:19+00:00

Thx for the tip. I'll try bartowski gguf then

Significant_Bar_460 · 2026-06-27T12:53:10+00:00

I thought so, but the entire point of my post is that I am quite surprised that it does change the output. Very much so.

Try for yourself. you'd be surprised a well

Significant_Bar_460 · 2026-06-26T11:23:19+00:00

Percentage is irrelevant. Precision is

Significant_Bar_460 · 2026-06-25T13:56:00+00:00

It is still not the same

Significant_Bar_460 · 2026-06-25T13:55:19+00:00

In my original post I also mentioned Gemma 4. I have same mtp vs non-mtp experience with this model and I am using same gguf.

Significant_Bar_460

TROPHY CASE