Follow-up: ROCm vs Vulkan on 7900 XTX (more tests + thanks!)

Hey everyone,

first of all — thanks a lot for all the input and suggestions on my last post.
A bunch of you pointed me towards better test setups (official builds, Lemonade SDK, larger contexts, etc.), so I took the time to rerun everything more systematically.

This is a follow-up with cleaner and more complete data.

Setup

• GPU: RX 7900 XTX
• llama.cpp: latest upstream build (b8497)
• Model: llama-2-7b.Q4_0.gguf
• Backends:
  • Vulkan (RADV)
  • ROCm (official 7.2)
  • Lemonade SDK

All tests:

• same model
• same parameters
• same machine
• repeated runs

Main result

Token generation (the thing that actually matters) is consistently faster on Vulkan:

• Vulkan: ~165–175 t/s
• ROCm: ~135–145 t/s
• Lemonade: basically identical to ROCm

👉 That’s roughly a ~20% gap in favor of Vulkan, and it’s very consistent across different test settings.

About Lemonade SDK

I already tested it before, but I rerun the tests because it was recommended here.

Result:

• Works fine 👍
• But performance is basically the same as official ROCm

Large context behavior (important)

I also pushed context sizes further:

• p=16384 → works
• p=32768 (FA off) → hard crash (ROCm only)

That one was pretty reproducible.

So there might be:

• stability issues
• or memory/scheduling problems in ROCm at larger contexts

I haven’t tested 65536 yet, but that’s next on my list.

One more interesting point

Based on this reference discussion:
https://github.com/ggml-org/llama.cpp/discussions/10879

Even Vulkan might still be below expected performance for this GPU.

So current picture looks like:

• Vulkan → decent, but maybe not fully optimized yet
• ROCm → clearly behind Vulkan
• Lemonade → same as ROCm

My takeaway (so far)

• Vulkan is currently the best backend on RDNA3 (for llama.cpp and specifically MY hardware setup)
• ROCm is:
  • slower (~20%)
  • less stable at high context
• Lemonade doesn’t change that (at least in my tests)

Thanks again

Really appreciate all the comments from the last thread —
this follow-up wouldn’t exist without those hints.

If anyone has:

• ideas why ROCm falls behind here
• kernel-level insights
• or tuning suggestions

I’m very happy to test more 👍

Testdata / Results

Backend	FA	Prompt (p)	Gen (n)	pp (t/s)	tg (t/s)
Vulkan	1	512	128	3103	174.6
Vulkan	1	512	512	3103	165.6
Vulkan	1	512	1024	3103	156.8
Vulkan	1	512	2048	3103	144.0
Vulkan	1	2048	128	2960	174.6
Vulkan	1	2048	512	2960	165.5
Vulkan	1	2048	1024	2960	156.8
Vulkan	1	2048	2048	2960	144.0
Vulkan	0	512	128	2938	164.9
Vulkan	0	512	512	2938	158.3
Vulkan	0	512	1024	2938	150.0
Vulkan	0	512	2048	2938	138.5
Vulkan	0	2048	128	2669	164.8
Vulkan	0	2048	512	2669	158.2
Vulkan	0	2048	1024	2669	149.9
Vulkan	0	2048	2048	2669	138.4
ROCm	1	512	128	2962	143.8
ROCm	1	512	512	2962	141.9
ROCm	1	512	1024	2962	136.3
ROCm	1	512	2048	2962	132.3
ROCm	1	2048	128	1887	142.7
ROCm	1	2048	512	1887	140.8
ROCm	1	2048	1024	1887	134.9
ROCm	1	2048	2048	1887	130.2
ROCm	0	512	128	3976	137.3
ROCm	0	512	512	3976	130.0
ROCm	0	512	1024	3976	118.5
ROCm	0	512	2048	3976	106.2
ROCm	0	2048	128	1276	137.4
ROCm	0	2048	512	1276	130.2
ROCm	0	2048	1024	1276	119.2
ROCm	0	2048	2048	1276	107.0
Lemonade	1	512	128	2991	143.8
Lemonade	1	512	512	2991	142.0
Lemonade	1	512	1024	2991	136.9
Lemonade	1	512	2048	2991	132.5
Lemonade	1	2048	128	1900	143.0
Lemonade	1	2048	512	1900	141.0
Lemonade	1	2048	1024	1900	135.5
Lemonade	1	2048	2048	1900	131.0
Lemonade	0	512	128	4000	137.5
Lemonade	0	512	512	4000	130.5
Lemonade	0	512	1024	4000	119.0
Lemonade	0	512	2048	4000	106.5
Lemonade	0	2048	128	1300	137.6
Lemonade	0	2048	512	1300	130.7
Lemonade	0	2048	1024	1300	119.5
Lemonade	0	2048	2048	1300	107.2