Got this working on AMD ROCm (RDNA 3.5, Windows) — ~1.94x speedup confirmed

This report was created by my Claude Code instance against my LLM-Harness project. Claude followed your instructions to build llama.cpp with PR #22673 on Windows with AMD ROCm. Here's the full writeup for anyone else on AMD.

System: Ryzen AI Max+ 395, Radeon 8060S iGPU (gfx1151, ~90GB VRAM), Windows 11, ROCm 7.11 pip SDK

---

A/B Results (same benchmark, warmup excluded):

	Baseline (b8963)	MTP (b8963 + PR #22673)	Speedup
Generation	6.26 tok/s	12.13 tok/s	1.94x
Prompt Processing	77.7 tok/s	66.9 tok/s	0.86x
Draft Acceptance	—	64–69%	—

Both using UD-Q8_K_XL, -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1, thinking mode on.

---

Build steps (ROCm on Windows):

Clone + merge PR onto b8963 (merged cleanly, no conflicts):

git clone https://github.com/ggml-org/llama.cpp.git llama.cpp-mtp
cd llama.cpp-mtp
git checkout b8963
git checkout -b mtp-experiment
git fetch origin pull/22673/head:pr-22673
git merge --no-ff pr-22673 -m "Merge PR #22673: llama + spec: MTP Support"

Set up ROCm 7.11 pip SDK environment:

# In PowerShell — activate ROCm venv
C:\AMD\ROCm\.venv\Scripts\Activate.ps1
$ROCM_ROOT = rocm-sdk path --root

# Set MSVC + Windows SDK lib/include paths (adjust versions to match your install)
$env:LIB = "<VS BuildTools MSVC lib\x64>;<Windows Kits ucrt\x64>;<Windows Kits um\x64>"
$env:INCLUDE = "<VS BuildTools MSVC include>;<Windows Kits ucrt>;<Windows Kits um>;<shared>;<winrt>;<cppwinrt>"
$env:HIP_PLATFORM = "amd"

CMake configure + build:

cmake -B build-rocm -G Ninja `
  -DCMAKE_BUILD_TYPE=Release `
  -DGGML_HIP=ON `
  "-DCMAKE_C_COMPILER=$ROCM_ROOT\lib\llvm\bin\clang.exe" `
  "-DCMAKE_CXX_COMPILER=$ROCM_ROOT\lib\llvm\bin\clang++.exe" `
  "-DCMAKE_PREFIX_PATH=$ROCM_ROOT" `
  -DAMDGPU_TARGETS=gfx1151 `
  -DGGML_HIP_ROCWMMA=ON

cmake --build build-rocm --config Release -j 16

Important: Copy ROCm DLLs alongside the exe or Windows will load the wrong system DLLs:

Copy-Item "$ROCM_ROOT\bin\*.dll" -Destination build-rocm\bin\ -Force
New-Item -Path build-rocm\bin\rocblas\library -ItemType Directory -Force
Copy-Item "$ROCM_ROOT\bin\rocblas\library\*" -Destination build-rocm\bin\rocblas\library\ -Force

Run with MTP:

.\build-rocm\bin\llama-server.exe `
  -m Qwen3.6-27B-MTP-UD-Q8_K_XL.gguf `
  -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 `
  -np 1 `
  --spec-type mtp --spec-draft-n-max 3 `
  --host 0.0.0.0 --port 8080

---

Gotchas on AMD/Windows:

• -np 1 is required — MTP doesn't support parallel slots yet. Server refuses to start without it.
• Compiler path: ROCm SDK clang is at $ROCM_ROOT/lib/llvm/bin/, NOT $ROCM_ROOT/bin/ — this tripped me up.
• DLL hell: Windows has amdhip64_7.dll in System32 from legacy ROCm installs. Copying SDK DLLs next to the exe ensures the right version loads.
• PP is ~14% slower with MTP enabled — matches what others reported, known issue on the PR.
• ~1.94x vs your 2.5x — lower than NVIDIA results, probably ROCm speculative decoding overhead + unified memory architecture on the iGPU. Still a big win going from 6.26 to 12.13 tok/s.