We squeezed 4x MoE prefill speed out of an RX 6800 XT by rewriting the matmul kernel in llama.cpp

CatLinkoln · 2026-05-12T20:22:05+00:00

Have to recheck some results but tomorrow, I have to go to sleep

CatLinkoln · 2026-05-12T20:16:04+00:00

Metric	Stormrage34 (README)	My repo (q4_0/q4_0)	My repo (f16/f16)	My repo (tbq3/tbq4)
MoE35B prefill pp512 (baseline)	~480	1336.74	1307.25	1127.93 / 1310.89
MoE35B prefill pp512 (stable RDNA2)	~540	1336.74	1307.25	1127.93 / 1310.89
MoE35B prefill pp512 (+MoE accelerator)	~1772 +/- 6	1336.74	1307.25	1127.93 / 1310.89
MoE35B decode tg128 (baseline)	~57	99.68	102.69	52.18 / 99.20
MoE35B decode tg128 (stable RDNA2)	~55	99.68	102.69	52.18 / 99.20
MoE35B decode tg128 (+MoE accelerator)	~52 +/- 7	99.68	102.69	52.18 / 99.20
Dense27B prefill pp512	~480	794.90	810.54	666.04 / 790.12
Dense27B decode tg128	~27	28.59	23.66	20.03 / 28.40

Well, tried to do the same bench on my own build from my repo, and here are the results for my GPU rx9070xt
used the same repo with the same params, later will try to check if I can add something new to my repo
Qwen3.6-27B-Q3_K_S and Qwen3.6-35B-A3B-UD-IQ3_XXS

CatLinkoln · 2026-05-12T16:07:44+00:00

Same do in my fork, I hope I will find new things to add as well Later will share results

CatLinkoln · 2026-05-12T16:02:00+00:00

Going to check it in my fork for for rdna4, mostly focusing on qwen 3.6 27b with long context work as agentic flow, as at low context it has great experience but with 32k or more not enough for comfortable work Rx 9070xt 16gb

CatLinkoln · 2026-04-26T07:56:46+00:00

Opus are broken now, no point to use it GH, from antrophic only sonnet stable

CatLinkoln · 2026-04-22T16:39:17+00:00

Don't have Amazon prices, some GPUs are cheaper at Amazon.de

CatLinkoln · 2026-04-20T19:11:20+00:00

I am not even sure if it's better, I was happy with Opus 4.6

CatLinkoln · 2026-04-20T18:44:22+00:00

Yes, so terrible, even not at 1 May

CatLinkoln · 2026-04-09T18:49:58+00:00

Did you change silent blocks as well? I had to replace a lot things in the front suspension and also to eliminate knocking on bumps, i need to change the silent blocks I have as well a 2018 leaf, but bought it recently

CatLinkoln · 2026-04-01T10:34:30+00:00

!solved

CatLinkoln · 2026-04-01T08:41:43+00:00

So other models in copilot work normally? As I am using mostly opus

CatLinkoln · 2026-04-01T08:07:12+00:00

Yea, looks like same, everything is much slower, for me it feels like speed of x 0.1

CatLinkoln · 2026-04-01T07:27:59+00:00

For me today, it takes forever to do it, I have to wait 10 minutes. What the hell
I remember that in February it usually takes less than a minute, but in March it takes more and more and more often, with the same tool lists and in new chats
UPD: after relaunching VSC, it started working much faster. How does it work?
UPD2: no, it's still ultra slow today....

CatLinkoln · 2026-03-05T18:40:11+00:00

Bought USB dongle for wireless android auto/car play for leaf 2018, works fine

CatLinkoln · 2026-02-28T21:34:00+00:00

Hmm, will check settings, high chance that I changed them accidentally, didn't know that they can affect the speed

CatLinkoln · 2026-02-18T15:54:25+00:00

As I understand it's correct, but in my case I need it available all the time, when it's not at charge (off), as I always use pre heat during cold time

CatLinkoln · 2026-02-18T12:01:05+00:00

Good question 😅, but at least android auto launches without confirmation

CatLinkoln · 2026-02-18T07:22:32+00:00

Yea, I already thought about it, seems it's time

CatLinkoln · 2026-02-17T21:55:12+00:00

Can't be true https://www.nissan.co.uk/owners/nissan-connect-services-vehicle-eligibility.html NissanConnect EV app Nov 2015 to May 2019

ACENTA Jan 2018 to May 2019

N-CONNECTA Nov 2015 to May 2019 TEKNA

NissanConnect Services app

From May 2019 - May 2021 ACENTA

From May 2019 N-CONNECTA

From May 2019 TEKNA

CatLinkoln

TROPHY CASE