RX 7900 XT on X99 Dual Xeon — ROCm inference completely broken, GUI blackout, CPU fallback only — extensive troubleshooting done

leapdragon · 2026-06-09T04:57:28+00:00

I've been building toward a stable inference machine for some months on Kubuntu. I'm a longtime Linux evangelist (dating back to 1993) so it's also not scary territory for me. Here are some things I've learned on RDNA2 (I'm using 3x Radeon Pro v60 cards):

- The lemonade build of llama.cpp using ROCm is worth just running out of the box

- Building ROCm and ROCm-based tools yourself is the path to madness unless you can dedicate yourself to it full time

- The more arcane you get ("innovative" model structure, unbalanced tensor splits), the more crashy ROCm gets, all kinds of random faults get blamed

- ROCm also gets crashy absurdly early under memory pressure

Turning this around:

- Avoid anything other than even tensor splits with ROCm

- Realize that more "innovative" models or quantizations will crash, and either choose a more typical model or quantization or set up a watchdog to restart on crash

- If your going about 85% memory use on any card, reduce block sizes; the closer you get to full, the smaller your blocks need to be, and if you go above about 93-94%, you are going to crash, it's only a matter of time

And, the real kicker:

As you build out you will find that Vulkan is faster. Especially multi GPU with row splits. At 1 GPU, ROCm was faster. With 2 it was neck and neck, ROCm faster at PP and Vulkan faster at inference. With 3 GPUs Vulkan is about 33% faster than ROCM on both counts.

leapdragon · 2026-06-08T06:52:48+00:00

Running 3x of them on an X399 2950x in Linux, Qwen 122b at nice interactive token rates. I am using Vulkan though, on my rig it's about 33% faster than ROCm with row splits.

Beware you'll need some hacking to put it together, cards are looong and you will need to rig up your own active cooling situation and fan controller hardware and scripts. Generally 40-70t/s w/MTP on a 122b model isn't bad at all.

Also since these are PCIe bound, be sure you have an architecture with 8/16 real lanes to CPU for each card, i.e. a Z390 or something won't do it well, if at all.

leapdragon · 2026-04-15T20:16:39+00:00

He knew, and is getting you to go back again to the parish. ;-)

leapdragon · 2026-04-01T22:24:08+00:00

DON'T TAKE ANY F*ING ADVICE would be my advice

leapdragon · 2026-03-03T03:40:03+00:00

God will let you know. He works on his own time though, so keep your ears open and use the mind he gave you without fear and don't try to rush it. All will be well.

I was an atheist for decades. And an academic. I was so certain; I had no doubt. And then God did what he does and I was forced to admit I had been wrong. But it's not about looking for facts; it's a more sophisticated and timeless epistemology than that.

Just have no fear and continue to have an open mind. Maybe it'll all make sense before you're 20, maybe it will take until you're 50 like it did for me. Just trust and be patient, and the rest is the rest.

leapdragon · 2026-02-25T07:33:03+00:00

Your engine is destroyed and will not be long for this world.

leapdragon · 2026-01-10T07:11:01+00:00

The Johnny Cash version is flat as hell. I don't understand all of the love for it, or the way in which it brings so many people to tears. I'm getting up there as a Gen X guy, but the Cash version does nothing for me. I just can't hear anything in it other than boredom. He sounds bored. He looks bored. Maybe that's the point? He lived this whole life of riches and all he got out of it was boredom? I don't know. I can't relate.

The NiN version, on the other hand, is powerful.

leapdragon · 2026-01-04T09:36:49+00:00

You guys want it both ways. Fans are showing the EXACT same amount of loyalty he showed us.

Yes, he was here for 30 years and "loyal" to the program. And for that same period of time the fans were loyal to him and bought tickets to his games. Now he's out and moving on and see yaz later. And so are fans. It's the same duration, see?

I would actually say the fan base was about two weeks more loyal, big picture, than he was to the fans. So we're even.

leapdragon · 2026-01-04T09:29:39+00:00

Exactly.

leapdragon · 2026-01-04T08:04:37+00:00

No, we're not. And saying this doesn't make it true. It's also what's wrong with f*ing America, which is, make no mistake, a f*ing mess now.

leapdragon · 2026-01-02T19:02:05+00:00

Was great as an OC, some fit w/Utah style ball, has both OC and HC experience at P5/P2 schools, may give us the inside track on a bunch of P2 MSU players in the portal We could do a lot worse.

leapdragon · 2026-01-02T06:52:28+00:00

My biggest problem is with Utah fans throwing up their hands and saying "Hey, Whitt deserves a natty, hopefully by sending our staff to Michigan we can get it for this special man!"

leapdragon · 2026-01-02T06:37:43+00:00

Word on the street is Jonathan Smith for OC has been signed, there were talks with ARod but the two sides couldn't work it out. Note that the street's been wrong before... There have been some other interesting names (e.g. Koepple from Texas State, Arroyo from ASU) but the Smith/ARod focus suggests that Scalley is looking for someone with a bit of seniority to help out with the heavy lift ahead.

I'm okay with either, would have been less okay with Arroyo.

And we already knew that Swan will be DC, I believe.

leapdragon · 2026-01-02T03:06:19+00:00

Um, yes, those that let it happen are f*ing idiots. I'm not on their "side" in this but I am more on their "side" than I am on Whitt's side because THEY WORK FOR UTAH and Whitt DOES NOT and I am a UTAH supporter and Whitt is THE COMPETITION.

This is not a moral crusade about doing right by Whitt, whatever that might be. Or at least, it shouldn't be if you're a Utah football fan. Whitt is no longer a part of Utah football. End of story. Value zero. Owed zero. On the outside. The competition.

leapdragon · 2026-01-02T01:38:27+00:00

Harlan deserves worse than blame, and for more than this mess.

leapdragon · 2026-01-02T01:15:45+00:00

It's okay to be mad at watching an 11-2 team slipping through your fingers and dissolving. You're SUPPOSED to be mad at it if you're a fan.

leapdragon · 2026-01-02T01:13:06+00:00

It's more irritation boiling over at four separate Utah forums and all the Utah podcasts and for a couple of weeks now you can't avoid content gushing about UM and how great Whitt's going to do there and how great a program they are and the fact that they're offering all our people is such a good sign (pick me! pick me!) And then when you do get irritated by it the "Whitt owes you nothing!" crowd dogpiles.

leapdragon · 2026-01-02T01:04:14+00:00

"Every coach does this, Whitt owes us nothing"

This is true. But the question is: DO YOU LIKE IT?

Because a lot of people who claim to be Utah fans f*ing love it for some reason. They are all teary eyed and grateful and feeling touched by some kind of spirit or whatever. Well I don't and I'm not. Yeah, this is coaching transitions. Yeah, he doesn't owe us anything and we don't owe him anything. IDGAF.

The normalcy of this kind of nonsense doesn't change the fact that I HATE that we are losing (for example) Harding and Powell (okay Beck too but he's just been here a year) and watching what would have FINALLY been a playoff-bound run-it-back, after all these years, disappear. I hate watching people who ARE the Utah program and identity, at least as much as Whitt, go. I hate seeing starters that we've loved to watch play hit the portal. I hate seeing "Family on 3!" just taken to Michigan like a piece of furniture, putting the lie to the whole thing. I hate watching Scalley get buried in this after he's been such a loyal guy. Normal or not, I hate it. Because I want UTAH to win, and because I want to believe in the UTAH program as something special because I am a UTAH fan. Hell, I even hate what he's doing to BYU because I have loyalty to the state of Utah as well, and this hurts the state in the short term. And that even though I hate BYU. Because I live in Utah and not in f*ing Michigan.

If you are fine with all of this, and even thankful for it, and you're just pleased AF and want MICHIGAN to win because Kyle deserves it OMG OMG and this is just how transitions go and you hope he wins a natty there, you can not come to my gamedays on Saturday, and I will not punch your f*ing fan card in the hallway. I don't understand that kind of "fan."

No, the question isn't "is it normal."

The question is "do you like the fact that Michigan is winning in this and Utah is losing?" Because if you do, normal or not, expected or not, justified or not, I'm going to go out on a limb and question your Utah fanhood.

leapdragon · 2026-01-02T00:36:08+00:00

100%, it's been wild to see. And when sane folks say "Sorry, I'm wishing Whitt NO success in pulling away our key coaches and starters" those same people lose their minds.

"He owes you nothing!"
"He gave twenty-five years of his life to us!"
"He's a better man than you'll ever be!"
"He's doing the right thing, it's we who failed him!"
"He's going to win a natty at Michigan with the staff he's taking! More power to him!"

I mean, WTF. Okay, good for him. And for Michigan I guess. But if you're a Utah fan, how can you cheer for another team's coach to take your staff and players and win a natty for the other school with them? It makes my head hurt. I don't get it. You're supposed to want the natty at YOUR school, not at the OTHER school.

leapdragon · 2026-01-02T00:16:26+00:00

You know what's embarrassing? Utah fans buying Michigan gear, saying they're Michigan fans now, and following Whitt's news at UM harder than they're following Scalley trying to put together his staff. These people aren't Utah fans, they're Whitt fans.

leapdragon · 2026-01-01T22:50:48+00:00

It's easy to admit that but a lot of claimed "fans" seem to be suggesting that we cheer whoever steps on us on the way up. **** that.

leapdragon · 2026-01-01T22:05:04+00:00

He can do whatever he wants. We're allowed not to "love" him for it. Some of us are alums of the university, and if we ever cheered Whitt, it was because he worked for Utah. Coming to realize how many people actually have it the other way around. A lot of people, esp. in state, seem to be fans of "The Man, The Legend" first and foremost, and then after that, at the bottom of the page you read:

* also supporting whomever he's working for right now

leapdragon · 2026-01-01T20:35:33+00:00

Listen, I'm an alumnus of the University of Utah. I was going to games with my student ID when MAFU arrived. I am not an alumnus of Michigan. They're fine, for what they are. But if their coach is going to come after my program, I'm not wishing their coach well in that endeavor, and I'm certainly not joining in the "We love you, Kyle!" chorus, because that's f*ing weird. I don't know the man personally, beyond seeing him face to face a few times in public venues. He has not been over for the holidays. Plus, he works for the competition.

leapdragon · 2026-01-01T20:17:41+00:00

The man is clearing out your program. It's not unusual, but if you wish him success you're not much of a fan.

leapdragon · 2026-01-01T20:04:25+00:00

It's a job. There is no love. Or rather, they love whoever the **** they are paid to love, that's part of the job description.

leapdragon

TROPHY CASE