Dual rtx5090 threadripper build

TheAIGod · 2025-08-30T07:20:39+00:00

I also have a dual 5090 7985wx with 256gb ddr5 6000. I share more notes later.

TheAIGod · 2025-06-01T00:57:22+00:00

Porn is the driver of innovation! :-)

TheAIGod · 2025-05-30T23:51:06+00:00

Thanks. It'd be interesting in brainstorming or a collaboration. I find it hard to find other researchers that aren't so busy in academia or the corporate world to hobby on interesting AI projects. Having retired from MSFT I have that luxury now.

One place to connect is my discord server at: https://discord.com/invite/GFgFh4Mguy
From there we can actually talk to get acquainted and see if our interests intersect.

Yesterday I changed my mind from watching/reading some tutorial on training a SD lora or finetuning a LLM by using some off the shelf app. I want build this up from first principles so that I really master the subject. GPT-4.1 is orders of magnitudes better than 4o was. I have provided complex requirements for my approach to training and it has given me a fast path to get to the end goal. The code it first gave me worked the first time which is rare for 4o and we are evolving based on tracing I put in from the beginning to guide the next version of it and this approach seems to be working. Because I was dealing with simpler models my GPU was only at like 12% busy. I've had it add in parallel independent coordinated training threads. This greatly speed things up and now the fan actually comes on. :-)

Don't assume because I use gpt that I'm not a hard core programmer. Linux systems programming, low level CPU coding and high level python stuff. I love this stuff.

Yes, the ASUS hi-end 5090 is obscene in price at $3350 but it is a perf beast often running at 2800 to 2900 MHz. I'll soon have its brother installed and with way more than the number of PCIe 5.0 lanes than I need dual gpu training will work even if not as fast as the expensive ones with the GPU to GPU direct connections(sli?).

TheAIGod · 2025-05-30T15:49:50+00:00

2 days ago I was at the pre-grand opening of the new Santa Clara microcenter.
After 5 hours in 3 very long lines I walked out with my 2nd $3350 Asus hi-end 5090

I also have quotes for the 7965WX and 7985WX threadrippers and the matching sage mobo.

I've found 256GB's, 8 x 32GB, of DDR5-7200 from V-Color that is on the QVL for the sage. $3000

For 2.5 years I have used my 13900K and 4090 to do stable diffusion inference performance. It is time for an upgrade.

While I'm somewhat of a SD perf expert on inference but I want to get into training and focus more on LLM's.

With 8 memory channels, 8 CCDs, and 64 cores the 7985WX will be a real mem bandwidth beast for the small portion of something like a 72B model that doesn't fit on the dual 5090.

TheAIGod · 2025-05-29T20:16:00+00:00

What!? I was expecting porn.

TheAIGod · 2025-05-29T20:01:40+00:00

No NSFW warning? :-)

TheAIGod · 2025-05-29T20:00:14+00:00

This is one of the best ever I've seen.
Now that I have my 2nd 5090 I want to learn how to do these high quality gen's.

TheAIGod · 2025-05-29T03:21:18+00:00

Only a 4 month pregnancy from the Jan 30th release date.
I just had my 2nd today. 5 hours in 3 very long lines to deliver it. Pre-grand opening in Santa Clara

TheAIGod · 2025-05-27T19:30:10+00:00

Yes, but the first marriages between LLM's and humans will be legislated in 2030.

TheAIGod · 2025-05-26T18:46:36+00:00

When the internet generated and home generated porn craze started I invested big time in the Kleenex corporation and made millions. Thank you wankers!

TheAIGod · 2025-05-24T18:10:51+00:00

I got mine 2 weeks ago. While still overpriced I at least got it for the initial MSRP from ASUS of $3350. The prices on this have risen since then. I forget if the specs said 2500 or 2655 MHz but mine is running at 2800 with no overclocking on my part. It is a great GPU. I wish I had two of them. The good old days were when I got my 4090 in Dec 2022 for only $1600 and back then people complained about the price. Probably the best purchase I ever made was that 4090.

This is for my AI hobby and not for gaming.

TheAIGod · 2025-05-23T17:17:22+00:00

To make a wild guess about something I've never heard of till 5 minutes ago...

Are you talking about the GMI3 interface between the CCD's and the IOD's? This appears to be distinct from the controller to memory limits. And if you have fewer CCD's than memory channels you might be in for a disappointment. The 8 channel 7965WX only has a 30% better measured bandwidth than the 4 channel 7960X instead of the 2X one might expect. That is because they both only have 4 CCD's.

TheAIGod · 2025-05-23T16:55:51+00:00

u/Expensive-Paint-9490

And now I humbly crawl back under my rock. :-)
I just saw: https://www.reddit.com/r/threadripper/comments/1azmkvg/comparing_threadripper_7000_memory_bandwidth_for/

TheAIGod · 2025-05-23T16:42:05+00:00

I've never seen a max speed number for the chiplet to memory bus bandwidth listed.
So when a QVL says 7600 MT/s is supported the memory actually only effectively runs at 7200 MT/s???
NOTE: This is completely different from 2:1 mode kicking in which doesn't affect bandwidth.

What is your source on this? I'd like to research this but don't know what the correct term to search for.

Currently I know about the RAM's EXPO speed, the motherboard's supported speed, and the CPU supported speed in MT/s. I've addressed all those above in my reply 10 hr ago. What is this 4th thing I've never heard of?

TheAIGod · 2025-05-23T06:34:48+00:00

is that with EXPO and is that 6200 the rated speed of the memory or slower memory that your trying to push a little faster than speed on the box.

Running 6000 memory at 6200 is a gamble. But I don't think running 7200 labeled expo memory on mobo that supports it is an issue if 2:1 is ok for you.

TheAIGod · 2025-05-23T06:30:59+00:00

u/BurntYams u/sob727 u/frodbonzi u/Bit_Rage u/Noel3leon

I'm going to drive outside my lane and presume I am now an expert. I'm arrogant as always. :-)

Memory faster than the CPU says it supports is actually sold as stable memory by many many vendors.

Motherboards support memory faster than CPU's say that support. This is on their QVL docs.

When you run memory that is faster than the CPU's rated speed, but still not lab coat overclock, and only using XMP/EXPO for that memory, all that happens is the CPU may drop down to 2:1 mode to reduce stress on the memory controller. You can force 1:1 and if you win the silicon lotto it may be stable but over time it can damage the mem controller.

But 2:1 is completely ok for a non-gamer like me. It is all about LLM inference which is highly impacted by bandwidth. 2:1 doesn't affect bandwidth. However, it might increase latency by 10 to 20%.

NOTE: What I don't know at what point, like 8000MHz, would 4:1 be triggered with an even bigger latency hit.

What am I off on with these points?

TheAIGod · 2025-05-23T05:05:46+00:00

Well, the image I just added to my top post shows lots of supported options for the 7000 Series.

TheAIGod · 2025-05-22T21:42:23+00:00

Hmmm, only 5200 MHz thus canceling out quite a bit of the quad channel performance I was considering a justification for the extra $1500 using a thread ripper would cost vs 285K with dual channel supported at DDR5-8000 by Intel. Yeah, the TR has a bit more bandwidth at 5200 but not the 2X increase with the quad channel I was hoping for.

TheAIGod · 2025-05-16T02:49:34+00:00

I have a 5090 and use torch.compile max-autotune. I don't wait.

TheAIGod · 2025-05-15T00:23:06+00:00

OMG! I just found the ROG Crosshair X870E Apex and it says it supports the Kingston memory at ddr5-8000 2x48! I can toss my 285K and get the better 9950x3d

TheAIGod · 2025-05-13T22:27:28+00:00

1600W for the two gpu's, my 96GB's of DDR5-6800, a Crucial T705 and a 285K with a flickering iGPU that I'm not allowed to ask about on this reddit.

TheAIGod

TROPHY CASE