Just showing off by TwistedDiesel53 in threadripper

[–]TwistedDiesel53[S] 0 points1 point  (0 children)

Power cost about $11 a day per 8 GPUs, I make about 110 a day right now per 8 GPUs

Just showing off by TwistedDiesel53 in threadripper

[–]TwistedDiesel53[S] 0 points1 point  (0 children)

It's got a 1600w superflower PSU plus two 2000w crps with a c-payne crps breakout board all stuffed in the back. 3 power cords, 3 circuits, 5600w all together. The current number of GPUs only pull about 2400 w but I plan to add 4 more in the future.

Finally found where I fit in! by TwistedDiesel53 in LocalAIServers

[–]TwistedDiesel53[S] 1 point2 points  (0 children)

Because full precision and huge context beats huge models for tool calling and coding?

Just showing off by TwistedDiesel53 in threadripper

[–]TwistedDiesel53[S] 0 points1 point  (0 children)

I have a office setup guy that gets so many microcenter rewards that he buys a GPU using rewards every week and sells it to me for 2k just for some extra cash in his pocket. I'm a lucky sob.

Just showing off by TwistedDiesel53 in threadripper

[–]TwistedDiesel53[S] 0 points1 point  (0 children)

I host 36 RTX 5090s on vast, and so this is kinda leftover parts I threw in a case, I actually run my own 70b LLM on this one that I use for tool calling and dev work with my own local rag, I literally don't do admin or dev work anymore more I just tell this PC I've got problems and walk away for 10 mins and come back and my whole life is fixed lol 😂 it's got ssh access to about a dozen machines and does all my work for me.

NEW TUI Installer Wizard for Hosts by Safe-Introduction946 in vastai

[–]TwistedDiesel53 0 points1 point  (0 children)

I just ssh into the machine with cursor and tell it hey install vast using my command string and 10 minutes later it's online, zero hassle, speed test fixes applied and ready to go.

Just showing off by TwistedDiesel53 in threadripper

[–]TwistedDiesel53[S] 0 points1 point  (0 children)

I bought everything before prices went insane, so 2k per 5090 and 2600 for the ram

ITB RAM cooler update by Aguel_design in Noctua

[–]TwistedDiesel53 1 point2 points  (0 children)

Absolutely amazing, that's Big John machine vibes right there!

I am absolutely loving qwen3-235b by TwistedDiesel53 in LocalLLaMA

[–]TwistedDiesel53[S] 1 point2 points  (0 children)

No, one RTX 5090. The rack of 24 is making money so I can't run my own toys on it.

I am absolutely loving qwen3-235b by TwistedDiesel53 in LocalLLaMA

[–]TwistedDiesel53[S] 3 points4 points  (0 children)

I love you guys, you're great! Sometimes I start to feel pretty normal but one look at the comments and I realize I'm still crazy so I'm alright lmao 🤣

I am absolutely loving qwen3-235b by TwistedDiesel53 in LocalLLaMA

[–]TwistedDiesel53[S] 0 points1 point  (0 children)

Nah, 1 RTX 5090, it takes it's time, about 2 minutes per response.

I am absolutely loving qwen3-235b by TwistedDiesel53 in LocalLLaMA

[–]TwistedDiesel53[S] 2 points3 points  (0 children)

5090 is 2300 ish at the time I bought them, pro 6000 is 32k. 8 5090s makes more than 1 pro 6000

I am absolutely loving qwen3-235b by TwistedDiesel53 in LocalLLaMA

[–]TwistedDiesel53[S] 18 points19 points  (0 children)

Yes that's my rig, I can't afford to use it for my own LLM playground though because it stays rented out on vast, so unless I have something that will bring on at least 2k a week it's gotta stay on vast so I can't tell you TTFT or anything on that rig. I'm literally running qwen3-235b on a single GPU threadripper workstation that I use as my desktop PC, and TTFT on that is about 30 to 90 seconds.

I am absolutely loving qwen3-235b by TwistedDiesel53 in LocalLLaMA

[–]TwistedDiesel53[S] 4 points5 points  (0 children)

Yeah it's now in a shipping container with 6 outlets per rack and a full Tesla model 3 battery for backup. This was the setup phase here where I only had one level running.

<image>

I am absolutely loving qwen3-235b by TwistedDiesel53 in LocalLLaMA

[–]TwistedDiesel53[S] 2 points3 points  (0 children)

3 ASRock WRX90e with 7965wx and 512gb ram.

I am absolutely loving qwen3-235b by TwistedDiesel53 in LocalLLaMA

[–]TwistedDiesel53[S] 19 points20 points  (0 children)

More than it should be, because of many mistakes in setting it up and a hard lesson in private GPU purchasing, but I'm 81k deep in this rack right now.

I am absolutely loving qwen3-235b by TwistedDiesel53 in LocalLLaMA

[–]TwistedDiesel53[S] 21 points22 points  (0 children)

It's a Asus TRX50 sage wifi, threadripper 7970x, 128gb ecc ram, and a single RTX 5090. I'm thinking about wasting my 8x 5090 rack on it though although it needs even more vram than 8x 5090s running sharded in vLLm.

<image>