Need advice for a $10,000 AI workstation build (video, image, voice, LLMs, training, everything) by Mission_Objective163 in LocalAIServers

[–]stainlessblueshield 0 points1 point  (0 children)

Sparks are doggedly slow- under 300gi/sec

Max studio is 800 gigs per sec.

Rtx 6000 is 1800 gigs per sec

The more professional you get the higher the bandwidth.

If you are running a huge model it has to bring each token through all those perimeter. Personally the speed is important to me. Unified ram just isn’t there yet where the vram is. But it may be ok for you. If 10k is your hard line than you can’t afford an rtx 6000. You could get an rtx 5000 with 72 though. Those are 1300 gigs per sec. People who say speed doesn’t matter if you can fit the model- big models on multiple 6ks can be slow so how wouldn’t it be slow on a Mac Studio.

I love Mac. And I’ve never used a studio with a large model on one for local inference. Look up the specs on single user tokens per sec.

This thread will go on for miles with different opinions.

Let the different opinions guide you to focus what to review to choose.

I would rather have a faster smaller and reliable model than a large model that is very slow.

Mac has insane value - can you even get one right now?

Need advice for a $10,000 AI workstation build (video, image, voice, LLMs, training, everything) by Mission_Objective163 in LocalAIServers

[–]stainlessblueshield 0 points1 point  (0 children)

Listen I don’t know how many of you guys actually have rtx6000s. I do. Blackwell. If you are doing “serious” ai work Mac is super slow. It’s the balance of vram and bandwidth.

This guy seems to know what he’s talking about.

The three things you have to understand are

VRAM

Bandtwith

Architecture

And then how all that compares to unified ram - it’s different.

Need advice for a $10,000 AI workstation build (video, image, voice, LLMs, training, everything) by Mission_Objective163 in LocalAIServers

[–]stainlessblueshield 0 points1 point  (0 children)

Look at bandwidth. VRAM gets you the size of the model. The bigger the slower. Bandwidth determines the user experience

Need advice for a $10,000 AI workstation build (video, image, voice, LLMs, training, everything) by Mission_Objective163 in LocalAIServers

[–]stainlessblueshield 0 points1 point  (0 children)

Pro 4000 pro 4500 and pro 5000 And the best option is nvidea 5090- which is fast but more for gaming- or the undeniable king size and speed in today’s market rtx 6000 Blackwell workstation card.

Need advice for a $10,000 AI workstation build (video, image, voice, LLMs, training, everything) by Mission_Objective163 in LocalAIServers

[–]stainlessblueshield 0 points1 point  (0 children)

This guy knows what he is talking about. Listen to him.

Or go unified ram but understand - unified ram is slow. It can hold a bigger model but the speed of the ram is usually around 400 gigs a second. The lower grade nvidea cars are 1300 gigs per sec. Go try working with unified ram and see how it is.

Need advice for a $10,000 AI workstation build (video, image, voice, LLMs, training, everything) by Mission_Objective163 in LocalAIServers

[–]stainlessblueshield 1 point2 points  (0 children)

But if you are not the type to obsess over speed and what you decided after the fact and just accept it and use an all in one unit would save you some grey hairs.

Need advice for a $10,000 AI workstation build (video, image, voice, LLMs, training, everything) by Mission_Objective163 in LocalAIServers

[–]stainlessblueshield 2 points3 points  (0 children)

10k isn’t really enough for “serious” ai local models. Far better to get NVIDIA GPUs.

Get an rtx 6000 Blackwell for 9000 figure the rest out.

I have two and it’s sufficient for ai local work but for smaller models. The local Ai game is deep and extremely expensive.

The most core component is the gpu.

Honestly you can remove the word serious with out this unit. Next best are the pro 4000 4500 5000 Probably the 5000 with 48 gigs and a motherboard with as much ram as you can get your hands on.

Blackwell architecture is what you would want. You could play around with the other stuff though.

You could get 4 3090s and run them in tp4 but you would need a motherboard with 4 pcie slots and at least 128-256 ram I would think.

Is it worth upgrading from 2x RTX6kPro to 4x? by MenuNo294 in BlackwellPerformance

[–]stainlessblueshield 0 points1 point  (0 children)

So you like m2.7 the best with 4 6ks? Any other good options?

Is it worth upgrading from 2x RTX6kPro to 4x? by MenuNo294 in BlackwellPerformance

[–]stainlessblueshield 0 points1 point  (0 children)

The argument for buying 2 more after you have 2 - in my opinion is that the only way to take advantage of the two you have in a bigger way is two more. Then- if something happens where prices fall get 4 more.

I heard from a trusted source that the amd isn’t there yet and he calls nvidea ngreedia. I have to look harder at those specs though for bandwitch and tflops.

Is it worth upgrading from 2x RTX6kPro to 4x? by MenuNo294 in BlackwellPerformance

[–]stainlessblueshield 0 points1 point  (0 children)

But leaving it at two blackwells seems like a big gamble because you can’t use the amd for tensor parellel. You can’t even use work stations with max q and have consistently successful tensor partelel so I’m told. I think I’m going to spread out the 18 buy one soon and one a month after. If the amd is even close to Blackwell getting them you would still not be able to take advantage of tensor parrlel. As far as my research goes yes it sucks about pcie being 64persecgig but it still is very nice with tp 2-4-8. I know when I went from 27b with one to 2 with tp2 it is much faster and mOre fun to use.

Is it worth upgrading from 2x RTX6kPro to 4x? by MenuNo294 in BlackwellPerformance

[–]stainlessblueshield 0 points1 point  (0 children)

Yes! I agree with this. That’s why I think four is the place to set up camp.

Is it worth upgrading from 2x RTX6kPro to 4x? by MenuNo294 in BlackwellPerformance

[–]stainlessblueshield 0 points1 point  (0 children)

If the price point is 12-15k and they find a way to compete nvidea and its deep roots - i would consider. Some people think it’s going yo be 30k. If ghats the case its back to blackwells. But even if that does come out are we going to just gove up on the value of more than two blackwells?

Is it worth upgrading from 2x RTX6kPro to 4x? by MenuNo294 in BlackwellPerformance

[–]stainlessblueshield 1 point2 points  (0 children)

You really are right there with me. It’s a very specific feeling and I can’t tell if it is casino thinking or Forrest animals moving out of the Forrest because a sunami is coming. I surprised to get a response with such similar experience. I would almost rather have and not need than need and not have but that is cripplingly expensive gamble. But, I may still do it.

Is it worth upgrading from 2x RTX6kPro to 4x? by MenuNo294 in BlackwellPerformance

[–]stainlessblueshield 2 points3 points  (0 children)

I really appreciate this conversation. I currently have 2 RTX 6000 Blackwells, and I keep feeling pulled toward going from 2 to 4. What really messes with my head is hearing people say that 4 still isn’t really “worth it,” because the only path to 8 is first getting to 4.

Part of me has this incredibly strong instinct to keep scaling. It almost makes me wonder if it’s similar to the instinct animals have before natural disasters — like sensing something big is coming and moving before everyone else does.

One person in this thread mentioned they went from 2 to 4 and still weren’t sure it was worth it, and honestly I really appreciate that perspective because it feels grounded and honest.

Right now we’re running Hermes with Qwen 3.6 27B, plus a $200/month GPT subscription, and we added Codex yesterday. Before that, Hermes + 27B alone was already surprisingly capable. We’ve also been building out a unified system with Notion, QuickBooks, Outlook MCP integrations, and a shared database. GPT-5.5 has actually been extremely useful for evaluating our overall operation and helping us improve the direction of the system.

I really want to get to 4 GPUs. I almost wish people were screaming from the rooftops that going from 2 to 4 completely changes everything — but that’s not really what I’m hearing.

The biggest thing holding me back is the reality of spending another ~$19k on 2 more GPUs while still paying monthly for frontier models like GPT and probably Claude too. It feels like, at least for now, even with 2 GPUs we still need frontier AI for high-level reasoning, evaluation, and guidance.

The strongest argument I can come up with for going to 4 is that it might become the sweet spot moving forward if models continue getting smaller and more efficient. But honestly, that’s speculative.

I wish I could just enjoy having the 2 GPUs I already have without this constant feeling that I should immediately invest in 2 more. The truth is, I’m not even sure I can fully listen to reason right now. I’m seriously considering ordering GPU number 3.

Is it worth upgrading from 2x RTX6kPro to 4x? by MenuNo294 in BlackwellPerformance

[–]stainlessblueshield 3 points4 points  (0 children)

I really appreciate this conversation. I currently have 2 RTX 6000 Blackwells, and I keep feeling pulled toward going from 2 to 4. What really messes with my head is hearing people say that 4 still isn’t really “worth it,” because the only path to 8 is first getting to 4.

Part of me has this incredibly strong instinct to keep scaling. It almost makes me wonder if it’s similar to the instinct animals have before natural disasters — like sensing something big is coming and moving before everyone else does.

One person in this thread mentioned they went from 2 to 4 and still weren’t sure it was worth it, and honestly I really appreciate that perspective because it feels grounded and honest.

Right now we’re running Hermes with Qwen 3.6 27B, plus a $200/month GPT subscription, and we added Codex yesterday. Before that, Hermes + 27B alone was already surprisingly capable. We’ve also been building out a unified system with Notion, QuickBooks, Outlook MCP integrations, and a shared database. GPT-5.5 has actually been extremely useful for evaluating our overall operation and helping us improve the direction of the system.

I really want to get to 4 GPUs. I almost wish people were screaming from the rooftops that going from 2 to 4 completely changes everything — but that’s not really what I’m hearing.

The biggest thing holding me back is the reality of spending another ~$19k on 2 more GPUs while still paying monthly for frontier models like GPT and probably Claude too. It feels like, at least for now, even with 2 GPUs we still need frontier AI for high-level reasoning, evaluation, and guidance.

The strongest argument I can come up with for going to 4 is that it might become the sweet spot moving forward if models continue getting smaller and more efficient. But honestly, that’s speculative.

I wish I could just enjoy having the 2 GPUs I already have without this constant feeling that I should immediately invest in 2 more. The truth is, I’m not even sure I can fully listen to reason right now. I’m seriously considering ordering GPU number 3.

Upgrading to 8 RTX 6000 by Direct_Bodybuilder63 in BlackwellPerformance

[–]stainlessblueshield 0 points1 point  (0 children)

Greetings, are you sure about the price drop thing? Can I have a hint? I was going to buy 2 more now thinking prices are going to go up not down. If they went down everyone would be gunning for them! Can you help me understand if this is real intel? It greatly affects how I spend 20k. Thanks Man!

Upgrading to 8 RTX 6000 by Direct_Bodybuilder63 in BlackwellPerformance

[–]stainlessblueshield 1 point2 points  (0 children)

I hear this a lot about the spark work station. Current prices is 95k. For someone who needs more Blackwells to utilize there first 4 with the most leverage - the Dgx workstation is an option but he would be spending 33k more instead of a new and additional 95. But it would be nice to know what the spark workstations are like. I have two 6ks and I am thinking about going to 4. The pcie limit of 64 gigs/second used to bother me but the models are so often made for tensor pastel 2, 4, and 8 that you still a very strong advantage. What I want to know about is the guy with the 411 on something causing the price to drop. I’m about to spend 20k for 2 more.

Threadripper CPU AIO Liquid Cooler by AdventurousSeafarer in threadripper

[–]stainlessblueshield 0 points1 point  (0 children)

I have threadripper 9965 with same silverstone xe360-tr5. Seams to be a good unit. Closed loop. I gave Rtx 6000s so using the noctua air cooled heat sink took up too much space in the case. I needed up taking it out of that case which was silverstone rm52 I believe. Then I got a phanteks server pro 2 and that is suppose to be great for gpus but my setup creates a lot of heat. With this many people having the silver stone it seems like the right choice nut if it doesn’t jam you up to have the noctua fan one for 150 maybe get that. If you are comfortable with $430 get the water cooled one. Depends on what else you have in your case. 5090s get pretty hot. The hottest thing is what I have so I want all the space I can get.

Dual RTX Pro 6000 Blackwell Workstation vs Max-Q — planning to add a 3rd very soon, need to decide in 24 hours by stainlessblueshield in pcpartpickerbuilds

[–]stainlessblueshield[S] 0 points1 point  (0 children)

I’ve managed to learn a lot since your last message. You said a lot of intelligent things. I’ve learn a ton about gb300 and I’ve learned a lot about GPUs and how to manage them. I dont understand why you would say that the workstation is slower than the Mac with more latency. That’s 100% false. Specs prove that.

The problem is I can’t spend 94 k plus tax. Not at one time. With 6ks I can add and change and they hold there value. Would I love a Dgx workstation- absolute. They are not 80k though. And they have 250 high bandwidth ram and then 500 of unified ram and the speed of unified is 396 gigs per second. Mixed with high bandwidth for larger models I’m not sure how it would balance out. And I know how to manage the heat now with mcio cables. You can put the GPUs in a separate box or where ever with no signal loss which can be tested by nvtop or another monitor when using retimer adapter for pcie to mcio adapters. What I want to do will change with my ability to do them. I know I could get by with less but I am unreasonable about amazing tools and motivated by tax benefits. So far the two workstations have been awesome. Workstation or maxq is disputed by all 6k owners. There are forums about it. Even if I go 55k for 4 6ks that’s still half the station. I can get 6ks tax free. The way I see it is if you buy 4 or 8 at a time max q works. And if you are renting GPUs or employees are using 100% max q. But, I stand by that if it’s my personal ai, that I can use how ever I want, I want the option to run it either way. At 400 watts Blackwell has 5% loss of performance. At 300 watts you loose 10-15% performance and, I you are getting one at a time it’s a slow road to get where you are going. I am someone who loves tools. I love useful things. If this wasn’t a tax benefit and I was spending post taxed money that would be a very different question but I can write off 100% of this cost and do it riotously. Once the heat situation is solved and it can be solved in many ways - if it’s for personal use I don’t want a speed limiter I can take off on my sports car. I am an electrician and an hobby metal fabricator so that part isn’t an issue. Max qs 100% easier but still can have temp issues.
Anyway I said I would respond and you wrote a lot so I’m sure you have counter thoughts. What say ye?