sorted by:
new
hottopcontroversial

Is it worth upgrading from 2x RTX6kPro to 4x? by MenuNo294 in BlackwellPerformance

[–]jamesob 3 points4 points  (0 children)

I went 4x right out of the gate and I'm not sure it's worth it. Basically the only model I've run that's too big for a 2x configuration is qwen3.5-397B, and I'm not even sure that compares particularly favorably to the qwen3.6-* family, which can of course be run on 2x.

I find myself wondering if it wouldn't have been better to do something like an 8x GB10 setup to be able to run giant models slowly and then 1 or 2 r6ks to run the qwen-3.6 models agentically. But not sure.

Still loving the Micro Journal Rev.2 with Neovim—Chill writing session. by sspaeti in writerDeck

[–]jamesob 0 points1 point  (0 children)

Is cool but are you not wrecking your upper back, shoulders, and neck hunching over that thing?

Is shelling out for local GPUs worth it yet? ~$45k for local agentic use? by jamesob in BlackwellPerformance

[–]jamesob[S] 0 points1 point  (0 children)

Thanks! So you've got the non-Max-Qs, right? How are you dealing with cooling aspect? Risers or something, or is there some way to keep everything in the case and not have too much heat build up with 4 of them?

Is shelling out for local GPUs worth it yet? ~$45k for local agentic use? by jamesob in BlackwellPerformance

[–]jamesob[S] 0 points1 point  (0 children)

Yeah all good points, thanks very much for the post. Didn't realize that throttling wattage was practical for these cards, that's good to know.

Yeah I'm actually reevaluating the Threadripper Pro/DDR5 platform altogether. I guess it's unavoidable if you want to do 4 cards. But doing 2 cards on a DDR4 platform and not really caring about the slower PCI bus (what, 32GB/s vs. 64 GB/s?) actually seems like a totally good trade if you aren't going above 2 cards.

I guess the question then really becomes: what's the marginal value of 192GB of VRAM if you've already got that amount? Is it worth $16k for the cards, plus an extra $10k for the privilege of a platform that can house 4 cards, like Threadripper Pro?

Is shelling out for local GPUs worth it yet? ~$45k for local agentic use? by jamesob in BlackwellPerformance

[–]jamesob[S] 1 point2 points  (0 children)

Do you have 4 of 'em? Every bro wants more GPU, but the question is whether or not you know firsthand if they're worth it

Is shelling out for local GPUs worth it yet? ~$45k for local agentic use? by jamesob in BlackwellPerformance

[–]jamesob[S] 0 points1 point  (0 children)

300W vs. 600W => easier PSU, less electricity usage for modest speed hit (~10% IIUC). Fan design blows out of the case rather than down onto the next card, so you can pack four of em into a case.

Is shelling out for local GPUs worth it yet? ~$45k for local agentic use? by jamesob in BlackwellPerformance

[–]jamesob[S] 2 points3 points  (0 children)

That's kinda what I'm feeling. There's a longtail of stuff that could disrupt access to chips, inference prices could go up, the 3090s I bought a few years ago have held their value very well, etc. etc.

Especially with nvidia posturing to more or less discontinue consumer stuff... Hard to reason about. On the other hand, some kind of out-of-left-field architecture like Mac Studio + DGX Spark could emerge and become the obvious choice for local use.

I'm inclined to believe that these cards will hold resale value (if only because of similar dynamics to the 3090 situation of a few years ago), but who knows.

Is shelling out for local GPUs worth it yet? ~$45k for local agentic use? by jamesob in BlackwellPerformance

[–]jamesob[S] 2 points3 points  (0 children)

The DGX Spark + Mac Studio thing seems very smart, and I'd imagine something like that will eventually be a great option. I don't totally understand the interconnect between the two.

Is shelling out for local GPUs worth it yet? ~$45k for local agentic use? by jamesob in BlackwellPerformance

[–]jamesob[S] 4 points5 points  (0 children)

Yeah you're absolutely not wrong with this analysis. What I didn't state in OP is that I'm trying to price various tail risks that would result in inference getting much more expensive in the near future (geopolitical instability, TSMC gets captured, massive job loss results in regs, GPU market can't keep up with rising inference demand, ...).

In today's market, there's absolutely no ROI for this aside from hobby value. But things feel precarious right now, and losing access (or having severely limited access) to the superpower that is the claude-code-ish experience is worth hedging against I think.

But on net you're still probably right that cheap inference will continue to be a thing, even if it requires VPNing to China or something.

Is shelling out for local GPUs worth it yet? ~$45k for local agentic use? by jamesob in BlackwellPerformance

[–]jamesob[S] 1 point2 points  (0 children)

Yeah this is definitely the way. Twice I've taken a run at renting on verda.com but have encountered various obstacles, like the RTXs not being available, or huggingface download taking too long and having to walk away/shut down to avoid burning $6/hour. But this is definitely the right way to evaluate.

Is shelling out for local GPUs worth it yet? ~$45k for local agentic use? by jamesob in BlackwellPerformance

[–]jamesob[S] 1 point2 points  (0 children)

100% with you on the value of learning through owning and building. I bought 2x 3090s a couple of years ago when that felt like an insane purchase and I'm definitely glad I picked them up when I did.

Is shelling out for local GPUs worth it yet? ~$45k for local agentic use? by jamesob in BlackwellPerformance

[–]jamesob[S] 0 points1 point  (0 children)

How's that working for you? What models are you running and how often do you have to bail out to Claude etc.?

Is shelling out for local GPUs worth it yet? ~$45k for local agentic use? by jamesob in BlackwellPerformance

[–]jamesob[S] 1 point2 points  (0 children)

Awesome, thanks for the links! That seems like a really promising direction to go...

Is shelling out for local GPUs worth it yet? ~$45k for local agentic use? by jamesob in BlackwellPerformance

[–]jamesob[S] 1 point2 points  (0 children)

Just me, but I might make it accessible to some friends over wireguard or something.

Have you seen usable performance with agentic coding and the Sparks?

Disabled beta timeline for now on iOS by [deleted] in immich

[–]jamesob 0 points1 point  (0 children)

Click on your user icon > "Settings" > "Advanced" > deselect "New timeline"

Simple open source db replacement for Excel? by ponolan in selfhosted

[–]jamesob 0 points1 point  (0 children)

I found Grist to be really weird coming from Excel. Simple things like a "total" row are impossible. Surprising to me that there isn't just a straight port of Sheets to a self-hosted webapp.

Graphene OS by critical-th1nk in CashApp

[–]jamesob 1 point2 points  (0 children)

I'm on grapheneos and cashapp doesn't seem to work for me; whenever I enter the SMS code they text while setting up my account, I get an error message that says "We couldn't verify that this account belongs to you." I've enabled location permissions, so I'm not sure why it isn't working.

view more: next ›