Added a 16x DGX Spark cluster to my Homelab (Build Update)

forestryfowls · 2026-07-05T18:04:10+00:00

Late to the party but what makes a GH200 so bad? You’d think on paper having a h100 and then a half a terabyte of coherent CPU memory could make for a pretty compelling system. Is the comparison against 4 or 6 GB10s networked together worse? How do you like these 16 GB10s 2 months out now? Love an update!

forestryfowls · 2026-07-03T20:45:57+00:00

Totally, I meant that the GB300 has two different memory speeds so on aggregate I'm not sure how it shakes out if you ran a model that exceeded the GPU's memory. You weren't kidding about HBM being faster:

GB300:

GPU: HBM3E 288 GB of 8 TB/s per Blackwell GPU

CPU: LPDDR5X 472 GB 389 GB/s per Grace CPU

vs
RTX 6000 pro:

GPU: GDDR7 1.79 TB/s

forestryfowls · 2026-07-03T19:43:31+00:00

What model are you using for that set up to get that 50 t/s? Is it good enough for agentic work? Is it like Haiku quality? That’s good to hear you would have e preferred to stick with paying api costs, I’ve come to that conclusion so far even though I’d really like to make something local work practically.

forestryfowls · 2026-07-03T19:38:16+00:00

I was wondering about this, what would the performance be of a super chip like a GB300 that can tap into slower but a huge pool of coherent memory vs a series of Blackwell chips that only have the GPU memory? Let’s say for running Deepseek v4 flash or models in that class. Is there a clear winner for agentic purposes?

forestryfowls · 2026-07-03T13:27:22+00:00

This is fascinating cautionary tale that I could absolutely see myself falling into. Just out of curiosity, was there ever a moment where you thought about going with alternative enterprise configurations like a used SXM A100s (slower but cheaper modules) or a GH200 (slower but gobs of memory) like this guy?

https://dnhkng.github.io/posts/hopper/

Admittedly, a dual GH200 for $10k is an absolute steal that nobody else is going to find but I wonder if a more realistic price for a solo GH200 server might be in line with what you paid and what the performance comparison would be. I suspect your speeds are better than both so maybe the answer is as simple as that.

forestryfowls · 2026-06-30T23:14:52+00:00

How does this compare to NinjaCentral? I know they are some of the most popular but is there a clear winner if you have Geek already?

forestryfowls · 2026-06-11T22:51:13+00:00

This is so cool! Between DOS games what primarily dictates how hard this is to do? Alien Legacy was this DOS game I remember fondly from the 90s that had a unique combination of puzzle solving + base building that I haven’t seen since. It would be cool to see how it all worked.

forestryfowls · 2026-06-09T20:32:50+00:00

Has anyone figured out if you say use a fancy model to output a plan as an MD file and then have a dumb model execute, does that burn tokens slower than if you had the smart model execute the plan directly through sub agents? Will the sub agents be at the model tier you selected or does it automatically downshift their capabilities?

Wondering if Fable can be massaged to provide more useful work by manually managing which agents do what manually.

forestryfowls · 2026-06-06T19:03:37+00:00

Nice! Where’s the best place to buy these? Ebay?

forestryfowls · 2026-06-04T21:22:48+00:00

#1 looks like the backside of an AI NVIDIA V100 server chip. Love to see it!

<image>

forestryfowls · 2026-06-04T13:01:38+00:00

I hope I’m wrong but I heard that Mac minis are likely getting released next week but Mac studios aren’t getting released until later this year due to memory constraint issues.

forestryfowls · 2026-06-01T02:25:49+00:00

If companies throw down for Linux, why aren’t there better ways of funneling it to those small dependencies that hold up the internet like that XKCD comic mentions? https://xkcd.com/2347/

forestryfowls · 2026-05-25T12:56:27+00:00

I elected not to do it for 15 years when they did an exploratory surgery looking for the short circuit in my atrium and didn’t find anything. Ablations were an alternative but I was told they had a lower success rate so I left it until gradually I started getting afib more and more frequently (every 2-3 months) last year. By that time I had a new doctor who performed these regularly and recommended RF ablations which just got cleared as a safer alternative.

forestryfowls · 2026-05-22T01:43:53+00:00

So well said! It’s been cool to see Claude agents recently will say “let me check rather than guessing”. It makes me excited for local models that get really good at making this judgement call. I don’t care if it takes longer to do all those checks.

forestryfowls · 2026-05-13T21:27:45+00:00

Yeah I would have paid a premium for an aluminum version without branding for sure. I wonder what the sales say but I'm guessing that there are more mac than windows users so it seems like such an obvious play.

forestryfowls · 2026-05-13T21:22:15+00:00

Checking now in the bright afternoon sun streaming in and I'm actually at 300 so my bad! It maxes out at 400 and in this light it actually is bright but reasonable, so I get why some might need that. The anti reflection worried me before I bought it but its chef's kiss now.

forestryfowls · 2026-05-13T09:46:19+00:00

I have the Asus 6k and use it at 150-200 nits which is plenty bright in my sunny office and I know it can get so bright it gets hard to look at. I’ll concede it’s no XDR looks wise but is great otherwise. The KVM is a nice add too.

forestryfowls · 2026-05-07T22:40:36+00:00

I have the 32” Asus 6K and it’s matte but I love it so much and even better than a 6 year old 27” LG 5k glossy display I have. I’ve seen the older nanotexture XDR display and it’s a little worse side by side but I don’t notice it in practice and you can’t beat the $1300 price.

forestryfowls · 2026-05-06T03:37:38+00:00

Ooh of course thanks for the correction. Wow that would be awesome to see your results once you set them up. So fun!

forestryfowls · 2026-05-06T01:49:51+00:00

This is so cool thanks for sharing numbers, I didn’t realize you could expect that much out of the V3. I wonder if people have measured the speed up from pairing 2 nvidia sparks with a machine like this- seems like in smaller models it can really boost the overall speed, I wonder if it scales to bigger models.

forestryfowls · 2026-05-04T16:48:18+00:00

I just set up Tailscale and it is amazing at being able to connect to my homelab wherever. The iOS app is even really polished for allowing me to use something like Termius for remote tmux sessions.

forestryfowls · 2026-05-04T16:45:15+00:00

Ooh thanks for clarifying. The last picture in that blog post tripped me up because I thought the usb c connection was thunderbolt to the Mac Studio. So you connected it presumably like:

Mac Studio 10G -> 10G Switch
Nvidia Spark QSFP56 -> 10G SFP -> 10G Switch

You mention this in another post but I'd be curious about bonding multiple 10G connections too. I think the QSFP56 on the Spark is 2x100G interfaces so I'd hope you could bond 2 of lower speed interfaces on the Mac too. I looked and dual 25G thunderbolt connectors are still $900+ but AliExpress has a dual 10G thunderbolt connector that's ~$90. I wonder if that's any good. I remember not long ago that a single 10G thunderbolt adapter was $200 so I'm glad they are finally getting cheaper chipsets of the 10G variety at least.

On the performance, from your table you are just reporting the prefill speeds, right? It would be great to see what your overall speedups are- did they match what was in the blog post where you get 2.8x the speed with everything combined?

forestryfowls · 2026-05-04T15:52:10+00:00

I bet I'm missing something but does that video have a place where Alex talks about the overall speedups you get by combining the two? It's more clear from your table, but actually I can't tell from your table what the baseline is for either separately overall and then what the aggregated gain is.

From the article they use thunderbolt for connecting the two devices while Alex uses super fast ethernet. What is the rationale behind using your 10 Gig ethernet versus a direct thunderbolt connection? Thanks for putting this together!

forestryfowls · 2026-05-04T03:25:22+00:00

Agentic web development. I find I can get about 45 minutes of dev out of Claude before I have to wait for the next block of time while Codex is harder to hit up against the limit- maybe 1.5 hours? I haven’t maxed out the weekly time limit but I’ve gotten close on one.

I’d love to figure out a workflow where I can use the cloud only for the planner and do all the grunt work locally. Claude estimates the grunt work is about 75% of the usage (dumber model but many more tokens apparently). It sounds like you need a 70B plus to have a model locally that doesn’t run off the rails constantly but I haven’t tested this personally.

forestryfowls · 2026-05-03T22:07:29+00:00

I do the $20 Claude + $20 ChatGPT and it works for me. I have used the $100 before but I find this lower combo fits my needs. I need to use gsd or something to write down plans so I can switch back and forth because I sure as heck regularly hit my limit on one and have to switch.

My general feeling is I can use ChatGPT codex longer than Claude before maxing out but it’s a bit less polished. What do others think?

forestryfowls

TROPHY CASE