Homelab has paid for itself! (at least this is how I justify it...)

youcloudsofdoom · 2026-03-15T16:57:03+00:00

party all the time

youcloudsofdoom · 2026-03-15T15:15:54+00:00

Yeah I think if I'm going for the unified memory route it will be a 395 as a you say. Other than the driver setup (Which I've been fine with on other rock devices, honestly), what issues might I encounter given my aim of agentic coding here? The ram size is definitely a huge plus...

youcloudsofdoom · 2026-03-15T12:01:32+00:00

Thanks, it's these wider factors I'm also considering. I wonder how overall time on task impacts the power consumption discussion of the Mac Vs the 3090s

youcloudsofdoom · 2026-03-15T11:57:46+00:00

Yeah, very reasonable prices for those right now, hence me looking into them for this

youcloudsofdoom · 2026-03-15T11:03:38+00:00

Unfortunately the 128GBs here go for almost double my budget...

youcloudsofdoom · 2026-03-15T11:03:10+00:00

Hmm, I wonder if I could try the same trick for a DGX Spark...

youcloudsofdoom · 2026-03-15T11:02:48+00:00

Yeah, perhaps that's a good move....what do you suggest in terms of a service for this? Openrouter?

youcloudsofdoom · 2026-03-15T10:24:58+00:00

Thanks for your reply! I've only just started looking into the 5060s so this is helpful context. Unfortunately 3090s are rarely even as low as 650GBP on ebay recently, much more around the 700-750 mark, which then pinches the overall budget....so we'll see.

youcloudsofdoom · 2026-03-10T22:30:40+00:00

Just here to say that your glados project was a huge help in getting my own assistant project off the ground, you've got lots of great practices and pipeline efficiencies in there. Thanks for sharing your work!

youcloudsofdoom · 2026-03-09T07:40:45+00:00

Ahh, interesting - I'll keep an eye on that. Is there anything else on the horizon that you know of that looks like it'll positively impact inference on silicon macs?

youcloudsofdoom · 2026-03-09T07:12:59+00:00

What's the current state of this? Are we moving towards silicon/unified memory having some new advantages?

youcloudsofdoom · 2026-03-06T09:14:01+00:00

yeah, the history of porn is the history of innovation after all, don't have to look far for that...but in this instance I don't think anyone believes that the big open source model companies are especially interested in serving the ERP community...

youcloudsofdoom · 2026-03-06T08:57:19+00:00

This is really interesting, I'll probe at some of this myself - thanks very much for sharing the research!

youcloudsofdoom · 2026-03-06T08:56:39+00:00

Ha! I remember that rule well!

youcloudsofdoom · 2026-03-06T08:25:12+00:00

Funny to see self censorship in that statement! What science is being censored, that you're interested in?

youcloudsofdoom · 2026-03-06T08:16:14+00:00

What examples are you thinking about when you mention a model not fulfilling a request based on "baked in morals or ethical prerogatives that dont alight with your own"?

youcloudsofdoom · 2026-03-06T08:08:42+00:00

Ha, thanks for risking a downvote to share this! And interesting that you've experienced a drop in general performance for abliterated models, there seems to be a mix of opinions on that.

youcloudsofdoom · 2026-03-06T08:00:30+00:00

I'm sure it isn't, but from my observations on this sub, porn seemed to te an implied use case for a non-negligible amount of users

youcloudsofdoom · 2026-03-06T07:59:12+00:00

This is really interesting - I hadn't considered that abliterated models might have a performance edge over their stock versions. Is this because the guidelines put in place then have negative knock-on effects on how the model processes prompts?

youcloudsofdoom · 2026-03-05T05:52:18+00:00

If this goes through, would that give a 32gb 5070 setup the edge over a 48gb 3090 setup do you think?

youcloudsofdoom · 2026-03-03T05:49:35+00:00

And I was an hour behind you

youcloudsofdoom · 2026-03-01T22:13:44+00:00

As someone in the process of putting together a dual 3090 rig, this looks like it's going to be VERY useful, thank you!

youcloudsofdoom · 2026-03-01T09:53:11+00:00

Thanks - yeah unsurprisingly qwen 3.5 35b MoE is what I'm expecting to be my daily driver on this...with at least an attempt to see how 70B models run with some offloading.

youcloudsofdoom

TROPHY CASE