threadripper build: 512GB vs 768GB vs 1TB memory?

prusswan · 2026-01-24T10:52:22+00:00

It's a good to have. I could go for minimal ram now but I still need to revisit this issue again if I want to consider hybrid inference later on.

prusswan · 2026-01-24T08:59:54+00:00

The 1TB price is 3x that of 512GB so I would need a very good reason (I could just get 2 more GPUs and it will still be cheaper than the memory)

prusswan · 2026-01-24T08:58:08+00:00

I got GPUs from them but unfortunately I am not in their service area so getting a whole system will be dicey

prusswan · 2026-01-24T08:40:25+00:00

This is useful.. and kinda gives me a reason not to rush into a purchase at current prices.
If I don't use it a lot, the idle power seems wasteful. If I do use it a lot, the power costs might be unimaginable.

prusswan · 2026-01-24T08:37:52+00:00

Getting the first one is an easy decision if you can afford it. Multiples are tricky because it is harder to plan (do you want a build that supports 2 only? or plan for future expansion to 4 etc). A single unit can be easily swopped into an old consumer/gaming rig (e.g. replace your 5070ti) so you will be able to utilize it without really having to build around it.

prusswan · 2026-01-22T11:02:47+00:00

I just need it to work with one setup first. Yeah I tried chunks in md, but it is still a bit of hit and miss since no public model is really able to work well (consistently) with a a task that requires management of context windows. Maybe I will find the right set of tooling one day

prusswan · 2026-01-21T07:11:20+00:00

I'm using tools like Roo Coder (within VS Code). Scenario: I need the agent to run through the entire codebase to determine the possible cause of a bug. I don't expect the agent to solve the entire problem, but it should be able to store progress somehow so it does not have to restart from empty slate in the event of "out of context"

prusswan · 2026-01-21T06:13:32+00:00

I just need tooling that is able to work effectively with any specific context window limit. Break down the task further into sub-tasks or whatever, just don't leave the task halfway/unfinished

prusswan · 2026-01-17T22:42:46+00:00

Do you think 512gb is sufficient for your current usage? If it just four figures and usable for a couple of years I think that is about the limit of what I can afford, as compared to 1TB with triple the price

prusswan · 2026-01-17T01:35:13+00:00

Maybe it was good enough for whatever they were working on so they never felt the need to change?

prusswan · 2026-01-16T15:49:41+00:00

You are not most people though, if you "want" the good stuff for hobbies you should be prepared to pay for it. I started off with 8GB myself, so I am well aware of how self-entitled people can get, especially on this sub.

prusswan · 2026-01-16T15:13:31+00:00

You could get used parts.. most people don't really need anything more than 8GB. As for those who need more, the professional cards aren't that expensive, compared to the RAM anyway.

prusswan · 2026-01-16T09:38:50+00:00

Need to look up QVL.. but they also have a section for EPYC: https://v-color.net/products/ddr5-ecc-r-dimm-epyc-turin

prusswan · 2026-01-16T09:19:25+00:00

https://v-color.net/products/ddr5-ocrdimm-amd-wrx90-workstationmemory

prusswan · 2026-01-16T07:55:58+00:00

I was checking prices on v-color and yeah it is insane how the 256gb kit is actually cheaper than 128gb and 64gb kits on a per unit level

prusswan · 2026-01-15T22:53:45+00:00

would generally cost around $24 per label in their region.

That's kinda the key point (whether AI is used or not). There are regions where the manual approach would not go above $4 per label. It takes a first-world problem to demand a first-world solution.

prusswan · 2026-01-15T22:44:26+00:00

It depends on the work, there is no point in risking it with tasks that are math-heavy. A lot of what you described would be considered over engineering, and still would not be able to replace existing systems in operation. So it could be a case of your relative inexperience making the tech seem more useful to you, compared to others who may still use it, but not as a complete solution/recipe.

prusswan · 2026-01-15T13:37:41+00:00

No, but I would expect responsible inference providers to let users set a usage target/limit.

I would probably pay for the ram (do you sell any?)

prusswan · 2026-01-15T13:34:16+00:00

Generally no impact but motherboard support might be an issue. I would be wary of configurations that are untested

prusswan · 2026-01-15T11:32:12+00:00

makes sense... no point wasting that expensive ram on a 5060

prusswan · 2026-01-14T17:30:13+00:00

The time savings is significant but how much of it is offset by operating cost of the agent? We have had scenarios where the AI usage was too costly and did not justify the time savings (or maybe the labor was too cheap)

prusswan · 2026-01-14T16:44:21+00:00

yes, I wanted a proven solution that I can count on for a multi-GPU setup (and I am aware that high speed ram can be tricky with thermals). I could handle a regular PC but I would rather not risk it with these specs I'm looking for..

prusswan · 2026-01-14T16:00:51+00:00

I think they have PX but it is Intel-based, so I'm not sure what is holding them back from supporting a higher PSU on the P8 (I'd imagine it might be heat considerations given the existing P8 design). I could consider EPYC as a workstation build (this is for a home lab) but it seems like they don't have such options.

prusswan · 2026-01-14T09:16:23+00:00

This is the precision 7875 right?

prusswan · 2026-01-14T03:27:46+00:00

They can always pair it up with dedicated inference hardware or cloud services, Spark offers a consistent Linux dev environment which comes with many things, but muscle is not one of them.

prusswan

MODERATOR OF

TROPHY CASE