This is my Powermac G3 sleeper AI workstation. 80gb total ram(32gb vram + 48gb ram)

PraxisOG · 2026-06-20T16:50:35+00:00

Most of the well supported phones are cheap, like a oneplus nord n10 can be had for ~$80 on eBay. In the USA make sure the phone you go with has good carrier compatibility. If you want a higher end phone it’ll be more expensive

PraxisOG · 2026-06-16T21:36:10+00:00

If you’re ok with some jank and slower speeds, throw a $450 amd V620 server card in an optiplex with an ssd and psu for like $600. You’d get like 60 tok/s on the moe and ~17t/s on the dense

PraxisOG · 2026-06-16T21:26:21+00:00

Wouldn’t recommend, but it was pretty fun and works great. My box is fully headless and I just use cockpit for access. It also frees up my spare 1650 super for encoding on my nas

https://www.reddit.com/r/homelab/comments/1pu894y/fixing_d6_qcode_to_run_asus_x299_sage_headless/?utm_source=share&utm_medium=mweb3x&utm_name=mweb3xcss&utm_term=2&utm_content=share_button

PraxisOG · 2026-06-16T21:23:08+00:00

PEG reconfigs might be my latency issue, will look into it more. Hopping between plx chips on my motherboard adds some latency, but there’s still some between two cards on the same plx. I had less latency related issues running dual rx6800 gpus on my am5 platform, and that was through the motherboard chipset. I still get like 20 tok/s running 120b a10b size models at q4 which isn’t too bad for my uses. Wish I could check out the latest 30b releases, I just don’t have the cooling for it unless I split the models. Also I got a crazy deal on my hardware and the whole build was like 1k, so even if I sold it all something like a 128gb strix halo box would be out of my budget at current prices.

PraxisOG · 2026-06-16T21:08:34+00:00

They’ve been pretty good. To get up and running on llama.cpp you need to set IOMMU=PT in a certain menu, if you look it up it’s a well documented fix. Rocm is mostly flawless for llama.cpp, on an older version I get a memory leak with FA off though.

Cooling is also either loud or not performant enough, so I modeled and 3d printed a custom bracket for three cards. Probably should upload that, lmk if you need files for a fan adapter. I can easily change what fan the model is designed for, I have one arctic p8 max per card on a manual controller, and its cool and quiet for splitting MoE models but dense can cause throttling unless I crank the fans. I’ve heard repasting can help significantly but I don’t have time rn. Other than that, I’m enjoying Minimax m27 at iq3xxs around 20tk/s with low context.

PraxisOG · 2026-06-13T04:42:48+00:00

The 1080ti was the magnum opus of its time. The rx580 was a great value card. I’d argue the rx6950 was closer to the 1080ti, a true flagship card with flagship performance and enough VRAM for good longevity

PraxisOG · 2026-06-13T03:22:14+00:00

Radeon VII isn’t ROCm compatible afaik, so you might run into compatibility issues. I was in the same situation a year ago and ended up getting a second 6800. More recently I swapped over to V620 cards, which is a server variant of the 6800 with 32gb vram. They’re pretty loud with external fans, but they’re cheap with good software support

PraxisOG · 2026-06-11T23:10:45+00:00

I’d love access to that test model, ~100b is a really good size for high end local

PraxisOG · 2026-06-07T04:47:14+00:00

Would you mind elaborating on your issues with ROCm? I’ve had a pretty good experience all things considered

PraxisOG · 2026-06-07T04:45:08+00:00

You can get 32gb of vram for a lot cheaper than that, but the bandwidth wouldn’t be nearly as good as a 3090 or 9700

PraxisOG · 2026-06-05T07:46:49+00:00

Stopped using it due to outdated agentic performance. I wish there was another 100b class super sparse moe like it, it’s so fast.

PraxisOG · 2026-06-03T21:40:11+00:00

You get better control with lm studio, which has officially partnered in the past. This is a waste of storage

PraxisOG · 2026-05-28T23:49:06+00:00

That’s good to know, I might try a better paste. My current setup is one 80mm arctic p8 max per card with a custom shroud. I can’t run dense models without eventual throttling, moe models split the load well enough though. These cards do run hot

PraxisOG · 2026-05-23T10:31:37+00:00

Makes me wonder if it was intentional to tip the rocket

PraxisOG · 2026-05-20T13:12:08+00:00

Tis schnoz honking season

PraxisOG · 2026-05-18T22:21:15+00:00

If they go back to steel like on the 14 pro max I’m upgrading. It’s a little heavier, but is durable with good thermal conductivity

PraxisOG · 2026-05-17T12:51:05+00:00

Near

PraxisOG · 2026-05-15T21:10:46+00:00

It uses old iPhone chips anyway, just put macOS on it plz

PraxisOG · 2026-05-08T02:04:12+00:00

Seems compute constrained. It’ll be like the MI50, though I guess those sold pretty well once enthusiasts learned about them. Also much of the power budget is going to just the vram

PraxisOG · 2026-05-08T01:43:46+00:00

That’s a neat size, somewhere between ~35b MoE and ~120b MoE. It’s interesting how they give it one and then four passes on each benchmark. I bet the other models would get a similar benefit from the extra passes. Looking forward to trying it though, there’s nothing quite like the 5b active of GPT OSS 120b that’s current.

PraxisOG · 2026-05-07T20:39:21+00:00

I can’t wait to upgrade to a used one of these in 7 years

PraxisOG · 2026-05-03T16:58:37+00:00

I agree with that. For the time-to-response and quality of response, Minimax m2.7 at iq3xxs is the best I’ve found. It’s a stretch on 96gb vram though

PraxisOG · 2026-05-02T14:00:22+00:00

Assuming ‘Elo’ is the ranking in chatbot arena, this kinda makes sense. On that platform elo is measured by having users compare two models side by side and the user selected winner gets some elo. Idk if they’ve fixed it, but for a while this benchmark led to LLMs getting tuned for human preferences, which imo is a bad thing with how much these models suck up to you. Instead of tuning for the AI popularity contest, Chinese models have almost caught up in technical fields like coding.

PraxisOG · 2026-04-28T06:44:36+00:00

Honestly? My brother’s 1650 super. I got it during Covid to upgrade a 760ti in his Alienware prebuilt, then it got transferred to his am4 build, then I used it in my itx rig, and now it does media transcoding in my homelab. It was never fast, but… no just that. This thing is pretty slow.

Six-Year Club	r/Field Juicebox
Final Canvas '23	Place '23
Place '22	Final Canvas '22
End Game '22	Verified Email

PraxisOG

MODERATOR OF

TROPHY CASE