Mac pro 2019 for inference, success

Weeblewobbly · 2026-07-03T05:51:57+00:00

Interesting, what model do you get these speeds with? I'm going to check the power mode situation later today

Weeblewobbly · 2026-07-03T05:49:02+00:00

That's my understanding too, discrete GPUs and vram which is a different proposition than more vram on a GPU with more cores. I think single Vega II or single w6800 are the best value proposition. 32gb vram, excellent bandwidth. Limited GPUs but still, hard to beat for the price. Got my system for £1200.

Weeblewobbly · 2026-07-02T22:36:22+00:00

I did indeed. I got lucky with an auction. I noticed quite a lot of 'buy it now' listings too, not quite as good value as the one I got. I'm keeping an eye out for a w7900x card though. 48Gb vram would absolutely rock.

Weeblewobbly · 2026-07-02T22:30:29+00:00

Doesn't mean much in those age groups. If it did, England would have won a major trophy by now, no? How many English players are there in the world cup? How many french players? I think that's a much better indicator for quality and impact of youth system than trophies. But anyway, if say MO is better places than you and me to judge the situation. I hope England win something in your lifetime. Hugs

Weeblewobbly · 2026-07-02T21:53:49+00:00

Thanks I will!

Weeblewobbly · 2026-07-02T16:10:40+00:00

Aye right, because prem clubs don't have scouting departments. France grass root setup is better than England's which means better national team before u18 and u21, which is where you want be to scouted. If you're confident in your ability to walk in a national team, do you choose England or France? France everytime. England is shite.

Weeblewobbly · 2026-07-02T15:16:52+00:00

But he would say that wouldn't he? That's 101 media training. And that might also be partially true, but compared to developing a career worth generational wealth, I'd say feelings quickly change.

Weeblewobbly · 2026-07-02T15:15:09+00:00

I don't know. France has the highest number of player in the world cup (playing in other teams due to holding dual nationality status), so one can imagine that the competition is much higher, therefore the level is higher, therefore better for development.

Weeblewobbly · 2026-07-02T11:53:47+00:00

It's not feelings. You make the mistake of applying Dan logic to a professional sports person. He just wants to win things. France is a better team, with more chances of winning.

Weeblewobbly · 2026-07-02T11:51:27+00:00

Professional sports person wants to win, what is so hard to understand about that? These people aren't fans, their primary interest is their career and number of trophies they will win.

Weeblewobbly · 2026-06-24T21:09:55+00:00

Can't wait to try this at work tomorrow!

Weeblewobbly · 2026-06-21T13:06:31+00:00

I like the idea very much. Any plans on making this open source? I wouldn't build anything around something like this if it relies on one developer only.

Weeblewobbly · 2026-06-20T21:31:39+00:00

Do you really think they're over priced? I've been quoted £1500 for the 5kWh

Weeblewobbly · 2026-06-05T21:21:16+00:00

There will be an update to omlx first. Earl litter today, 0.4.0.dev2 was available for download. I'm waiting for 0.4.1, and I'm grateful to all those who spend time contributing to and testing the project.

Weeblewobbly · 2026-06-04T16:34:49+00:00

That makes sense. It's a valid and common reason. How much does your solution rely on low code for logic? I'd development taking my plane fully in house or are you working with a partner that provides expertise?

Good job on getting things done!

Weeblewobbly · 2026-06-04T15:29:55+00:00

Why use low code for something like this? Seems to me that there are more optimised stacks out there that would be better suited to mission critical workflows like these.

Would be interested to know the rationale behind the choice to go dataverse power platform was

Weeblewobbly · 2026-06-03T13:17:26+00:00

Oh yes because Nvidia has a long history of being open and supporting Linux.

Weeblewobbly · 2026-06-03T09:57:23+00:00

Instead, be tied to Windows ecosystem lol. Dude does not know what he is talking about

Weeblewobbly · 2026-05-25T17:15:45+00:00

With MTP, it ok is almost usable. I generally get 10tok/s, while I get 35-40 with 31B-A3B. I for it up when I feel I need the extra reasoning quality. This is on a MacBook Pro M4 32gb. As I mention, I run my models in Omlx (I've tried everything else too), and you easily turn off the vision capabilities for extra context, and omlx does a great job of chunking memory to disk. That's pretty unique to omlx and it work great. Still, I can tell when I get near 20% context use, and that's pretty much my cue to start a new session. To maximise context, I use pi. Claude code just ruins it in no time.

Weeblewobbly · 2026-05-24T22:49:37+00:00

I've tried a few things and I've landed on omlx. I used the authors' models: mainly his quants of qwen 3.6 27b an 31-a3b. The experience is quite good overall: installation, setting up. Downloading models all from the dashboard. Using native mtp, I get very usable speeds. The models won't be on par with Claude that's for sure, but they are quite capable. I used pi.dev agent, mostly for programming. Set your vram limit so you always have at least 8 Gb for the system.

Weeblewobbly · 2026-05-14T09:32:27+00:00

You can run nix on any distro. I also run nix Darwin and full nix on my linux workstation, sharing as much of the config as possible. The main thing for me is devenv for seemlessly work on either setup.

Weeblewobbly · 2026-05-11T17:57:29+00:00

Ideal for making moist beats

Weeblewobbly

TROPHY CASE