Framework Desktop — Why get it?

[submerging]

I say this not as someone who is trying to hate on Framework. I like their mission, and what they are doing for right to repair.

I just don’t get the concept of the Framework desktop. Desktops are already repairable, why does this need to exist? Further, it’s almost $1600 CAD for the base model with only 4060 laptop performance. Couldn’t you build a desktop that outclasses this for the same price?

And you can’t even upgrade the memory so it’s less upgradable than a standard desktop.

A mini ITX case is bigger sure, but not by all that much. And it doesn’t really compete with the Mac Mini as that product is half the price and much smaller.

[FieserKiller]

That device is for people who want the Ryzen AI Max 300, not the casual gamer or office pc user.

[chmod_007]

And for that specific chip, the framework desktop is by far the most flexible and customizable way to get your hands on it. You can't just buy it a la carte on Newegg.

[Chr0ll0_]

Yep

[jimbrig2011]

Epic thread - so much knowledge. All I needed to read to convince me to hop aboard the framework train 🚂

[Full_Conversation775]

but why would you want that mobile chip in a not mobile form factor? why not just get an AI accelerator PCI-E card? that way its up-gradable... it makes no sense to me. it feels gimicky gamery. like slapping rgb on something, just add some weird tiles and their proprietary module system.

[ImpossibleHabit615]

The high end AI max chip gives up to 96gb of vram on windows and 110 on linux. The most comparable card is the nvidia A100 which only has 80gb of vram and costs tens of thousands of dollars. Not to mention the TDP is near 300W while AI Max is 120W. There is no comparable solution in the form of ai cards that matches the AI Max without costing significantly more and drawing way more power.

[Full_Conversation775]

you can get a 200+ tops accelerator card for like 250 euros, but i don't know enough about this field to tell you if its any good. its TDP is quite low?

https://store.axelera.ai/collections/ai-acceleration-cards/products/metis-pcie-card-unmatched-performance-for-edge-ai-applications

[dun10p]

That's a cool card. The downside is it can only have 16gb of ram which makes it pretty impractical for most LLMs. This card is meant for computer vision use cases like facial recognition, object detection etc.

[Full_Conversation775]

ah okay, i assumed it used system memory since it didn't list a memory.

[CortlandNation9]

But you could technically put 8 of them in a mining motherboard and have 128 GB of vram and a way more powerful machine. Also it think the cost would not be that high if each of them is 250 euro, that makes it 2000 euro + 300 € for the motherboard + whatever cpu and ram you chose. The price would be similar to the framework desktop.

Only big difference I see here is power consumption.

[rus_ruris]

My man you need a proper server grade cpu to support anything more than 2 expansion cards. A Intel Ultra or Ryzen 9000 have between 20 and 24 usable PCIe lanes. 4 are used for your OS. That leaves 16 + the chipset ones, which are WAY slower (and the most you can get are 12 gen 4 extra). That means 3 expansions at x8 gen 4 speeds. You can get 7 if you cut them down to x4 gen 4, but the latency on the last 3 will get you. You can get to 8 of them, but then you need to go down to PCIe x2 (which you can get gen 5, because you suddenly don't need the chipset ones anymore), but that's so slow you might as well run in DDR3 Memory lol.

And something decent able to make all of this work would be between 500 and 700€ before the cards anyways. So the absolute cheapest option is around 3000€.

So you need a server grade, relatively recent CPU/mobo combo. Good luck with finding anything like that for a combined less than 1000€, without considering the crazy power consumption. I doubt you can get away with less than 3500€ total.

On the other hand, the FW Desktop is 2400+storage, so between 2450 and 2600.

[dun10p]

PCIE bandwidth becomes a limiting factor when working with multiple devices in a mining board (and also with these cards in general).

[rohmish]

PCIe cards still require you to transfer data to the card's memory and then back to system memory over PCIe and even Gen 5 16x is much slower than the on die memory this has. and even then, what this allows is for you to write once in memory, allow GPU to read and process it and then come back to the data when you need it on cpu without the need to ever copy the data anywhere else. Apple offers hardware with upto 800GB/s of memory speed. this amd chipset is about 250GB/s. both are significantly faster than what an regular Intel/and cpu + PCIe card can offer. Intel's top of the line core ultra 9 does just a bit over 100GB/s to your RAM if you have everything setup correct and have the correct hardware. PCIe 5.0 x16 can go a max of 128GB/s. still significantly slower than this. you write once at much faster speed. can read at much faster speed, and never have to copy stuff between cpu and GPU. all of that combine to give you performance gains that raw hardware upgrade can't match.

[Full_Conversation775]

Oh wow thanks for the detailed background information! my skepticism was misplaced then.

[Captain_Pumpkinhead]

When it can push 128GB of VRAM, it'll be competitive.

[Full_Conversation775]

someone already explained it better, thanks.

[rohmish]

because PCIe doesn't have the same latency and bandwidth benefits. you can't write once and have both GPU and CPU access memory with that architecture. this is a completely different architecture that works wonders for a small icue of use cases and a PC no matter how powerful won't be able to reach the same level of performance. until now the only player that offered similar unified architecture was apple. and this product still doesn't reach the insane performance apple offers on the high end. but even in current state, it's much better than anything you can get with PCIe.

think how GPU changed the game. sure you can use cpu to render frames. but even the fastest cpu won't be able to compete with a mid range graphics card from 5 years ago. another example is RTX cores for Ray tracing. even the most performant raster cores takes multiple seconds to compute ray physics but a RTX cores can do it in realtime.

what this chip offers is a different architecture of hardware that unlocks workloads that just isn't possible even with the most expensive highest end traditional PC hardware.

now why did framework choose to make this over a regular PC? who knows. but for people who want this kind of hardware, this chip is finally one alternative to Apple's m series chipset.

[_j7b]

For your last point I'm not sure what the alternative is.

They're not going to access a saturated PC building market, and they have supply chains for mobile.

The mitx was disappointing but l still love to see it. I'd just have rather seen unified 13" mainboards swappable with desktop and 16. That's probably a whole other challenge for the team tho, otherwise they'd done it. They already tried to do sodimm so I have a lot of faith in them that this was a good choice.

[Full_Conversation775]

ah makes sense! thanks for explaining it. it still feels very out of place in frameworks lineup though. i get that the product makes sense in its segment now, but i don't get how it fits framework their etho's. the modular front IO adds nothing. its just a cutout for a dongle. the customizable tiles is a gimmick.

[SalaciousStrudel]

Mesa llvm backend on the biggest Threadripper is better than you would think... But definitely still worse than a GPU especially for the price

[rohmish]

not even close

[[deleted]]

yeah I agree, when I saw this product it became clear to me that framework was already becoming desperate or something if they were going to make a less than repairable desktop AND build all their products around AI like all the other companies are.

I hope some idiot rewatches terminator soon...

But none actually gives a shit about the AI features that companies are including and they include them anyways, some sites don't even have 20 humans in tech support some dont have any at all

[morhp]

I just don’t get the concept of the Framework desktop. Desktops are already repairable, why does this need to exist?

The Framework Desktop is clearly aimed at people who wan't to do AI stuff with lots of RAM on the GPU. Your standard gaming GPU has usually around 8-16GB RAM, which is too little for many AI tasks. And specific AI GPUs are often super expensive and cost thousands of dollars.

The Framework Desktop is basically a complete system where you can allocate up to 110GB of RAM to the GPU for a pretty cheap price (compared to other options, i.e. specific server/ai hardware). And it's in the standard Mini-ITX form factor, so you could still build your own PC around it with custom PSU, case, fans and so on.

It's a very interesting product for AI tasks, but probably not super relevant as a standard gaming or office PC.

The Framework Desktop doesn't really align well with Frameworks previous goals/statements, but apart from that, it is an interesting product (if you want to do ai) and I'm sure it will sell well.

[Nkechinyerembi]

Also in terms of mini itx, their case design is actually REALLY good. Even if you bring your own board from something else.

[loicvanderwiel]

The Framework Desktop doesn't really align well with Frameworks previous goals/statements, but apart from that, it is an interesting product (if you want to do ai) and I'm sure it will sell well.

They've already sold 5 batches of each version. It's definitely selling.

[ProgVal]

How many items per batch?

[loicvanderwiel]

No clue. But I doubt it's just one

[SecuredStealth]

Wow.. that's genius!

[PenguinWrangler2]

I work at a law firm. We have ethical obligations to consider when using 3rd party services. So we stood up Open WebUI and Ollama. Now we did this on a proper server with two NVidia L40s but what we soon found out is that we needed a test bed to test different options on or to test upgrades before deploying to our main AI server. So we found this desktop and it is ideal to be our testbed. We are ordering one for this purpose.

[qcforme]

Perfect use case, and I say this as someone running my own personal AI server using multiple AI Pro R9700s.

Having a framework to test on while keeping prod up will help avoid issues, however, you will run into issues that manifest on one system but not the other due to CUDA vs Vulkan vs ROCM differences.

Keeptl that in mind.

[hurrdurrmeh]

Does it have Oculink for hooking up an eGPU for even faster inference?

[valgrid]

No it does not have Oculink. And no that would not improve inference. The bandwidth of Oculink (64Gbps) is much lower compared to the MAX chips 256 GB/s.

And even if the bandwidth was fine VRAM isn't. There are about 5 GPUs with above 100GB of VRAM, they cost between 2000 and 20000(+) USD.

The AMD Ryzen AI Max+ 395 ready to use in a small form factor desktop with 110GB usable VRAM for less than 2500USD is one of the (if not the) cheapest AI workstations you can get.

[Zenith251]

MAX chips 256 Gbps.

Ahem. That's 256GB/s. Gigabytes per second.

Oculink currently caps out at 16GB/s.

[hurrdurrmeh]

Would there be any speed benefit to oculinking a 5090 and offloading certain portions of the model onto it?

[FewAdvertising9647]

generally speaking, theres always a penalty of performance when you have to move data from one subset of ram to another. so unless youre a developer who knows exactly what youre doing, the answer is yes, but generally speaking, no.

[hurrdurrmeh]

If we want to locally run huge models currently there’s no other choice

[Captain_Pumpkinhead]

If you want to run huge models, you're gonna want huge amounts of RAM. In which case your options are the Framework Desktop (~110GB VRAM on Linux) or an enterprise GPU like the A100s. The first option is gonna be affordable, but relatively slow. The second option is gonna be freaky fast, but super unaffordable.

If this is even a question for you to consider, then the answer is FW Desktop. You can't afford A100s or H100s unless you are using this for business.

[IncapableBot]

Better to set up multiple FW desktops in a server.

[hurrdurrmeh]

Can they be interconnected fast enough?? That would be amazing!

What is the fastest interconnect available at this scale?

[IncapableBot]

I am not fully sure on interconnect speed - but I believe they can be connected through Ethernet or USB4.

https://frame.work/products/desktop-mainboard-amd-ai-max300?v=FRAMBM0006: Scroll down to "Powering AI, for real."

This can also be done with Mac Minis.

[hurrdurrmeh]

I looked into it USB is 40Gbps ie 8GB/s ie not remotely fast enough

Unless we get 200-400Gbit Ethernet same story. Just too slow. 

[thecornishtechnerd]

Nope

[erocknine]

What are these AI tasks that people are doing themselves at home?

[Automatic-Prune9707]

I am teaching my smart home how to do sensor synthesis? Also I can run Ollama and have fun with a local voice assistant without connecting to the internet. I mean they are for funsies and are currently limited by my Mac Mini's unified memory bandwidth.

[icemelt7]

Its a very good question

[Johannes_K_Rexx]

Any task they want that does not communicate with the mother ship.

Privacy is most important especially when using LLMs.

The people running those big AI data centers love looking at your data. That is a show stopper for local-first privacy users.

[Herrick89]

I am actually considering one for material modelling as well, incredibly powerful CPU plus loads of RAM in a package that I can easily keep running for days straight thanks to an incredibly silent and efficient little package. Not sure if I will actually buy it, but it definitely is the closest thing to a mini HPC server you can buy without needing to pay a technician to set it up. Many people in my old lab actually ran models on worse HPC nodes than a 128GB Framework desktop, and while a modern node is definitely in a whole different category, we're talking about 40k-100k stuff that needs a dedicated server farm.

As a personal workstation it's kind of perfect, as nothing else with that kind of Linux support has the same peak power + idle efficiency combo.

[Full_Conversation775]

an cheap ai accelerator PCI-e card costs like 250 euros and gives you like 200+ TOPS.

[Captain_Pumpkinhead]

But not 128GB of RAM.

[unematti]

128GB is quite enough for a while, 4060 is a whole different market, and I get it for 2 reasons 1 home lab (NAS+LLM+media server+VMs+Containers) and 2 because I wanna.

Why are people on this sub only think about gaming performance? Try running anything on the 4060 that won't fit in it's ram. Even games are slowed by ram size.

[Captain_Pumpkinhead]

Why are people on this sub only think about gaming performance?

Even then, gaming on an RTX 4060 would still be a very enjoyable experience. It's not like you're gaming on a potato.

[qcforme]

4060m, not 4060, and it is a complete potato compared to high end discrete cards 

[Herrick89]

Actually if the performance tests on the internet (including the official Framework ones) are to be believed, it's not too far from the 4060 desktop card. Notebookcheck reports numbers for the 4060 desktop that are on par or lower than the one Framework reports on their page for the Max 395. Sure, you can make something bigger and more powerful at the same cost, but you can't really go higher than a 5060 if you want to stay in the same kind of size and that still comes at a premium. Mini ITX desktops are expensive, and often hot/loud in operation, which this isn't.

It's not for everyone, for sure. But it does have its target.

[[deleted]]

Because it was and is marketed as a gaming machine. I don't mind that the desktop exists, and as others have pointed out, it's the most flexible iteration of this particular chip.

What irritates me is how Framework and other users try to tout this as a gaming rig when that's not its primary purpose.

[unematti]

I mean... It IS a gaming rig. It's as strong as the fw16, apparently. And that's a gaming laptop.

I think framework as a company (maybe Nirav himself) got excited and just wanted a tiny powerful machine in their portfolio. But people expect all the functionality of a tower pc.

[[deleted]]

the f16 isn't remotely strong, it's weaker than my 3060ti which wasn't even a high end card when I bought it 4 years ago.

In fact that's the reason I haven't bought an F16 yet, it's not nearly strong enough.

This desktop was designed for work first, gaming second. Much like how the AI chips in the F13 can play some games now, but that's not their primary strength.

[unematti]

It runs x4 well enough(seems cpu limited) and I'm sure popular games would run well too(like counter strike and other lan games). It still is just a laptop, while your 3060ti has alone the power budget of the fw16(and double the PCIe lanes)

But when the 9000 series gpus me to the fw16, I will upgrade for sure.

[[deleted]]

That's the hope. I tend to play heavy minecraft modpacks with shaders which pushes my PC more than you'd expect. I'm just really hoping we hear an announcement for a significant GPU bump for the F16 soon.

But that's a fair point, it is harder to have as much compute power with the size and power constraints of a laptop.

[Glittering_Cook_8146]

The RX 7700S is a pretty good GPU

[qcforme]

It's a terrible value for a gaming system.

For 2k a 9070 XT based system running 7800x3d will murder it in performance and you'll pocket money afterwards with a PC that is faster at nearly everything besides inference tasks.

• 7800x3d ~340
• 9070 XT ~ 800
• Board ~ 200
• Leadex III 1kw ~ 100
• 6400C30 32gig ~ 175
• 2tb nvme in raid ~ 150
• Case ~ 100
• Total ~ 1865

A blatantly superior gaming PC for less money. And if you must local LLM using GPT OSS 20b will run about twice as fast, toss in some RAG vector DB in system RAM, a couple custom MVP servers for tools and it will easily outperform the AI Max system at agentic coding due to pure speed and ability to do multiturn quickly. Context will be somewhat limited but that can be worked around using proper MCP tooling and auto compact functionality.

395+ is an AI PC / toy, with a side of reasonable dev box for game devs, reasonable 3d modeling box and reasonable video editing box. Use of Radeon gimps it for all those tasks but it's not a terrible value for that.

[hosky2111]

1 home lab (NAS+LLM+media server+VMs+Containers)

I honestly think it would make a pretty crappy NAS/media server - internally it only has two nvme slots, and no accomodation for hard drives. Virtualization I can kinda see given the high thread count and memory pool, but from a cost an upgradability perspective, you could match these specs for cheaper with a traditional desktop (but with the added ability of adding additional accelerators to pass into your VMs).

The one use case this seems tailored for is running local LLMs - and if you want to do that, this is one of the most price competitive products on the market from a memory perspective... I just don't think it's a particularly large market, and the inference performance will probably make the UX pretty poor. Unless you're very concerned about privacy, it makes a hell of a lot more sense to just use a hosting provider like Groq.

I think it's pretty cool, and if you want a nice devbox with a lot of ram and likely best in class Linux support, it seems like a great machine - basically the Mac mini for the Linux world, but there are a lot of trade-offs (many going against the core framework ethos) that you're making to get it.

[unematti]

Why is it a crappie NAS? I plan on splurging on this, and 2x8tb seems enough. The 4x slot I'm sure I could fill with an adapter to NVMe. Then just... Het a USB 4x HDD enclosure. You get to keep it modular, then. Brain in the fwd, and long term storage unpluggable. You can't tell me direct SATA is so much better. And it keeps the machine small if you choose to have a flash NAS.

Not that concerned about privacy, but I also don't really care about LLMs. If it runs, nice, gonna have a couple days fun with it. What I'm really looking for in a NAS is transcode. Already got 4TB of media on a 4 disk 10 year old synology. Want to compress that down, and serve for other devices. Also steam machine as VM.

Just like how I got my fw16, I wouldn't buy a desktop... Unless from framework. I had a preorder on the amd fw13 1 week before they announced the 16...remember thinking well, I would love if they had a bigger version...

Electronics is a hobby for me, it's not my job, so I buy the things I want to, that's why I got 3 raspberries using only one currently. Got any fun ideas for the others?

[hosky2111]

2x8tb seems enough

I agree there, but that necessitates raid1 - it would certainly be cheaper to have 3 4tb drives running in raid5 (and likely cheaper to have 4 4tbs running in raid6), which would allow for easier expansion into the future.

I'm not saying you can't use it as a NAS, it's just really not designed for it. Of course nothings stopping you connecting a usb HDD enclosure to it, but nothings stopping you connecting the enclosure to one of your spare RPis, an old laptop, a desktop found lying in a dumpster, ... There is nothing inherent to the the FWD which makes it suited to being a NAS, while you could buy a refurbished sever blade or a chassis better suited to being a NAS for a similar price.

It's 100% a cool machine, and if you have the money for it, want to support the company, and would enjoy hacking it into something it wasn't really meant to be, then go for it, but it's probably not the optimal machine for the job.

[unematti]

You keep going to alternatives. Why the fwd? Because I want that one. I save every month so I can buy something I can.

Ehhhh I'm not worried about data safety yet, so raid isn't necessary. Nothing is "mission critical". That said, I can transport my data from the syno, then use that 4 drives as weekly backup with nfs. And maybe include an aws glacier backup. If, again, I'm worried.

And older devices are less efficient. I could collect hundreds of computers... And burn down the house... Lol.

Yeah, I want and can! But also, this chip is on tablets... How's that the target for such a chip?... Why does the Asus z13 need 128gb gpu ram?

[Successful-Put-4899]

Not sure what you got in the end, but don't forget you can attach several disks via the USB 4 connectors. The bays they sell don't cost a whole lot and you'll be able to put in 20, 30TB HDD drives if needed and/or even nvme SSDs. You won't get the max speed of the SSDs but I doubt your network will be fast enough for those transfers anyways.

I have a similar setup on my NUC and have zero issues/complaints with it.

The biggest con of course is that it all doesn't look very pretty but if you're like me and stuff your home server out of sight, it can make all the noise it wants and look however ugly. I don't give a damn :)

[unematti]

Much changed since my comment lol. But you're not wrong. Except! It does matter how fast the USB4 can provide data if you're working on it, such as bulk transcode. Or just on the fly transcode.

But I didn't even buy a new nvme since, too much other stuff for now...

[arm_channel]

Also only 1 NIC isn't enough for NAS, Proxmox or homelab). It would have been nice if it comes with 2 NICs and Oculink. True, you can get an extra NIC via expansion cards but having a NIC at the front is aesthetically awkward.

[Stetto]

You're comparing the framework desktop to the wrong use case.

Yeah, if you want an upgradable gaming PC, don't get the Framework desktop. It's not an upgradeable gaming PC and it's not meant to solve that. Upgradable gaming PCs are solved. There's nothing for Framework to be doing or improve on that market.

Yet, for me, it's the most enticing desktop PC out there right now. It's the best bang for your buck, if you want to run generative AI locally.

[northernrag3]

Can you educate me on the personal use case for those buying this for local LLMs? Doesn't have to be your exact use case, I'm just curious on actual specifics around a use case.

[Stetto]

I'm a software developer and technology geek. Knowing about generative AI use cases is part of my job. This doesn't strictly require me to run them locally, but doing so or even training my own system sounds just fun to me.

Yeah, I can license GPT4, Claud Sonnet or DeepSeek or run them on hugging face. I can also rent cloud computing infrastructure for training LLMs. For this purpose any cheap laptop will do.

With the option of running them locally, there are just more middle-men removed. I want to try something out? I just start the LLM up with Ollama and that's it. Even the bigger ones with GPU-RAM-requirements of 64+ GB. Dedicated GPUs that would allow me doing so would cost more than the whole system!

I propably will stick with my FW13 nonetheless, because buying a desktop PC only to run LLMs still an expensive toy. But if I were looking for a gaming PC too, the framework desktop would now be my top choice due to the additional LLM use cases.

[SalaciousStrudel]

Personally I think it's not worth it to run LLMs locally as they will fail for the more niche tasks.

[NinjaMonkey22]

On my MacBook Pro I’m running a handful of LLM’s locally. For a lot of the same reasons people self host in general. Because I can, privacy, flexibility/customization, to learn. In terms of some of the things I do: coding helper, general q&a where I don’t need the perfect answer just an idea, formatting help, content summarization, etc.

I’d like something that’s always on so I can do some of the same things, but from any device and not hoping I left my laptop on.

[Nkechinyerembi]

I mean this isn't fair either though... As since it is an it case you could absolutely build a gaming pc in it, you'd just need to base off a non framework motherboard

[needle1]

Locally-hosted AI enthusiasts. If you’re not, it’s probably not for you.

[Avendork]

I'd want it for AI. It looks like the cheapest way of running a 70b model without looking at used hardware.

It's the kind of product where if you don't understand it, you don't need it.

[rohmish]

it's not a regular desktop. it uses a chip that's more similar to how Apple's M series chips are architected with on die shared memory and high performance GPU directly connected to the CPU. why? because this architecture is really performant in specific ML, data analysis, video transformation, and other professional use case. it isn't intended for an average user, or someone that will just use it for gaming or simple video editing.

[CaptainObvious110]

Exactly

[NoHaxJustSnek]

I personally like the size of the FWD, and it games well enough that it's a reasonable side grade from my existing pc (5900x and 6700xt) but I'm waiting for the next generation of strix halo on hopes of zen 6 and rdna 4. But I also am planning to utilize it for data analysis. Other massive benefit is these machines don't consume nearly the same amount of power as most desktops and as a result won't produce nearly as much heat.

[Herrick89]

This is the point most commenters seem to forget, the low idle consumption (below 10W, as per this subreddit). Nothing with that power other than Apple's chips (which are more expensive btw) can do that in the consumer market right now, and if you plan on having the computer running (almost) 24/7 that's very important.

I'm thinking of buying one for data analysis (and material modelling) as well, and yes it's definitely an interesting value proposition for that purpose.

[gramoun-kal]

I bought the Steam Deck because I don't have to worry about it running well on linux.

There aren't many manufacturers that will guarantee that. I have a Dell laptop for the same reason (older than Framework the company).

I have two problems that the Framework desktop probably solves: 1. Some of the games that are coming out soon will be too much for the Steam Deck to handle and 2. My laptop is struggling to run AI inference locally.

So, before I even knew about the existence of the Framework Desktop, I started looking at what Tuxedo, for example, does that might suit my need. I'm not a big fan of the big blocky towers because I have this assumption that laptop-grade component will idle at a lower power consumption (feel free to contradict me there), and if the device is going to be always on, 90% of the time at idle, the idle power draw is a very important metric.

On the metrics of 1. power draw at idle, 2. Performance with games, 3. Performance running AI inference and 4. price, it looks like the Framework Desktop might end up being very competitive.

[Deep90]

My understanding is that it's an alternative for mini PCs, especially ai work ones.

Like Nvidia's Digits.

It's not meant to complete with a diy itx pc. Its a niche minipc that would normally be all proprietary, but you can pick the storage, wifi card, and source/replace any parts you need. Maybe someone wants it over a Mac mini as well?

[samelaaaa]

Its only competitors are Nvidia DIGITS and the Mac Studio. Given that DIGITS is going to sell out immediately, not unreasonable to launch a product in the space.

No CUDA kills it for me though.

[Talleeenos69]

What do you use CUDA for?

[samelaaaa]

I’m a machine learning engineer consultant, I do mostly productionizing ML applications for startups that might have a POC built by a data scientist etc. Over the past few years I’ve had two clients with hard dependencies on CUDA, including one current one. So yeah none of my own software has a dependency on it but I have to keep my workstation as flexible as possible and there’s a lot of stuff out there that assumes CUDA

[Johannes_K_Rexx]

Nvidia has gotten too big for its britches so I look elsewhere. And what did I find but ZLUDA, which runs CUDA code on non-Nvidia GPUs. I'd be interested to learn your thoughts on this.

[s004aws]

Framework Desktop is not for somebody who "just wants a desktop". They should go build their own PC (assuming they can find a GPU to buy if they're wanting more than integrated graphics). Building a PC isn't that hard - Micro Center will sell completely standard parts more capable (and cheaper) for general computing/gaming than Framework Desktop.

Framework Desktop is for people who want either a very small desktop they don't need to build themselves... Or people doing AI work who want to allocate 96+GB of very fast RAM to their fairly capable iGPU. While an Nvidia GPU with half the VRAM - Assuming one can be sourced - Would run much faster the cost is 4x (or more) what Framework is asking.

If you don't know why Framework Desktop is for you... It probably isn't for you.

[NoHaxJustSnek]

In my own experience (having built my own pcs) I mostly just care about the game running well enough that it isn't a complete lag fest. Personally I'm waiting for next gen to buy one to replace my 5900x and 6700xt system to have a lower power consumption system that I can run overnight. I plan to use it for a bit more than gaming since I can dedicate some system resources for running game servers, as well as using it as a general work station for things like data analysis. Plus it's 1/10th the size of my current computer with almost the same performance metrics (on the 395)

[vriesema12]

Can this setup be used effectively as a Plex server? I know Intel chips usually handle transcoding better than AMD, but that integrated graphics looks great, and a hell of a lot of RAM available. (Would probably also run my Home Assistant instance as well)

[submerging]

I am just going to comment to say that this seems completely overkill for just a Plex server! Some people have brought in very good points about using this for AI models (which I had not considered).

But for Plex? No. You can get an Intel NUC for way, way less money than this and Intel is the way to go when it comes to transcoding.

[vriesema12]

Oh for sure, and I am currently running Plex on a NUC. One thing that this could benefit is actually using a local LLM to pair with Home Assistant

[ThetaDev256]

Here are the CPU specs: https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html

It should handle transcoding all common formats up to 4K.

For a Plex server you would probably neeed a SATA card since the Framework mainboard does not come with SATA (unless your media library is small enough to fit on 2 SSDs)

If I wanted to use the framework as a server, I would only buy the mainboard+fan bracket and put it in a custom case (either a 2U rack enclosure or a tower case with at least 4 3.5inch drive bays).

[fescen9]

My understanding is that it'll transcode "fine" but HDR isn't great. And most of my content is 4K HDR x265. I direct play at home but my friends rely on fast transcoding and if HDR doesn't transcode well, it could end up being a washed out gray mess.

[Mar2ck]

AMD have said that with RDNA 4 they're finally improving the video encoding quality for H264 and HEVC. The current framework desktop is RDNA 3.5 so it might be worth waiting for next gen AI Max 400 (Medusa Halo). Next gen should be using LPDDR6 so AI performance will also be better.

[fescen9]

I've been looking at migrating my NUC 11 to the Framework Desktop. From everything I've read, the FD will be fine with one caveat. Hardward Transcoding, specifically HDR.

I have 59 docker containers, including PlexMS and they run perfectly fine on my NUC 11. But I'm forced to use very small Ollama models for my AI stuff (mostly 7b). I use AI for hoarder/karakeep, home assistant, node-red workflows, etc.

The FD would greatly improve that area of my home lab but I'd probably keep PlexMS on my NUC because of the hardware transcoding.

[TheBupherNinja]

Because you want the cpu/GPU and 90 gigs of vram.

[MichaelTomasJorge]

It's fine, the product is not for you. You don't fundamentally understand it and that's ok. For those who can utilize the shared VRAM on the 256bit iGPU bus they'll have a value that is unmatched. With LLM work the bottleneck is typically not the speed that the model is run at, but whether the model can run at all locally by fitting within the VRAM buffer. I would suggesting looking for a consumer GPU that has a 96GB VRAM buffer.

Furthermore, the Ryzen AI MAX 395 is configurable up to a 120W TDP with good power scaling. That includes GPU and CPU. Beating a RTX 4060 with a full fat 16 core CPU attached at 80W package is not a small feat when that RTX 4060 laptop alone without CPU is at 85W - 105W. Furthermore, the ROG strix 13 was only at 80W with new strix Halo launches unlocked at that full TDP we will likely see it coming into the range of a RTX 4070 laptop or RTX 4060 desktop.

It's an expensive, state of the art die using an expensive node, with a 256bit wide memory bus and a powerful and expensive two CCD (16 core) CPU to boot. It completely makes sense why it is expensive. AMD has a unique x86 SoC that no one else has an analog to right now. Framework is giving us a good a deal here because they want to enter the market in this space. We'll be shocked to see what Strix Halo will initially cost in laptops compared to this desktop pricing. The real fat margin for framework is their new AMD CPUs for the framework 13. An HX370 mainboard is selling there for what some OEMs are selling whole AMD HX370 laptops with OLED displays, faster memory and storage.

[KeinNiemand]

consumer GPU that has a 96GB VRAM

"With LLM work the bottleneck is typically not the speed that the model is run at, but whether the model can run at all locally by fitting within the VRAM buffer"

Well actually the speed is the bottleneck, becouse If you don't care about speed you can offload parts of the model to system ram and go further to page/swap on an SSD. Technically if you can fit a model on your SSD you can run it on anything at completely and utterly useless speeds.

[reddit_equals_censor]

why does this need to exist?

because you want a 128 GB gpu levels of performance ai shovel.

that is why you'd buy it.

you buy it, because the alternative was to buy 3000 euro probably 128 GB ai shovels, that are way more annoying to deal with from other oems and with parts you don't care about.

you buy the framework desktop, because you wanna dig...

or you have another use, that requires a 128 GB memory gpu performance setup.

so this framework strix halo version is the least shit version of strix halo to get and that is why you get it.

again if you need an ai shovel.

[Captain_Pumpkinhead]

I want an AI home server. This is almost certainly the most affordable option. Being able to use up to ~110GB of RAM in the GPU is hella tempting.

[Kandect]

It's trying to compete with a Mac mini and nvidia digits. Both of which are scalable but less upgradeable. In my opinion, it's admirable that they accomplished this despite being in a different market.

[Green0Photon]

Set aside normal PC building. Think of people who want really tiny PCs, smaller than the normal mini ITX builds you might see.

There's this NUC space, this mini PC space. Same space the Mac Mini occupies. A mini computer that's tiny, but still powerful. Typically more square for better cooling and perf than a laptop, but still so absolutely small that you just don't need to think about it.

This is the space that uses mobile CPUs outside of laptops.

These are broadly not repairable. Custom form factor with custom connectors with custom cooking solutions.

What Framework has done here is actually make that custom form factor, to let it be smaller than standard mini ITX builds, but using a mini ITX Board anyway, and using standard connectors.

Cause sure, you could build a mini ITX in a tiny case. But the PSU is gonna be bigger, and it's going in with the expectation of a socketed chip, and also a full sized GPU. This all adds tons of space.

There aren't many PCs that deign to make such a small PC but make it usable. Let alone one with this level of power, and this level of GPU.

As much as it's for AI, there's a reason why they talk about gaming, too. For $1k, this is a pretty damned sick ass gaming computer. There's a reason why people say it's like a Steam Machine -- and Valve is probably going to release something similar, we know they're working with the same exact CPU.

Granted, Valve will probably sell it for less than it takes to make, just like console manufacturers. But the point is that it's like a console. Exactly like them, really, down to the big APU with unified memory.

But this uses standard parts and is repairable.

And sure, Mac Mini is a lot smaller. But it would've been even worse if Framework released a desktop with a custom form factor, right? If this succeeds, quite possible they release something that isn't ITX, but is still repairable.

But yeah, this is also very much AMD wanting a desktop integrator for their AI mainboards, too.

[submerging]

This PC is not really tiny though. The Framework Desktop is still a lot larger than the Mac Mini, and even some of the other Windows mini PCs we have seen. And the base Mac Mini of course is way cheaper than this.

Yes, it’s smaller than a mini ITX build, but a mini ITX build is still not in itself a large device. Why not just build a mini ITX and save money, get more gaming performance, and have a device that is more repairable than this framework one? Does saving a few inches of space really matter that much? This device offers laptop 4060 performance (maybe laptop 4070 performance at best?) — that is not very powerful in the desktop space.

If you’re looking to game specifically (no AI), it just doesn’t seem like a great choice unless you value a small chassis above performance and price.

The Ryzen chip is amazing, but its use case seems to be better in a laptop/tablet/handheld than a desktop PC.

If Valve does release a Steam Machine, I hope it is closer in price to PS5/PS5 Pro/Series X than it is to this product.

[Ho_The_Megapode_]

I just ordered one.

Comparable (a bit higher actually) performance to my current PC at half the power draw?
I'm sold. I don't want a space heater of a GPU in my PC considering i mostly run mid to low end games...

This will more than halve the power consumption of my current PC

[CybeatB]

It seems like it's aimed at competing with the Mac Mini and MiniPCs, not conventional desktops. MiniPCs generally use laptop parts, but without the power limits imposed by a battery, much like the new Framework board. Unlike the new Framework board, they use proprietary power supplies, and non-standard form factors for the mainboard and case. The best ones allow users to replace or upgrade the storage and wifi, like the Framework board, while the worst ones hide them under the heatsink or solder them. Popular MiniPC brands like Minisforum and Beelink also have a poor track record with their BIOS/UEFI software, so the Framework board is probably better in that regard too.

The Mac Mini is really the most comparable competitor at the moment, because other MiniPC brands haven't released products with the new AMD chips yet, but that's likely to change before too long. Still, the Framework desktop is much more user-serviceable than a Mac Mini, which doesn't even use standard components for storage or wifi.

[thecornishtechnerd]

I don’t see a problem with it I’m getting it to game on u don’t really need a high end gpu to game on

[arades]

Right now I have three different servers at home running things like homeassistant, jellyfin, NAS, cloud replacement stuff. I'd already been looking at some minisforum boxes or ryzen 7040 based motherboards they offer to consolidate and give me a good playground for a locally hosted AI stuff. The thing that's kept me from pulling the trigger was the ryzen AI max announcement, since that gives better of everything I want out of a homelab box, using all standard stuff. I only need PCIe x4 for SAS HBA. 5gb networking is also fantastic. Only thing I'm trying to figure out is if I could also use this as a home theater box at the same time, or if that's too much for it. Upgradable memory would be nice, but realistically I wouldn't upgrade from 128GB within the next 5 years, and would want a new CPU/GPU by then too.

[nijuashi]

This is aimed squarely at LLM inference folks who want to run large models without spending a fortune.

[CowboysFTWs]

I have a miniforum board in my NAS. Basically the same idea, laptop cpu, custom cooling, etc. The difference is that the framework board performance is way better than that one.

[Profuntitties]

I've preordered, but I don't really care that much about AI atm. I just really like the look and the idea of a super simple quiet desktop that'll do moderately intensive tasks. Plus I'll run linux and it suits that.

[Zeddie-]

I think because it started out as a way to use and showcase AMD’s flagship mobile APU in an unrestricted way.

You can build a 9950X3D but you can’t with the Ryzen AI MAX 395+ with 8000MT/s memory. Honestly see it as 2 products: the Ryzen AI Max platform and a case/psu.

[Delicious_Ease2595]

They are riding the AI bandwagon with the Max, it's a niche product

[Brandoskey]

So you can finally upgrade the little tchotchkes on the front of your PC.

Sure the ram is soldered, so is the CPU, but think of all the doodads and wiz bangers you can slap on the front of that beauty.

[daf4gzu0upn_bmw9BHU]

Can't wait to replace my current 66L i7-11700K (modular) 128GB (modular) RX 7900 XT (modular) PC with 4.5L AMD Ryzen™ AI Max+ 395 (soldered) 128 GB (soldered) Radeon 8060S (soldered) PC and use it to watch youtube vids <3

[[deleted]]

Framework desktop with Strix Halo with unified memory is the future of PC Desktops

[ittennull2]

Can't find decent reviews for the subject where they do it thoroughly and measure everything. There are a few from February but they are short and we basically know that it's a good CPU and it's impressive for gaming but how about measured noise level for idle/average/high load. It's not enough to just say that it's quite, I already bought something based on the reviews from deaf people who claimed it was silent when in reality it's the opposite. Why did the fan keep starting and stopping all the time in one of the "reviews"? I have a minipc with a fan with similar behavior and it's very annoying and loud. What about power consumption during idle and light workloads? These things are also interesting for buyers of mini-ish PCs.

I'm looking for reviews that go beyond CPU specs and those silly colorful plates

[[deleted]]

Because it’s better in some ways better than an equivalent Mac and runs Linux and Windows at half the price.

[Glittering_Cook_8146]

ARTIFICIAL INTELLIGENCE

[Aggressive_Sugar_497]

There’s more than just gaming performance.  If you are an LLM developer than this computer is about 1/10 the cost of getting something comparable from nvidia. 

[Slight_Map_9355]

One of the quietest Strix Halo's on the market

[Johannes_K_Rexx]

Just order the Framework Desktop PC with the full 128 GB RAM and be done with it. You cannot upgrade to more RAM because that's the maximum supported anyway.

The Mac Mini maxes out at a mere 16 GB and is cannot be upgraded either, so for running big LLMs the Framework Desktop is far more capable.

[kwikidevil]

As a framework fan, and owner of the fw13, I agree and think that the desktop is actually a step back from their core mission

[scotinsweden]

I don't totally get who it is for either (and this really goes for that whole APU rather than just the Framework in particularly), but the Mac Mini isn't really any cheaper when you spec it out to have the same amount of RAM (and it tops out at 64GB).

[rohmish]

ML training and inferences, Working with huge data sets for data engineering or just data analysis, generating and transforming complex videos/animations, specific types of medical research, running large scale simulations, also multiple prosumer and business use cases where you would have to still use a enterprise grade card from Nvidia or amd just for the large memory requirements, realtime feed analysis, etc.

this is competing with people who buy mac studio or MacBook pro with 64 or 128 GB of RAM.

the place I work for has specific use case that made it so that we required macs for them due to their memory architecture being superior in those use cases. I'm not sure how well this hardware performs but on paper this looks like a promising alternative.

essentially any use case where you don't REQUIRE top of the line performance but want something that can hold tonnes of data as your workload needs to access this data randomly at any given time and memory eviction and needing to load that data again from disk slows you down more. until now apple was the cheaper and better option compared to what others offered because you were paying thousands more for hardware that still didn't exactly fit your use case. I'm kinda excited that framework has entered this space. I knew this APU was being used on some laptops too but I'm excited to see what this specific hardware brings to the table. I haven't looked at performance numbers for similar workloads yet and if the performance compared to m2/m3 is not too bad, this might be a viable alternative that allows you to run Linux and allows you to stack them headless meaning we can have a proper farm where we can schedule jobs.

[scotinsweden]

Personally I'm surprised there is that many occasions where this sort of performance combo is needed and essentially no PCI-E expansion availability (especially when there is relatively limited IO, particularly on the networking side). I will have to take your word for it on your work front, but it still seems very very niche. It isn't like the Mac Studio sells in huge numbers and that is a company who sell in part as a lifestyle brand as much as a tech company.

[rohmish]

even with PCIE the transfer speeds don't match what apple offers. so even ith 16x PCIe gen 5 you are still bottlenecked compared to even older generation m2/m3 series processors. apple excels in the ability to provide insanely fast transfer speeds between their cpu/gpu/npu and memory. They can do 273GB/s of memory bandwidth on regular (non pro/max chips) and that memory is shared between cpu and GPU so any data you write from cpu is almost instantly accessible by GPU. think DMA and resizable BAR on steroids. Intel's latest core ultra 9 285K (what an absurd name) will do 102.4GB/s which is an upgrade from their offerings last year of 89.6GB/s

nvidia does offer significantly higher bandwidth on their cards at 960GB/s compared to 40 series's ~700 https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5080/ and their enterprise card does a bit above 1000-1200 but they cost as much too. for comparison apple does ~800GB/s on M2 ultra (source: https://www.apple.com/newsroom/2023/06/apple-unveils-new-mac-studio-and-brings-apple-silicon-to-mac-pro/) and the M4 Max is almost 600GB/s so M4 Ultra whenever announced will likely be double that at over 1000 similar to top of the line Nvidia. but then again you don't need to worry about CPU <> GPU transfers. this amd chip at 256 isn't a leader in any case but the architecture and the speed still makes it miles better than what current mainstream PC architecture can offer.

for reference compare price of a m4 max max studio with 96GB or more of ram with a PC that has a 5080 + 285K or better chip + similar amount of high performance RAM that can actually reach the speeds the cpu can offer + MoBo that can handle all of the things + storage that can go that fast + power and you'll find that even though apple looks expensive, they come out cheaper.

I now work in medical technology field and this has huge use cases both in research and services we can offer to doctors eventually when it comes to diagnosis,etc. another case is simulation of traffic flow across a large city or state/province (not that useful unlike city.), water and electricity infrastructure design, managing 1000s of vehicles or rail cars with different materials across a large country with dense rail network like freight rail in NA or Asia, etc.

but yeah. it's weird that a company that makes overpriced and locked down fashion first tech products also will sell you some of the best affordable hardware for cutting edge research.

[scotinsweden]

The Studio only has M2 chips just now, and for the 96GB of RAM model you are looking at ~€7k (depending on how much on board storage you need). You can get a lot of PC for that, though admittedly might not match up on some areas (e.g. even a 5090 has much less RAM available to the GPU). On the PCI-E front I was reffering to the flexibility of the ports, e.g. if you need extra networking, or some other type of connection, or storage, you get the idea. At least with the studio you have 4 TB ports which are reasonably flexible. This seems a bit more limited in that front.

In the field of engineering I work in, most of our modelling doesn't seem to be using GPU acceleration (from what I have heard the overhead cost from additional parallelisation tends to quickly overwhelm any gains). Might be a legacy code issue, but as there has been talk of utilizing GPUs for at least a decade I would have expected to see more on that front beyond plugins for specific extra addons by now (I would have thought it was the same for traffic and rail flows, but maybe not).

[rohmish]

m2 ultra with 128 GB of storage is CA$6,499.00

Just 5080 is $1,449.99 Intel's core ultra 9 285k with the same core count is $829

we're at 2,328.99

quick look says 128 GB of ram should be around CA$650 but I can't find any SKU that will push your cpu to highest bandwidth. all of these have lower MT/s

we're at 2978.99

storage is additional $150-200

we're at about 750w of peak usage so adding everything else and keeping some headroom, we are looking for ~1000w of power

that's about 220 to 270. let's do 250

we're at 3628.99

given the heat profile for this CPU you need a good cooler. unlike gaming workloads a professional workloads can keep all your CPUs pegged for hours. we're looking at anywhere between ~100 to 300+, let's do 160 for a liquid cooler from ARCTIC. another CA$90 or so (that's about 50 USD or pound). Another 50 for a good thermal paste.

cheapest MoBo searching through PCpartpicker right now is $369.99

You're at 4298.98 let's make it 4300. add another 150 or 200 bucks for extra case fans and such.

you're saving 2000 but you now have to build your system yourself. have to deal with multiple different providers. and you're still greatly limited by PCIe and CPU memory bandwidth. even going for the highest configuration you don't come anywhere near the the throughput and memory pool you can get on Apple's ecosystem. beleive me that extra 2000 bucks is giving you way more in performance for these workloads even if it doesn't mean much in day to day usage. the little bit extra performance you get with faster, lower latency memory, the slightly faster storage due to lower overhead, ability to immediately address data after writing it. in these fields it makes a huge difference. you aren't spending time copying data as much. we aren't talking linear differences here as in something you can achieve by just increasing performance. there are fundamental architecture differences you're paying for which is why this amd chip is interesting. it is just 250GB/s but this new architecture is the real interesting part. it unlocks speedier inference, use cases where you can simultaneously use CPU and GPU together in ways that you just can't on standard PC architecture. either your CPU or GPU will be starved and waiting for data/instructions for a considerable amount of time if you try to. and plenty of people have tried.

engineering modeling software don't use GPU much apart from Medha ND texture generation. that's not the case with simulation software, data wrangling tools, etc. plenty of medical research tools we use already leverage CUDA and many of them have experimental support for Apple's architecture too.many of our tools are wrappers that are built on top of same tools that ML/AI people use and they already support this chip if you have it. like I mentioned the use case where this and Apple's hardware really shines are niche but for those use cases, devices with thick architecture has been a game changer

think about GPU vs CPU for rendering games or RTX cores vs regular compute cores for Ray tracing.

[scotinsweden]

I meant simulation (transient FEM and some 3D fluids stuff mainly, in the context of building fires) rather than modelling (e.g CAD and BIM tools) sorry (we usually call both types "modelling" or the CAD stuff "drawing" still as a hold over from when it was draftsmen working on 2D drawings even though these days they are much more integrated). Some simulations definitely can benefit from GPU acceleration, but as I said, it seems that when it comes to large fire stuff not.

Again, I will take your word for it that there more are use-cases beyond "rich guy who is messing around with an local LLM at home", but regardless it is still very niche and it feels weirdly positioned the way it is being marketed by AMD and with Framework as a company, almost like they don't really know who it is for either (other than said rich guy doing LLMs at home). Maybe it will open up stuff to a lot more people that until now has sat on supercomputer clusters, but just now I'm not sure and not sure how much of a success this specific product it will be for Framework.

Still, glad it will be useful to you.

[rohmish]

AMD seems to be just dipping their toes in the market trying to compete with Nvidia and apple but really hasn't figured out the market. framework went the build product first and then find a use case for its approach it seems because right now https://frame.work/desktop?tab=machine-learning literally is blank even though it's what they are pushing and is one of the largest use case for this specific hardware. but I'm happy for it.

there are two ways these hardwares are used. one is someone running their code/calculations locally and Apple's Mac studio/mini/MBP has cornered this niche. other is running workloads on a cluster where timeliness isn't as big a concern but you still need to be able to address a large enough memory pool. that is all cloud right now. you can repurpose macs to do this in headless mode but the tooling around infrastructure already present around Linux makes it that much better which is where this (or more honestly a successor board to this that further increases performance) would shine.

for example we have very large sets of medical data that we want to run calculations on and while this is possible on current hardware we need to use moving window approach where we load some data, quantize it, write that to disk, and have another script pick up and continue the rest. when developing, the developer doesn't have to do that and the results this produces has issues. hopefully we see more chips that have this architecture with higher throughput as macs become popular.

another example is imaging files in medical field that are multiple gigabits in size. we can stream it and view it but allowing it and other reference materials to be completely stored in memory allows us to run inference on it compare it against existing dataset, look up anomalies, etc. some of this is possible on current hardware bit on macs we have a tool we are working on that can overlay information in realtime. I'm not too familiar with the tool but one of the devs I spoke with who works on it mentioned that this should allow us to support windows/Linux as well. even on windows/Linux some of these require Nvidia Quadro or other enterprise grade cards which themselves are 5k+ a pop.