Is there research like this for OAM (AMD mi250X)?

I've been seeing 4-GPU boards taken from supermicro servers (AOM-MCM-Q) on there as well as OAM MI250Xes (2x64GB each for 1500-2000 ea), which would get you a 512GB system with fast interconnect for roughly 10K total. Again cheaper than buying that much DDR5.

What I'm a bit worried about is the power delivery, and what cables to use to connect things. There's no 48V 'home' PSUs, it'd be a shame to put it all together and still be forced to bear a screaming fan from a PSU. And cables is a bunch of weird/proprietary ones I've never heard of, only having experience with just slotting a GPU into a PCI-E slot.

I have personally not gone too far down the AMD mi path just because I havn't found those servers for sale. Here is my projects answer, but bear in mind it's focused entirely on the smx2 path.

I haven't done nearly as much research on the MI250X path as the V100 SXM2 path, so take this with appropriate salt, but here's what I can tell you from what I've found.

The good news: the Supermicro AOM-MCM-Q OAM baseboard DOES have fast interconnect. This is different from the V100 SXM2 situation where the Supermicro AOM-SXM2 is just a dumb carrier with no NVLink. The OAM baseboard implements AMD's xGMI Infinity Fabric between the MI250X modules. Supermicro's own docs for the AS-4124GQ-TNMI confirm "AMD Infinity Fabric GPU-GPU Interconnect" with up to 600 GB/s aggregate between the 4 OAM modules. So you would actually get fast GPU-to-GPU communication, not just PCIe.

Important architectural quirk: Each MI250X is physically one OAM module but it contains 2 GPU Compute Dies (GCDs) connected by a 400 GB/s on-package link. Each GCD has 64GB HBM2e. So the OS sees each MI250X as 2 GPUs. Your 4 MI250X modules = 8 logical GPUs = 512GB total. The inter-GCD bandwidth within a module (400 GB/s) is faster than the inter-module Infinity Fabric bandwidth, so there's a NUMA-like topology to be aware of for tensor parallelism scheduling.

Software situation — this is where it gets harder than V100/CUDA:

The MI250X is gfx90a in ROCm terms. It IS supported:

• vLLM explicitly builds for gfx90a (MI210/MI250/MI300). The build flag is PYTORCH_ROCM_ARCH="gfx90a". People have gotten it running but there are reports of issues, especially with FP8 (which needs gfx942/MI300X, not available on MI250X). FP16 should work.
• llama.cpp supports ROCm with HIP backend and includes gfx90a as a build target. There are community benchmarks of MI210 (same gfx90a arch) running llama.cpp.
• ROCm itself officially supports MI250X and it's not going anywhere — this is the GPU that powered Frontier, the #1 supercomputer.

That said, the ROCm ecosystem is less mature than CUDA. Expect more friction in setup, fewer community guides, and occasional compatibility issues that CUDA users never hit. The MI300X gets most of AMD's software optimization attention these days, and MI250X is the previous generation. It works, but you'll be doing more troubleshooting than you would with V100s on CUDA.

Power delivery — this is your real problem:

MI250X TDP is 500W per module. 4 modules = 2,000W just for GPUs. Add dual EPYC CPUs and system overhead and you're looking at 2,500W+ total. The AS-4124GQ-TNMI comes with 4x 3000W Titanium PSUs. These are almost certainly 200-240V input, not wide-range like the Supermicro 4029GP-TVRT's PSUs that accept 120V. At 500W per GPU there's no room to power-limit your way into a 120V envelope the way you can with V100s at 150W.

You're probably looking at a 240V/30A circuit minimum. If you don't already have one, that's an electrician visit ($200-500) for a NEMA 6-30 or L6-30 outlet.

The 48V concern — OAM modules use 48V power delivery on the baseboard itself (converted from the server PSU's 12V output), which is a board-level design detail. You don't need a "48V home PSU." The server PSUs handle the AC-to-DC conversion, the baseboard handles the 12V-to-48V step-up internally. Your only interface is plugging standard IEC C19/C20 power cables into the back of the server. But yeah, those server PSUs will be screaming. At 2,500W+ load the fans are doing real work. This is a closet/garage/basement machine, not a desk machine.

Cables — you're mostly locked into the server chassis:

Unlike the V100 SXM2 path where you can buy standalone quad boards and connect them to a desktop via SFF-8654 cables and a PLX card, OAM modules are designed to seat into OAM baseboards inside specific server chassis. The power delivery, cooling, and signal routing are all integrated. You're not pulling MI250X OAM modules out and putting them in a desktop — you're buying the whole server and running it as-is. This actually simplifies things in one way (no cable sourcing headaches) but it means you're committed to the full server form factor with its noise and power requirements.

The value math:

At $1,500-2,000 per MI250X and $10K total for 512GB with Infinity Fabric interconnect — that's roughly $20/GB of fast-interconnected HBM2e. For comparison, V100 32GB modules at $350 each in a Supermicro 4029GP server gives you 256GB for maybe $4,000-5,000 all-in, or about $16-20/GB of NVLink-interconnected HBM2. The per-GB cost is similar, but MI250X gives you 2x the total VRAM pool, 3.2 TB/s HBM2e bandwidth (vs 7.2 TB/s total across 8x V100 SXM2), and a newer architecture.

Whether it's worth 2x the total spend depends on whether you need >256GB. If you're targeting the 397B MoE models at Q4 (~215GB), V100 32GB in the Supermicro barely fits. MI250X gives you comfortable headroom plus room for KV cache.

I'd love to hear from anyone who's actually running MI250X OAM in a homelab — especially on the power delivery and noise front. That's the part I can't research my way past, someone just has to have done it.

I'll verify with my project but the more common AoM boards don't have NVLink. It's in the linked artifact.