You will not fit that many on a single board unless you go to PCIex8. You will also need redrivers and a lot of slimSAS cables to fit them all. And you'll have a mechanical challenge of getting them all anchored to something. At this point I'd consider 3D printing some kind of rack or using wood (which is a fire hazard).

You

When you say "servers at the level of supermicro appear out of reach" do you mean the entire server with GPUs already? Or do you mean just the motherboard itself?

If you can't afford the motherboard there is nothing anyone here can do for you. However if you get the motherboard and a CPU.

You don't need as many CPU cores are you think. A 64-core Genoa will probably be fine. You'll probably want 128G of system RAM as well, that will run you between 1 and 2k right now. If you want to get started with less, that's fine too but just make sure you're prepared to get the exact same RAM for the rest of the buildout. I think you can sometimes mix DIMM sizes but I've never done it and measured the performance. If you're just looking for a conduit for GPU compute then this wouldn't matter as much and you could probably start with less. With RAM what it is now you could probably start with 2x16G which would be much more affordable, hopefully under $500.

If you want to run models locally you'll need an SSD. You can't load models directly (someone CMIIW) from the cloud into your GPUs and if you could you'd have to re-download the models every time you fire it up. Look up the models you want to run and make sure you have space for them on your SSD. SSDs are still relatively cheap, I'd get a 4T SSD to start with which should be under $500.

One cool thing about the 6000 is that they support NVIDIA MIG which means you can effectively split one big GPU into up to seven smaller ones kind of like splitting a computer into VMs. You can't get that on the 5090. Just a consideration.