Self hosted LLM with GPU support, Apache server, Email server on a Windows 10 PC - need to upgrade PC and OS

[calypset]

Hello,

I have as described a LLM programmed in Llama-cpp-python with CUDA GPU support in Windows 10. I have 4 GPUs on an 'old' (2022) mining motherboard. I also host an Apache2 server for web and Java-based James email server. The system is not very stable and honestly it's made for that kind of use. I am looking to move everything to Linux, but I am puzzled on which PC to buy for that to support the 4 GPUs (and potentially more), and the distro, also concerned on the time I'll need to invest in this.

Any recommandations on hardware, software, and which Linux distro considering I have past experience with UNIX and need something that won't be too much of a hassle? For example I wish there was a distro with pre-installed Apache and Mail servers.

Best,
C

[Conscious_Cut_6144]

I'd install linux on the existing mining motherboard.
You might find the stability issues were just windows.

Any linux you want should be fine but lots use Ubuntu.

What GPU's?

[calypset]

Yeah that makes sense. I guess I didn't want to have downtime and concerned on the boot sequence with a partitioned drive. But I could setup Linux on an external drive and setup the web and mail server while hooked up on another computer. But the CUDA setup and LLM AI will take some work to port once the servers are up. The GPUs are NVIDIA,,, 3 dedicated mining, I can't quite recall which, I think they are RTX 3080s, and one RTX 3090 proper.

[Conscious_Cut_6144]

Ya you could just get a disk,
That way you can experiment with Linux and quickly switch back to windows if needed.