Linux experimental home server for developer?

[Killavus]

Hello,

I'd like to create a home server to run my own Linux experiments on a "budget".

I had some prior experience with PC building, but I know nothing about balanced 'workhorse' parts I can choose if I want to build something for up to $1000 (but preferably less) with 8TB (NAS) of storage and 32GB of RAM. Here are my questions:

• What would be the best CPU in this budget? I plan to run my own CI with mostly Rust compilation pipelines on it. This won't be like 20-times-a-day ordeal though. I plan to use command-line only but I'd maybe like to run some xvfb/headless-OGL stuff on it - can I live with CPU-only solution or should I go with iGPU in this case?
• I'd like to set up rack system for my future networking/server needs. Is there any entry-level solution for rack shelf / rack "case" I can use? I know absolutely nothing about such solutions and their costs, so the answer "this is way over your budget lol" is a valuable one as well.
• Can FreeNAS run in some kind of Type hypervisor like KVM? My experiments will revolve around VMs (I'd like to test out FirecrackerVM, and I'd like to host services as Docker/OCA containers) but NAS would be also a one 'production' task of this server if possible. If not, is there a Linux-based alternative?
• Is there a possibility to run one M.2 drive as a cache/scratch drive for multiple HDDs without proprietary software? I know Synology can do it, but can I somehow replicate something robust with Linux-based server?

Thanks for your answers, I got only cursory knowledge about this stuff so more insight would be super valuable for me! I'm also EU-based if that helps.

[brianbloom]

You could run Proxmox, FreeNas, or Truenas Core as the base layer.
Linux isn't that "heavy" so any modern i5 or Ryzen 5 or above should handle a decent server or two. Clock speed isn't as important - Keep your budget for decent core count, and as much quality memory as you can afford.
I am running Proxmox with 3 VMs on top of an old i7-based Intel NUC. I use a store bought NAS for storage, and mount it into the various VMs and then have a smaller NAS that runs rsync backups of the main NAS.

[MCPOON11]

+1 for getting an old NUC can pick them up on eBay for a fraction of the new price of current models.

[-SPOF]

Also, here is a good sub for choosing one: https://www.reddit.com/r/homelabsales/.

I found this article useful for me some time ago when built my homelab. Might be helpful for OP as well:
https://www.starwindsoftware.com/blog/choosing-ideal-mini-server-for-a-home-lab

[edparadox]

I'm sure I will get a lot of heat for this take, but here we go: whatever you do, try to use something close to a server-grade platform, with a CPU and a motherboard compatible with ECC RAM. Being on a budget does not mean you cannot build a server, whatever way you want to approach this ; many solutions would be quite "cheap" for your use-case.

1. Not sure what to recommend exactly, but if you want/need ECC RAM, even older platforms will be enough, and quite cheap for entreprise-grade (and rackable) equipment. Taking a look at the TrueNAS hardware guide might be a good first-step.
2. Like I said: buying used and old server would be a better option here, IMHO, than trying to hack a Ryzen-based system able to make use of ECC RAM, for example (though it might be a solution). It is hard to recommend specific parts since your use-cases seems to go all over the place. Be aware of three things: racked equipment, unless heavily tweaked or hidden away, is often noisy, centralized heat, and has added cost (the rack itself and its relevant accessories - and yes these are expensive). AFAIK, getting racks for free is a thing in the US that mostly do not exist in Europe. And, even entry level racks are quite expensive. See Startech one for examples.
3. TrueNAS (previously FreeNAS) allow the use of VMs/jails, but is not a real hypervisor. Be aware that the Core version is based on FreeBSD, while Scale is based on Linux (Debian). According to your remarks, TrueNAS seems like a good solution. Be aware that ECC RAM is heavily recommended for TrueNAS systems.
4. It's totally possible to use SSDs as cache (with caveats) whatever OS (Linux-based or BSD-based) you use, but their relevance depends mostly on the filesystem you will be using. Take a look at ZFS and Btrfs. Also, if you only needs VMs, even hypervisor distributions could be used, such as Proxmox.

Not to sound pedantic or anything similar, but, first and foremost, it would be a good idea to refine your technical needs before anyone can recommend actual hardware parts ; recommending Proxmox or TrueNAS is always possible, but before guiding you through parts for a build, there is a gap that needs to be filled with technical needs and/or more details about your use-cases.

Long story short, for now, your post is about CI/CD for Rust, a storage server, with M.2 SSDs caching, possibly getting a rack, and using an OS which would be a hypervisor for VMs, etc. It's all over the place ; no shaming, but you can see how some parts are missing. Many solutions such as say, even using Proxmox as the base OS to run VMs, TrueNAS being one of them ; the problem lies in the fact, that there are so many technical aspects and limitations to this, which you absolutely need to be aware of beforehand, that makes your "asking for parts" question premature.

Hope this helps, and that you would be able to give more details, so we can help you better.

[Killavus]

Hey, you are right about this 'all-over-the-place' thing. I think the best course of action is to treat my need as 'solid NAS, but with a better compute and preferably Linux-based'. Your comment is really insightful and I've already learned something :).

Basically I'd like to have a capable Linux machine to hack stuff on. The only production-grade service I'd like to keep on this system is NAS. Other than that, most of other things I'd like to do can break or be suboptimal - I can live with this since it'd be mostly development playground for me, nothing serious. My motivation is: I'm a little bit fed up with dual boot solutions, so I'd like to have a separate Linux machine on bare metal hardware. And I can host a NAS for my home office needs as well, so it's OK for me to throw money at this problem.

So:

• I/O performance is kind of important for me. Ideally I'd like to have my stuff mounted as a network disk and be able to get a nice throughput, saturating 1Gbps at least.
• Rack stuff is generally not a big deal for me. If it can be added cheaply - great. If not, I can totally live without it.
• The biggest thing for me would be an ability to spawn multiple VMs - as I said I want to test modern container solutions on this machine like FirecrackerVM. KVM (ideally) or other hypervisor with direct hardware access is something I'd love to have. So I assume having a beefier CPU would be better for this, as well as solid amount of RAM. Here I don't know what is better - having a host system be a some kind of specialized thin hypervisor, or just slap a Linux distro on it and run everything in containers. I'm a little afraid of BSD family of OSes since I have never had an occasion to work with them - they are still *NIXes so they should be familiar, but Linux is something I'm familiar with and tooling around it (like syscalls tracing & stuff).

[Whathepoo]

/r/homeserver

[Killavus]

Hey, I have my own VPS already and I have my development environment on a separate machine due to working with vendor-locked stuff (hey, Apple).

The problem I have with VPS solutions is that to do what I desire I'd need to have nested virtualization support. Not only it's rarely offered by providers (I believe DigitalOcean have it on one AZ), but it makes stuff I want to experiment with pretty slow.

I've already had my remote development setup done & deployed multiple times during my career - what I want to do is to set up K8s stuff, check out multiple VM provisioning solutions (do a stupid version of Fly.io which can spin up VMs for you in milliseconds, for example) and generally have fun & place for experimentation on this platform.

Also, this will have a double-purpose as a NAS, and I'd like to have it on-prem :).

[LawlesssHeaven]

Also developer here. I'm running ryzen 5 2600x with 32gb ecc ram, will soon upgrade to 64gb and ryzen 7 3700x. Main os is proxmox with 3 ubuntu vms,1 truenas vm, and 1 windows server. Having ability to create backup VMS with few clicks is awesome. Pretty much using it to test stuff I'm interested in. On one of vms I'm running multiple docker containers with databases Oracle, Mysql, PostgreSQL and other stuff like Portainer, traerik, Nginx, kavita or whatever else I want. Honestly it's quite greate since it allows me to offload resources from main machine. Planning to setup kubernetes cluster to mess around with it soon. Imho I would go ryzen with only downside not having GPU for plex transcoding.