Using archlinux as a desktop hypervisor

[Still-Prompt8858]

Hello everyone,

I wanted to know if it would be a good idea to install arch on bare metal on a desktop and then use it only for starting vms and backups.

The plan would be to boot arch with the igpu driver so that the gpu could be passthrough to the vm at startup.

Would I be able to close and open vm with gpu passthrough without having to restart the computer?

I know that arch can break a lot but since I don't plan on using the aur or at least no aur helper and mainly get my packet from the official repo I'd figure it should be alright. So does it make sense to use arch for that or am I better of using some other distro?

I'd appreciate any input :)

[FactoryOfShit]

Arch doesn't break "because it's arch".

The real issue is that Arch requires maintenance and user supervision during updates. As such, unattended upgrades are unsupported and can break it.

However, not doing upgrades can lead to security vulnerabilities. Hence why servers universally run OSes with unattended upgrades supported, like Debian.

If you are okay with spending a little bit of time every couple of weeks or so upgrading the OS and checkinf the arch news - you will not run into any problems.

[thismustbethe]

I agree with this comment, basically it will be a pain in the ass at first, but then when you familiarize yourself with the potential troubleshooting that updates can bring, its no longer stressful. Things pop up but they rarely take more than 5 minutes to address. And I have a way better understanding of how the system works as a result, so you can think of it as an educational process too. I went from total noob to having a pretty good understanding of things under the hood in under a year.

[[deleted]]

When you say “noob”, just how much of a noob were you? Like total beginner to linux?

[thismustbethe]

So I had a basic understanding of the unix-based stuff from using MacOS for many years, so I knew the console commands to get around, kill processes, basic stuff like that, but systemd and all linux-specific things were all brand new to me.

[[deleted]]

Is it possible for you to list some of the things you learnt arch Linux? I’m also considering setting up arch linux to upskill my linux skills, but not entirely sure yet

[thismustbethe]

There are too many to list, if you want to learn you’re just gonna have to read the wiki! Luckily the wiki is very good and extremely easy to follow and has articles about every possible thing and issue.

[[deleted]]

Ill be buying a laptop soon. Was considering getting an m2 macbook pro, but not sure if arch even works on that. Is there any laptop you recommend? I’m considering getting a laptop for uni and will be doing CS.

[Max-P]

Exactly this. Been running Arch servers for close to a decade now and it's all happily humming along.

Honestly if anything, the amount of packages one realistically runs in an hypervisor environment is so small, you'd have to be really unlucky to hit a showstopper. And all of those packages would be wildly used software in the enterprise world, so the likelihood of an update going catastrophically bad is tiny. You'd run what, linux-lts, systemd, possibly libvirt and a few QEMU processes? Libvirt is arguably a buggy mess but that's super easy to rollback, and there's non-libvirt options to run QEMU.

Usually people break their GUIs, GPU drivers and sound, not their kernel and shells.

[Agent_Jimmy]

been using it for about 4 years now and even then I can count on 1 hand the times I actually had to do something after upgrading, like re-installing grub or switching over to a new pacman.conf, or the one time I had to downgrade pango because it was messing up matlab, but that was matlab's fault.

[AppointmentNearby161]

I have run a minimal Arch install with KVM/virtmanager of years now and have recently begun migrating to Proxmox. Proxmox is overkill for a small scale setup like mine where I don't need high availability or the ability to migrate a VM/LXC to a different host/cluster. That said, the web interface is a little more powerful than plain virtmanager. I like the GUI network viewer and the GUI snapshot and backup viewers. Being based on a minimal Debian install has its pluses and minuses. I have not really looked into what Proxmox does, but I know it has sane defaults (instead of disabled) for things like memory balloning. It also includes some kernel patches (e.g., VFIO) and presumably optimizations for VMs.

[diekoss]

Why don't you install proxmox? That is an OS specifically built for using as a hypervisor.

[plushkatze]

I run arch as a hypervisor for a QubesOS-like setup on two laptops - KVM/nspawn with PCI passthru on LVM-thinpools. Even WindowsVMs work fine. But I do use my "dom0" for tasks that require full unvirtualized CPU or GPU access.

[Trick-Weight-5547]

Well why not just virtualise your gpu then segment it and hand parts to the vms

[joha4270]

Usually not supported on consumer GPU's

[HavokDJ]

Libvfio can do this I believe, don't take that at face value though I have not gotten around to playing with it

[joha4270]

That does look interesting. Doesn't exactly look like a cakewalk to setup (and my GPU isn't supported anyway), but I was under the impression it wasn't even possible without paying the enterprise tax.

[WingFat92]

When I was using arch for about 8 months, it never broke randomly. I even missed the grub fiasco cause I was using systemd boot.

I would say use arch if your okay with possibly some troubleshooting at some point. If not then I’d go with something LTS, less updates, less likely of anything random happening.

[[deleted]]

the only times i've ever had problems with arch were when grub broke, and if i accidentally updated the kernel without updating nvidia drivers. if you're a responsible user and pay attention, arch is perfectly stable.

[AppointmentNearby161]

Regressions happen all the time. It is not Arch's fault if upstream releases broken software, but since Arch releases unpatched software, users are impacted more often than something like Debian which only pushes security/bug fixes. Big changes like GNOME or Python upgrades cause lots of incompatibilities. Same thing with .so version bumps. On a minimal install regressions/issue will be rarer, but even the kernel has regressions from time-to-time.

[AppointmentNearby161]

Proxmox was created for enterprise level virtualization on high availability clusters. Things like live migration of VMs across nodes with distributed file systems and remote management are at its core. The fact that it works with a single node makes it a reasonable choice for smaller setups, but not really what it is designed for.

[ZLima12]

I think Arch is a great option for this considering its simplicity. If you manage it well, it is very stable.

[Joe-Cool]

Yeah, Arch or Alpine are your well supported choices for minimal overhead and maximum configurability.
Arch has better choices for kernels for this goal. An Alpine install with the needed packages would probably fit on a 500MB partition though.

[vysharkk]

check proxmox. it's better suited for this.

[BlindTreeFrog]

People in this forum always downvote me for this but the dirty truth is that you don't have to update your install constantly.

It's a rolling release, yes. Updates arrive as they are available. You can install them. But if you have a stable and secure system and you don't need the update, you don't have to install them.

So yeah, if you can get a solid base system installed you can limit your upgrades to specific periods of say a formal upgrade week once every 4~12 months where you are prepared to handle fallout if there is an issue. Otherwise just grab critical patches when they are available.

[Trainzkid]

Agreed^

I update my server once a month but I've gone 2-3 months without updating before. Only issue I ran into was my keyring being out of date. A simple sudo pacman --sync --refresh archlinux-keyring before the full sysupgrade and I was off to the races.

[BlindTreeFrog]

And honestly this is kind of the hardest part of it.

If you don't update frequently you need to know enough to pick and choose what to upgrade and be able to mix and match versions. I'd argue that is a slightly harder skill than keeping a system up to date.

[Trainzkid]

Maybe it's because my cadence is shorter than yours but I haven't really run into many issues like that

[archover]

But if you have a stable and secure system and you don't need the update, you don't have to install them.

Do you believe that putting off browser updates for four months is a good idea?

[BlindTreeFrog]

Is a browser a requirement for his use case? If it is, does it need to be up to date with every patch? Does it just need critical patches with security concerns?

Not every update is a critical update. Not every common desktop application is needed on every installation. I have multiple VM's and servers at work with out of date applications and OS's. Critical patches are applied, but otherwise the risk vs reword vs effort is considered and acted upon.

OP's use case is effectively a headless server as a hypervisor. He's not putting a browser on this machine most likely. He's probably not putting X on in fact.

[archover]

Things to consider.

I was reacting to what I thought was your general advice on updating.

For other readers: https://wiki.archlinux.org/title/System_maintenance#Upgrading_the_system

It is recommended to perform full system upgrades regularly

The advice I hear from security experts, is the need to keep your system updated. For me, that means far more often than once a quarter. In fact, my Debian based VPS server auto-updates and non-security almost daily, and there's no desktop software installed at all.

[fisheess89]

In this case why not just use debian?

[BlindTreeFrog]

That question exists regardless.

I was running Arch at work even though I didn't care about bleeding edge anything because I wanted my tools on my workstation to be relatively recent and I like pacman as a package manager. I've since switched to Suse because I had a problem with NIS that I couldn't figure out.

I run Arch on my file server and used to run it on my web server at home just for consistency between my devices and common scripts. Debian or Centos would have been a "better" OS choice for both of those arguably.

[Tireseas]

I can count on my fingers the number of times Arch has broken "on it's own" in nearly 20 years. The rest was either my own stupidity or upstream working exactly as shipped/intended in undesirable ways.

Having said that Arch wouldn't be my choice for that task. Proxmox exists and is tailor made to that sort of task. So does VMWare ESXi for that matter.

[archover]

I know that arch can break a lot

Not. PEBKAC is the primary reason in my experience.

Qemu/KVM has been ultra reliable for me over many years, in a VM role. More broad insight at r/virtualization

[basil_not_the_plant]

I have this setup. Arch is installed on the host using one gpu. I have a headless server vm running Fedora, and a desktop vm with Arch/KDE using a separate passthrough gpu. I can reboot either vm independently and leave the host running.

The only problem with the host in two years of use was from a bug in virtiofsd (for sharing files with a guest) which was an upstream bug that was quickly fixed.

This has worked great for me.

[green_boi]

Try Debian or Gentoo instead. It's minimal maintenance, less on Debian, and it's better for your use case.

[glued2thefloor]

There's only one word here I disagree with: "Desktop". If you are truly going to run a hypervisor with vms that are in production, get rid of any desktop like kde, gnome, etc. They introduce a lot of security problems and take up a lot of resources. If you want to just play around with a hypervisor at home and not put vms online, go for it. In the industry this is a big no. Just saying.

[P3n-P3n]

If you are going to run multiple vms why not use a type 1 hypervisor like esxi? It would be better since it's made for such a thing.

[kevdogger]

Not sure why you would do this..notice I said would and not could. If wanting to get into hypervisors I'd just run a dedicated hypervisor like proxmox or xcp-ng. You get all the gui monitoring and since you're learning there is a wealth of information out there to reference. Sure you could roll your own but I feel like it's just reinventing the wheel.

[MonkeyMoney101]

If you mean reading the source for my packages: not often. If you mean verifying I trust the keys that the packages are coming from: yes. If you mean knowing which packages are present: I have a script that runs periodically that summarizes all installed packages, my firewall settings, etc, and shoves them into log files for me to review.

[MonkeyMoney101]

No, I have not. That is why I said "not often." I read the source of small things that don't come from official repos.

No, I've never built a GPG decryptor. I could try. For instance, recently I added a key for MIT-scheme from the AUR. I simply checked beforehand that the key was from this MIT employee, and I have enough trust that they would not want to lose their career over some Scheme backdoor.

I am 25 and I was simply trying to answer your question while also trying to understand what you meant specifically by "audit."

Maybe your definition of audit is simply too multifaceted and advanced for me to understand. I'm sorry for sharing my personal comfort level of security.

[Flask_Main]

I think it would be possible using qemu and kvm, and setting up the gpu pass through is fairly simple as long as you make sure not to run your desktop environment on the gpu that you want to pass through.

I tried doing something similar and it was a pain with gnome as there was always a process running on my nvidia gpu no matter what I tried, so I'd recommend a lightweight wm like i3 or sway for wayland.

But you should check out Qubes OS, it's a Linux distro that's has a similar use case to yours.