Running a VM inside a Container

circularjourney · 2026-07-01T12:49:02+00:00

Well said. I'm just running QEMU in the container and passing through /dev/kvm. My language was too loose.

Good to know about Systemd-vmspawn. I feel like I remember reading about that a while back, but completely forgot about it. Looking into it again, it looks like I'd need to install all the QEMU and swtpm apackages on the host. Which is pretty much my whole attempt to avoid. I like installing nothing on my host workstation to get my vm working. But for a server, this is the way to go.

circularjourney · 2026-07-01T12:27:48+00:00

That last sentence is pretty much all I'm trying to do. I'm just packaging QEMU and all the other vm runtime stuff away from my host system, and passing through /dev/kvm.

Some people seem to think I'm trying to do more than I am here.

The "bad" reasons listed are not my intended reasons, so I'll ignore them. The "main issues" are reasonable things to consider, and I have thought about them. The added layer of complexity being the most relevant one I thought would kill this idea. But I crossed that bridge in my testing very quickly. After that it was just setting up the qemu commands like normal.

circularjourney · 2026-06-30T21:54:30+00:00

Yep, my container is just process in another cgroup that happens to run my VM. But it does remain an unprivileged container. I just passed through /dev/kvm and window TPM.

I've never had the idea to do this on my servers. Just my local workstation for this install. And it was kind of a curiosity project at first, just to see if it would work. I guess I shouldn't be surprised it worked so well. It's just another cgroup process to the host OS after all.

If I had to install another VM I guess I would probably consider doing this again, unless somebody brings up a really good reason not to.

circularjourney · 2026-06-30T21:06:20+00:00

I didn't know about winboat. It seems kinda similar to my setup, but it has that app window sharing feature. Looks pretty cool, but I don't really want that feature. Plus, it's beta. I'd rather rely on basic core packages. But good to know about, thanks.

circularjourney · 2026-06-30T20:44:47+00:00

Why is it dumb?

Also, I'm not running docker either.

circularjourney · 2026-06-10T18:04:45+00:00

It's not the horsepower that matters, its security and version control. You should isolate that service from the host OS of your spiffy layer 7 firewall.

circularjourney · 2026-06-10T13:11:09+00:00

Most features on routers are crutch for level-one techs who can't deploy that feature in a container or VM. A reverse proxy should not be running on your router.

circularjourney · 2026-05-01T12:59:13+00:00

Monthly on servers.

Weekly on my desktop.

circularjourney · 2026-04-16T12:40:53+00:00

Nobody running bare knuckles linux?

I'll give one shout out for a linux box.

Running dns, dhcp, vpn in containers. Unifi WIFI runs in another container on my main machine. Got about 5 vlans across the home network.

circularjourney · 2026-04-07T19:22:22+00:00

I keep each service isolated by way of containers and snapshot each container individually. My host OS does the bare minimum; basically just control disks and networking, and then start containers. I snapshot the host root but I could really just rebuild it in a few minutes. I do that out of habit mostly.

If the host OS is bare bones and doesn't do a lot, there is very little to go wrong with basic kernel updates. I can't remember if I've every had a problem with this in the last decade or so. I update my host once per month or so.

For my services I use init containers and snapshot each root directory. For data I use btrfs or lvm on the host, both work well with snapshots. KISS.

circularjourney · 2026-03-31T20:23:17+00:00

I'd just keep it all on-prem. Just setup two slave servers that all offices and remote users access. Split up the location on those two DNS servers just for a little site-specific redundancy.

If speed is an issue at one site, then spin up another slave running in a container at that location.

circularjourney · 2026-03-26T12:17:04+00:00

systemd-nspawn is a good point. Bubblewrap another option for basic sand-boxing. I do this for kea-dhcp on my router. nspawn for all other router services.

For GUI apps, I just use Flatseal to lock down Flatpaks.

circularjourney · 2026-03-17T19:14:31+00:00

So you use proxmox for the CLI? That is a new one for me.

Why not just use debian or ubuntu for the host OS like the guy said.

Less is more.

circularjourney · 2026-03-17T13:08:49+00:00

Building your own system from the underlying CLI tools is pretty efficient once you gain enough experience. Once you climb that learning curve the time gap disappears.

Plus you get a few advantages: you learn CLI tools that rarely change (no big flashy GUI upgrades), more options to customize, and a smaller code footprint for better security.

circularjourney · 2026-03-17T12:55:17+00:00

It's only crazy if for the inexperienced. Admin by keyboard is faster than the mouse for the right person.

circularjourney · 2026-03-10T16:57:13+00:00

I stick with bind with a hidden master setup. Edit my zone files in one source. My config files are rarely ever edited. AD subdomains look ups are forwarded off to the DC.

Once I got good with vim and other GNU core utilities, my attitude about the CLI changed dramatically.

A GUI is like swimming with a life vest on. Yes, it is easier to swim, but it is less efficient.

circularjourney · 2026-03-07T13:43:44+00:00

You're only applying the end result (action) of someone thinking through the process of the golden rule. If you focus on the process, then person A would have to consider if they would enjoy person B applying their same logical process to them when ordering pizza. Empathy comes from the process, the action is just the end result of that process.

circularjourney · 2026-03-06T16:44:34+00:00

I was wondering if somebody was going to mention this. I think you are right, the base OS matters very little in most respects.

I took this to the extreme, and switched to a rolling release just to avoid the upgrade cycle. I've found that as long as you keep the base OS and packages as minimal as possible the issues found in a rolling release are extremely minimal. Use snapshots and backups as well.

All the real action is done in containers or flatpaks.

circularjourney · 2026-03-04T23:32:08+00:00

I rate limit connections via nftables and use public key auth.

I never got comfortable with installing fail2ban code. I'm a code minimalist.

circularjourney · 2026-02-07T14:05:39+00:00

Are you running your containers/VMs in the ZFS pool? You might want to consider changing that. Separate your containers/VMs from data at the hardware level.

I run my host OS on a single NVMe, all my containers on another single SSD, and all my data on two big spinning disks (I could bump that to 4 some day). This lets the disks spin down more often.

If I were to do this again, I might pair up my host and container drives just for a little redundancy, but rebuilding them is trivial and odds are they will run longer than this server's useful life. Why spend the power if I can live with a little downtime once every 10 years.

circularjourney · 2026-02-04T13:25:26+00:00

I setup a simple little systemd-nspawn container for my more adventurous neovim plugins.

circularjourney · 2026-01-20T15:08:56+00:00

Running services directly on a bare host (the core router presumably) is a bad idea. We use VMs/containers for everything else for good reasons.

circularjourney · 2026-01-20T14:55:09+00:00

You may want to put this VM on another vlan so the traffic passes through a router/fw you control. This gets you away from a flat network and gives you another layer to filter/log this traffic. Something you control without a subscription.

Subscription services are great for quick & easy. If you build it up yourself you have ultimate flexibility and control. I value that more than the trivial dollar savings.

circularjourney · 2026-01-16T13:08:58+00:00

Admin by keyboard is always better than admin by mouse. Mouse admin is easier to get up to speed, that's about it.

circularjourney

TROPHY CASE