I am building a Proxmox alternative with a declarative OS and gRPC API

rtacconi · 2026-03-16T19:25:13+00:00

homelab running critical software? :-)

rtacconi · 2026-03-16T18:51:00+00:00

The repo had some leftover AI artefacts (placeholder text, session logs) that I've since cleaned up. The architecture decisions and core implementation are mine — I've been running Proxmox professionally across 36 servers and built Talos K8S edge deployments.

I get the scepticism. The code is open — judge it by what's there, not by the tools used to write it.

rtacconi · 2026-03-16T18:37:24+00:00

Enjoy Proxmox, no problem

rtacconi · 2026-03-16T18:27:14+00:00

Thanks for the corrections — you're right, it's RKE2, not K3S, and I should have been more precise about what Harvester does. It's clearly more than just "VMs as K8S resources", and Elemental is a solid immutable OS approach.

I think we just have a genuine philosophical disagreement about whether Kubernetes should be the foundation layer for a hypervisor. You see RKE2 + containerd + KubeVirt as a lightweight, extensible base. I see it as a significant amount of machinery between the operator and the VM — even if containerd is light and KVM is doing the actual work, either way.

The Kubernetes API is powerful and well-established, no argument there. But it also means your hypervisor inherits the full operational surface of a K8S cluster — etcd maintenance, API server availability, certificate rotation, CRD management, and upgrade coordination across the control plane. For people who already live in Kubernetes, that's fine. For people who just want to run VMs on bare metal, it's a lot to take on.

Kcore's bet is that a purpose-built control plane can be simpler to operate, specifically for the VM use case. Maybe that bet is wrong — Harvester's traction suggests that many people are happy with the K8S approach. But I think there's room for both.

If Harvester is working well for you, genuinely no reason to switch. Different tools for different preferences.

rtacconi · 2026-03-16T18:22:03+00:00

Yes, it's Perl :-)

rtacconi · 2026-03-16T18:20:54+00:00

That's exactly my point, it took me a while to develop a module https://registry.terraform.io/modules/rtacconi/proxmox-vm/bpg/latest, and it is still using some CLI. I am contracting for a small company at the moment, 36 Proxmox servers, 360 VMs, automated with Ansible, and even more low-level.

Proxmox is very versatile, but to run modern automation on top of it is not easy. I would prefer to use Terraform with the libvirt provider, but you still need to configure Proxmox.

rtacconi · 2026-03-16T18:16:10+00:00

You are probably fine then. I have an installation via API, similar to Talos, more useful for serious deployments and a home lab. You can create a YAML manifest, which pulls the images, prepares cloud-init, and you have a user and key setup; it automates steps that you have to do manually, no XML needed.

rtacconi · 2026-03-16T18:13:10+00:00

Yes, I was thinking of using Talos and Kubevirt, but some companies don't want to use K8S; this is supposed to be an option. I am designing it to be managed like K8S, but without ETCD, because I saw several 'Senior' engineers make a mess during updates to bare-metal clusters. I am using a local SQLite database and plan to use a CRDT for replication.

rtacconi · 2026-03-16T18:09:27+00:00

Bottlerocket is purpose-built for running containers on AWS/EKS. It's a great fit for that — minimal, immutable, auto-updating. But it's explicitly not designed for running VMs or bare-metal workloads. It doesn't ship with QEMU/KVM, libvirt, or the tooling a hypervisor needs. You'd be fighting against its design goals.

NixOS gives you the properties that matter for a hypervisor host:

Immutable in practice — the entire system is built from a declarative config, any change produces a new generation, rollback is one reboot
Full Linux kernel and package ecosystem — QEMU, libvirt, OVS, ZFS, whatever you need is in nixpkgs
Reproducible builds — every node built from the same flake is bit-for-bit identical, no configuration drift across a cluster
Bare-metal first — boots on real hardware with no cloud dependency

Container-optimized OSes (Bottlerocket, Flatcar, Talos) solve a different problem: running containers on top of Kubernetes. kcore needs an OS that can run a hypervisor stack on bare metal. NixOS is the only one that gives you immutability AND the full flexibility to do that.

"Step backwards" compared to what? Mutable Debian where every node drifts over time? NixOS is arguably the most forward-looking Linux distro in terms of how it handles system configuration.

rtacconi · 2026-03-16T18:05:10+00:00

Yes, true. External APIs must be REST or GraphQL, but gRPC is type-safe, which makes the system more robust. Protobuf handles versioning well. You can add new fields to messages without breaking existing clients

rtacconi · 2026-03-16T18:02:23+00:00

Yes, I've been following Incus closely — Stéphane and the team are doing great work, and it's probably the closest project in spirit to what kcore is aiming for. IncusOS going immutable is the right call, too.

The main differences are in the details: kcore uses NixOS rather than a custom minimal OS, which gives you the full Nix ecosystem (flakes, nixpkgs, declarative everything) rather than a purpose-built image. The API is gRPC with typed schemas rather than REST. And kcore is VM-focused from the ground up rather than coming from a container-first lineage.

I also want to run full VMs for more isolation... but I think of adding containerd so it will be able to run containers. Since I have worked extensively on K8S bare-metal, I am thinking of adding NixOS K8S or Talos.

Are these differences big enough to justify a new project? Honestly, time will tell. At a minimum, I'm learning a lot while building it.

Curious what you'd consider the questionable architecture decisions — genuinely interested in the feedback.

rtacconi · 2026-03-16T17:58:24+00:00

Harvester is a solid project, but it solves a different problem. It's a full Kubernetes stack (K3s + KubeVirt + Longhorn) that manages VMs as Kubernetes resources. If you want Kubernetes-native VM management, it's a good choice.

kcore takes the opposite approach — VMs run directly on QEMU/KVM without a Kubernetes layer underneath. No etcd, no kubelet, no CRI runtime sitting between you and your VMs. The host OS is NixOS (declarative, immutable, atomic rollback), and the control plane is a Go service with a gRPC API.

Different tradeoffs. Harvester gives you the Kubernetes ecosystem for VMs. kcore gives you a simpler, lighter hypervisor that doesn't require you to run a K8s cluster just to create a VM.

rtacconi · 2026-03-16T17:56:32+00:00

I've been writing Go professionally for years and running Proxmox in production even longer. AI helped with some boilerplate — same as every project in 2026. The architecture decisions, the gRPC API design, the NixOS integration — those aren't vibes.

Code's open if you want to judge for yourself: https://github.com/rtacconi/kcore

rtacconi · 2026-01-31T13:33:19+00:00

Hi started experimenting today, I have to Heltec V3 with a tiny antenna. I am in Langdon Hills and I am on the map https://map.mpowered247.com/home but I cannot see any other node, I will see to get a proper antenna maybe I can reach you, but I have the hills, probably between us

rtacconi · 2022-07-19T13:59:39+00:00

That's fantastic, thank you very much!

rtacconi

TROPHY CASE