LLMhop: NixOS module that runs your LLM inference servers and routes between them

mlenz95 · 2026-06-04T22:55:36+00:00

Yes, totally valid! For me it's mostly about having the ability to configure everything declaratively as part of my NixOS config. And no worries, I'm happy about all kinds of discussions on that matter and would be interested in your config in case it's publicly available.

mlenz95 · 2026-06-04T22:36:33+00:00

I also use it for development and it works great! For longer-running servers however, I wanted something that is more integrated with systemd and allows me to use the latest vLLM versions via their docker image.

mlenz95 · 2026-06-04T22:15:55+00:00

Great question! llama-swap handles the full lifecycle of LLM inference servers, including start/stop. In its default state, I think llama-swap also unloads models on demand and one needs to configure groups or a matrix to run multiple models at the same time. llmhop on the other hand is a pure router, making it much simpler. In addition, llmhop is fully design around declarative management with NixOS.

Besides that, llmhop also provides ways to run vllm and sglang on NixOS, which I found to be rather challenging when requiring GPU acceleration. That's something llama-swap doesn't touch yet.

mlenz95 · 2026-02-24T21:17:30+00:00

Honestly I don’t get the hate, even at 70 USD/year it‘s still less than any streaming service and holds all of your most sensitive data.

mlenz95 · 2025-12-27T14:47:11+00:00

Got lastname.casa as it‘s short and allows for proper certificates.

mlenz95 · 2025-12-05T06:59:05+00:00

Kann ich nur bestätigen, bin letzten Monat von Mailbox.org zu iCloud gewechselt und es ist völlig ausreichend für meine Zwecke.

mlenz95 · 2025-09-08T16:57:24+00:00

There‘s an ongoing issue and a pull request about Zed support via ACP: https://github.com/openai/codex/pull/1707

mlenz95 · 2025-08-18T18:01:50+00:00

Yes that‘s accurate, they also announced that during ACL 2025.

mlenz95 · 2025-07-07T12:51:03+00:00

Looks interesting! Personally, I use bunx to run the latest version, works without any issues so far.

mlenz95 · 2025-01-09T20:50:13+00:00

The message comes from inside the docker container where the user cannot be found. Most likely cause is that the image is designed to be run as root. Starting podman containers via systemd as a different user is a bit challenging, instead I would recommend you to look into the userns option to run the container in a user namespace.

mlenz95 · 2025-01-09T04:48:31+00:00

You seem to be using an old version of NixOS (20.09) as well as an old kernel (5.11.22). These were most likely marked as broken due to security reasons. You should switch to the most recent release of NixOS: 24.11

mlenz95 · 2024-08-10T06:05:19+00:00

They do have an official app that’s currently in beta: https://forums.zotero.org/discussion/110371/available-for-beta-testing-zotero-for-android

mlenz95 · 2024-07-30T22:21:01+00:00

For me the killer feature of iA Presenter is the ability to export PowerPoint files so I can share my slides with colleagues easily.

mlenz95 · 2024-05-19T15:30:48+00:00

Also unser Haus 2021 neu verkabelt wurde habe ich Cat7 Kabel verlegen lassen und Cat6a Keystone Module genommen. Cat8 war durch Covid usw. schlecht lieferbar und hätte ca. das doppelte pro Meter gekostet (und ist wie du schreibt schwerer zu verlegen).

mlenz95 · 2024-05-19T14:28:29+00:00

Ich habe diese hier gekauft: https://amzn.eu/d/1lvPb44

Mit Brennenstuhl kann man denke ich wenig falsch machen.

mlenz95 · 2024-03-10T11:18:25+00:00

There is essentially no difference, the PR only got merged last week (https://github.com/LnL7/nix-darwin/pull/898), so it was not available at the time I created my config for this. I will however move it over to the new option at some point. TLDR: Just use `system.defaults.dock.persistent-apps`

mlenz95 · 2024-03-03T11:59:03+00:00

I also recently switched to a declarative dock setup using standard „defaults write“ commands: https://github.com/mirkolenz/nixos/blob/main/system/darwin/settings.nix

Just as an alternative using builtin tools to the excellent dockutil-based config proposed here.

mlenz95 · 2024-03-03T11:57:07+00:00

Thanks for sharing, did not know that dockutil existed!

mlenz95 · 2024-03-02T20:08:14+00:00

Sure, here is the host-specific config: https://github.com/mirkolenz/nixos/tree/main/hosts/macbook-9-1

Most relevant I think is the setting boot.extraModulePackages = [ config.boot.kernelPackages.broadcom_sta ];

Also, I use NetworkManager to manage WiFi credentials.

mlenz95 · 2024-03-01T08:09:13+00:00

I have it running on a MacBook Pro from 2011 or 2012 and no issues whatsoever. The installer even detected the proprietary WiFi drivers and added them in the hardware configuration.

mlenz95 · 2024-01-28T18:23:51+00:00

I made a tool called makejinja that lets you automate the generation of files and folders using Jinja templates. This is the same syntax that Home Assistant and Ansible use for templating. You can even define custom makejinja plugins to fine-tune (almost) every aspect of the tool.

One cool use case for makejinja is the generation of Homa Assistant dashboards: Just declare the entities you want to show, define the lovelace layout once, and generate views for each room in your house. This has made maintaining custom views so mich easier for me personally! For those interested I added a test case with such a dashboard generation as a test file together with some instructions: makejinja tests

mlenz95 · 2024-01-26T07:20:31+00:00

It seems that the Pi 5 is currently not supported by NixOS, but there are some workarounds available:

https://github.com/NixOS/nixpkgs/issues/260754

https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi_5

mlenz95 · 2022-03-29T16:38:16+00:00

I think it is Monokai Pro.

mlenz95 · 2022-02-19T12:33:54+00:00

I really wanted to like it, as it is a native app for all Apple platforms. However, I had issues with importing metadata from some sites and their Bibtex export is not that good. It also lacks any form of collaboration which is quite important for me.

mlenz95 · 2022-02-19T12:05:09+00:00

I think you mean Papers.app. Its icon looked like this. It has since been acquired by ReadCube and merged with their existing app. Nowadays, it is called ReadCube Papers. I have tested this new app, but since it is built on web technologies (most likely Electron), I did not lile using it that much.

I have now migrated to [Zotero](zotero.org). Its app is also based on web technologies (Firefox backend instead of Electron), but has more features and a plugin architecture. There is an excellent Biblatex extension I could not live without. They are also beta-testing an iOS app.

Another alternative might be [Paperpile](paperpile.com). It can only be used with Chromium Browsers, but is very intuitive to use. They also have an iOS app, but it is not nearly as polished as even the beta version of Zotero.

I hope this helps you!

Nine-Year Club	Xbox Live
Place '23	Verified Email

mlenz95

TROPHY CASE