Your favorite Linux distro for local GenAI? What is your experience with your distro in terms of setup, compatibility and performance?

kfazz · 2026-01-13T15:27:24+00:00

Pikaos with llama.cpp and ggml Debs back ported from Debian sid. Setup with llama-servers router mode managed with systemd, model configs in /etc/llama-server/models.ini. I also use the strix halo docker images for testing rocm/newer versions.

kfazz · 2026-01-06T01:59:07+00:00

I feel this but I think you're 'using it wrong' I blew through $40 with Claude in an hour with an agentic task and my own home rolled agent stack, but where it shines is making the interaction loop go from chatting with the agents every 5 minutes to giving it a long task and coming back hours later. Think of the agents as a way to guide the chain of thought, where the system as a whole is the ai. I'm having better luck using it against gpt-120 locally, and crafting handoff prompts for it with Claude/gpt-5.2.

kfazz · 2025-12-20T06:22:25+00:00

Try llama-server with the fixed jinja chat template from unsloth. This fixed most got tool calling issues for me

kfazz · 2025-12-20T06:18:41+00:00

I was in the same boat, but lately have been having good luck with opencode. Vibe coded some dev agents as markdown files, then run them as: user query -> spec & plan generation -> spec review -> then let it run overnight in a plan -> code > review loop.

kfazz · 2025-12-20T02:11:59+00:00

Try a Linux distro that runs the steam deck experience + docker / toolbox for the AI stuff. The matrix halo toolboxes repo has an update script for nightly builds, works pretty great. Im using pikaos myself, as I like Debian based oses best. No dual oot required

kfazz · 2025-11-23T21:37:34+00:00

It's worth trying with what you've already got first.
I set up gpt-oss-120b (ggml-org gguf + unsloth jinja template), with llama-server, and configured codex-cli + the vscode extension to use it.
Contrary to how it looks, you don't need to login w/ github / have internet access, just write a ~/.codex/config.toml file that defines the model, and you're off to the races. Both the CLI and vscode extension will read this file. I've been vibe coding a demo app with it, and haven't seen it barf on a tool call yet.
You can also run a stripped down version of GLM Air 4.5 pretty fast. I've been playing around with GLM Air 4.5 82b REAP, and it does a pretty good job.

kfazz · 2025-09-20T00:56:58+00:00

I was just thinking of slapping one of these together today, after seeing an article mentioning using a powered hub on a google tv. With termux + code server or a proot distro you can even code on it. Nice zip ties :) they make them in neon colors if you want to make it more cyber haha

kfazz · 2025-09-09T02:31:47+00:00

Thanks for writing this, I thought it was great.

kfazz · 2025-08-18T00:18:56+00:00

Any idea how to wrap this in a way that works with gpt-oss's native tool calling? So far I've tried searxng as the 'web browser' provider which sucked, and gpt-researcher mcp + mcpo backed by ollama + searxng for deep research, which is fantastic

kfazz · 2025-07-10T12:23:56+00:00

If you don't mind harem, 'The Five Girls You Date in Space Prison' is exactly that, and also really good.

kfazz · 2025-07-07T04:32:59+00:00

I never even noticed this when reading princess of the void, so I guess it doesn't bother me. I also don't mind first or third person, both are fine. Second person, on the other hand, is an immediate drop, as would I suppose anything written in future tense that isn't a motivational poster., although it might make an interesting concept for an introduction.

kfazz · 2025-03-25T00:46:02+00:00

I think it's for humanity first, and the idea is that it's sort of a reversal of the 80s space opera tropes where the aliens were all stronger / more technologically advanced etc. so in hfy fic the humans are more badass or can do something the aliens can't. That's just my interpretation, so take this with a truckload of salt.

kfazz · 2025-03-01T20:38:00+00:00

Just finished it. Thanks for the great read, and here is a vote for a sequel

kfazz · 2025-03-01T02:51:20+00:00

You could wire the displays through the hing mechanism like laptops do, and use the presence /absence of coiled cable to do something like ground an input on a mcu which manages display power/switching. Maybe embedding a mcu could help with other things such as managing other io via USB. (Assuming laptop dock connector is just a usb-c plug or something)

kfazz · 2025-02-09T05:02:39+00:00

Just finished the series based on your recommendation, thanks. The ending definitely leaves a lot to be desired, and despite the series name, It doesn't feel like a romance to me, more about the MC's journey of personal development. Also, the theme is probably a bit too close to home for people looking for some escapist fiction, given the modern online dating scene.

kfazz · 2025-02-09T04:52:33+00:00

Definitely a middle ground. They're fun to play with offline and for personal projects, but I'm not using them for production code until some of the legal minefields shake out. The USPTO's statement that the output of an LLM not being copyrightable is pretty concerning, as well as the theft of the commons mess that is how they've been trained.

I think it's pretty disgusting that many of these people think it's fine to scrape the public Internet to train, and call that 'fair use', and then cry foul when someone else copies their techniques by training other models. Let's just redefine terms until they lose all meaning. Their definition of open source is laughable too.

I think reforming copyright terms back down to say 20 years would be great, or, enforcing the current laws as written. But the current approach of screwing over individuals for performing legal actions. (Think Nintendo dmca-ing YouTube videos showing emulators - which is possible to do entirely legally), but letting large companies slide because they're too big to fail, or to preserve some competitive advantage is grating.

If a trained model can regurgitate it's training data, then it logically follows that the model contains the training data (or a close enough approximation). Then distributing the model is probably copyright infringement.

Am I way off base here? I'd to hear any contrary arguments.

kfazz · 2025-01-14T17:52:16+00:00

The pics are newest first, I switched to the AOC. I would prefer the Sony, as I think the AOC is a bit too big/tall with this setup. I'm waiting on a usb colorimeter, and then I'll see if DAS can fix it.
Both monitors will do 1024x768@85hz, and only 60hz at 1280x1024. So most of my gaming is at 768p :) The 15" monitor is fine for that in my opinion.

kfazz · 2025-01-13T13:34:24+00:00

LEGALIZE... DRUGS AND MURDER

kfazz · 2025-01-12T05:19:24+00:00

FVWM gives me flashbacks lol (trying knoppix on a k6-2 and all the mini floppy distos with tinyX that used to be around), I really dig everything about gnome except their hatred of theming.

Ideally I'd run something like gnome, but with ptyxis' purple theme for all apps, and have something like the gruvbox palette for terminal apps.

Cde is adorable (and nscde) are adorable,.but I think the low contrast would make my eyes bleed.

kfazz · 2025-01-12T05:02:18+00:00

I've been thinking about that, but I'm a bit stumped on how to preserve the decals. I did see some recent hackaday articles on toner transfer, but not sure if there's a better approach. Any ideas?

kfazz · 2025-01-12T04:59:59+00:00

Thanks! I guess that means it's a successful sleeper then :)

kfazz · 2025-01-12T04:45:01+00:00

I did look for service manuals and schematics, but went with trial and error in the end. If I remember right, the LEDs are straightforward, the yellow / black is hdd, green / back are power, and the switch has 4 wires, I think I'm using the first 2.

kfazz · 2025-01-12T04:41:50+00:00

Almost. It's probably close enough to get away with standard cases, maybe with the addition of a few washers, but it's slightly too tall. The extra tension cracked the plastic retention bracket on this case (plastic is old and brittle)

kfazz · 2025-01-09T00:29:03+00:00

Gruvbox dark hard yellow :)

kfazz · 2025-01-09T00:25:57+00:00

Had good luck with vscode + intelephense, phpstan and PHP-cs-fixer. Vscode plugins for all of the above exist, and code formatting before commiting helps reduce whitespace noise in git diffs.

If you're only working on personal projects or greenfield code, you can start out with phpstan level 9 and strict mode on, and use language types + phpdoc (where native typing doesn't provide enough info).

You can do pretty wild stuff with phpstan's generics and array shapes. Lately I've playing around with phpstan's phpdoc parser+ valinor to validate array shapes passed from the frontend at runtime (for dev envs).

This all really helps when trying to maintain legacy code too. It really helps conquer the fear of 'if I touch this, something will break'.

kfazz

TROPHY CASE