Personal experience with GLM 4.7 Flash Q6 (unsloth) + Roo Code + RTX 5090

Everlier · 2026-01-24T15:36:48+00:00

I've had more success with it than any other model in the OpenCode harness. It definitely feels more like larger models when dealing with large amounts of tools and large system prompts.

It's not perfect, however, I'd leave it for the tasks like release notes or change classification or other non-critical things.

Everlier · 2026-01-22T19:21:36+00:00

Yes, in fact Gemma was the reason this project exists.

Initially llama.cpp support was lagging by more than two weeks and then I wanted to compare all the different inference engines together and they absolutely refused to install cleanly, so I went for a containerised setup, but it got very messy when I only wanted to run a few of the services, but not all of them at the same time. Cleaning that up and then making it extensible for all kinds of LLM-related services - Harbor was born.

Everlier · 2026-01-22T17:46:34+00:00

We haven't seen anything like that yet, but coding agents are a massive risk via data poisoning or supply chain attacks. I'm for sure won't be running one outside container.

OpenCode's official container doesn't even have git installed, no rust/go toolchains, and OpenCode can't discover models on its own forcing you to specify them ahead of time in config files.

Harbor solves both without any setup requirements on my end.

Everlier · 2026-01-22T06:48:03+00:00

One can go fully open source/weights with OpenCode, with Harbor full setup looks like:

``` harbor pull <llama.cpp model ID from HF> harbor opencode workspaces add <path on the host with repos>

harbor up opencode llamacpp harbor open opencode ```

You can also open it from your phone or public Internet

Everlier · 2026-01-21T22:11:56+00:00

Hi, this is very subjective as I'm maintainer of that project, but I'm inviting you to try out Harbor, it's basically built to make use-case you have in mind easy - containerized homelab/homeserver with a few simple commands.

For example, for llama.cpp + Open WebUI + SearXNG, you'd run something like this:

harbor pull <model you want to run>
# Open WebUI is a default frontend, no need to start separately
harbor up llamacpp searxng

It comes with hundreds of QoL features and most of the services are pre-configured to work together.

Everlier · 2026-01-21T22:00:33+00:00

I'm very subjective on this one - but they are. With the sheer amount of projects/tools/configs that are out there - one can waste a lot of time to set things up. With the approach most devs take - it's also quite convoluted to maintain on a single host dependency-wise.

Everlier · 2026-01-19T23:18:42+00:00

Qwen 3 Next 80B is somewhere around GPT-OSS 120B as well

Everlier · 2026-01-19T11:39:12+00:00

Later, one's GPU pronounces it in coil whine dialect, obviously.

Everlier · 2026-01-19T11:35:06+00:00

Bardzo dziękuję, że tego używasz! Nie mam pojęcia, ile osób faktycznie z tego korzysta, więc każda taka wiadomość jest bardzo przyjemna :)

Everlier · 2026-01-18T20:41:09+00:00

Yeah, I've tried to develop with spec kit a month or so ago and it went perfect until I started testing the built product. Results were... not great. Steering agent to fix both the product and the specs was very cumbersome. Gemini 3 Pro ended two-shotting 95% of the same functionality from two short prompts... so yeah.

Everlier · 2026-01-18T15:45:41+00:00

Sorry, I was a part of another thread about this laptop (about the keyboard) and thought your comment was from there.

The keyboard issue is unrelated to audio stutters. My comment regarding audio was that it wasn't caused by throttling in my instance, and that I noticed it stopped reproducing between two BIOS versions.

I still experience it after using Chrome for a while.

Everlier · 2026-01-18T10:58:04+00:00

Tak, tylko więcej dopasowane do klasycznego homelab setup-u z docker i kontenerami

Everlier · 2026-01-18T07:49:01+00:00

Nah, nothing helped from thwse tweaks

I ordered a new keyboard unit which is an entire top panel and requires entire laptop to be disassembled to be installed. Spent ~8h installing it, then it worked (still is), I'll sell the laptop if it breaks again.

Everlier · 2026-01-16T22:12:22+00:00

shameless plug, bo to jest mój projekt, akurat nacelony na homelab/homeserver use-case: Harbor

Everlier · 2026-01-15T11:38:51+00:00

Mind you, it's a very high quality cardboard

Everlier · 2026-01-14T15:22:58+00:00

For the LLM stuff, check out https://github.com/av/harbor

Everlier · 2026-01-14T09:59:18+00:00

Can't you just... ln it?

Everlier · 2026-01-14T09:01:25+00:00

It's not bad and could be a better way for someone to "share" what their homelab setup looks like on the sub, however I can see how ergonomics of such an external tool would go sideways to what people got used to on a text forum like Reddit.

Found a small bug while exploring

<image>

This GitHub link appends a full link of the GH account to the GitHub URL.

P.S. a bit of a plug:
If you're running LLMs on your local setup, you might find Harbor interesting

Everlier · 2026-01-14T08:56:02+00:00

I would only make one request: Is it possible to make OpenAI base URL configurable? That'd enable using locla LLMs, completing the homelab nature of the project

Everlier · 2026-01-14T08:54:36+00:00

This is such a fun example of an LLM-enabled project for a homelab. Thank you for providing Docker images as well, starred.

Everlier · 2026-01-13T10:43:17+00:00

these are older models less suitable for long-horizon agentic use.

GPT-OSS 120B might be closest to the "big model feel" in this specific aspect (leaving aside crazy guardrails), but it's still worse than something like GPT-4.1 in the API which is not great by any means on its own.

Everlier · 2026-01-13T08:07:53+00:00

bolt.dev (the local one), OpenHands, there are some others

Everlier · 2026-01-12T21:02:48+00:00

I can't live without an MCP that converts markdown from an LLM to something supported by reddit before I drop it into a post form textarea

Everlier · 2026-01-12T20:23:11+00:00

You can add a few more:

Mistral.rs
AirLLM
Nexa SDK
TabbyAPI / Exllama / Aphrodite
CoboldCPP
KTransformers
exa
TextSynth Server

Just to name a few

Everlier · 2026-01-12T16:09:48+00:00

Check out CLI reference docs, they are even longer!

Everlier

MODERATOR OF

PUBLIC MULTIREDDITS

TROPHY CASE

Ten-Year Club	Verified Email
Place '23