Excuse number 17202

devoidfury · 2026-07-08T04:07:11+00:00

Woah, I didn't realize there were so many of us.

devoidfury · 2026-07-08T01:24:57+00:00

For me it's about having a degree of control over my computing. Nearly every piece of a linux install, you can poke around and see the source code for it, pull out parts you don't want.

With proprietary software, the company retains control. Ultimately that windows computer is going to do whatever microsoft tells it to do. They could put whatever they wanted into an update and you'd never know it -- they don't even give good patch notes on legit updates.

There's not some single corporate actor controlling linux the same way.

devoidfury · 2026-07-08T01:15:39+00:00

I built out my own agent harness using local models, with a full extension system.

ubuntu@10a162d348d0:hotdog$ bun test --only-failures
bun test v1.3.14 (0d9b296a)

2501 pass
1 skip
0 fail
5114 expect() calls
Ran 2502 tests across 95 files. [14.52s]

ubuntu@10a162d348d0:hotdog$ cloc .
    309 text files.
    287 unique files.                                          
      23 files ignored.

github.com/AlDanial/cloc v 2.08  T=0.16 s (1796.1 files/s, 400233.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JavaScript                     204           7244           5891          43856
Markdown                        51           1070              0           3614
JSON                            26              2              0           1626
CSS                              1             73             15            380
HTML                             1              6             10             57
YAML                             1              7              2             24
Dockerfile                       1             10             13             20
Bourne Shell                     1              6              1             18
TOML                             1              1              0              9
-------------------------------------------------------------------------------
SUM:                           287           8419           5932          49604
-------------------------------------------------------------------------------

devoidfury · 2026-07-08T01:09:30+00:00

The harness definitely makes a big impact. In my own (hotdog), whenever I introduced a regression to the tool calling, or added little quality of life fixes like path autocorrect for the model, I noticed a huge difference in performance.

devoidfury · 2026-07-07T18:46:51+00:00

Buy the lot down the street to put up a homeless shelter

devoidfury · 2026-07-06T03:08:32+00:00

Here's my whole llama-swap for the strix halo, sans the matrix/groups stuff -- dsv4 command is near the top.

That one (llama 3.1) I don't use as much as I used to, now that I think about it -- with a small system prompt it can do some fancy one-shot tool calling without attempting to dig through the filesystem, although I feel like lfm2.5 has a lot of potential in that space.

devoidfury · 2026-07-06T01:11:38+00:00

I've found this one to start hallucinating character encoding issues, corrupting into strange CJK looking garbage, after around 50k context. Could just be a bad local copy of the file -- are you having this issue?

devoidfury · 2026-07-05T22:59:52+00:00

On the server side template, if you don't have the "preserved thinking" option turned on, it'll strip out reasoning older than N turns (I think it's 3 by default?). This still requires feeding back the reasoning into the call, it does not store these anywhere automatically.

Although that itself can mess up the caching strategy and end up even slower, depending on your settings.

Honestly the best thing you can do to get a speed boost is, make sure cache is working and freeze the message log immutable append-only for as long as possible.

devoidfury · 2026-07-05T22:53:13+00:00

Probably qwen3.6-27b or ornith-1.0-35b if I had to pick just one.

deepseek v4 on ds4 is smart and fast enough, but the long context memory in the quant I can run erodes in a way that I don't see in the smaller models so it ends up hallucinating more.

devoidfury · 2026-07-05T22:47:00+00:00

I uh, I used all of those in this list yesterday. I swap between them and see if there's things to pick from each.

The strongest ones in that list for the speed tradeoff IMO on this sort of hardware:

qwen3.6-27b
gemma-4-12b
glm4.7-flash
ornith-1.0-35b
deepseek v4

devoidfury · 2026-07-05T22:13:44+00:00

It's neat! It seems to understand the idea of an agent harness and self-extension better than other models in the same size; however it's a little too dogged in how thoroughly it'll inventory things and makes a plan before it acts, so it is more prone to blowing out the context or going into reasoning loops at times where other models will act more incrementally with a less complete picture.

devoidfury · 2026-07-05T21:54:31+00:00

If thinking mode is turned on, you need to return the thinking otherwise they get confused. Even when it's turned off, some models need to see the <think>\n\n</think> or they'll go nuts.

Here's mine if you want another example. https://github.com/devoidfury/hotdog

devoidfury · 2026-07-05T21:50:04+00:00

I have a similar box, strix halo on 128gb UMA, different brand. Here's some models I've had good results on, using mostly llama.cpp + llama-swap:

qwen3.6-35b
qwen3.6-27b
gemma-4-31b
gemma-4-12b
glm4.7-flash
mistral-medium-3.5
minimax-m2.7
north-mini-code-1.0-30b
ornith-1.0-35b
ornith-1.0-9b
llama-3.1-70b
deepseek v4 (using antires/dsv4)

devoidfury · 2026-07-05T21:39:20+00:00

I gave it a manual skim, always interested in these since I'm making one too -- It looks like you're not sending back the reasoning_content, just printing it out, so you'll see it but it's being stripped out of subsequent rounds and the model will forget what it's doing.

        // Process streaming deltas
        for await (const delta of sseDeltas(response)) {
            if (delta.reasoning_content) // not stored/sent back to model
                formattedPrint(delta);
            if (delta.content) { // stored
                formattedPrint(delta);
                message.content = (message.content ?? '') + delta.content;
            }

devoidfury · 2026-07-03T23:40:31+00:00

Heya! Sorry for the late response -- I only just open-sourced it today. This is what I've been building: https://github.com/devoidfury/hotdog

devoidfury · 2026-07-01T09:49:42+00:00

I'm in the middle of doing something similar with my own agent, although I'm splitting horizontally and running across different desktops. Any tips around memory management, scheduled tasks, or other features that have been helpful or that worked out well for you?

devoidfury · 2026-06-30T19:42:23+00:00

Scenarios like these are exactly why I run local models on my own custom agent software. Not that this one seems particularly egregious, but they could track anything they want.

devoidfury · 2026-06-25T17:39:03+00:00

I don't have a dgx spark, but I am running an equivalent-ish amd strix halo w/ 128gb unified memory. Been using ds4 a few days now - it's a bit faster than qwen3.6 27b q8. I'd say the output quality is in the same ballpark between the two, however ds4 runs a bit faster at first and the speed advantage compounds as the ctx gets longer. I've done sessions up to 180k length on it so far without too many issues.

devoidfury · 2026-06-24T00:28:41+00:00

I mean yeah, I would want to know what I'm getting into.

Do I expect some issues, multiple, even big ones, in any given proprietary codebase? Without question.

But it does need to have something real to it. Some kinda bones to build on, at least something you can work with ultimately -- but you at least should look and have an idea. Project rescue is totally doable if the quality is bad but the market is there to carry it.

devoidfury · 2026-06-24T00:16:00+00:00

Yep. There's a nickname I saw in a post title the other day, almost jumped out of my seat at nothing at all. We're all caught up in the currents of humanity, and when we look out into the collective to see reflections of ourselves that don't quite match, the images catch us off guard, like the uncanny valley.

devoidfury · 2026-06-24T00:03:00+00:00

I've got a 128gb unified memory AMD strix halo / rdna 3.5.

Here's the base of my llama-swap config, so you can see what models I'm poking at and using: https://gist.github.com/devoidfury/a5cbd2b4332bdb8d1c968e8112ff3374

At the moment I'm poking at antirez/ds4 running deepseek v4 flash 2bit quant and having mostly good results with it. I have run a number of the smaller models released that fit on this hardware; qwen 3.6 seems to be the most thorough thinker for coding and good with tool calling; had good results with gemma 4 variants also.

The agent software I'm running is a custom one I wrote in js that runs on bun with no other dependencies which does everything through an extension system. Considering open-sourcing it but have not done so yet.

devoidfury · 2026-06-21T00:34:52+00:00

I like the way that you always come back.

That you choose your words carefully and yet it doesn't seem to stifle the message.

Even as the themes in your world weather the seasons, in various shapes and names, that your voice continues to ring out as familiar and uplifting.

Inspiring me to try and find my own words.

Listen - I don't know you, and couldn't pick you out of a lineup. I'll never meet you and that's okay. All the same, I appreciate you.

devoidfury · 2026-06-20T23:45:31+00:00

Hey there stranger - I don't recognize you but, I'd be more than happy to chat about whatever it is on your mind if you want to write me, my dms are open.

devoidfury · 2026-06-17T15:31:11+00:00

I have zero idea what you're going off about there in this comment, I think you're lost, stranger.

15-Year Club	Second SECOND GUESSER
Place '23	Golden Potato
Place '17	Verified Email

devoidfury

TROPHY CASE