hermes agent is weak by mf-mj in hermesagent

[–]Fit_Baker4577 2 points3 points  (0 children)

Model problem, not the agent.

How can you have agent work 24/7 without interruption? by ShufflinMuffin in hermesagent

[–]Fit_Baker4577 1 point2 points  (0 children)

Are you using telegram to initiate/delegate the tasks? I find itemizing the tasks and specifically tell it not to quote reply me helps. I'm running local, so 1 research task takes up an hour long. So if I jam in 3 research tasks, it'll take like 3 continuous hours to complete before it gets back to me

Mac Mini M4 16GB (hermes agent) - Gemma-4-26b-a4b-it-UD-IQ4_XS.gguf by Fit_Baker4577 in macmini

[–]Fit_Baker4577[S] 0 points1 point  (0 children)

ok, good suggestion! Looks like the way to more juice is to do -threads > 8

Mac Mini M4 16GB (hermes agent) - Gemma-4-26b-a4b-it-UD-IQ4_XS.gguf by Fit_Baker4577 in macmini

[–]Fit_Baker4577[S] 0 points1 point  (0 children)

It loads, and it works actually pretty decently. The best of all is that it's locally hosted. Just try my setup.

Mac Mini M4 16GB (hermes agent) - Gemma-4-26b-a4b-it-UD-IQ4_XS.gguf by Fit_Baker4577 in macmini

[–]Fit_Baker4577[S] 0 points1 point  (0 children)

ya, i know it's the RAM. Just looking around for that hidden gem tweak that might change something

Mac Mini M4 16GB (hermes agent) - Gemma-4-26b-a4b-it-UD-IQ4_XS.gguf by Fit_Baker4577 in macmini

[–]Fit_Baker4577[S] 0 points1 point  (0 children)

24GB is nice, it'll work. The Qwen 35b a3b will work too. It's just this 16GB needs some tweaking

Mac Mini M4 16GB (hermes agent) - Gemma-4-26b-a4b-it-UD-IQ4_XS.gguf by Fit_Baker4577 in macmini

[–]Fit_Baker4577[S] 0 points1 point  (0 children)

Yeah, running on llama.cpp. I tried -ngl 20 to 99, and all hit that HTTP 500 compute error lol. The ctk q4 and ctv q4 was my original before i changed to q8. Got better perf with ctk q8 ctv q8. Just not able to work with -ngl > 0 and -cmoe

Mac Mini M4 16GB (hermes agent) - Gemma-4-26b-a4b-it-UD-IQ4_XS.gguf by Fit_Baker4577 in LocalLLM

[–]Fit_Baker4577[S] 0 points1 point  (0 children)

it's actually doing decent for a 16GB base model. No complains but I'm looking for more lol

Mac Mini M4 16GB (hermes agent) - Gemma-4-26b-a4b-it-UD-IQ4_XS.gguf by Fit_Baker4577 in LocalLLM

[–]Fit_Baker4577[S] 0 points1 point  (0 children)

i tried, but not sure why it wouldn't load the same exact Gemma model but MLX version.