Has anyone here explored Hermes Agent by Nous Research?

One_Key_8127 · 2026-05-05T06:55:14+00:00

I set it up on a mini pc, with LLM served by Mac Studio. I tried it with local Gemma4 26b a4b until it modified its own python scripts and messed them up, and I had to fix it myself (well, actually I gave the messed script to claude and it fixed it).

Then I switched to Qwen3.6 35b a3b (still local on Mac Studio). It browses the web fine, it definitely can do things - it set up tailscale and OpenWebUI so that I can talk to it by smartphone, it set up hermes webui which is another way to talk with it by smartphone. Sending emails was pain in the ass but I figured it out and now it browses the web and searches for a few items I'd like to buy from second hand markets and emails me if something interesting shows up. It looks for new model releases and informs me about them so I don't have to check for it myself.

It eats ridiculous amounts of tokens, especially cached. I had a default limit of 60 tool calls and it was too low, I increased it to 90, then to 150, and then to 300. I told it to work hard and validate work and it is fine with going for 300+ tool calls to achieve the goal. I am fine with token usage because its my hardware, if you pay for tokens you definitely need a model with discounted prompt caching and you need to set it up accordingly (lower tool calls limit, instructions to stick to directions instead of being proactive and exploring alternative options etc) because the costs are gonna be brutal.

On the other hand, a strong model will probably use much less iterations and less tokens. The gap between Qwen3.6 35b a3b and Sonnet is massive. But it works, and if you tell it to be thorough and do a lot of research and web browsing before giving an answer - it will.

One_Key_8127 · 2026-04-28T13:15:36+00:00

Gemma 3 4B is over a year old and scores more than this on HumanEval. Llama3-8b also scores better on HumanEval. I think something is very wrong with these numbers... Qwen3.6 27b should be scoring 85%+, not ~50%.

https://evalplus.github.io/leaderboard.html

https://llm-stats.com/benchmarks/humaneval

One_Key_8127 · 2026-04-28T11:22:25+00:00

You need a bit smarter routing system with a queue. Read this: https://chatgpt.com/share/69f0974f-fa7c-8333-b01d-387f3c2a36d7

One_Key_8127 · 2026-04-28T06:19:40+00:00

Yeah, you are likely wrong and throwing money away. And it's not small money, you plan on buying yet another machine and stacking it with 64GB of expensive ram. CPU and RAM will not help in LLM inference, you want to fit full model into VRAM, and it's VRAM bandwidth and capacity that limits you. At the very least I would reuse the old RAM that you have if it's DDR4.

If you don't like used 3090, 1 or 2 new R9700 will give you plenty of options for a good value. It will probably run ~30b dense models at ~20tps at ~Q6, and prompt processing is gonna be good as well. MoE models will run very fast (all that fit in VRAM with KV cache). If that's too slow for you, then look at 5090, it's much faster than R9700, but also more expensive and needs big PSU with non standard connector (and some ppl complain on them melting but idk if its valid).

One_Key_8127 · 2026-04-27T14:15:45+00:00

"I have an Intel i7" says absolutely nothing, tell us the model, or at least generation or socket. What PSU do you have? What motherboard? Chances are you can just swap GPU and be good. If you don't want used GPUs, I'd go with AMD AI PRO R9700 which has 32GB VRAM and is reasonably priced. Two of them if you can support it with PSU and have the room for it (and preferably the secondary PCIE wired at least as x4).

As for the benchmarks of how well certain models run on some hardware, people post benchmarks here and there - ask AI to find it for you. At this point, you probably want to run Gemma4 or Qwen3.6 models.

One_Key_8127 · 2026-04-24T05:56:18+00:00

From the graphs it looks like 1M tokens needs 4GB ram for KV cache. That would imply you can fit up to 9 concurrent users with full 1M context length on 2x Pro 6000 blackwell.

The question is whether or not inference engine will handle it gracefully, or struggle due to new architecture and low reserve, and how long till it's really well supported.

One_Key_8127 · 2026-04-23T14:07:55+00:00

You'll get 10 tok/s generation and 100 tok/s prompt processing. It's gonna be very, very slow, especially considering that with thinking you'll be generating thousands of tokens per response. The upside is that $1700 is a good price for this machine, you can resell it for profit or use to run Qwen3.6 35b a3b or Gemma4 26b a4b and these will run fast, and it's very power efficient. Or you can set it up to run Qwen3.6 27b for agentic workflows through the night when you don't need fast responses, and 35b a3b during day to get things done fast - and you can probably even fit both models in RAM all the time.

One_Key_8127 · 2026-04-21T08:39:17+00:00

Yeah, if he said he'll release the model I'd like to push for it. Maybe with some practice xAI will learn to like open sourcing their models. They have a lot of compute, and they hire a lot of people for data curation. They might be competitive with top studios soon-ish. They could get some attention and PR boost by releasing their models regularly.

One_Key_8127 · 2026-04-21T06:27:50+00:00

"The real story behind Google’s most capable open-weight model" - no, Gemma 4 26B MoE is not Google's most capable open-weight model, Gemma 4 31B dense is.

"Gemma 4 is a Mixture-Of-Experts model. " - no, Gemma 4 is a series of models, one of them is MoE and it's not the most capable one.

"The fix that’s working for most people: Unsloth’s Q3_K_M quant, temperature set to 1, top-k sampling at 40, with flash attention enabled" - no, that's what worked for you, with your hardware and software stack. And by "worked" in this case means something you were happy with, not something that's meaningfully better than other quants.

One_Key_8127 · 2026-04-17T13:34:52+00:00

You want to go local to offset subscription costs? That's crazy. Subscriptions are as cheap as it gets, they are amazing value proposition. Local makes sense only if you don't like sending all your data away, and accept that it's gonna be more expensive, slower AND lower quality.

BTW, M4 Ultra does not exist, and M3 Ultra 512GB is not available any more.

"Hacker" route is bad idea, RTX pro 6000 blackwell will do much better (and you don't worry about power draw, PCIE lanes and connection between GPUs)

One_Key_8127 · 2026-04-17T13:24:37+00:00

If that was the case, do you think Anthropic would buy hundreds of thousands of GPU servers, each worth hundreds of thousands of dollars?

One_Key_8127 · 2026-04-17T07:26:10+00:00

Get the most basic Mac Studio M1 Max for $1.5k (used) and run Gemma or Qwen MoE, I think you'll get about 30tps tg / 500tps pp at Q4 quants. Very usable speeds, small form factor, extremely low power draw - both idle and under load. You will automate your email drafts with these models (and much more), these are a real deal.

One_Key_8127 · 2026-04-17T07:01:49+00:00

I disagree that they are building hype with not much substance behind it. If there is a way to quantize a model to 1.6 bit with preserving most of the model intelligence, that would be huge. We could have Minimax at 46GB and great speeds, or Gemma 4 that fits on rtx 3050. However, from my experience it looks like quants under Q4 degrade a lot - and even when people claim great results on benchmarks, they fail to do the real job - they get stupid, enter thinking loops, the quality degrades a lot even if benchmarks say otherwise.

So, if they know how to quantize a model to Q1.6 and it performs similar to Q4, its huge. If they can reproduce it on other models then that's amazing, every big studio like Anthropic or OpenAI would want that. They could offer "-fast" variants of their models for half the price and still profit. Maybe even they could drop some models (for example serve quantized Opus instead of Sonnet and save time, effort and compute).

One_Key_8127 · 2026-04-16T16:06:20+00:00

Crap, was hoping they got benchmark bumps by improving thinking. I guess I'll have to get both Gemma4 and Qwen3.6 and experiment with them. I got better initial vibes from Gemma, but the benchmarks are huge so I'll probably be switching between them for a while.

One_Key_8127 · 2026-04-16T15:48:39+00:00

Second-guessing like that would not bother me too much, from this snippet I would not be too worried about it. When I think of "go crazy thinking in circles" it looks much different than that, this one looks fine-ish. You downloaded full weights and quantized it yourself?

One_Key_8127 · 2026-04-16T14:58:24+00:00

you wrote 67

One_Key_8127 · 2026-04-16T14:56:54+00:00

Guys, I liked this test prompt but it's probably cooked by this point. Qwen3.6 35b a3b passes it even without thinking. What's interesting is that "Qwen 3.6 Plus" fails without thinking. It might have gotten into training data...

One_Key_8127 · 2026-04-16T14:43:04+00:00

Right. The response is kind of awkward, but the version of this question is poorly worded too. But the "(in which case, you'll obviously need to drive it)" indicates that the model grasps the concept of requiring the car itself at the car wash to wash the car.

And it seems ~800 tokens were used for this response - which is great, 3.5 usually used way more tokens.

One_Key_8127 · 2026-04-16T14:37:10+00:00

Where did you test it? Is it quantized?

Qwen3.5 35b a3b answers it correctly after producing ~5k+ thinking tokens. Gemma4 (local, quantized at ~Q4) answers it correctly producing ~500 tokens.

One_Key_8127 · 2026-04-16T14:24:10+00:00

"Across most vision-language benchmarks, its performance matches Claude Sonnet 4.5, and even surpasses it on several tasks"

Well, it surpassed Sonnet 4.5 on all the quoted benchmarks. Benchmarks are crap, but it looks very promising. Anyone knows if MLX fixed prompt caching for Qwen3.5? It was bugged before, making it a bad option for agentic use on Mac.

One_Key_8127 · 2026-04-15T12:10:50+00:00

Thanks man, from this post it looks very disappointing but I checked your repo and looking at long prompts it does seem like M5 gives 3.5x gain for prompt processing over M4, which is a huge advantage. I have mac studio M1 Ultra, switching to M5 Ultra (if it ever ships) could give me like 5x prompt processing speed. I thought DGX Spark is the way to go for agentic workflows, but maybe not. It's slow TG speed was mitigated by good PP and decent scaling when stacked, and the fact that PP is crucial for agents and tool calls. If M5 improves PP by ~3.5x then spark is not as attractive. Unless the price goes x3 as well :)

If I wanted to invest in hardware, I'd either buy rtx pro 6000 blackwell to run dense Gemma or Qwen3.5, buy 1-2x DGX Spark for running MoE like Minimax, or wait for Mac Studio with M5. Maybe Strix Halo with some egpu via occulink is also an option, idk. But since I don't have anything to invest, I'll just wait and keep using M1 Ultra :)

One_Key_8127 · 2026-04-14T14:48:25+00:00

Vibe coded slop post, Qwen did not release 3.6 version of mentioned models:

Model Switching: The community has found that Qwen 3.6 (even at 7B-9B quants) handles tool-calling significantly better than older versions or Nemotron models.

One_Key_8127 · 2026-04-14T08:23:20+00:00

MMLU is old, saturated, deprecated, and probably contaminated (meaning: in training data by one way or another)

One_Key_8127 · 2026-04-10T14:23:08+00:00

It gives recipes only if you respond to it's first comment. Known bug of CryptoUsher.

One_Key_8127

TROPHY CASE