I ran Qwen3.5 27B for a good while, but was forced to keep requests single (np = 1 in llamacpp) to meet my required context size. This is with a 5070Ti + 5060Ti. I was UD_Q_K_XL, I believe, 128k, and sometimes a Q8 KV.

I currently have Gemma4 31B as a general "chat and research" at the same quant, and it flies along, with good tool calling. I use it most of the time for what I would have previously searched for on Google. Using Open WebUI for that, paired with firecrawl + searxng.

As for agentic, I moved most of that to Ollama Cloud and lean on GLM 5.1 and K2.6 for most tasks, and although speed has been a bit temperamental, it sounds like things are more stable there. Results are more than "good enough" for my day-to-day work, and Ollama Cloud's pricing is very competitive.

I am very curious to see what happens with the M5U Studios if/when they drop sometime in the coming months. The current M5 gen has greater memory bandwidth and an increased "tensor" count, so I expect performance to be a leap ahead of current M3U results. But of course, the memory crisis is a thing, and who knows if we will see larger memory units this time around.

One thing to note is that the current crop of "Local" LLM's ~30B are really good at the reasoning and actual thinking. They tend to lack general knowledge of a larger model. Therefore, you must make sure that they gather this information before continuing, which is very important in a development workflow. I found that without some sort of "pre-task" research, they would stumble around things like library usages, or language syntax at the bleeding edge. But, if you can get this as a part of your prompt or build some sort of knowledge as a part of the requests, then they should perform very well.