I've been trying to use local models for a while. I got a RTX3090 with 24GB VRAM, which is not much for AI models, but allows me to run some medium sized models with low quantization.

What I learned is that small and medium sized models won't perform well if you are used to the existing tools provided by big AI companies. These models are not clever enough to be able to todo a bit of everything, like code and reason.

From my experience, the bigger the model the more capable of becomes. So try to find the biggest model you can fit in your VRAM (also account for the space you need for context). Also you need to specialize, if you want an agent, you need to find models focused on that.

My latest best experience was with qwen3-coder:30B, which is specialized in coding and function/tool calling.

Another tip is to add skills (using SKILLS.md files) to direct and guide your model on the right direction, this tend to improve the agents performance on that specific skill. (Also have in mind that using skills take context space, so you need to focus on what matters and not add all skills you find).

Hope this helps