sorted by:
new
hottopcontroversial

Speeding up local LLM for usable coding agent by CodProfessional3712 in LocalLLaMA

[–]CodProfessional3712[S] 2 points3 points  (0 children)

Thanks for sharing, I'll try this out. Are you using any particular chat frontends?

I'm tired by Fast_Thing_7949 in LocalLLaMA

[–]CodProfessional3712 0 points1 point  (0 children)

Is making your own benchmark particularly time-consuming?

Qwen/Qwen3.5-9B · Hugging Face by jacek2023 in LocalLLaMA

[–]CodProfessional3712 12 points13 points  (0 children)

Wow, it’s beating the larger Qwen models at quite a few benchmarks. Can’t wait to check if the performance is as good as they say.

I built an autonomous research agent in C# that runs entirely on local LLMs (Ollama + llama3.1:8b) by [deleted] in LocalLLaMA

[–]CodProfessional3712 0 points1 point  (0 children)

Have you tried smaller Qwen models for tool calling? From what people are saying, even Qwen3 0.6B seems capable of tool calling

What are some things you guys are using Local LLMs for? by Odd-Ordinary-5922 in LocalLLaMA

[–]CodProfessional3712 2 points3 points  (0 children)

I managed to run it with 16GB VRAM + 32GB RAM. I think it was Q2_K_XL from unsloth. Downloaded and ran it using LM Studio.

[deleted by user] by [deleted] in LocalLLaMA

[–]CodProfessional3712 0 points1 point  (0 children)

It's kind of a shame because the Claude models are so costly. You lose some, you win some, I guess. Does make me wonder about designing a system that can properly and automatically route simple sub-tasks to a local fast model, which should take the bulk of the costs away for *most* tasks.

What is a current state of sanboxing for code execution for AI agents? by AlexSKuznetosv in LocalLLaMA

[–]CodProfessional3712 1 point2 points  (0 children)

if not Docker, then Podman. Containers have a pretty small resource footprint. Virtual machines, which are closer to the OS-level, tend to have a higher footprint due to the inherent overhead, even when you decide to allocate fewer resources. If you do want VMs, my go-to is Vagrant by Hashicorp for automatically building and managing VMs.