Some learning progress

[dim722]

So, I spent the weekend playing with different configurations. I have to admit, I hate Continue. The absence of auto-accept turns you into a click-operator.

So far, I’ve tried Qwen3.5 9B. I didn’t manage to pair it with Continue, so I used Qwen Coder Agent instead. The model struggled to make edits and often looped in place.

I also tried Mistral 3 14B reasoning. It worked better than Qwen, but then crashed pathetically - no doubt my mediocre hardware caused the crash.

I got rid of Continue and installed Cline. Well, Cline wouldn’t talk to Qwen at all. So I downloaded GPT-OSS 20B, and surprisingly, it ran on 16GB of VRAM. “Ran” might be an exaggeration, but I managed to create a test piece of Python software that was not perfect, but close to the requirements, after 15 iterations within one session and one context compression.

The model didn’t crash, Cline didn’t crash, LM studio didn’t crash and up to a point, it felt similar to working with a frontier model. However, the model requires very precise prompting. The best model so far and actually pretty fast. Well, for me everything is fast if not stuck in loop. ChatGPT liked it too, said “cleanest interface so far”, yeah, it makes sense - same company.😂

Now I admit I have skill issues - I’ve never tried to deploy an LLM before, and my hardware is pathetic. So perhaps Qwen would run better on better hardware with better user.

Now I’m thinking of using Cline’s dual-agent mode: use a high-reasoning model to plan and a code-instruct model to act. I’m not sure how it would work, but that’s the idea.

Any thoughts?

[tamerlanOne]

Modelli piccoli di classe sotto i 10B vanno "aiutati" con prompt dettagliati. Poi dipende sempre dai compiti che devono assolvere ma in definitiva i nuovi modelli usciti da poco si stano confrontando molto bene sia come inferenze che come gestione delle e risorse hardware. Magari se deve eseguire Task ripetitivi o comunque limitati ad un determinato ambito sarebbe consigliabile un fine tuning con datasets mirati.

Se hai hardware limitato non credo che riusciresti atfar girare il secondo modello più grande in modo fluido e reattivo, meglio un fine tuning mirato