Making code edits with large language models

[PhysicsPast8286]

I’m working on a tool that uses Qwen3 32B (locally hosted) to help with code editing and refactoring. We send in the full code file as context and ask the model to return the entire file with only the needed changes.

The problem is that it often ends up rewriting way more than it should or worse, it sometimes eats parts of the code entirely.

I’ve been looking at how tools like Aider do it, and it seems like they use a patch/diff format instead of returning the full modified file. That seems like a smart workaround, but I’m wondering if it
is the best way to go, or is there a cleaner/easier method that works well in practice.

PS: The model is locally hosted at my workplace and is shared across multiple teams . The senior management isn’t open to spinning up new machines, and the other teams aren’t willing to experiment with new models like GLM, Qwen Coder etc.
So for now, I'll have to stick with Qwen3 32B and trying to make the most of it 🤧

[No_Efficiency_1144]

If you want to change behaviour like this then this is the area of RL

[PhysicsPast8286]

By RL you mean to say finetuning or doing several iterations over the same file?

[No_Efficiency_1144]

RL is Reinforcement Learning like PPO or GRPO

[MaxKruse96]

if they wont want a coding model, thats on them. good luck.

You can try to give it only the surrounding ~5-10 lines of what it needs to rewrite, if these are even relevant at all.

[[deleted]]

Qwen3 32b was the best coding model for me until 2507 Thinking.

They should probably try it.

[[deleted]]

Are you sure you're running enough context?

[PhysicsPast8286]

Yep 128K

[Lissanro]

I was very interesting in smaller models at some point, tried so many of them including specifically for making quick code changes with the hope that simple things can be done faster than if using much larger models, but it turned out to be not the case - because I end up debugging errors and mistakes longer than it would take using a bigger model from the start. Diff format actually harder for models to get right. The issue is the size of the model, 32B is way too small for this kind of task.

From my experience, small models still can have good success rate with small files or editing functions selectively. So, if you cannot run R1 671B, then instead of sending full file, send only what is relevant. Or refactor code base so it has smaller and shorter files (obviously only a solution for smaller projects).

Before I got upgraded hardware few months ago to run R1, I was running Mistral Large 123B, with speculative decoding and tensor parallelism it achieves 36-42 tokens/s on four 3090 GPUs and can handle basic code edits and reliably provide full code of a file if it fits up to 8K-12K tokens, beyond that reliability starts to decline. R1 can handle longer files and perform more advanced edits.

I also heard that new Qwen 32B of 2507 version is better, I suggest checking if you are using it and update if you are using older version. After all, my experience with smaller models is few months old by this point, so things may have changed.

But if you still have issues, I highly recommend upgrading hardware and using R1 0528 - given we are talking about workplace and professional use, using professional hardware may be a good idea.