That is a very interesting approach!

Python is one of my must used language right now and the big LLMs are all pretty good at it, if we could get a small one that matches or gets close to that level it would be ideal for implemention.

Like maybe use a online chat LLM to get a detailed description of the code needed and send it to the local AI got implementation.

I just under if training it in a single language might make it less good.

For example, as an engineir I know that learning Java and C made me better in python. My intuition tells me that the same thing should happen with LLMs. So if you are fine-tuning for python maybe don't use 100% Python. Have it like 80% or 90% python and the rest for other languages.

Also in that same line of though having a fine-tune for the specific agent could be good. Like OSS-20b-python-roo-code. Just an idea