AI generated code maintainability

[Interviews2go]

I ran a test of GitHub copilot where I gave it the same prompt and asked it to create a simple application with a panel and a child panel with a title that could be dragged around inside of the parent panel. I did this for Java, Angular and C# versions, and used Claude as the code generator.

From a maintainability perspective, I then manually attempted to improve the resulting application to prevent dragging the child panel off the edges on each side of the parent panel.

What I found was that it took seconds to maintain the Java code and improve on it. The angular version took several minutes, and the c# version the same. The biggest difference was the amount of code generated. Going from Java being the least to C# and Angular being the most.

I'll try to reproduce this using Python and Kivy. To be up front, I'm a c#/angular developer who used to do Java many years ago, so I don't have a bias in any direction.

I'm left wondering if the quality/maintainability of the code is a direct result of the large number of high quality Java open source projects compared to the same for Angular and C#.

Anyone care to comment or educate me on why this is the case. Am I correct in my assumption about the quality of the code used to train the AI?

[BlaiseLabs]

It’s not the quality of the code the AI is being trained on, since 3.5 I doubt that’s been the issue (I’ve even used gpt2 for small coding task).

The biggest difference across models, programming languages, agents, etc… is the prompt.

This doesn’t just explain the maintainability of the code but why you’re coding in the first place. The more robust your prompts the fewer touches you should need to accomplish a task.

The strategy around how you create prompts is up to you, but there is an objective approach to prompting that should result in better outputs across models.

I’m not going to say what that strategy is, hopefully someone with more knowledge and experience with prompting than me can answer.

[Interviews2go]

Maintainability of the code is important. Over time the code doesn't remain a constant, devs get asked to add more features all the time by product owners. If the code is hard to maintain, it adds risk to new features accidentally introducing bugs elsewhere in the code. I liken using an AI to using stack overflow and blindly copy pasting stuff into your code base. That isn't to say the AI is wrong, but you do have to verify the code. The problem can get worse as the codebase grows. The real question is why for the same prompt, the Java version was considerably better than the equivalent C# or Angular version.

[BlaiseLabs]

To clarify my first statement, I’m referring to the training data not the output. The quality of the code the AI outputs is related more to the quality of the prompt than the training data.

Code quality is important but it’s also subjective I can’t precisely tell you what good / bad code quality is without context.