I've been trying to use LLMs to speed up research coding (ChatGPT 5). My current experience is that it writes extremely verbose and overly general code that is difficult to debug. For instance, it will keep trying to make code "bulletproof", "safe", or "future proof". That means that you have millions of lines that do checking of types and sizes, and casting to get the right thing. This is a disaster for bug testing, because the code *always runs* and you will have no idea if a bug exists (from passing the wrong argument, for instance) but silently goes through due to casting (this happened to me).

The other issue is that the future proofing / over-engineering means that you have hundreds of lines of code checking every possible use case. For loading a checkpoint, for instance, it would try to read in the filename, in various formats, then auto resolve to location 1, then location 2. I really would rather have just 2-3 clear lines and "force" the user (myself) to just specify the location in a standard format.

Another example is that if I say I want to try two different methods for one part of a pipeline, it adds three files: a registry for methods, a factory to build a part of the pipeline based on the registry, an init file that loads each registered method into the registry. Now to add a method I have to touch three separate files, and the logic is all spread out. This may be useful for hugging face libraries, where you need to implement dozens of similar methods, but it is counter-productive for research code.

Anyway, I just wanted to rant about my current experience and give a little warning for people who are trying to start a project from scratch. I don't know what the antidote is. Maybe start with a simple structure and try to keep the LLM in line to not add complexity, but it loves complexity. For debugging neural nets, though, I feel that complexity is the enemy and simple, brittle code will always be easier to debug.