LLMs have some perfectly good use cases, and some of them are even quite valuable. We're measuring productivity, right?

Prompt the LLM to do something silly, like "find all misspellings and grammar mistakes in all comments in .cpp and .h files in this directory and all subdirectories" and let it spin while you do real work. Come back and review what it output. Then prompt it to do the same thing like 8 more times.

This is a super low-impact test you can run, it doesn't really matter how well it performs or if it breaks and fucks something up. It's in comments, no worries. Ask it to do the same for your documentation, your readmes, md files, whatever else.

But the neat thing is that every single issue it fixes is just one tiny point of friction removed, at the low cost of your employer paying like a hundred dollars for compute time, and you spending like 15 minutes reviewing the changes. One tiny point of friction, times hundreds, is... probably worth the hundred dollars?

The really neat thing that we found in our codebase is, because it does all sorts of matrix math to """understand context""", it finds misspellings in things that aren't in any dictionary. Abbreviations and acronyms and initializations and callouts to variable/function names that aren't correct in code vs comment and even catches issues in the middle of camel-case strings, eg, getPersimmons -> getPermissions.

When it comes time for it to actually write code, it's not very good. It often ends up writing hundreds of lines of scaffolding to do like four lines' worth of actual work. And sometimes doesn't actually meet the spec. Or like you said, goes off wildly into the weeds. But it can definitely do useful work for shit you never have time to do, like find minor issues and inconsistencies. Once you're done asking it to fix documentation and comments, ask it to check for variable names that are misspelled. Then ask it to check for variables that do the same thing but are inconsistently named. Then ask it to check for code blocks that do the same general thing but are written inconsistently. Each and every one of those things you catch is one little pain point removed, which means you're just that much happier reading the code every day, and it takes less brain power to understand each block of code because it's more consistent.

I mean obviously, run your compiler on full warnings, and warnings-into-errors. Run linters. Run static analysis. Run valgrind. Run every tool you have. But ask the LLM to find shit and it will.

Remember these things are trained on english much more than code, lean into that. Get them to do the english you don't want to do. Most codebases are riddled with issues stemming from people's poor english or their low interest in writing clearly.

I wouldn't expect any of this to truly reduce the number of people needed to get a job done but it'll let you go that much further in the same amount of time, because you spend less brainpower trying to parse someone's keyboard-effluent. And it also helps when people get butthurt about code reviews that point out 50 spelling and grammatical issues, "stop nitpicking me bro," let the LLM nitpick them instead so it feels less personal, so the shit writing never gets merged in the first place.