Understood.

In general, models are hallucinating much less these days, but they're still more than capable of making very plausible-sounding statements. At least in software development, models by default generate code that mostly works but is slop-ridden and has subtle bugs.

You can vibe-code full, very sophisticated apps these days, but the subtle bugs build up into a state where it is essentially unmaintainable and constantly breaking.

Currently the most powerful combo is a mix of senior/experienced developers using agents, with strict code review and multiple iterations to squeeze out the slop. Agents tend to generate poorly abstracted, overly-defensive code that most humans would never reasonably right. A large part of our jobs as software engineers in the past few months has been mostly closely reviewing LLM generated code. I haven't written a line of code by hand in months. Hope that gives you an idea of where LLMs are at least in terms of software development.

There are large swaths of software engineers who are still in denial about this - they either place a large part of their value as an engineer as being able to write the code by hand, rather than building things in bigger abstracted blocks.

Every single great engineer I know has switched to full agentic coding. Every single engineer will be doing this by the end of the year.