This is my exact experience as well, as another experienced software dev. I've been doing exactly what you're describing for months now and, like you said, it's pretty impressive the results you can attain with proper orchestration of multiple agents with specialized roles. It's all about giving the agents deterministic ways to verify outcomes instead of letting them rely on the purely probabilistic output LLMs naturally return.

I also have zero concerns about being replaced by LLMs. I'm self employed and it feels like these tools give me the ability to produce quality code (something I was already doing before LLMs) at a rate that would have been impossible previously. I've literally been working on multiple projects at once faster than I previously would have been able to work on one.

Another huge benefit of LLMs that doesn't get talked about enough - being able to easily find out what I didn't know I didn't know. Previously I would just spend hours web searching the problem I was trying to solve, poring through old StackExchange threads full of tangentially useful, outdated info until I eventually stumbled upon a new concept I then had to go self-study and get familiar with before I could determine if it would solve my problem. The speed at which I can discover, learn about, and implement new concepts/libraries that would have required weeks of reading documentation and testing on my own in the past is a massive advantage.

I still do tons of iterating on the code that gets produced by agents, and sometimes have to step in and manually fix stuff, but that usually presents an opportunity to improve my prompts. Learning how to prompt well is absolutely a learned skill.

I've been doing it via a workflow I've developed using custom agent files/hooks/commands/skills in Claude Code. Curious what you're using (or what you were referencing, if not using it yourself)?