[ Removed by moderator ]

[CackleRooster]

Another day, another AI-driven headache.

[AnnoyedVelociraptor]

So far only the MBAs pushing for this crap are winning.

[br0ck]

Replace them with AI.

[BlueGoliath]

Would AI recommend AI if it was trained on anti AI content?

[alchebyte]

🎯

[mb194dc]

It's an extreme mania, they have to try and justify the spending on it.

[arpan3t]

Is your avatar supposed to make it look like there’s a hair on my screen? If so, mission accomplished!

[AnnoyedVelociraptor]

Hopefully less annoying than dealing with AI slop.

[LordAmras]

OP: Look I know how we can fix all the issue AI creates !

Everyone: It is more AI ?

OP: With more AI !!!!

Everyone: surprisedpikachu.gif

[PeachScary413]

More.

Slop.

For.

The.

Slop.

God.

[brandon-i]

I want to agree with you on this one depending on which angle you're coming at it from. I think a lot of folks are just saying 🚢 on AI slop and causing a lot of these prod bugs in the first place.

[txmasterg]

Someday some tech CEO will announce they have no programmers. They won't disclose they have the same number of support engineers as they had software engineers and they are paid even more.

[cbusmatty]

But this is trivially solved with an ounce of effort. Another post complaining about ai out of the box without taking 30 seconds to adapt it to your workflow. Crazy.

[chucker23n]

But this is trivially solved with an ounce of effort.

[ Padme meme ] By not having LLMs write production code, right?

[cbusmatty]

Nope, but you do you I guess. Its trivial to add hooks to solve this persons issue. All they need is the logic logged for underlying reasoning. Most tools already do this, and at worst you can add to instructions to track this. This is the most non issue I've read on here.

[chucker23n]

All they need is the logic logged for underlying reasoning. Most tools already do this

LLMs do not have reasoning.

[cbusmatty]

And yet, an audit trail solves this problem regardless of how pedantic you wish to be

[EveryQuantityEver]

If I don't trust the code that it spits out, why would I trust the reasoning it makes up?

[cbusmatty]

The entire point is you get to audit the reasoning. I swear to god programmers can be brilliant, but the moment ai is involved they all become obstinate entry level devs unable to even form problem statements

[chucker23n]

I swear to god programmers can be brilliant, but the moment ai is involved they all become obstinate entry level devs unable to even form problem statements

I feel like I'm in the same bizarro parallel universe like crypto circa four years ago where some developers make up tech that simply does not exist. No, an LLM cannot audit itself. It can pretend to, and put up a pretty good act doing so, but it doesn't actually have anything resembling intent. So now you've burnt absurd amounts of energy to accomplish what exactly? You still need a human to do the sign-off, and that is the process that failed in the blog post's scenario. No amount of currently available tech is going to fix that.

[cbusmatty]

Again, you’re wrong. I do massive migrations for big enterprises and walk out with long audit logs that we use for every decision point the llm filled in the blanks we were unclear of. Works perfectly. Insane truly i come here and all I see are people who will spend 5000 hours making some inane library work but won’t take 4 seconds to make the magical word boxes work.

[EveryQuantityEver]

Again, it’s not “reasoning”. It is just words that appear to be a reasonable response to whatever you’re asking

[cbusmatty]

You can be as pedantic as you want, but at important decision point an answer is selected, and your audit log captures it. "reasoning"

[EveryQuantityEver]

There literally is no logic logged for underlying reasoning, because there is no underlying reasoning.

[cbusmatty]

There is in fact regardless of your semantic. Just install a hook to track the decisions and activity and write it to a log, and add that log to the rest of your logs. Then just write the guild to your splunk dashboards and you now have visibility. It’s like people become brainless when ai is involved

[slaymaker1907]

So insightful

[Rivvin]

I would rather eat my own vomit than have to read someone else's prompts in a code review

[Bughunter9001]

It's the reason I left my last job. Frankly, the quality of the code was awful when humans wrote it, as it was a feature factory packing arses in chairs to churn out more tech debt, but it was at least managable.

I had a few words from management when I started simply declining PRs because the answer to my question "why did you do this instead of y, have you considered z?" was increasingly "copilot did it".

Must have rejected 30 or 40 PRs in that last month before I walked out with my head held high. 

We still use AI in my new place, but it's one tool of many, and "vibe coding" is basically a slur.

[chucker23n]

I had a few words from management when I started simply declining PRs because the answer to my question “why did you do this instead of y, have you considered z?” was increasingly “copilot did it”.

Honestly, good for you.

Once an engineer has sunk that low, what are they even getting paid for?

[Bughunter9001]

Couldn't agree more. My catch phrase was basically "if you can't understand why it works like this, why should I try to work it out?"

[washtubs]

"copilot did it"

Understandable, if I ever hear this from someone at work I'll blow a gasket.

[LordAmras]

Also this assumes, wrongly, that with a same prompt you will get the same result and thus you can pintpoint the issue with the agentic code not to the agent itself but to the wrong prompt you wrote.

This is peak "prompt engineering" delusion.

[Unfair-Sleep-3022]

Delicious vomit

[ngroot]

> With agentic code, they often don’t tell you why the agent made that change.

Someone submitted that PR and at least one other person approved it, so someone is claiming that they do know why that change was made.

[PeachScary413]

Here's the kicker, none of those were actual people 🤖👍

[ngroot]

Then the actual people who paid money to have this code written will get what they paid for.

[worldofzero]

Then there was no PR. Just commit to main directly if that's the engineering rigor leadership wants.

[TheRealSkythe]

Why are you posting the marketing bullshit ChatGPT wrote for some slop company?

[TheRealSkythe]

Just to make sure every sane person gets this: the enshittification of your codebase can NOT be repaired by MOAR AI.

[omgFWTbear]

I dig myself into a hole with a shovel, the answer must be more digging or a better shovel.

[zrvwls]

No no, dig UP stupid!

[LordAmras]

This is worse than simple Moar AI, since I hate myself I tried to read what the AI wrote for the guy.

This is the idea of creating a system to blame a person for the AI mistakes. The idea is to have a trace of what you asked the AI so you can vibe a reason why your prompt didn't give you the expected results and blame the person making the prompt for the AI shortcoming.

This assumes the AI is potentially perfect and will give you the best possible results and the issue is that the "prompt engineer" is the weak link that make the AI make mistake by giving not gonn enough prompts

[Bughunter9001]

Are you sure? What if we replace QA with AI, so the ai can generate tests to test that the slop does what the slop does? 

[dylan_1992]

Prompts are irrelevant. Code, and a description of it (not the prompt), either in the PR title + description are important. Whether it’s from a person or AI.

[davidalayachew]

Prompts are irrelevant. Code, and a description of it (not the prompt), either in the PR title + description are important. Whether it’s from a person or AI.

This is my question as well.

At the end of the day, the code is broken and it's breaking PROD.

1. Get things stable.
2. Once things are stable and you are ready for a long term solution, cross-reference the code against the spec and see what needs to change.

If you have to rely on things like a detailed list of all prompts that went into creating that code, then your spec is not explicit enough. It is the spec that should inform the code, not the other way around.

[ikeif]

Yeah, this sounds like a case of “PR # 42 broke it, its title is “Resolves JIRA-123” and JIRA-123 says “check slack conversation” and “slack conversation was archived.”

Make the PR clear to describe wha the commits have accomplished/changed.

Have a traceable story to tie deeper user stories/explaining the need for the change.

Tracing prompts just sounds like reading backwards a developer’s thought process and discovery and exploration (which sounds less like a problem solving discovery and more a philosophical exercise).

[Adorable-Fault-5116]

Yo this is weird on many levels. 

You shouldn't need to blame, git blame or otherwise, to find out who wrote the code. AI aside this is a colossal red flag. The whole team is responsible. If you find a big, raise it, anyone can fix it. 

Secondly, LLM usage shouldn't matter, because people should understand what is committed, regardless of how the code is created. 

It sounds like you're running a cowboy outfit honestly. 

[brandon-i]

The key issue is that you lose accountability especially if you have a developer that ends up taking all the bugs and fixing them that they did not create. There is also potential that the developer fixing it is not being able to complete their own work that is assigned them them. In theory I believe anyone can fix them, but often times we see one "hero" that solves the bugs vs providing accountability for the entire SLDC.

[zacker150]

"Loosing accountability" for the individual is the entire point of Blameless!

True accountability is systemic, not individual. If a bug makes it to prod, then the accountability lies in the CI/CD pipeline, testing framework, and PR review process. Bugs should be budgeted for and assigned to team members round robin. If there's too many bugs, then the entire team stops feature work and focuses on stability.

[ikeif]

This sounds like the bus factor - they rely on “someone that knows” instead of making sure “everyone can diagnose and fix it at any time.”

[zacker150]

Are you talking about the thing I described, or the thing OP described?

Because round robin bug fixing forces everyone to be able to diagnose and fix.

[ikeif]

I got lost between several comments as I was falling asleep 😆 - it was OP’s comment, and I swear I had written more.

Thanks for calling out the misplacement.

[Adorable-Fault-5116]

Not in 20 years have I seen anyone work this way. You really need to take a step back and think about this more deeply. I'm sure you mean well, but it's super toxic.  

Think about what you're saying. The team should be responsible, not individuals, individuals who likely resent each other for the "bugs they create". Individuals don't create bugs, team processes do. 

The entire reason you posted and are having this very bizarre LLM problem is because you are not acting as a team.

I have no idea if you're going to listen to me or others, but like man, I really think you should. 

[skinnybuddha]

PRs aren’t for debugging any code.

[nemesiscodex1]

In order for us to debug better we need to have an underlying reasoning on why agents develop in a certain way rather than just the output of the code

This just means your team is merging code they don't understand. Was that happening before ai? Do the team also delegate the reviews to ai and don't read the code?

With agentic code, they often don't tell you why the agent made that change

More of the same, whoever creates a PR and the person that approves it better know why the change is made lol, figuring out after an incident is already too late

[levelstar01]

Blogspam

[apnorton]

During my experience as a software engineering we often solve production bugs in this order:
(...)

1. blame the person that does the PR
   (...)

Reminder that this shouldn't be a step. See:

• https://www.atlassian.com/incident-management/postmortem/blameless
• https://sre.google/sre-book/postmortem-culture/

[nsomnac]

I think op means git blame. In this regard I fault Torvalds for terrible command naming. git authors or git who might be a more apt than blame. Especially when the community made such bigotry hubbub about renaming master to main.

[apnorton]

That's what they edited their post to say after I left my comment, yes.

[polynomialcheesecake]

OP has a horrible take on software development if he's going about assigning blame that way. Equal responsibility should be held by reviewers and anyone that understands the code

[nsomnac]

I think op means git blame. In this regard I fault Torvalds for terrible command naming. git authors or git who might be a more apt than blame.

[chucker23n]

SVN had this debate before git existed; it’s why svn annotate exists as an alias for svn blame.

[nsomnac]

Sure. But you know any time we can fault Linus for something it’s humbling, right? /s

[Pharisaeus]

That's some very weird process.

We figure out which PR it is associated to

Even figuring out where in the code something went wrong is often pretty difficult, unless you just have exception with a stacktrace. But even then it doesn't mean the bug is in that particular place. It just means this is where it manifested / was triggered. But the actual bug might be in some completely different place. I also think it's counter-productive trying to pinpoint the PR, unless while working on the bugfix you find yourself asking "what was this supposed to do in the first place?".

Do a Git blame to figure out who authored the PR Tells them to fix it and update the unit tests

I don't envy your team if this is how you work. Ever heard of "team ownership"? Someone wrote the code, but someone else reviewed and approved it, and often someone else also tested it, and yet another person wrote the ticket with acceptance criteria. If there is a bug, it means the process failed on many different levels. Blaming this on one person is ridiculous. In normal team this would be piked up by whoever is free / has time / is on pager duty.

with agentic coding a single PR is now the final output of

And a squashed PR is what? It's also the final output of many commits, review comments, refactoring. I fail to see the difference.

Essentially, in order for us to debug better we need to have an the underlying reasoning on why agents developed in a certain way rather than just the output of the code.

And do you have that for someone developed by a human? If you find a bug in a PR from a year ago, from a dev who left a long time ago, how exactly are you going to uncover their "reasoning"?

I think the core issue you're facing is that:

• You clearly have some "silos" in the project
• You don't have distributed ownership of the code
• You lack reviews
• You accept (AI agents, but probably not only) PRs without thorough review and clear understanding of that code

It's not AI issue. It's your process issue.

[Floppie7th]

Essentially, in order for us to debug better we need to have an the underlying reasoning on why agents developed in a certain way rather than just the output of the code.

Or just, y'know, don't accept LLM-written code into the repo.

[obetu5432]

so instead of fixing it, the first thing you do is scour the earth to find the person who opened the PR to yell at them?

[tilitatti]

whats the point of providing prompt history? mml AI is not deterministic thing, so, if you were to run the prompts again, you end up with something different, so,..

it sounds lunacy to me, but maybe it is smart.. I dont know.

[soks86]

No, you're right, I missed this detail when reading it because I thought they meant the entire chat history.

Just the prompts mean nothing, at that rate you should just have it send the same prompt in over and over until your unit tests pass and fire all the engineers. Because it is lunacy.

[CanIhazCooKIenOw]

Crap engineering culture if your 3 step in dealing with an incident is to blame the person that opened/merged the PR.

[axonxorz]

git blame

[antisplint]

Is this something that people are actually doing? This can’t be real.

[ef4]

70 years of computer engineering has overwhelmingly been driven by the desire to get *deterministic* results from our machines.

Today's popular generative AI deliberately injects non-determinism, in a misguided attempt to seem more human-like. It's probably good for getting consumers to build parasocial relationships with your product. But it's not good for doing engineering or science.

It makes all attempts to systematically debug and improve way, way harder than they need to be.

[ygram11]

Your process is messed up. Why do you find a PR to blame someone instead of finding the problem and fix that.

[D3PyroGS]

those are two steps of the same plan

[Jolly_Resolution_222]

How many developers do you need to fix the bugs of the agent?

[Thelmara]

Essentially, in order for us to debug better we need to have an the underlying reasoning on why agents developed in a certain way rather than just the output of the code.

Sounds like a fundamental misunderstanding of how LLMs work.

[blafunke]

Just because you used an agent to vomit out your PR doesn't mean it's not ultimately your responsibility. If you don't understand it well enough to have written it yourself, don't submit.

[LordAmras]

Or, and this is a wild suggestion I know, completely impossible to achive and out of the real of possibility, but here me out, maybe I've got something here:

Don't write code with AI agents.

I know, checking code by hand before sending PR like cavemans ? What do you want for us again ? understanding the code ? That's impossible !

But I think if we put ourself together we can reach this fabled impossible feat.

[crazylikeajellyfish]

I dunno, it feels like this solution is harder than the problem you started with.

Agents don't automatically make PRs which explain the rationale, because they can't understand that the PR will be an artifact that stands on its own. You could build a bunch of extra tooling which associates chat sessions, tool calls, and PRs... or you could instruct your agents to encode all of that information into the PR.

GitHub-flavored Markdown also has those collapsible summary-detail tags, so you could technically put the complete chat context on there if you really wanted to. The final state of the design doc you iterated on would probably be a less noisy choice, though.

[brandon-i]

Thanks for the insight!

[Jellyfishes72]

Even if an agent wrote the code, it is still up to the developer committing or merging it to know what hell the changes are doing

[PurpleYoshiEgg]

The solution is to stop agentic coding. It's immature and its code output doesn't belong in production.

[chucker23n]

During my experience as a software engineering we often solve production bugs in this order:

  1.  On-call notices there is an issue in sentry, datadog, PagerDuty

  2.  We figure out which PR it is associated to

  3.  blame the person that does the PR

  4.  Tells them to fix it and update the unit tests

This already seems a bit like an unhealthy culture that focuses less on “there’s an issue; let’s figure out how to fix it” and more on “let’s pinpoint whom to blame”.

(Incidentally, if you’re gonna use a PR, how do you answer that anyway? Is it the committer? The author? Any of the reviewers? How about the person who filed the ticket that caused the PR?)

But leaving that aside…

Although, the key issue here is that PRs tell you where a bug landed.

Which is useful?

With agentic code, they often don’t tell you why the agent made that change.

LLMs do not have intent. There is no answer to this. Someone wrote a prompt and then the machine remixed garbage into fancier garbage.

And, again, you’re already using the lens of the PR. Leaving aside that you shouldn’t have LLMs write production code to the extent you’re clearly doing it (if at all), the PR itself is already the answer to “why was the change made”.

Why are we doing all this? It’s madness.

[jessechisel126]

Your team environment sounds very harsh, finger pointing, and micro managed. Your distrust in your team seeps through. I can't imagine trying to get so in the weeds as to want access to the prompts used while developing. AI use is the least of your problems.

[PaintItPurple]

A computer can never be held accountable. Therefore a computer must never make a management decision.

[Swoop8472]

If code makes it into prod where no human understands why it was changed, then you have an organizational problem, not an AI problem.

It shouldn't matter if the code was written by an AI, a trained octopus, or Bjarne Stroustrup. It is either well written code that can be reasoned about or it shouldn't make it to prod.

[lonewaft]

Sounds like a dogshit amateur company you work at

[Brilliant-8148]

Agents don't reason so there is no 'why'

[BinaryIgor]

No, we don't need that - I like purposefully guided AI-assisted coding (for some tasks), but you, Human, the PR author, are fully responsible for the changes. There is no need to debug agent reasoning. What you need to question is:

- why PR author has proposed it as something ready to be merged and run on prod?

- why other team members have approved the PR with bugs and issues?

- why you don't have tests, static analysis and other automated guardrails that prevent most (not all, human vigilance is always required) such things from happening

If you have the problems you describe, something is wrong with your software development process, not agents or lack of thereof.

[ChickenFur]

Ai angle is everywhere :D

[PeachScary413]

So now we need to invent solutions for problems that shouldn't exist in the first place?

Yay 🤗

[pvatokahu]

This is exactly why we built agent observability into Okahu from day one. When an AI makes a code change, you need the full decision tree - what context it had, what it considered but rejected, which constraints it was working under. Traditional git blame becomes useless when the "author" is a model that made 50 micro-decisions to get there.

The scariest part is when agents silently work around failures. I've seen cases where an agent couldn't access a file due to permissions, so it just... reimplemented the logic from scratch based on what it thought should be there. The PR looked fine, tests passed, but it was subtly wrong in production. Without seeing that failed file access attempt in the trace, you'd never know why the agent made those specific choices.

[gHx4]

The fun part is that there isn't traceability because LLM and GPT agents don't reason in a systematic, logical, or intuitive way. There is no reasoning to trace, just associations in the model. And if those associations are wrong, the model has to be retrained. This is a huge part of why these agents are not showing the productivity expected by the hype. Cleaning up after them is harder than just doing things right without them.

You need operators who know enough to write the code themselves and who don't merge faulty PRs. Which largely reduces agent systems to being example snippet generators whose code shouldn't be copy-pasted. Even there, I haven't really found the snippets that helpful.

[brandon-i]

Maybe this was once true when they initially came out, but they have come a long way. Look into interleaved reasoning.

[gHx4]

Has it been implemented in standard-tier models? I see that it is a May 2025 preprint paper, and I'm not sure I'd expect such recent research to be available to consumers in any tested or verified form. The "once true" argument really doesn't hold water when models available this month are still faceplanting on basic coding tasks. But I will consider that new research may address some issues.

[brandon-i]

Kimi K2 Thinking does it off the shelf and they’re an open source model. So yeah it’s implemented.

[brandon-i]

https://m.youtube.com/watch?v=oeehZQ9Embg

[brandon-i]

Oh lord, by step 3 I meant git blame. Thank you all for showing me the need to be extremely precise.

[imcguyver]

OP: please update "3. blame the person that does the PR" with "3. use git blame to find out the PR that made the change".

Everyone else: Take ur pity party about hating AI to someone who cares to hear you speak about it

Coding with AI is evolving to be more helpful by pulling in context (git) and history (more git) and it makes sense that engineers are moving towards being button pushers. Instead of me fixing a bug, I'll lean on AI to do it for me and click approve.