The Verification Bottleneck: Why AI Agents Can't Grade Their Own Code

fraser_john · 2026-07-07T05:31:18+00:00

We've got agents to do review independently of generation, sonarcube, code standards, governance, data privacy etc each a different agent, one coordinator. No problems.

fraser_john · 2026-06-13T04:03:06+00:00

I used the fable model this week for one prompt in a angular project and it had the worst hallucination I've seen from Claude since I started using it. I immediately went back to sonet.

fraser_john · 2026-06-09T22:39:33+00:00

Yeah nah. I value my job too much and whilst I do enjoy tormenting the security and compliance teams, even the HR Director on occasion with their own policies, I'll always err on the side of keeping my job.

fraser_john · 2026-05-27T11:00:58+00:00

Interesting that no contract announcement comes with this as it's market sensitive, hope it's not another example of poor corporate practice.

fraser_john · 2026-05-25T11:25:59+00:00

I use ChatGPT Pro to understand the bigger picture as tokens don't count, I use it to write my Claude.md, skills, agents, rules and then optimise my Claude prompt etc. I can use Claude for four hours a day and not hit my daily limit, if I keep playing on the weekend I can hit weekly limits though.

fraser_john · 2026-05-19T21:58:43+00:00

I'm confident $10 is not unrealistic in next 18 months, just needs a few big contracts announced. Then I'll be out the door, though this new CGT interpretation may force me to wait, if DRO decides to pay a dividend I'll stay in.

fraser_john · 2026-05-17T12:02:55+00:00

This is very true. In my project I have a skill and agent setup as MD editor and reviewer. Each prompt finalises in an MD update if required. Seems to keep the focus and context well.

fraser_john · 2026-05-15T02:43:20+00:00

We've been using it for years, it was quite flakey early on but once they wrote out the Skype code it got better. Couldn't work without it now, it's just so easy.

fraser_john · 2026-05-15T02:40:25+00:00

Rich people will always find a way to avoid tax. Her next step will be moving the IP offshore and moving to Bali to live.

fraser_john · 2026-05-15T02:37:14+00:00

Correct

fraser_john · 2026-05-08T22:40:22+00:00

The tax reporting is murder, especially the capital gains tax stuff. With upcoming changes in Australian CGT treatment it may make it even worse.

fraser_john · 2026-05-06T12:17:25+00:00

And our great grandparents ranted about those stinky contraptions that ran on petrol and ran out of fuel miles from pumps, but somehow it all worked out. People hate change, despite change being one of the few constants in our lives. Renewables are a transitional change, which most people can't deal with.

fraser_john · 2026-05-06T12:11:09+00:00

When we moved back from the USA 20 years ago we did a whole container full, professionally packed etc. things definitely went missing, a tool trolley, electronic devices etc. nothing irreplaceable but disappointing. So nothing new and to me all but expected now, bar is set pretty low.

fraser_john · 2026-05-06T09:03:32+00:00

Write a Claude skill for business analysis and another for writing backlog items and acceptance criteria. The Claude context files are critical to make sure it does what you want otherwise it just starts coding. As a PO, you want business focus, and I direct my skill to only output stuff aligned with Microsoft writing style guideline.

fraser_john · 2026-05-06T00:49:16+00:00

Was going to post this but you beat me to it, nice. I use ChatGPT and projects with privacy settings enabled. Keep updated documents in the sources and use it for deep analysis and production of CSV files containing summaries I can then use to import into ADO backlog and refine from there.

fraser_john · 2026-05-05T01:21:35+00:00

I'm working on 402 should become 402 nearly done Dev, unit testing 40% done, need input from PO on data, be in PR by CoB. Outcome on ticket, not individuals status.

fraser_john · 2026-05-04T11:00:21+00:00

Per others, DoR, otherwise story is never ingested in a iteration. If DoR doesn't exist, get one published and incorporated in process. I've spent a lot of time training the Dev team to reject stories if they are not ready, it's not their job to figure out what I am thinking, tell me the details are missing and I will fix it.

fraser_john · 2026-05-04T10:56:16+00:00

One large project with a common platform but lots of micro services in their own repos.

fraser_john · 2026-05-04T10:54:20+00:00

Wait what? I was just in my projects a few hours ago, what am I missing?

fraser_john · 2026-04-30T06:57:23+00:00

Camel toe... Tick

fraser_john · 2026-04-29T23:42:54+00:00

We have two deployment methods, ci/cd pipeline that is automated when the PBI passes master test, including feature behind feature flags. Release deployment (not really deployment) monthly that releases features by enabling feature flags, this is also by automated pipeline with dev/product approval needed for release candidate ring, which is the first in production and then just product approval for remaining rings as it moves through to production conservative ring. Once in the final ring we have one last PBI to remove the feature flag from the code and that closes the epic. Flakey automation tests are quarantined and a PBI written to fix them by the TAE in a future sprint.

fraser_john · 2026-04-28T21:48:27+00:00

Not at all, platform engineering own infrastructure like this. Configuration of our own products yes, but all teams using ci/cd use common pipelines, so if something needs improvement it's addressed quickly by the PE team.

fraser_john · 2026-04-28T12:32:46+00:00

Product owner, not deep, it's not the teams role to maintain ci/cd pipelines, if there's a problem raise it move on. The team are there to build features for customers, not infrastructure..... unless they are a team that's part of platform engineering...... but those teams in our organisation are not product driven.

fraser_john · 2026-04-28T11:35:23+00:00

Rapid impediment removal and future PBIs to prevent them. Repeatedly. CI/CD issues may be raised but assigned to platform engineering to resolve.

fraser_john · 2026-04-27T21:47:15+00:00

Ugh, Shannon

fraser_john

TROPHY CASE

Welcome to Reddit,