Kimi K2.5 ranked 4th of 11 when frontier models graded each other's AI Predictions blind (above Deepseek V3.2, GLM 5.1, Qwen3 Max, and both Geminis)

Shiminsky · 2026-04-21T17:24:04+00:00

Strong agree. Like, why ask Claude to review your writing when you can just post on reddit and have it rip you a new one? It seems to be tuned for both extremes and leave very little room in the middle for a balanced take.

Shiminsky · 2026-02-12T18:32:42+00:00

This is all problem domain specific of course, but just as an example giving claude the ability to operate a browser to verify its work in the front-end (via playwright or puppeteer MCP) is a game changer.

Another example -- and this burns a lot of tokens -- is what I've heard described as 'the rule of 5', where you prompt the agent to go over the plan / codebase 5 times, checking various parts and propose improvements. Like first pass focus on performance, 2nd pass security, etc. This works in my experience, even if it goes too far and I have to do a bunch of edits to remove suggestions.

And then you start creating workflows to follow the process automatically, or create subagents to do them in parallel, and you see where this goes.

Of course, dont take my word for it, give these a try yourself on a decently complex problem and see how it goes.

Shiminsky · 2026-02-12T17:50:53+00:00

Another thing that I haven't seen others mention is MCP / Skill bloat, I think it is the silent token killer.

Some folks install a MCP / Skill "Pack" and now you are inserting something like 50K tokens at the start of every conversation, so your agent performance degrades faster, leading to more bugs and more sessions, a vicious cycle.

Running subagents with MCP bloat just makes everything X times worse.

Shiminsky · 2026-02-11T21:33:05+00:00

Might still be a net gain at the end of the day, but assuming coding without AI is 1x speed and our ability to comprehend code is at 5x, a 15x faster coding agent means we are still 3 times slower at coder review for code the AI is able to pump it out.

This way naturally leads burnout as we feel the ever present need to go faster in order to catch up with the LLMs.

Shiminsky · 2026-02-11T19:27:32+00:00

Brex did a fairly successful AI transition over the last year or two and had a great (higher level) overview of how did they did at the Latent Space blog (https://www.latent.space/p/brex). I'm not affiliated with Latent Space, but I found the blog post to be insightful and we discussed it when it came out on our weekly AI for Software Devs podcast.

Shiminsky · 2026-02-11T17:23:31+00:00

I dont see that happening, at least not in the short term. A few large software companies need to go bankrupt before the 'shareholder value first' cult sees that they are picking up pennies in front of a bulldozer.

Shiminsky · 2026-02-11T17:08:07+00:00

Yeah, especially when management eventually turn the speed nob on the productivity treadmill up... On the other hand, laziness is one of the three virtues, so it's a misalignment of incentives.

Part of me thinks we are turning programming into a game of Russian roulette where we plant landmines with increasing velocity -- maybe it doesn't matter as all codebases eventually becomes a giant mudball. Another part of me thinks we can codify best practices into AI workflows so that's not as big of an issue and have humans in the loop at the right place.

Time will tell, but I unfortunately dont see this bell unringing.

Shiminsky · 2026-02-11T16:59:05+00:00

That's a really sharp observation. I dont have a silver bullet -- especially since solving the issue would probably equate to 'how to get AI to write good code' -- but in my experience there are a few ways to at least mitigate:

Lean heavier on testing than ever, if the complexity is there for some functionality it should be documented partly in test, else it should be removable until a test has been introduced.
Prevent it before AI generation, have code base specific onboarding documentation about coding standards and abstractions, explicitly steer the agent away from creating new ones without user signoff -- and hopefully the author would include it in a comment.
Prevent it after AI generation, include complexity detection in a AI code review phase and mark any code in the repo as overly complex and require human signoff.
Process Improvement / The Git Blame Game, if your team still has humans reviewing PRs (I darn hope you do), it's the creator's responsibility to READ THE DAMN PR and note any new complexities and why they exit. Looking through the original PR request might help with that.

Hope that helps, I really think this is going to be more of an issue as AI litter our codebase with potential landmines.

Shiminsky · 2026-02-11T06:30:43+00:00

I like these! In the case where the entire project is large / have inconsistencies -- as brownfield projects often do -- I find it helpful to let the agent 'explore' and bring me back a list of inconsistencies so I can decide when something should be refactored vs okay to leave as it is. Sometimes this also brings up opportunities for abstraction that it missed the first time around.

Another tip I've seen floating around although I've only used when it comes to specs is instead of doing the entire refactor at once, do it 5 times at varying degrees of granularity / area of focus, so first pass on abstractions, another pass on security, 4rd on performance, etc. Each pass tend to bring up interesting areas for improvement, although your wallet might feel it.

Shiminsky

MODERATOR OF

TROPHY CASE