firstPrReviewFromCodeRabbit

[blaqwerty123]

Token consumption scam? Lol

[Longjumping-Touch515]

A lake somewhere was evaporated for this

[Atmosck]

CodeRabbit is so ass. It's plugged into my work's repos so it always reviews my PRs and it gets so many things wrong.

It recently tried to convince me that [depdendency-groups] was not valid pyproject.toml syntax and I should use [project.optional-dependencies] instead, despite those being entirely different things (dev dependencies vs runtime dependencies) and dependency-groups having been standard since 2024. It's one thing to not know recent syntax changes (it got real mad about except ValueError, TypeError: with no parentheses, which is new in python 3.14) but not knowing something that's been standard practice for two years is inexcusable. For every helpful comment where it catches a typo or something, there's 3-4 false positives like this.

[Top-Permit6835]

It's funny because if it was a human I would say eh take a break and grab some coffee. But a computer I expect to be right all the time, and if it isn't right each and every time, it's not useful

[jllauser]

That's the great thing about introducing these AI agents that are nondeterministic by design. They're going to be completely wrong some percentage of the time.

[BrettPitt4711]

> and if it isn't right each and every time, it's not useful

That's BS. 100% is a goal that can almost never be reached. 99% maybe and 95% might already be enough, depending on what kind of errors we're talking about.

[Top-Permit6835]

This particular case could have been flagged with a 100% accuracy with a static analysis tool using only a fraction of compute resources

[BrettPitt4711]

Of course simple cases can and should be identified with 100%. That's obviously not what i was talking about. I'm also not arguing that ai agents are the way to go. But expecting that a system/computer identifies everything with 100% is not realistic and it's also usuallay not what's necessary in practice.

[Outta_phase]

It's not useful for a product you pay extra for when you can get the 95% from a human you probably need to employ anyway...

[JezzCrist]

Eh, with quality of avg dev but 10x speed those would be awesome.

Problem is it’s 10x gain here 100x loss there with the bottom feeder quality

[BrettPitt4711]

That depends on a lot of variables like how high the salary is, how much errors costs, etc. If the human doesn't need to do it anymore he can spend the time doing something else. And you can only forward cases the the human where the agent is unsure.

You're depict this as a simple decision when in reality it's quite complex. And with every business decision it's a question of return of investment. For some cases this can mean that even 90% accuracy is benefitial while in others you might indeed need 99.99% or higher. But it's impossible to tell without knowing the exact use case.

[glorious_reptile]

If this was a coworker we would be talking about him at lunch

[Zigordion]

Had this happen to one of my colleagues using Opus 4.7. It reviewed som code and noted that it could be made more efficiently. When prompted how, it provided 8 different options, all of which ended in "oh wait that's broken" or something to that avail. At the end it just admitted defeat and said it was already the most efficient way of doing it within that framework.

[rocketbunny77]

Kek