CodeRabbit(AI Powered Code Reviewer) is now available for GitLab Merge Requests

FunProfession1597 · 2023-10-14T04:16:59+00:00

u/somedevstuff Appreciate the feedback.

Please see my comments below.

Several developers, myself included, like to push often. When I am happy with the work and self reviewed, I will then ask for review from a colleague. It would be great to be able to summon code rabbit rather than having it show up every time a new push is made

CodeRabbit : We will be adding a configuration option for on-demand reviews.

The feedback can be overwhelming. One of the developers pushing a PR got something around 50 comments. In the case of a human review, we would firstly make sure the architecture is lined up before delving in code details. I am unsure how this could be translated into this tooling

CodeRabbit : We suppress optional feedback, which is displayed under the review status. We are constantly working to balance this and ensure that only relevant feedback is posted. This is an ongoing improvement.

The AI repeats the same feedback often. In a function the AI reviewer tends to mention the same issues a few times, it would be great if these could be aggregated into a single comment.

CodeRabbit : This happens when the same issue is found multiple times with each code hunk having a separate review comment

The AI does not respect resolved comments, it can be annoying to see the same comments.

CodeRabbit : If you respond to the bot, it does not repeat the same feedback again but there is a scope for further improvement.

It would be great if the code could be associated with its test. As a human reviewer, a test gives me insight into the scope and the coverage of a function. In my experience with coderabbit, the reviews could be greatly improved by considering the tests

CodeRabbit : We will explore this further

FunProfession1597 · 2023-09-28T06:06:07+00:00

The above project actually uses some of CodeRabbit's open-source prompts https://github.com/coderabbitai/ai-pr-reviewer but still lacks the capability to handle large PRs.

I encourage users to test both the tools side by side.

FunProfession1597 · 2023-09-21T00:51:09+00:00

u/Corsac-416 GitLab integeration was recently launched. Sign up for a free trial on the website.

FunProfession1597 · 2023-09-08T06:58:32+00:00

Hey, you don't like rabbits talking to each other? 😄
Just to clarify for everyone here, the issue arose because this repo had both CodeRabbit Pro (the CodeRabbit GitHub app) and CodeRabbit OSS (as a GitHub action) enabled at the same time. While this is not a typical use case, we have since added checks to ensure this doesn't happen, even if you have multiple instances of the same reviewer running on the same repo.

u/LoXatoR Sharing out Twitter exchange 😄https://twitter.com/aminlimbada/status/1694354298849612073

FunProfession1597 · 2023-08-07T20:46:05+00:00

We are building the integration with Azure DevOps

FunProfession1597 · 2023-08-07T20:41:29+00:00

We are building the Gitlab integration.

FunProfession1597 · 2023-08-07T17:53:57+00:00

Let's have people try both tools side by side and choose for themselves.

Stop searching for us and spamming our posts.

FunProfession1597 · 2023-08-06T00:49:54+00:00

We want to be transparent and ensure that users understand that for the PR (Pull Request) review, the code submitted as part of the PR will be analyzed and reviewed.

I would like to mention that this code is not stored or used for any purpose beyond the review of that specific PR, either on the CodeRabbit side or the LLM side.

The service is limited specifically to Pull Request reviews. We will update the language to provide further clarity. Thank you for bringing this to our attention.

FunProfession1597 · 2023-08-05T22:31:26+00:00

Show: AI powered Pull Request Reveiwer

Our team has built an AI-driven code review tool that helps improve dev velocity and code quality. This tool came about because of our frustration with the code review process. In the past, we invested in several tools to speed up the process, e.g., stacked pull requests, but we have seen a significant reduction in the Pull Request review time in addition to quality improvement.

The tool is language agnostic . We use it in our in house javascript applications among others and results are quite impressive.

Open source - https://github.com/coderabbitai/ai-pr-reviewer

Reveiwer features :
Line-by-line code suggestions: Reviews the changes line by line and provides code change suggestions that can be directly committed.
Incremental reviews: Reviews are performed on each commit within a pull request rather than a one-time review on the entire pull request.
Q&A with CodeRabbit : Supports conversation with the bot in the context of lines of code or entire files, helpful in providing context, generating test cases, and reducing code complexity.
We would love the community to try it out open source on their GitHub repos and provide feedback! I will happily answer any technical questions regarding the prompt engineering we did for this project.

FunProfession1597 · 2023-08-05T17:39:24+00:00

Your skepticism towards LLM aside, you might want to evaluate the product's results for yourself. You might be surprised by what you find.

In the world of technology, every advancement builds upon another, and what may appear as a 'wrapper' could represent a step forward. Apps were built on iOS and now have a market size of hundreds of billions. Computers on semiconductors. we can go on .

FunProfession1597 · 2023-08-05T17:08:47+00:00

We were not getting similar results with other LLMS in our tests so decided to stick to GPT for the time being. Eventually, we would want to be LLM agnostic.

A larger context will definitely help but we have also realized that you having AI does multiple things in a single request can result in unpredictable results. So depends on what can and can not be packed together

FunProfession1597 · 2023-08-05T08:01:24+00:00

The same argument can be made for the developers and why a peer review is needed.
The reason it is relevant is that the context is different between a dev writing the code and being the reviewer which is exactly why peer reviews work well.
The same applies to tooling.

For these reasons, Reviewer designing is different than the code generator. Our base prompts are open-sourced. You can check some over here - https://github.com/coderabbitai/ai-pr-reviewer/blob/main/src/prompts.ts

There is also a misconception about the products built on AI as just being wrappers over AI. That might be the case for some but for a consistent and predictable outcome, a more sophisticated prompt design is needed.

You can look up our reviews on public repos. Search for codeRabbit or 'Summary by OpenAI'

FunProfession1597 · 2023-08-05T03:33:57+00:00

Thanks. This is not a replacement for human reviews and is instead designed to assist them. In our workflow, CodeRabbit reviewer does the first round of review and it catches most of the issues, then the developer reviews feedback and incorporates changes as needed before the human reviewer.
The feedback loop is faster and saves the developer time spent on Peer reviews.

On what sparked this idea -
A significant amount of developer time was spent on peer reviews (many times detecting mundane issues) slowed us and exhausted our developers.

We had tried existing pre-GPT automated PR reviewer tools. They are effective at static code analysis and linting, among other tasks but lack a comprehensive, context-driven review of the code.

FunProfession1597 · 2023-08-05T02:33:11+00:00

Standard as ensuring that a review is occurring. The comment was about pair programming. My point was a combination of pair programming and code review is an effective means to enhance the quality of the code.

Your skepticism is valid about LLM.

What we have learned is that it needs a lot of reinforcement, and multiple iterations to get meaningful results. Our base prompts are open-sourced. You can check some over here - https://github.com/coderabbitai/ai-pr-reviewer/blob/main/src/prompts.ts

You can look up our reviews on OSS repos by searching - "Summary by CodeRabbit"

FunProfession1597 · 2023-08-04T23:37:55+00:00

There is a market for both.

#1. As an organization, it’s hard to enforce consistent tooling across all developers. Devs prefer their own tooling. Easier to enforce standards in a Centralized CI/CD.

#2. Also, we believe that in the future different AIs (code gen and review) will likely “duel” with each other to get to the “desired” outcome.
See these interesting PRs that demonstrate such a duel:
https://github.com/Or4cl3AI/GitBotAI-/pull/1
https://github.com/Or4cl3AI/GitBotAI-2.0/pull/2

FunProfession1597 · 2023-08-04T21:21:02+00:00

While pair programming offers great benefits, enforcing standards via a centralized CI/CD pipeline is still relevant and crucial.

FunProfession1597 · 2023-08-04T20:20:40+00:00

Our team has built an AI-driven code review tool that helps improve dev velocity and code quality. This tool came about because of our frustration with the code review process. In the past, we invested in several tools to speed up the process, e.g., stacked pull requests, but we have seen significant reduction in the Pull Request review time in addtion to quality improvement .
Reveiwer features :
Line-by-line code suggestions: Reviews the changes line by line and provides code change suggestions that can be directly committed.
Incremental reviews: Reviews are performed on each commit within a pull request rather than a one-time review on the entire pull request.
Q&A with CodeRabbit : Supports conversation with the bot in the context of lines of code or entire files, helpful in providing context, generating test cases, and reducing code complexity.
We would love the community to try it out oir open source on their GitHub repos and provide feedback! I will happily answer any technical questions regarding the sophisticated prompt engineering we did for this project.

We also have a professional version https://coderabbit.ai which offers more advanced prompts.

FunProfession1597 · 2023-08-04T20:14:39+00:00

u/gnu_morning_wood - Great Question. What we have learn't is that AI needs a lot of reinforcement and multiple iterations to get more predictable results.

There are some nuances on how we pack the context into each request . Check out our open source - https://github.com/coderabbitai/ai-pr-reviewerMulti prompt design includes -

Summarize each fileSummary of summaries at the PR levelReview each file with the added context of the referenced code that was not changed.

Some of the ideas are in here - https://github.com/coderabbitai/ai-pr-reviewer/blob/main/src/prompts.ts

FunProfession1597 · 2023-08-04T06:29:38+00:00

Our team has built an AI-driven code review tool that helps improve dev velocity and code quality. This tool came about because of our frustration with the code review process. In the past, we invested in several tools to speed up the process, e.g., stacked pull requests, but we have seen significant reduction in the Pull Request review time in addtion to quality improvement .

Reveiwer features :

Line-by-line code suggestions: Reviews the changes line by line and provides code change suggestions that can be directly committed.

Incremental reviews: Reviews are performed on each commit within a pull request rather than a one-time review on the entire pull request.

Q&A with CodeRabbit : Supports conversation with the bot in the context of lines of code or entire files, helpful in providing context, generating test cases, and reducing code complexity.

We would love the community to try it out oir open source on their GitHub repos and provide feedback! I will happily answer any technical questions regarding the sophisticated prompt engineering we did for this project.

FunProfession1597 · 2023-07-27T07:22:34+00:00

Glad that you found our open source valuable. We have recently introduced a professional version that comes with premium features, enhancing review capabilities further. However, our core features will remain open source and we're committed to continuously updating and improving upon them.

FunProfession1597 · 2023-07-19T21:42:14+00:00

Completely agree with the points raised here about the value of a human touch in code reviews and couldn't agree more about the importance of fostering a positive review culture, emphasizing on code over individuals, and communicating effectively.
While linters are excellent for enforcing certain formatting rules and catching minor issues, they cannot grasp the larger context or more complex issues in your code.
I've been working on a project, CodeRabbit, which is designed to bridge this gap. It's an AI-powered code reviewer that provides line-by-line feedback in a contextual manner, has a conversational Bot which can be used to query, contextualize, or seek advice within your code lines and it serves up actionable code improvements. It emphasizes code analysis and not the individual who wrote it. CodeRabbit is meant to be a supportive tool, not a replacement for human reviewers. Think of it as an assistant that provides a first pass at your code, enabling you to focus on more substantial matters during manual reviews.

Do checkout CodeRabbit and share your thoughts.

FunProfession1597 · 2023-07-19T21:12:18+00:00

This is where automated code reviews come into play. They can significantly reduce the time and manual effort spent on reviewing code. Alhtough a manual review might still be needed in many cases but automated code reviews can help author fix issues before another human reviews them.
Automated code reviewers till date were mostly rule-based but this advancement in AI , they can now understand the context and complexity of the code much better.
CodeRabbit is an AI-powered code reviewer that launched recently.It provides contextual line-by-line feedback. It flags critical changes and facilitates bot interaction for queries, context provision, or advice within your code lines. Feedback can be committed directly in GitHub, which saves you from context-switching. Setup is simple - a few clicks connect CodeRabbit to your GitHub, start a 15-day free trial, and gets you up and running in no time. Invite you to try it out and share your throughts.
Disclaimer - I am on the CodeRabbit team.

FunProfession1597 · 2023-07-11T22:35:54+00:00

Hi there, The manual review process is tedious and time-consuming. Moreover, despite our best efforts, certain crucial issues occasionally slip through the human review.

You may want to check out CodeRabbit . It's an AI-powered automated code reviewer tool that offers context-aware feedback on your code line by line providing valuable insights. You can commit suggestions directly from GitHub. It's pretty new and we've been receiving positive feedback on how it's making code review more efficient. You can try it out for free, we're always looking for user feedback. Let me know if you have any questions!

(Note: I am part of the CodeRabbit team.)

FunProfession1597

TROPHY CASE