What’s the Best LLM for Code Review? (2026 Benchmark)
Edvaldo Freitas
Picking an LLM to write code is very different from picking one to review it.
A model can be great at solving programming problems, writing functions and shipping new features, and still not be good at finding bugs in a pull request.
Code review is a different job. The model has to read changes it didn’t write, understand how they affect the rest of the project and spot problems that often aren’t obvious.
There’s another catch, too: finding lots of bugs doesn’t help much if the tool also posts a pile of wrong comments. Over time, developers start ignoring the suggestions.
That’s why general coding leaderboards aren’t enough to judge how good an LLM is at code review.
At Kodus, we built the LLM code review benchmark, which compares AI models by having them review real pull requests from open source projects.
We put 13 models side by side to see how many bugs each one finds and how many of its comments are right. We also worked out what each review costs, because a model that finds a lot of bugs can end up too expensive for a team’s PR volume.
Updated: November 10, 2026
TL;DR:
- We evaluated 13 LLMs: Kimi K3.1 Pro, GPT-6.1 Sol, GPT-6 Luna, Claude Opus 5.5, Claude Sonnet 5.5, GLM-5.3, GLM-5.3 Flash, DeepSeek V4 Pro, DeepSeek V4.1 Flash, Qwen3.8 27B, MiniMax M3, Gemini 3.8 Flash and Muse Spark 1.2.
- The models reviewed 30 real pull requests from five open source repositories: cal.com (TypeScript), Discourse (Ruby), Grafana (Go), Keycloak (Java) and Sentry (Python).
- Kimi K3.1 Pro finds more confirmed bugs than any other model (68.6% of the 121, or 83 bugs).
- GPT-6.1 Sol has the highest precision of the 13 models (62.7%), so a smaller share of its comments miss a confirmed bug.
- DeepSeek V4.1 Flash and GLM-5.3 Flash have the best cost per confirmed bug, five to almost seven times lower than the leaders in recall or precision.
- Running nine specialist agents alongside the main agent (heavy mode) adds 9.9 points of recall on average, but costs 2.5 times as much and cuts precision by 7 points, and the gain varies a lot from model to model.
What are the best LLMs for code review?
Based on our benchmark results, six models are worth a look, each for a different reason:
- Kimi K3.1 Pro: finds the most bugs, with 83 of the 121 confirmed bugs (68.6% recall).
- GPT-6.1 Sol: the highest precision of the 13 models, with 62.7% of its comments matching confirmed bugs.
- Claude Opus 5.5: a balanced result, with 42.2% recall and 52.6% precision.
- DeepSeek V4.1 Flash: recall in the same range as GPT-6.1 Sol and Opus (40.5%), at $0.134 per PR, between a fifth and a sixth of what those two cost.
- Claude Sonnet 5.5: 57.0% precision, at a lower cost per PR than Opus.
- GLM-5.3 Flash: the lowest cost per bug found in the test, $0.069.
These numbers come from balanced mode, where each model reviews the PR with a single generalist agent.
Comparison table of all 13 models
The table below lists the 13 models we evaluated, sorted by F2 in balanced mode.
| Model | Provider | Recall | Bugs found | Precision | F2 | Cost per PR | Cost per bug found |
|---|---|---|---|---|---|---|---|
| Kimi K3.1 Pro | Moonshot | 68.6% | 83 of 121 | 49.1% | 63.6 | $1.244 | $0.450 |
| GPT-6.1 Sol | OpenAI | 43.0% | 52 of 121 | 62.7% | 45.9 | $0.787 | $0.454 |
| Claude Opus 5.5 | Anthropic | 42.2% | 51 of 121 | 52.6% | 43.9 | $0.715 | $0.421 |
| GLM-5.3 | Zhipu | 41.3% | 50 of 121 | 43.5% | 41.7 | $1.171 | $0.702 |
| DeepSeek V4.1 Flash | DeepSeek | 40.5% | 49 of 121 | 43.4% | 41.0 | $0.134 | $0.082 |
| Claude Sonnet 5.5 | Anthropic | 37.2% | 45 of 121 | 57.0% | 40.0 | $0.542 | $0.361 |
| Muse Spark 1.2 | Meta | 37.2% | 45 of 121 | 33.6% | 36.4 | $0.725 | $0.483 |
| GLM-5.3 Flash | Zhipu | 35.5% | 43 of 121 | 39.8% | 36.3 | $0.098 | $0.069 |
| DeepSeek V4 Pro | DeepSeek | 32.2% | 39 of 121 | 47.6% | 34.5 | $0.490 | $0.377 |
| Qwen3.8 27B | Alibaba | 28.1% | 34 of 121 | 51.5% | 30.9 | $0.198 | $0.175 |
| MiniMax M3 | MiniMax | 19.8% | 24 of 121 | 38.1% | 21.9 | $0.154 | $0.193 |
| Gemini 3.8 Flash | 15.7% | 19 of 121 | 50.0% | 18.2 | $0.349 | $0.550 | |
| GPT-6 Luna | OpenAI | 6.6% | 8 of 121 | 61.5% | 8.1 | $0.023 | $0.086 |
Source: leaderboard, balanced mode, methodology v3 (the heavy-mode figures in this article come from the same leaderboard), data pulled on November 10, 2026. Cost is each provider’s list API price, with prompt caching discounts applied where available, priced between November 7 and 9.
Small gaps between models need to be read carefully. The test runs once per model and mode, and resampling the same 30 PRs moves recall by about 20 points (95% bootstrap confidence interval).
Kimi’s lead holds up against that margin: its balanced interval runs from 59.0% to 78.0%, and no other top-5 model’s upper bound goes past 54.8%. GPT-6.1 Sol, Claude Opus 5.5, GLM-5.3 and DeepSeek V4.1 Flash, on the other hand, have intervals that overlap a lot, so this sample can’t settle the exact order from second to fifth place.
How we evaluated the models for code review
We took real, already-merged pull requests from open source projects and had each model review them the same way a code review tool would day to day, using the Kodus review system.
In total, the test covers:
- 30 real pull requests, already merged into the projects.
- 121 confirmed bugs, used as the answer key.
- 5 open source repositories: cal.com, Discourse, Grafana, Keycloak and Sentry.
- 5 languages: Go, Java, Python, Ruby and TypeScript.
- 13 models, each tested in two review modes.
To build the answer key, we reviewed and relabeled the bugs in each PR. Claude Opus 5.5 reviewed each PR, a second reviewer checked the result, and disagreements were debated.
Then each model reviewed the same 30 PRs. The posted comments were compared against the answer key by a judge model, Claude Haiku 5.5. To count as a hit, a comment has to point to the same problem in the same place as the reference bug, and a second comment on the same bug counts as a false positive.
To validate Haiku as the judge, we compared its decisions with Claude Opus 5.5’s on 252 comments. The two agreed on 249.
Every model ran on the same PRs, under the same rules and with the same judge, so they can be compared under the same conditions.
The two review modes: balanced and heavy
Each model was tested two ways.
Balanced: a single generalist agent reviews the PR and posts the comments on its own. This is the setup used in the table above.
Heavy: the same generalist agent runs alongside nine specialist agents, in parallel. Each specialist looks at the same PR through a different lens, such as security, performance or business logic. Then everyone’s findings go through the same verification and deduplication steps before they become posted comments.
In heavy mode, the output isn’t ten separate reviews. It’s a single list of comments, merged from ten passes over the same diff. Comparing the two modes shows how much recall changes when the system puts more agents on the same PR, and what that costs.
What recall, precision and F2 mean
Three metrics help make sense of the results.
Recall: how many bugs the model found. Picture a pull request with ten known bugs. If the model catches six of them, recall is 60%. The higher the recall, the more real bugs the model can find.
Precision: how many comments match confirmed bugs. Now picture a model that posts ten comments on a PR, and only four of them match confirmed bugs. Precision is 40%. Low precision means more comments for the team to sort through and dismiss.
In this methodology, a comment that doesn’t match any bug in the answer key counts as a false positive. That doesn’t prove every extra suggestion is technically wrong, since a comment can point to something valid that wasn’t in the answer key.
F2: one number that combines both. F2 combines recall and precision but gives recall double the weight. The reasoning is that letting a bug through usually costs more than dismissing a comment. That’s why F2 is what orders the ranking.
That choice doesn’t fit every team equally. On projects with a high PR volume, low precision can create a lot of triage work, and precision starts to weigh more in the decision.
A closer look at the best models for code review
1. Kimi K3.1 Pro: finds the most bugs
Kimi K3.1 Pro, from Moonshot, had the best overall result in the benchmark. In balanced mode, it found 83 of the 121 confirmed bugs, or 68.6% recall. That’s 31 more bugs than the runner-up, GPT-6.1 Sol, which found 52.
| Metric (balanced) | Kimi K3.1 Pro |
|---|---|
| Recall | 68.6% |
| Bugs found | 83 of 121 |
| Precision | 49.1% |
| F2 | 63.6 |
| Cost per PR | $1.244 |
| Cost per bug found | $0.450 |
It’s also the most solid lead in the ranking, and the only one that holds up even after accounting for the test’s margin of error.
The trade-off is precision. Kimi posted 169 comments across the 30 PRs, and 83 of them matched confirmed bugs. A little under half of its comments count as hits. That can be fine when the goal is broader review coverage. For a team already drowning in comments from automated tools, though, triage becomes a real cost.
You’d expect nine extra agents to help the model that was already in the lead. That’s not what happened: in heavy mode, Kimi found one fewer bug (82, versus 83 in balanced) and posted 49 more comments. Precision fell from 49.1% to 37.6%, and cost per PR nearly doubled, from $1.244 to $2.306.
When to consider Kimi K3.1 Pro: when finding as many bugs as possible is the priority and the team is willing to go through a larger volume of comments.
2. GPT-6.1 Sol: the highest precision in the test
GPT-6.1 Sol, from OpenAI, came in second on F2 in balanced mode, at 45.9. Its standout is precision: of the 83 comments it posted, 52 matched confirmed bugs, which works out to 62.7%, the highest precision of the 13 models.
| Metric (balanced) | GPT-6.1 Sol |
|---|---|
| Recall | 43.0% |
| Bugs found | 52 of 121 |
| Precision | 62.7% |
| F2 | 45.9 |
| Cost per PR | $0.787 |
| Cost per bug found | $0.454 |
Its behavior is close to the opposite of Kimi’s. Kimi finds more problems, while GPT-6.1 Sol posts a higher share of correct comments. For teams that have already been burned by false positives from other code review tools, that difference matters a lot.
The price of that caution shows up in recall. In balanced mode, GPT-6.1 Sol missed 69 of the 121 bugs.
In heavy mode, recall rises to 55.4% (67 bugs) and precision drops to the same 55.4%. It’s the only model in the test where the two metrics end up at exactly the same value. Cost per PR goes from $0.787 to $1.741.
When to consider GPT-6.1 Sol: when the team wants more accurate comments and needs to cut the share of suggestions that don’t match confirmed bugs.
3. Claude Opus 5.5: a balance between recall and precision
Claude Opus 5.5, from Anthropic, came in third in balanced mode, with an F2 of 43.9. It found 51 of the 121 bugs (42.2% recall), with 52.6% precision.
| Metric (balanced) | Claude Opus 5.5 |
|---|---|
| Recall | 42.2% |
| Bugs found | 51 of 121 |
| Precision | 52.6% |
| F2 | 43.9 |
| Cost per PR | $0.715 |
| Cost per bug found | $0.421 |
On recall, Opus is basically tied with GPT-6.1 Sol: 51 bugs versus 52, a gap that fits comfortably within the test’s margin of error. On precision, it trails by 10 points. Its cost per PR is a bit lower, $0.715 versus $0.787.
Opus’s weak spot shows up in heavy mode. Recall rises to 49.6% (60 bugs), but cost per PR reaches $3.10, more than four times the balanced cost, for a gain of 7.4 points of recall. It’s the most expensive configuration in the whole test, and the cost per bug found climbs to $1.55.
When to consider Claude Opus 5.5: when the team wants a balanced result between recall and precision, especially if it already runs on Anthropic’s infrastructure.
4. DeepSeek V4.1 Flash: the best value among the top performers
DeepSeek V4.1 Flash came in fifth on F2 in balanced mode, at a much lower cost than the models above it. It found 49 of the 121 bugs (40.5% recall) at $0.134 per PR.
| Metric (balanced) | DeepSeek V4.1 Flash |
|---|---|
| Recall | 40.5% |
| Bugs found | 49 of 121 |
| Precision | 43.4% |
| F2 | 41.0 |
| Cost per PR | $0.134 |
| Cost per bug found | $0.082 |
Claude Opus 5.5 found 51 bugs, two more than DeepSeek, at more than five times the cost per PR. GPT-6.1 Sol found 52, three more, at almost six times the cost per PR.
DeepSeek’s precision (43.4%) is lower than both, so it isn’t automatically the best pick. Still, catching bugs at a competitive level doesn’t require one of the most expensive models. For a team reviewing thousands of PRs a month, that cost gap adds up fast.
In heavy mode, DeepSeek V4.1 Flash climbs to 52.9% recall (64 bugs), ahead of Claude Opus 5.5 in the same mode, at $0.16 per bug found. In heavy mode, Opus costs $1.55 per bug.
When to consider DeepSeek V4.1 Flash: when the goal is to balance bug detection against API cost, especially for operations with a high review volume.
5. Claude Sonnet 5.5: Anthropic’s middle option
Claude Sonnet 5.5 found 45 bugs in balanced mode, with 37.2% recall and 57.0% precision.
| Metric (balanced) | Claude Sonnet 5.5 |
|---|---|
| Recall | 37.2% |
| Bugs found | 45 of 121 |
| Precision | 57.0% |
| F2 | 40.0 |
| Cost per PR | $0.542 |
| Cost per bug found | $0.361 |
Compared with Claude Opus 5.5, Sonnet found six fewer bugs but had higher precision and a lower cost ($0.542 per PR, versus $0.715). Among the six models in this analysis, only GPT-6.1 Sol has higher precision.
In heavy mode, Sonnet’s recall rises to 47.1% (57 bugs) and its precision holds at 57.0%, the highest of any model in that mode. Cost per PR goes up to $1.754.
When to consider Claude Sonnet 5.5: when the team already uses Anthropic and wants an alternative to Opus with a lower cost and more accurate comments.
6. GLM-5.3 Flash: the lowest cost per bug found
GLM-5.3 Flash, from Zhipu, stands out mainly for its cost. It had 35.5% recall (43 bugs) and 39.8% precision, at $0.098 per PR.
| Metric (balanced) | GLM-5.3 Flash |
|---|---|
| Recall | 35.5% |
| Bugs found | 43 of 121 |
| Precision | 39.8% |
| F2 | 36.3 |
| Cost per PR | $0.098 |
| Cost per bug found | $0.069 |
Its cost per bug found, $0.069, is the lowest in the entire test. GLM-5.3 Flash ranks eighth on F2 and is still the cheapest model for each bug it finds. In heavy mode, recall rises to 47.9% (58 bugs) and cost per bug stays low, at $0.114.
The downside is precision: of the 108 comments it posted in balanced mode, 65 didn’t match any bug in the answer key. Before picking this model, factor in the time your team will spend going through those comments.
GPT-6 Luna costs even less per PR ($0.023), but it finds only 8 bugs, so its cost per bug ($0.086) ends up higher than GLM-5.3 Flash’s.
When to consider GLM-5.3 Flash: when the per-review budget is very tight and the team can live with lower bug coverage.
The other seven models we tested
| Model | Provider | Recall (balanced) | Precision (balanced) | Notes |
|---|---|---|---|---|
| GLM-5.3 | Zhipu | 41.3% | 43.5% | Recall close to Claude Opus 5.5, but 1.7 times the cost per bug found ($0.70 versus $0.42). |
| Qwen3.8 27B | Alibaba | 28.1% | 51.5% | Fairly high precision for a 27B-parameter model, but less than half of Kimi K3.1 Pro’s recall. |
| MiniMax M3 | MiniMax | 19.8% | 38.1% | The third-lowest recall in the test, with a cost per bug above the two cheapest flash models. |
| Muse Spark 1.2 | Meta | 37.2% | 33.6% | Same recall as Claude Sonnet 5.5, but the lowest precision of the 13 models. |
| DeepSeek V4 Pro | DeepSeek | 32.2% | 47.6% | DeepSeek’s non-flash model trails V4.1 Flash on recall and costs about 4.6 times as much per bug. |
| Gemini 3.8 Flash | 15.7% | 50.0% | Precision near the test average, but the second-lowest recall. Cost uses Google’s promotional price, valid through December 31, 2026. | |
| GPT-6 Luna | OpenAI | 6.6% | 61.5% | The lowest recall in the test. At the provider’s default reasoning setting, it produces about 25 reasoning tokens per PR, which is very little for a review task. |
What the results show about AI code review
Beyond how each model performed, a few behaviors repeat across several of them and help set expectations for AI in code review.
1. Even the best models let bugs through
Kimi K3.1 Pro led the test with 68.6% recall. Even so, 38 of the 121 confirmed bugs slipped past it in balanced mode. GPT-6.1 Sol missed 69, and Claude Opus 5.5 missed 70.
These numbers apply to this sample of pull requests and aren’t the exact rate at which each model would miss bugs on any project. Still, they help explain why using AI in code review doesn’t remove the need for automated tests and human review. A model can catch problems that slipped past other steps, but it doesn’t guarantee the code is correct.
2. The model with the highest recall isn’t necessarily the most precise
Kimi found 83 bugs at 49.1% precision, while GPT-6.1 Sol found 52 at 62.7% precision. In practice, Kimi finds more problems, but a larger share of its comments don’t match confirmed bugs.
If you work on critical code, like payments or authentication, a model that finds more bugs, like Kimi, tends to be the better deal, even if it posts more wrong comments. Going through a few extra suggestions takes work, but it’s usually cheaper than finding the bug in production. On a project with hundreds of PRs a day, though, GPT-6.1 Sol may be the better option for the team, because it posts fewer comments and a larger share of them point to a confirmed bug.
3. More expensive models aren’t necessarily more efficient
DeepSeek V4.1 Flash found 49 bugs at $0.134 per PR. Claude Opus 5.5 found 51 at $0.715 per PR, and GPT-6.1 Sol found 52 at $0.787.
Divide cost by the number of bugs found and the gap between models gets wider:
| Model | Cost per bug found (balanced) |
|---|---|
| GLM-5.3 Flash | $0.069 |
| DeepSeek V4.1 Flash | $0.082 |
| Claude Sonnet 5.5 | $0.361 |
| Claude Opus 5.5 | $0.421 |
| Kimi K3.1 Pro | $0.450 |
| GPT-6.1 Sol | $0.454 |
These figures are API cost within the benchmark. They don’t include developer time spent reviewing suggestions, infrastructure or other operating costs. Even with that limitation, price and quality don’t scale together. A more expensive model can have higher recall or precision and still not be the most efficient option for your project.
4. Adding more agents doesn’t always improve the review
Averaged across the 13 models, heavy mode, with nine extra specialist agents, delivered:
- 9.9 percentage points more recall.
- 7 percentage points less precision.
- About 2.5 times the cost per PR.
In practice, more agents tend to find more bugs, but they also post more wrong comments and drive cost up considerably.
The effect also varies a lot from one model to the next. DeepSeek V4 Pro had the biggest gain, 15.7 points of recall (from 32.2% to 47.9%). Kimi K3.1 Pro was the only one that got worse, if only slightly: it found 83 bugs in balanced mode and 82 in heavy.
More agents on their own don’t guarantee a better review. Since the effect changed so much from one model to another, the review mode needs to be tested together with the model you pick.
How to choose an LLM for code review
If your priority is finding more bugs
Look at recall first. In our test, Kimi K3.1 Pro stood out, finding 68.6% of the confirmed bugs, and it’s the only lead that holds up within the margin of error.
Check precision too, especially if your team already gets a lot of comments from automated tools. Payment code, authentication and sensitive data are where letting a bug through usually costs the most.
If your priority is cutting false positives
Here, precision is the metric that matters most. GPT-6.1 Sol had the best result in the test (62.7%), followed closely by Claude Sonnet 5.5 (57.0%).
High precision here means a larger share of comments matched confirmed bugs within the evaluation.
If your priority is cutting costs
DeepSeek V4.1 Flash and GLM-5.3 Flash deserve a look. Both cost a fraction of the most expensive models and can make sense for teams with a high review volume.
If we had to pick a model today for a team with lots of PRs and a tight budget, we’d start with DeepSeek V4.1 Flash in heavy mode: in our test, it found 64 bugs, more than Claude Opus 5.5 in either mode, at $0.16 per bug. We’d only move up to a pricier model if its precision (37.7% in that mode) started to bother the team.
Compare cost per bug found, not cost per token or per PR. And keep in mind that a cheap model can get expensive in practice if it generates lots of suggestions that need manual investigation.
If you work on a large codebase
Beyond the model, it matters how the model gets the context it needs to review the change. A bug can live in the function changed in the PR, but also in a contract defined in another file, a configuration default or a business rule implemented somewhere else.
Of the 121 confirmed bugs, 5 weren’t found by any of the 13 models, in either mode. In some of them, the problem only becomes visible with information that sits outside the diff. One example is a duplicate-email check that’s case-sensitive, while the blocklist, in another part of the code, already lowercases everything.
That’s why, before adopting a model for code review, you should test it on pull requests from your own project, especially ones that touch several files or modules.
Does code review quality depend only on the model?
The model carries a lot of weight, but it doesn’t decide the quality of an automated review on its own. The same model can produce very different results depending on the context it gets, the tools it can use to investigate the code and the verification steps before a comment is posted.
With the same model, changing only how the review is organized, from balanced to heavy, moved DeepSeek V4 Pro’s recall by 15.7 points.
The same goes for false positives. A model can raise a hypothesis that looks valid but falls apart once you look at another part of the code. If the review system checks that hypothesis before posting the comment, the unnecessary suggestion never reaches the developer.
For teams that have already decided which model to use, whether because of data policy, a contract or preference, Kodus lets you plug in your own API key and pay the provider directly, with no Kodus markup on model cost. The trade-off is that the team takes on managing its own provider account, rate limits and billing. That way, the models in this article can be tested in the team’s own environment without switching code review tools.
Conclusion
Choosing an LLM for code review means looking past general coding leaderboards. In our test, Kimi K3.1 Pro found the most bugs, GPT-6.1 Sol had the highest precision and DeepSeek V4.1 Flash showed that a cheap model can also reach competitive results.
Even the best models let problems through, and more agents don’t guarantee a better review. So choosing the model has to go hand in hand with evaluating the system it will run in: repository context, the available tools, the review rules and how findings get verified all shape the final result.
If you’re deciding which model to use, our suggestion is to start with this benchmark’s data and then test the most promising options on your own team’s pull requests. That’s what we did here with 30 PRs from other projects, and your PRs will tell you more about your code than any average.
FAQ
What is the best LLM for code review in 2026?
In our test, Kimi K3.1 Pro had the best overall result, with an F2 of 63.6 and 68.6% recall. GPT-6.1 Sol had the highest precision, at 62.7%, and DeepSeek V4.1 Flash stood out for value among the top performers.
Is Claude or GPT better for code review?
On precision, GPT-6.1 Sol is ahead of Claude Opus 5.5 in balanced mode (62.7% versus 52.6%). On recall, the two are basically tied (43.0% versus 42.2%), a gap that fits within the test’s margin of error. Claude Sonnet 5.5 costs less per bug found than either of them.
Which LLM finds the most bugs in pull requests?
Kimi K3.1 Pro, which found 83 of the 121 confirmed bugs (68.6%) in balanced mode and 82 in heavy. It had the highest recall of the 13 models, with a lead that holds up even after accounting for the margin of error.
What is the cheapest LLM for code review?
GPT-6 Luna has the lowest cost per PR, $0.023, but found only 6.6% of the bugs at its default settings. By cost per bug found, GLM-5.3 Flash is the cheapest, at $0.069. DeepSeek V4.1 Flash pairs higher recall (40.5%) with a low cost, $0.082 per bug.
Which tool lets you run code review with different LLMs?
Some AI code review tools let you choose between different models without building your own integration for each one.
Kodus, for example, lets you configure models from providers like OpenAI, Anthropic and Google Gemini through BYOK (Bring Your Own Key). That way, teams can test different LLMs, compare their results and pay the provider directly, with no markup on token usage.
Does a more expensive model find more bugs?
In balanced mode, DeepSeek V4.1 Flash costs $0.134 per PR and finds 49 bugs, two fewer than Claude Opus 5.5, which costs $0.715. In heavy mode, Opus has the highest cost per PR in the test ($3.10) and ranks fourth on F2. In this sample, price and the number of bugs found don’t move together.
Does running more review agents always find more bugs?
On average, yes, with 9.9 more points of recall in heavy mode. But the effect ranges from a small loss for Kimi K3.1 Pro (0.8 points lower) to a 15.7-point gain for DeepSeek V4 Pro, and it comes with an average 7-point drop in precision and about 2.5 times the cost per PR.
Can an LLM replace human code review?
The test results don’t support that conclusion. Even the model with the highest recall missed about 31% of the confirmed bugs. Models help catch problems before human review and work as an extra layer of verification, but automated tests and human review are still necessary, especially for critical changes.