The diff is nine hundred lines. It compiles, the tests pass, the comments are better written than mine, and the engineer who opened it cannot tell you why the retry sits in the client instead of the service. Not because they were careless. Because they didn't write it, and neither did anybody else in the thread.
Reviewing AI-generated code
is a verification task rather than a comprehension task, in which the reviewer establishes what a change actually does and whether it should exist at all, because nobody in the thread accumulated the reasoning that produced it.
Code review was built on an assumption that has quietly stopped holding: that behind every pull request is a person who made each decision in it on purpose, and who can be asked why. Remove that person and most review conventions keep running while doing something other than what they were designed for.
Disclosure first, because it matters for what follows. I build Codelit, an AI tool that designs systems — architecture diagrams, product boards, agent workflows. It has nothing useful to say about your diff. It will not review a pull request, it does not read your branch, and if you came looking for a tool that catches a broken permission check in review, this is not that tool and I am not going to pretend otherwise. What follows is process. You can adopt all of it without buying anything from me or from anyone else.
The review rules, in one table
Six rules cover most of it: a size ceiling, a verified-versus-accepted note, an automated first pass, an escalation tier for money and auth and data, an explainability rule, and conventions that live in AGENTS.md rather than in review comments nobody remembers.
| Rule | What it says | Why it exists | What it costs you |
|---|---|---|---|
| Size ceiling | No agent-authored PR over ~400 changed lines without a written reason in the description | Defect-finding falls off past a few hundred lines, and that limit is a property of the reviewer | Driving an agent in small steps is slower and more irritating than letting it run |
| Verified vs accepted | The description separates what the author ran and read from what they took on trust | Makes the trust boundary visible instead of leaving the reviewer to guess where it sits | Authors resent writing it, and a lazy one will claim they verified everything |
| Automated first pass | No human looks at the diff until CI, types, lint, tests and dependency scanning are green | Human attention is the scarce input, and spending it on formatting is malpractice | CI minutes, plus the setup work nobody volunteers for |
| Escalation tier | Money, auth, permissions, personal data, migrations and destructive queries need a second named reviewer | Nearly half of generated samples fail a security test, and these are the paths where that lands hardest | Slows down exactly the changes people most want to ship today |
| Explainability | "I don't know, the agent wrote it" blocks the merge | An author who can't explain a line also can't own the pager for it | Uncomfortable conversations, usually with senior people |
| Conventions in AGENTS.md | Anything you'd reject in review has to be written in the file the agents read | Otherwise you retype the same review comment forever and nothing learns from it | One more document that rots if nobody owns it |
Why do agent-authored pull requests need a size ceiling?
Because the ceiling belongs to the reviewer, and agents removed the one thing that used to keep pull requests small: how long it takes a human to type them. Nothing about a 900-line diff got easier to review just because it got easier to produce.
The evidence here is old, which is exactly why I trust it. Cisco Systems ran a case study with SmartBear covering roughly 2,500 reviews across 3.2 million lines of code, published in Best Kept Secrets of Peer Code Review in 2006. Two findings have held up: a reviewer's ability to find defects degrades badly beyond about 200 to 400 changed lines, and reviewers moving faster than roughly 450 lines an hour showed below-average defect density in 87% of cases.
That study predates every tool this post is about. It measured the human, and the human has not been upgraded.
Batch size is where the delivery research points too. Google's 2024 DORA report estimated that a 25% increase in AI adoption was associated with a 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability, and named small batch sizes and strong testing as the fundamentals that decide whether faster authoring turns into faster delivery. The 2025 DORA report, published 23 September 2025, found the throughput relationship had flipped positive — and the stability relationship had not. Speed got easier. Not breaking things did not.
Enforce the ceiling with a bot comment rather than a hard block. A hard block teaches people to game the line count. A comment saying "this is 1,100 lines, add a sentence explaining why it can't be split" gets you the reason, and about half the time the reason is good.
How do you make the trust boundary visible in a pull request?
Every pull request should state, in two lists, what the author personally ran or read line by line and what they took on trust. This is the highest-value change on the whole list, and it costs nothing but a template edit.
Before agents, the boundary was implicit and roughly correct: the author had thought about all of it, more or less, because they typed all of it. Now it moves per pull request and stays invisible unless somebody writes it down.
Two lists fix that. "I ran this locally and exercised the new path by hand" is a different claim from "the tests pass," and both are different from "I read the migration but I did not trace the rollback." A reviewer who can see which is which knows where to spend the next forty minutes.
The explainability rule is the same idea with teeth. If a reviewer asks why a line is there and the answer is "the agent put it there", the change does not merge until someone can answer. Not because the agent was wrong — often it isn't — but because merging code nobody can explain means you have accepted a maintenance obligation you cannot service at 3am.
Which checks should move to automation and which stay human?
Everything mechanical moves to CI, and everything that requires knowing what this system is for stays with a person. The split is not new. What changed is that the mechanical pile regenerates itself endlessly if you only ever complain about it in review comments.
| Check | Where it belongs | Why |
|---|---|---|
| Formatting and lint style | Automation, entirely | It was never human judgement, and an agent will reproduce the same violation forever unless the rule is machine-enforced |
| Type errors and compilation | Automation, entirely | Free, instant, and nobody's opinion is required |
| A new code path has a test that fails without the change | Automation gate, human sanity check | Coverage can be gamed by tests that assert nothing, and generated tests are unusually good at that |
| Known-vulnerable dependencies | Automation, entirely | Software composition analysis does this better than any reviewer |
| Injection, XSS and crypto misuse patterns | Automation first pass, human confirmation | The scanners catch the common shapes and miss the contextual ones |
| Duplicated logic that already exists elsewhere | Automation flags, human decides | A clone detector finds the copy; only a person knows whether the existing helper was the right one |
| Is this the right change at all | Human only | No tool has your roadmap, your incident history, or your customer |
| Does the error handling match how this system actually fails | Human only | Requires knowing what has broken here before |
| Does the abstraction fit the domain | Human only | The model has read a million codebases and none of yours |
The reviewer's job shrinks to the bottom four rows, and those four are the hardest ones. That is the trade. Anyone who tells you review gets easier is selling something.
Which changes deserve a stricter review than everything else?
Money, authentication, permissions, personal data, schema migrations and anything with no undo. These are the paths where a plausible-looking mistake costs the most, and the security research says generated code lands mistakes there at an uncomfortable rate.
Veracode's 2025 GenAI Code Security Report tested more than 100 models across 80 curated coding tasks and found 45% of generated samples introduced an OWASP Top 10 vulnerability. The breakdown matters more than the headline: SQL injection and cryptographic choices are getting better, while cross-site scripting failed 86% of the time and log injection performed similarly badly. Models are not uniformly unsafe. They are unsafe in specific, predictable places, and predictable is something you can write policy around.
| Tier | What puts a change in it | What review requires |
|---|---|---|
| Normal | Everything else | One reviewer, standard checklist, merge when green |
| Sensitive | Auth, sessions, permissions, personal data, anything that moves money | Two reviewers, one from the owning group, and the reviewer traces the changed path by hand instead of reading the diff top to bottom |
| Irreversible | Schema migrations, destructive queries, deletion logic, key rotation | Everything in Sensitive, plus a written rollback plan and a run against production-shaped data in staging |
Tracing the path by hand is the part people skip. Reading a diff tells you what changed. Following the request from endpoint to database with the diff open beside you tells you what now happens, which is a different question and the only one that matters on a billing path. Every genuinely nasty thing I have seen in payment code lives in the gap between those two questions, and integrating Stripe into Codelit taught me how wide that gap gets.
Two rules that sound right and are not
Two popular proposals fail for the same reason: labelling pull requests as AI-generated, and banning agents from security-sensitive directories. Both regulate the provenance of the code instead of the risk of the change, and provenance is the attribute that is about to stop carrying any information at all.
Label the pull request as AI-assisted. In Stack Overflow's 2025 Developer Survey, 84% of respondents said they use or plan to use AI tools in development. A label that ends up on nearly every pull request tells a reviewer nothing. Worse, it implies that unlabelled changes are the safe ones, which is precisely backwards — the unlabelled pile will contain both the hand-written code and the code from whoever couldn't be bothered to tick the box. Label the blast radius of the change. That stays meaningful no matter who typed it.
Ban agents from /auth and /billing. I find this one genuinely reasonable, and I still think it is wrong. It is unenforceable, because nobody can tell from a diff. It pushes usage underground, which costs you the signal you most need. And it quietly asserts that hand-written authentication code is safe, an idea the entire history of CVEs would like a word about. The constructive version is the escalation tier above: don't restrict who writes it, raise what it takes to merge it.
Does reviewing AI-generated code actually take longer?
Nobody has published a clean before-and-after measurement of review time, and anyone quoting you one is guessing. What the research does show is that the cost moved rather than disappeared — from reading code to verifying claims about it.
Stack Overflow's 2025 survey is the clearest picture of where the time goes. The single biggest frustration, cited by 66% of developers, is "AI solutions that are almost right, but not quite." Second, at 45%, is that debugging generated code takes longer. Trust matched the frustration: about 46% said they distrust the accuracy of AI output against roughly 33% who trust it. Almost-right is the expensive category. Wrong code fails a test. Almost-right code passes every test you thought to write and fails the one you didn't.
The number I find hardest to argue with is METR's randomised controlled trial, published in July 2025. Sixteen experienced open-source developers completed 246 tasks on their own mature repositories, randomised into AI-allowed and AI-disallowed conditions. They forecast a 24% speedup, reported a 20% speedup afterwards, and were measured 19% slower. If practitioners are that wrong about their own throughput on their own code, self-reported review effort is worthless as evidence — which is why this post is a rulebook rather than a results table.
There is one review target the research points at directly. GitClear's January 2026 analysis of 623 million changes from 2023 to 2026 found refactoring collapsed to 3.8% of changed lines, down from 21% in 2022, while duplicated code blocks rose 81% since 2023 and error-masking constructs rose 47%. So add one question to every review: does a function that already does this exist in this codebase? A clone detector will flag the copy. Only a reviewer with the codebase in their head knows whether the original was the right one to reuse, which is the same context problem I hit when building the feature that reads a GitHub repo and draws its architecture.
The pull request template and CODEOWNERS snippet
Two files carry most of this. A pull request template puts the verified-versus-accepted note and the blast radius checklist in front of the author before they hit submit, and a CODEOWNERS file routes the escalation tiers to the right people without anyone having to remember to.
Drop this at .github/pull_request_template.md:
And the routing, at .github/CODEOWNERS:
# Default owner. Anyone on the team can approve.
* @acme/engineering
# Sensitive tier. Second approval must come from the owning group.
/src/billing/ @acme/engineering @acme/payments
/src/auth/ @acme/engineering @acme/security
/src/**/migrations/ @acme/engineering @acme/data
/infra/ @acme/engineering @acme/platform
# The files the agents read. Changing the conventions is a team decision.
/AGENTS.md @acme/staff-eng
/CLAUDE.md @acme/staff-eng
One caveat that catches people: on GitHub, CODEOWNERS on its own only requests reviewers. It blocks nothing until you turn on the branch protection rule that requires review from code owners. A file full of good intentions and no enforcement is worse than no file, because everyone assumes it is working.
The last two lines are the part I would fight for. AGENTS.md is a plain Markdown file describing your build commands, test commands and conventions for coding agents — the same role a README plays for humans. It is now stewarded by the Agentic AI Foundation under the Linux Foundation and, according to agents.md as of July 2026, used by more than 60,000 open-source projects and read by 20-plus agents. If you find yourself leaving the same review comment twice, the comment belongs in that file. Review is a terrible place to store a convention, because it only ever reaches one person at a time.
What this is really about
None of this is about distrusting the tools. I ship agent-assisted code on my own products constantly, and Codelit exists because I got tired of doing work a machine should do. The rules exist because writing software has become the cheap part, which makes every other part the bottleneck, and review is where that bottleneck first shows up.
What review demands has shifted with it. It used to be reading comprehension. It is now judgement: which changes deserve scrutiny, which trade-off got made silently, which abstraction will be wrong in six months. That is the same gap I keep hitting with engineers learning system design — everyone knows what a message queue is, and far fewer can say whether this system needs one.
Generating code stopped being the hard part. Deciding it deserves to exist never was easy, and now it's the whole job.
Questions people actually ask
- How should you review code that an AI agent wrote?
- Treat it as verification rather than comprehension. Ask what the author personally ran and read, and what they took on trust. Push every mechanical check into continuous integration so human attention goes to whether the change should exist at all, whether the error handling matches how the system really fails, and whether the abstraction fits the domain. Treat an author who cannot explain a line as a blocking problem.
- How big should a pull request be?
- Small enough that one person can hold it in their head, which in practice means a few hundred changed lines. One of the largest published peer code review case studies, run at Cisco Systems with SmartBear and published in 2006, found that a reviewer's ability to find defects degrades badly beyond roughly two to four hundred lines. That ceiling is a property of the reviewer, so it does not move just because an agent wrote the code faster.
- Does AI-generated code need a stricter code review?
- Not stricter across the board, which mostly produces theatre. Raise the bar on the paths where mistakes are expensive and hard to reverse. Changes to billing, to authentication and permissions, to personal data, and to schema migrations or deletion logic should require a second named reviewer, a hand traced walk of the changed code path, and a written rollback plan. Everything else can stay on the normal path.
- Should you label pull requests as AI-generated?
- No. Provenance labels stop discriminating almost immediately, because nearly everyone is using these tools. In Stack Overflow's 2025 Developer Survey, 84 percent of respondents said they use or plan to use AI tools in development. A label that lands on every pull request carries no information, and worse, it tells reviewers that unlabelled changes are safe. Label the risk of the change instead.
- Is AI-generated code less secure than human-written code?
- The published evidence is not encouraging. Veracode's 2025 GenAI Code Security Report tested more than one hundred models across eighty curated coding tasks and found that 45 percent of the generated samples introduced an OWASP Top 10 vulnerability, with cross site scripting failing 86 percent of the time. That is an argument for stronger automated scanning and an escalation tier on sensitive paths, not for banning the tools.
- What should go in a pull request template for AI-assisted work?
- Four things. One sentence describing what the change does, a list of what the author verified by running or reading it, a list of what the author accepted on trust, and a blast radius checklist covering money, authentication, personal data, and irreversible operations. The accepted list is the important one. It converts an invisible trust boundary into something a reviewer can see and argue with.