Mo Sharif
Back to writing

Code Review Rules for AI-Generated Pull Requests

In this article

The diff compiles. The tests pass. The author still cannot explain why the retry lives in the client instead of the service.

That is the review problem I care about with AI-assisted code. Faster authoring makes it easy to submit more change than anyone has actually evaluated.

Reviewing AI-generated code

combines code comprehension with verification of behavior, assumptions, and ownership before a generated change becomes part of a maintained system.

I use agents on my own products. I want the leverage without turning "the agent wrote it" into an explanation for decisions nobody owns.

A useful policy makes scope, evidence, risk, and ownership visible before review starts. It should tell an author what good preparation looks like and give reviewers enough context to test the important claims without reconstructing the entire task.

RuleWhat I want to seeWhat it prevents
Coherent scopeOne behavioral change or an explained exceptionUnrelated edits hiding inside a large diff
Verification noteExact checks run and paths not exercised"Tests pass" standing in for complete QA
Automated baselineTypes, lint, relevant tests, dependency checksReview time spent on mechanical defects
Risk classificationMoney, auth, data, or destructive behavior called outSensitive changes taking the ordinary path
Named ownershipSomeone can explain and maintain the resultA merge nobody can support later

The policy should apply to senior engineers in a hurry as well as new contributors. A rule that disappears for the busiest person is not much of a rule.

Pull request size matters because reviewer attention is limited, while generated changes can grow quickly. The useful boundary is a coherent behavior that someone can understand and verify, not a universal line count that promises safety.

I would use a few hundred changed lines as a prompt to inspect scope, not an automatic rejection. A thousand-line generated fixture and a hundred-line permission rewrite demand different reviews.

Ask the author to split independent behavior, separate formatting, and explain unavoidable bulk. Avoid arbitrary splits that leave each intermediate commit broken or force the reviewer to reconstruct one operation across five pull requests.

Ask authors to distinguish what they ran, what they inspected, and what remains untested. The goal is a useful map of uncertainty. It is not to force an empty "unverified" section or reward people for claiming they checked everything.

A useful note might say:

  • Ran the new endpoint with a valid account and an expired session.
  • Verified that the regression test fails before the patch.
  • Read the migration and its rollback.
  • Did not test a production-sized table or a provider timeout.

That is much more informative than "verified locally." It tells the reviewer where independent work still matters.

Automation should catch repeatable, machine-checkable defects and provide evidence for human review. It cannot establish that the proposed behavior is the right product decision, that the tests represent real failures, or that a passing result covers every sensitive path.

CheckAutomation's roleReviewer's role
Formatting, types, buildEnforce the baselineInvestigate meaningful exceptions
Regression testsRun assertions consistentlyCheck the assertions and failure case
Dependency vulnerabilitiesFlag known advisoriesAssess exposure and remediation
Injection and unsafe patternsFind common shapesTrace input and permission boundaries
Duplicate logicSuggest candidatesDecide whether reuse fits the domain
Product behaviorExercise specified expectationsChallenge the expectations themselves

Do not confuse generated tests with independent evidence. A test can repeat the same mistaken assumption as the implementation. For a fix, I want a test that fails for the original reason and passes for the intended reason.

Changes involving authorization, billing, private data, migrations, or destructive operations need review proportional to their consequences. Trace the whole affected path, not just the changed lines, and identify how the system recovers if deployment succeeds but behavior is wrong.

For billing, that means following a webhook through persistence and entitlement checks. For authorization, it means checking both who may call the endpoint and which records they may affect.

RiskExtra evidence
PermissionsCross-user and cross-tenant denial cases
BillingDuplicate, delayed, and out-of-order events
Personal dataAccess, redaction, retention, and deletion behavior
MigrationProduction-shaped test data and recovery procedure
Irreversible operationPreview, authorization, and verified blast radius

Use synthetic or properly sanitized data for staging. "Production-shaped" should describe volume and edge cases, not permission to copy private production records.

A short template should capture intent, verified behavior, known gaps, and risk. It works best when the fields answer real review questions and the author can complete them without producing another essay that nobody reads.

Use four sections: What changes, Verified, Not verified, and Risk and recovery. Under verification, include commands and observable results, not just checked boxes. Under risk, explain who or what could be affected and how the change can be reversed or contained.

Keep project conventions in the files agents and humans actually read. If the same review comment keeps recurring, decide whether it belongs in a test, lint rule, or repository guide.

That also helps repository-aware tooling: explicit conventions reduce the amount of intent a tool has to guess from the code.

CODEOWNERS identifies responsible reviewers; repository rules determine whether their approval is required. Multiple names on one ownership rule do not mean every listed team must approve. Configure and test the enforcement you actually intend.

GitHub documents that approval from any listed owner can satisfy a code-owner requirement. A separate required-approval count can demand more reviewers, but does not by itself guarantee one approval from each specific team.

That distinction matters for sensitive code. A comment saying "payments and security must approve" is documentation, not enforcement. Verify the repository rules with a test pull request before relying on them.

AI changes how quickly code can be produced, but it does not remove the need for ownership. The author should understand the important behavior, know what remains uncertain, and be able to maintain the result after the tool session ends.

I would not infer a universal productivity result from one study or one team's experience. Measure review time, rework, escaped defects, and rollback frequency in the actual workflow.

The useful question is the same one behind system design judgment: can you explain why this choice fits this constraint? "It compiled" and "the model suggested it" answer different questions.

I want AI-assisted work to reach review with more evidence, not just more code. Small scope, honest verification, and a named owner make that possible.

Questions people actually ask

How should you review code an AI agent wrote?
Use the same correctness standard as any other change, with explicit evidence of what was tested and understood. Trace the affected behavior, inspect failure paths, and check whether the change fits existing code. Generated code still needs a human owner who can explain its important decisions.
How large should an AI-assisted pull request be?
Small enough to review as one coherent change. A few hundred changed lines can be a useful review prompt, but line count is not a universal safety threshold. Separate generated files and mechanical edits from behavioral changes, and ask large pull requests to explain why they cannot be split.
Which changes need extra review?
Increase scrutiny when a change affects money, authorization, personal data, data migrations, or irreversible operations. Require an owner to trace the changed path, test relevant failure cases, and document recovery. Use a second reviewer where available rather than treating a green test suite as sufficient.
Should pull requests disclose AI assistance?
Follow the team's disclosure, security, and compliance rules. An AI-assistance label can be useful provenance, but it is not a risk assessment. Review requirements should still depend on the changed behavior, affected data, and verification evidence, including for entirely human-written code.
Does CODEOWNERS require two approvals from listed teams?
No. Listing multiple owners for a path does not require approval from every owner. GitHub can require code-owner review through repository rules, but one listed owner can satisfy that requirement. Additional approval counts and specific team policies need separate enforcement.