ProjekinAja. Software Engineering Stage 2 · Craft Lesson 2.10
● Stage 2 · Lesson 2.10

Code Review: Humans, Egos, and AI-Generated Diffs

Review is the bottleneck skill of the AI era — knowledge transfer first, bug-catching second, rubber-stamping never.

⏱ ~20 min read 🤖 AI-era skill 🚀 Stage 2 closer

🎯 By the end you'll understand

What code review genuinely catches (and doesn't), how to size and structure a PR for fast, meaningful review, the comment severity ladder and the tone that doesn't crater team morale — and most critically, how to review AI-generated diffs adversarially, because plausible-by-construction code is the new way a rubber-stamper becomes a code-monkey. You'll review a realistic AI-generated TraceLab PR, catch its defects red-handed, and ship TraceLab v1.

KEY INSIGHTReview is knowledge transfer that sometimes catches bugs

The diff is a pretext. What you're really doing is telling two people the same story about the codebase — the author who wrote it, and the reviewer who now has to live with it.

Every code review study you'll find puts bug-catch rates somewhere between 60–90% for code inspections done well — but "done well" means hours, structured checklists, and often a team. A typical async PR review catches maybe 15% of the defects a pre-release inspection would. This isn't an argument against review. It's an argument against treating review as your main QA layer, and an argument for what review is actually optimized for: spreading understanding of a change across the people who have to maintain it.

🔑 The mental model that changes everything

Think of your PR as a story you're telling the codebase's future maintainers — including yourself in six months. A reviewer who understands your change well enough to extend it, fix it, or roll it back without you present: that's a successful review. A reviewer who approved it in thirty seconds and promptly forgot it: that's a rubber stamp, and you've just created a knowledge silo. The defect that slipped through is the less expensive consequence.

CORE CONCEPTWhat review actually catches — and what it doesn't

The honest picture, drawn from published inspection literature and post-mortems:

Logic/correctness errors API misuse Knowledge transfer ★ Security vulns (subtle) Concurrency/race bugs catches well catches poorly relative effectiveness →
Review is good at logic errors and knowledge transfer; poor at subtle security and concurrency bugs — which need dedicated tooling (fuzzing, taint analysis, race detectors) to catch reliably.

The implication: do not use code review as your security posture. A subtle SQL injection or a path traversal in a TraceLab file upload will likely make it through even a conscientious reviewer. Use review for what it's good at, and layer in dedicated tooling (static analysis, dependency audits, fuzzing) for the things it misses.

PR sizing — the one lever reviewers will thank you for

Empirically, review quality collapses after about 400 lines of diff. Reviewers don't get worse — they get tired, they start pattern-matching instead of reading, and they start deferring to the author's judgment because they can't hold the whole change in working memory. The research on this is consistent enough to treat as settled: keep PRs under 400 lines, ideally under 200. When a feature is bigger, split it: a structural refactor first (diff that changes architecture, not behavior), then the feature on top.

💡 The "newspaper test" for PR size

If you can't summarize what your PR does in one clear sentence, it's too big. "Refactor CSV logger to use csv.writer and handle encoding" — yes. "Add logging, fix the schema, update CLI flags, and also rework the compare command" — that's four PRs you failed to split.

HOW IT WORKSAnatomy of a review: comment severity and tone

Not every comment is the same weight. A reviewer who annotates code style at the same severity level as a correctness bug is training the author to ignore them both. Use an explicit severity ladder — many teams do, and those that don't should. A common and functional one:

LabelMeaningExampleBlocks merge?
BLOCKER Correctness defect, security hole, or broken contract — the PR cannot ship as-is "Off-by-one: rows[page*n:(page+1)*n] returns empty when page=0, n=len(rows)" Yes
CONCERN Not necessarily broken today, but a fragile design or unhandled edge case worth discussing "No validation on run_id — could this be passed user input?" Author's call
NIT Style, naming, or formatting preference — feel free to disagree or ignore "nit: prefer int(page) over page // 1 for clarity" No
QUESTION Genuinely curious — not a critique, the reviewer wants to learn the intent "Why append rather than truncate-and-rewrite? Is there a concurrency reason?" No

Tone matters as much as the label. Every comment should be about the code, never about the author. "This function is confusing" is an accusation; "I found this function hard to follow — could we add a docstring explaining the page-offset math?" is an invitation. Senior engineers leave the second kind by habit, not just kindness: it produces faster resolution and doesn't torch the relationship you depend on for the next review.

💡 The rule about blockers

If a comment is a BLOCKER, say why — provide the specific failing input or scenario. "Off-by-one" is not enough; "Off-by-one: calling paginate(rows=[], page=0, n=10) raises IndexError" is a blocker with evidence. The author should be able to reproduce the failure from your comment alone, without a context-switch back to you.

SEE ITAI-output review trainer

Five realistic AI-generated TraceLab functions. Each one looks clean — it uses type hints, has a docstring, passes a linter. Each one hides exactly one defect of the kind AI code is specifically prone to: confident-sounding code that handles the happy path perfectly and fails at the boundary, the edge case, or the invariant that wasn't in the prompt.

Work through each function, identify the defect, then check your answer. The timer runs once you pick a candidate — speed matters because review pressure is real. After you submit, you'll see the test that would have caught it automatically.

🔍 Interactive · AI Code Review Trainer — TraceLab functions

Five functions, five hidden defects. Pick one, find the bug, submit your guess. Scored on catches (correct BLOCKER) vs false alarms (wrong call) vs misses.

Timer: 0s starts when you load a function

Where is the defect?

Catches
0
False alarms
0
Misses
0
Reviewed
0 / 5

Load a function, find the defect, submit your call.

The pattern you'll notice: every function has a clean happy path. The defect lives at the edge — empty input, type mismatch on a realistic value, a subtle off-by-one when pagination wraps, or user-controlled data reaching a file path. AI models are trained on code that handles the typical case; they consistently underspecify the atypical case because the prompt didn't mention it.

IN PRACTICEReceiving review without ego

Half of review culture is about the author, not the reviewer. Engineers who resist review don't usually say "I hate feedback." They say things like "my teammate is being nitpicky," or "they don't understand my approach," or "they would have done it differently so they're wrong." These are all ego-adjacent reactions to perfectly normal review comments, and they're corrosive to teams that depend on shared ownership.

The reframe that works: your reviewer is your best chance to be right before production. Every BLOCKER comment is a defect that didn't ship. Every QUESTION is a future maintainer telling you the code is not yet clear enough. The comment is data about how your code lands on a reader — and that reader is the metric that matters, because software is read far more than it's written. (You learned that in Lesson 2.1.)

1

Read every comment before responding to any

Don't respond to comments as they arrive. Read the full review first. Many comments that feel like attacks are fair points; many apparent contradictions resolve when you read the whole context. Reactive responses to individual comments are where "this is nitpicky" arguments start.

2

Distinguish "they're wrong" from "I disagree"

Both are valid, but they require different responses. "They're wrong" requires evidence: a failing test, a spec reference, a concrete counterexample. "I disagree" means you have a different preference — state it, explain your reasoning, and be prepared to defer. Teams that can't distinguish these two get stuck in style arguments that block real work.

3

Respond to every comment, even if it's just "done" or "I disagree because…"

Silent resolution (you made the change but didn't reply) leaves the reviewer uncertain whether you understood their point or just applied it mechanically. A one-word "done" closes the loop. A "I went with X instead because Y" teaches your reviewer something about the code's constraints.

4

If you added tests after the review, say so

Reviewers should never have to re-read the whole diff. "Added a unit test for the empty-input case in the paginate section" tells them exactly where to look and closes the feedback loop efficiently.

CORE CONCEPTThe new discipline: adversarial review of AI-generated code

Here is the failure mode this lesson exists to prevent. You use an AI coding assistant — which you already do, and should — and it generates a 150-line pull request implementing TraceLab's pagination. You read it. It looks clean: type hints, docstring, passes the linter, plausible variable names. You approve it. Three weeks later, a robot logging session that rolls over a page boundary corrupts its index, and you spend six hours in a debugger before you find the off-by-one that was right there in line 11.

The root cause isn't that you used AI. It's that you reviewed the diff as if it were human code — code written by someone who understood the invariants, tested the edges, and made intentional tradeoffs. AI-generated code is different in a specific, systematic way:

Human-authored code fails… • Misunderstood requirements • Off-by-ones in hurried code • Wrong API (misread docs) • Inconsistent style, clues left • Reviewer can ask the author • Failure mode is usually local Signal: code looks uncertain AI-generated code fails… • Plausible code, wrong invariant • Boundary conditions not in prompt • Silently drops errors • Confident style, no uncertainty cues • No "author" to ask — you own it • Failure mode can be systematic Signal: code looks confident
The key difference: human code gives you uncertainty signals — messy variable names, comments explaining the hack. AI-generated code is uniformly clean whether the logic is correct or wrong. Confidence is not a correctness signal.

The practical review discipline for AI-generated PRs is adversarial by default. Treat every function as if the prompt that generated it said "implement the happy path" — because statistically that's what the prompt said. Then ask four questions before approving anything:

1

What does this function do when the input is empty or zero?

Empty lists, empty strings, zero-length pages, missing keys in dicts — these are the most common uncovered cases. Run the function mentally with empty input before you read anything else.

2

What does this function do when input types are realistic but unexpected?

A metric passed as a string "0.31" instead of a float 0.31. A run ID that happens to be numeric. A filename that contains a slash. Type hints in Python are unenforced annotations, not guarantees.

3

Does this function swallow errors?

AI code has a persistent habit of except Exception: pass or except Exception: return None. Both are silent failure — the caller gets nothing and never knows why. Search for bare excepts explicitly.

4

Does any user-controlled data reach a file path, query, or system call?

A run ID that becomes part of a path. A metric name interpolated into a log line. These are injection surfaces. The AI generated the code against a clean test value; it did not think about what run_id = "../../etc/passwd" does.

⚠ The rubber-stamp is the new code-monkey trap

Pre-AI, the code-monkey was the engineer who translated specs into code mechanically, without judgment. Post-AI, the code-monkey is the engineer who approves AI diffs mechanically, without adversarial review. The output is the same: code that runs, ships, and eventually fails in the way a reviewer would have caught — if they'd been reviewing instead of approving. The AI does the generation. Your job is now entirely the judgment layer.

TRACELABTraceLab v1 shipped — with one AI-generated PR reviewed and caught

This is the Stage 2 close. TraceLab v1 is the v0 script restructured into a real package — typed, tested, proper src/ layout, locked dependencies, designed error handling, clean git history. Here is the structure you've built across Stage 2:

tracelab/ (v1 package layout)Stage 2 milestone
tracelab/
├── src/
│   └── tracelab/
│       ├── __init__.py         # public API surface
│       ├── cli.py              # tracelab init / log / compare
│       ├── storage.py          # CSV I/O with csv.writer, proper encoding
│       ├── models.py           # Run dataclass with typed fields
│       └── compare.py          # metric diffing between two runs
├── tests/
│   ├── test_storage.py
│   ├── test_compare.py
│   └── test_cli.py
├── pyproject.toml              # locked deps, entry point
└── README.md

Now the AI-generated PR. An assistant produced a paginate_runs() function for the storage.py module — pagination is a real feature (the compare command needs it for long run logs). Here is the diff as submitted:

storage.py — AI-generated paginate_runs()PR #7 — find the defect before reading on
def paginate_runs(runs: list, page: int, per_page: int) -> list:
    """Return a single page of runs (0-indexed).

    Args:
        runs: full list of Run objects
        page: page number, starting at 0
        per_page: number of runs per page

    Returns:
        A list of Run objects for the requested page.
    """
    start = page * per_page
    end = start + per_page
        return runs[start:end + 1]

Did you catch it? end + 1 on the final line. The AI added a fencepost correction that's completely wrong — slicing is already exclusive in Python, so runs[0:10] gives you elements 0–9. The + 1 gives every page one extra element, drifting the count; on the last partial page it silently returns an extra run from the next page. It's confident, it passes a linter, and the docstring even says "for the requested page" — just not correctly.

🔑 The test that would have caught it automatically

The test you should have written before merging (and would write in TDD, as you learned in Lesson 2.6):

tests/test_storage.pythe missing test
def test_paginate_runs_exact_page_size():
    """Each page must contain exactly per_page items (except possibly the last)."""
    runs = list(range(20))               # 20 fake runs
    page_0 = paginate_runs(runs, page=0, per_page=5)
    page_1 = paginate_runs(runs, page=1, per_page=5)
    assert len(page_0) == 5              # FAILS: returns 6 due to +1
    assert page_0[-1] != page_1[0]      # no overlap between pages

def test_paginate_runs_last_page():
    """Last page may be partial — must not overshoot the list."""
    runs = list(range(7))
    last = paginate_runs(runs, page=1, per_page=5)
    assert len(last) == 2               # PASSES despite the bug: Python clips slice at list bounds, masking the +1 on partial pages

def test_paginate_runs_empty():
    """Empty run list returns empty page, no exception."""
    assert paginate_runs([], page=0, per_page=10) == []

Three tests, fifteen lines. They would have caught the defect instantly — before it ever reached review. This is the TraceLab v1 lesson: ship the PR with the tests that prove it's correct, and review AI-generated diffs with the adversarial question list, not a rubber stamp. TraceLab v1 is now locked, typed, tested, and has a clean git history. Stage 3 starts from here.

WEAKNESSWhere code review goes wrong

⚠ Review-as-gatekeeping — the death of velocity

When review becomes a power asymmetry — senior engineers blocking PRs over style, or teams where "approved" requires a specific person who's always busy — the result is a queue of blocked work, authors who game the system by making tiny PRs to avoid review, and knowledge that flows in one direction only. Review should be lateral and distributed. If your team has a "must be approved by X" rule for most PRs, X is a bottleneck and the team's throughput is capped at X's availability.

⚠ Approving for social reasons

The approval you give because the author is a senior engineer, because you don't want to hold up the sprint, or because the review has already been waiting three days — that's a rubber stamp with extra steps. Every engineering post-mortem on rubber-stamp culture finds the same thing: the defect was in the diff; several people saw it; nobody said anything. The social pressure to approve is the single most common cause of code review failure at teams that nominally have review. Say the thing. Blockers block.

⚠ Reviewing AI diffs as if they were human code

This one is new and deserves its own callout. AI-generated code passes superficial review almost every time — clean formatting, no obvious typos, plausible structure. The defects live at the boundary conditions, in the swallowed exceptions, and in the security surfaces. If you don't have an explicit adversarial checklist for AI diffs, you don't have a review — you have a syntax check with extra steps. TraceLab's paginate_runs was right there.

KEY INSIGHTHow this connects

  • Code Is Read 10× More Than It's Written (2.1) — the reason review matters: the reader is the metric. A diff that's clear to its author and opaque to the reviewer has failed the primary purpose of code.
  • Testing I: What Tests Are Actually For (2.6) — the three tests that would have caught paginate_runs's off-by-one in fifteen lines. Review and testing are not substitutes; they catch different things, and AI-generated PRs need both working in concert.
  • 3.1 · HTTP: The Lingua Franca → — Stage 3 opens with the protocol layer. TraceLab v1 you've just shipped will grow a REST API; every decision you made about error handling and typed interfaces in Stage 2 pays its rent there.

Code review is the skill where everything else in Stage 2 comes together — readable code, tested invariants, typed contracts, clean git history. Review is how a codebase stays coherent across multiple contributors and across time. Now that AI generates a significant fraction of every team's diffs, adversarial review isn't an advanced technique. It's the baseline.

GLOSSARYKey terms introduced

BLOCKER
A review comment identifying a correctness defect, security hole, or broken invariant that must be resolved before the PR can merge. Always accompanied by a concrete reproduction case.
Rubber stamp
An approval given without genuine review — for social, time-pressure, or authority-deference reasons. The single most common cause of review-stage defect escapes.
PR sizing
The practice of keeping pull requests small enough (typically under 400 lines of diff) for a reviewer to hold the whole change in working memory. Above that threshold, review quality degrades empirically.
Adversarial review
A review mode appropriate for AI-generated code: assume the happy path is correct and actively probe boundary conditions, silent error handling, and injection surfaces.
Fencepost error
An off-by-one where a boundary index is wrong by one — classic in pagination and slice arithmetic. AI models reproduce fencepost errors at high rates because they optimize for common-case plausibility.
Silent failure
A code path that swallows an exception and returns None or an empty result without signaling the error. AI-generated exception handlers are a frequent source. The caller has no way to distinguish success from failure.
Injection surface
Any point where user-controlled data reaches a file path, system call, or query without sanitization. Particularly dangerous in AI-generated code because the model was prompted with a clean test value and never encountered a malicious one.

ACTIVE RECALLCheck your understanding

Answer in your own words first, then reveal.

1What is code review primarily optimized for — and what does it catch poorly?›

Review is primarily optimized for knowledge transfer — ensuring the change is understood by at least one other person who has to maintain it. It also catches logic errors and API misuse reasonably well. It catches subtle security vulnerabilities and concurrency bugs poorly — these require dedicated tooling like static analysis, fuzzing, and race detectors to find reliably.

2What is the practical PR size limit for maintaining review quality, and what should you do when a feature is larger?›

Review quality degrades empirically above roughly 400 lines of diff, ideally keep it under 200. For larger features: split the PR — a structural/refactor PR first (changes the architecture but not the behavior), then the feature on top. A PR you can summarize in one clear sentence is the right size.

3What makes reviewing AI-generated code different from reviewing human-authored code?›

AI-generated code is plausible by construction — it looks clean and confident regardless of whether it's correct. Human code leaves uncertainty signals (messy names, hedging comments). AI code systematically underspecifies boundary conditions not mentioned in the prompt, swallows errors, and may expose injection surfaces because the model was prompted with a clean test value. The appropriate response is adversarial review: assume the happy path is correct and actively probe the edges.

4What are the four adversarial questions you should ask on every AI-generated function before approving?›

(1) What does this do with empty/zero input? (2) What does it do with realistic but unexpected types? — type hints are unenforced in Python. (3) Does it swallow errors? — search explicitly for bare except clauses and return None in exception handlers. (4) Does user-controlled data reach a file path, query, or system call? — injection surfaces the model never tested.

Stage 2 complete — TraceLab v1 shipped. Stage 3: systems thinking begins →