The Blind Spots of AI Code Review: Regressions and Decision Drift Beyond the Diff

Most AI code review tools work the same way: they read the Pull Request diff, glance at a few surrounding files, and comment on what they see — missing null checks, unhandled Promises, hand-concatenated SQL strings. They've gotten quite good at this, and if your team is already using one, keep using it.

But there's an entire class of bugs that's invisible to this approach. The diff looks fine; the problem lives somewhere the reviewer never looks. These fall into two categories: regressions outside the diff and decision drift.

Blind Spot #1: Regressions Outside the Diff

Imagine this scenario: a function gains a new required parameter. The author updates the function in the PR, along with the two call sites they remember. But three more call sites are scattered across other packages. They're outside the diff, so the reviewer never reads them. If one of those call sites passes through an any-typed adapter, the type checker won't catch it either. And as long as nobody tests that path, the tests pass fine.

The conclusion is almost ironic: the diff itself is correct, but the change is wrong.

This problem gets worse as coding agents become more common. When an agent modifies a function, it rarely goes looking for every call site unless you explicitly ask it to — and the PRs it generates tend to look clean, confident, and airtight. A reviewer reading only that PR won't find anything to object to.

To catch these issues, a reviewer must answer one question before reading the diff: Who depends on the code being changed?

Coding agents are AI systems capable of autonomously writing, modifying, and committing code — tools like GitHub Copilot Workspace, Claude Code, and Cursor Agent. Unlike traditional code completion, these agents accept high-level natural language instructions, plan and execute multi-step code changes, and produce a complete PR. Their output is typically well-formatted and internally consistent, but because they work by "looking things up on demand" rather than scanning globally, they tend to miss impact assessments on other modules unless explicitly prompted. As these tools become more prevalent in engineering teams, reviewers often place more trust in agent-generated code than in code written by human colleagues — which actually increases the risk of latent regressions slipping through.

Blind Spot #2: Decision Drift

The second category is quieter. Suppose your team made a deliberate decision a few months ago: all LLM calls must go through a shared client, so cost tracking and retry logic stay centralized. That decision lives in a merged PR, a review thread, and one line in CLAUDE.md.

Now a new PR calls the provider SDK directly from a new file. It runs. It has tests. But it's exactly the beginning of what you collectively decided not to do — and nothing in the diff signals that.

A general-purpose reviewer can't flag this, because it's not a general rule. It's your team's rule, living in your history, not in the changed code. To catch it, the reviewer needs to answer a second question: Which of our decisions apply to the files being changed?

Answering "Who Depends on This Code" Is Harder Than It Looks

The most straightforward tool for the first problem is a code graph: index the entire repository, then query the callers of every changed symbol. The authors built their regression check on exactly this — but they did something critical: they actually measured its accuracy, because missing one caller means missing one regression.

They used TypeScript LanguageService to annotate real call sites as ground truth, then compared against off-the-shelf code graph tools. On their own TypeScript monorepo, one out-of-the-box graph tool performed as follows:

• Recall of only 57.4%: found 27 of 47 real callers;
• 17 of 26 functions returned "zero callers", even though every single one had real callers;
• Some misses were within the same file, not just cross-package;
• Every empty result was labeled "exact" with risk level LOW.

That last point is what makes it genuinely dangerous. A tool that says "I don't know" is acceptable. A tool that says "nobody calls this, safe to change" when three places do call it is precisely why regressions get approved.

The authors ran a similar study on the open-source project Vite using a different engine. That engine correctly named a caller 67.9% of the time, and all 185 call sites fell within some caller it reported — it just misidentified the outer wrapper function rather than missing the call entirely. That's a much better failure mode, because a reviewer following the reported caller will still find the actual call.

Three lessons emerged:

1. An empty caller list doesn't mean safe. Treat it as "unknown" — search the codebase anyway, and read likely consumers before declaring a change has no impact.
2. Measure your graph against a real oracle on a real repo. A clean, small test case will show 100% accuracy and tell you nothing — gaps only surface at scale.
3. Pass uncertainty to the reviewer, not just the answer. Their impact analysis results carry an explicit "approximate" label plus a list of reasons the results may be incomplete (dynamic dispatch, re-exports, reflection calls). The review model can see all of this.

Answering "Which Decisions Apply" Requires a Decision Store

Team decisions are scattered everywhere: merged PRs, review threads (someone wrote "please don't do this, we use X"), ADR documents, READMEs, CLAUDE.md. To check a PR against them, you need them in one place — each decision with a scope (the paths it covers) and a source link.

The authors' approach is to automatically mine candidate decisions from history, but mined decisions are just guesses. So they exist first as "proposals," which team members approve, edit, or reject one by one. Findings that cite approved decisions can block merges; findings that cite unconfirmed proposals are surfaced with lower weight.

The link back to the source turns out to matter more than expected. "This violates a decision" is easy to dismiss. "This violates a decision from that PR six months ago, here's the thread" is much harder to ignore.

Keeping the Noise Down

Both types of checks are extremely prone to becoming noisy. A reviewer that fires on every caller of every changed function becomes wallpaper within a week. What the authors found to work:

• Two-stage review. A cheaper model pre-filters candidate findings; a stronger model then verifies each one against the actual code before anything is published.
• Every finding needs evidence. Either name the affected callers (with names and files), or cite the decision and its source. No evidence, no comment.
• Feedback persists. A finding that a team ignored doesn't reappear on the next push.

Parallel to Existing Reviewers, Not a Replacement

The core argument in the original piece is clear and pragmatic: diff-level review and context-level review catch different bugs, so you should run both. The diff reviewer catches the null check; the context check catches the caller in another package and the decision from six months ago.

The authors built these capabilities into a GitHub PR review tool called Tently, which checks every change against dependent code and team decisions. The same decision store is also available through an open-source MCP server (tently-team/tently-mcp) while an agent is writing code — so Claude Code, Cursor, or Codex can query the rules before the PR even exists.

Whatever tools you use, these two questions are worth asking for any PR — human-written or agent-generated:

1. Who depends on the code being changed?
2. Which of our decisions apply to the files being changed?

As the authors put it: if your reviewer can't answer them, then some team member is answering them from memory. And memory is the least reliable review baseline there is.

MCP (Model Context Protocol) is an open protocol proposed by Anthropic to standardize how AI models interact with external tools and data sources. Through an MCP server, developers can provide structured context — codebase information, documentation, APIs — to AI clients that support the protocol (such as Claude Code and Cursor), enabling models to actively query project-specific rules and constraints during code generation, rather than stuffing all context into a prompt. The tently-mcp mentioned here is a server implementation that exposes a team's decision store as an MCP tool, allowing agents to be aware of team conventions at the code-writing stage rather than discovering violations after the fact. This "shift left" approach — moving compliance checks earlier in the development lifecycle — aligns with the broader software engineering trend of pushing testing and security validation upstream.

Background Notes

A code graph is a static index of dependency relationships between symbols in a codebase — function calls, module imports, class inheritance — stored as a directed graph. Common construction approaches include semantic analysis via the Language Server Protocol (LSP), static analysis via AST (Abstract Syntax Tree) parsing, and dedicated tools like Sourcekit or tree-sitter. Accuracy is affected by multiple factors: dynamic dispatch (where the actual call target is only known at runtime), re-exports through files like index.ts (which truncate the import chain), and reflection calls (e.g., obj[methodName]()) — all cases where static analysis tools cannot trace the real caller. TypeScript LanguageService, Microsoft's official TypeScript language service API, performs semantic-level "find all references" queries with higher precision than plain-text or AST-level tools, making it a suitable ground truth for benchmarking other tools.

An ADR (Architecture Decision Record) is a lightweight documentation practice for capturing important architectural or technical decisions — including context, options considered, the final choice, and the reasoning behind it. ADRs are typically stored as Markdown files in the codebase (e.g., in a docs/adr/ directory), with each record corresponding to a single decision and a status such as "proposed," "accepted," or "deprecated." Unlike READMEs or inline comments, ADRs emphasize preserving the historical context of a decision so future team members can understand why something was done, not just what was done. CLAUDE.md, mentioned in the article, is a project convention file aimed specifically at AI coding assistants — a way to communicate coding standards, prohibited patterns, and team conventions to models like Claude. It represents an emerging form of "machine-readable team norms" in modern AI-assisted development.