A mid-sized software company set itself a public goal in June 2025: double engineering productivity with AI, measured as merged pull requests per engineer per month. By April 2026 it had done it. PRs per developer were at 2.09 times the early-2025 baseline.

That's the headline. The part I keep coming back to is the line underneath it. Company-wide PR volume grew 3.1 times. The pool of people actually reviewing those PRs grew 1.5 times. Load per reviewer roughly doubled. And the company kept up the only way it could: it stopped having humans review a large share of the code.

Bar chart: PRs per developer 2.09x, company-wide PR volume 3.1x, developers acting as reviewers 1.5x, load per reviewer 2.0x

This comes from a longitudinal study by researchers at Carnegie Mellon and Stanford, covering 802 developers and 196,212 pull requests (arXiv:2607.01904). Its title says it all: AI Writes Faster Than Humans Can Review.

In September I argued in The AI Productivity Lie that AI didn't remove the delivery bottleneck, it moved it downstream onto review, testing and operations. This study is the next chapter, backed by more than two years of telemetry from a single company. It shows what an organisation actually does when the review queue fills up. Mostly, it routes around it.

I run agentic control points in production. In my own estate, an agent turns production traces into pull requests several times a day, and autonomous review gates check every change before a human sees it. So I'm not writing this as a sceptic of agent PRs. I'm writing it because I've watched the constraint move, and most dashboards haven't moved with it.

Key takeaways

Doubling output without doubling judgement doesn't make you twice as fast. It makes you twice as exposed.

Where the work actually went

Merge rates stayed flat in the 2× study, and revert rates, if anything, went down. That's the result an AI mandate wants to put on a slide. The authors are careful to call these "coarse, short-horizon proxies" that miss defects, incidents and maintainability. They read them as evidence against an acute quality collapse, "not a clean bill of health."

The more useful story is how review changed shape:

Grouped bars: any human review 89% to 68%; automated AI review about 19% to 84%; substantive human review about 39% to 21%; silent human approval about 50% to 50%

Read that last pair again. The extra load didn't go into deeper review. It went into approve buttons.

The supplement makes the queue visible. The 90th-percentile cycle time for PRs that got a substantive human review peaked near 114 hours and was still around 73 hours in 2026, above its roughly 63-hour level before the ramp. That queue never decongested. Aggregate cycle time only recovered because fewer PRs went through it: substantive-review coverage among merged PRs dropped from a pre-mandate peak of about 48% to about 21%. In the authors' words, the company "absorbed the queue by routing work around substantive human review, not by reviewing faster."

If your engineering report shows throughput going up and cycle time coming down, it's worth asking which of those two things is happening in your org.

A review bot alone is not a reviewer

The obvious answer to review overload is more review bots. The evidence says bots help as a layer, and fail as a replacement.

A 2026 MSR study looked at agent-authored PRs in the public AIDev dataset and compared those reviewed only by humans against those reviewed only by code review agents (arXiv:2604.03196).

Horizontal bars: merge rate 68.4% human-only vs 45.2% bot-only; closed without merge 21.6% human-only vs 34.9% bot-only

That's correlational, and the sample of bot-only PRs is small (281). But it lines up with what an earlier industrial study found when an LLM review tool was introduced: 73.8% of its comments were resolved, yet average PR closure time went up from 5 hours 52 minutes to 8 hours 20 minutes (arXiv:2412.18531). More comments aren't the same as more judgement.

The MSR authors' own advice is to configure review bots for narrow, specific checks, such as security issues or style rules, rather than general-purpose review, and to keep human approval before merge. I agree. A bot that enforces one standard sharply is a gate. A bot that comments on everything is just more for the human to read.

Not every agent PR costs the same

The good news is that review effort isn't spread evenly, and that makes it manageable.

Another MSR 2026 paper studied 33,707 agent-authored PRs (arXiv:2601.00753). It found two regimes. 28.3% merged almost instantly: small, narrow changes that rarely touched configuration. The rest went into back-and-forth review, where acceptance dropped to 68.7% and agents sometimes abandoned the PR after subjective feedback.

The useful part is that the expensive tail was predictable before any human looked at it. Using only signals available when the PR is opened (patch size, file types, whether the description includes a plan) the authors' model ranked high-effort PRs with an AUC of about 0.96 (about 0.83 on repositories it hadn't seen). Reviewing just the top 20% by predicted effort caught 69% of the high-effort PRs. PRs that stated a plan were less likely to be abandoned. Large, multi-file changes with no plan were the riskiest. Their practical advice is blunt: flag PRs over 500 additions, fast-fail those without plans, and time out stale ones after 14 days.

Who opened the PR matters too. A study of 37,623 provenance-labelled PRs from five agents and a matched human baseline found 90-day revert rates ranging from 6.1% for one agent to 14.5% for another, against 11.5% for humans (arXiv:2609.17598). The other three agents were statistically indistinguishable from humans. "AI PRs" is not a single risk category.

And for perspective, agent PRs are mostly acceptable work. In a study of 567 PRs made with one agentic tool across 157 open-source projects, 83.8% were merged, against 91.0% for matched human PRs, and 54.9% of the merged ones went in without further changes (arXiv:2509.14745, now in ACM TOSEM). The issue isn't that agent code is bad. It's that the other 45% of merged PRs needed human revisions, and human time is now the scarcest thing in the pipeline.

What I'd change in the pipeline

1. Report review as a capacity, not an activity

Throughput metrics were built for a pipeline where writing code was the slow part. That's no longer true. Next to merged PRs, put these on the same page:

If substantive review is falling while throughput rises, you're not faster. You're less checked. The 2× authors make the same recommendation: don't govern by merge count alone. I made a similar argument about what a CTO actually wants to hear about quality; this is the agentic version of it.

2. Triage every agent PR into a lane at creation time

You don't need a machine-learning model to start. Static signals got the MSR team most of the way, and you can write them as rules:

The point of lanes isn't to review less. It's to stop spending the same attention on a typo fix as on a change to how money moves.

3. Use bots as gates before humans, not as reviewers instead of them

This is how I run it: autonomous review gates enforce mechanical standards ahead of human review, so by the time a person opens the PR, the boring failures are already gone. The gate either blocks or passes. It doesn't leave twenty comments for someone to triage.

Keep each bot narrow, measure its signal ratio (how many of its comments led to a change), and turn off the ones that are mostly noise. A bot with a 20% signal ratio is costing your reviewers time.

4. Make the agent bring its own evidence

An agent PR should arrive with the things a reviewer would otherwise have to dig for:

That turns review from reconstruction into verification. It's also where the "has a plan" signal from the triage research becomes a requirement rather than a lucky feature.

5. Put a WIP limit on agent PRs

Agents don't get tired, so they'll happily open more PRs than your team can read. Cap open agent PRs per repository or per owning squad, and let the agent queue rather than the reviewers. When the cap is hit, that's a capacity signal for a leadership conversation, not a reason to approve faster. XP taught me to limit work in progress long before agents existed; it matters more now, not less.

A policy you can start with this week

agent_pr_policy:
  on_open:
    require: [plan, risk_lane_claim, linked_ticket_or_trace]
    bug_fix: at_least_one_new_test_fails_on_base_commit
  lanes:
    fast:
      when: { additions_lt: 100, components: 1, paths: [docs/**, tests/**, .github/**, deps] }
      approve: 1 human on green gates
    standard:
      approve: 1 human reviewer, substantive comment or explicit sign-off
    deep:
      when_any: [touches_auth, touches_payments, touches_pii, touches_infra_config,
                 additions_gt: 500, multi_file_without_plan]
      approve: 2 humans, one domain owner
  bots:
    role: blocking_gates_before_human_review
    track: signal_ratio_per_bot
  limits:
    max_open_agent_prs_per_squad: 10
    close_if_no_agent_response_days: 14
  report_weekly: [pickup_time, substantive_review_rate, silent_approval_rate, queue_depth_by_lane]

Tune the numbers to your own estate. Start with the weekly report; you can't manage a queue you can't see.

The leadership point

For years, the cost of software was dominated by the cost of writing it. Agents have made that cost close to zero for a lot of changes, and every org I talk to is measuring the speedup. Far fewer are measuring where the effort went. It didn't disappear. It moved into review, which is paced by human judgement, and in the best-documented case so far, the organisation coped by quietly doing less of it.

That's not automatically wrong. Plenty of agent PRs deserve a fast lane. But it should be a decision you made deliberately, with the risk written down, not something that happened because the queue was too long. The 2× authors end with a line every engineering leader should read: a single throughput number "captures the speed and hides that shift."

If your agents doubled your PRs this year, ask one question at your next quality review: who read them?

Continue reading

Sources