AI teams ship more code and deliver less: what to measure

PR volume is up and the product hasn't moved. Four measurements that expose where AI turns into debt: review queue, 30-day rework, mutation score and real customer use.

PR volume went up and the review queue is where the AI gain disappears.

LeadDev ran a piece last week called "AI-powered teams ship more code but deliver less". The short version of why: writing code was never the bottleneck. Reviewing, integrating, fixing what broke and keeping the thing alive for two more years is the bottleneck. AI made the cheapest step faster and dumped volume onto the three expensive ones.

Want to know whether it's happening on your team? Ignore lines written and ignore how fast people feel. Look at four numbers: pull requests opened per week versus changes that reached production, how long a PR sits waiting for review, how much of what shipped comes back as a bug or rework within 30 days, and how much of your test suite actually fails when the code breaks. The first three come out of your own board in an afternoon. The fourth one tends to ruin the day.

More material on site, same floor of the building

On site, a truck unloaded at the gate isn't progress. Same should apply to your board.

Before software I signed off on steel structure assembly. On a construction site nobody measures progress by trucks unloaded at the gate. You measure by sections erected, aligned and cleared by inspection. Material stacked on the ground is inventory, and inventory on site takes up room, rusts and gets in everyone's way.

A pull request waiting for review is inventory. Work you already paid for that hasn't turned into product yet, and it ages fast: every day it sits there, more chance of conflict with what someone else touched, more context gone from the author's head, more expensive for the reviewer to page it all back in. A team that doubled PR volume without doubling review capacity didn't get twice as productive. It got twice as much stuff parked.

The ugly part is that the dashboard looks great in this scenario. Commits climb, the activity graph is green, everyone feels quick. The date the customer gets to use the thing doesn't move. When we looked at what changed in time to market between 2025 and 2026, that gap between activity and delivery was exactly where teams fooled themselves.

The review queue is where the gain disappears

An AI code review CLI pulled 284 points and 73 comments on Hacker News. New tools only make that kind of noise when the pain is already widespread. And the pain is easy to state: you multiplied code output by a large factor and kept the same headcount able to say "this part is wrong".

A bot handles the first layer well. Style, patterns, a missing call, an odd dependency, an unused import. What it doesn't catch is the business rule somebody agreed with sales in 2019 and never wrote down anywhere. I've gone through where automated review replaces a human and where it does not, and the line hasn't moved much, even with better models.

So measure the queue, which costs nothing. Median time between opening a PR and getting the first human comment. If that's consistently over one business day, writing stopped being your bottleneck a while ago, and hiring more people to write will make the bill worse. Worth checking against the PR review time benchmark for remote teams before you argue with your team about whether the number is good.

92% coverage, 41% mutation score

92% coverage on the report and roughly 41% mutation score on the module that carries revenue.

I picked up a project reporting 92% test coverage. The kind of number that makes a steering committee relax. I ran mutation testing on the module that carries revenue and roughly 41% of the mutants died. In plain terms for whoever signs the budget: you could change more than half the system's behaviour and the suite stayed green, and the deploy went through.

The cause was tests with no assertion. Code that calls the function, claims nothing about the result, and passes. Coverage measures lines visited. Guaranteed behaviour is a different measurement, and almost nobody takes it. Same theatre as the health endpoint that returns 200 without checking the database, the queue or the third party: complete on the outside, hollow underneath.

What AI did here was make it cheap. Generating 300 test cases takes two minutes now, and a slice of them are born without an assertion that matters. Coverage climbs, the feeling of safety climbs with it, and the number of incidents reaching customers stays flat. Across the broken codebases we opened and measured, that pair of symptoms shows up with boring regularity.

"But my team has been faster since we adopted AI"

Probably true. And on plenty of fronts it's true with no catch at all: internal scripts, mechanical file migrations, CRUD boilerplate, integration glue, the first draft of a test a senior later hardens. There the tool hands real hours back, and refusing that on principle is fighting the wrong fight.

The maths changes where the decision matters: data modelling, concurrency, transaction boundaries, what the system does when the partner API returns a 500 halfway through the flow. In those places writing was maybe 20% of the job. The rest is understanding the business and choosing the behaviour for the bad day.

One limit on my own argument, so I'm not selling rigour where it hurts: if you're three people, no legacy, chasing traction, measuring mutation score every week will cost you more than it protects. This whole checklist is for teams with paying customers and a system that can't go down on invoice day.

The four numbers, and how to pull each one this week

1. Pull requests merged this month against changes a customer actually used this month. If the first column grew 60% and the second stood still, what you have is inventory, not throughput.
2. Median time from opening a PR to the first human comment. Pull it straight from Git, no new tool, and use the median rather than the average, because one forgotten three-week PR skews everything.
3. Share of what shipped that came back as a bug, hotfix or revert within 30 days. That's the number that tells you whether the speed was real or you just borrowed work from next month.
4. Mutation score on one critical module, the one that touches money. Run it once, compare it to the coverage figure, and in an afternoon you'll know whether your suite is a guarantee or a decoration.

None of the four needs a committee, a paid tool or a six-month programme. It needs someone willing to look at the result and not like it.

The bill doesn't arrive in the sprint, it arrives in the contract

None of this is a tooling problem. It's an incentive problem. The common contract pays for a fixed scope, on a short deadline, with nothing written about who keeps the thing running afterwards. Under that design, code volume is great for the vendor and terrible for you, and AI just made the misalignment cheaper to execute. Nobody needs bad faith for this, the contract paying for the wrong thing is enough.

When the person writing is the same person paged at 3am on a Tuesday, the standard fixes itself. Nobody approves 900 generated lines in four minutes knowing they'll be woken up by them. That's why we embed a senior squad inside the client's team, with the names of the people actually in the repository written down, and why knowing which team will really ship your code matters more than the price gap between two proposals.

At your next team meeting, ask two questions in this order: how many pull requests did we merge last month, and how many of those changes did a customer use. If the second answer takes longer than the first, the material is already unloaded on site and the building is still on the same floor.