Reliability
The Friction Was the Signal

In 2025, METR ran a randomized trial with 16 experienced open-source developers using early-2025 AI tools in mature codebases they had contributed to for years. With the tools, they completed tasks 19% slower. When asked afterward, they estimated they were 20% faster.
The slowdown isn’t the interesting part. The gap is. These people were best positioned to know: years in those repositories, a direct comparison in front of them, and a sense of their own pace that overstated it by nearly 40 points. Their gauge wasn’t broken. It was reading a place the difficulty had already left.
Think about what the gauge used to read. An engineer adding a small feature to a tangled module had to read enough to be annoyed by its shape. The annoyance wasn’t virtue. It was information. It made the shortcut hurt before it became someone else’s problem, and it kept the engineer from fooling themselves about what they understood.
Engineering work has always carried a rough signal in its friction. An unexpectedly hard change meant the system wasn’t understood yet. A painful refactor revealed accumulated complexity. A week that felt slow was slow. It was never a clean instrument. Teams ignored it, normalized it, and mistook exhaustion for diligence. But it was ambient: messy code couldn’t hide its cost from the people touching it. The signal ran at every level, for free, and nobody had to design it.
AI didn’t eliminate that difficulty. It shifted the difficulty from producing a change, where the cost was early and felt by the person making it, to verifying, integrating, and operating one, where it’s late and felt by someone else. The important change isn’t that code got easier to produce. It’s that ease became a weaker proxy for understanding. Most of the industry has spent a year reading the quiet at the keyboard as good news.
The Struggle Stops Showing
DORA’s research on AI-assisted development identifies the first risk: lowering the barrier to a working answer can bypass the productive struggle that builds deep expertise. That’s the learning cost. But the struggle did a second job that gets less attention. It was observable. When an engineer got stuck on the same problem twice, the organization had something to notice: a pattern, a question repeated in review, a pull request that kept coming back. Output looked junior when understanding was junior, and the mismatch showed where help was needed.
Now output arrives looking finished regardless of what sits behind it. In LeadDev’s 2026 survey of engineering leaders, 42% said their teams had become less able to debug without AI assistance, while 10% said their teams’ ability had improved. It was the only competency in the survey where decline beat improvement, and it’s the one you need exactly when the tool has no answer. That moment used to arrive early and often, as something not working. It can arrive much later, in production, without anyone noticing it was overdue.
The Codebase Stops Hurting
The codebase didn’t get cheaper. It stopped charging the person writing the patch. Duplicated logic was still expensive to read, and tangled files were still slow to change. When the author was the one paying, refactoring felt like relief rather than overhead. Instead, the next person to touch the code pays, and the signal fires for someone who didn’t make the decision.
GitClear’s series of 116 million line changes from 2022 through 2025 shows a pattern consistent with a weakening signal. The share of changes that were moves or refactors fell from 20.6% to 6.9%, and copy-pasted lines rose from 9.6% to 16.9%. The data can’t say why. One plausible mechanism is that the tool did the pasting, so the cost of reading the result never entered the decision. No one had to decide to stop refactoring, and the prompt to do it may have come less often.
I’ve argued that the dread of maintaining your own code was the enforcement arm of every maintainability principle. It was also a reporting line. Agents don’t feel dread. A team that hands them enough of the friction can stop noticing it too.
The Organization Stops Seeing It
At the top, the same quiet reads as a dashboard. Faros AI’s 2026 telemetry study, tracking more than 4,000 teams from each organization’s lowest to highest period of AI adoption, found bugs per developer up 54%, monthly incidents up 57.9%, and incidents per pull request more than tripled. Median review time rose fivefold, and 31% more pull requests now merge without any review. Developers felt faster, and by individual output measures they were. The organization wasn’t. The difficulty didn’t vanish between those facts. It moved into the review queue and the incident count, which is exactly what the report finds.
The same report takes direct aim at a DORA finding from the year before: that strong engineering practices protect teams from AI’s downsides. Faros’s telemetry shows high-performing organizations seeing the same downstream deterioration as everyone else. That’s the METR gap again, one level up. A survey asks people how it went. Telemetry counts what happened. The two keep disagreeing the same way: the closer you are to making the change, the better things look.
Self-reported gains follow the same pattern. In LeadDev’s 2026 survey, a quarter of engineering leaders said AI had raised productivity by more than 25%, double the share a year earlier. The harder numbers measure different things. Sundar Pichai put Google’s internal estimate of company-wide velocity gain at about 10%, while METR measured controlled task time for one group on repositories they knew well. Neither tells a leader whether the engineers behind it still understand the systems they’re changing. Only 31% of organizations measure AI’s impact at all. Among those who do, the single most-tracked number is token usage: an input, counted because the outcome reading never arrives. Most of what gets called an AI investment decision is conviction standing in for proof.
That isn’t carelessness. When implementation was hard to fake, elapsed time and felt effort were honest proxies for output, and a leader could run an organization on them without naming them. Once the local signal weakens, its proxies move together. Confidence rises, activity counts rise, and apparent velocity rises with them, each seeming to confirm the others.
A Quiet Dashboard Isn’t a Clean One
The old instrument had three properties the replacement lacks: it was free, implicit, and already installed at every level. Rebuilding it costs something at every level, and isn’t about preserving struggle for its own sake. Aviation paid that cost on purpose: in 2013, after its data showed a rise in manual handling errors, the FAA urged airlines to build hand-flying back into line operations and training.
The few software teams doing the same are recognizable by what they ask for. Authors explain the design, not just present the diff. Senior output gets measured, not estimated. Review load and post-release rework are tracked alongside throughput, not under it. Some hard debugging is intentionally reserved for the people who need the reps, the same investment a team that only hires finished engineers keeps deferring.
Until the instrument is rebuilt, a quiet dashboard means one of two things, and the organization can’t tell which: the work is healthy, or the part of it that used to complain has been moved somewhere nobody is reading. If it’s the second, the bill arrives as it always does, in an incident, on a night when the tool has no answer. What’s new is that nobody will know, until that night, whether the person on call has ever had to find one without it.








