Harvard Researchers, the Linux Foundation and Google Report AI Coding's Effects With Different Instruments: Firm Data, a Survey and Agent Traces

Overview

Between October 9 and November 10, 2026, three pieces of research put numbers on the effects of AI coding tools beyond the speed of writing code, and outlets reported each one. Harvard University researchers analysed engineering-analytics data, as covered by Ars Technica and TechRadar. The Linux Foundation surveyed open source contributors and maintainers, as covered by The Register and TechRadar. Google published an engineering post on how coding agents react to stale code comments, as covered by InfoQ. This Analysis compares what each one counted and with what instrument. The three are the Herald’s own selection, not a census of such studies. The Herald’s reading, labelled as such under Implications, is that the headline figures sit on different scales and cannot be read against one another.

Background

Ars Technica opens its report by noting that AI coding assistants and agents can generate large amounts of working code, but that the output still has to be reviewed. The Herald frames the three reports as addressing where that effort lands: in the review queue, in the maintainer’s inbox, or in the agent’s own session.

The Herald has covered neighbouring ground. A News report on the Linux Foundation survey has a corrections record noting that its sources did not state the sample size or how “AI slop” was defined; the full report, read for this piece, does. An Analysis of how organisations describe measuring AI coding use dealt with usage counts. This one deals with reported effects.

Sourcing, counted by outlet: four outlets independent of the study authors are cited, Ars Technica, TechRadar, The Register and InfoQ. TechRadar covers two of the three cases. The Register restates the Linux Foundation release and InfoQ restates Google’s post, and each is described as such below. The Linux Foundation and Google pages are first-party and are not counted as independent.

Current situation

Harvard researchers’ firm data

Ars Technica reported on October 9 a study by Harvard University researchers Fiona Chen and James Stratton, using aggregated analytics data from Jellyfish. Per Ars, the data covers 300 million “work events” across more than 700,000 employees at over 700 firms, from 2021 through March 2026. The researchers dated each firm’s adoption of AI tools from “a mix of directly measured AI usage and analyses of GitHub activity” and ran a “difference of differences” regression before and after adoption.

Per Ars, introducing AI coding agents at a firm led to 30 percent more lines of code, 20 percent more commits and 23 percent more pull requests on average, while the resolution rate for Jira issues and epics “did not change in a statistically significant way”. The average time between a pull request being submitted and merged rose 49 percent, the share of pull requests with changes requested nearly doubled, comments per pull request rose 35 percent, and the share of workers performing reviews rose 14 percent. Ars also reports that 80 percent of measured firms used some AI code review by March 2026, with AI agents responsible for 23.3 percent of review comments and 10.8 percent of pull requests. It notes that there have been “significant updates and upgrades” to agents’ output since the study’s March 2026 data cutoff.

TechRadar, on October 12, gives the same headline percentages for code output, review time, comments and reviewers, and adds that the average review process went from 7.03 to 10.48 days. Two differences from Ars are worth noting. It attributes the 30 percent to “using AI” generally, where Ars says AI coding agents. And its summary line says “Comments per pull request are up 49%”, while its body gives 49 percent for review-and-merge time and 35 percent for comments, as Ars does. The Herald uses the body figures. Both outlets are accounts of the study; the Herald did not cite the paper itself and did not check these figures against it or against any later revision.

The Linux Foundation survey

The Linux Foundation announced the report on October 20, published with sponsorship from Red Hat. Its release says 25 percent of incoming pull requests “are estimated to be low-quality” and that respondents estimate a 52 percent average time reduction for code authoring. The full report describes a web survey distributed through Linux Foundation newsletters, social channels and partner channels, fielded April to June 2026, with 234 qualified respondents and a stated margin of error of +/- 6.41 percent at 95 percent confidence.

Two of the survey’s headline figures rest on smaller groups than the 234. The 25 percent comes from a chart labelled “Average estimated percentage of incoming PRs that are low-quality and/or AI-generated”, with 111 respondents. The 52 percent comes from respondents who use generative AI estimating how long a typical task took before and after, with 96 respondents; the same chart puts code review and feedback at a 44 percent estimated reduction. A separate chart on review time, whose caption gives 174 respondents, shows 52 percent in its “All maintainers” row saying AI-assisted contributions increased average review time and 17 percent saying they reduced it; the figures for large projects are 64 percent and 7 percent. The report also gives 134 respondents for its maintainer subsample, and the Herald did not reconcile that count with the 174.

The Register, on October 21, restates the release and keeps the qualifier: “Respondents estimated that a quarter of incoming pull requests” were low-quality “AI slop”. It adds the curl project’s announcement that week and a quote from maintainer Daniel Stenberg. It does not give the sample size. TechRadar, on October 22, writes that “Researchers found that artificial intelligence can reduce the amount of time it takes to write code by a staggering 52%”, without saying that the figure is a respondent estimate.

Google’s agent traces

Google’s Agent Distraction post, dated November 6, uses inline TODO(bug/…) comments as what it calls “the perfect proxy” for stale context. Drawing on traces from thousands of engineers and hundreds of millions of agent steps, it says 13 percent of file-reading sessions exposed the agent to such a reference. In 6 percent of encounters the agent acted on the bug, and 83 percent of those actions were unprompted. Of the unprompted detours, 41 percent concerned already closed tickets and 57 percent concerned still-open bugs outside the request. A regression controlling for developer, task type and model tier, among other variables, found those detours added 70 percent more steps, 77 percent more tokens and 105 percent more wait time per session. Google also says its cleanup agents removed thousands of obsolete references over six months with “exceeding 95% acceptance from human reviewers”.

InfoQ, on November 10, restates the post, adds comments from social media users, and reports no measurement of its own. Its headline refers to technical debt generally; its body says the researchers “focused on TODO(bug/…) comments”. Neither Google’s post nor InfoQ’s report states the period the traces cover.

Implications

The following is the Herald’s own comparison of the three accounts above, not a finding of any source.

Different instruments. The Harvard figures are changes in engineering-analytics data around firms’ adoption dates, per Ars. The Linux Foundation figures are respondent estimates and self-reports from a web survey, per the report’s own methodology page. Google’s figures come from a regression on agent traces described by the post’s Google authors, using one kind of comment as a proxy.

Different places and periods. The pull request queue, the maintainer’s inbox and the agent’s own steps are three locations. The Harvard data ends in March 2026 and the survey was fielded from April to June 2026, so the two do not cover the same months. Google’s period is not stated.

Numerals that collide. The Linux Foundation report uses 52 percent for at least two different quantities: respondents’ average estimated cut in code-authoring time, and the share of the “All maintainers” row saying review time increased. TechRadar’s summary line, which pairs 49 percent with comments rather than review time, shows how a figure can attach to the wrong quantity.

One apparent disagreement, explained by definitions. Harvard’s review measure is elapsed time from submission to merge, which rose 49 percent per Ars. The Linux Foundation’s review estimate is respondents’ own estimate of how long a review task takes them, down 44 percent. These are different quantities, and the Linux Foundation report also has maintainers reporting longer review times (52 percent in its “All maintainers” row), the same direction as Harvard’s. The sources do not test whether the two measures reconcile.

What follows for readers. The figures can be quoted accurately only with their instrument named, as The Register does by writing “Respondents estimated”. TechRadar’s “Researchers found” for a survey estimate drops the qualifier that separates an estimate from an observation. Among the three accounts above, only Google’s describes a tool its authors built, the cleanup agents, and it reports reviewer acceptance of that tool’s changes, not a measured change in detours.

What the sources do not settle

Ars notes that agents’ output has been updated and upgraded since the Harvard data cutoff, and neither Ars nor TechRadar reports figures for tools released after it. The Linux Foundation’s 25 percent is a mean of 111 respondents’ estimates, not a count of pull requests. Google’s post and InfoQ’s report name neither the agent tools nor the traces’ period. The Herald’s selection of three reports leaves out other studies, and no inference about the wider literature follows from it.