Harvard Study: AI Coding Agents Raise Output Metrics, Not Software Output
A Harvard study of more than 700,000 employees found AI coding agents boosted lines of code and commits, but not Issue or Epic resolution rates.
A Harvard University study found that introducing AI coding agents at software firms produces sharp increases in code-production metrics but no statistically significant change in overall software output, according to arstechnica.com. The research examined aggregated analytics data from Jellyfish covering more than 700 software development firms and over 700,000 employees between 2021 and March 2026.
The study reported that firms adopting AI coding agents saw an average 30 percent increase in total lines of code, a 20 percent rise in total commits, and a 23 percent increase in pull requests. However, it found no statistically significant change in the resolution rate for Issues and Epics tracked by tools such as Jira after the AI tools were introduced.
The authors concluded there is little evidence that firms increase software output or reduce employment by using AI coding tools, with human code review acting as a bottleneck.
Background
Modern AI coding assistants and agents are efficient at generating large amounts of functional code, but substantial effort is needed to review AI-generated output for accuracy.
Quick answers
What did the Harvard study measure?
It analyzed aggregated analytics data from Jellyfish covering more than 700 software development firms and over 700,000 employees from 2021 through March 2026.
Did AI coding agents increase software output?
The study found no statistically significant change in the resolution rate for Issues and Epics tracked by tools like Jira, and concluded there is little evidence that firms increase software output by using AI coding tools.
What metrics did AI coding agents increase?
On average, they led to a 30 percent increase in total lines of code, a 20 percent rise in total commits, and a 23 percent increase in pull requests.