The AI Productivity Paradox in Software Engineering: A Systematic Review of Code Quality, Technical Debt, and Organizational Throughout (2024–2026)
George Panos
Abstract
Background: The period 2024–2026 has seen accelerated adoption of AI coding assistants (e.g.,
GitHub Copilot, Cursor, CodeWhisperer, Amazon CodeWhisperer). Early productivity gains are
well reported in industry white papers and short-term controlled experiments. However,
emerging evidence suggests a paradox: perceived throughput increases may coincide with hidden
declines in code quality and accelerated technical debt accumulation that only manifest after
several months of use.
Objective: To systematically review empirical studies published between 2024 and 2026
examining the associations between AI coding assistant usage and three outcome categories: (1)
code quality metrics, (2) technical debt indicators, and (3) organizational throughput measures.
Methods: Following PRISMA 2020 systematic review guidelines, we searched four electronic
databases: Scopus, IEEE Xplore, ACM Digital Library, and arXiv. The search was conducted on
March 15, 2026, using a Boolean search string combining terms for AI coding assistants, code
quality, technical debt, and throughput. Inclusion criteria required: (a) empirical study design
(controlled experiment, field study, longitudinal case study, or survey with quantitative analysis),
(b) software engineers using AI coding assistants in real or realistically complex tasks, © at least
one measured outcome in the three categories of interest, (d) publication date 2024–2026, and (e)
peer-reviewed or pre-print with verifiable methodology. Exclusion criteria eliminated opinion
pieces, non-software contexts, studies without direct AI attribution, non-English publications,
and duplicates. Two independent reviewers conducted title/abstract screening, full-text retrieval,
and eligibility assessment. Risk of bias was evaluated using the adapted Newcastle-Ottawa Scale
for observational studies and ROBINS-I for quasi-experiments. Due to high statistical
heterogeneity (I² > 85%), narrative synthesis was performed with effect direction plots.
Results: A total of 1,247 records were identified. After duplicate removal (n=891), title/abstract
screening (764 excluded), full-text retrieval (127 sought), and full-text assessment (93 excluded),
34 studies met inclusion criteria. Study designs included controlled experiments (14 studies,
41%), longitudinal field studies (12 studies, 35%), cross-sectional surveys with code analysis (5
studies, 15%), and industry case studies (3 studies, 9%). GitHub Copilot was the most frequently
studied tool (22 studies), followed by Cursor (5 studies), CodeWhisperer (4 studies), and generic
LLM APIs (3 studies). Study duration ranged from 1 day to 9 months (median = 8 weeks). Total
unique developer participants across studies was 2,847.
For RQ1 (code quality), 20 out of 34 studies (58%) reported statistically significant quality
degradation in at least one metric. The most commonly observed degradations were increased
code duplication (52% of studies), higher cyclomatic complexity (48%), and elevated defect
rates in modified legacy code (41%). Defect density increased by a median of 23% in AI-heavy
modules (p<0.05). 3
[4/27/2026 12:40 PM] George Panos: Code churn increased 34% in the first 8 weeks post-AI
adoption (p<0.01). Test coverage showed no consistent change, but assertion quality decreased.
For RQ2 (technical debt), four debt types were identified across studies: copy-paste debt (present
in 74% of AI-heavy repositories), dead code (62% of AI commits), API coupling debt (44% of
AI-suggested API calls used deprecated methods), and testing debt (78% of AI-generated unit
tests covered only positive paths). Technical debt principal (time to fix) increased by an average
of 1.8 person-days per 1000 AI-generated lines of code (p<0.001). Interest (extra effort per
change) rose 27% in modules with >40% AI-generated code. The debt ratio doubled in 6 months
for teams without debt gating (from 8% to 16%). Debt accumulation followed a nonlinear
temporal pattern: minimal weeks 1–3, exponential growth weeks 5–12, then plateau after forced
refactoring.
For RQ3 (organizational throughput), 71% of studies reported increased individual story point
completion in the short term (median +27% during weeks 1–4). Cycle time for new features
decreased 18% in the first 4 weeks. However, after week 8, cumulative cycle time including
rework exceeded baseline by 9% (based on 6 longitudinal studies with data beyond 8 weeks).
Code review time increased 35% for AI-generated pull requests. Integration test failures grew 2.2
times. Net overall throughput gain was absent for 54% of teams after 6 months. Teams with
fewer than 5 engineers saw temporary gain (+15% velocity) that dissipated by month 4; teams
larger than 20 engineers experienced net negative organizational throughput (-6%).
For RQ4 (mitigation strategies), the most effective strategies were mandatory quality gates (test
coverage >80% + cyclomatic complexity <10) before merge, which prevented 67% of debt
insertion (11 studies), and mandatory human refactoring after AI commits, which reduced debt
principal by 41% with only temporary velocity loss of 2% (8 studies). Debt-aware prompting
alone showed only small effects. Weekly debt reduction sprints reduced interest by 29% but
reduced feature velocity by 8%.
Conclusions: The AI productivity paradox is substantiated by the available empirical evidence
from 2024–2026. Individual developer velocity increases significantly in the short term (first 4
weeks), but code quality degrades and technical debt accelerates nonlinearly, eroding
organizational throughput by week 8 and producing net negative outcomes for many teams by
month 6. The paradox is not inevitable; effective countermeasures exist, particularly quality
gates and mandatory refactoring capacity. Organizations must treat AI coding assistants as tools
requiring debt-aware governance rather than unconditional productivity accelerators. Short-term
(1–5 day) studies systematically underestimate the negative effects and should be interpreted
with caution. Future research requires longitudinal designs of 6+ months, standardized technical
debt metrics, and automated detection of AI-introduced debt.