AI is consuming the internet faster than humans can refill it. A growing body of research warns that 'model collapse' — the gradual degradation of AI models trained on AI-generated content — could quietly undermine the coding tools your team depends on. Here's what it means, why it matters for engineering strategy, and what forward-thinking leaders are doing about it.
There is a strange feedback loop building at the center of the AI industry, and it doesn't get nearly enough attention in engineering strategy conversations. Here it is, plainly stated: the large language models that write our code, summarize our documents, and generate our content are trained on human-produced text. But AI tools are now producing an increasingly large share of everything published online — code on GitHub, articles on the web, comments in forums, documentation on developer portals. Which means the next generation of AI models will be trained, in significant part, on output from the previous generation of AI models.
Researchers call this 'model collapse.' The term first appeared in formal literature in 2023, but the concern has reached mainstream engineering discourse in early 2026. A widely-circulated Hacker News thread titled 'The Internet Is Getting Quieter — Who Will Feed the Next Generation of AI?' captured the anxiety well: 'We're not just burning through the training data we have. We're actively polluting the source it comes from.'
For most engineering leaders, this might sound like a theoretical concern for AI researchers at large labs. It isn't. The models your team uses to write code today are already the product of this dynamic. And the trajectory matters — because your AI-assisted development strategy is being built on an assumption of continuous model improvement that may not hold.
Model collapse, at its technical core, is what happens when a generative AI model is fine-tuned or retrained on data that was itself generated by an earlier version of the model. The new model learns from its predecessor's outputs rather than from diverse, independently-produced human data. Over successive iterations, the distribution of outputs narrows: the model loses access to rare but important patterns, amplifies common errors, and produces text — or code — that is progressively more homogeneous and less reliable.
The analogy that has gained traction in research circles is photocopying a photocopy. Each generation of reproduction degrades the signal. The first copy is close to the original. By the tenth, detail is gone. By the twentieth, the output is a blur that bears only structural resemblance to the source.
The empirical evidence is beginning to accumulate. A 2025 study published by researchers at Oxford and the University of Edinburgh demonstrated measurable quality degradation in text generation models retrained through five successive rounds on synthetic data. The degradation was not uniform — factual recall declined earlier than stylistic coherence, meaning models could continue to sound plausible while becoming less accurate. In code generation specifically, a separate analysis found that models trained on high proportions of AI-generated code showed increased rates of subtle logical errors and security anti-patterns — errors that passed syntax checking and superficial review, but failed in edge cases.
The concern is not hypothetical. It's a structural property of how these systems learn.
Key Takeaways
For software teams, the most direct risk is in code generation. GitHub Copilot, Claude Code, Cursor, and their equivalents now produce — by some estimates — close to half of all new code committed in active engineering teams. That code lands in repositories. Repositories are a primary training source for the next generation of coding models. The signal that future models learn from is already, at significant scale, AI-generated.
This creates a compounding problem. Consider the lifecycle: a developer uses an AI coding assistant to write a service. The output is reviewed — but quickly, because the team is moving fast and the code looks reasonable. It gets committed. Six months later, it becomes part of a public dataset that a model provider uses for fine-tuning. The fine-tuned model now incorporates the patterns — including the subtle anti-patterns and overfitted solutions — from that AI-generated code. The next developer who uses the fine-tuned model receives code that carries the accumulated biases of prior AI outputs, not the diversity of human engineering judgment.
The practical result: the coding assistant that your team relies on to accelerate delivery may be subtly guiding your codebase toward patterns that are internally consistent but brittle at the edges — hard-to-catch bugs, weak error handling in uncommon paths, security assumptions that were valid for the training distribution but not for your specific context.
This is not a reason to abandon AI coding tools. It is a reason to be precise about how you use them, review them, and what you don't delegate to them — and to build a development process that doesn't assume AI output is getting unconditionally better with each model release.
Key Takeaways
Most engineering roadmaps in 2026 carry an implicit assumption: the AI tools we use will continue to improve. Model releases will be more capable. Code quality from AI assistants will trend upward. Productivity gains will compound. The investment thesis for AI-assisted development is partly built on this trajectory.
Model collapse puts a question mark on that assumption — not by suggesting AI tools will suddenly stop working, but by introducing a plausible scenario where improvement plateaus or selectively reverses in domains that matter. If the next generation of a widely-used coding model has better benchmark performance but higher rates of subtle security vulnerabilities — the kind that pass automated scanning and basic review — engineering teams that have reduced human review intensity in reliance on that model are exposed in ways they didn't anticipate.
The risk is particularly acute for teams that have systematically reduced senior engineer review in favor of speed and AI throughput. The value of a senior engineer reviewing AI-generated code is not just catching syntax issues — it's catching the judgment failures that the model cannot self-identify because the error is in the conceptual framing, not the syntax. If that review layer has been thinned, model quality degradation hits without a safety net.
There's also a competitive dimension. Organizations that build proprietary datasets of high-quality, expert-reviewed, human-validated engineering decisions — architecture choices, incident post-mortems, design documents, code review commentary — are accumulating a training asset that will not be affected by the degradation of public training data. This 'signal advantage' is not yet widely recognized as a strategic asset, but it is one.
A small but growing set of engineering organizations has begun responding to this risk with concrete process changes. The patterns worth noting are not dramatic — they are adjustments in emphasis that reflect a more nuanced model of what AI coding tools are and aren't.
The first pattern is investing in human code review quality rather than reducing it. Teams that have responded to AI-assisted development by speeding up review — approving AI-generated PRs faster because the code looks clean — are making the model collapse risk worse. Teams that have maintained or deepened senior review intensity, focusing reviewers specifically on the judgment-heavy layers (security boundaries, error handling, data model assumptions), are building a meaningful quality signal that will distinguish their codebases over time.
The second is building internal signal assets. Some organizations have begun systematically capturing high-quality engineering decisions — detailed architectural decision records (ADRs), post-incident analyses, design review notes — in structured formats that could eventually support internal fine-tuning or retrieval-augmented generation. The teams doing this aren't necessarily planning to train their own models. They are creating an institutional knowledge base that is independent of public model quality trends.
The third is maintaining what some engineering leaders now call a 'human-generated code core' — a set of system boundaries, security-critical paths, and architectural foundations where the policy is human-authored code, human-reviewed to a high standard, period. This is not Luddism. It is triage: recognizing that AI coding tools add enormous value in the 80% of work where the specification is clear and the stakes are recoverable, while preserving human authorship in the 20% where the failure mode is catastrophic.
The fourth, and most strategically interesting, is seeking external engineering partnerships that bring diverse, independent engineering judgment — not just additional throughput. The risk of model collapse is partly a monoculture risk: teams that use the same AI tools, trained on the same datasets, converge on the same patterns and the same blind spots. Nearshore engineering partners who apply different development cultures, different tool configurations, and different review processes introduce the kind of diversity that is actually protective against the feedback loop.
Key Takeaways
There is an underappreciated inversion happening in the value of human engineering judgment. For the past several years, the dominant narrative was about AI augmenting human developers — making individual engineers more productive, reducing the need for large teams, compressing delivery timelines. That story is real and it's not going away.
But model collapse introduces a second narrative that runs in parallel: human-generated, expert-validated engineering work is becoming a scarcer and more valuable signal precisely because the total volume of AI-generated code is expanding so rapidly. The ratio of high-quality, expert-authored code to total code is declining. The organizations that maintain that ratio — that keep senior engineers deeply engaged in consequential code, that build cultures of genuine review rather than superficial approval — are accumulating something that will be harder to replicate as the dynamic matures.
The practical implication for CTOs and engineering leaders: the question isn't whether to use AI coding tools. It's whether your development process is designed to preserve and amplify human engineering judgment, or to substitute for it. Those are different bets with different risk profiles — and in a world where model quality is not a guaranteed upward trend, the first bet has better downside protection.
The internet is getting quieter. The training data problem is real. The most resilient engineering organizations will be the ones that understood early that human judgment was not just a complement to AI tools, but the signal that keeps the whole system honest.
Model collapse is not a catastrophe arriving in a single event. It is a gradient — a slow narrowing of AI output quality that is difficult to detect in any single model release, but accumulates across generations. Engineering leaders don't need to panic about it. They need to design for it: by preserving the human review layers that catch what models miss, by building internal knowledge assets independent of public training data, by treating high-quality human engineering judgment as the strategic asset it is becoming. The teams that do this in 2026 will not just navigate the model collapse risk better — they will have built the organizational practices that produce genuinely better software, regardless of where the AI tools trend.
StepTo helps European and US companies build senior-led nearshore engineering teams in Serbia. Let's talk about what your next engagement could look like.
Start a conversationDarja
stepto.net