Karpathy coined 'vibe coding' and the internet ran with it. Now the data is in — and it's not what the hype predicted. 45% of AI-generated code has security flaws, experienced developers are measurably slower, and enterprises are quietly inheriting a quality debt they didn't budget for. Here's what the research actually says, and what it means for your next development engagement.
In February 2025, Andrej Karpathy posted a short note on X that would quietly reshape how the industry talks about AI-assisted development. 'There's a new kind of coding I call vibe coding,' he wrote, 'where you fully give in to the vibes, embrace exponentials, and forget that the code even exists.' The concept: describe what you want in plain English, accept whatever the LLM generates, ship it, and move fast.
The reaction was immediate and mostly enthusiastic. Collins English Dictionary named 'vibe coding' its Word of the Year for 2025. Developers on Reddit and X shared demos of weekend projects built in hours. Venture-backed startups claimed they were shipping with one-person engineering teams. The narrative wrote itself: AI has finally removed the barrier between idea and product.
By early 2026, something interesting happened. Karpathy himself called vibe coding 'passé' and moved on to championing 'agentic engineering' — a model with far more human oversight and judgment in the loop. The phrase hadn't aged a year before its own inventor was walking it back. That pivot is a signal worth paying attention to.
The question for engineering leaders isn't whether vibe coding is a good meme. It's what the actual production data shows about AI-generated code quality — and whether your outsourcing strategy, your code review processes, and your technical risk posture are calibrated for what's actually happening in 2026.
Key Takeaways
Vendor benchmarks for AI coding tools have been uniformly optimistic. GitHub reports Copilot users complete JavaScript tasks 55% faster in controlled experiments. Internal studies show PR cycle times dropping from 9.6 days to 2.4 days. These are real numbers from real studies — but they're also the numbers vendors publish.
The independent picture is substantially more complicated. The most striking data point of 2025 came from METR — a safety-focused AI research organization — in a randomized controlled trial with 16 experienced developers working on large, real-world open-source codebases (repositories with 22,000+ stars and over a million lines of code). The finding: AI tools made these experienced developers 19% slower, not faster.
The reason isn't that AI tools are bad. It's that on complex, established codebases, generating candidate code is fast — but evaluating, verifying, and integrating that code takes time. Developers accepted less than 44% of AI-generated suggestions, meaning the majority of generation cycles became review overhead, not productivity gains. The simple tasks AI handles well are often already fast. The hard tasks where speed would matter most are exactly where AI-generated output demands the most scrutiny.
The perception gap makes this particularly interesting for organizational decision-making. In the METR study, developers expected AI to speed them up by 24%. Even after experiencing the measurable slowdown, they still believed AI had made them 20% faster. This isn't a character flaw — it's a structural challenge for any CTO trying to make honest assessments of where AI tooling is actually delivering. When developers feel productive, it's hard to surface a data signal that says otherwise.
Bain & Company described real-world AI productivity gains as 'unremarkable' in a September 2025 report. Faros AI's analysis of engineering metrics across enterprise clients found no measurable improvement in delivery velocity or business outcomes, despite more than 75% of developers using AI assistants. These aren't anti-AI arguments — they're calibration signals.
Key Takeaways
If the productivity data is ambiguous, the security data is not. Veracode's 2025 GenAI Code Security Report found that nearly 45% of AI-generated code contains security flaws. When models are given a choice between a secure and an insecure implementation path, they choose the insecure path nearly half the time — not because they're malicious, but because insecure patterns are statistically common in training data and often syntactically cleaner.
CodeRabbit's analysis of 470 open-source pull requests found that AI-co-authored code contains 1.7x more 'major' issues than human-written code, with 2.74x higher security vulnerability rates and 75% more misconfigurations. GitClear's longitudinal study of 211 million lines of code written between 2020 and 2024 found an 8x increase in duplicated code blocks — a pattern consistent with LLMs copying and pasting from training data without architectural judgment.
Real incidents are accumulating. In March 2025, a vibe-coded payment gateway approved $2 million in fraudulent transactions due to inadequate input validation that had been faithfully reproduced from AI training data. The code worked. The tests passed. The logic was wrong in a way that only appears under adversarial conditions.
The underlying dynamic is structural: AI models generate plausible code. Plausible code passes superficial review. Security vulnerabilities are disproportionately concentrated in edge cases, adversarial inputs, and interaction effects between components — precisely the areas where 'does this look right?' review is least effective. The code that ships with a security flaw doesn't look wrong. That's the whole problem.
For enterprises building or extending software through outsourcing partners, this creates a specific risk: if your partner's developers are using AI heavily to accelerate delivery, and your review process hasn't adapted to catch AI-specific failure modes, you may be accepting technical and security risk that doesn't appear on any sprint dashboard.
Key Takeaways
The quality and security risk of vibe coding doesn't stop at the code generation layer. In November 2024, Anthropic open-sourced the Model Context Protocol (MCP) — a standard for connecting AI models to external tools, APIs, and data sources. The adoption has been genuinely extraordinary: within three months, over 1,000 community-built MCP servers existed. By December 2025, there were 97 million monthly SDK downloads, 10,000+ active MCP servers in production, and support from OpenAI, Google DeepMind, and Microsoft.
MCP is, in effect, becoming the npm of AI agent infrastructure. That's both what makes it powerful and exactly what makes it risky. The npm ecosystem's security history — typosquatting attacks, malicious packages, supply chain exploits — is a template for what happens when a fast-growing, community-driven package ecosystem scales faster than its security practices.
The current MCP landscape: approximately 75% of MCP servers are built by individuals, not companies, with no centralized security review and no vendor accountability. Command injection affects 43% of tested MCP implementations. Real incidents include a malicious MCP server exfiltrating a user's entire WhatsApp history, and a fake 'Postmark MCP Server' that silently BCC'd all email communications to an attacker. CVE-2025-6514, with a CVSS score of 9.6, affected 437,000+ downloads.
For engineering teams building AI-augmented workflows — which is increasingly every team — MCP is becoming unavoidable infrastructure. The question isn't whether to use it. The question is whether your development process has adapted to evaluate and audit MCP integrations with the same rigor you would apply to any other third-party dependency. Most haven't.
Key Takeaways
The practical implication for CTOs isn't 'stop using AI tools.' AI coding assistance is real, adoption is irreversible, and the productivity gains on well-scoped tasks — boilerplate, test generation, documentation, simple integrations — are genuine. The mistake is treating those gains as uniform across all development contexts.
The research suggests a specific segmentation: AI tools create measurable acceleration for junior developers on routine tasks, and measurable slowdown for experienced developers on complex systems. The typical offshore or staff-augmentation model — built around volume hiring of mid-level developers executing well-defined specs — is the model that AI tooling accelerates most. But it's also the model most likely to generate high volumes of plausible-looking AI code that doesn't undergo sufficient architectural review.
The development partners who are building durably in this environment share a few characteristics. They have clear internal policies about where AI generation is used and where senior human judgment remains non-negotiable — particularly in security-critical paths, API boundary definitions, authentication flows, and data handling. They treat AI-generated code as a first draft that requires expert review, not a finished product that requires acceptance. They have adapted their code review practices to specifically check for AI failure modes: duplicated logic, missing edge case handling, insecure defaults, and architectural inconsistencies that look fine in isolation but break under integration.
The 'AI-augmented senior team' model — small, senior-heavy, with AI tools accelerating output and senior judgment governing quality — is emerging as the pattern that actually delivers. It's more expensive per-developer than a large junior team. It's substantially cheaper than the hidden cost of security incidents, architectural rework, and the maintenance debt that vibe-coded systems accumulate over time.
Key Takeaways
If you're evaluating a development partner, extending an existing engagement, or reviewing how your internal teams are using AI tools, the questions have changed. 'Do you use Copilot or Cursor?' is not a useful signal. Every team does. The useful questions are about governance.
Ask how AI-generated code is flagged, reviewed, and tested differently than human-authored code. Ask which parts of the codebase are explicitly designated as requiring senior human authorship — security, authentication, data handling, financial logic. Ask what their process is for auditing MCP integrations and third-party AI tool dependencies. Ask for examples of AI-generated code they rejected or substantially rewrote, and why. Ask how their review capacity has scaled relative to their AI tool adoption.
The organizations that are managing this well have answers to these questions that reflect genuine process thinking, not marketing language. The ones that haven't thought about it yet will give you the marketing answers: 'We use AI to go faster.' That's true. It's also not the answer to the question you actually need to evaluate.
The vibe coding era taught the industry something important: generating code is now cheap. The constraint has moved upstream to judgment — the expert assessment of what to build, how to structure it, where the risks are, and what the AI got wrong. The development partners who understand that shift, and who have built their teams and processes around it, are the ones worth working with in 2026.
Key Takeaways
Vibe coding was never really about the code. It was about the promise that the friction between idea and implementation was finally gone. That promise was real — but incomplete. Generating plausible code is now cheap. Generating correct, secure, maintainable code is not, and the gap between those two things is where technical debt accumulates and security incidents originate. For CTOs making decisions about how to build software in 2026 — whether in-house, nearshore, or through any other model — the fundamental question has shifted. It's no longer 'how fast can your team write code?' It's 'how does your team govern the code that gets written?' The organizations that are answering that question well are building software with AI that holds up. The ones still riding the vibe will find out differently.
StepTo helps European and US companies build senior-led nearshore engineering teams in Serbia. Let's talk about what your next engagement could look like.
Start a conversationIvan
stepto.net