Tokenmaxxing Is Over. What Should Engineering Leaders Measure Instead of AI Usage?
Leaderboards ranked engineers by tokens burned, employees pointed agents at pointless work to climb them, and the companies that built them spent 2026 switching them off. Usage was never the outcome. Here is what to measure instead, and what to ask of any team, in-house or outsourced, that reports its AI adoption to you.
What Is Tokenmaxxing, and Why Did It Spread So Fast?
Tokenmaxxing is the habit of treating the number of AI tokens an employee consumes as a proxy for how productive, modern or committed they are. The term started as a Silicon Valley joke and became a management practice with surprising speed. Its logic is easy to follow: leadership has spent heavily on AI coding tools, wants to see them used, and tokens are the one number every vendor dashboard already exposes. Counting them is free. Interpreting them is the hard part, and most organisations skipped that step.
The cultural signal came from the top. In April 2025, Shopify CEO Tobi Lütke circulated a memo saying AI use was now a baseline expectation, that staff would need to show why AI could not do a job before requesting headcount, and that the company would be adding AI usage questions to its performance and peer review questionnaires. Other leaders copied the framing, and in many companies a reasonable instruction, learn these tools, quietly turned into a measurable one, use these tools more than your colleagues.
HR practice followed. HiBob's 2026 study of 1,200 AI decision-makers, fielded between February and March, found that 67% of companies link AI skills to promotion criteria and 50% tie them directly to performance ratings, while only 36% of respondents who expect managers to build AI capability see them as highly prepared to do it. That combination is the recipe for tokenmaxxing: a metric that affects careers, and managers who are not equipped to look beneath it.
The most famous artefact of the era was a dashboard at Meta nicknamed Claudeonomics. Built by an employee rather than by the company, it ranked the top 250 of Meta's more than 85,000 employees by token usage and awarded titles such as "Token Legend" and "Cache Wizard". It was meant to be fun. It also made explicit what many engineers already suspected: in the eyes of their organisation, burning tokens had become a way to be seen.
How Did the Biggest Adopters End Up Switching Their Dashboards Off?
Amazon is the clearest case study because the incentive and the outcome are both on the record. The company set a target of more than 80% of its developers using AI every week and tracked activity on Kiro, its developer platform, through an internal ranking called Kirorank. According to Financial Times reporting summarised by The Decoder, some employees pointed AI agents at pointless tasks to climb the rankings, driving up cloud costs. Senior Vice President Dave Treadwell reportedly told staff not to use AI just for the sake of using AI, and Amazon replaced the leaderboard with a metric it calls normalised deployments, intended to capture AI-generated code that is actually useful.
Meta followed a similar arc. Claudeonomics was taken down two days after the press reported on it, with the shutdown notice citing data being shared externally. Then, in September, WIRED reported that Meta had revised its performance-review guidance so that AI adoption dashboards and token counts would not be used to evaluate employee impact. Meta's position is that contributions were always the basis of evaluation. The fact that it needed to say so in writing tells you how employees had been reading the signals.
Neither company abandoned AI. Both still want their engineers using it heavily, and both are investing at enormous scale. What they abandoned is the idea that the amount of AI consumed is evidence of value produced. That is a narrower and more useful lesson than "AI mandates failed", and it applies just as much to a twenty-person product team as to a company with tens of thousands of engineers.
There is also a less visible version of the same story playing out in smaller organisations. Few mid-sized companies built public leaderboards, but many added an "AI adoption" line to quarterly engineering reports, asked team leads to explain low seat utilisation, or praised the heaviest users in all-hands meetings. Engineers notice which numbers leadership celebrates. If the celebrated number is usage, usage is what you will get.
Why Does Token Volume Fail as a Productivity Metric?
Goodhart's law, that a measure which becomes a target stops being a good measure, is usually invoked loosely. Token counts are an unusually pure example, because the cost of inflating them is close to zero for the individual and paid entirely by the company. An engineer can double their consumption by asking an agent to re-explore a codebase it has already indexed, by choosing a larger model than the task needs, or by running speculative branches nobody intends to merge. None of that is visible on a usage dashboard as waste. All of it is visible on the invoice.
Even honest usage is a weak signal, because the relationship between AI activity and delivered value is not linear. Faros AI's telemetry from more than 10,000 developers across 1,255 teams found that developers on high-adoption teams complete 21% more tasks and merge 98% more pull requests, while PR review time increases 91%. Individual output rose. The constraint simply moved downstream, to the people who have to read, test and own what was produced. A dashboard that celebrates the first number without showing the third is rewarding the creation of a bottleneck.
Google's DORA research reaches a compatible conclusion from a different direction. Its 2025 report, based on nearly 5,000 technology professionals, found nearly universal AI adoption and a positive relationship with throughput, alongside continued increases in delivery instability, and described AI's primary role as that of an amplifier. An amplifier makes a good system better and a weak one worse. Measuring how loudly it is turned up tells you nothing about which kind of system you have.
Finally, usage metrics punish exactly the engineers you most want to keep. Senior engineers often use AI selectively: they know which tasks a model handles well, they write tighter prompts, and they skip the tool when the change is faster to make by hand. On a token leaderboard they look like laggards. Junior engineers, who may lean on agents for everything, look like champions. If performance reviews follow that ranking, you are paying a retention cost to get a worse signal. We covered the wider skills side of this in our piece on AI deskilling.
Key Takeaways
- Inflating token counts costs the individual nothing and the company real money
- AI raises individual output while shifting the bottleneck to review, testing and ownership
- Usage tells you how much AI is running, not whether the delivery system is getting better
- Selective, expert use of AI looks like low adoption on a leaderboard
What Does It Cost When Usage Becomes the Goal?
The direct cost is the bill. Meta's dashboard offers a sense of scale: over 30 days, employees consumed more than 60 trillion tokens, and the top user averaged 281 billion, a volume that could have cost more than $1.4 million at the price of the cheapest Claude Opus model. Most companies are nowhere near that scale, but the dynamic is the same at any size. Once tools are billed per token, an incentive to consume becomes an incentive to spend, and finance teams discover it a quarter later. We looked at the budgeting side of this in our analysis of usage-based billing for AI coding tools.
The observability industry has noticed. In September, Splunk introduced Tokenomics, a capability that traces consumption across active users, teams, models, providers and tools, with integrations planned for Claude Code, Codex, Cursor, Windsurf and GitHub Copilot. The same announcement makes the key point plainly: token counts alone cannot tell leaders whether an AI investment is working. Spend has to be correlated with quality and impact data to mean anything.
The indirect costs are harder to see and usually larger. Speculative agent runs generate pull requests that someone has to triage or close. Oversized changes inflate review queues. Engineers who feel watched on usage stop doing the unglamorous, low-token work, reading code, talking to users, writing a careful design note, that prevents expensive mistakes. None of this shows up as a line item, but all of it shows up in lead time and in the defect rate a few months later.
There is a trust cost as well. When employees believe a metric is being used to judge them and leadership insists it is not, the gap becomes a story in itself. Amazon told staff its leaderboard would not feed into reviews, yet the dashboard still had to be withdrawn once gaming became visible. Clear, written separation between cost monitoring and performance evaluation is cheaper than repairing that trust afterwards.
When Does an Activity Metric Become a Legal Problem?
Usage data stops being an internal curiosity the moment it influences employment decisions. In July 2026, 26 former Meta employees filed suit in federal court in Oakland alleging that the company relied on factors such as productivity and AI token usage when selecting people for layoffs, and that this disadvantaged employees on medical leave or with disabilities. Meta denies the allegations and says workforce decisions were made by people, not AI. The case is unresolved, and nothing here should be read as a finding against the company.
The allegation itself is the lesson. An employee on parental leave, recovering from surgery or working with an accommodation will consume fewer tokens than a colleague who is not. Any metric that mechanically tracks presence and activity carries that bias by construction. If such a metric touches promotion, bonus or redundancy decisions, you need to be able to show that it was tested for disparate impact, that a human reviewed the outcome, and that the employee could contest it.
In Europe the bar is set by law rather than litigation. The EU AI Act lists AI systems used to evaluate worker performance or make promotion and termination decisions among the high-risk use cases in Annex III, which brings obligations around risk management, data quality, human oversight and documentation. A simple usage dashboard is not automatically a high-risk AI system. A tool that scores engineers on AI activity and feeds those scores into reviews starts to look a lot more like one, and the safest time to design around that is before you build it.
Works councils and data protection officers add a further layer in many EU countries, where monitoring employee activity at this level of detail often requires consultation or a documented legitimate interest. For companies with teams in several jurisdictions, the practical answer is the same everywhere: collect token data at team and project level for cost management, and keep individual-level usage out of evaluation altogether.
Key Takeaways
- Usage metrics track presence, so they encode bias against people on leave or with accommodations
- Feeding activity scores into reviews, promotions or layoffs creates legal exposure
- EU rules treat AI used for worker evaluation as a high-risk category
- Aggregate token data by team and project for cost, not by person for performance
What Should You Measure Instead?
The replacement is not a single clever metric. It is a small set of outcome measures that are hard to inflate without doing real work, read together. Most of them are familiar from DORA and from ordinary engineering management. What changes in the AI era is the emphasis: because generation is cheap, the measures that matter most are the ones that capture what happens after code is written.
Start with flow and stability. Lead time from commit to production and deployment frequency tell you whether more work is actually reaching users. Change failure rate and time to restore service tell you whether that work is safe. If AI adoption is helping, the first pair should improve without the second pair getting worse. If throughput rises while failures rise with it, you are buying speed with reliability, and a usage dashboard would have shown you only good news.
Then measure the bottleneck that AI creates. Review time per pull request, pull request size, and the share of AI-assisted changes reworked or reverted within 30 days are the clearest early warnings that generation is outpacing verification. Escaped defects per release and the ratio of time spent on new work versus fixes round out the picture. Our earlier piece on verification capacity covers why this is where most teams are now constrained.
Finally, keep cost in view, but as a unit cost rather than a volume. Cost per merged, shipped change, AI spend included, is a far better number than total tokens, because it rewards efficiency rather than consumption. A team that ships the same work with half the tokens should look better on your dashboard, not worse. This is the spirit of Amazon's move from raw consumption to useful deployments, and it is achievable without enterprise tooling: most teams can compute it from their CI system, their issue tracker and their AI vendor invoices.
| Question you are trying to answer | Usage metric (avoid) | Outcome metric (use) |
|---|---|---|
| Is AI making us faster? | Tokens consumed per engineer | Lead time from commit to production |
| Is the extra output safe? | Agent sessions per week | Change failure rate and time to restore |
| Is review keeping up? | Lines of AI-generated code | Review time per PR and PR size |
| Is the code any good? | Share of commits marked AI-assisted | Rework and revert rate within 30 days |
| Is it worth the money? | Total monthly token spend | Cost per shipped change, AI spend included |
| Are people adopting it? | Leaderboard rank | Team-level adoption, reviewed in retrospectives |
Key Takeaways
- Read flow metrics and stability metrics together, never one without the other
- Track review time, PR size and rework to catch generation outpacing verification
- Replace total token spend with cost per shipped change
- Discuss adoption at team level in retrospectives, not on an individual leaderboard
What Should You Ask a Development Partner About AI?
Tokenmaxxing has a commercial cousin. Many software vendors now lead with AI adoption statistics: every engineer has an agent, most commits are AI-assisted, usage is up quarter on quarter. Those claims are the outsourcing equivalent of a token leaderboard. They describe inputs. They tell you nothing about whether you are getting better software sooner, and they can hide the same problems, oversized changes, thin review, rising rework, that sank internal dashboards.
The questions worth asking a partner mirror the internal ones. What has happened to lead time and change failure rate on comparable engagements since the team adopted AI tools? How is AI-generated code reviewed, and by whom? What is the rework rate on AI-assisted changes? How are AI tool costs handled, are they absorbed in the rate, passed through, or invisible until an invoice arrives? A partner who can answer with numbers from real delivery is measuring the right things. A partner who answers with adoption percentages is measuring what is easy.
Contract structure matters too. If you pay purely per hour or per seat, a vendor has no particular reason to care whether AI makes the team more efficient, and some reason to prefer that it does not. If the engagement is anchored on delivery, a stable dedicated team with agreed outcomes, transparent reporting and shared visibility into the pipeline, efficiency gains show up as more delivered scope for the same spend. We discussed the commercial side of this in our piece on pricing outsourcing in the AI era.
Visibility is the underrated part. When your partner works in your repository, your CI pipeline and your issue tracker, the outcome metrics in the previous section come for free, and you do not have to trust a slide deck. When work arrives as a zip file or a monthly status report, you are relying on the vendor's own account of its productivity, which is exactly the situation tokenmaxxing taught everyone to distrust.
How Stepto Approaches AI-Assisted Delivery
At Stepto, we use AI coding tools extensively, and we deliberately do not report usage as a success metric to clients. A dedicated development team from Stepto works inside your tooling, so lead time, deployment frequency, change failure rate, review time and rework are visible to you in the same systems your in-house engineers use. Those are the numbers we discuss in reviews, because those are the numbers that describe what you are paying for.
Our engineering practice is built around the bottleneck AI creates rather than the output it generates. Senior engineers own review on every change, AI-assisted or not. Pull requests are kept small enough to be read properly. Agent-generated code is held to the same testing and security standards as anything else, and AI tool costs are managed at team level so they never become a hidden line item on your invoice. The goal is the one Amazon arrived at the expensive way: useful code in production, not activity on a dashboard.
The nearshore model helps here in a practical way. Our teams in Serbia work hours that overlap fully with the European business day and substantially with the US East Coast morning, which means review conversations, design discussions and incident follow-ups happen in real time rather than across an overnight handoff. That matters more than ever when the expensive part of delivery is verification and judgement rather than typing.
If your organisation is rethinking how it measures AI-era engineering, whether for an internal team, a vendor relationship or both, we are happy to share how we instrument delivery on our own engagements. You can talk to us about a team that is measured on outcomes from the first sprint.
Key Takeaways
- Stepto reports delivery outcomes to clients, not AI usage statistics
- Teams work inside the client's repository, CI and tracker, so metrics are directly visible
- Senior review on every change and small PRs keep generation from outpacing verification
- Time zone overlap with Europe and the US East Coast keeps review and design work synchronous
Measure the Software, Not the Spend
Tokenmaxxing was an understandable mistake. Companies had invested heavily in AI tools, needed evidence that the investment was being used, and reached for the number that was easiest to collect. The result was predictable in hindsight: engineers optimised for the metric, costs rose, trust eroded, and the companies that went furthest were the first to switch their dashboards off. The better approach is already clear. Track AI spend for budgeting at team and project level, keep individual usage out of performance evaluation, and judge teams on lead time, stability, review load, rework and cost per shipped change. Apply the same standard to every partner who builds software for you. AI is a powerful amplifier, and the only way to know what it is amplifying is to measure the work that reaches production.
Building a team in Eastern Europe?
StepTo helps European and US companies build senior-led nearshore engineering teams in Serbia. Let's talk about what your next engagement could look like.
Start a conversationMore from StepTo on this topic
- Your DORA Scores Look Great. Your Engineering Team Is Still Stuck. Here's Why.
Deployment frequency is at historic highs while change failure rates climb. Why elite DORA scores are now misleading engineering leaders.
- The AI ROI Reckoning: Why Your Board Is Right to Ask Hard Questions, and How Engineering Leaders Should Answer
McKinsey: 88% of enterprises use AI, just 39% see any enterprise-level EBIT impact. It's a measurement failure, here's how value-capturing teams differ.
- The Code Took a Morning. The Story Took a Week. How AI Coding Agents Broke Software Estimation
Coding agents made writing code close to free, and estimates did not get better. They got worse. Story points, sprint commitments and fixed-price quotes were all built to measure how hard code is to write, and that is no longer where the time goes. Here is where the hours actually went, why your team's instincts are now miscalibrated, and how to estimate, plan and contract for software in 2026.
Written by
Igor GazivodaFounder & CEO · StepTo
Igor has 15+ years in software engineering and business development. He specializes in scaling engineering teams, nearshore strategy, and AI-driven product development. He holds a Master's in Computer Science from the University of Belgrade.
LinkedIn →