You Cannot Switch Models. That Is the Problem, and It Cost the Average Enterprise $315,000 to Find Out

Why This Lock-In Is Different

Enterprises have decades of experience with vendor lock-in and a well-developed instinct for managing it — abstraction layers, standard formats, exit clauses, multi-vendor sourcing. Those instincts underestimate this case, because the coupling forms in a place the old playbook does not cover.

Traditional lock-in is about interfaces and data. You are locked in because migrating a database is hard, or because a SaaS platform holds your records in a proprietary schema. Both are painful and both are tractable, because the semantics are stable: a customer record means the same thing after migration as before.

LLM lock-in attaches to behaviour, and behaviour does not port. Analyses of AI lock-in identify the sources consistently: provider SDK coupling, prompts tuned to one model, tool and function schema drift, embedding and retrieval dependencies, and operational tooling bound to a single vendor. The crucial difference is that your business logic is now partly encoded in prompts and model-specific integrations that have no equivalent on another provider. A prompt refined over eighteen months against one model's quirks is a real asset with real value, and it is worth considerably less against a different model — not zero, but not what you paid for it.

The financial shape is documented. Migrations cost an average of $315,000, and Gartner projects that by 2027, 60% of organisations will face significant LLM migration costs driven by vendor pricing changes, API deprecations or strategic pivots. None of those three triggers are within your control, which is the entire point: lock-in is not a state you choose, it is a state you discover when the provider changes something.

The Five Places Coupling Actually Forms

Portability work is only tractable if you know where the coupling lives. It is five specific places, and they differ enormously in how expensive they are to unwind.

Provider SDKs in application code. The cheapest to fix and the one everybody fixes first, which creates a false sense of progress. Calling a provider's client library from a hundred places in your codebase is annoying but mechanical to abstract. If this is your only portability work, you have addressed the least significant layer.

Prompts tuned to one model. Substantially harder. Prompts accumulate model-specific compensations — phrasings that work around a particular failure, formatting that a specific model respects, few-shot examples selected because they fixed a specific regression. Move to another model and much of that tuning is noise, and some of it is actively harmful. This is the largest single component of a real migration.

Tool and function-calling schemas. Providers differ in how tools are declared, how arguments are returned, how parallel calls and errors are handled. An agent system with twenty tools has twenty integration points shaped by one provider's conventions, and the differences are not always expressible as a mapping.

Embeddings and the retrieval layer. The most underestimated of the five. Embeddings from one provider's model are not comparable with another's — different dimensionality, different vector space, no meaningful translation. Switching embedding models means re-embedding the entire corpus, revalidating chunking and retrieval quality, and potentially reindexing a vector store with millions of documents. For large corpora this is the single most expensive line in a migration, and it is frequently discovered late.

Operational tooling. Observability, cost tracking, guardrails, caching, evaluation harnesses — all commonly built against one provider's telemetry format and semantics. Individually small, collectively a quarter of the work.

The Gateway Pattern, and What It Does Not Solve

The standard architectural answer is a gateway between your applications and providers — a unified endpoint that abstracts provider-specific API formats, handles routing and fallback chains, and enforces governance centrally. It is the right first move and it is genuinely valuable.

What a gateway buys you is real. One integration point instead of many. Centralised credential management, rate limiting and cost attribution — the last of which alone often justifies it, because per-team and per-feature AI cost attribution is nearly impossible without a central choke point. Fallback chains when a provider degrades. And a place to enforce guardrails and logging uniformly rather than hoping every team implemented them.

What a gateway does not buy you is the ability to actually switch. It normalises the API surface, which is the SDK coupling layer — the cheapest of the five. It does nothing about prompts tuned to one model's behaviour, nothing about embeddings in a provider-specific vector space, and nothing about whether the alternative model performs acceptably on your workload. A team that installs a gateway and declares portability solved has addressed roughly 20% of the problem and acquired confidence disproportionate to that.

The honest framing: a gateway is necessary infrastructure and insufficient strategy. It makes switching mechanically possible. What makes switching safe is knowing, in advance and in numbers, what happens to output quality when you do — and that is a different investment entirely.

The Evaluation Set Is the Actual Exit Strategy

Here is the part that gets left out of every portability discussion and is, in practice, the whole thing.

The reason organisations cannot switch models is not that the code will not compile against a different API. It is that nobody can say whether the new model is better or worse on their specific workload. Without that answer, switching is an act of faith with a customer-visible blast radius, and no sensible engineering leader authorises it. The lock-in is epistemic before it is technical.

The fix is a workload-specific evaluation set: a few hundred real inputs from your actual usage, with expected outputs or a scoring rubric, covering the normal cases and — more importantly — the edge cases you have already been burned by. Run it against the incumbent to establish a baseline. Then, when evaluating an alternative, run it again and read the difference. A team with this artefact can evaluate a new model in a day and make the switch decision on evidence. A team without it cannot switch at any price, because the risk is unbounded and unquantifiable.

This also converts a defensive investment into an offensive one, which is how it gets funded. The same evaluation set is what lets you adopt a cheaper model when one appears, and inference pricing keeps falling sharply enough that this is not a theoretical benefit — capability that costs a euro today has repeatedly cost a fraction of that within a year. Organisations with portable stacks and evaluation infrastructure capture those reductions within weeks. Organisations without them keep paying the old price because they cannot prove the cheaper option is safe. Over a large deployment, that gap dwarfs the cost of building the evaluation set several times over.

Build it from production traffic, keep it versioned in the repository, run it in CI on every prompt change, and expand it every time something goes wrong in production. It is the highest-return AI infrastructure most teams have not built.

How Much Portability Is Worth Buying

Full portability is expensive and usually wrong. The interesting question is how much to buy, and it varies by system.

For high-volume, cost-sensitive workloads — classification, extraction, summarisation at scale — portability pays for itself quickly. These workloads are typically simple enough that model differences are manageable, volume makes price differences material, and smaller or cheaper models often perform adequately. Build full portability here: gateway, model-agnostic prompts, an evaluation set, and ideally a live second provider handling a small traffic share so the fallback is proven rather than assumed.

For complex agentic systems with deep tool use and long reasoning chains, full portability is often not worth it. The coupling is intrinsic to how the system works, the engineering cost of abstracting it is large, and the honest strategy is concentrated dependency with mitigations: a gateway for operational control, an evaluation set so you can assess alternatives when forced, documented prompts and tool schemas, and contractual notice periods on deprecation. Accept the dependency, price it, and monitor it.

For embeddings specifically, make the decision consciously and early, because it is the most expensive to reverse. If your corpus is large and growing, the cost of re-embedding rises every month you defer. Two mitigations are worth their cost: store the source text alongside the vectors so re-embedding is always possible without reconstructing the corpus, and prefer embedding models with a credible open-weights alternative so you are not exposed to a single vendor's deprecation schedule for the most expensive layer you own.

Finally, price the dependency explicitly. If your incumbent raised prices 40% tomorrow, what would it cost to move, and how long would it take? If you cannot answer that in numbers, you have an unmeasured position on a vendor's future pricing decisions. Financial institutions have started pushing model portability into procurement guidelines — requiring that any AI used be swappable, exportable and transparent — and that is a reasonable standard for anyone, regulated or not.

The Sovereignty and Regulatory Dimension

There is a second driver of portability work that has nothing to do with pricing, and for European organisations it is often the stronger one.

Where a model runs, where data is processed and under whose jurisdiction have moved from procurement footnotes to board-level constraints. Sovereignty requirements can arrive mid-contract from a regulator, a customer's procurement team or a sector rule. An organisation whose product is welded to one provider's hosted API has no response to a customer who requires processing within a specific jurisdiction, and the sales cycle simply ends.

Portability is the answer to that question too, and it points at a slightly different architecture. Being able to run an open-weights model in your own environment — even if it is not your primary path and even if quality is somewhat lower — converts an unanswerable requirement into a deployment option with a known cost. Several organisations now maintain exactly this: a hosted frontier model as the default, and a validated self-hosted path they can activate for the customers or jurisdictions that demand it.

The engineering prerequisite is the same evaluation set. You cannot offer a self-hosted deployment path unless you can state what it costs in quality, and you cannot state that without measurement. Every road in this topic leads back to the same missing artefact.

What to Do This Quarter

A sequence that is achievable inside a quarter and leaves you materially less exposed.

Week one to two: map the coupling. For each AI feature in production, record which provider it uses, how the SDK is called, whether prompts contain model-specific tuning, what tool schemas exist, which embedding model the retrieval layer uses, and how large the embedded corpus is. Most teams have never assembled this and are surprised by at least one entry — usually an embedding decision made eighteen months ago by someone who has left.

Week three to six: build the evaluation set. A few hundred real inputs per significant feature, drawn from production traffic, with expected outputs or a rubric. Establish the incumbent baseline. Wire it into CI so every prompt change is measured. This is the load-bearing step and the one most likely to be deferred, because it produces no visible feature — do it anyway.

Week seven to ten: introduce a gateway for the operational benefits, and use it to run a real comparison. Send a small share of non-critical traffic to an alternative provider and score it against your evaluation set. The output of this exercise is a number: how much quality you would lose and how much cost you would save by switching. That number is your actual portability position, and almost no organisation currently has it.

Week eleven to twelve: decide deliberately per system. Full portability for high-volume cost-sensitive workloads. Priced, monitored concentration for deeply agentic ones. Explicit embedding strategy with source text retained. And write down what would trigger a migration — a price rise above a threshold, a deprecation notice, a sovereignty requirement — so the decision is made in advance rather than under pressure.

At StepTo we have run senior engineering teams out of Serbia since 2014, and this specific work — evaluation harnesses, gateway layers, retrieval portability, and the migration itself when it comes — has become a large share of what our AI strategy and AI engineering teams take on. It is well-defined project work with a measurable endpoint, which makes it a poor fit for permanent hiring and a good fit for a senior team that builds it, proves it against your traffic, and hands it over. Engineers are assigned by name and work European hours, which matters when the comparison run produces an ambiguous result and someone needs to decide what to do about it today.