Embedding Model Lock-In and What Switching Costs

Vectors are only meaningful to the model that produced them. Change the embedding model and every vector in your index becomes garbage, so the change is not a configuration edit — it is a full re-embed of the corpus plus a re-validation of quality. This is the most consequential lock-in in a retrieval stack, and it is created by a decision usually made in the first week by whoever set up the prototype.

You cannot avoid it. You can make it cheap, and you can make it a decision you take deliberately.

What a switch actually involves

Four pieces of work, in order, and only the first is the one people think of.

Re-embed the corpus. Every chunk through the new model. Mechanically simple, cost is proportional to total tokens, and it is usually the smallest term in the whole exercise.

Rebuild the index. New vectors, possibly a different dimensionality, so a new index rather than an update. If dimensionality changed, storage and memory sizing change with it.

Re-validate quality. The new model ranks differently. Better on average is not better on your queries, and you cannot know which without measuring. If you have a labelled set, this is a day. If you do not, this is where the project stalls — and it is why the switch tends to be postponed indefinitely rather than evaluated.

Re-tune everything downstream. Similarity thresholds, top-k, any score cutoffs, and the hybrid weighting if you fuse with keyword search. Scores from different models are not comparable, so every tuned constant in your pipeline is now unjustified.

Running both, which is how you avoid a flag day

The naive migration is a cutover: re-embed, swap, hope. The better pattern costs more in infrastructure and much less in risk.

Build the new index alongside the old one and route a share of traffic — or better, replay logged queries — through both. Compare on the same queries. Promote the new index only when it is at least as good on the set you care about. Delete the old one after a grace period rather than immediately.

That means paying for two indexes for a few weeks, which is a small, quantifiable cost. Compare it against a cutover that silently degrades retrieval for a fortnight before anyone connects the complaints to the change. The parallel-run cost is easy to justify precisely because the risk it removes is invisible failure.

Pricing the switch

Switch cost = corpus tokens × embedding price per token
            + parallel index infrastructure × weeks of overlap
            + engineering to run, compare, and promote
            + re-tuning downstream constants

Worked illustration, all inputs hypothetical. Take a corpus of 6 million tokens. At a hypothetical embedding price of $0.02 per million tokens, re-embedding is about $0.12 — genuinely nothing, and this is the term everyone worries about. Now the rest: two weeks of a parallel index at, say, $200 a month prorated is roughly $100, and a week of engineering time at a hypothetical $4,000 is $4,000. Substitute your own figures.

The engineering is 97% of the cost. Which yields the useful conclusion: embedding compute is not the lock-in; validation is. A team with an eval set can switch models in days. A team without one cannot switch at all, and will be running whatever the prototype picked years later — not because it is good, but because nobody can demonstrate that the replacement is not worse.

The corollary is that the investment that buys you optionality here is not architectural, it is the labelled query set from the team you need to run retrieval in production.

When a switch is actually worth it

Given the real cost is validation, the bar is “does this justify a week,” not “is the new model better.”

Trigger	Worth switching?
A new model tops a public leaderboard	No, on its own — general rankings do not predict your corpus
Your provider deprecates the model you use	Forced. Budget it the moment the notice arrives
Measured recall failures traced to embeddings	Yes — you have evidence, and a set to validate against
You need a language your model handles poorly	Yes, and this one is usually clear-cut
Domain vocabulary the model appears to collapse	Test first — often a hybrid keyword layer is cheaper
Dimensionality is driving memory cost	Maybe, and quantifiable up front
You want to move off a hosted API for residency reasons	Forced by the constraint, not by quality; see data residency as an architecture constraint

The first row is the common one and the answer is genuinely no. Leaderboards are averages over benchmark corpora that are not yours. The only evidence that counts is your queries against your documents.

Reducing the lock-in before you need to

Four choices, none expensive, all much easier before launch than after.

Keep source documents and chunking logic outside the store. If you can regenerate every chunk from originals with one command, a re-embed is a batch job. If chunk boundaries only exist as rows in a vendor’s index, you are rebuilding your pipeline as part of the migration.

Store the model identity and version with every vector. Obvious, routinely omitted, and the thing that makes a mixed-state index detectable rather than mysterious.

Do not let embedding scores leak into your product. If a similarity number is displayed to users, or written into a contract or an SLA, you have coupled a product surface to a specific model’s score distribution. Use ranks and thresholds internally; expose neither.

Build the eval set early. It is the actual escape hatch. Everything else is plumbing.

The self-hosting question is separate

Whether to run the embedding model yourself is a cost and control decision, not a lock-in decision — worked through in self-hosting your embedding model. Worth noting though: self-hosting an open-weight model removes deprecation risk entirely, because nobody can retire a model that is sitting on your disk. If forced migrations are the specific thing you are worried about, that is a real argument, and it is separate from the price comparison.

The recommendation

Choose an embedding model on your own retrieval quality, measured on thirty or more real queries, and then stop thinking about it. Model churn is constant and mostly irrelevant to a working system. The right cadence for revisiting is annually, or when something forces you.

Spend the effort on the eval set instead of on model selection. It converts a locked-in choice into a reversible one, and it is the same asset that lets you change chunking, add reranking, or upgrade the generation model — every other component decision in the stack.

Always parallel-run. The infrastructure overlap is cheap and the failure it prevents is the invisible kind, as described in choosing an architecture by how it fails.

The threshold that flips it: switch when you can point to specific queries that fail for embedding reasons and demonstrate a candidate model fixing them without breaking others. Absent that evidence, a switch is churn with a re-validation bill attached — and the fact that you cannot produce the evidence is the finding, not the model choice.