Data Residency as an Architecture Constraint
A residency requirement is not a preference to be traded against cost — it is a boundary that removes options from the table. The useful work is establishing precisely what the boundary forbids, because “our data cannot leave the country” is usually shorthand for several different rules with very different consequences, and teams routinely over- or under-constrain themselves by not asking.
This is not legal advice and nothing here substitutes for your own counsel’s reading of your own obligations. It is a structural map of how such constraints interact with a retrieval architecture, so you can ask the right questions of the people who can answer them.
Establish what is actually constrained
Four separate questions, and the answers frequently differ from each other.
1. Where may the documents be stored? The corpus at rest — originals, extracted text, and chunks. Often the strictest of the four.
2. Where may the documents be processed? Sending text to an embedding model is processing, even though nothing is retained. Some rules distinguish storage from transient processing; some do not. This distinction determines whether hosted embedding APIs are available to you.
3. Where may the vectors live? Vectors are derived from the text and are not human-readable, and a common instinct is to treat them as anonymised. Treat that instinct with suspicion: they are derived from the source and carry information about it. Whether your obligations follow the derivation is a question for counsel, and getting a written answer early is worth more than any engineering decision on this list.
4. What may go to the generation model? Retrieved passages travel in the prompt, so every answer sends document content to whatever serves the model. This is usually the binding constraint, and it is the one people discover last.
Get those four answered before designing anything. The combination determines your architecture more completely than corpus size or volume.
What each answer forbids
| If the rule says | Then you cannot |
|---|---|
| Storage must stay in-region | Use a store or backup target outside it — check where backups and replicas actually land |
| Processing must stay in-region | Use a hosted embedding or generation endpoint outside it |
| No third-party processing at all | Use any hosted model; you are self-hosting both embedding and generation |
| Vectors are in scope as derived data | Host the index yourself, or in-region under acceptable terms |
| Content cannot reach a general-purpose model | Send only non-sensitive material, or self-host generation, or do not generate |
The bottom row is the one that most often reshapes a project. If retrieved passages cannot go to a hosted model, your options are a self-hosted model — with its own hardware, capacity, and quality trade-offs — or an architecture that does not generate at all. A search product with good snippets keeps the document inside your perimeter by construction, which makes do you need generation, or just better search? a compliance argument as much as a cost one.
What the constraint costs
Three costs, in ascending order of how much they hurt.
Fewer vendors, worse terms. In-region managed options may be fewer, more expensive, or on older versions of a product. Recognisable and quantifiable.
Self-hosting where you would have bought. The residency case is one of the two legitimate reasons to run your own index, per build or buy your retrieval stack, and it converts a subscription into a standing operational commitment. Likewise for the embedding model, which is the third of the three real reasons in self-hosting your embedding model.
Self-hosting generation. By far the largest, and it is not primarily about hardware cost. Serving a model well is a specialism, capacity for peak traffic must be owned rather than rented per request, and the strongest models are generally not available to self-host. So the cost is partly a bill and partly a quality ceiling — and the quality ceiling is what should worry you, because it changes what the product can be rather than what it costs.
Worked illustration, all inputs hypothetical. Suppose hosted generation for your volume would run $1,500 a month. Self-hosting equivalent capacity might be several GPU instances at, say, $1.50 an hour each running continuously — roughly $3,300 a month for three — plus the engineering to serve, scale, and monitor them, plus accepting whatever open-weight model you can run. Substitute your own figures. The bill roughly doubles; the more consequential change is that your answer quality is now set by what fits on your hardware.
Designing within the constraint
Four moves, roughly in order of how much they buy.
Classify the corpus rather than the project. Very often only part of the corpus is constrained. Split it: the unconstrained majority uses hosted services; the sensitive minority uses an in-perimeter path, or is excluded and handled by a person. This turns an all-or-nothing constraint into a routing problem, and it is nearly always the cheapest available answer.
Keep the constrained path simpler. If a subset must run on self-hosted infrastructure, give that path a narrower job — retrieval with source display and no generation, for instance. You are not obliged to offer identical capability on both sides, and pretending otherwise is what makes the constrained path expensive.
Check where your data actually goes, including the parts you did not choose. Backups, replicas, log aggregation, error tracking, and analytics all move content or fragments of it. A residency design that covers the primary store and not the log pipeline is a design that will fail its first audit.
Get the vector question answered in writing. Whether derived vectors inherit the source data’s obligations is the pivot for your entire storage decision. Do not let it stay an assumption; an engineer’s guess here can commit the project to the wrong infrastructure for a year.
The one thing not to do
Do not build a general architecture and plan to add residency later. Unlike most requirements, this one is not a layer — it determines which services you may call, which means it determines your dependencies, which means it determines almost everything. Retrofitting it is a rebuild rather than a refactor, and it is one of the requirements worth pre-empting even at low probability, in the same category as the permission thresholds in scaling thresholds for a retrieval system.
The recommendation
Answer the four questions — storage, processing, vectors, generation — with counsel, in writing, before choosing any component. This is a half-day of work that determines a year of decisions, and it is the only ordering that avoids expensive rework.
Classify the corpus and route by sensitivity. Constraining the whole system to satisfy a minority of documents is the most common and most expensive error here. Most corpora are mostly ordinary.
If generation cannot happen outside your perimeter, seriously consider not generating. A high-quality search and extraction product inside the perimeter frequently serves the same need without a self-hosted model, at a fraction of the cost and with better failure characteristics per choosing an architecture by how it fails.
The threshold that flips it: when the constrained subset is small enough to be handled by people, handle it by people and keep the automated system unconstrained. When it is the majority of your value, accept the self-hosted path and budget for the quality ceiling as well as the bill — and price it before committing, using the framework in pricing a RAG project before you have requirements.