The Team You Need to Run Retrieval in Production
A RAG system needs five distinct capabilities to stay good, and only two of them look like the work in the prototype. Teams that staff the prototype’s skills and stop end up with a system that launches well and degrades quietly. The staffing question is therefore an architecture question: if you cannot cover a capability, choose an architecture that does not need it.
Here is what the capabilities actually are, who usually already has them, and which ones you can substitute your way out of.
The five capabilities
1. Pipeline engineering. Ingestion, chunking, embedding, the retrieve-and-assemble path, the deployment. This is ordinary backend work — the specific concepts are learnable in weeks by someone competent with data pipelines and HTTP services. It is also the only capability most teams budget for, and it is front-loaded: heavy until first release, then modest.
2. Retrieval quality. Deciding what “good” means for your queries, building a set of labelled examples, and using it to judge whether a change helped. This is the capability that distinguishes a system that improves from one that drifts, and it is not a machine-learning specialism — it is closer to careful product analysis with a spreadsheet. It is also permanent. There is no version of this work that finishes.
3. Corpus ownership. Someone who knows which document is authoritative, notices when a policy changes, and decides what happens to the three superseded copies. This is not an engineering role and putting an engineer in it does not work, because the knowledge is domain knowledge. It is the capability most often left unassigned, and its absence is the single most reliable predictor of a dead retrieval project — see when not to use RAG.
4. Operations. Whatever is carrying your index has to be patched, backed up, monitored, and recovered. If you bought the storage layer, this is small. If you self-hosted it, it is an on-call commitment that exists whether or not the pager fires.
5. Answer support. When a user escalates a wrong answer, someone has to reconstruct what was retrieved, decide whether the retriever, the corpus, or the prompt was at fault, and close the loop. In the first months after launch this arrives steadily, and it lands on whoever is nearest rather than whoever is assigned.
Who these people already are
You are almost certainly not hiring five people. You are assigning five hats, and the useful exercise is mapping them to roles that exist.
| Capability | Usually sits with | Realistic ongoing load |
|---|---|---|
| Pipeline engineering | A backend or platform engineer | Heavy to launch, light after |
| Retrieval quality | A product-minded engineer or a technical analyst | Small but permanent, and never zero |
| Corpus ownership | The team that authors the documents | Small per change, unpredictable in timing |
| Operations | Platform or SRE, or the vendor | Near zero if bought, standing if self-hosted |
| Answer support | Whoever is on the product’s support rota | Front-loaded after launch, decays if quality work happens |
Two observations from that table. First, the two capabilities that never end — quality and corpus ownership — are the two with no obvious home, because neither maps to an existing engineering job title. Second, operations is the only one you can buy outright, which is most of the argument in build or buy your retrieval stack.
Pricing it as a standing commitment
The mistake is pricing the launch. Price the steady state, because that is the number that recurs.
Ongoing team cost per month =
(quality fraction + ops fraction + support fraction)
× fully loaded cost per engineer
+ corpus-owner time, valued at that person's costWorked illustration, all inputs hypothetical. Suppose steady state needs 15% of an engineer for quality work, 10% for operations, and 10% for answer support — 35% of a person. At a fully loaded engineering cost of, say, $16,000 a month, substitute your own, that is $5,600 a month, or roughly $67,000 a year, before a single token is billed. Add a domain expert at 5% for corpus ownership and it is higher.
Now compare that against your inference and storage bill from what a RAG system actually costs to run. For a system serving internal volumes, the people number is usually the larger one by a wide margin. That is the finding that should change your architecture, not your hiring plan: at low query volume, an architecture that needs fewer permanent hats beats an architecture that is cheaper per request.
What to do when you cannot fund all five
This is the common case, and it has better answers than proceeding anyway.
If you cannot fund quality work, do not build a generated-answer product. Ship retrieval with a results page instead. Search degrades visibly — users see an unhelpful list and rephrase — where a generated answer degrades invisibly. Choosing the architecture with legible failure is how you survive not having someone to measure it. The argument in full is in do you need generation, or just better search?.
If you cannot fund operations, buy the storage and search layer. This one is unambiguous. Self-hosting an index without an on-call commitment behind it is deferring an incident, not saving money.
If you cannot name a corpus owner, shrink the corpus until you can. Forty maintained pages beat four thousand unmaintained ones on accuracy, on user trust, and on the cost of everything downstream.
If you cannot fund pipeline engineering, you are not ready to start, and there is no substitution that fixes it. This is the one capability with no workaround, which is ironic given it is the only one most plans include.
The recommendation
Staff the two permanent capabilities before the launch, not after. Name the person who owns retrieval quality and the person who owns the corpus, by name, in the design document. If either line is blank when you ship, the system’s useful life is measured in months regardless of how good the launch is.
Buy operations unless you have a specific reason to own it. It is the only capability with a clean market substitute, and paying for it converts a standing on-call obligation into a line item — which is a good trade for almost everyone.
Size the project by the hats it needs, not by the code it needs. Two architectures with identical per-request costs can differ by a fifth of a person a month in permanent attention, and over eighteen months that difference dwarfs the infrastructure difference. Prefer the architecture with fewer permanent hats until volume makes the per-request term dominate.
The threshold that flips this: when your monthly inference and storage bill exceeds the monthly cost of the fractional people maintaining the system, you have crossed into the regime where per-request efficiency is worth staffing for. Below that line — where most internal tools live permanently — optimise for the smallest number of people who have to care.