Tagged “latency”
-
Agentic Retrieval or a Fixed Pipeline?
Letting the model decide how many searches to run trades a predictable bill for a better answer. How to price the variance and where to cap it.
-
Latency Budgets for Retrieval
Where the milliseconds go in a RAG request, which stages you can cut when your target is under two seconds, and what each cut costs you.
-
Caching and What It Does to a RAG Bill
Four things in a retrieval pipeline can be cached, they save wildly different amounts, and one of them is a correctness risk. Which to do, in order.