Shipping RAG systems that actually hold up in production

Most RAG demos work great on ten curated documents. Production RAG systems break down when they meet the real mess of enterprise knowledge: inconsistent formatting, outdated documents, and duplicate information across systems.

The single highest-leverage investment we've found is not the retrieval algorithm — it's the ingestion pipeline. Chunking strategy, metadata tagging, and a deduplication pass before anything hits the vector store will save you more debugging time than any embedding model swap.

Second, evaluation cannot be an afterthought. We build a golden test set of real questions with verified answers before writing a single line of retrieval code, and we re-run it on every pipeline change. Without this, you're flying blind on whether a 'fix' actually helped.

Finally, plan for the escalation path from day one. Every production RAG system we've shipped includes a confidence threshold below which the system defers to a human — this single decision has prevented more customer trust issues than any prompt engineering technique.