RAG Answer Verification: Are Your Answers Actually Supported by the Retrieved Documents?

Retrieval puts the right documents in front of the model. It does not make the model faithful to them.

Retrieval-augmented generation was supposed to solve hallucination: ground the model in retrieved documents and it can only say what the documents say. Every team that has operated a RAG system in production knows how that promise turned out. RAG dramatically reduces fabrication-from-nothing — and leaves intact an entire class of subtler failures that occur after retrieval succeeds.

The generation-side failures retrieval can't fix

• Over-synthesis. The model retrieves three true passages and welds them into one conclusion none of them makes. Retrieval was perfect; the synthesis is fabricated.
• Relation invention. Chunk A says traffic doubled; chunk B says a campaign ran. The answer says the campaign doubled traffic. Adjacency became causation in generation.
• Hedge stripping. Sources say "may," "preliminary," "pending review." Answers say "is," "will," "confirmed." Certainty inflates in the rewrite.
• Attribution drift. "The report discusses X" becomes "the report concludes X." One verb, entirely different claim.
• Stale-context leakage. The model answers partly from training memory when retrieved context is thin, blending your documents with the internet's.

Why your evaluation layer probably misses these

Standard RAG evaluation scores answer-level faithfulness against the retrieved context, usually with embeddings or an LLM judge producing one number. The failures above are precisely the ones aggregate scoring hides: the answer overlaps heavily with the retrieved text — it was built from it — so similarity is high even where a relation was invented or a hedge was stripped. The fabrication lives in the connective tissue between true statements, which is the one place a single score cannot look.

What claim-level RAG verification looks like

Take the final answer and the retrieved context. Decompose the answer into atomic claims. For each claim, check the facts against the context and — separately — check every asserted relation: was it stated in the context, or does it actually follow from the supported facts? Report the failing component, not just a flag. In production this changes triage completely: "facts supported, causal link unsupported" routes to prompt or synthesis fixes, while "fact unsupported" routes to retrieval and documentation fixes. Teams stop re-reading whole answers and start fixing named components.

Closing the loop: from flag to fix

The final step most pipelines lack: for each unsupported claim, generate the statement the context would have needed to contain. Sometimes that statement exists in your corpus and retrieval missed it — a retrieval fix. Sometimes it exists in no document you have — a documentation gap with a named owner. Sometimes it cannot exist because the context contradicts the claim — a generation fix. Knowing which is the difference between an evaluation dashboard and a repair workflow.