A staff member asks an assistant how many days of leave a policy allows. A general answer about employment practices is not enough: the response needs to reflect the organisation's actual policy. Retrieval-augmented generation, usually shortened to RAG, addresses this kind of problem by supplying relevant external information to a model.
The foundational RAG paper combines retrieval from a document collection with text generation. In a typical application, a system searches a collection, selects relevant passages, and provides them as context for an answer. This separates access to a body of information from relying entirely on knowledge encoded in model parameters.
Imagine a small organisation with three versions of its leave policy. A search system finds a passage mentioning twenty days, and the assistant presents that number. The answer may still be wrong if the passage comes from an outdated policy or applies to a different employment category. Retrieving a document is useful only when it is the right document for the question.
A practical evaluation can therefore separate three checks. Did the collection contain the authoritative policy? Did the search retrieve the relevant passage? Did the final answer accurately reflect that passage and its conditions? This proposed workflow makes failures easier to diagnose than marking the whole system simply right or wrong.
References should also be useful to the person reading them. A link to a long handbook leaves the user to find the relevant rule again. A document title, section, and short supporting passage can make verification easier. If the source does not answer the question, an honest statement of that gap is preferable to a plausible guess.
For a first project, choose a small collection with clear ownership and version dates. Prepare questions with known answers, including questions the documents cannot answer. Review both the retrieved material and the generated response. RAG is best understood as a way to connect a system to evidence; whether that connection works must be demonstrated with the material and questions the application will actually encounter.
Reference: Lewis and colleagues, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks: https://arxiv.org/abs/2005.11401