TermAIJuly 26, 20261 min

RAG (Retrieval-Augmented Generation)

Giving a language model your documents at question time, so its answer is grounded in your content rather than its training data.

Retrieval-Augmented Generation (RAG) is a technique where a search step runs before a language model answers: the user's question is used to retrieve relevant passages from your own documents, and those passages are passed to the model as context. It is how an assistant answers questions about internal material it was never trained on, and it makes answers citable — but it only works as well as the quality and curation of the underlying document set.

A language model only knows what it was trained on. RAG closes that gap by retrieving relevant passages from your own material at question time and passing them to the model as context.

How it works

Documents are split into passages and indexed, usually as embeddings in a vector database.
A question is embedded and used to retrieve the closest passages.
Those passages are placed in the prompt alongside the question.
The model answers from the supplied context, and can cite which passage it used.

Where it goes wrong

Almost always in the corpus, not the model. Point RAG at every shared drive a company has ever had and it will faithfully retrieve the superseded policy next to the current one. Curation and recency filtering matter more than the choice of vector store.

RAG (Retrieval-Augmented Generation)

How it works

Where it goes wrong

Glossary

ERP (Enterprise Resource Planning)

ZATCA / Fatoora

DocType