What is retrieval-augmented generation?

How a RAG pipeline works

A RAG pipeline works in a short loop: find the right passages, add them to the prompt, then generate an answer from them. The documents are prepared in advance so they can be searched quickly, often by turning each passage into an embedding, a list of numbers that captures its meaning.

Take an illustrative sales case. A rep asks what the company promised a customer about onboarding. The system searches the shared files, finds the signed order form and the kick-off notes, and passes the relevant lines to the model, which answers and points to both documents.

• Prepare: split documents into passages and index them for search.
• Retrieve: find the passages most relevant to the question.
• Augment: add those passages to the prompt with the question.
• Generate: the model writes an answer from the passages and cites them.

What RAG means in AI research

In AI research, RAG means combining two kinds of memory: what a model learned during training, and a separate, searchable index of documents it can consult at answer time. The research paper that named the approach in 2020 framed it this way and pointed to two benefits: answers can show where they came from, and knowledge can be updated without retraining the model.

Since then the term has broadened. Today it covers almost any system that fetches material from outside the model and hands it over before the model writes.

RAG vs fine-tuning

RAG and fine-tuning solve different problems: RAG changes what a model can look up, while fine-tuning changes how the model itself behaves. Fine-tuning retrains a model on extra examples so it adopts a style, format or specialist vocabulary.

For facts that change often, such as prices, product details or account history, RAG is usually the better fit, because updating a document is quicker and easier to audit than retraining. Fine-tuning suits stable behaviour. Many systems use both.

What is graph RAG

Graph RAG retrieves from a knowledge graph, a network of entities and the relationships between them, instead of, or as well as, plain passages of text. It lets a system follow links, from a company to its subsidiaries to the people who run them, and bring back connected facts that no single document holds.

It is one of the places where structured knowledge and language models meet. The graph supplies explicit, checkable relationships; the model turns them into a readable answer.

Limits of retrieval-augmented generation

RAG is only as good as what it retrieves. If the search returns the wrong passage, an outdated version or two documents that contradict each other, the model will write a confident answer from bad material.

Even with the right passages, a model can misread them, blend them with its own assumptions or cite a source for a point it does not make. Access matters too: a system must not fetch documents the person asking is not allowed to see. RAG makes hallucination less likely and easier to catch; it does not end it.