RAG Definition: Retrieval-Augmented Generation

Start here

Retrieval-Augmented Generation (RAG) in plain language

Imagine asking an assistant when a customer can return a product. A model answering from memory may invent a policy. A RAG application first finds the relevant policy passage and supplies it with the question. The model then has evidence it can use to construct an answer.

Retrieval can use keywords, vectors, structured queries or a combination. A vector database is one implementation choice, not part of the definition. Sources also need not be verified simply because a system retrieved them. Their quality, version and relevance still matter.

How it works

The visual guide

From your documents to a grounded answer

Prepare the knowledge

Source documents
Policies, product guides or other authorised information.
Make searchable
Organise passages and keep source references and permissions.
Source collection
Search using keywords, vectors or a combination.

The source collection supplies evidence to the retrieval step below.

Answer a question

Ask + retrieve
Use the question to find relevant passages the user may access.
Generate
Give the model the question and retrieved evidence together.
Check + respond
Check that sources support the answer; flag missing evidence.

An illustrative RAG workflow: sources are retrieved at answer time, not learned through retraining. Source references make checking possible; they do not guarantee a correct answer.

Prepare sources
Select documents, preserve their provenance and permissions, and make them searchable. Some systems index passages; others query existing sources.
Retrieve evidence
Use the question and relevant filters to select useful information. Check whether the passages actually address the question.
Generate and check
Supply the evidence to the model, request supported answers and expose source references. Evaluate the answer separately from retrieval.

Illustrative example · Fictional data

Answering a return-policy question

Input: Illustrative policy: unopened accessories may be returned within 14 days. Opened accessories are excluded unless faulty. Question: can I return an opened, working charger after 7 days?
Process: Retrieve both the time limit and the condition exception. Supply them together rather than only the sentence containing “14 days”.
Expected output: Expected answer: the ordinary return policy does not cover this opened, working accessory. Link to the condition exception.
How to check it: If the assistant says “yes, within 14 days”, the answer failed even though retrieval found a relevant policy. Test questions about exceptions and missing information as well as straightforward cases.

Where it helps

Support answers grounded in product documentation.
Internal knowledge assistance over authorised policies and procedures.
Research assistance where readers need to inspect the source passage.

RAG, search or fine-tuning?

Search: Returns records or passages for the reader to interpret. It may already solve the task.
RAG: Adds generation using retrieved material. Useful when synthesising evidence helps the reader.
Fine-tuning: Changes model parameters through additional training. Useful for repeatable task behaviour; it is not a substitute for maintaining a source collection.

Common pitfalls

Relevant-looking passages can omit an important exception or come from an obsolete document.
RAG can reduce unsupported answers, but it does not eliminate them. A citation must support the associated claim.
Access control must apply before sensitive content reaches the model. Prompt instructions alone do not enforce document permissions.

Put the idea to work

Try it yourself

Write three short fictional policies and five questions, including one exception and one question the policies cannot answer. Manually select the evidence before drafting each answer. Record retrieval errors separately from answer errors.

Common questions

Does RAG need embeddings?

No. RAG needs retrieval, which can use keyword search, vector search, database queries or a mixture. Embeddings are useful for many semantic-search designs. Hybrid search, which combines keyword and vector results, often works well for business documents that contain product codes, names and specialised terms.

Does RAG eliminate hallucinations?

No. Retrieval may miss evidence, sources may be wrong, and the model may misinterpret them. Test both evidence selection and answer support. Ask the model to cite the passages it used, check that the citations support the claims and measure retrieval quality and answer faithfulness on a test set.

Does private hosting make RAG secure?

Hosting is only one consideration. Document permissions, identity, logs, model-provider data handling and the application’s access controls also matter. Make sure retrieval respects each user’s document permissions, so the system never surfaces content a person could not open directly.

Your next step