Skip to main content
Mobius
Advanced

What is RAG

Full name: Retrieval-Augmented Generation

Also known as: retrieval-augmented generation, retrieval augmented generation

Definition

RAG (retrieval-augmented generation) is a method where AI answers from your documents: it first finds the relevant passages, then writes the answer from them.

Why it matters

RAG is a common way to reduce made up answers, by giving the model your own text to answer from, and it is how a customer facing assistant can quote your real cancellation rule instead of a plausible one.

Common mistakes

Believing RAG prevents made up answers, when it reduces them and the model can still misread or add to a passage.

Related terms

Worked exampleIllustration

"What happens if I cancel on the day?" at a fictional pilates studio

Question

What happens if I cancel on the day?

Retrieved from the sheet

  • You can cancel a class up to 24 hours before it starts.
  • A later cancellation counts as a used class.
  • To move a class, reply with the day you prefer.

Not retrieved

  • Classes are for adults only.
  • Bring a towel and water.

RAG off

  • You can cancel up to 12 hours before at no charge.

What happened

  1. RAG off Answering from its general training, the assistant writes "You can cancel up to 12 hours before at no charge", a rule the studio does not have.

  2. Search RAG on: the question is matched against the policy sheet.

  3. Retrieve The 3 closest passages go into the prompt, with "answer only from these, and say which one you used".3 passages

  4. Answer "Cancellations are up to 24 hours before class. A same day cancellation counts as a used class. (Source: policy sheet, cancellations)"

RAG changed where the answer came from. If the sheet is out of date, so is the answer, so keep the sheet current.

Illustration

Where the answer comes from

  1. QuestionWhat happens if I cancel on the day?
  2. SearchThe question is matched against the 40 passages of the policy sheet
  3. The 3 closest passagesCancellationsLate cancellationMoving a class
  4. Into the promptAnswer only from these passages, and say which one you used.
  5. AnswerUp to 24 hours before class. A same day cancellation counts as a used class.Source: policy sheet, cancellations

RAG on: the answer comes from the sheet and names its source. If the sheet is wrong or out of date, so is the answer.

RAG searches the studio’s own sheet first, puts the closest passages into the prompt, and the model answers only from them. Switch RAG off and the model answers from its general training.

Definition

RAG, retrieval-augmented generation, connects a language model to your documents. When a question comes in, the system first searches the documents for the most relevant passages, usually with embeddings stored in a vector database, then puts those passages into the prompt and asks the model to answer from them, ideally naming the source. The model itself does not change: what changes is what it reads before it answers. The idea comes from a 2020 research paper that paired a language model with a searchable index of documents. Many chat tools do RAG for you when you upload files to a project.

Why it matters

It also moves the work: the answers are only as current and correct as the documents it searches, so keeping one current policy sheet matters more than any setting.

Directly related: Vector Database, Embedding, Hallucination.

Improvement tips

  • Keep one current version of each document the assistant searches, and remove the old ones.
  • Ask the assistant to answer only from the retrieved passages and to name the source.
  • Give a customer facing assistant only customer facing documents.
  • Tell it what to do when the answer is not in the documents: say it will check, and pass the chat to a person.

Common mistakes

  • Letting the assistant search old files, so it quotes last year's schedule as confidently as this year's.
  • Putting internal documents in the search of a customer facing assistant.
  • Blaming the model when the real problem is that the search did not find the right passage.

Related terms

AI and DataAdvanced

Vector Database

A vector database stores embeddings and finds the items closest in meaning to a question, even when the question and the document use different words.

AI and DataAdvanced

Embedding

An embedding is a list of numbers that stands for the meaning of a text or image, so texts that mean the same land close together, even with different words.

AI and DataBeginner

Hallucination

An AI hallucination is an answer an AI model makes up and states with confidence, such as an invented price, rule or source, because it predicts likely words.

AI and DataIntermediate

LLM

An LLM (large language model) is AI trained on masses of text to predict the next piece of a sentence. That is how chat tools write answers, right or wrong.

AI and DataAdvanced

Fine-tuning

Fine-tuning is extra training of a ready AI model on your own examples, so it learns a style, format or task. It changes how it answers, not what it knows.

AI and DataIntermediate

Context Window

A context window is how much text, in tokens, a model can see at once: instructions, files, the chat so far and its reply. What falls outside, it cannot use.

Quick check

An assistant with RAG quotes last season's class schedule. What is the most likely cause?

Choose an answer

Frequently asked questions

What is RAG in simple words?
A way to make an AI answer from your documents. It first finds the passages that match the question, then writes the answer from them, like an open book exam where the model looks up the page before it answers.
What does RAG stand for?
Retrieval-augmented generation. Retrieval is finding the passages, generation is writing the answer, and augmented means the answer is written with the found passages added to the prompt. The term comes from a 2020 research paper by Lewis and colleagues.
Am I already using RAG?
Possibly. Tools that let you upload files and then answer from them are often RAG done for you. Anthropic's help page on Projects, for example, describes Claude using a project knowledge search tool to retrieve relevant information from your uploaded documents.
What is the difference between RAG and fine-tuning?
RAG gives the model your documents to read with each question, and the model stays the same. Fine-tuning trains the model further on your examples to change how it writes. For facts that change, such as hours and rules, RAG fits, because you just update the document.
Why does our assistant still invent answers with RAG?
RAG reduces made up answers, it does not end them. The search may miss the right passage, or the model may add to what it found. Instruct it to answer only from the passages and to say it will check when the answer is not there, and have a person check what reaches customers.
Why does the assistant quote an old rule?
Because an old document is still among what it searches. RAG answers from whatever it finds, with the same confidence. Keep one current version of each document and delete the rest.
How should I prepare documents for RAG?
Short passages, one rule or topic each, in plain words, with a clear heading. Remove duplicates and outdated versions. One current policy sheet usually beats twenty old files.
Does RAG need a vector database?
Most RAG systems search with embeddings stored in a vector database, because it finds text by meaning. In a ready tool you never see it, because the tool keeps one for you.
Is it safe to connect confidential files to RAG?
Check the vendor's terms on storing and using your files before you connect them, and give each assistant only the documents it should quote. A customer facing assistant should never search internal files or customer records. Israel's Privacy Protection Law, 1981, as amended by Amendment 13, in force since 14 August 2025, governs collecting, keeping and using personal details, including details you paste into or upload to an AI tool. Educational material, not legal or tax advice. Check your own case with a lawyer or an accountant.
Does RAG make the AI smarter?
No. It changes what the model reads before answering, not how capable it is. A good passage makes a good answer more likely, and a stale passage gives a stale answer with the same confidence.
Do I need a developer for RAG?
Not for a ready tool where you upload files. A setup that searches your own systems, such as a shared drive or a booking database, usually needs someone technical.
Can RAG show where an answer came from?
Yes, if it is set up to. Ask the assistant to name the document or section it used, as in "(Source: policy sheet, cancellations)". That makes checking fast.

Sources: Lewis, Perez, Piktus et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", NeurIPS 2020 (arXiv 2005.11401), read 2026-09-25, Anthropic Help Center, "Retrieval augmented generation (RAG) for projects" (support.claude.com/en/articles/11473015), read 2026-09-25, Privacy Protection Authority, Amendment 13 questions and answers (gov.il/he/pages/tikun13_qa), read 2026-08-31

Last reviewed: 2026-09-25

I have worked with

What Clients Say

From different fields, at different stages of business

Dan Manto

Eclipse Capital

Real Estate Investment and Finance, USA

Arty McLabin

GameReady

Game Development Education and Outsourcing, International

Anna, Beautician

Anna

Beautician

After I finished a cosmetology course at a leading company, I was confused. I didn't know how to open a business, what to sell from home, or how to bring in clients. Since I reached out to Mobius, everything has changed. We built an organized plan with clear steps and no unnecessary risks. Today I see results, and I'm growing and developing every month.

Mark, Massage therapist, Gan Yavne

Mark

Massage therapist, Gan Yavne

When I finished a massage course, I imagined a different world. I found out it's a hard field, and financial stability isn't easy to find. When I came to Mobius, something changed. We started an organized process and built a business plan. Without Mobius? I probably would have given up. Today I believe in myself and in my own path.

Wondering where AI actually fits your business?

Alex helps Israeli businesses use AI and data where it pays off, and skip it where it does not. The first call is free.

+972 055-248-6151. Better on WhatsApp: I am in meetings most of the day and answer as soon as I am free.