Every model has a knowledge cutoff. RAG is how you get around it without retraining anything, and why retrieval quality decides your answer quality.
Every model has a knowledge cutoff. At some point training stopped, and everything after that date simply does not exist for it. RAG fixes that without retraining anything.
The practical version of the problem: you want an AI to answer questions about your internal documentation, your support tickets, your product manual. None of that is on the internet. Fine-tuning takes weeks and costs money. There is an architecture for exactly this.
Retrieval augmented generation means relevant information is retrieved from an external source and injected into the model's context before it generates an answer. The model does not need to know the answer. It needs to be handed the right page to read first.
The open book exam analogy carries the whole concept. A student does not memorise everything. They look it up, but they need to know which book, which page, which paragraph. The AI is the student, your documents are the books, and the retrieval system finds the page.
The pipeline has three steps. Ingestion: split documents into chunks, convert each into a vector, store them in a vector database. Retrieval: the question also becomes a vector, and the system finds chunks that are semantically similar, meaning matching rather than keyword matching. Generation: the relevant chunks get added to the context just before the model answers.
Two honest limits. RAG does not make the AI smarter, it gives it better input, and the reasoning stays the same. And RAG fails when retrieval fails: pull the wrong chunks or find none, and the model either hallucinates or says it does not know. The quality of your retrieval decides the quality of your answers.
| Aspect | RAG | Fine-tuning |
|---|---|---|
| What it changes | Keeps knowledge external and retrieves it | Bakes knowledge into the model's weights |
| Cost | Cheap | Expensive |
| Speed | Fast to set up | Slow, takes weeks |
| Updating | Updates the moment you update your documents | Static after training |
| Best for | Adding facts the model does not have | Changing behaviour, not adding facts |
| Auditability | The answer can cite which chunk it came from | No trace back to a source |
| Failure mode | Wrong chunks retrieved means wrong or refused answer | Wrong training data means wrong behaviour |
RAG, retrieval augmented generation, is a technique where relevant information is retrieved from an external source and injected into the AI's context before it generates a response. The model does not need to know the answer, it needs to be given the right page to read first.
Three steps. Ingestion: documents are split into chunks, each chunk becomes a vector, and the vectors go into a vector database. Retrieval: the user question also becomes a vector and the system finds semantically similar chunks. Generation: those chunks are added to the model's context right before it answers.
For most use cases, yes, because they solve different problems. RAG adds facts and updates instantly when your documents change. Fine-tuning changes behaviour and is expensive, slow and static after training. Fine-tuning is for changing how a model behaves, not for adding knowledge.
No. It gives the AI better information to work with, which is a different thing. The model's reasoning is unchanged, only the input improves.
When retrieval fails. If the wrong chunks are pulled, or no relevant chunks are found, the model either hallucinates or says it does not know. RAG is a pipeline and every step can break, so the quality of your retrieval determines the quality of your answers.
Because the answer is generated from retrieved chunks, the system can point back to where each piece of information came from. That auditability is a key advantage over pure generation, where there is no trace back to a source.
No manual subtitle track is available for this video. This page therefore uses a checked chapter summary instead of publishing auto-generated captions as a transcript.
Every model has a knowledge cutoff. Training stopped at some point and everything after that simply does not exist for it. RAG is how you fix that without retraining a single thing. This is part five of the series.
The practical problem: answering questions about internal documentation, support tickets or a product manual, none of which exists on the public internet. Fine-tuning a model for that takes weeks and costs money.
The definition. Retrieval augmented generation retrieves relevant information from an external source and injects it into the AI's context before generation. The model does not need to know the answer, it needs the right page in front of it.
The open book exam. A student does not memorise everything, they look it up, but they need to know which book, which page, which paragraph. The AI is the student, your documents are the books, the retrieval system finds the page.
The three-step pipeline. Ingestion splits documents into chunks and converts each into a vector stored in a vector database. Retrieval turns the question into a vector and finds semantically similar chunks, matching meaning rather than keywords. Generation adds the best chunks to the context right before the model answers.
The LangChain documentation shows the full pipeline as a diagram: loading, splitting, embedding, retrieval, generation. Each step is modular, so vector stores, embedding models and retrievers can be swapped. LangChain handles the plumbing.
What the output looks like in practice. The user asks, the system retrieves two or three relevant chunks, the model reads them and writes a grounded answer that cites where the information came from. That auditability is a key advantage over pure generation.
A common confusion: RAG and fine-tuning are not interchangeable. Fine-tuning bakes knowledge into weights, is expensive, slow and static. RAG keeps knowledge external, is cheap and fast, and updates the moment your documents do. Fine-tuning changes behaviour, RAG adds facts.
What RAG does not do. It does not make the model smarter, it gives it better input. The reasoning is unchanged. And it fails when retrieval fails: wrong chunks or no chunks means hallucination or a refusal.
The summary: build a pipeline that retrieves, injects and generates. The model does not need to know, it needs to be told at the right moment. But everything gets injected into a context window, and that window has a hard limit, which is the next part.
I work 1:1 with freelancers, consultants, coaches and small teams on practical AI workflows and automation. First call is free, 15 minutes, no pitch.
Book a free intro call