Technology
How retrieval-augmented generation works, how it differs from uploading and fine-tuning, why it reduces errors and which privacy questions to ask.
With RAG, retrieval-augmented generation. Your documents are split into passages and put in a search index beforehand. When someone asks a question, the system finds the best matching passages and sends them to a language model together with the question. The model writes an answer based on those passages and names the source.
Searching for RAG AI or an AI search engine for your own documents leads to the same technology. Why pasting a client file into a chat window is not a good route is explained on the main page about AI chatbots and the GDPR.
A language model knows what was in its training and what you put in the conversation. Your own client files, contracts and notes are not included. RAG addresses this by supplying the model with relevant text for each question, rather than teaching the model something new.
The name comes from a research paper published in May 2020 by Patrick Lewis and eleven co-authors. They combined a language model with a searchable index of Wikipedia. They called the knowledge within the model parametric memory, and the index alongside it non-parametric memory. They identified two open problems for language models: establishing where an answer comes from, and updating a model's knowledge. According to the paper, their models wrote more specific and factual text than a model without an index.
An ordinary search engine gives you a list of documents and leaves the reading to you. An AI search engine based on RAG does two additional things: it searches by meaning instead of exact words, and lets a language model summarise the passages found into an answer. The search engine does the finding; the language model does the wording.
The model therefore does not decide what your documents contain. It receives a handful of passages and an instruction to base an answer on them.
There are two stages: indexing beforehand, then searching and answering for each question.
Searching by meaning uses embeddings. An embedding model converts a piece of text into a series of numbers representing its meaning. Passages on the same topic get numbers close to one another, even if they use different words. The question “when can the tenant give notice” can therefore find a passage about “termination of the agreement by the occupier of the premises”.
This is called semantic search. It is strong with descriptions and weak with exact identifiers, such as a client file number or article code. Many systems therefore combine it with searching for literal words. Anthropic describes that combination and an additional step afterwards: an initial broad selection is ranked again, and only the best passages go to the model.
The model does not get access to the index. It receives text: the question, an instruction and the passages found. Because each passage carries its location, the answer can reference a document and page.
That reference lets you check an answer: open the source and read what it says.
The search system may be part of the same application as the model, or a separate service to which an AI assistant connects. In the latter case, this often uses the Model Context Protocol. How such a connection works and how an assistant signs in is explained in What is an MCP server?.
These are three ways to make a model work with your own information. They differ in what the model sees, the maintenance required and what happens to personal data.
You put a document in the conversation and ask a question about it. This is quick and requires no setup. The whole file goes to the provider, including pages you asked nothing about. It works for a few documents, not an archive. The risks are explained in Uploading documents to an AI chatbot.
If you have very little text, uploading may sometimes be the simplest technical solution. Anthropic writes that a knowledge base small enough can be supplied to the model in full, and that RAG only becomes necessary as the volume grows. That is a technical argument. It says nothing about whether the text may go to the provider.
Fine-tuning means training an existing model further on your own examples. OpenAI makes a clear distinction in its guide to accuracy. RAG gives the model access to knowledge it lacks. Fine-tuning changes how the model behaves: format, tone and reasoning style. If the model lacks information, fine-tuning does not help. According to OpenAI, the two do not exclude each other.
For anyone wanting a model to answer from client files, fine-tuning is therefore usually the wrong tool. There is also a privacy objection. Data used to train a model is not easy to remove. In their December 2024 opinion on AI models, European supervisory authorities wrote that whether a model trained with personal data can be considered anonymous must be assessed case by case. This is only so if it is very unlikely that people can be identified or that their data can be extracted from the model through questions.
| Approach | What the model sees | Maintenance | Privacy |
|---|---|---|---|
| Uploading a file | The whole file, for each conversation | No setup; upload again each time | Everything in the file goes to the provider; no permissions per person; no prior check |
| Fine-tuning | Nothing per question: the examples are incorporated into the model | Retrain with every change; knowledge becomes outdated | Data is in the model and difficult to remove or correct |
| RAG | Only the passages found for the question | Update the index when documents change; the model stays the same | Depends on setup: what the index contains, who may search and which passages go to which provider |
The final cell is the honest one: RAG can use data sparingly, but does not do so automatically.
A language model without a source fills in what sounds probable. If it receives the passage containing the answer, it does not have to guess. This explains why RAG reduces fabricated answers. It does not eliminate them, for three reasons.
If the system finds the wrong passages, the model answers from the wrong text. OpenAI identifies another variation in the same guide: too many barely relevant passages drown out the one containing the answer.
Even with the right passage, the model can turn it into something else. OpenAI puts it this way: the model can receive the right context and do the wrong thing with it. A notice period of three months then becomes three weeks, or an exception from the next paragraph disappears.
RAG retrieves what is there. If the index contains an outdated version of a contract, the answer is outdated. The answer's quality is never better than the folder's quality.
Three questions, following the data flow: what is in the index, who may find what, and what goes to the model.
A search index is a copy of your documents in pieces. If the documents contain personal data, the passages contain it too. Embeddings are also derived from that text. Therefore ask a supplier:
An index making everything searchable for everyone also exposes what was shared incorrectly. A document buried deep in a folder that nobody ever found now appears with the first matching question. Good RAG checks what the questioner may see for every search and filters results accordingly. If the system only checks when indexing, it falls behind as soon as someone changes department. How this works out with an assistant searching an entire environment is explained in Microsoft 365 Copilot and the GDPR.
The passages found become part of the conversation. If the language model runs with a different party from the index, those passages go there with everything they contain, and become subject to that party's retention policy. The user's own question goes there too.
This is where data minimisation becomes concrete. Article 5(1)(c) GDPR requires data to be limited to what is necessary. With RAG, that translates into three controls: how many passages are included per question, how long they are, and whether they still contain names. For which agreements each provider offers, see Which plan for business use?.
A RAG application on client files is processing and belongs in your record of processing activities, with the parties providing the index, embeddings and language model. Whether a DPIA is needed depends on what the documents contain and the scale. An outline is in DPIAs for AI tools.
A short list to compare with an offer.
PrivaSearch follows this search pattern, with one step beforehand: before a document becomes searchable, personal data is detected and replaced with pseudonyms, and a colleague approves the result. The same name always receives the same pseudonym within a project, keeping a passage readable. Until approval, a document is not searchable. The steps are on How PrivaSearch works.
The index therefore contains checked, pseudonymised text. Semantic search uses a European AI provider with a data processing agreement, and only that text. Personal data detection takes place inside PrivaSearch itself, without an external AI provider. There is one exception, explained on Hosting in the EU: a scan without a text layer goes as a file to the same European provider for text extraction, followed by the same pseudonymisation and review. Originals, the search index and detection run in data centres in the EU. Which party handles each stage is listed in the sub-processor overview.
Search runs per project and returns passages with source references, only for people with access. With a connected SharePoint library, each request checks live whether the user may still see the document there. If someone types a name in PrivaSearch's search field, it is replaced before the question reaches the model. Each project has an end date. Afterwards, copies, fragments and pseudonyms are removed unless a legal hold applies.
To have an assistant write an answer, connect Claude, ChatGPT Business or Mistral Vibe (formerly Le Chat). For each search, the assistant receives at most eight passages from the same released text, never a file. What is returned becomes part of the conversation and reaches that assistant's provider. That is why it is pseudonymised and limited. Pseudonymised data remains personal data for your organisation: this is not anonymisation. The difference is explained in Pseudonymising or anonymising.
Want to see what a question with source references looks like on your own fictional documents? Book a demo.
RAG stands for retrieval-augmented generation. A system first finds the best matching passages in its own document collection and supplies them to a language model together with the question. The model writes an answer based on those passages, often with a source reference. The model learns nothing new: the documents are in a search index alongside it.
RAG supplies a model with information it lacks for each question, from an index updated separately from the model. Fine-tuning trains the model itself further on examples and mainly changes how it behaves: format, tone and approach. If you want answers from your own client files, RAG is usually the appropriate approach. Data used to train a model is also difficult to remove.
That depends on the setup, not the technology. Ask three questions: what is in the index, who may find what, and which passages go to which provider. A system that checks permissions for each search, supplies little text per answer and replaces personal data beforehand processes far less than a system making everything searchable for everyone.
Yes. RAG reduces fabricated answers because the model receives the text containing the answer. But search can return the wrong passages, the model can misrepresent a correct passage, and the source itself may be outdated. Every answer should therefore include a source reference that you open before relying on the answer.
An embedding is a series of numbers representing the meaning of a piece of text. An embedding model creates one for every passage and every search query. Texts with similar meanings receive numbers close to one another. This lets a search system find passages about the same subject even if they use different words. Embeddings are derived from the text and therefore share its retention period.
Not the documents in full. But the passages found for a question do: they become part of the conversation and fall under the model provider's retention policy. The user's question goes there too. How much is included depends on the number of passages, their length and whether they still contain personal data. Ask your supplier about all three.
What the Model Context Protocol is, how a connection differs from uploading a file, what an assistant receives and which risks you should check.
What a language model does, what tokens and a context window are, why a model invents things and what it does and does not remember from your input.
What an AI detector measures, why its result is a probability rather than proof, and what to arrange before challenging a pupil or employee about it.
We show it with fictitious client files from your sector: upload, detection, review, search and the connection with your AI assistant. Afterwards you'll know exactly what your team can use safely.
Book a demo →