PrivaSearch

Technology

Automatically detecting personal data in documents: how reliable is it?

Why software that finds names, addresses and numbers in client files always misses something or masks too much, and how a human review catches it.

In brief

  • Automatic detection of personal data always makes two kinds of mistake: it masks words that are not personal data, and it leaves personal data in place.
  • Data with a fixed format, such as a BSN or an IBAN, is easy to find; names, job titles and circumstances that make someone recognisable are much harder.
  • The GDPR calls pseudonymisation a measure that reduces risk, not a way to fall outside the law: what you pseudonymise remains personal data for you.
  • A human review is not a formality but the part that catches what the software misses, especially indirect data and health or criminal matters in a subordinate clause.
  • Record who reviews, what they check and when a document may move on, and measure what is missed instead of relying on a percentage on a website.

How reliable is automatic detection of personal data?

Good enough to save a lot of work, never good enough to trust blindly. Software finds numbers with a fixed format fairly reliably, but misses names that look like ordinary words and rarely sees that a job title or a place makes someone recognisable. Conversely, it masks words that are not a person. That is why a human belongs in the process.

This article is about the technology behind that detection and about what the GDPR expects of it. The difference between pseudonymising and anonymising is explained in Pseudonymising or anonymising: the difference, and why it matters for AI. Which data you may put in a chatbot is covered in Can you put client data in ChatGPT, Claude or Mistral Vibe?.

What does the software actually have to find?

More than names. In Article 4, point 1, the GDPR covers any information about a natural person who can be identified directly or indirectly, for example by a name, an identification number, location data or one or more factors specific to the person's physical, economic, cultural or social identity. Recital 26 adds that you take into account all the means reasonably likely to be used to identify a person.

For a document, that means three layers.

Software is strongest in the first layer. The second and third require an understanding of the text and of the world around it.

Which two mistakes does every detection make?

Every detection, by hand or by software, makes two kinds of mistake. In the language of statistics they are called false positives and false negatives. In a client file it means: too much masked or too little masked.

Too much masked

A word that looks like a name is replaced although it is not a person. "De bakker" (the baker) in a lease, the month of May, the colour green, the name of a municipality or the Belastingdienst as an authority, or a label such as "Name:" above an empty field. The result is a text that reads worse and answers that are less useful. A search for the notice period at the baker finds nothing once "baker" has become a code.

This is mainly a quality problem. Anyone who gets too much of it starts to distrust the detection or to work around it, and in the long run that becomes a privacy problem too.

Too little masked

Personal data stays in. A surname that is also an ordinary word, a name in an email signature at the bottom, a misspelt name, or a name in a scan whose text was poorly extracted. Then there is indirect data: "the only female partner of the firm in the east of the country" contains no name and still points to one person.

This is the privacy problem. Whatever stays in goes along to whoever reads the text next, including an AI model.

Why you cannot get both to zero

The two mistakes pull against each other. Set the detection stricter and it misses less but masks more without reason. Set it looser and the text reads better but more slips through. A supplier quoting a high percentage therefore says little as long as it is unclear which mistake was counted, on which documents and in which language. Measure it on your own kind of client files.

Why are some data harder than others?

The difference lies in the format. A BSN, an IBAN, a postcode and a number plate have a fixed structure. A pattern finds them reliably, and a mistake is usually a number that happens to have the same format. A name has no fixed format. That needs a language model that works out from the sentence whether a word is a person.

Three things make it harder.

That is why it makes sense to think differently per type of data. Numbers with a fixed format you leave to the software and check by sampling. Names you check per document. Indirect and sensitive data need someone who knows what the file is about.

What does the GDPR require of automatic detection?

The GDPR does not prescribe a technology, but it does prescribe an attitude. Article 25 requires data protection by design and gives pseudonymisation as an example of an appropriate measure. Article 32 mentions pseudonymisation and encryption under the security of processing. Recital 28 says that pseudonymisation can reduce the risks to data subjects, and that it does not exclude other measures.

A measure, not a way out

That last point is the core. Pseudonymising reduces the risk, but it does not take a document outside the GDPR. Recital 26 says so explicitly: pseudonymised data that can be linked to a person with additional information is to be considered information about an identifiable person. For the organisation that keeps the key, it therefore remains personal data, with a lawful basis, a register entry and, where needed, a DPIA.

What the supervisory authorities say

The European supervisory authorities published guidelines on pseudonymisation in January 2025, with a legal and a technical part. That page holds the consultation version; no final version can be found there (as of November 2026). The Autoriteit Persoonsgegevens (AP), the Dutch data protection authority, summarised the guidelines in its announcement: pseudonymising is a security measure that makes data harder to trace back and can reduce the impact of a data breach.

A detection that systematically misses something is therefore a weak measure. Whatever still ends up with an AI service may be a data breach. The AP has already warned about this after reports of personal data that employees had entered into an AI chatbot. How to assess whether you must notify is covered in Data breach through an AI chatbot: when must you notify the AP?.

How do you set up the human review?

A review only helps if it actually happens and if the reviewer sees what the software did. Five agreements make the difference.

  1. Show the original next to the edited text. Anyone who sees only the edited text cannot see what was missed.
  2. Make approval a deliberate step. A document moves on only once someone has explicitly approved it, not because a deadline passed.
  3. Assign the review to whoever knows the file. The case handler recognises the rare job title or the small town; a random colleague does not.
  4. Give extra attention to sensitive files. Sickness absence, medical records, criminal cases and debt counselling get a stricter look than a supply contract.
  5. Keep track of what was missed. Anyone who notes which types of data the reviewer adds knows after a few weeks where the detection is weak and can adjust the agreements.

Include those agreements in your AI policy. See Writing an AI policy for your organisation: steps and examples.

What PrivaSearch does here

PrivaSearch sits between your documents and the AI model. A language model inside PrivaSearch and Dutch patterns find names, addresses, BSNs, IBANs, number plates and dates of birth. No outside AI provider looks in at this step. The data found is replaced by pseudonyms that stay the same for the same name within a project.

Because automatic detection is never complete, the human review is mandatory. A colleague sees the original and the pseudonymised text side by side, corrects what the model missed and explicitly approves. Only then does a document become searchable, in PrivaSearch itself or through a connected AI assistant. The steps are on How PrivaSearch works.

An AI assistant searching through the connection receives only released, pseudonymised passages, no original files, file names or links. What you type into the assistant yourself goes straight to the provider, outside the pseudonymisation. So do not put names there. The wider picture is on ChatGPT, Claude, Mistral Vibe and the GDPR.

Would you like to see what the review looks like with your own kind of client file? Book a demo.

Frequently asked questions

Can software find all personal data in a document?

No. Software finds data with a fixed format, such as a BSN or an IBAN, fairly reliably, but misses names that look like ordinary words, names in a poor scan and data that only makes someone recognisable in combination. That is why automatic detection always needs a human review.

What is a false positive when detecting personal data?

A false positive is a word that is masked as personal data although it is not, such as "baker" as a profession, the month of May or the name of an authority. It leaks nothing, but the text reads worse and search returns less. Too many of these mistakes make people work around the detection.

Is a document anonymous after automatic pseudonymisation?

No. Pseudonymised data remains personal data for the organisation that keeps the link, says Recital 26 of the GDPR. In addition, a text without names can still make someone recognisable through a job title, a place or an event. A document is only anonymous once nobody can reasonably work out who it is about.

Is a human review mandatory under the GDPR?

The GDPR does not mention a human review for pseudonymisation. It does require appropriate measures and data protection by design. A detection known to miss things, with no review behind it, is hard to defend as appropriate. In practice, the review is therefore the part that makes the measure work.

How do you know how well a detection works?

By measuring it on your own documents. Have a colleague review a series of client files and note what was added and what was masked without reason. A supplier's percentage says little as long as it is unknown which mistake was counted, on which documents and in which language.

May an AI model do the detection?

Yes, provided you know where that model runs. If the whole document first goes to an external AI service to find the personal data in it, that service has already seen the personal data, and everything that comes with sharing with a provider applies. Detection that happens inside your own environment prevents that.

Read more

  1. Power BI Q&A ends in December 2026: how do you still ask questions about your own data?

    Microsoft is retiring Q&A in Power BI and points to Copilot, which needs Fabric capacity. What goes, what the alternative needs and what about your documents?

  2. New in PrivaSearch: see in your workspace what your AI assistant does

    A small window in the workspace shows in which project your assistant searched and which documents it found. The assistant itself does not get the names.

  3. Connecting GreenPT to your own client files: what MCP and PrivaSearch can offer

    GreenPT is working on MCP with sign-in. What a connection with PrivaSearch would then offer: searching your client files with pseudonymised passages.

See in twenty minutes what happens to your documents.

We show it with fictitious client files from your sector: upload, detection, review, search and the connection with your AI assistant. Afterwards you'll know exactly what your team can use safely.

Book a demo →