PrivaSearch

Rules

Data breach through an AI chatbot: when must you notify the AP?

When entering personal data into an AI chatbot is a data breach, when you must report it and what to do during the first 72 hours after discovery.

In brief

  • According to the Autoriteit Persoonsgegevens (AP), the Dutch data protection authority, an employee entering personal data into an AI chatbot against agreed rules causes a data breach.
  • Not every data breach needs reporting: the duty depends on the risk to the people the data concerns.
  • If notification is required, you do so without undue delay and, where feasible, within 72 hours of becoming aware of the breach.
  • You inform the data subjects themselves only if the breach is likely to present a high risk to them.
  • You document every data breach in an internal register, even if you do not report it.

When must you notify the AP of a data breach through an AI chatbot?

You notify the Autoriteit Persoonsgegevens unless the breach is unlikely to present a risk to the people the data concerns. Sensitive data, such as medical data or a Dutch citizen service number (BSN) with name and address, generally presents that risk. You notify within 72 hours where feasible.

That rule is in Article 33 GDPR. The test has two thresholds. A risk is enough to notify the supervisory authority. Informing data subjects themselves requires a high risk under Article 34. You therefore always make an assessment and document it.

What does the GDPR call a data breach?

The GDPR does not use the shortened term data breach. Article 4(12) refers to a “personal data breach”: a breach of security leading to accidental or unlawful destruction, loss, alteration, unauthorised disclosure of, or access to, personal data.

On its page what is a data breach, the Autoriteit Persoonsgegevens (AP) summarises it as access to personal data that is unauthorised or unintended, caused by a security breach. It distinguishes three types.

Intent is not required: the definition says “accidental or unlawful”. With an AI chatbot, it is almost always the first type. The data is not gone or changed. It has reached a party that should not have received it.

What does the Autoriteit Persoonsgegevens say about AI chatbots and data breaches?

On 6 August 2024, the AP published Take care: using an AI chatbot can lead to data breaches. It says it received several data breach notifications after employees shared personal data about patients or clients, for example, with an AI chatbot. Entering this data can give the companies behind the chatbot unauthorised access.

The AP gives two examples. At a general practice, an employee entered patients' medical data into an AI chatbot against agreed rules. At a telecommunications company, an employee entered a file containing information including clients' addresses.

The distinction the AP makes

The core of the notice distinguishes two situations.

The AP says organisations must prevent both situations. The second point is often overlooked. A policy allowing use means the input is not a data breach, but does not resolve the rest: you still need a lawful basis, an agreement with the provider and, where necessary, a transfer basis. This is explained on ChatGPT, Claude, Mistral Vibe and the GDPR.

What the AP advises

The AP ends with three recommendations. Make clear agreements with employees on AI chatbots. If you allow use, explain which data may and may not be entered. And explore whether you can arrange for the provider not to store entered data. If things nevertheless go wrong, the notice says notification to the AP and data subjects is “mandatory in many cases”.

When is chatbot input a data breach, and when must you report it?

The table compares common situations. It is an aid to your own assessment, not a substitute. Where it says “depends”, the risk assessment in the following section determines the outcome.

The risk assessment

On whether to report a data breach, the AP gives factors for making an objective assessment:

The recipient matters, but do not overestimate the protection

With a chatbot, the question is who receives the data and what they do with input. This varies by account. Personal accounts have consumer terms, input may be retained and, depending on the user's settings, used to train models. Your organisation does not know that colleague's settings.

As an example of a trustworthy recipient, the AP mentions a party with whom you have a business relationship, such as a regular supplier. A provider with no agreement with your organisation does not qualify. If you can have input deleted and the provider confirms it, document that. It reduces risk but does not undo the breach. What providers do with uploaded files per subscription is in uploading documents to an AI chatbot.

If in doubt, the AP is clear: err on the side of caution and report it.

What do you do in the first 72 hours?

The period starts not with the incident but when you become aware of it. European supervisory authorities explain this in their guidelines on data breach notification: you are aware once you know with a reasonable degree of certainty that personal data has been compromised. A short initial investigation is allowed if it begins immediately.

  1. Record the moment. Note when and how you learned about the input. Your investigation starts then.
  2. Establish what was entered. Which document or text, into which chatbot, on which account and when. Ask the employee to show the conversation before deleting it.
  3. Determine who is affected. Which people, types of data and how many records.
  4. Limit the damage. Have the conversation and uploaded files deleted, disable training if it was enabled and ask the provider for deletion where possible.
  5. Involve the right people. Your data protection officer if you have one, and those responsible for privacy and IT.
  6. Assess the risk. Work through the AP's factors and write down the outcome and reasons.
  7. Notify if required. Use the reporting portal on the AP's report a data breach page. If information is incomplete, you may provide it in stages. If you notify after 72 hours, give the reason for the delay.
  8. Inform data subjects for a high risk. Without undue delay, in clear, plain language: what happened, likely consequences, what you did and whom they can contact.
  9. Record everything in the register. Even if you decide not to notify.

What the notification contains

Article 33(3) lists four elements: the nature of the breach with, where possible, categories and approximate numbers of data subjects and records; contact details for the data protection officer or another contact point; likely consequences; and measures taken or proposed.

If your processor discovers the breach

If the breach comes from a provider with whom you have an agreement, it must inform you without undue delay. The provider need not assess the risk first: you do that. How quickly a provider commits to notify varies by contract. Differences are in the data processing agreement with an AI provider.

What do you record in the internal register?

Article 33(5) requires you to document all breaches: facts, effects and remedial action. This documentation lets the supervisory authority check whether you meet notification duties. European supervisory authorities encourage an internal register of all breaches, whether reportable or not. On its data breaches page, the AP offers an example register.

For a chatbot breach, record at least:

How do you prevent an AI chatbot data breach?

The AP's advice is the start. Practice adds more.

What PrivaSearch does here

PrivaSearch addresses the cause behind this type of breach: an employee wants an answer about a client file, and the whole file goes along. In PrivaSearch, personal data is first detected and replaced with pseudonyms. A colleague sees original and pseudonymised text side by side and approves. Only then is a document searchable. The process is on how PrivaSearch works.

A connected AI assistant searches only those released passages and receives no original files, filenames or links. If a search question to PrivaSearch nevertheless contains a name, it is replaced before searching. The connection works with Claude, ChatGPT Business and Mistral Vibe. Actions are logged.

Two limits apply. Automatic detection is never complete; human review is therefore mandatory. And anything employees type into the assistant goes directly to its provider, outside PrivaSearch. Pseudonymised data also remains personal data for your organisation. The reporting duty and assessment therefore remain yours. The risks of uncontrolled use are on the risks of AI on your client files.

Employees complete short AI literacy training in PrivaSearch, repeated periodically. To see how it works, book a demo.

Frequently asked questions

Is it a data breach if an employee puts client data into ChatGPT?

According to the Autoriteit Persoonsgegevens, yes, if the employee does so on their own initiative and against agreements with their employer. The chatbot's provider then gains access to personal data without authorisation. If use is instead part of organisational policy, the supervisory authority says it is not a data breach but often not legally permitted.

How quickly must you report a data breach?

Without undue delay and, where feasible, within 72 hours of becoming aware of it. The period therefore starts not at the incident itself but when you know with reasonable certainty that personal data has been affected. If you report later, give the reason for the delay. Missing information may be supplied afterwards in stages.

Must you report every data breach to the Autoriteit Persoonsgegevens?

No. You report unless the breach is unlikely to present a risk to the people the data concerns. You make that assessment yourself, using factors including sensitivity, identifiability and the recipient. If in doubt, the supervisory authority advises reporting. Recording it in your own register is always required.

When must you inform people themselves about a data breach?

When it is likely to present a high risk to them. That threshold is higher than for notifying the supervisory authority. Inform them without undue delay and in clear language about what happened, possible consequences and measures. This is not required if data had already been rendered unintelligible, for example by encryption, or if you subsequently removed the high risk.

What must a data breach register contain?

The facts of the breach, its effects and remedial action taken. This applies to every breach, including unreported ones. Also document your risk assessment and reasons for reporting or not. The supervisory authority may request the register to check whether you meet notification duties.

Can you be fined for not reporting an AI chatbot data breach?

Yes. If you wrongly fail to report a breach to the Autoriteit Persoonsgegevens, it may impose a fine. If you wrongly fail to inform data subjects, it may require you to do so. Its judgement in a specific case depends on the circumstances. For each incident, therefore document your assessment and why you made it.

Read more

  1. AI Impact Assessment (AIIA): what it is and how your organisation can use it

    What the AIIA of the Dutch Ministry of Infrastructure is, who must use it, how it is structured and how it relates to the IAMA and the DPIA.

  2. Digital omnibus: what the Council of the EU wants to change in the GDPR for AI

    Member states want AI processing to rest on legitimate interests and pseudonymisation to be regulated differently. What the text says and what to do now.

  3. The AI Act in autumn 2026: what applies now and what has been postponed?

    The AI Act timeline after the digital omnibus: what applies already, what has moved to 2027 and 2028, and what AI chatbot users need to arrange now.

See in twenty minutes what happens to your documents.

We show it with fictitious client files from your sector: upload, detection, review, search and the connection with your AI assistant. Afterwards you'll know exactly what your team can use safely.

Book a demo →