ChatGPT and social work case notes: is it a GDPR breach?

OpenAI is unusually clear about this. For its consumer products, the ones most care workers sign up to on their own phone, it says:

There is an opt-out in the settings, and a Temporary Chat mode that is not used for training. Business products are different: OpenAI states that by default it does not train on inputs or outputs from ChatGPT Business, ChatGPT Enterprise or the API. But a care worker pasting a note into the free app on their phone is not on a business plan, has usually not changed a setting, and has no agreement with OpenAI at all.

Training is only part of it. Even with training switched off, the text has left your organisation, been stored on a third party’s systems, and been processed for a purpose the person it describes was never told about. That is the data protection problem, and it exists whichever AI service is on the other end.

Four things go wrong at once when a case note goes into a chatbot.

• It is personal data, and usually special category data. Health, disability and safeguarding information sit in the most protected class under Article 9 of the UK GDPR, which needs a specific condition on top of an ordinary lawful basis.
• There is no processor contract. UK GDPR Article 28 requires a written contract with anyone processing personal data on your behalf. A consumer AI account gives you a terms page, not a controller-processor agreement.
• It may be an international transfer. Many AI services process data outside the UK. Transfers need a lawful mechanism and, under the ICO’s guidance, a risk assessment.
• Nobody assessed the risk. The ICO’s AI guidance says that using AI will usually involve processing "likely to result in a high risk to individuals’ rights and freedoms", which triggers the legal requirement for a data protection impact assessment.

The ICO also draws the line that matters for the rest of this guide: anonymous information is outside data protection law entirely, and personal data is not.

None of the bodies below say "do not use AI". All of them say the same three things: do not put personal data into generic tools, keep a human responsible for the output, and assess the risks before you start.

The British Association of Social Workers published the first practice guidance on generative AI in social work in March 2025. As Community Care reported, it advised practitioners to:

The same report noted that "accountability ultimately sat with the social worker using the tool" and that "data protection assessments should also be undertaken before introducing AI products within services".

The Department for Education’s National Workload Action Group looked specifically at AI in children’s social care case recording in September 2025. It was positive about the potential and blunt about the risks:

The Care Quality Commission set out its expectations for providers using AI in May 2026. It does not assess or approve individual tools, but it expects a data protection impact assessment, trained staff, human oversight and transparency with the people who use services. Ofsted takes a similar line: inspectors "can consider the impact that the use of AI has on the outcomes and experiences of children" and may ask leaders how they make sure AI use supports children’s best interests. Both are covered in detail in our guide to what regulators expect.

The maximum penalty for the most serious breaches under the Data Protection Act 2018 is £17.5 million or 4% of annual worldwide turnover, whichever is higher. In practice the ICO uses its full range of powers, from reprimands and enforcement notices upwards, and the consequences that hurt a care provider first are usually the ones that are not fines at all: a reportable breach, a safeguarding review, a regulator asking questions at the next inspection, and a family finding out.

It is also worth being honest about the reverse risk. Staff use these tools because the writing load is real. A policy that just says "no" tends to push the behaviour out of sight rather than stop it, which is exactly the situation the DfE report describes: "fragmented approaches and lack of oversight and support pose risks".

The workable answer is to take the personal data out before the text goes anywhere. If the AI only ever sees an anonymised version, most of the problems above disappear at the same time: there is no personal data in the transfer, nothing for the model to learn, and nothing to breach.

The ICO’s guidance on pseudonymisation describes the mechanism precisely. Replacing identifiers with placeholders and keeping the key separately means the data is still personal data in your hands, and may be anonymous in the hands of a recipient who cannot re-identify anyone:

Doing that by hand on every note is slow and error-prone, which is why it does not happen. Caredact does it automatically: paste the note, and names, addresses, NHS numbers, references and care-specific identifiers are detected and replaced with labelled placeholders. A person checks the highlights and marks anything the detector missed, the AI works on the checked version, and the export puts the real details back. The placeholder map is never sent to the AI, and it is deleted when the document is finished or reaches its retention date. The practical steps are in our guide to anonymising case notes.

• Write the policy so it names an approved route rather than only a prohibition. Our free AI policy template is built that way.
• Do the DPIA once, for the approved tool, instead of pretending the unapproved ones are not being used. There is a DPIA template pre-filled for AI in social care.
• Get a Data Processing Agreement with whatever you approve. If a vendor cannot offer one, that is your answer.
• Keep a person in the loop and keep a record of what was sent.