Choosing an AI tool for your school: a checklist built on the DfE's product safety standards

Most schools now get a steady stream of AI products pitched to them, and the same question lands on several desks at once. Is this safe to use here? The Department for Education’s Generative AI: Product Safety Standards give a shared starting point. They are written for the companies that build AI products for education, but they work just as well as a frame for the people who have to choose one.

This guide turns those standards into a practical checklist. It is our reading of them, not the DfE’s text, so read the standards and the DfE’s guidance on procuring EdTech before you rely on it.

Who this is for

Choosing an AI tool is rarely one person’s decision, and each role tends to ask different questions.

• Headteachers and senior leaders decide whether a tool fits the school’s priorities and is worth the risk and the cost.
• Designated safeguarding leads need to know what pupils could see, say or be exposed to, and how concerns reach them.
• Data protection officers need to know what personal data goes where, on what legal basis and under which contract.
• IT leads and network managers need to know how staff sign in, how the service is secured and what it connects to.

The checklist below is grouped so that each of you can take your section, then compare notes.

Start with two questions

The standards open with a product’s stated purpose and its educational use cases. Before anything else, ask the vendor to say plainly what the tool is for and who uses it.

That second point changes everything that follows. The DfE has 13 standards in all, and five of them are written for learner-facing products, the ones pupils use directly. They cover filtering, monitoring and reporting, cognitive development, emotional and social development, and mental health. If pupils will type into the tool or read what it generates on their own screens, all five apply in full. If only staff use it, they may not apply at all.

Do not take “teacher-only” on trust. Check whether the product has pupil accounts, shareable links pupils can open, a class code or any chat a pupil could reach. If it does, treat it as learner-facing, and all five learner-facing standards apply.

The checklist

Five themes, each with the question that matters most and the answer that should make you pause. The detailed questions for each theme follow the table.

Theme	The question that matters most	Pause if you hear
Filtering and safety	How is harmful content stopped, and is it re-tested when the AI model changes?	“The AI provider handles that.”
Data protection	Does any feature need pupil personal data, and is anything used to train AI?	“Only anonymised data is used for training.”
Monitoring and reporting	How does a concern reach your DSL, and how fast does the vendor respond?	No named contact for safeguarding.
Transparency	Is there a published statement against the DfE standards?	“We can talk you through it on a call.”
Teacher control	Does a teacher review every output and decide every mark?	“It sends feedback to pupils automatically.”

Filtering and safety

• If pupils use the tool, how does it stop harmful or inappropriate content, both in what pupils type and in what it generates?
• Is filtering on by default, and who in school can change it?
• How has the vendor tested the tool against deliberate attempts to get round its safeguards?
• When the vendor moves to a new AI model or version, is it tested again before release?
• Can staff sign in with your own Microsoft or Google accounts, so your password and multi-factor authentication rules apply?
• Is data encrypted in transit and at rest, and how often is security tested?

Data protection

• Does the tool need pupil personal data at all? The strongest answer is that no feature needs a pupil’s name, contact details or identification number.
• Are prompts, uploads or outputs used to train AI models, by the vendor or by the AI providers it uses? Is that the default for every account, or a setting someone has to find?
• Where is data stored and processed? If any of it leaves the UK, which safeguards cover the transfer?
• Will the vendor sign a Data Processing Agreement with the school as controller, and give you its list of sub-processors?
• Will it help with your data protection impact assessment?
• Does it connect to your management information system, and if so, what data flows each way?
• Are teachers’ resources and pupils’ work used to develop the product? Who owns what the tool produces?
• What happens to your data when you stop using the service?

Monitoring and reporting

• For a learner-facing tool, can staff see what pupils have done, and how are safeguarding concerns brought to the designated safeguarding lead?
• For any tool, can a school administrator see how staff are using it? Is use of AI logged?
• How do you report a concern about an output, and how quickly does the vendor acknowledge and resolve it?
• Is there a named person at the vendor responsible for data protection, and one for safeguarding?

Transparency

• Are the privacy notice, terms and cookie policy published, in language a parent could follow?
• Has the vendor published how its product measures up against the DfE standards, or will it share a statement on request?
• Does it say which kinds of service providers handle your data, such as hosting, AI models and email?
• Does it say plainly that AI output can be inaccurate or incomplete?
• Is the product free of advertising, dark patterns and features designed only to keep people using it? Are plan limits and prices stated plainly?

Teacher control

• Is every output a draft that a teacher reviews before it reaches a classroom? Does the tool ever send anything to pupils on its own?
• If the tool marks work, does the teacher decide every mark? Does it make decisions about pupils, or profile or rank them?
• Can teachers edit what the tool produces and set the criteria it works to, such as a mark scheme or rubric?
• If pupils use the tool, does it help them think, or does it do the thinking for them?
• Was the product designed and tested with teachers, across the subjects and key stages you teach?

Questions to ask any vendor

If you only have time for one email, these ten questions cover most of the ground. Copy them, send them and compare the answers side by side.

1. What is the product for, and do pupils use it directly?
2. Does any feature need pupil personal data?
3. Is any of our data used to train AI models, by you or your AI providers?
4. Where is our data stored and processed, and which safeguards cover any transfer outside the UK?
5. Will you sign a Data Processing Agreement and share your sub-processor list?
6. How do you test for harmful output and for attempts to get round your safeguards, including when models change?
7. What can our administrators see about how the tool is used?
8. How do we report a concern, and how quickly will you respond?
9. Does a teacher review every output, and every mark, before it reaches a pupil?
10. Have you published how you meet the DfE’s Generative AI: Product Safety Standards?

A good vendor answers in writing, specifically, and points you to documents rather than reassurance. A vague answer to questions 2, 3 or 9 is worth a follow-up call before you go any further.

How Madlen answers these

We build Madlen, an AI copilot for teachers, so it is fair to say how we answer the same list.

Madlen is teacher-facing. Pupils do not have accounts and do not use it, so the five learner-facing standards do not apply to it. Its tools are designed to work without pupil personal data, and we ask teachers to remove names before uploading work. We do not use prompts, uploads or outputs to train AI models, and neither do the AI providers we work with. Data is hosted in the UK and the European Union, and the few transfers outside them, to some supporting providers and our group company in Turkey, are covered by recognised UK transfer safeguards. Every output is a draft that a teacher reviews, and the teacher decides every mark.

Some answers cannot sit on a public page. The Data Processing Agreement, our sub-processor list and completed security questionnaires are available on request. Our Trust and Compliance page sets out how Madlen aligns with each of the 13 standards, and answers the DfE’s EdTech procurement questions for us.

Whichever tool you are looking at, the test is the same. If a vendor cannot answer these questions clearly, that is an answer in itself.