AI in Schools: Understanding Data Protection, Privacy, and Student IP

Artificial Intelligence (AI) is quickly finding its way into classrooms — from tools that can analyse assessment data to apps that create lesson resources in seconds. For many school leaders and teachers, the question isn’t whether to use AI, but how to use it safely.

And yet, when it comes to data protection and privacy, the rules can feel blurry. You likely already know that safeguarding student information is important, but what exactly counts as “data”? Why is student intellectual property worth protecting? And how can you tell if an AI tool is genuinely secure or simply wrapped in clever marketing?

This article breaks down three key areas — data, intellectual property, and transparency — so you can confidently decide what is and isn’t safe to share. We’ll define each term in a school setting, explain why it matters, suggest questions you should ask, and give real examples of safe practice.

1. Understanding What ‘Data’ Really Means

When teachers hear “data”, many think of assessment results, target grades, or progress tracking spreadsheets. These are certainly important, but in a legal and safeguarding context, data means much more.

Personal Data

Personal data is any information that can be used to identify a student. That includes:

• Names, addresses, and dates of birth.

• Background details, such as pupil premium eligibility.

• Medical information or learning needs.

• Photos and videos — even group shots, like a class photo.

  
  

It’s easy to overlook just how many everyday teaching tasks involve personal data. For example, you might want to create a class poster using AI that turns your students into superheroes. To do this, you’d need to upload their photos — but that’s sharing personal data, even if it feels harmless.

How and Where You Share Data

Even if your intention is good, where you share data matters just as much as what you share. You cannot paste students’ personal or performance information into a public tool like ChatGPT and assume it’s safe.

Instead, any AI tool you use must:

• Be secure and meet data protection standards.

• Ideally, be approved by your school’s IT lead or leadership team.

• Store data in a way that complies with relevant regulations (such as the UK GDPR).

Questions to Ask Before Using AI with Student Data

1. Is this an approved tool? Has someone in leadership checked its security and privacy policy?

2. Where will the data be stored? Is it hosted securely and protected from unauthorised access?

3. Will the data be used to train the AI? If yes, stop — that’s a red flag.

4. Do we have clear guidance for staff and students? Is everyone aware of what can and can’t be shared?

   
   
   
   

Good Practice Example

Some schools keep a short, up-to-date list of approved AI tools. Staff are encouraged to stick to these, and any new tool must be checked by leadership before use. This avoids the risk of well-meaning teachers experimenting with apps that don’t meet data protection requirements. Policies also make it clear to students what “data” means and why it matters.

Section 6 of our privacy policy tells you what data we use, and how we store it, at stylus.

2. Protecting Student Intellectual Property (IP)

When we talk about intellectual property in education, it’s easy instead to think of copyright — for example, knowing you can’t photocopy and distribute an entire textbook without permission, or that you shouldn’t present someone else’s work as your own. But in a school context, it also means recognising that students own the rights to their own creations — whether that’s a poem, a science project, or a piece of artwork.

Why AI Creates New Risks

Large Language Models (LLMs) like ChatGPT are trained on vast amounts of text from across the internet — some of it public, some of it taken without permission. This has sparked debates (and legal battles) about whether these systems have been built using other people’s intellectual property.

Because of this, many organisations — including schools — are now choosing to protect their own work and their students’ work from being used to train these systems. If student work is fed into an LLM that uses it for training, it could, in theory, contribute to another user’s output.

Example: A student writes a deeply personal narrative for an English assignment. If that work is uploaded into a system that uses all inputs to improve its AI model, the personal details and style could potentially be echoed in someone else’s generated text later on. Even if the risk is small, the ethical and safeguarding implications are serious.

Training vs. Improving Prompts

There’s a big difference between training an AI model and improving the way you use it.

• Training: Feeding new examples into the AI so it permanently learns from them (which can put student work into the model’s knowledge base).

• Improving prompts: Looking at where the AI output differs from what you expect, then refining the instructions you give it — without storing student work in the AI’s memory.

  
  

Good practice example: At stylus, we never use student work to train LLMs through our assessment process. Instead, if we identify a difference between an AI-generated mark and what a human marker would give, we adjust the detailed prompt the AI uses, so future marking better reflects the intended standard. This keeps the student’s work private while still improving accuracy.

Questions to Ask Before Sharing Student Work with AI

1. Will this work be stored or used to train the AI?

2. Can I get a clear “no” from the provider, in writing?

3. Is there a safe, offline way to refine results without giving the AI permanent access to the work?

4. Does our school policy explicitly cover student IP rights?

3. Transparency in AI Tools

When education companies talk about AI, you might hear the term “GPT wrapper”. In simple terms, a wrapper is when a company builds a website or app around an existing AI tool like ChatGPT. They add an attractive interface, maybe some well-crafted prompts, and market it as a unique service.

The tool may slightly adjust your input (or sometimes leave it unchanged) before sending it to the LLM, and then returns exactly the same response it received from the LLM.  

Underneath, however, it’s still the same AI engine you could use yourself — meaning, with a slight additional effort, you could achieve the same response by interacting with the LLM directly.

Why This Matters

If the core engine is ChatGPT (or another LLM), the tool will have the same strengths and weaknesses as the original:

• Strengths: Speed, fluency, and ability to generate useful first drafts or ideas.

• Weaknesses: “Hallucinations” (made-up facts), overconfidence, and occasional inaccuracy.

  
  

So, while these tools may save time — especially for teachers who aren’t confident with AI prompting — they’re not inherently more accurate or reliable than ChatGPT itself. The risk is that you may assume the output is more trustworthy simply because it comes in a polished package.

Good practice example: Many schools now have AI usage policies which make it clear that the teacher is responsible for checking any AI output. Resources created using AI are shared among the team for quality assurance, and then stored in a central resource bank. Prompts which generate high-quality resources are shared with the rest of the team. We’d recommend subscribing to the Teacher Prompts newsletter for suggestions of effective prompts which generate usable results first time. 

At stylus, we work with a team of moderators who help us ensure our marking is teacher-quality — so you don’t need to worry about hallucinations or misconceptions filtering through to our feedback. 

Questions to Ask AI Vendors

When a company offers an AI tool for education, ask:

1. What makes your tool different from ChatGPT?

2. How do you prevent hallucinations or factual errors?

3. Do you have a quality assurance process for the outputs?

4. How do you handle sensitive or personal data?

5. If the AI makes a mistake, who is responsible for correcting it?

If the vendor can’t clearly explain their safeguards, you may still need to proofread and fact-check everything yourself — meaning the time saved might be less than you think.

Practical AI Safety Checklist for Schools

To make decisions quickly and confidently, school leaders can use a simple, repeatable checklist:

✅ Define “data” for your context — include personal data, assessment results, photos, and anything that could identify a student.

✅ Approve AI tools before use — and keep an up-to-date list for staff.

✅ Protect student intellectual property — get clear confirmation that their work won’t be used for AI training.

✅ Ask about transparency — understand what’s really behind a tool and what quality checks are in place.

✅ Train staff and students — make sure everyone knows what they can and can’t share, and why.

✅ Review regularly — technology changes fast; so should your policies.

Conclusion

AI can be an incredible tool for schools — helping teachers save time, access new resources, and gain insights from data. But its benefits are only truly realised when used safely and ethically.

By understanding what counts as data, protecting student intellectual property, and demanding transparency from AI providers, schools can avoid the pitfalls while making the most of the opportunities.

Ultimately, the goal isn’t to avoid AI — it’s to use it confidently, responsibly, and in a way that always puts students first.