AI integration- What to consider when partnering with an external provider

Understanding how a provider handles data and manages security before committing to a partnership.

Staff

AI may offer opportunities to support administration and planning, but choosing a tool starts with a question: what problem are you trying to solve?

At the Education Summit, technology and education leaders discussed the importance of understanding a tool’s purpose, how it handles information and who remains responsible for decisions. Before entering a partnership, consider the following:

What will the tool do? Ask the supplier to explain its intended use, limitations and potential risks. Decide how staff will check its output as AI can make mistakes and should not replace professional judgement.

What information will it use? Establish what data will be entered, where it will be stored, who can access it and when it will be deleted. Ask whether information is used to develop or train the supplier’s systems. Do not enter identifiable or sensitive information about children into an unapproved tool.

Who else handles the data? Ask about subcontractors, where processing takes place and what arrangements protect information shared with them. Review the contract as well as the supplier’s privacy information.

What security is in place? Discuss access controls, multi-factor authentication, incident reporting and staff training. Relevant independent certifications can help your assessment but check their scope and validity as a certificate alone does not guarantee that a product is suitable or compliant.

Is a data protection impact assessment needed? Assess the proposed use before introducing it. A DPIA is required where processing is likely to result in a high risk to individuals, it is not automatically required for every tool.

Finally, make sure staff know what has been approved, how to use it safely and where to raise concerns.

 

Disclaimer: This article aims to provide practical suggestions only. Always follow your setting’s safeguarding procedures and seek specialist or professional support where appropriate.

 

References:

https://www.gov.uk/government/publications/accessing-ukhsa-protected-data/approval-standards-and-guidelines-data-security

https://www.gov.uk/government/publications/generative-artificial-intelligence-in-education/generative-artificial-intelligence-ai-in-education