AI Governance in Education

AI governance and regulation is reshaping education more profoundly than any other public-sector domain. Under the EU AI Act, AI systems used in educational admissions, student assessment, learning outcome evaluation, and exam proctoring are classified as high-risk — placing them in the same regulatory tier as AI in law enforcement and critical infrastructure. As high-risk enforcement provisions take effect in 2026, every institution deploying adaptive learning platforms, automated grading, or AI-powered proctoring must comply with conformity assessments, human oversight mandates, and technical documentation requirements. In the United States, the intersection of FERPA, COPPA, and emerging state-level AI laws creates a patchwork of compliance obligations that ed-tech vendors and school districts are still struggling to navigate.

The EU AI Act: Education as a High-Risk Domain

Annex III, Category 3 of the EU AI Act explicitly designates four education use cases as high-risk: AI determining access to educational institutions, AI assigning students to programs, AI evaluating learning outcomes, and AI monitoring behavior during assessments. This means platforms like Turnitin's AI writing detection, Proctorio's exam monitoring, and adaptive learning systems from companies like DreamBox and Carnegie Learning must undergo conformity assessments before deployment in the EU market.

The practical requirements are substantial. High-risk education AI systems must maintain risk management systems with continuous monitoring, implement data governance protocols ensuring training data quality and representativeness, provide technical documentation sufficient for third-party auditing, and guarantee meaningful human oversight — meaning a teacher or administrator must be able to understand, interpret, and override AI decisions. The AI literacy obligations that took effect in February 2025 require every organization deploying AI to ensure staff have sufficient understanding of how these systems work, creating an urgent professional development mandate across European education systems.

US Regulatory Patchwork: FERPA, COPPA, and State Laws

The American regulatory landscape for education AI is fragmented but intensifying. At the federal level, FERPA's "school official exception" — the legal mechanism by which ed-tech vendors access student records — was never designed for an era of large language models trained on vast datasets. When a school district deploys an AI tutoring system under a school official agreement, questions arise about whether student interaction data becomes training data, whether it flows to model providers, and whether the "legitimate educational interest" standard stretches to cover AI model improvement.

The FTC's updated COPPA Rule, effective since 2024, strengthened data minimization and retention limits for children under 13, directly impacting AI tools deployed in K-8 classrooms. Companies like GoGuardian and Lightspeed Systems, which monitor student activity on school-issued devices, must now demonstrate that data collection is strictly necessary and that retention periods are minimized — a tension with AI systems that benefit from large historical datasets.

At the state level, Colorado's AI Act (SB 24-205) requires algorithmic impact assessments for high-risk AI systems, which encompasses educational tools that make consequential decisions about students. California's AI legislation and its CCPA/CPRA framework add additional consent and transparency requirements. As of early 2026, at least 15 states have introduced or passed legislation specifically addressing AI in education, creating a compliance maze for vendors operating nationally.

Academic Integrity and the Detection Arms Race

The governance of AI writing detection tools has become one of education's most contentious regulatory flashpoints. Turnitin, which processes submissions from over 16,000 institutions, reports AI detection accuracy above 98% for fully AI-generated text with a sub-1% false positive rate. But independent research — including a widely cited 2023 Stanford study — found that AI detectors disproportionately flag writing by non-native English speakers, raising serious equity and civil rights concerns.

Competitors like GPTZero and Copyleaks have entered the market with alternative detection methodologies, but the fundamental governance question remains unresolved: should AI detection tools be treated as high-risk AI systems themselves? Under the EU AI Act's framework, an AI system that effectively determines whether a student is accused of academic misconduct — potentially resulting in expulsion — arguably meets the threshold. Several European universities have already paused AI detection pending regulatory clarity, while others have shifted their academic integrity policies to emphasize AI-transparent assessment design rather than detection-based enforcement.

Institutional AI Governance Frameworks

Forward-thinking institutions are building internal governance structures rather than waiting for regulation. The University of Michigan's GenAI Committee established one of the first comprehensive institutional frameworks, covering acceptable use policies, data handling standards, and faculty guidance for AI-integrated pedagogy. Stanford's Human-Centered AI Institute (HAI) has influenced governance thinking beyond its campus, publishing research and frameworks adopted by institutions worldwide.

In K-12, the trajectory has been dramatic. New York City's Department of Education banned ChatGPT in January 2023, reversed course by May 2023, and has since developed one of the most detailed district-level AI governance frameworks in the country. The Los Angeles Unified School District went further, developing "Ed.AI" — a custom AI chatbot for students — with an internal governance structure covering data handling, content safety, and bias monitoring. UNESCO's September 2023 guidance recommending a minimum age of 13 for generative AI use has been adopted as informal policy by many districts, though enforcement varies widely.

PowerSchool, serving over 45 million students across its platforms, has embedded AI governance features into its administrative systems, including audit trails for AI-assisted decisions and role-based access controls that align with FERPA requirements. Instructure (Canvas) and Anthology (formerly Blackboard) have similarly integrated governance tooling into their learning management systems, recognizing that compliance features are now competitive differentiators.

The Speed Problem: Governance vs. Innovation

Education AI governance faces the same temporal mismatch that Jon Radoff has documented across the broader AI landscape: the technology evolves faster than institutions can regulate it. The 92% inference cost deflation over three years means AI capabilities that were prohibitively expensive for schools in 2023 are now commodity services. By the time the EU AI Act's high-risk provisions are fully enforced in 2026-2027, the AI systems they were designed to regulate may have been superseded by architecturally different successors — AI assistants that blur the line between tool and tutor, compound AI systems that combine multiple models in ways that resist categorization under existing risk frameworks.

This speed mismatch is particularly acute in education because the stakes involve minors. A flawed AI system in enterprise software causes productivity losses; a flawed AI system in education can misdirect a child's learning trajectory, create discriminatory admissions outcomes, or expose student data in ways that follow them for decades. The governance challenge is not just getting regulation right — it's getting it right fast enough to matter.