June 2026 - Minaz Jivraj My Take: AI Security Under Scrutiny: Why Facial Recognition and Behavioural Analytics Are Facing Global Regulation

Artificial intelligence has rapidly transformed the modern security industry. AI-enabled surveillance platforms now monitor campuses, analyze behavioural patterns through CCTV systems, automate access control, verify identities through biometrics, and assess potential threats in real time. These technologies are increasingly embedded across schools, universities, hospitals, transportation hubs, businesses, and public institutions.

At the same time, regulators around the world have begun to question whether such systems create unacceptable risks to privacy, civil liberties, human rights, and democratic freedoms. The European Union’s Artificial Intelligence Act (EU AI Act), formally adopted as Regulation (EU) 2024/1689, represents the most comprehensive regulatory framework yet created for artificial intelligence. Although implementation is being phased in over several years, major obligations for high-risk systems are tied to August 2026 under the current legislative framework.

The EU AI Act introduces a risk-based approach to AI governance. AI systems are categorized into four broad levels of risk: minimal risk, limited risk, high risk, and unacceptable risk. Security technologies that use biometric identification, behavioural analysis, automated decision-making, or emotion recognition are among the systems most likely to fall into the “high-risk” category. Certain uses are prohibited altogether.

While the legislation applies directly within the European Union, its global influence is already extending beyond Europe. Similar to the impact of the General Data Protection Regulation (GDPR), many multinational organizations are expected to adopt EU AI Act standards globally to simplify compliance, maintain market access, and demonstrate responsible AI governance.

If Canadian organizations, schools, institutions, and businesses voluntarily adopted the principles and obligations of the EU AI Act, significant operational, technical, legal, and ethical changes would be required. These changes would particularly affect AI-enabled security systems used in educational environments and public-facing institutions.

This article examines how AI-enabled security technologies are likely to be categorized under the EU AI Act, which applications may be prohibited, and how Canadian institutions would need to adapt if they chose to align themselves with the European regulatory model.

Understanding the EU AI Act’s Risk-Based Framework

The EU AI Act regulates AI according to the degree of risk a system poses to safety, fundamental rights, and society.

Under Annex III of the legislation, several AI applications relevant to security and surveillance are specifically classified as “high-risk.” These include:

• Remote biometric identification systems

• Biometric categorization systems

• Emotion recognition systems

• AI systems used in education and vocational training

• AI systems affecting access to essential services

• AI systems used in law enforcement or border management

The regulation imposes extensive obligations on providers and deployers of high-risk systems, including:

• Risk management processes

• Human oversight requirements

• Transparency obligations

• Accuracy and cybersecurity standards

• Technical documentation requirements

• Data governance obligations

• Continuous monitoring and post-market surveillance

The EU AI Act also prohibits several categories of AI practices considered incompatible with fundamental rights.

These prohibited uses include:

• Social scoring systems

• Certain manipulative or deceptive AI systems

• Exploitation of vulnerable populations

• Certain forms of predictive policing

• Emotion recognition in workplaces and educational institutions

• Real-time remote biometric identification in public spaces for law enforcement purposes, except under narrow legal exceptions

The legislation therefore distinguishes between systems that are permitted but heavily regulated and systems considered fundamentally unacceptable.

Why AI-Enabled Security Systems Are Likely to Be Classified as High-Risk

Access Control and Identity Verification

Modern access control systems increasingly rely on AI-driven facial recognition, fingerprint analysis, behavioural authentication, gait recognition, and identity verification platforms.

These systems are often marketed as improving convenience, reducing unauthorized access, and increasing operational efficiency. However, regulators are increasingly concerned about several issues:

• Bias and discrimination in biometric matching

• False positives and false negatives

• Unauthorized surveillance

• Mass collection of biometric data

• Lack of informed consent

• Function creep, where systems are later used for broader surveillance purposes

Under Annex III of the EU AI Act, remote biometric identification systems are specifically identified as high-risk technologies.

A facial recognition system used to control entry into a school, workplace, or public facility could therefore require extensive compliance measures, including:

• Human oversight

• Accuracy testing

• Risk assessments

• Bias mitigation

• Data governance controls

• Cybersecurity protections

• Auditability

Canadian organizations adopting EU AI Act standards would likely need to move away from passive acceptance of vendor claims and instead implement formal governance frameworks around biometric technologies.

Organizations would also need to justify whether biometric identification is necessary and proportionate compared to less intrusive alternatives such as smart cards, PIN systems, or multi-factor authentication.

CCTV Analytics and Behavioural Detection

AI-powered CCTV systems now extend far beyond traditional video recording.

Advanced systems can:

• Detect unusual movement patterns

• Identify loitering behaviour

• Analyze crowd behaviour

• Flag “suspicious” activity

• Recognize faces

• Infer emotional states

• Generate automated alerts

• Track individuals across multiple camera feeds

These capabilities create significant concerns around civil liberties and algorithmic bias.

Behavioural detection systems are especially controversial because the concept of “suspicious behaviour” is highly subjective. Academic researchers and civil liberties organizations have repeatedly warned that behavioural AI systems can disproportionately target racialized communities, neurodivergent individuals, students with disabilities, or individuals whose behaviour falls outside socially expected norms.

The EU AI Act reflects these concerns.

Emotion recognition systems used in educational institutions and workplaces are specifically identified as prohibited practices under Article 5 in many circumstances.

For schools and universities, this has major implications.

Some educational technology vendors have explored AI systems that monitor student attention, emotional engagement, stress levels, or behavioural indicators during classroom activities or online learning sessions. Under the EU framework, many of these applications would face substantial legal and ethical barriers.

Canadian educational institutions adopting EU AI Act principles would likely need to prohibit:

• AI systems that infer emotional states from facial expressions

• Classroom surveillance systems assessing student engagement through emotion analysis

• AI systems that score or profile students based on behavioural data

• Automated behavioural risk scoring platforms

This would significantly reshape the educational technology market.

Institutions would need stronger procurement standards, independent assessments, and ethics review processes before implementing AI-enabled monitoring tools.

Perimeter Protection and Automated Decision-Making

AI-enabled perimeter security systems increasingly integrate multiple technologies, including:

• Automated video analytics

• Thermal imaging

• Intrusion detection

• Drone surveillance

• Predictive alert systems

• Automated threat classification

Many modern systems use machine learning models to determine whether an event should trigger a security response.

The problem is that automated threat assessment systems can produce errors with serious consequences.

False alarms may lead to unnecessary interventions, while missed detections can undermine safety. More importantly, automated systems may replicate historical biases present in training data.

If organizations in Canada adopted EU AI Act standards, perimeter security systems using automated decision-making would likely require:

• Human review of automated alerts

• Clear escalation procedures

• Documented risk assessments

• Transparency regarding system limitations

• Independent testing for discriminatory outcomes

• Continuous monitoring for performance degradation

Educational institutions would face particular challenges because schools involve minors, who are considered vulnerable populations under many privacy and human rights frameworks.

The EU AI Act places strong emphasis on protecting vulnerable groups from manipulative or harmful AI systems.

As a result, Canadian schools adopting EU standards would likely need to limit the use of fully automated threat detection systems that make consequential decisions without human oversight.

Real-Time Biometric Identification and Prohibited AI Practices

One of the most controversial provisions of the EU AI Act concerns real-time remote biometric identification.

The legislation generally prohibits the use of real-time biometric identification systems in publicly accessible spaces for law enforcement purposes, except under narrowly defined exceptions.

The European Commission and supporting guidance documents identify major risks associated with these systems, including:

• Mass surveillance

• Chilling effects on democratic participation

• Misidentification

• Discriminatory impacts

• Violations of privacy and freedom of assembly

The debate surrounding facial recognition technology has intensified globally.

Researchers, human rights advocates, and privacy regulators have repeatedly documented concerns regarding:

• Racial bias in facial recognition systems

• Higher error rates for women and minorities

• Lack of transparency in algorithmic training

• Permanent biometric tracking of individuals

• Data retention and misuse risks

Several academic studies have argued that biometric surveillance fundamentally alters the relationship between individuals and public spaces.

If Canadian organizations adopted EU AI Act principles, the implications would be substantial.

Public institutions, schools, universities, transit systems, shopping centres, and municipalities would likely need to prohibit or severely restrict:

• Live facial recognition in public environments

• Continuous biometric tracking systems

• AI systems that categorize individuals by sensitive attributes

• Emotion recognition technologies

• Behavioural profiling systems

This would represent a major shift away from current trends in “smart surveillance.”

Organizations would also need explicit legal justification, documented proportionality assessments, and independent oversight mechanisms before deploying biometric technologies.

The Impact on Canadian Schools and Educational Institutions

Educational institutions would likely experience some of the most significant operational impacts if EU AI Act standards were adopted in Canada.

Student Privacy and Surveillance

Schools increasingly use AI-enabled systems for:

• Campus security

• Visitor management

• Attendance monitoring

• Online exam proctoring

• Behavioural analysis

• Threat assessment

• Student engagement monitoring

Many of these technologies raise profound ethical questions.

The EU AI Act specifically identifies emotion recognition systems in educational institutions as prohibited in many circumstances.

This reflects growing concern that AI systems should not infer emotional or psychological states from facial expressions, body language, voice patterns, or behavioural signals in learning environments.

Critics argue that such systems can create:

• Constant psychological monitoring

• Chilling effects on learning and participation

• Increased anxiety among students

• Bias against neurodivergent students

• Misinterpretation of cultural communication styles

• Reduced trust between students and institutions

Canadian schools adopting EU-style safeguards would likely need to redesign their approach to AI surveillance entirely.

Online Proctoring and Behavioural Monitoring

During the COVID-19 pandemic, many educational institutions adopted AI-based online proctoring systems.

These systems often monitored:

• Eye movement

• Facial positioning

• Voice patterns

• Background activity

• Keyboard behaviour

• Head movement

Some systems generated automated “suspicion scores” for students.

Privacy advocates and student groups criticized these technologies for:

• Invasive surveillance

• False accusations of cheating

• Discrimination against students with disabilities

• Algorithmic bias

• Lack of transparency

If EU AI Act principles were applied in Canada, many AI-driven proctoring systems would likely require extensive reassessment or removal.

Educational institutions would need to demonstrate:

• Necessity and proportionality

• Human oversight

• Non-discriminatory performance

• Transparent decision-making

• Appeal mechanisms

• Minimal data collection

Some systems may become economically or legally impractical under such standards.

Safeguarding Without Over-Surveillance

Schools still face legitimate safety concerns, including violence prevention, unauthorized access, vandalism, and emergency response.

The challenge is therefore not whether security should exist, but how it can be implemented without creating environments of constant algorithmic surveillance.

If Canadian schools adopted EU AI Act principles, institutions would likely need to move toward:

• Privacy-by-design security systems

• Minimal data retention practices

• Strong parental notification and consent frameworks

• Independent AI impact assessments

• Clear governance policies

• Human-centred security oversight

• Transparent procurement standards

This would likely slow the deployment of experimental AI surveillance technologies in education.

However, proponents argue that it would also protect students from becoming subjects of continuous biometric monitoring during critical developmental years.

Governance Changes Canadian Organizations Would Need to Implement

If organizations in Canada voluntarily aligned themselves with EU AI Act requirements, compliance would involve far more than simply purchasing compliant software.

The regulation requires comprehensive governance structures.

AI Risk Assessments

Organizations would need formal AI impact assessments before deployment.

These assessments would likely examine:

• Human rights implications

• Privacy risks

• Bias and discrimination risks

• Cybersecurity vulnerabilities

• Data quality issues

• Accuracy limitations

• Potential harms to vulnerable groups

Such assessments would become particularly important in schools, healthcare facilities, and public institutions.

Human Oversight Requirements

The EU AI Act repeatedly emphasizes that high-risk systems must not operate without meaningful human oversight.

Canadian organizations would therefore need:

• Trained oversight personnel

• Escalation procedures

• Human review of automated decisions

• Clear accountability structures

• Documentation of intervention protocols

This requirement alone could significantly alter how automated security systems are deployed.

Fully autonomous decision-making systems would become difficult to justify.

Procurement and Vendor Accountability

Organizations would also need more rigorous vendor due diligence.

Security vendors would likely be required to provide:

• Technical documentation

• Accuracy testing data

• Bias assessment results

• Transparency regarding training data

• Cybersecurity certifications

• Ongoing monitoring mechanisms

Schools and institutions could no longer rely solely on vendor marketing claims.

Procurement teams would need specialized expertise in AI governance, privacy law, and cybersecurity.

Transparency and Public Trust

One of the central goals of the EU AI Act is increasing public trust in AI systems.

Canadian organizations adopting similar standards would likely need to improve transparency by:

• Clearly disclosing AI use

• Publishing governance policies

• Explaining automated decision-making processes

• Providing complaint and appeal channels

• Conducting public consultations

For educational institutions, transparency would become especially important.

Parents, students, faculty, and communities would increasingly expect visibility into how AI systems are used and how personal data is processed.

Broader Societal and Ethical Implications

The debate surrounding AI-enabled security systems is ultimately about more than technology. It concerns the balance between safety, efficiency, privacy, autonomy, and democratic freedoms. Critics of expansive AI surveillance argue that constant monitoring can normalize:

• Mass data collection

• Behavioural profiling

• Predictive risk scoring

• Loss of anonymity in public spaces

• Reduced freedom of expression

Supporters of AI security systems argue that these technologies can improve safety, reduce crime, accelerate emergency response, and enhance operational efficiency.

The EU AI Act attempts to navigate these competing interests by allowing some uses under strict safeguards while prohibiting others entirely.

If Canadian institutions adopted these principles, organizations would likely face difficult questions:

• How much surveillance is proportionate?

• Should students be continuously monitored by AI systems?

• Can emotion recognition ever be reliable enough for educational settings?

• Who is accountable when automated systems make harmful errors?

• How should vulnerable populations be protected?

• Can public trust exist without transparency?

These questions are likely to define the next decade of AI governance.

Conclusion

The EU AI Act represents a historic shift in the regulation of artificial intelligence.

AI-enabled security technologies; particularly those involving biometric identification, behavioural analytics, emotion recognition, and automated surveillance, are among the systems most directly affected by the legislation.

Many of these systems are likely to be classified as high-risk under the EU framework, while certain applications are prohibited altogether.

Should Canadian organizations, institutions, schools, and businesses voluntarily adopted EU AI Act principles, the implications would be profound.

Security technologies would require:

• Stronger governance

• Independent oversight

• Human-centred decision-making

• Transparency obligations

• Privacy-by-design architecture

• Formal risk assessments

• Accountability mechanisms

Educational institutions would face particularly significant changes.

AI systems that monitor emotional states, behaviour, engagement, or student conduct would likely face major restrictions or outright prohibition under EU-style standards.

At the same time, schools and organizations would still need to maintain safe environments.

The central challenge will therefore be finding a balance between legitimate security needs and the protection of fundamental rights.

The EU AI Act does not eliminate AI-enabled security systems. Instead, it establishes the principle that technologies capable of affecting rights, freedoms, and human dignity must be subject to rigorous oversight.

As governments around the world consider their own AI governance frameworks, the European model is likely to influence global regulatory expectations far beyond Europe itself.

For Canadian institutions, the question may no longer be whether AI governance standards will evolve, but how quickly organizations can adapt to a future in which responsible AI deployment becomes both a legal expectation and a societal demand.

References

1. European Commission. “Commission publishes the Guidelines on prohibited artificial intelligence (AI) practices, as defined by the AI Act.” https://digital-strategy.ec.europa.eu/en/library/commission-publishes-guidelines-prohibited-artificial-intelligence-ai-practices-defined-ai-act

2. Council of the European Union. “Artificial intelligence act.” https://www.consilium.europa.eu/en/policies/artificial-intelligence-act/

3. European Parliament. “EU AI Act: first regulation on artificial intelligence.” https://www.europarl.europa.eu/topics/en/article/20230601STO93804/the-ai-act-eu-rules-to-regulate-artificial-intelligence

4. European Commission AI Act Service Desk. “Article 5: Prohibited AI practices.” https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-5

5. European Commission. “Navigating the AI Act.” https://digital-strategy.ec.europa.eu/en/faqs/navigating-ai-act

6. European Commission AI Act Service Desk. “Annex III.” https://ai-act-service-desk.ec.europa.eu/en/ai-act/annex-3

7. Reuters. “EU lays out guidelines on misuse of AI by employers, websites and police.” February 4, 2025. https://www.reuters.com/technology/artificial-intelligence/eu-lays-out-guidelines-misuse-ai-by-employers-websites-police-2025-02-04/

8. Le Monde. “First measures of European AI Act regulation take effect.” February 2, 2025. https://www.lemonde.fr/en/pixels/article/2025/02/02/artificial-intelligence-the-first-measures-of-the-european-ai-act-regulation-take-effect_6737691_13.html

9. Wired. “The EU Just Passed Sweeping New Rules to Regulate AI.” https://www.wired.com/story/eu-ai-act

10. The Guardian. “What will the EU’s proposed act to regulate AI mean for consumers?” https://www.theguardian.com/technology/2024/mar/14/what-will-eu-proposed-regulation-ai-mean-consumers

11. Sousa e Silva, Nuno. “The Artificial Intelligence Act: critical overview.” arXiv. https://arxiv.org/abs/2409.00264

12. Ho-Dac, Marion. “First Analysis of the EU Artificial Intelligence Act: Towards a Global Standard for Trustworthy AI?” arXiv. https://arxiv.org/abs/2408.08318

13. Kieslich, Kimon and Marco Lünich. “Regulating AI-Based Remote Biometric Identification. Investigating the Public Demand for Bans, Audits, and Public Database Registrations.” arXiv. https://arxiv.org/abs/2401.13605

14. Genicot, Nathan. “Scoring the European Citizen in the AI Era.” arXiv. https://arxiv.org/abs/2505.02791

15. Reddit discussion. “EU draft legislation will ban AI for mass biometric surveillance and predictive policing.” https://www.reddit.com/r/privacy/comments/13fftuz/

16. Reddit discussion. “AI systems with ‘unacceptable risk’ are now banned in the EU.” https://www.reddit.com/r/neoliberal/comments/1igmfoj/

17. Reddit discussion. “EU AI Act enforcement hits August 2026 — what are mid-market companies actually doing to prepare?” https://www.reddit.com/r/AI_Governance/comments/1sjt3ft/eu_ai_act_enforcement_hits_august_2_how_are/

18. Reddit discussion. “EU AI Act high-risk obligations hit August 2. How are EU tech companies handling classification?” https://www.reddit.com/r/eutech/comments/1sr0z98/eu_ai_act_highrisk_obligations_hit_august_2_how/

19. Official EU AI Act information portal. https://artificialintelligenceact.eu/

Minaz Jivraj MSc., C.P.P., C.F.E., C.F.E.I., C.C.F.I.-C., I.C.P.S., C.C.T.P.

Disclaimer:The information provided in this blog/article is for general informational purposes only and reflects the personal opinions of the author. It is not intended as legal advice and should not be relied upon as such. While every effort has been made to ensure the accuracy of the content, the author makes no representations or warranties about its completeness or suitability for any particular purpose. Readers are encouraged to seek professional legal advice specific to their situation.