The AI Act – Europe leads the charge on AI regulation

Unacceptable risk

AI systems with an “unacceptable” level of risk to safety or human rights are strictly prohibited. The list of prohibited systems was extensively negotiated between the institutions of the EU last year. In the finalised text, it includes:

• “real-time remote biometric identification systems in publicly accessible spaces” – which would capture live facial recognition systems. Interestingly, in a significant amendment to the draft approved in May 2023, the scope of that ban is now confined to use “for the purposes of law enforcement” (subject to various exceptions such as searching for victims of abduction, preventing terrorist attacks, etc.) Other uses of such systems are not prohibited outright but are instead classified as high-risk, as discussed further below;
  
  
• systems that create or expand facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footage;
  
  
• systems that “infer emotions” in the workplace or educational institutions (other than for “medical or safety reasons”); and
  
  
• “biometric categorisation systems” used to categorise individuals and draw inferences about their personal characteristics (subject to a specific exception for law enforcement).

Some of the other categories reflect the fears of dystopian misuse that underpinned the original draft legislation in 2021 – including social scoring systems, systems that predict the likelihood of a person committing a criminal offence, or systems that deploy “subliminal techniques” to distort behaviour (an example given is “machine-brain interfaces").

High risk

Systems that create a “high risk” to health and safety or human rights are not banned, but are subject to various quality management obligations including strict “conformity assessments” (to assess conformity with existing laws where applicable, and to confirm compliance with various other safeguards set out in the Act). The majority of these obligations apply to the “providers” (i.e. the developers) of high-risk AI systems. 

High risk systems also include systems designed for certain designated uses. The list is lengthy but includes, by way of example, systems used for:

• determining access to educational and vocational training or for the recruitment, evaluation or promotion of employees;
  
  
• credit applications or credit scoring (subject to a helpful new exception for systems used to detect financial fraud);
  
  
• risk assessment and pricing in relation to life and health insurance;
  
  
• certain law enforcement purposes, for example to evaluate the reliability of evidence; and
  
  
• biometric identification (other than for law enforcement purposes, which are generally prohibited as noted above). This is subject to certain exceptions including where systems are used to verify the identity of a “specific natural person.”

Low / minimal risk

For lower risk activities, the AI Act imposes more general requirements, such as transparency obligations. That requires that AI systems should make clear to users that they are interacting with an AI system (unless that is obvious based on the context), and any limitations on the system’s capabilities should be made clear.

Providers of all AI systems are encouraged under the AI Act to adhere voluntarily to codes of conduct. These are expected to set out requirements related to a range of factors including sustainability, accessibility for persons with a disability, and diversity of development teams.