European AI Act: What It Is, Why It Matters, & What to Do About It

Agentic AI Payment fraud

On Mar 13, 2024, the European Parliament voted to pass the world’s first comprehensive legal framework for the regulation of artificial intelligence: the Artificial Intelligence Act (otherwise known as the “AI Act”).

The increasing use of artificial intelligence (AI) tools, and the deployment and wide use of generative AIs such as ChatGPT, have raised concerns about their ethical, legal, and societal implications. In this context, this new European regulation aims to regulate the design and use of AI. Its objective is to strengthen trust around AI and control its impact on society, businesses, and individuals (and in particular on the fundamental rights of the latter), while creating a context favorable to research and development, the economy, and innovation.

The AI Act was entered into force on Aug 1, 2024—which is the date that determines when compliance to different provisions of the Act will be enforced.

What is the AI Act?

The regulation defines an artificial intelligence system (AIS or AI System(s)) in order to distinguish them from simpler software systems. Article 3(1) of the AI Act defines an AI System as “a machine-based system designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments”.

This risk-based approach is one of the fundamentals, and one of the original aspects of this regulation, establishing progressively increasing restrictions on uses of the AI that are deemed to produce growing risks.

AI Systems Categories

The different AI Systems have been categorized into four types according to their potential levels of danger, each with their own levels of associated obligations to comply with the AI Act:

Minimal risk or no risk AI Systems (e.g. spam filters)
Limited-risk AI Systems (e.g. chatbots)
High-risk AI Systems (e.g. recruitment)
Prohibited AI Systems (e.g. social scoring)

Minimal Risk or No Risk AI Systems

This category covers, for example, AIS that perform predictive maintenance. In this case, their deployment and use are permitted without further restrictions or obligations.

Limited-Risk AI Systems

AI Systems that interact with humans (e.g. chatbots) and systems that generate or manipulate content (e.g. spam filters) enter this category. As these AIS are likely to interact with the public, they will be subject to limited information and transparency obligations (in particular by the mention made to the user that their output is “generated by artificial intelligence”).

High-Risk AI Systems

The rules for characterizing high-risk AI systems are defined in Article 6 of the regulation. In summary, high-risk systems:

Meet both of these conditions:
- The system is intended to be used as a safety component of a product, or is itself a product, and is covered by the Union harmonization legislation listed in Annex I.
- The system or the product it’s a part of is required to undergo third-party conformity assessment.
Or are those referred to in Annex III, which covers:
- Biometric systems (emotion recognition, biometric identification systems, categorization, etc.).
- Critical infrastructures (e.g. road traffic, supply of water or gas, etc.)
- Educational or vocational training (systems used to determine access or admission, learning evaluation systems, etc.).
- Employment, workers management, and access to self-employment.
- Access to and enjoyment of essential private services and essential public services and benefits (e.g. systems used to evaluate the creditworthiness of natural persons or establish their credit score, or systems used by public authorities or on behalf of public authorities to evaluate the eligibility for healthcare services).
- Law enforcement.
- Migration, asylum and border control management.
- Administration of justice and democratic processes.

High-Risk AIS Exemptions

AI Systems that don’t pose significant risk of harm to health, safety, or fundamental human rights aren’t considered high-risk, as defined by fulfilling one of these criteria:

They are performing narrow procedural tasks.
They are making improvements to the results of previously completed human activities.
The AI Systems are detecting decision-making patterns without replacing human assessments.
They are intended to be mere preparatory tasks to a risk-assessment.

On the other hand, AI systems that perform profiling are always considered high-risk.

High-risk AI systems will be subject to a set of strict requirements which could include the obligation of documentation, the implementation of compliance measures guaranteeing in particular the monitoring of biases, the robustness and cybersecurity of the system, its consistency with data privacy standards, human oversight as well as technical documentation, and registration of the system in the EU Database.

Prohibited AI Systems

This category of AI Systems is entirely prohibited, and includes the following AI practices:

Human behavior manipulation.
AIS that exploit any vulnerabilities of a person or specific group of persons due to their age, disability or specific social or economic situation.
Biometric categorization systems categorizing natural persons to deduce or infer their race, political opinions, trade union membership, religious or philosophical beliefs, sex life, or sexual orientation.
Classification of natural persons or groups based on their social behavior.
Risk assessments of natural persons in order to assess or predict the risk of a natural person to commit a criminal offense, based solely on the profiling of a natural person or on assessing their personality traits and characteristics.
Real-time remote biometric identification systems (subject to exceptions).

Law enforcement authorities will be allowed to use real-time remote biometric identification systems in public-accessible spaces under very specific conditions, most relating to the prevention of criminal harm to others. This use will also be strictly supervised.

General Purpose AI Systems

The AI Act creates a specific category for “general purpose AI models” (GPAI), which includes AI models trained with large amounts of data that “displays significant generality and is capable to competently perform a wide range of distinct tasks regardless of the way the model is placed on the market and that can be integrated into a variety of downstream systems or applications”. Essentially, AI systems like ChatGPT would be included in this category, as it was built as a general information system.

Providers of GPAI models have certain obligations:

Draw up, maintain, and make available to the public technical documentation of the model, including its training and testing process and the results of its evaluation.
Draw up, maintain, and make available information and documentation to providers of AI Systems who intend to integrate the general-purpose AI model in their AI System.
Put in place a policy to respect the European Union’s copyright law.

Furthermore, these GPAIs can potentially be classified as “General Purpose AI Models with Systemic Risk” if they meet either of these two criteria: it has high impact capabilities and/or when the cumulative amount of compute used for its training exceeds a certain threshold.

Thus, the GPAIs considered a systemic risk are subject to a series of additional obligations such as:

Performing model evaluation with a view to identify and mitigate systemic risks.
Assess and mitigate at the level of the European Union possible systemic risks that may stem from the development, placing on the market, or use of GPAI models with systemic risk.
Monitor, document and report without undue delay to the AI Office (the European Commission) and, as appropriate, to national competent authorities, relevant information about serious incidents and possible corrective measures to address them.
Ensure an adequate level of cybersecurity protection for the GPAI model with systemic risk and the physical infrastructure of the model.

Who does the Act apply to?

The AI Act will apply to all “deployers”, distributors, importers, manufacturers, and suppliers of AI Systems who have a connection with the EU. Basically, these actors (natural or legal persons) will be subject to the AI Act if they have a stable establishment on European territory or market products or services on the European market. This is a very broad scope (very similar to the scope of the GDPR for example) and effectively means that most entities trading on a global scale will be subject to it.

In terms of territorial application, this regulation is in line with the major compliance regulations created by the European Union over the last two decades; Like the General Data Protection Regulation (“GDPR”), it is of extraterritorial application. It will have a global reach and, thus, create new obligations on organizations across sectors and throughout the provision chain that markets and uses AI Systems in the European Union.

How will the Act be enforced?

On January 24, 2024, the European Commission designated a European Office for Artificial Intelligence. Its role is to support and ensure the proper implementation of the AI Act. By proceeding in this way, the EU wanted to ensure coordinated implementation at European level of the future regulation.

The tasks of this office will include:

Contributing to a strategic, coherent and effective European Union approach to international initiatives on AI in coordination with Member States and in line with the European Union’s positions and policies;
Cooperating with all relevant bodies, offices and agencies of the European Union;
Cooperating with authorities and bodies of the Member States on behalf of the European Commission.

Regarding the designation of national AI control authorities, each Member State shall establish or designate at least one notifying authority (also called single point of contact) and at least one market surveillance authority (also called national competent authority) for the purpose of this regulation as national competent authorities. These national competent authorities shall ensure the application and implementation of this regulation at a local scale.

Fines

AI actors who fail to comply with the obligations provided for by the AI Act will be exposed to fines, the amount of which may vary (in particular depending on the type of AIS involved). Thus, the text distinguishes in particular breaches involving prohibited AI Systems from other AI Systems, and provides for a specific sanctions regime for GPAIs.

The most serious sanctions for non-compliance amount to:

35,000,000 euros; or
7% of total worldwide annual turnover for the preceding financial year for legal entities, whichever is higher.

Implementation Timeline

The AI Act has been entered into force as of Aug 1, 2024, and its final requirements and obligations will take effect in phases. This implementation will happen gradually (being fully applied 36 months after its entry into force), depending on the level of risk from the day of its publication in the Official Journal of the European Union.

Subjects to the AI Act will have:

6 months (Feb 2, 2025) to comply with the rules on prohibited AI Systems;
12 months (Aug 2, 2025) to implement compliance with GPAI requirements;
24 months (Aug 2, 2026) to implement global compliance with AI Systems;
36 months (Aug 2, 2027) to comply with the rules for high-risk AI Systems.

How to Anticipate Business Changes for the Act

So what does the EU AI Act mean for your business?

Entities that are potentially impacted by this new regulation should consider actions focused on anticipating the AI Act, as well as other artificial intelligence regulations that are likely to follow. Here are some suggested actions to take:

1. Audit your organization’s AI structure.

This first step should include:

A mapping of both internal and external AI systems and models (purpose, data processed, contractual framework) used within the company.
Identifying the various organizational frameworks covering their use (decision-making procedure, etc), their current impact, and monitoring through the prism of the key functions within the entity: purchase, security, IT, legal, compliance, etc.

2. Reassess this structure in light of the AI Act.

This action should include:

Creating an inventory and a monitoring framework for the identified internal and external AI systems and models.
Evaluating how each AI System could be classified under the AI Act’s risk classification.
Assessing the needs in terms of governance and monitoring, reporting, or other compliance requirements that might apply to each use—including whether the use might be prohibited.

3. Dedicate a team to monitor & strategize about AI problematics.

AI will be a key concern for public authorities and media during the upcoming decade; consequently, it will be a major issue for companies. Companies will have to approach AI’s legal framework globally to ensure compliance with multiple complex regulations across jurisdictions.

Managing these AI-related risks and the ethics and compliance issues they will raise in businesses will be necessary for maintaining trust with customers.

Organizations will need to create a consistent strategy to monitor and anticipate these new risks and concerns. To achieve this objective and to secure compliance with the AI Act, organizations will have to seek cross-functional competences.

A centralized, dedicated team is the most efficient way to determine and provide a global, scalable strategy. This team will be necessary to establish a consistent, efficient framework adapted to control and manage emerging AI risks and compliance requirements.

Conclusion

Adaptation to the legal framework for AI is a new pivotal moment, as was adaptation to data privacy during the previous decade. Organizations can anticipate it—and even make it a positive business element and differentiator from their competition—or suffer from it. As is often the case, the difference will quickly be noticeable between the companies that anticipated the change and those that will suffer from it.

DataDome is working hard to ensure full compliance with the EU AI Act and other AI legislation that may follow.