THE EUROPEAN AI ACT
Securing democracy’s primacy over technology
by Paul Nemitz
Principal Advisor at the EU Commission
Artificial intelligence (AI), much like nuclear power, has the potential to transcend borders, bringing both enormous benefits and significant risks. As Swedish philosopher Nick Bostrom of Oxford University warns, AI could be humanity’s last invention if it leads to catastrophic outcomes. Yet, if developed and used responsibly, AI can serve the public interest and benefit all of humanity. This requires stringent regulations to ensure safe and beneficial development, something the market alone cannot guarantee. Recognising this, the G7 leaders even invited the Pope to their meeting in Italy to discuss AI. On the 57th World Day of Peace, January 1, 2024, the Pope addressed the global community on “Artificial Intelligence and Peace.”
Download as PDF
The European Union, with the AI Act coming into force in August 2024, acknowledges that democracy cannot rely solely on the ethics or self-regulation of private tech giants like Google, Amazon, Facebook, Apple, Microsoft (GAFAM), Elon Musk’s ventures, or Chinese companies like Tencent, Alibaba, and Baidu to establish AI rules. Democracies share a common interest in agreeing on rules and institutions for the international governance of AI, not only when strong AI surpasses human intelligence across multiple domains, as posited by Howard Gardner1.
The EU’s AI Act is the world’s first legal regulation for AI, following the precedent set by the 2016 General Data Protection Regulation (GDPR)2. These instruments demonstrate that innovation occurs not just in technology but also within the political system of democracy. Technical innovations that pose risks to individual rights, democracy, and state sovereignty must be accompanied by democratic, innovative, and binding regulations. Only the law, with its binding nature and enforceability, can guarantee a level playing field for competition and the protection of fundamental rights. Ethics, self-regulation, and non-binding political declarations cannot achieve this.
Without the AI Regulation, Europe could have faced 27 different national AI regulations. A unified European law is thus a programme of less regulation, serving business interests by reducing regulatory fragmentation.
Negotiations on a multilateral agreement for AI regulation are nearing completion in the Council of Europe3, with participation from the USA and many other non-European countries. While non-European states can join Council of Europe conventions, the direct protective effect of the AI Convention will be limited due to exclusions of key areas and the lengthy implementation process in member states.
It’s essential to remember that technological innovation and its regulation by law, along with supervision by public authorities, are not unusual. This parallel is evident in the regulation of nuclear power, which saw international and national supervisory authorities established before commercial nuclear reactors were connected to the grid. The International Atomic Energy Agency (IAEA) was founded on July 29, 1957, with the ratification of its statute by President Eisenhower.
Within a decade of producing electricity from an experimental reactor in the USA, legal and administrative structures were in place to protect against nuclear risks. Similarly, the European Commission and the AI Office, established by Commission Decision on January 24, 2024, bear great responsibility. The AI Office within the Commission will be a centre of excellence for enforcing the AI Regulation, protecting fundamental rights and democracy in the technological age. The knowledge gained here and across the AI Act’s governance system must be utilised for democratic control of technological power.
The implementation of the AI Regulation will present European and national courts with challenging interpretation questions. Courts must consider the power and information imbalance between citizens and AI actors when interpreting the law. The introduction of a fundamental rights impact assessment in the AI Regulation underscores its focus on protecting fundamental rights and democracy. This new law, designed to safeguard democracy and fundamental rights in technology, presents new tasks for courts and engineers alike.
We need engineers who, in the spirit of Eugen Kogon4 and Hans Jonas5, take responsibility for the consequences of their AI developments and actively contribute to creating human-centred AI that respects and strengthens fundamental rights and democracy. Additionally, courts must guide and contain new AI technology through law, considering constitutional principles and the EU Charter of Fundamental Rights6.
A critical question in the age of AI is whether individuals have enforceable rights against those who develop, market, and use AI systems, as well as against supervisory authorities. In data protection, this is undoubtedly the case, as confirmed by the European Court of Justice (ECJ) Schufa7 ruling on December 7, 2023. A similar discussion is now emerging regarding the AI Act. The European Parliament strengthened the AI Act’s fundamental rights orientation by adding a fundamental rights impact assessment. It remains to be seen if individuals can hold authorities accountable for failing to implement the AI Act.
Other authorities, such as consumer protection or data protection agencies, might also enforce the AI Act more effectively. The ECJ’s Bundeskartellamt/Facebook ruling on July 4, 20238, has paved the way for modernising the enforcement of economic administrative law. In a world dominated by AI and multi-sided platforms, we need a platformisation of enforcement, where authorities assess facts holistically and consider various legal bases.
Democracy requires openness to change, reflected in electoral outcomes. This means AI systems used for administrative tasks must be evaluated for their adaptability to democratic changes. If humans can implement necessary changes more flexibly and cost-effectively, AI should not be used. AI, working with data from today and yesterday, lacks the critical attitude towards the present, dissatisfaction with routine, and the imagination for a better future—qualities essential for democracy and innovation. Philosopher Kant’s insight that one cannot derive what should be from what is remains a guiding principle in evaluating AI’s potential.
1
Howard Gardner is an American psychologist and educator best known for his theory of multiple intelligences, which he introduced in his book Frames of Mind (1983). The core idea of Frames of Mind is that intelligence is not a single, unified ability but rather a set of distinct types, including linguistic, logical-mathematical, spatial, musical, and interpersonal intelligences, among others. This theory challenges traditional views on intelligence and has significantly influenced education practices.
2
The 2016 General Data Protection Regulation (GDPR) is an EU law that sets strict guidelines on how personal data is collected, stored, and processed. It aims to protect individuals’ privacy and data rights, with significant penalties for non-compliance, and applies to all organizations operating within the EU or handling EU citizens’ data.
3
The Council of Europe Framework Convention on Artificial Intelligence focused on ensuring that AI respects human rights, democracy, and the rule of law. It was adopted on 17 May 2024 by the Committee of Ministers of the Council of Europe in Strasbourg, and will be opened for signature on the occasion of the Conference of Ministers of Justice in Vilnius (Lithuania) on 5 September 2024.
4
Eugen Kogon was a German political scientist, sociologist, and Holocaust survivor, known for his work on the analysis of totalitarian regimes and his influential book on Nazi concentration camps, The Theory and Practice of Hell. He was also a prominent advocate for European integration.
5
Hans Jonas was a German philosopher known for his work in ethics, particularly in bioethics and environmental philosophy. His most influential work, The Imperative of Responsibility, argues for a moral duty to protect future generations in the face of technological advancements.
6
The EU is currently considering how to adapt the interpretation of the Charter of Fundamental Rights, which enshrines the fundamental rights and freedoms of individuals in the European Union, to the challenges and opportunities posed by AI. While no formal amendments have been made, there is ongoing debate on how to ensure that AI technologies respect fundamental rights like privacy, non-discrimination, and fairness. The European Commission and other bodies are working on guidelines and potential regulations to ensure that AI development aligns with the Charter’s principles.
7
The European Court of Justice (ECJ) issued a significant ruling on December 7, 2023, regarding SCHUFA, a major German credit rating agency. The court found that SCHUFA’s practice of generating credit scores based on automated data processing could be considered a form of “automated individual decision-making” under Article 22 of the General Data Protection Regulation (GDPR). This type of decision-making is generally prohibited if it significantly affects the individual, as it does in the case of credit scoring, where a low score can lead to the denial of credit.
8
The European Court of Justice (ECJ) issued a landmark ruling on July 4, 2023, in the case between Meta (formerly Facebook) and the German Federal Cartel Office (Bundeskartellamt). The court confirmed that national competition authorities, like the Bundeskartellamt, can assess compliance with the General Data Protection Regulation (GDPR) as part of their investigations into abuse of dominance under competition law. This decision underscores that violations of data protection laws can be considered as evidence of anti-competitive behavior, particularly in the digital economy where access to and processing of personal data are critical factors.