According to the regulation for artificial intelligence (AI) passed by the EU Parliament on Wednesday, operators of basic models such as GPT from OpenAI, Gemini from Google or LLaMA from Meta must examine and, if necessary, mitigate foreseeable systemic risks. It concerns, for example, the areas of health, safety, fundamental rights and the environment. The MPs were able to enforce particularly strict obligations for “foundation models” with general purpose and “high efficiency”. They also have to carry out tests with enemy attacks, report serious incidents to the EU Commission and report on their energy efficiency.
Advertisement
According to Article 51 of the AI Act, the Brussels government institution should, among other things, use the computing power used for training as a quantitative threshold for the classification of such models. High effectiveness can therefore be assumed above all if “the cumulative amount” of calculations used for training is more than 10 to the power of 25 (10^25) floating point operations, measured in floating point operations per second (flops). But Dragoş Tudorache, parliamentary co-rapporteur for the AI law, has admitted that this requirement will soon become obsolete.
By the time the rules come into force in around 12 months, there could be four or five large base models that exceed this threshold, the negotiator told the Euractiv portal. But a new technological leap is also possible, which will massively reduce the computing requirements for powerful foundation models. Currently only OpenAI with ChatGPT and Google with Gemini would probably have to meet the very high requirements. The computing power required to train the current OpenAI model GPT-4 is estimated to be just over 10^25 flops. A cluster with around 10^18 flops is said to have been used for the predecessor GPT-3. For comparison: The performance of the Nvidia server DGX H100, which is specifically designed for AI purposes, is around 10^16 flops.
Limit value is controversial, additional criteria should apply
The flop threshold “confuses computing power with risk,” which are two different things, Sandra Wachter, professor of technology and regulation at the Oxford Internet Institute, criticized Euractiv. Regardless of their size, these models carried all sorts of risks in terms of bias, misinformation, privacy, and hallucinations. The part of the AI law on basic models is the result of massive lobbying by European players such as Aleph Alpha and Mistral, added Merve Hickok, President of the Center for AI and Digital Policy. At the same time, engineers in Silicon Valley are working hard to reduce the immense computing effort required for AI training for cost reasons alone. Regulatory exemptions generally apply to open source models.
The 10^25 flops are not set in stone. Recital 111 of the Regulation states: “This threshold should be adjusted over time to take into account technological and industrial changes, such as algorithmic improvements or increased hardware efficiency, and should be supplemented by benchmarks and indicators of model capability.” But a revision is unlikely to be easy after the co-legislators only managed to agree on a common text after enormous efforts. According to an annex, the Commission should also take into account other criteria when making the classification, such as the number of parameters of the model, the quality or size of the data set, the ability to learn new, different tasks and the number of users.
(mki)