World experts create document on AI capabilities and risks

Director of the Institute of Mathematical and Computer Sciences at the University of São Paulo collaborates with work developed in the UK to guide governments.

Agência FAPESP – Professor André Carlos Ponce de Leon Ferreira de Carvalho, Director of the Institute of Mathematical and Computer Sciences at the University of São Paulo (ICMC-USP) in São Carlos, Brazil, has been invited to participate in an international advisory committee formed by the British government to prepare a report on the capabilities and risks of artificial intelligence (AI).

Carvalho coordinates the Center for Applied Research in Artificial Intelligence Recreating Environments (ARC-IARA), supported by FAPESP and the Ministry of Science, Technology and Innovation (MCTI). He is also a principal researcher at the Center for Mathematical Sciences Applied to Industry (CeMEAI), a FAPESP Research, Innovation and Dissemination Center (RIDC) based at ICMC-USP.

According to him, in November 2023, the UK organized the first global summit on the safety of artificial intelligence at Bletchley Park, where in the Second World War, a team led by Alan Turing, one of the pioneers of AI, cracked the code of the German Enigma machine.

“The summit was attended by representatives of national governments and the European Union, business, academia and civil society, including the Prime Minister of the United Kingdom, Rishi Sunak, the Vice President of the United States, Kamala Harris, and the President of the European Commission, Ursula von der Leyen,” Carvalho told the ICMC-USP Press Office.

The event discussed the risks of the latest and most powerful AI systems, frontier AI, and participants received a report entitled “AI Safety Summit – Capabilities and Risks from Frontier AI,” which outlined the capabilities and risks of frontier AI and the need to support research on the topic.

The report was reviewed by a committee of experts coordinated by Professor Yoshua Bengio of the University of Montreal (Canada). Along with Professors Geoffrey Hinton and Yann LeCun, Bengio was presented with the 2018 Turing Award, the equivalent of the Nobel Prize for computing. The three are known as the “Godfathers of AI.”

Carvalho highlighted an excerpt from the report: “We are in the midst of a technological revolution that will bring major changes to the way we live, work and relate to each other. AI could transform almost every aspect of our economy and society.”

“To seize the opportunities, we must understand and address the risks. AI poses risks in ways that do not respect national borders. It is important that governments, universities, businesses and civil society work together to address these challenges, which are complex and difficult to predict, to mitigate the potential dangers and ensure the benefits of AI for society as a whole,” another excerpt points out.

According to the ICMC director, the power of this transformative technology, with its increasingly sophisticated and complex techniques, is no longer debatable. For him, this futuristic scenario, like a science fiction movie script, has already arrived.

What cannot yet be predicted are the consequences of the misuse of technology and the emergence of AI that is more intelligent than human beings. “That’s why it’s so urgent to discuss and produce documents like this, written by AI experts. It’s also been agreed that we’ll continue to produce documents with information that can help governments prepare, providing them with the necessary knowledge, according to the state of the art, about what could happen in this area, especially its risks,” he emphasized.

The full report is available at: assets.publishing.service.gov.uk/media/65395abae6c968000daa9b25/frontier-ai-capabilities-risks-report.pdf.