Artificial intelligence for more efficient public spending audits

A startup supported by FAPESP has developed software to process legal cases, winning an important order from Brazil’s federal court of auditors.

FAPESP Innovative R&D* – Based at the Science and Technology Park of the University of Campinas (UNICAMP) in São Paulo state, Brazil, NeuralMind is a startup that focuses on artificial intelligence (AI) for automation of operations, compliance and fraud detection. In partnership with Terranova, a statistics consultancy that specializes in jurimetrics, NeuralMind has developed innovative software capable of transforming public spending oversight. The two firms were selected in a public call for proposals to supply the solution to Tribunal de Contas da União (TCU), Brazil’s federal court of auditors.

Courts around the world are using AI to support judicial decisions. In Brazil, the TCU uses a tender analysis platform called ALICE (short for Análise de LICitações e Editais), to oversee governmental procurement processes. In 2021, it avoided more than BRL 504 million in losses to public coffers.

The new software solution to be supplied by NeuralMind will support another part of the TCU’s work: analysis of evidence in the 2,000 new complaints and similar cases, mostly relating to public accounts, that come before the court every year. Such proceedings correspond to more than 40% of all the cases judged by the court, which says the number is steadily rising.

“Being selected by the TCU reflects the unique quality of NeuralMind’s solutions. The firm is prepared to mitigate the risks of a project like this one. We’re honored to participate in this innovative initiative in collaboration with an authority that’s so important to Brazilian society,” said Roberto Lotufo, a professor at the School of Electrical and Computer Engineering (FEEC-UNICAMP) and a co-founder of NeuralMind.

The project is supported by the FAPESP Innovative Research in Small Business Program (PIPE). 

Technological order

The technological route chosen by the TCU proposes a generalist language model that can be used to perform several tasks. The product does not exist commercially yet, but far exceeds the capabilities of current state-of-the-art jurimetrics and legal drafting software and was commissioned via a technological order, a special type of public procurement proceeding established by Brazil’s federal innovation law (Lei de Inovação).

In this instance, the technological order remunerates the startup’s research and development (R&D) efforts even if they do not result in a viable product, because “it cannot be foreseen whether the technological solution requested is feasible or whether the desired minimum performance will be achieved”, in the words of the TCU, citing what it calls innovation-associated technological risk. 

The project is considered ambitious. The functionality called for by the order includes detecting meanings in registered case files. The “machine” has to be able to identify the alleged irregularities in initial complaints, for example. In a second phase, the model will have to be able to classify procedural elements such as admissibility criteria for precautionary measures or precedents, among others, and automatically create a quantitative jurimetrics dashboard.

Eventually, it will have to be able to generate texts such as summaries of lengthy pleadings and responses to court notices expressing logically and legally suitable interpretations.

“The goal isn’t to take the place of human beings but to get the court’s work done faster so as to increase human productivity. The really exhausting and repetitive work can be done by computers,” Lotufo said.

One of the challenges of the technology is estimating the trustworthiness of the solutions presented by the algorithm. “To ensure the ‘machine’ makes logical connections, we ask it to explain the reasons for its choices. The stronger the grounds for its decisions, the greater its trustworthiness,” he added.

The TCU’s selection committee noted that the solution, when ready, will be “less costly” and adaptable to other situations, so that it can be shared with other bodies, including audit courts in states and municipalities.

The project 

The project AI-Assisted Investigation (Instrução Assistida por Inteligência Artificial, or INA2) proposed by NeuralMind and Terranova stood out among the 18 proposals submitted by Brazilian companies and science and technology institutions. The experience accumulated by Lotufo and his business partner Rodrigo Nogueira, also a professor at UNICAMP who specializes in advanced AI techniques, was important in guiding definition of the architectural design for the solution.

“We’re based in UNICAMP’s Science and Technology Park, which gives us a strong and direct relationship with the university, its researchers and students, and lets us proceed as a deep-tech specializing in product R&D and launching innovations with cutting-edge models,” Lotufo said.

The system will be driven by few-shot learning, a type of machine learning designed to classify new data when only a few training samples are available. In the classic model, the computer has to be trained on a specific set of documents for each task and lawyers have to take notes manually. The new solution proposes to use the same model for all tasks. It has hundreds of billions of parameters and is “pre-trained” on billions of documents so that it can predict sentences and produce similar texts to humans. The utopian future glimpsed in movies where robots read books in a library has already arrived, according to Lotufo.

This type of machine learning has been chosen by such major players as Google, Facebook, Microsoft and OpenAI in their search for more precise models that require fewer resources. In 2020, NeuralMind produced BERTimbau, a Brazilian version of BERT, Google’s open-source neural network-based machine learning system for natural language pre-training.

It plans to train GPT-3 to handle Portuguese so that Brazilians can deploy the tool, considered the most powerful AI ever created. “We’ve been working since late 2021 on mega-models like OpenAI’s GPT-3. This order from TCU will enable us to apply and explore this accumulated knowledge,” Lotufo said.

*With information from INOVA UNICAMP’s press office.