AI can turbocharge government regulatory operations

Three considerations for realizing AI’s potential in regulatory operations

To harness generative AI’s potential, regulatory organizations should consider the following steps: 1) understand the unique capabilities of different AI tools; 2) employ multiple tools tailored to unique tasks, and; 3) adapt business processes to integrate AI and human judgment. Embracing these principles can help regulatory agencies keep pace with the industries they oversee by speeding up regulatory processes, automating internal processes to reduce burden on staff, and improving compliance rates through data-driven decision-making.

Understand the unique capabilities of different tools

Different AI tools work differently. Generative AI, for example, can do creative tasks that other forms of machine learning cannot, but at the cost of some degree of accuracy—for example, the hallucinations you may have heard about where generative AI responds with plausible, but incorrect facts. On the other hand, more traditional forms of machine learning can discern hierarchical relationships from data, but can’t easily show why it reached a certain conclusion.

This interplay of strengths and weaknesses means that some AI tools may be better suited than others for different tasks. Tasks requiring repetitively creating content, like transcribing hearing notes or generating reports or analyzing user sentiment on a call, could be given to generative AI. Tasks involving finding patterns in large volumes of data with a high degree of precision, like identifying duplicative legislation or detecting fraud, could be assigned to traditional machine learning technologies. Finally, the strengths of human judgment in dealing with high variability or social components means that AI should augment human expertise. Applying AI tools can free up humans to accomplish more challenging tasks more efficiently.

This is similar to a system already in place in Denmark. The Danish Business Authority uses AI to do the computationally heavy task of analyzing more than 230,000 financial statements a year for possible fraud and inconsistencies.⁷ But once an issue has been found, determining if fraud has occurred requires awareness of content that human judgment should provide.

Look for multiple tools tailored to their tasks

Because different AI models do different tasks, adopting the most sophisticated and most expensive tool may not yield the best results. Rather, the best results are likely to come from multiple, smaller tools, each playing to its particular strengths. Take AI augmented tax operations. An AI digital assistant can prepare tax returns in roughly five minutes.⁸ A natural language processing (NLP) enabled chatbot can ask tax-related questions to a taxpayer and autofill a tax return with the help of RPA. Finally, a taxpayer can review the tax return, validate the data, and then file the return. Further, simple tax returns can be done by the chatbot, leaving more complicated returns for humans.

A single generative AI tool would need extremely strict parameters for this task to avoid getting too creative with tax advice. The combination of different AI tools, each playing to their strengths, can generate results with a smaller footprint than a single massive tool.

Adapt business processes to integrate AI and human judgment

It can be counterproductive to layer new technologies onto regulatory processes built for the paper age. Agencies should rethink their regulatory processes with an understanding of what AI can deliver. A Deloitte survey found that when AI projects are accompanied by significant workflow changes, they are 36% more likely to succeed.⁹

An effective process can emerge by first determining the regulatory outcome and then identifying the most suitable technologies to accomplish the associated tasks. The Securities and Exchange Commission (SEC) has invested in a tool that incorporates NLP and machine learning to analyze regulatory filings of investment advisers. Instead of just aiming to make existing processes faster, the SEC is trying to use this tool to do something new: to find patterns that could flag potential violations. Backtesting analyses found algorithms are five times better than random inspections of filings at identifying language in a filing that qualifies for referral to enforcement.¹⁰

Such a monitoring tool could support risk-based targeting of compliance activities by identifying where the likelihood or impact of noncompliance is the highest. In other words, this tool could allow regulators to work differently. Rather than waiting for violations and issuing penalties or remediations, regulators could proactively inspect filings.

So how can AI transform regulatory operations?

Even from these three considerations, it is clear that AI can have a tremendous impact on regulatory operations. AI can help improve the efficiency of existing tasks, such as the Danish Business Authority’s use of AI to identify fraud. But it can also allow regulators to achieve desired outcomes in fundamentally new ways, such as the SEC’s use of NLP and AI to detect infractions by investment advisers. AI can also help improve both efficiency and effectiveness across the entire regulatory life cycle—including public consultation, developing regulation, and enforcing regulations           (figure 1).

Using AI for counter manipulation of public comments

Public comment periods are a major element of most regulatory decision-making. The importance and openness of public comment also, however, makes it vulnerable to manipulation.

When the Federal Communications Commission (FCC) proposed new rules for net neutrality in 2017, their online comment system received over 22 million comments. Researchers used tools like NLP and clustering algorithms to determine that more than 1 million of those comments—maybe far more—were written by bots.¹¹ The FCC hired a contractor to search comments for repeated phrases, word use, and sentence structures. They found up to 1.3 million “unique” messages in which words had been swapped for synonyms within identical sentences and paragraphs.¹² Unlike a cut-and-paste email campaign, where real people copy a message composed by an advocacy group, these messages had each been slightly changed, likely by a bot, to bypass filters and appear “unique.” One analysis found that only 800,000 comments were organically written rather than being spam messages or part of a larger email campaign.¹³

To help combat bot-generated pressure campaigns, leaders can sort through comments with a variety of tools. Reading all 22 million comments was infeasible for humans, but AI can cluster comments into groups, essentially narrowing those millions of comments down to 25 or so bulk messages. The top 25 comments represented 98% of the comments in favor of repealing net neutrality.¹⁴ “Sentiment analysis” can sort comments into pros and cons, providing a rough poll.

AI can also allow regulators to reimagine the process of open comment periods. Spam detection can help filter out bot attacks and AI-generated comments. NLP and sentiment analysis could summarize legitimate comments. Generative AI can cluster their arguments, and perhaps even make inferences about the constituencies at play. Finally, generative AI can summarize findings of multi-stakeholder discussions to help allow policymakers to focus on the most important issues.

Using appropriate safeguards, AI can assess not just the content of comments, but also the metadata. Factors like operating system, IP address, time of submission, and browser all help identify spam. A data analytics firm found a batch of suspected FCC bot comments based on invisible line breaks called “\n strings.”¹⁵

AI can include such factors in analysis as long as the system takes care to protect privacy. Deloitte’s trustworthy AI framework (see sidebar, “Six dimensions of trustworthy AI”), which demands transparency and limits on how data is used, can help AI assess relevant data without compromising privacy.¹⁶

Maintaining institutional memory

Regulatory agencies often rely on human memory. Regulators test regulatory strategies, close loopholes, and write regulations. They become experts in technologies, facilities, and industries. They learn how to provide a steady and predictable service while navigating government procedures. When a long-serving regulatory expert leaves an agency, it can mean a vast loss of knowledge. Turnover can also make it difficult for incoming regulatory and policy staff to hit the ground running, which could delay rulemaking or enforcement of regulations.

AI could help mitigate these problems. First, as employees leave, exit interviews can capture their tacit knowledge. Generative AI can then use those interviews to create onboarding and training documents tailored to a new hire’s specific job. Generative AI could be trained on existing legislation and regulations, and historical enforcement decisions so existing staff and new hires can query chatbots to better understand complex policy documents and speed up regulatory research processes.

Improving effectiveness of inspections

AI offers myriad options for improving inspections throughout the process: prioritizing where to inspect, reducing paperwork during inspections, producing first drafts of reports, and uncovering insights into historical patterns.

The tools for targeting inspections are well-established. Some health departments use AI to determine which restaurants are most likely violating code.¹⁷ First, a program assesses risk factors using everything from 311 complaints about garbage to Twitter posts about food poisoning.¹⁸ These risk assessments help target inspections toward more likely offenders, resulting in infractions that are more than a third of a control group with random inspections.¹⁹ Researchers estimated the software could save 9,000 food poisoning incidents and upwards of 500 hospital visits a year.²⁰

Likewise, building inspectors in New York City have used an AI tool to determine which factors correlate to structure fires and assign every building a risk score. The tool helps focus inspections, much like the tool for health inspectors targeting potentially unsafe restaurants. It automatically lists buildings in order of priority, including incorporating lower-risk buildings with mandatory annual inspections, like schools. While the department started out considering about six risk factors, the New York City Fire Department's latest iteration tracks over 7,500 datapoints from 17 city agency data streams.²¹ The AI learns from each new fire, identifying new factors to consider, and reassesses risk in real time. This feeds a real-time dashboard of fire risk, which fire station leaders can use to plan. Similar algorithms could predict high-risk situations in nearly any regulatory domain, from workplace hazards to environmental spills.

Generative AI could come into place once inspections begin. A new permit for a manufacturing facility might take days for an air quality inspector to understand. Does the provision about gas boilers larger than X cubic feet running at night apply at this facility? Generative AI could summarize the permit into key points to inspect and highlight pertinent sections. The inspector would still have to read it all, but at least they'd start with a summary. Here, again, is where institutional knowledge comes in. A chatbot with memory of past infractions from a specific facility can suggest where to look, and information gleaned from past inspectors can remind the new guy to check behind that one door where they keep all the chemicals. AI could help build a briefing form about every facility.

Apart from prioritizing inspections, generative AI can also help in documenting infractions, sending notices, and generating first drafts of inspection reports. Saving time on paperwork can add up. Based on an analysis of the US federal government workforce conducted in 2017, it’s estimated that automating manual tasks through AI could free up tens of millions of regulatory staff-hours. For example, up to 60 million hours a year could be saved for activities related to compliance and enforcement operations alone. Another 26 million hours annually of inspectors’ time could be freed up as well.²²

In the six years since this analysis was done, AI capabilities have vastly improved and, with the advent of generative AI, time savings likely would now be even higher. AI can now take on an even greater number of low-value tasks, while inspectors could work on high-value, cognitive tasks.

AI augmentation can allow inspectors to spend more time doing what they’re supposed to—inspect high-risk operations. 

Getting started

When thoughtfully paired with complementary AI tools and human judgment, generative AI can open up transformative possibilities for regulation development and enforcement. The path to that transformation features some significant challenges, but three considerations could help regulators improve their chances of success as they embark on their AI journeys:

Become familiar with underlying technology

Understanding the basic capabilities of different types of machine learning and generative AI can help leaders decide which technologies are appropriate for which tasks. The relative transparency of a traditional machine learning setup might be better for finding patterns in incident reports rather than a generative AI tool. Detailed knowledge about how the AI models work can also help mitigate challenges with privacy and security. For example, using “knowledge embedding” to restrict the scope of documents that an LLM will review can help improve accuracy and help keep queries out of the public domain. Knowing AI’s capabilities lets one imagine what is possible.

Put users at the center

Reimagined regulatory processes should work like any well-designed tech project: put the users at the center. Depending on the process that AI is augmenting, those users may be citizens, businesses, or even the regulators themselves. Centering around the user can help shape good design. The basic principles of human-centered design apply here. First, focus on the outcome that users desire and discover what factors may be currently hindering that progress. Next, develop progressively more refined solutions that can bring technology to help reduce those challenges. Finally, find metrics that correlate to serving the mission and continually measure them.

Invest in developing trust in AI

While economies evolve, and regulators should evolve with them, regulations are often a slow turning ship. Balancing trust and stability with the benefits of a fast-moving technology can be a challenge for regulators.

Trust is important to regulators. When it comes to trusting AI, a central tenet of Deloitte’s trustworthy AI framework is transparency. Developing trustworthy AI will require transparency in two ways. First, transparency of the AI models themselves. The opacity of black box AI models, where inputs enter and a result emerges with no one certain what happened in the middle, can be concerning especially if your life or livelihood is on the line. Public trust will likely become stronger when there is an oversight mechanism that stakeholders can understand. So, if regulators are using AI to augment decisions, they should be sure to choose an algorithm with explainable features. If AI is used to find hidden trends in data, then features of the data used to train the model should be public, if not the training data itself. If AI automates a process, the results should be consistent. AI in government is expected to be held to a higher standard of trust than experimental models in the private sector.

The second dimension of transparency is about the purpose of AI models. AI programs should be impartial, transparent, accountable, consistent, secure, and should protect privacy. Leaders should not only build in all the right governance safeguards from the beginning, but also be clear about when AI models are used and why. This clarity can help show the public and workforce alike that AI is not being used to infringe on privacy or replace workers but is designed to bring a real benefit to the public. Our research has shown that communicating benefits is among the surest ways to generate trust in government.²³

Toward a future of better, faster regulatory processes

AI makes big changes possible. The prospect of reengineering a process to create a better one incorporating machine learning can sound daunting, but it doesn’t need to be. Consider starting small if you’re new to AI. New York City built the original version of their AI-driven inspection tool by interviewing veteran firefighters about which factors, anecdotally, correlated with structure fires.²⁴ Within months, the first tool had collected enough data to help build a second tool. Both tools were an improvement on the old system of a card catalog, siloed by station.²⁵ Test new software. Automate a process. See if generative AI can reliably produce a few anodyne compliance reports. Find a real problem, with a clear, measurable outcome, and measure it. As regulators know, sometimes to understand a system, you have to inspect it yourself.