How can tech leaders manage emerging generative AI risks today while keeping the future in mind?

Exploring four gen AI risk categories impacting cyber strategies

Our analysis of the fourth edition of Deloitte’s Global Future of Cyber Survey, which surveyed nearly 1,200 cyber decision-makers at the director level or higher, identified eight potential risks specific to gen AI, ranging from integrity risks like hallucinations, to social engineering attacks, to inadequate governance of gen AI strategies. Original analysis of that data for this research finds that respondents are equally worried about all of them, with 77% saying they were concerned “to a large extent” about how these risks may impact their cybersecurity strategies.  

To provide a clearer understanding of the intersectional nature of these threats and the areas they impact, we can organize these gen AI risks into four distinct categories: risks to the enterprise, which include threats to organizational operations and data; risks to gen AI capabilities, which include the potential for AI systems to malfunction or their vulnerabilities to be misused; risks from adversarial AI, which include threats posed by malicious actors leveraging gen AI; and risks from the marketplace, which include economic, legal, and competitive pressures that could influence AI deployment and security (figure 1).

In the following sections, we explore each of these four risk categories in more detail to understand their potential implications and the strategies that can be employed to help manage them.

Risks to the enterprise

Gen AI introduces increased enterprise risk across data, applications, infrastructure, and processes. Our research shows some of the most prevalent are:

Data privacy, security, and intellectual property risks. Gen AI models are trained on large, diverse collections of text, images, and audio that are gathered from the web, generated by other models, or manually curated. Third-party data and models often don’t authenticate original sources, creator intentions, copyrights, or basic properties, making provenance hard to track⁴ and perpetuating potential hallucinations and misinformation.⁵  

Additionally, gen AI is now creating art, music, literature, software, and inventions, leading to questions about authorship, ownership, and protection. Regulation around who owns what innovation and thus who can monetize it is still in the works.⁶ This has increased copyright ambiguity.

Gen AI also raises privacy concerns. Personal details like names and addresses might be collected unintentionally, leading to accidental exposure or misuse of sensitive information, trade secrets, or confidential data. For instance, a health care company might use a retrieval system to access patient records, enabling medical professionals to query a patient’s medical history using natural language, but also exposing that private data to unintended risk.   

Security and employee risk across development processes. As gen AI is introduced into development processes, new security risks emerge. A Palo Alto Networks report found that AI-generated code was the top concern for surveyed security and information technology leaders.⁷ The Deloitte leaders interviewed for this article expressed concerns that AI might inadvertently amplify existing code-level vulnerabilities such as misconfigured code, increasing the risk of data breaches, malware infections, and reputational damage. The concern is grounded in the lack of transparency into third-party foundation models, which may introduce unknown vulnerabilities.

Additionally, employees are frequently using unsanctioned gen AI solutions. According to a 2024 AI Adoption and Risk Report from Cyberhaven Labs, usage of major gen AI tools at work is through personal accounts.⁸ Unauthorized use can unknowingly expose sensitive information to risk. For instance, Samsung banned the usage of gen AI tools among its employees after it was revealed that employees accidentally leaked sensitive data in public prompts.⁹

Emerging approaches to enterprise risk management

Given the evolving risk landscape, enterprise risk leaders and chief information security officers (CISOs) can consider the following actions to address risks to the enterprise:

• Implementing digital asset management and data privacy controls. Organizations need to know where data they bring into their organization came from, and where data they are sending out into the world has gone. One way to achieve this is through a digital passport or model cards that can help verify training data and history as well as model origins—a capability known as digital provenance. Authenticity techniques, data consent mechanisms, and data provenance standards can help companies achieve data traceability. Some organizations are working to develop infrastructure for the “consent layer” in AI data, which involves gathering opt-in and opt-out information from data creators and incorporating it into searchable databases, such as the Do Not Train registry.¹⁰

In the absence of universal standards, tracing data lineages can help. For instance, the Data & Trust Alliance’s Data Provenance Standard¹¹ is a collaborative effort among 19 corporations to document leading practices.

Strong intellectual property management strategies can embed trust into content creation. One method is using verifiable content credentials for images, videos, fonts, and audio files. Additionally, advanced digital rights management solutions can prevent unauthorized copying, provide real-time tracking, enforce licenses, and control digital asset distribution.¹² Some companies have started embedding hidden signals, or watermarks, in their data to identify machine origins or prevent future machine use.¹³

• Reimagining DevSecOps processes for a new age of prompt engineering: Deloitte research on engineering has shown that gen AI also can change the way software is developed, by creating increasingly integrated roles among individuals in the pod team.¹⁴ Risk and cyber executives can take a leadership role in shaping those programs while centralizing governance structures to confirm clear policies and processes are in place for everything from AI usage to network monitoring and that new processes are being systematically implemented and scaled.

Risks to gen AI capabilities

Gen AI also introduces new security risks that target the data and models that gen AI solutions depend on. Emerging threats include:

Prompt injection attacks. A distinctive feature of gen AI solutions is their use of prompts or instructions given to the AI. Prompt injections are an emerging technique where attackers design prompts to deceive gen AI systems into revealing secure data, spreading misinformation or tricking the prompt into performing a malicious action or access the model via a backdoor with a hidden trigger.¹⁵ Prompt injections are the leading security threat aligned to large language model (LLM) applications on the Open Worldwide Application Security Project Top 10.¹⁶

Evasion attacks. Traditionally, in AI systems, an evasion attack happens when the model is deliberately misled or tricked by conflicting samples, known as “adversarial examples,” leading to incorrect output. Gen AI systems are often vulnerable to these attacks at a greater scale than traditional AI. Anyone with basic query privileges can probe the model with prompts intended to understand its predictive labels and confidence scores to engineer the model into making different decisions.¹⁷ These attacks can be used to bypass intrusion detection systems and more.

Data poisoning. External models trained on public data introduce a host of unknowns. Data poisoning is one such risk: a deliberate attack on an AI system where an adversary alters the model’s training dataset. Data poisoning techniques may include altering data to become deceptive, incorrect, or misleading. The risk of data poisoning increases with retrieval augmented generation systems.   

Hallucinations in gen AI models. Gen AI models predict outputs from training data patterns, but when they hallucinate, the results may appear plausible yet be incorrect. Such inaccuracies can cause faulty decisions, damaged reputations, regulatory penalties, and lost opportunities.¹⁸ Like hallucinations, misinformation can be perpetrated through gen AI models, which may lead to loss of trust, financial loss, and negative impact on business decisions.

Emerging approaches to data, model, and application security

CISOs and chief technology officers should account for these increased risks and attack surfaces. This should include a broad approach to cybersecurity that encompasses various strategies and measures, including:

• Combatting prompt injections with input guardrails and model firewalls. First, consider input sanitization and validation that preprocess user inputs to eliminate or neutralize harmful content before it reaches the LLM. This can involve filtering, escaping, or transforming inputs to confirm they follow expected patterns and are free of malicious instructions. Confidential data should be thoroughly vetted or masked before connecting to an LLM application, and standard security practices such as least-privilege access to reduce the attack surface and parameterization to prevent the execution of malicious code should be adhered to reduce the risk of prompt injection.

Second, instead of depending on rules-based solutions, organizations could enhance their defenses by implementing an AI firewall to monitor data entering and exiting the model, which can enable better threat detection.¹⁹ While tools like prompt shields can help detect hidden instructions, human oversight remains critical to judge and approve the generated output.²⁰

• Fine-tuning to improve accuracy. Fine-tuning is a supervised process where a model is retrained with data to improve its content generation accuracy. We discussed emerging approaches when fine-tuning a gen AI model to combat hallucinations in part 2 of this series including bringing in authoritative external sources through retrieval augmented generation, implementing contextual guardrails, creating prompt libraries, and human oversight—each of which can help to reduce hallucinations.²¹
• Integrating gen AI into cyber capabilities. According to the fourth edition of Deloitte’s Global Future of Cyber survey, 36% of the responding organizations reported including AI and gen AI as part of their organization’s cybersecurity budget.²² Gen AI can identify subtle signs of threats hidden in network traffic and system logs, which might be missed by traditional security tools.²³ Gen AI can spot more complex phishing attempts by studying patterns in legitimate emails. For instance, NVIDIA introduced a spear phishing detection AI workflow which can reduce the time needed to develop a phishing detection solution with 21% higher accuracy when compared to existing methods.²⁴

AI-powered deepfake detection tools can also outperform humans in identifying subtle indicators to identify misinformation and machine-generated content—whether text, audio, images, or multiple formats in combination—on a large scale.

The National Institute of Standards and Technology recommends adversarial training for data models²⁵ to replicate cyberattacks and help identify security gaps and fortify defenses. General adversarial networks (GANs) and smart vulnerability detection tools can uncover security flaws that traditional scanners may overlook.²⁶

Risks from adversarial AI

Gen AI technologies commoditize the skills needed to orchestrate cyberattacks, lowering the entry barrier for malicious actors. The fourth edition of Deloitte Global Future of Cyber survey shows close to one-third of organizations are concerned about phishing, malware, or ransomware (34%), and threats related to data loss (28%).²⁷ Gen AI can increase the sophistication, scale, and ease of adversarial attacks. These types of risks may include:

AI-generated malware. Gen AI introduces more data and complexity into the digital ecosystem and enables attackers to automate malware and ransomware more easily, increasing both the scale and sophistication of known threats. For instance, hackers continuously create new malware and cybercriminals can now leverage gen AI to produce near limitless, sophisticated malware, potentially overwhelming conventional cybersecurity measures and response times.²⁸ A study uncovered about 100 machine learning models capable of injecting insecure AI code onto user machines.²⁹

Phishing attacks that seem more human. Gen AI has revolutionized phishing attacks. IBM’s 2024 X-Force Threat Intelligence Index found “AI” and “GPT” were mentioned in over 800,000 dark web posts in 2023.³⁰ Adversaries can exploit technology to craft believable messages and automate large-scale, sophisticated social engineering campaigns. Gen AI tools have commoditized the ability to draft convincing, context-sensitive messages, text and audio messages—overcoming language barriers, and even integrating cultural subtleties.³¹ Moreover, AI’s capability to self-improve could enable it to develop autonomous phishing tactics based on its acquired knowledge.³²

Impersonation attacks. The ability to create fake voices and videos through gen AI tools is an emerging risk that has matured over the last three years,³³ significantly reducing the barriers to perpetuating impersonation fraud. For example, in banking, gen AI is expected to magnify the risk of deepfakes for financial fraud. Deloitte’s Center for Financial Services forecasts that gen AI could enable fraud losses to reach US$40 billion in the United States by 2027, up from US$12.3 billion in 2023, a compound annual growth rate of 32%.³⁴ 

Emerging approaches to adversarial AI

CISOs can evolve their strategies to address adversarial AI risks by:

• Scaling traditional threat detection and response. CISOs have an opportunity to expand traditional AI and deep learning techniques to stay ahead of these threats. Mastercard is set to launch Decision Intelligence Pro, a system that evaluates the connections between various entities involved in a transaction to gauge its risk. The technology is estimated to increase fraud detection rates by an average of 20%, with potential peaks of up to 300% in certain cases.³⁵ Similarly Visa uses AI and machine learning to analyze over 500 attributes of each transaction to counter frauds.³⁶ Additionally, Microsoft noted that when monitoring its ecosystem, the company detects more than 65 trillion cybersecurity signals per day with the help of AI.³⁷
• Updating adversarial training programs: Trainings should be updated to cover the unique threats of gen AI being used by attackers and how employees should be vigilant in terms of digital interactions to avoid sophisticated phishing attempts. CISOs should drive security behavior and culture programs to help reduce human-related cybersecurity risks. These programs should aim to shift from merely increasing awareness to fostering behavior changes.

Risks from the marketplace

Broader market risks are still taking shape, and many are largely outside of organizations’ control, including regulatory risk, infrastructure resilience, third-party risk, and others. Some leading risks include:

Regulatory uncertainties. According to Deloitte’s fourth quarter State of Generative AI in the Enterprise report, being able to comply with regulations is the biggest concern reported by organizations surveyed. These can span global and regional regulations that govern what’s possible and dictate the use of data, security and privacy considerations integral to managing gen AI risks. For instance, in the United States, the 2023 executive order 14110 on “Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence” included a provision that directed federal agencies to impose reporting requirements on entities developing, acquiring, or possessing advanced AI models or computing clusters.³⁸ However, in 2025, the new administration issued subsequent executive orders that rescinded order 14110 and directed federal agencies to review existing AI policies and roll back any rules that may hinder AI innovation.³⁹ As a result, the reporting requirements have been rescinded. Organizations would have to be mindful of the evolving regulatory landscape’s implications on gen AI offerings.

Computing infrastructure risk as gen AI scales. Gen AI demands significant computing resources, putting pressure on the already strained electric grid. Public utilities are often not agile, and it can be challenging to accurately estimate future energy demand.⁴⁰ The aging infrastructure is struggling to keep up. According to Deloitte’s 2025 power and utilities industry survey, respondents cited grid infrastructure limitations as the key challenge in providing reliable power to data centers.⁴¹ According to Deloitte analysis, data centers, which currently consume 6% to 8% of total annual electricity generation in the United States, could potentially see their electricity needs grow to between 11% and 15% by 2030.⁴²

New value chains. Data centers face new uncertainties, and thus risks and opportunities, across their value chains. Many investment firms, real estate companies, and engineering construction organizations are working to access permits, land, and funding for new data centers. Many cloud hyperscalers, telecommunication companies, and tech infrastructure providers are working to manage increased computing demands. Limited electricity supply in US locations such as Northern Virginia, Columbus, and Pittsburgh can delay new project approvals.⁴³ In the United Kingdom and Europe, insufficient power has hindered the construction of new data centers.⁴⁴ Additionally, supply chain bottlenecks can cause delays and higher costs due to shortages of critical components and materials. For example, many operators are facing a delay in securing crucial equipment like generators, uninterruptible power supply batteries, transformers, servers, and building materials, forcing them to use available alternatives.⁴⁵ Along with these challenges, enterprises and data center operators have to manage these increased loads.

Limited application flexibility caused by vendor lock-in. Gen AI models and infrastructure are advancing faster than organizations can keep pace, introducing a risk that leaders could be paying for obsolete or duplicative capabilities or partnering with vendors that don’t ensure their products interoperate easily with other future technologies.

In addition, many organizations are rushing to secure advanced hardware for gen AI, but suppliers are struggling to keep up with the demand. NVIDIA’s Blackwell graphic processing units (GPUs) are sold out for the next year, prompting companies to seek alternatives.⁴⁶ Organizations that rely on a single vendor risk missing key hardware advancements in this competitive market.⁴⁷

Value realization concerns. The initial investment required for training and running large models along with the necessary computing hardware is substantial. According to Deloitte’s fourth quarter State of Generative AI in the Enterprise survey, about one-third of respondents believe that not achieving the expected value could slow the overall marketplace adoption of generative AI over the next two years.

Emerging approaches to marketplace risks

Based on the extent the organization is impacted by supply-side or demand-side risks (or both), leaders are likely exploring several solutions, including:  

• Cutting down on computing demand by shifting to small models and reducing workloads. While training small language models (SLMs) can be more expensive, upfront investment could save on GPU needs.⁴⁸ For example, while Salesforce launched their LLM XGen-7B with seven billion parameters in 2023, the company is also deploying small models for a specific task where greater efficiency may be needed to minimize the cost of computing and specialized hardware.⁴⁹ SLMs run on devices like a phone or a laptop,⁵⁰ reducing energy consumption. Google and NVIDIA have also launched their SLMs, which can be deployed directly to devices for low latency.⁵¹ For example, training bidirectional encoder representations from transformers on a large dataset can take 64 Tensor Processing Units (TPU) days, and substantial energy usage. In contrast, smaller datasets might only need a few hours, significantly cutting energy consumption.⁵² This approach is one of the strongest levers businesses have control over to reduce marketplace risks. Additionally, organizations are cutting down retraining time to reduce workloads. IBM, for example, introduced an associative memory module to enhance a model’s ability to manage longer contexts.⁵³
• Making strategic and efficient infrastructure decisions. Some organizations are making multimillion-dollar investments in on-premises data centers and hardware to build their own infrastructure; but in doing so, they’re rebuilding the walls they’ve spent decades trying to tear down by moving to the cloud and creating a hybrid cloud infrastructure that will require enhanced switching equipment to move across on-prem, edge, and cloud solutions. To that end, a recent industry report predicts the AI data center switching equipment market could reach US$1 billion by 2027.⁵⁴

The challenge with this approach is that companies have invested in these GPUs and still need scalable energy access. For example, an organization focusing on gen AI-based language translation and dubbing services initially purchased a four-GPU cluster, estimating a 60% to 70% cost-savings relative to the cloud. However, they hadn’t accounted for the power and cooling requirements that would come from running six machines. Ultimately the company opted for a hybrid approach with simple workloads performed on premises and more resource-intensive computing in the cloud.⁵⁵

Resource efficient hardware, along with efficient workload balancing, will play a crucial role in managing power consumption and efficiently utilizing available hardware. NVIDIA introduced its Blackwell GPU platform whose processors are designed to handle the demanding workloads of LLM inference while significantly reducing energy consumption.⁵⁶ IBM has also revealed its NorthPole chip was 25 times more energy-efficient than the other commonly used 12 nm GPUs and 14 nm CPUs.⁵⁷

On-premises alternatives—including edge AI infrastructure solutions—are also gaining prominence, and new leading practices in cooling, construction, and collaboration are emerging. Cooling requirements can constitute up to 40% of energy consumption in data centers.⁵⁸ As workloads increase and power demands generate more heat, liquid cooling systems—which circulate coolant directly to heat-producing components—have been used to manage data center power demand more efficiently.⁵⁹

• Managing energy consumption and addressing power grid stress. According to Flexential’s 2024 State of AI Infrastructure report, organizations also are turning to third-party colocation data centers for AI infrastructure needs. The survey found that only 24% of respondents use on-premises AI hardware, while 51% lease rack space at colocation centers for edge processing.⁶⁰ With a constrained public power grid, organizations are looking into microgrids with renewable energy for enhanced resilience and improved sustainability outcomes.⁶¹

Given the challenge goes beyond data centers to a larger energy load challenge, there’s also a need for alternative energy sources to fill the gap. While modernization of the energy grid and smart meters are certainly options, the innovation needed to support the scale of energy goes well beyond what a modernized grid can handle. As a result, alternative energy solutions are also being explored which can complement the existing infrastructure. For example, Google’s upcoming data center in Mesa, Arizona, expected to be finished by 2025, will use over 400 megawatts of clean energy from three sources: solar photovoltaic panels, wind turbines, and battery storage systems.⁶² Similarly, a German company colocation provider has placed data centers inside wind turbines for clean energy access.⁶³

Like alternative energy, nuclear energy and micro-nuclear strategies are on the rise. There have been reports of increases in micronuclear reactors—compact energy infrastructure available by road, rail, or sea—to meet the colocated demand.⁶⁴ Amazon acquired a data center site next to Pennsylvania’s Susquehanna nuclear power plant.⁶⁵ Microsoft agreed to recommission and purchase all of the electricity generated by Three Mile Island’s nuclear power plant for the next 20 years to support its data centers.⁶⁶ Similarly, the future viability of these energy approaches will be impacted by national energy policy and infrastructure decisions.

• Recommissioning and investing in data centers as AI factories. There’s been significant movement in the data center market, with 2.1 gigawatts of new leases signed in the United States from mid-May to late July 2023.⁶⁷ Amazon is investing US$10 billion in two data center complexes,⁶⁸ and 15 global telecommunications companies across a dozen countries are building gen AI data centers, or AI factories.⁶⁹ Given data centers are newly defined as critical infrastructure in the United States as of 2024, these commercial organizations could serve as the first line of defense to understand whether public and private US computing needs are met. Sovereign cloud and regional models to maintain national privacy will be important as well.⁷⁰

Organizations reliant on gen AI as a core business opportunity might need to invest more considerably to transform their infrastructure. For example, Meta has built its own colocated AI factory to support its large gen AI workloads while managing power or cooling needs through unified system design.⁷¹ The upfront cost of its colocation strategy is part of a larger investment strategy for the business.⁷²

• Decentralizing and creating new capacity through edge computing and low earth orbit. Edge data centers are small, localized data centers that are part of a larger distributed network. They can reduce latency and improve application performance needing real-time data processing as they are positioned near the areas they serve. Gen AI, being a computationally intensive technology, can benefit from edge computing’s reduced latency, local storage and processing, and fewer data transfers to the cloud. Edge data centers can also help in the reduction of central grid power consumption by distributing computing loads across the network. The resilience is further improved as the network structure can provide a collocated backup network.⁷³ According to one interview with a Deloitte leader, one option could be exploring low earth orbit for colocated data centers, liberating them from the constraints—and competition—for power on the same electrical grid supporting cities and communities. Japan’s NTT is exploring photonics and optical tech to send data to satellite networks for efficient processing and storage. The telecommunications firm plans to gather and analyze diverse data in space via a high-speed optical network, delivering only crucial information to users swiftly. This could potentially improve real-time space data usage and user convenience.⁷⁴
• Building trust and governance in gen AI deployments. Deloitte’s Trustworthy AI framework recommends operationalizing trust in AI through a thoughtful strategy.⁷⁵ The strategy should be coupled with governance process, standards, third-party risk management, and technology-enablement to manage the program efficiently and transparently. Depending on the scale of implementations, organizations can adopt a full stack approach in their gen AI deployments to help reduce the risks of obsolescence and maintaining cohesiveness between the components. This can help to understand the overall risk exposure from the ecosystem so leaders can take the necessary corrective action.

Reducing risk and strengthening resilience

Gen AI introduces internal and external risks that cut across the four categories discussed in this article—to the enterprise, to gen AI capabilities, from adversarial AI, and from the marketplace. The risks are multidimensional and intertwined in a way that organizations, regardless of industry and strategy, should assess the best way to protect their gen AI integrations. Leaders will likely need to play a crucial role in aligning their cyber and business resilience strategies based on this assessment. Risk leadership, including the CISO, is well positioned to help leaders understand the organizations’ exposure to gen AI risks, as well as how to address them with known and new approaches to protect the organization on all fronts.

A single solution can’t address all the risks, and organizations will likely have to align and customize multiple solutions based on their exposure. CISOs should also confirm that they are following a human in the loop along with a secure-by-design approach across their software development lifecycle to ensure that their organization is safeguarding customer privacy. They should build on their established practices to accommodate the evolved nature of existing risks as well as the nuanced risks of gen AI models and the rapidly evolving infrastructure needs to enable business resilience strategies accordingly.