Defending reality: Truth in an age of synthetic media
With the proliferation of AI tools, it’s now easier than ever to impersonate and deceive, but leading organizations are responding through a mix of policies and technologies.
You may have recently seen an ad with Tom Hanks pitching a dental plan. The actor himself didn’t participate in the shoot. Someone simply used his likeness, together with deepfake technology, to make it appear as though he had.1
Take it as a sign of the times, when anyone can be made to look as though they said or did anything. Artificially generated content, driven by rapid advances in generative AI, has reached a point where it’s almost impossible for people to separate what’s real from what was conjured from the depths of computers.
It’s not just celebrities in the crosshairs. With the proliferation of artificial intelligence tools, it’s now easier than ever for bad actors to impersonate others and deceive their targets. Many are using deepfakes to get around voice and facial recognition access controls, as well as in phishing attempts. AI applications themselves, which demand huge amounts of data, are rich targets for hackers. The security risks are multiplying with every new content-generation tool that hits the internet.
But leading organizations are responding through a mix of policies and technologies designed to identify harmful content and make their employees more aware of the risks. The same generative AI tools used by bad actors to exploit organizations can be used to identify and predict attacks, allowing enterprises to get ahead of them.
Now: The next generation of social engineering hacks
Social engineering hacks have always relied on convincing a person to hand over data or access to systems for illegitimate purposes. Though the strategy can be very effective, it also requires a lot of personal interaction between the bad actor and the victim. Artificially generated content enables attackers to create that personal touch with a much lower time investment. A wave of artificially generated content is now targeting enterprises, exploiting vulnerabilities by impersonating trusted sources. The problem is accelerating rapidly.2
Currently, there’s a large gap between AI’s ability to create realistic-sounding content and people’s ability to recognize it. A majority of people say they can tell the difference between AI- and human-generated content, but another 20% aren’t sure.3 However, the first group is likely being overconfident. Few people can reliably distinguish between the two precisely because AI content generators are trained on human-created content and developed to replicate it as closely as possible.4 People may expect artificially generated content to look or sound robotic in some way, but more than ever, it feels human.
Bad actors are likely to use artificially generated content to attack businesses in several ways (figure 1).
Improved phishing: Phishing is the most common type of cyberattack, with 3.4 billion spam emails sent every day. In 2021, cybercriminals stole an estimated US$44.2 million through phishing attacks.5 Phishing attacks typically succeed not because they’re high quality but because they’re sent out in massive volume—out of billions of emails, eventually a few will achieve their goal. Most recipients are generally able to identify phishing attempts because of the use of poor grammar and spelling, or because the sender clearly doesn’t know the recipient. But generative AI tools allow fraudsters to craft convincing, error-free messages quickly and easily and to provide relevant context, which enables them to tailor messages to each recipient, making the messages harder to ignore. The problem is likely to get worse as the quality of publicly available models improves.6
Deepfakes: Deepfakes have been around for years, but until fairly recently, they haven’t been convincing enough to be used in cybercrimes. Now, we’re starting to see them used to attack businesses. For example, the CEO of a UK-based energy firm was conned out of US$243,000 by scammers using deepfake AI voice technology to impersonate the head of the firm’s parent company.7 Deepfake tools have advanced significantly since this incident and are likely to continue improving rapidly, making it harder for people to know with confidence with whom they are dealing.
Prompt injection: Web browsers and email clients with virtual assistants could be leveraged by bad actors who leave malicious prompts in webpages or emails that instruct the assistant to forward data such as contact lists, banking information, and health data.8 Most types of social engineering hacks have historically worked by tricking people into handing over data or access to systems. But with prompt injection, hackers don’t even need to bother with this step. The prompts execute automatically, without the victim’s knowledge.
Misinformation: Social media campaigns against businesses are nothing new, but artificial content is adding fuel to the fire. AI tools can be used to create massive amounts of content quickly. Bad actors can use the tools to target enterprises, causing reputational harm or even threatening stock prices.9 In the past, attackers had to personally craft messages, but content-generating tools now give them the ability to churn out misinformation at scale, allowing them to experiment and test out messages with the public until they find one that resonates.
The wide availability of generative AI and the pace with which content-generating models are improving are likely to supercharge these problems. For little to no cost and with virtually no technical skill, anyone will be able to create convincing media to separate businesses from their money and data.
New: Arming the enterprise against an emerging threat
None of this means enterprises are powerless against the tidal wave of artificially generated content coming their way. Leading enterprises are taking proactive steps to make sure they don’t become victims.
Social engineering is nothing new, and while synthetic media may give hackers a new tool in their toolbox, many of the tried and true methods for preventing this type of attack are still applicable today. Being suspicious of online communications, verifying the identity of people with whom you’re communicating, and requiring multifactor authentication to access sensitive assets are all ways enterprises can guard against this new attack vector.
As with most types of social engineering threats, tackling the problem of synthetic content starts with awareness. “While AI is exciting and there’s a lot of cool things happening, it’s also giving a lot of capability to cybersecurity bad actors,” says Shamim Mohammed, chief information and technology officer at CarMax. “A big focus for me is making sure that we’re staying current and [even] ahead so we can protect and defend our company.”10
One way he does that is by working with a set of ecosystem partners. Mohammed says CarMax partners with both leading tech companies and AI-focused cybersecurity startups to get smart on the threat landscape and access the latest tools for preventing attacks.
“We have a very strong technology ecosystem,” Mohammed says. “We work with big players who are on top of the AI revolution as well as a lot of startups that are focusing on AI. So we have the best tools available to protect our information from this emerging trend.”
Effective tools are emerging to help enterprises identify potentially harmful content. Just as AI can create content, it can also assess images, video, and text for authenticity. These tools may soon be able to predict the sorts of attacks enterprises are likely to face.
When it comes to both creating and detecting artificial content, scale, diversity, and freshness of training data are paramount. When generative AI models first became publicly available, bad actors had an advantage because these models were trained by huge tech companies with access to the most powerful hardware and largest sets of training data. The first generation of detectors pushed out by large tech companies didn’t match that scale while training tools to identify synthetic content.11
That’s changing. Reality Defender, for example, trains its synthetic media detection platform on a petabyte-scale database of text, images, and audio, some of it artificially generated. When training on such a large corpus, subtle tells begin to emerge that indicate something was created by an AI tool. For example, AI-generated images often have specific deformations or pixelations. Text has a measurable degree of predictability. These things may not be obvious to the naked eye, but an AI model trained on sufficient data can learn to reliably pick them out.
Ben Colman, CEO at Reality Defender, says being able to identify harmful content and respond to it is critical for enterprises, particularly when it comes to misinformation and disinformation campaigns that may seek to harm the business’s reputation or that of its leadership. “Once something has gone viral, it’s too late,” he says. “If a brand is harmed in the court of public opinion, it doesn’t matter if it comes out a week or two later that the content was untrue.”12
Other tools exist to detect AI-generated content based on specific signifiers.13 Soon, synthetic media detectors will become even more finely tuned. Intel recently introduced a deepfake detection tool that looks beyond data and analyzes videos for signs of blood flow in the faces of people in the videos. When a person’s heart pumps blood through their veins, the veins change color slightly. This is something that can be measured in authentic videos but is very hard for AI models to mimic.14
Expect more efforts like this. According to some estimates, as much as 90% of online content will be synthetically generated by 2025.15 Much of it will be for legitimate purposes such as marketing and customer engagement, but cybercriminals will likely use generative tools for their own advantage. It has never been more important for enterprises to be able to identify the veracity of the content their employees interact with.
Next: The cat and mouse game continues
Many organizations were quick to add AI reinforcements to their arsenals a couple years ago,16 but generative AI has given bad actors a new weapon of their own. Enterprises are now catching up. Expect this process to continue in the future as new paradigms such as quantum computing mature and deepen AI’s capabilities.
Quantum computing is still a few years away from being broadly available, but it is rapidly maturing, and it may well become the next tool of choice for both hackers and enterprises. One of the most promising use cases for the technology looks to be quantum machine learning. Like any tool, what matters is how you use it. It has the potential to supercharge the problem of artificially generated content but also could be a boon to enterprises’ cyber defenses.
Quantum machine learning has shown the potential to generate more accurate predictive models on less training data.17 Classical computing data exists as a binary: Data is either a 0 or a 1. But quantum data can take on more than one state at a time, allowing quantum bytes to contain richer information. When applied to machine learning, this allows for the development of much more complex models than are possible today with even the most advanced graphic processing unit hardware.18
This could result in hackers creating better-targeted content without needing to gather more data about their intended victims. Instead of a model requiring hundreds of hours of video training data to create a convincing deepfake of a person, a few snippets could suffice in a quantum machine learning world.
However, for enterprises looking to improve their cybersecurity, quantum machine learning could also significantly improve synthetic media detectors. Rather than requiring billions of data points before they learn to recognize artificially generated media, detectors may learn to spot fakes after seeing a handful of examples.
Quantum computers may even enable enterprises to better predict the types of attacks they’re likely to face. Quantum machine learning excels at predictions, potentially exceeding classical machine learning. This is because quantum algorithms can explore the likelihood of various predictions being wrong and return an answer that is less likely to miss its mark.19 It may seem as though predicting the source of attacks is impossible today because they can come from almost anywhere, but the maturation of quantum machine learning could help make the problem more manageable. This could put businesses in the position of preventing attacks rather than responding to them.
Enterprises need to prepare for this reality now because bad actors aren’t sitting still. Getting ahead of the problem now will help keep them from being swept under by the tidal wave of artificial content that is set to come their way.