Autonomous generative AI agents: Under development

For example, Cognition Software launched “Devin” in March 2024 with the goal of creating an autonomous software engineer capable of reasoning, planning, and completing complex engineering tasks that require thousands of decisions.⁸ Devin was designed to perform programming jobs unassisted, based on natural language prompts from human programmers. These jobs include designing full applications, testing and fixing codebases, and training and tuning LLMs.⁹ Competitors like Codeium, which focuses on enterprise software development, and open-source versions of Devin, hit the market in summer 2024.¹⁰

Agentic AI software engineers share similar capabilities and vulnerabilities.¹¹ One vulnerability is that they currently make too many errors to handle full, or even partial, jobs without human oversight. In a recent benchmarking test, Devin was able to resolve nearly 14% of GitHub issues from real-world code repositories—twice as good as LLM-based chatbots,¹² but not fully autonomous. Big tech companies¹³ and startups are striving to make agentic AI software engineers more autonomous and reliable, so human coders—and their employers—can trust them to handle parts of their workload (figure 1). 

Augmenting and amplifying labor productivity

Agentic AI software engineers are just one example of how autonomous generative AI agents could transform how work is done (see Promising use cases for autonomous gen AI agents below). As agentic AI improves, its impact could be enormous. There are over 100 million knowledge workers in the US, and over 1.25 billion knowledge workers globally.¹⁴ Total factor productivity,¹⁵ a useful proxy for knowledge work, has stagnated in the United States, growing 0.8% from 1987 to 2023 and only 0.5% from 2019 to 2023.¹⁶ In most OECD countries, the story is the same.¹⁷ Attempts to increase the productivity of knowledge work by automating tasks have met with only partial success. Many companies also need more knowledge workers. Shortfalls of customer service representatives, semiconductor engineers, and seemingly everything in between persist. When new workers start, they need to be productive quickly.

Expert systems and robotic process automation (RPA) can falter when processes are ambiguous or require multiple steps. Systems based on traditional machine learning require extensive training, which is tailored for specific purposes. Built on LLMs, agentic AI can be more flexible, and it can address a broader range of use cases than machine learning or deep learning.

Agentic AI can significantly advance the capabilities of LLMs and could vindicate the investments companies are making in gen AI. The public release of gen AI tools has quickly captured the attention of executives. It was easy to imagine how their organizations could use the technology. Quantifiable business value from gen AI has often been elusive, however. Challenges with data foundations, risk and governance policies, and talent gaps make it hard for companies to scale gen AI initiatives.¹⁸ Only 30% of gen AI pilots make it to full production.¹⁹ Lack of trust in gen AI output, and potential “real world” consequences from gen AI mistakes, give executives pause.²⁰

Companies that develop and implement agentic AI need to consider the challenges of gen AI, plus the complexity of building bots that can reason, act, collaborate, and create. Most importantly, gen AI agents of all kinds need to be reliable for enterprises to use them: Getting the job right most of the time isn’t enough. There are some use cases and applications in late 2024 that show encouraging signs of being reliable enough for adoption in early 2025.

The potential payoff is worth the effort, however, and early results seem promising. Companies are learning how to boost LLM performance by combining these models with other AI technologies and training techniques. While autonomous and reliable agents are the goal, incremental increases in accuracy and independence could help companies reach their early productivity and efficiency goals for gen AI overall.²¹ With their range of applications—both horizontal and vertical—and clear business goals, agentic AI looks more like the gen AI solutions executives may have expected in the first place.

Gen AI agents explained: A closer look

Generative AI agents can break down a complex task into a series of steps, execute them, and work through unexpected barriers. They can sense their environment, which depending on the use case can be virtual, physical, or a combination of the two. To complete a task, agentic AI can determine which actions to take, recruit assistance from tools, databases, and other agents, and deliver results based on its goals set by humans.

Agentic AI is an emerging technology—and it continues to evolve—but it has some common characteristics and capabilities:

• Built on foundation models: Foundation models like LLMs enable agentic AI to reason, analyze, and adapt to complex and unpredictable workflows. This makes it more flexible than RPA and expert systems. LLMs are improving rapidly, with enhanced reasoning and ability to break tasks into smaller steps among the most recent breakthroughs.²² But foundation models alone can’t interact with their environment, make decisions, or execute tasks.²³ They must be augmented by other technologies and capabilities.
• Acts autonomously: While the degree of autonomy varies, agentic AI can be trained to plan and execute complex tasks largely on its own. By introducing reasoning tokens, chain-of-thought models can solve more complex challenges than previous LLMs. Chain-of-thought models are slower to respond but are more deliberative in how they reason through a problem; they can correct their own errors; and they can show the steps they have taken to reach a solution.²⁴
• Senses the environment: Agentic AI can perceive the environment, process information, and understand the context of the tasks it is given.²⁵ Advanced agentic AI can process multimodal data, such as videos, images, audio, text, and numbers.
• Uses tools: Agentic AI interacts with tools and systems to complete tasks, such as software, enterprise applications, and the internet.
• Orchestrates: Agentic AI can direct the participation of other systems and bots to complete a task. With multiagent systems, this means collaborating with other autonomous generative AI agents.
• Accesses memory: LLMs are stateless: Each interaction is processed independently, and information is not retained when an interaction is complete. With the addition of retrieval mechanisms and databases, agentic AI can access short-term memory to maintain context while performing a specific task, and long-term memory to learn and improve from experience.²⁶

Some of the latest models employ chain-of-thought functions that, while slower and more deliberative than prior large-scale models, enable higher-order reasoning on complex problems.²⁷ Multimodal data analysis can make agentic AI more flexible by expanding the kinds of data that can be interpreted and produced. Multimodal AI also shows that agentic AI can be even more powerful when combined with other kinds of AI technologies such as computer vision (image recognition), and transcription and translation.²⁸ Like agents themselves, multimodal AI is still developing.

True multiagent systems, in which work is orchestrated among a network of autonomous agents, are being developed now, with some pilots being launched in late 2024.²⁹ Multiagent models often outperform single-model systems by distributing tasks, especially in complex environments.³⁰ Startups and big tech are developing multiagent gen AI systems, including tools that can help organizations build their own custom agents.³¹

Promising use cases for autonomous gen AI agents

Big tech companies and startups are developing early-stage solutions that can partially automate functions like software development, sales, marketing, and regulatory compliance. What follows is a snapshot of today’s examples, not an exhaustive list of applications. Some are based on proofs of concept and demos that are promising but are not ready for enterprise deployment. While these examples are cross-industry, industry-specific agentic applications are also emerging.

Customer support: Customer service is an essential—and often stressful—job, with an annual turnover rate of 38%.³² Effective automation of parts of the customer support workflow could reduce stress and tedium for staff, and help companies serve more customers.³³ Agentic AI can handle more complex customer inquiries than today’s customer support chatbots, and they can act autonomously to resolve issues. In one example, an audio company is using agentic AI to help customers set up new equipment, a multistep process that usually requires a human agent. If a human agent is required, the agentic AI compiles relevant information and summarizes the issue before transferring the customer.³⁴ The next wave of customer support agents will likely integrate multimodal data such as voice and video in addition to text-based chat.

Cybersecurity: Cybersecurity experts epitomize the shortage of skilled knowledge workers: Globally, there’s a shortfall of four million today.³⁵ Meanwhile, malicious actors are using gen AI to infiltrate cybersecurity systems. Emerging agentic cybersecurity systems can make human experts more efficient by automating aspects of their work. They can autonomously detect attacks and generate reports, improving system security and reducing the workload of human experts by up to 90%.³⁶ Agentic AI can also help software development teams detect vulnerabilities in new code. It can run tests and communicate directly with developers to explain how to fix a problem—something human engineers must do manually today.³⁷

Regulatory compliance: Companies across industries, including financial services and healthcare, are required to conduct periodic regulatory compliance reviews. The increased size and complexity of relevant regulations, and the dearth of compliance professionals, makes compliance a growing challenge. Startups are developing agentic AI that can analyze regulations and corporate documents, and quickly determine whether the company is compliant. The agent can cite specific regulations, and proactively provide analysis and advice to human regulatory professionals.³⁸ Companies that use gen AI today cite regulatory compliance as their top barrier to developing and deploying gen AI, ahead of issues like a lack of AI technical talent, and implementation challenges.³⁹ Regulatory uncertainty plays a role, but so does the reach and complexity of new regulations. By helping companies understand and comply with regulations as they’re enacted, a more agentic AI solution could help accelerate wider gen AI adoption across enterprises.

Agent builders and orchestrators: Agentic AI solutions are emerging to help automate other cross-industry and vertical-specific workflows. Companies may not need to wait for the market, however. They can build their own agents and multiagent systems. With Google’s Vertex, companies can use no-code tools to create agents for specific tasks, such as building marketing collateral based on previous marketing campaigns.⁴⁰ LangChain uses open-source technology to help companies construct multiagentic systems. For example, startup Paradigm has launched a “smart spreadsheet” in which multiple agentic AIs partner to collect data from diverse sources, structure it, and complete tasks.⁴¹

Bottom line

Agentic AI has enormous potential to help increase the productivity of knowledge workers by automating entire workflows and discrete tasks. Its ability to take independent action, as single agents or in concert with other agents, sets it apart from today’s chatbots and co-pilots. Yet, agentic AI is in the early stages of development and adoption. As impressive as early agentic examples may be, these agents can make mistakes and get stuck in loops. In multiagent systems, “hallucinations” can spread from one agent to another; they can persuade other agents to take the wrong steps and give incorrect answers.⁴² Although agentic AI can be mainly autonomous, often having a human review decisions after they’ve been made (also known as “human on the loop” rather than the more restrictive “human in the loop”) can make agentic AI more suitable for deployment today. When gen AI agents get stuck, they can consult human experts who help them resolve the challenge and move forward. In this model, agentic AI is like a junior employee who can learn by experience while performing valuable work.⁴³

While some companies are investing billions to create consistent and reliable agentic AI, it’s not clear when this will happen, or under what circumstances. Will agentic AI reach widespread adoption in 2025, or within the next five years? Will ubiquity require breakthrough innovation, or tweaking current AI technologies and training methods? If the big companies and startups developing agentic AI are successful, the game will change quickly. Imagine autonomous gen AI agents that can process multimodal data, use tools, orchestrate other agents, remember and learn, and execute tasks consistently and reliably. Imagine further that custom agents can be quickly and easily developed by enterprises in “no-code environments” using just conversational text prompts.

Because the vision for agentic AI is compelling and the technology is evolving rapidly, companies should prepare themselves now. As they prepare, they should consider the following approaches.

Prioritize and redesign workflows for agentic AI: Consider which tasks and workflows are well-suited for agentic AI to execute, based on the technology’s capabilities and where the highest value is for your company. Redesign them to remove unnecessary steps. Ensure that agentic AI solutions have a clear goal, and access to the data, tools, and systems they will require. Although these agents can help other agents navigate their environment, cluttered and sub-optimized processes could deliver disappointing results. 

Focus on data governance and cybersecurity: For agentic AI to deliver value, it must have access to valuable and potentially sensitive enterprise data, as well as internal systems and external resources. Companies should put strong data governance and cybersecurity in place before getting started with autonomous generative AI agents. For gen AI early adopters, the top areas where they’re increasing IT investment are data management (75%) and cybersecurity (73%).⁴⁴ Despite these investments, 58% are highly concerned about using sensitive data in models and managing data security. And only 23% say they’re highly prepared for managing gen AI risk and governance. In short, many of today’s gen AI leaders seem unprepared for the advent of agentic AI. If these leaders are not ready, companies that are still on the gen AI sidelines surely have further to go.

Balance risk and reward: When starting with agentic AI, companies should consider the level of autonomy and data access agents are permitted. Low risk use cases with non-critical data and human oversight can help companies build the data management, cybersecurity, and governance for safe agentic AI applications. Once these are in place, companies should consider higher value use cases that use strategic data, access to more tools, and more autonomy.

Maintain healthy skepticism: Agentic AI is evolving and will likely be more capable in the next year, and will be applied to more horizontal and vertical-specific use cases. Expect impressive demos, simulations, and product announcements throughout 2025. But the challenges we’ve noted may take some time to resolve. Until these challenges are addressed, agentic AI performance in controlled settings is unlikely to deliver improved enterprise performance. Evaluate and question carefully.