Why hasn’t AI delivered on its promise?

AI exploded out of research some ten years ago, promising to deliver all manner of science fiction solutions. From autonomous cars and perfect prediction machines¹ for business, through to ushering in the singularity (where machine intelligence accelerates past human intelligence).² Pundits predicted systemic disruption as AI eliminated the need for humans in many fields of endeavor. Some even went so far as posit that we should stop training professions such as radiographers,³ as AI would soon be so superior that human radiographers would find it impossible to compete.

Despite all this promise, adoption of AI is not where many expected (or hoped) that it might be. Research continues to improve the underlying AI technology—the recent of development of both Midjourney and Stable Diffusion⁴ is a case in point—and firms continue to invest in AI.⁵ We even saw a bump in investment during the first couple of years of the pandemic.⁶ However, a majority of AI projects fail.⁷ Compelling demonstrations are not transitioning into value creating solutions. Autonomous cars are a prime example, where commercial, mass market, versions constantly seem to be a decade away, despite early success and significant investment. We hear a similar story from AI practitioners working in firms attempting to leverage AI, with carefully developed models and solutions left on the bench as they are either not compelling enough or too fragile to replace existing solutions. There are notable successes, such as machine language translation, however there appears to have been more misses.

Reframing the challenge

It’s possible that broad adoption of AI is being held back by the usual human adoption concerns, such as a lack of the technical implementation skills required, resistance to organizational change, and potential job displacement. Or it might be the case that deploying AI solutions requires significantly more work than anticipated, and it will take some time to work through many issues. We might even ask if AI is not the general-purpose technology⁸ many assumed it to be, as successful point solutions have not evolved into general business platforms (the way computers, once used in offices for discrete, complex calculations, eventually became part of the business infrastructure⁹). AI might be a solution to particular types of problems, but we’ve yet to develop the ability to identify the types of problems it is best applied to.¹⁰

Before we write off AI though, we might ask if the disappointment in AI is due to a failure in the technology being able to develop quickly enough¹¹ (or as quickly as we’d like), or if some other factor is at play. There has been some speculation that the shortfall in AI realization is due to an inability to translate theory into practice—something akin to the technology commercialization chasm that plagues academic research. This is an unsatisfying explanation, though, as there is no indication that AI is different from other technology domains that don’t suffer similar problems.¹²

Another possibility is that the emergence of AI was due to the development of complementary technologies—innovations that AI relies on—rather than developments in the core AI technologies themselves. For example, Deep Blue, the solution that triggered the computational chess revolution, could be attributed to a rapid drop in the cost of computing¹³ along with discovery (by AI researchers) of the Markov Decision Process,¹⁴ rather than a key development in the core AI techniques used.

Many (if not all) recent AI developments are at least equally due to the development or commoditization of other technologies, as the development of the core technology itself. The surprise emergence of machine translation in mid-1990s was more likely the result of dropping computation costs and access to digital, multilingual texts, rather than a significant improvement in the underlying theory.¹⁵ Similarly, autonomous cars went from research oddity¹⁶ to potentially transformative technology due to the development of portable LiDAR¹⁷ sensors and powerful portable computers¹⁸ that were both lightweight and had miserly electricity requirements.

Technology comes in packages, big and small

Technology comes in packages, as any practical solution relies on a range of complementary technologies as well as the key innovation.¹⁹ Consider how the first telegraphs (used to dispatch trains) were not much more than a battery, switch, two conductors, electromagnet, striker, and bell—a package of six complementary technologies.²⁰ New technologies rely on packages of earlier technologies, creating dependencies from our current technologies to their earlier, simpler, building blocks. Over time, these packages of technologies have grown in size. The modern equivalent of the telegraph—sending an emoji via text message—depends on a huge package of technologies that includes mobile computing and digital radios along with global telecommunications networks.²¹ The complex digital environment we live in means any modern solution will have many dependencies.

In the past, solutions that depended on large packages of technologies were at a disadvantage. It was quite possible, for example, to create impressive AI-powered legal analysis solutions in the 80s and 90s, though these solutions were hamstrung by the need to manually enter information from paper documents before the solution could do its work. Lack of easy access to digital data means that the solution was left on the shelf, uneconomic. Today, on the other hand, the same data is most likely digital, likely stored in a cloud-based document management system that can be accessed via an API, and a solution that was technologically possible but not economically viable now makes sense as it can leverage the many technologies in our technology-rich environment.

Our approach to realizing AI’s promise in the enterprise is predicated on there being some remarkable new development in AI. Assuming that the core AI technology is racing ahead, firms have attempted to seize the opportunity by investing in developing internal capabilities and then targeting these capabilities at problems that seem suitable. Machine learning (ML), for example, is seen as a machine for perfect predictions (given sufficient data) so the challenge is to develop expertise in ML and then find predictions to make perfect.

This approach, after a few early wins, appears to be running out of steam.²² A marketing team, for example, might use machine learning to develop a new customer segmentation model. At first, the solution provides novel (and valuable) insights, but repeated use provides little benefit. Bringing a fresh pair of (machine) eyes to the problem was valuable, but the enduring benefits that were expected from “constantly optimizing the model” never eventuated. A similar dynamic can be seen in other AI research efforts. It’s straightforward, for example, to hire a team of AI graduates, build a prototype autonomous car, and provide compelling demonstrations of the car’s capabilities. It’s very challenging, though, to move past that first prototype. A general autonomous car, one that can operate in the same diverse range of environments that humans can, remains beyond our reach.²³

What if, rather than focusing on particular AI technologies, we tease apart the packages of technologies that resulted in those technically possible solutions? Developments in the core AI technology likely have contributed to the solution’s success, but what if the success was also due to developments in complementary technologies?²⁴ Our earlier legal example is a case in point. The barrier to adoption of the first legal solutions was easy access to suitable digital texts, a barrier that was overcome by a combination of the automation of legal business processes (making the documents digital) and the shift to cloud computing (making the digital documents easily accessible online). While the core AI technologies were improved in the intervening years, it was developments in the complementary technologies that transformed the solutions from technically possible to economically viable.

There are two consequences.

First, other AI techniques that haven't yet reached the limelight might also have transitioned from the technically possible to economically viable as they can leverage similar complements. An organization that has used robotic process automation (RPA) to provide digital access to all the steps in a business process, for example, might swap their business process engine²⁵ for an AI-enabled real-time planning engine to dynamically manage business processes.²⁶ This has the effect of eliminating (static) business processes (as processes are constructed dynamically by the planning engine), along with business exceptions (as each process is constructed to meet the particular needs of a single transaction).

Second, and more interesting, rather than look for particular (novel) AI technologies and then try to find potential problems that the technology might address, we can look across the enterprise to find challenges and opportunities where the changes to the complements enable new approaches, and then pull in a range of technologies (new and established) to address these challenges and opportunities.

Our natural recency bias²⁷ leads us to focus on the novel and new. However, the history of big ideas tends to be a story of the drive to find a better way, coupled with the incremental development of otherwise banal technologies. The development of the global multi-modal container network is a great example²⁸—the genesis of which was in 1956 when Marshall McLean (1914-2001) launched an all-container shipping line built on the realization that up to half of shipping costs were due to the manual handling required to move cargo between transport modes (from boat to truck to train to truck and so on).²⁹ All-container shipping streamlined these mode changes to realize dramatic savings, and so usher in the next phase of globalization.

Big disruptive innovations tend to be the result of the incremental development of existing ideas which, at some point, come together in a way that amplifies their effect. The earlier legal example is one such instance, as is the story of machine translation. If we construct a figure that captures the technology novelty and the impact of the solutions, such as machine translation and the global multi-modal container network, then we find ourselves with the two-by-two shown in figure 1.