If organizations only pursue pilots, it can create a sense of overconfidence. As small-scale pilot projects succeed, organizations may mistakenly think that they have all the capabilities they need to tackle AI at scale. We observed signs of this overconfidence in our survey results. Seventy-three percent of government respondents believe that they are ahead of the private sector in AI capabilities. And as if to reinforce the optimism bias, 80% believed they are also ahead of their public sector peers.
Pilot purgatory
The problem is that AI at scale requires different organizational capabilities than pilots or proofs of concept. Pilots are typically smaller and narrower in focus than full-scale AI efforts. As a result, pilots can often make use of different technologies and data sources than would be required for full-scale use. They may not need to meet as rigorous security and privacy requirements. Further, the smaller scope of pilots means that they touch fewer parts of an organization so that change management is less of a factor in their success.
For these reasons, development of AI at scale just looks different than pilots. For example, a former CDO of a large US city describes initially being surprised at the slow pace of AI development among peers in the private sector. Only later did the CDO begin to realize that the slower pace may be needed to tackle larger AI projects. The limited scope of pilots may make them easier to pursue more quickly, but for larger-scale projects it takes time to make sure that the right data is gathered, the appropriate use case is chosen, and costly mistakes are not made while developing technological architecture. For those just starting out in their AI journey, it can seem counterintuitive that slowing down the process may be a way to achieving AI at scale quickly. Slow is smooth; smooth is fast.7
In short, organizations that have only experimented with pilot-scale AI cannot make it to the heights of at-scale AI simply by doing more of what they are doing. Without intentional action to acquire the organizational capabilities needed for at-scale AI, organizations can easily become stuck in “pilot purgatory” continually cycling through promising AI pilots but never realizing the transformational benefit that AI promises for their core mission.
Adapt, don’t just adopt
The good news is that government leaders appear to be increasingly aware of the gap between pilots and at-scale AI. The respondents of our survey again and again highlighted the gap between their goals for AI and where they currently assessed their AI capabilities. (figure 4).