Opening up to AI: Learning to trust our AI colleagues

New: From black box to glass box

Mistrust of AI can come from business leaders, front-line workers, and consumers. Regardless of its origin, it can dampen enterprises’ AI enthusiasm and, in turn, adoption. But leading organizations are working on solving issues that diminish trust in AI implementations. Some of the most effective approaches treat AI not so much as a point technology but rather as a piece in a larger process, considering the various stages where humans interact with the AI system and working to identify and address areas of potential mistrust. Acknowledging that AI tools are techniques to be woven into the larger tapestry of processes within an organization can make it easier to fix trust issues proactively. For more trusted AI, forward-thinking enterprises are leaning on data transparency, algorithmic explainability, and AI reliability (figure 1).

Data transparency

Transparent data-collection methods enable the end user to understand why certain pieces of information are being collected and how they’re going to be used. When users have this control, they can make informed decisions about whether the AI tool represents a fair value exchange.⁹

The Saudi Tourism Authority used this approach when developing a new application for travelers. The app uses AI to guide tourists through their stay in the country, recommending restaurants, attractions, and other activities based on location and preferences. But importantly, the user is in control of the data they provide to the app. Visitors can determine how much or how little data they hand over, or can opt out completely, with the understanding that giving the app less data access may mean less-tailored recommendations.¹⁰ This stands in contrast to many apps that have all-or-nothing data access requirements that generally serve as a poor foundation for trust.¹¹

Algorithmic explainability

One of the biggest clouds hanging over AI today is its black-box problem. Because of how certain algorithms train, it can be very difficult, if not impossible, to understand how they arrive at a recommendation. Asking workers to do something simply because the great and powerful algorithm behind the curtain says to is likely to lead to low levels of buy-in. 

One automaker in the United Kingdom is tackling this problem by bringing frontline workers into the process of developing AI tools. The manufacturer wanted to bring more AI into the vehicle-assembly process by enabling machine learning to control assembly robots and identify potentially misaligned parts before the vehicle gets too far into the assembly process. At the start of the development process, engineers bring in front-line assembly workers to gauge their perception of problems and use that to inform development. Rather than dropping AI into an arbitrary point in the production process, they use it where the assemblers say they most need help.

The tools ultimately built are interpretable because the workers’ input forms the basis of alerts and recommendations. In other words, it’s easy for assemblers to see how the AI platform’s recommendations map to the problems they themselves helped define. By bringing in workers at the start and helping them understand how the AI functions, developers are able to support the assembly team with trusted cobot coworkers rather than a silicon overlord dictating opaque instructions. 

AI reliability 

People have grown accustomed to a certain level of reliability from work applications. When you open an internet browser or word-processing application, it typically simply “behaves.” More specialized business applications such as customer relationship management platforms and enterprise resource management tools may be a bit more finicky, but their challenges are fairly well established, and good developers know how to troubleshoot them. 

With AI, the question isn’t whether it will work but rather how accurate the result will be or how precisely the model will assess a situation. AI is generally neither right nor wrong in the traditional sense. AI outputs are probabilistic, expressing the likelihood of certain outcomes or conditions as percentages—like a weather forecast predicting a 60% chance of rain—which can make assessing reliability a challenge. But workers need to know how accurate and precise AI is, particularly in critical scenarios such as health care applications.¹²  

AI is sometimes viewed as much as an art as a science, but that may need to change for robust adoption. Organizations that take a rigorous approach to ensuring AI reliability consistently see better results. Those that document and enforce MLOps processes—a set of procedures designed to ensure machine learning tools are deployed in a consistent and reliable manner—are twice as likely as those that don’t to achieve their goals and to deploy AI in a trustworthy way.¹³ Taking an operations-minded approach puts guardrails around AI and helps build confidence that it is subject to the same standards of reliability as any other business application. 

But reliable doesn’t necessarily mean perfect. Just as human coworkers will never deliver perfect results every time, AI too will make mistakes. So the bar for reliability is not perfection, but rather how often it meets or exceeds an existing performance standard. 

Next: Creative machines

As enterprises deploy AI in traditional operational systems, a new trend is taking shape on the horizon: generative AI. We’re already seeing the emergence of tools such as OpenAI’s Dall-E 2 image generator and GPT-3 text generator. There’s a generative model for music called Jukebox that lets users automatically create songs that mimic specific artists’ styles.¹⁴ AI is increasingly being used to automatically caption live audio and video.¹⁵ These types of content generators are getting more sophisticated by the day and are reaching the point where people have a hard time telling the difference between artificially rendered works and those created by humans. 

Concern over automation’s impact on jobs is nothing new, but it is growing ever more pronounced as we head toward this automatically generated future. In many cases, generative AI is proving itself in areas that were once thought to be automation-proof: Even poets, painters, and priests are finding no job will be untouched by machines.

That does not mean, however, that these jobs are going away. Even the most sophisticated AI applications today can’t match humans when it comes to purely creative tasks such as conceptualization, and we’re still a long way off from AI tools that can unseat humans in jobs in these areas. A smart approach to bringing in new AI tools is to position them as assistants, not competitors. 

Companies still need designers to develop concepts and choose the best output, even if designers aren’t doing as much of the manipulating of images directly. They need writers to understand topics and connect them to readers’ interests. In these cases, content generators are just another tool. As OpenAI’s CEO Sam Altman writes in a blog on DALLE-2, “It’s an example of a world in which good ideas are the limit for what we can do, not specific skills.”¹⁶

Workers and companies that learn to team with AI and leverage the unique strengths of both AI and humans may find that we’re all better together. Think about the creative, connective capabilities of the human mind combined with AI’s talent for production work. We’re seeing this approach come to life in the emerging role of the prompt engineer.¹⁷ This teaming approach may lead to better job security for workers and better employee experience for businesses. 

AI continues to push into new use cases through emerging capabilities that most people thought would remain the exclusive domain of humans. As enterprises consider adopting these capabilities, they could benefit from thinking about how users will interact with them and how that will impact trust. For some businesses, the functionality offered by emerging AI tools could be game-changing. But a lack of trust could ultimately derail these ambitions.