Start with the business case, assess your data and model requirements, invest in the right talent, and be willing to fail fast to reap the maximum benefits from cloud machine learning.
As organizations look to use machine learning (ML) to enhance their business strategies and operations, the retailer Wayfair looked to the cloud. Wayfair believes what so many other organizations seem to—that the cloud and the fully integrated suite of services it offers may give organizations the best approach to create ML solutions with time to market, existing resources, and available technologies. For its part, Wayfair pursued the cloud for its scalability—key for retailers with varying demand levels—as well as a full suite of integrated analytics and productivity tools service offerings to drive actionable insights. The results? Faster insights into changing market conditions to save time. Immediate access to the most current ML technologies. And tools of productivity that can drive efficient talent.1
Indeed, ML is transforming nearly every industry, helping companies drive efficiencies, enable rapid innovation, and meet customer needs at an unprecedented pace and scale. As companies increasingly mature in their use of ML, the heavy reliance on large data sets and the need for fast, reliable processing power are expected to drive the future of ML into the cloud.
The well-known benefits of the cloud—including modular, elastic, rapidly deployable, and scalable infrastructure without heavy upfront investment—apply when bringing ML into the cloud. Cloud ML additionally introduces cutting-edge technologies, services, and platforms—including pretrained models and accelerators—that provide more options for how data science and engineering teams can collaborate to bring models from the lab to the enterprise. Indeed, because of all these advantages, time, technology, and talent—which might otherwise serve as challenges to scaling enterprise technology—comprise the three-pronged promise of cloud ML (see sidebar, “The measurable value of cloud ML”).
The recognition of this potential has fueled rapid growth: The current cloud ML market is estimated to be worth between US$2 billion and US$5 billion, with the potential to reach US$13 billion by 2025. What such growth says is that adopters are increasingly realizing the benefits of cloud ML and that it is the future of artificial intelligence (AI). And with just 5% of the potential cloud ML market being penetrated according to one estimate, the community of adopters should only deepen across sectors and applications.2
To explore the potential cloud ML represents for organizations looking to innovate, we conducted interviews with nine leaders in cloud and AI whose perspectives largely inform the content that follows. In this paper, we provide an overview of three distinct operating models for cloud ML and offer insights for executives as they seek to unlock the full potential of the technology.
In the 2020 Deloitte State of AI in the Enterprise survey, 83% of respondents said that AI will be critically or very important to their organizations’ success within the next two years. Around two-thirds of respondents said they currently use ML as part of their AI initiatives. And our survey suggests that a majority of these AI/ML programs currently use or plan to use cloud infrastructure in some form. Our additional analysis of the survey data for this research showed that cloud ML, specifically, drives measurable benefits for the AI program (figure 1) and generally improves outcomes as compared to nonspecific AI deployments:
Importantly, these data points showed that cloud ML tools appear to make a notable impact on longer-term, strategic areas, such as competitive advantage and improved decision-making, when compared to general AI adoption. This is an important distinction and shift in the approach organizations are taking to run their AI programs (away from on-premise AI platforms), and a trend that is expected to continue as cloud ML takes greater focus for global organizations.
The addition of low-code/no-code AI tools, which are part of some cloud ML service offerings, can extend these advantages with responses showing additional gains in decision-making, process efficiencies, and competitive advantage and other areas. For instance:
The emergence of low-code/no-code tools is part of a broader trend toward greater accessibility to cloud AI/ML solutions and a more diverse user community. These positive survey results suggest that this trend may only strengthen in the near to medium term.
“AI is a tool to help solve a problem,” says Rajen Sheth, vice president, AI, Google Cloud. “[In the past,] people were doing a lot of cool things with AI that didn’t solve an urgent problem. As a result, their projects stopped at the concept stage. We now start with a business problem and then figure out how AI solves it to create real value.”
As organizations look to advance their ML programs supported by the cloud, there are three basic approaches to cloud ML (figure 2) they can take—cloud AI platforms (model management in the cloud); cloud ML services (including pretrained models); and AutoML (off-the-shelf models trained with proprietary data).
Selecting the right approach starts with understanding the business context—what problem the organization is trying to solve, the technology context (to what degree the organization has data and models on-premises or in the cloud already), and the talent context (to what extent the organization has the right people resources in place to build and train models).
There is no one-size-fits-all approach. Depending on the business problem, multiple approaches may be employed at the same company. As Gordon Heinrich, senior solutions architect at Amazon Web Services, says, “At the end of the day, it comes down to where you can put ML and AI into a business process that yields a better result than you are currently doing.” Building a model in-house requires significant investment of time in data collection, feature engineering, and model development. Depending on the specific use case and its strategic importance, companies can employ cloud ML services, including pretrained models, to help speed up time to innovation and deliver results.
Conversational AI: This is one use case that came up repeatedly during our research where cloud ML and pretrained models seem particularly well-positioned to help solve business, technology, and talent challenges. Conversational AI is frequently applied to help improve call center operations and enhance customer service. In fact, one estimate suggests that by 2025, the speech-to-text technology will account for 40% of all inbound voice communications to call centers.4
Organizations can use conversational AI to support customer service improvements in many ways—from helping answer customer questions directly via chatbots to supporting contact center staff in the background with next-best answer technology. These capabilities bring great value to customer service today. However, it takes a tremendous amount of work, time, and resources to continuously enhance conversational AI models, and it is not differentiating enough for most companies to build these models in house. So, most companies are employing large cloud vendors’ pretrained models instead of building their own conversational AI models.
Broadly speaking, there is no single approach to a cloud ML initiative. But any approach should contemplate a complex array of potentially deep and vast issues that are strategic, technical, and cultural in character and range the full arc of the project lifespan.
For general cloud ML use cases, companies should think through comparable aspects of business, technology, and talent to determine the right approach, while keeping in mind the following high-level recommendations:
Although not comprehensive by any means, thoughtful evaluation of these three representative areas should point teams in the right direction as they assess which applications are most appropriate for cloud ML. It can also help ensure valuable talent hours are utilized strategically toward the company’s most differentiating features and initiatives.
The broad applicability of cloud ML business archetypes has driven rapid adoption across industries. Analysts, however, have indicated a standout opportunity for cloud ML to solve business challenges at scale for industries with significant call center footprints as well as supply chain challenges.5
As Eduardo Kassner, chief technology and innovation officer, One Commercial Partner Group, Microsoft, says, “Certainly, there is a big evolution in cloud ML services and APIs across a number of industries. We are seeing increased activity in manufacturing where you bring together IoT, big data, and advanced analytics.”
In figure 3, we have compiled examples from six different industries where cloud ML has helped solve a range of tangible and definable business challenges. These companies have drawn on distinct cloud ML technology services such as speech and vision APIs and pretrained models for recommendation, fraud, and inventory management to advance their AI programs and achieve tangible business outcomes.
“When talking about talent in the AI/ML space, think of a staircase with four steps,” suggests Kassner. “The first step is the developers who leverage cognitive services APIs in their applications. The next step is developers who know how to call APIs for process automation, customer support, object detection, among other scenarios of cognitive services. The third level in the talent staircase is developers who understand big data, data cataloging, data warehousing, data wrangling, and data lakes. The fourth step is the one where specific algorithms and customer models are to be created with ML and AI Ops, which may require data scientists or data analysts."
To achieve the business objectives of any cloud ML initiative, data science professionals typically work with a well-balanced talent team—cloud engineers, data engineers, and ML engineers. But PhD-level data science talent is scarce.6 As a result—and as our interviewees repeatedly confirmed—many companies are training a growing number of developers and engineers to contribute alongside them as non-PhD data scientists.
Some speculate that pretrained model and AutoML capabilities could reduce the need for scarce PhD-level data science talent and thereby help “democratize” ML. However, we recommend viewing these capabilities as a way to extend the reach of deep data science expertise across a broader team made up of new types of specialists. With cloud ML, pretrained models and AutoML tools have “hidden” the complexity and blurred the lines between previously distinct roles. This often empowers ML engineers to fill the data science PhD talent gap and affords adopters greater flexibility in how they staff teams. And, as these tools get more nuanced and powerful, such flexibility should only grow. Ultimately, however, as an organization’s modeling needs evolve and become more complex, a PhD-trained data scientist may become indispensable.
The term MLOps, or “machine learning operations,” refers to the steps that an organization takes as it develops, tests, deploys, and monitors cloud ML models over their life cycle. In effect, these steps come to represent a set of governance practices that ensure integrity throughout the life cycle. As development operations (DevOps) become increasingly automated due to intensifying pressure for frequent model releases, these steps take on greater importance and urgency—especially as companies become more cloud-focused in their infrastructure.
Commenting on the importance of this trend, Heinrich notes, “[MLOps] is about explainability and getting a notification when a model is starting to drift. It is about making the data pipeline easier to use with data tagging and humans in the loop to achieve high-quality results. It really helps businesses feel more confident about putting algorithms into production.”
While there is no single accepted global standard for MLOps process and best practices, a comprehensive end-to-end MLOps program should typically include four basic elements: versioning the model, autoscaling, continuous model monitoring and training, and retraining and redeployment (figure 4).
Model bias: A related concept within a broader ML governance program is model bias. In the past few years, we have seen instances of bias creeping into AI/ML models. In fact, a recent experiment revealed that an AI vision model associated various pejorative stereotypes with gender, race, and other characteristics.7 ImageNet is one of a number of projects underway to identify and mitigate the role that bias plays in AI.8
According to Sheth, as cloud ML moves toward providing decision support or recommendations, the issue of bias becomes even more acute because multiple human judgment factors are manufactured into the model. “There are tools that help explain why the model gave a particular outcome and where there are biases and where there aren’t and what impact such biases will have on people in general. Organizations will have to dig deep to understand what is happening and supplement the data accordingly. It’s a painstaking process,” he says.
One cloud ML area where bias manifests frequently is pretrained models. When developing and training models in-house, the development team can implement safeguards and testing protocols to lessen the effect of bias since control of the process and training data rest with them. In contrast, they are less likely to be aware of bias when using pretrained models, as they have typically limited visibility into how a model was trained and thus how much bias, if any, was present prior to model deployment. Under such circumstances, testing for bias after model deployment requires heightened scrutiny using a variety of known data sets so that the results can be detected based on their output. (For more information on AI ethics and avoiding bias, see Deloitte's Trustworthy AITM framework.)
Cloud ML can be transformational, and there is strong evidence that organizations are already reaping its benefits in wide-ranging initiatives across industries, use cases, and archetypes. Based on our research and interviews with industry leaders, here are some important points for organizations to keep in mind while starting out on their cloud ML journey:
By putting some forethought into these key recommendations and gaining a thorough understanding of what they want to achieve, organizations can optimally employ cloud ML. They can thus reap the three-pronged promise of cloud ML—extending the reach of critical ML talent and technology investments and cutting time to innovation.
