The edge of cloud

Evolution: To the edge and back

Diana Kearns-Manolatos: As a technology pioneer and innovator who has been at the forefront of distributed computing for the last three decades, what are you seeing next for the edge of cloud?

Satya: We’re roughly a decade from when cloud computing started. Then, the cloud was about driving economies of scale to their limit, and we largely achieved that vision with gigantic data centers concentrated in remote places. However, fundamental challenges involving the speed of transmission, round-trip times, and bandwidth remain.

The time is ripe for the next inflection point—could we get the kind of compute that cloud computing is able to give but deliver it without those limitations of latency and bandwidth? That’s what edge computing is about. Once you think about it this way, it’s clear why it’s the next step in the evolution.

The next chapter in computing is going to be about the creation of this edge computing infrastructure worldwide to enable brand-new, edge-native applications you simply couldn’t create if you only had to rely on the cloud. What is going to be very exciting is how new edge-native applications are going to bring productivity enhancements to industries that have not yet been transformed. For example, there’s a whole class of industrial troubleshooting applications, which we can talk more about later.

Diana: In some of your prior research, you’ve talked about blurring the boundaries between tiers. How do you see the mobile edge, intelligent edge, and cloud tiers functioning to create innovative computing architectures?

Satya: Mobility is a big driver of modern computing and, by its very nature, imposes demands on computing. First, the phone has to fit in your pocket, the virtual reality (VR) glasses on your face, or the wristwatch on your wrist. The device has to be small and light; it cannot get too hot; and it has to have decent battery life. Let’s call these fundamental challenges to mobility the “mobility penalty.” It is the price you pay for making things mobile.

My next challenge is: How do I get cloud computing’s power and capabilities at the extreme mobile edge to overcome latency (distance) and bandwidth limitations? Edge computing emerges as a solution. It uses what we’ve learned in cloud computing—managed infrastructure, multitenancy, virtualization, virtual machines, and containers—but instead of putting the data into an enormous, exascale data center, it puts it closer to the mobile devices in a much smaller data center (a data center in a box!), a “cloudlet” that’s really a small cloud close by.

The intelligent edge

Diana: What advice do you have for organizations looking to bring AI to the edge of cloud?

Satya (laughs): I’m laughing because I just finished writing a paper⁶ on exactly this topic. So, a few tips:

First, the sheer speed with which things are changing in AI is dramatic. You have to pay attention to the algorithms, but by the time you use them, newer algorithms appear. You may have to slightly change how those models work in your system.

Second, because of this rapidly changing frontier, the optimal hardware and software combination is very hard to pin down. For example, the iPhone® 12 has neural engine chips for deep neural network inferencing.⁷ If your application is written to use this advantage, your phone can compute in a way not previously possible.

Companies should think about what should run on the mobile device, what should run on a cloudlet, and maybe what should run on the cloud. Nonetheless, these decisions should not be viewed as rigid decisions that are architected permanently into the system, but as decisions that you’re making for here and now. The next generation of IT hardware and IT technology might actually require you to rethink that partition.

Dave Linthicum: The big thing that I see evolving is the intelligent edge and the ability to put AI engines in edge-based architectures. A couple of years ago, we worked on a connected bike concept where a hardware device served as a knowledge engine communicating to the cloud.⁸ These tiered systems can support big data analysis on the back-end cloud systems and more transactional knowledge, such as the ability to shut the bike off if it was on fire, at the edge. We are partitioning the “brain,” but each section has access to other sections. Things are done faster, but each section is still able to operate in an independent mode.

This was a more advanced organization. Most companies are still figuring out what an AI engine is, how to leverage the training data, how to be effective with AI meeting data analytics, and how to leverage data virtualization and the operations around it. They're learning how to get their computing infrastructure up, running, and operationalized. The potential is just going to be amazing.

Implementation challenges and solutions

Diana: How are organizations implementing their infrastructure across edge and cloud tiers?

Dave: The challenge that we see organizations running into is architectural. Essentially, they need help with dividing the architecture into tiers. As technologists, we hear questions, such as Where do you keep the data? Where do you keep the knowledge? What’s the most efficient way to do it? Moving forward, whether it’s for robots or edge-based private cloud services, organizations will need to create a configuration management system that can dynamically manage information flow across architecture tiers.

The biggest fear I hear about when I talk to executives moving their organizations to edge computing is not that edge computing can’t be done but whether it's able to operate the infrastructure or tiers in a way that allows for dynamic changes. I had a mining company client, and they put out sensors that were satellite connected to gather information around working mines across the Northwest United States. They had the sensors, but they needed to be able to manage the data across the edge and the cloud on demand with the same ease as on a mobile device.

Diana: How are organizations starting to address these challenges?

Dave: That ability to push software changes remains a challenge, as does tiering of the system. Suddenly, there are 10,000 devices, and managing a large number of dispersed devices can be incredibly complex and exhausting. It requires the ability to have dynamic tiering that leverages different technologies in different ways, with operational configuration management. More vendors will get into this space and make purpose-built tools with AI ops, configuration management, identity access management distributed encryption, and key management systems that work with edge-based systems.

Optimization

Myke: I spend a lot of my time working in energy and resources with organizations like Wichita State University,⁹ which has a 60,000 square foot smart building on a smart grid with robotics, AI/ML, and 3D printers. Are there manufacturing use cases where the four-tier architecture is well-suited? Are there economic considerations around smart metering in power and utilities?

Satya: I have different thoughts for the manufacturing and metering use cases. To me, the potential opportunity for the edge-computing architecture is different in different cases.

The robotic factory you described is very interesting, but it is a capital-intensive investment. The robots are expensive, and each production change requires reprogramming and debugging. Therefore, each specification change comes with a certain fixed capital cost for the hardware. For each product change, there is a nontrivial cost for the software, testing, and troubleshooting. Today, the break-even point for robotics hardware may well be in hundreds of thousands, or millions, of units. Regardless, manufacturing is moving toward shorter, faster production runs—such as, producing 1,000 units quickly. At these levels, making a business case for large fixed costs gets harder.

The whole point of the robots was to reduce the people costs, but if I produce 100 widgets, it’s going to be faster and cheaper for a human with cognitive guidance to produce 100 units than to program and test those robots for such a small run. The truly unique opportunity for four-tiered edge computing (cloudlets in factories) are those situations where there is a human in the loop and the production is not multimillion-unit volumes but shorter runs, where some kind of cognitive assistance, guidance, automation to detect human error is needed.

Below a certain volume, having the sensing done by the system but leaving the actuation to the human is more cost-effective, and it aligns with the emerging desire to have short-run, highly agile manufacturing.

For the utilities and data collection, the opportunity for cloudlets and edge computing really is in the privacy space. The reason is increasing consumer concerns about how much data they’re exposing and all the inferencing that can be done with it. For example, the data collected every day by a smart water meter can reveal when your dishwasher runs. From there, it is a short step to inferring your activity, which is concerning. Edge computing, however, can alleviate some of these concerns by keeping the data in escrow at the edge, giving the consumer the opportunity to receive, for example, a more personalized experience, in exchange for detailed consumption data.

Dave: It is going to be an evolving and learning process on how to convert this to make money. The ability to keep track of the devices out there and their operations, and then to sync with software, security, and governance issues is a problem still to be solved.

To give one example, one of our major retail clients had a recommendation engine that used customers’ browser data and behavior to manage demographics targeting and determining logistics to recommend products that have a higher chance of interest. It raised sales by 25%. If they could integrate their mobile, IoT, and edge data from stores and their supply chain, they could slice and dice that data in new and different ways. The challenge right now to bringing in that data is privacy, security, and orchestration management.

Myke: We’re evaluating options, such as deploying AWS/Outposts or Azure/Stack for clients and, at this point, they may be cost prohibitive. We’re seeing some of the same challenges around 5G. I think economics shouldn’t be underestimated when you talk about edge computing.¹⁰ You can’t penalize the first few use cases with the whole cost of the infrastructure because it can scale with much smaller incremental costs to accommodate future use cases.

Dave: There has to be a business case. My client, a government contractor, is deploying AWS Outposts because their data standards forbid them from leaving top secret data at their data centers. They would pay a premium for it. Ultimately, with the right business case, edge computing can make businesses more innovative and creative and can be a disruptor. Many businesses just look at the bottom line, and they’re missing the core point and end up falling by the wayside.

Innovation: Across technology and industry

Technology innovations

Diana: What developments are you seeing with the telecommunications and cloud providers that are shaping what is possible for this four-tier architecture model?

Satya: I think the big players, the cloud computing players, have all realized that edge computing is not just vapor; it’s real. But none of them quite knows yet how to translate that abstract concept into their edge strategies or to determine how similar their edge approach should be to the cloud.

In 2018, Microsoft announced that the intelligent edge and the intelligent cloud both would be equally important.¹¹ They have followed through on this with the introduction of Azure Edge Zones.¹² They created a brand-new profit and loss group called Azure for Operators, which is basically edge computing. So, not just Microsoft, but many corporate giants, are seeing the significance of edge computing—and they are embracing it.

Initially, Amazon started with pure serverless compute for sensing. They now have an edge and fiber play to reduce latency and improve connectivity speeds¹³ and continue to expand their large-scale software distribution ecosystem across the Alexa devices and skills network.

Vodafone and AWS are partnering on an innovation program for companies in the United Kingdom and Europe to incentivize the use of edge computing.¹⁴ Additionally, Amazon has announced additional edge innovations with the AWS Snow Family.¹⁵

From the viewpoint of the developer and the customer, the best scenario would be if they don’t know where the computing is taking place. If they can run an application and the edge looks exactly like the cloud, no one needs to change the workflow or software. That should be an enormous source of opportunity.

Industry innovations

Diana: Looking forward, Satya, given your work with the Living Edge Lab and the Open Edge Computing Initiative, you get the opportunity to look at the latest research, technology, and innovations in terms of edge-native applications, platforms, and networks. What are you most excited about that you’re seeing?

Satya: The truth of the matter is edge computing is more expensive than cloud computing, and it is always going to be. You’re never going to be able to hide the additional cost. The only way you can make it profitable is by delivering end value to the customer that more than makes up for the premium. If what you’re doing is taking what’s running in the cloud and just moving it into the edge, the cost of moving it is small, but the value you are giving the end user, customer, is very little.

This is why with edge-native applications starting with the business problem is going to be crucial. Let me give you an example. The class of applications I’m most excited about is what I would call “industrial troubleshooting.” When an aircraft that is delayed at the gate costs the airline an estimated $600 per minute. Multiply that cost by the average one-hour delay across all delayed flights, and that’s a multimillion-dollar business problem. (Data has shown that in 2019 alone, delays cost airlines US$8.3 billion.¹⁶) Edge computing and AI with some kind of troubleshooting to find an employee at the gate to minimize the delay and save the airline money and the customer time—that’s the kind of thinking that is going to lead to the most exciting changes in the next few years.

Myke: We’ve talked about manufacturing already where the potential to drive business value is significant and tied to the Department of Defense regulatory compliance requirements, such as NIST SP 800-171. Additionally, in health care, we’re starting to see wearables, IoT, robotics, edge computing, and the cloud come together to manage smart beds that automatically adjust when patients need to move based on real-time analytics at the edge while maintaining that data in aggregate in the cloud for larger diagnostic and personalized health care goals.