Digital reality A technical primer

Approximately every decade since the 1970s, there seems to have been a major transformation in the technology industry. Each transformation is typically illustrated by a leap forward in how we interact with and apply technology and data in our daily lives. The 1970s saw the initial transition from large mainframes in clean rooms to desktop personal computers, where interaction was facilitated through the keyboard and keystrokes. In the 1980s, the introduction of the graphical user interface (GUI) allowed interaction with data through the mouse and the pointer. The 1990s saw the commercial adoption of the Internet, allowing the sharing of data globally. In the following decade, the explosive success of the smartphone led to touch—and later, voice—becoming our primary interface. In this decade, we are moving into the next transformation—digital reality^TM, where our means of interface is expected to be the gesture, our emotions, and our gaze.¹

In this technical primer, we refer to the gamut of augmented reality (AR), virtual reality (VR), mixed reality (MR), 360° video, and immersive experience capabilities as digital reality, except when alluding to the individual technology in its specific sense.² In later sections, we define these technologies and describe their drivers, possible challenges in implementation, market potential, and some industrial applications.

Defining digital reality

Digital reality is generally defined as the wide spectrum of technologies and capabilities that inhere in AR, VR, MR, 360° video, and the immersive experience, enabling simulation of reality in various ways (see figure 1).³

To comprehend digital reality, we need to understand the concepts that it encompasses. While VR enables users to immerse themselves in artificial surroundings that portray actual places or imaginary worlds, AR overlays contextual information on the actual physical environments users see, thus combining digital components and experiences with real life.⁴ MR characterizes the controlled impact of the AR/VR and the Internet of Things (IoT) trends. MR brings together the virtual and real worlds to generate new environments in which both digital and physical objects—and their data—can coexist and interact with one another.⁵ 360° video provides a new perspective that allows users to look in every direction. This is achieved by shooting with an omnidirectional camera or a collection of cameras. Immersive experience creates a multisensory digital experience that can be delivered using VR, AR, MR, and 360° video, among other technologies.⁶

Building blocks of digital reality

Digital reality enables the seamless use of digital information to make choices in real time as we accomplish tasks.

The three key foundational elements that reinforce digital reality (figure 2) are as follows:

• Source of data: The process begins at the point where information is created, taking us outside the territory of pure digital reality and into another connected technology, the IoT. Users of digital reality need to access digital information about the world in real time from various sources via sensors, and communicate that information so that it can be collected, evaluated, and acted upon, which is what the IoT is all about.⁷ Enterprise data—customer relationship management (CRM), enterprise resource planning (ERP), computer aided design (CAD)—can present the user with content and help him/her identify the next action. Analytics, platform computing, and network can also be important components of an overall solution.
• Method of presentation: An information overload can overwhelm users, which could in turn lead to poor performance. The data provided should be sufficient but not excessive, depending on the complexity of the application. Much of the current research into digital reality focuses on how to present relevant digital information in increasingly natural, easy-to-comprehend, contextual ways, so that users can absorb it effortlessly.⁸
• Interaction with and use of data to drive action: Even the right data presented in the right way would be of no use if it doesn’t create value. Digital reality not only enables the display of digital information, but also the control of it in increasingly natural ways. Digital reality takes the comparison of human and machine one step further by enabling us to ponder the question: “What happens if humans and the machines collaborate?” It is fundamentally about making the human-machine team work as naturally as possible.⁹

Exploring the digital reality market potential

In order to understand the reach of the global digital reality market in terms of numbers, we will analyze the market potential of the core AR/VR/MR elements of the digital reality ecosystem. Although the AR and VR markets comprise the overwhelming majority of this core digital reality market, the MR market has begun to demonstrate potential, and may evolve in the years ahead.

Augmented reality market

• The global AR market was estimated at $5.2 billion in 2016, and is expected to reach ~$27 billion by 2019 (see figure 3). The increasing penetration of 4G—and prospectively 5G, thanks to strong government as well as industry support in various countries—are key factors in achieving this forecasted growth.¹⁰
• The market is further divided into three categories—smart glasses, mobile devices, and head-mounted displays (HMDs) used for AR.¹¹ Each of these categories is further divided into software, hardware, and platforms.
• Mobile devices accounted for ~48 percent of the market in 2016, and as the awareness and interest in AR technology increases, users viewing AR content will likely increase with the development of consumer-specific AR apps, and the rise in the number of phones integrated with depth-sensing cameras.
• HMDs comprised ~36 percent of the market and are widely used in health care, aerospace and defense, and commercial and industrial operations.¹²
• Smart glasses accounted for ~16 percent of the market in the same year, with the enterprise sector being the major adopter.¹³

Virtual reality market

• The global VR market was estimated at ~$3.7 billion in 2016, and is forecast to reach ~$13 billion by 2019 (see figure 3).¹⁴ Increasing compatibility of smartphones with VR technology, along with growth in the mobile gaming market, are opening new opportunities for VR headset manufacturers.¹⁵
• The VR market is further divided into hardware and software categories.

  The hardware category accounted for ~85 percent of the total market in 2016, with gaming being the major end-user segment driver. The software category comprised ~15 percent of the total market in the same year, driven by VR-enabled apps in segments including music and entertainment, sports, and fitness.¹⁶

Mixed reality market

• The global MR market was valued at ~$68 million in 2016, and is forecast to exceed $300 million by 2019 (see figure 3).¹⁷ The advent of entry-level VR headsets in the market and a marketing push by smartphone manufacturers have boosted MR market growth.¹⁸
• The market is further segmented into the hardware and software categories.

  The hardware category (comprising HMDs, sensors, motion tracking systems, and haptic devices) accounted for ~66 percent of the market in 2016, while the software market, comprising integrated development environment (IDE), software development kit (SDK), and web browsers, accounted for the remaining 34 percent.¹⁹

• In terms of end-user segments, aerospace and defense, automotive, and entertainment and gaming accounted for over 70 percent of the market, while the health care and retail and e-commerce segments accounted for the remaining market share.²⁰

Key players in digital reality

In terms of key players, the digital reality space can be divided into areas of activity:

• Tools/content—platforms, apps, capture tools, etc.
• Application content—information from industry, analytics, social, etc.
• Infrastructure—hardware, data systems, HMDs, etc.

Increasing investment in infrastructure may drive the growth of software and content, leading to new practical applications and, possibly, an infusion of digital reality software development talent.²¹

Enabling the digital reality market and addressing key limitations to its growth

The expansion of the core digital reality market over the last few years can be attributed to a number of beneficial factors, some of which are discussed below:

• Proliferation of mobile apps along with the increased use of smartphones: The large-scale global adoption of smartphones, coupled with a rise in the number of mobile applications and the development of the application and content ecosystem, has driven digital reality growth.²² The overall mobile app market was estimated at ~$61 billion in 2016, and is forecast to expand at a CAGR of ~23 percent to reach ~$173 billion by 2021.²³
• Increasing use of digital reality in advertising: The use of digital reality for highly immersive advertising and marketing is creating opportunities for users and brands to connect real time with the product, driving an increase in purchases. People’s growing dependence on smartphones is compelling many advertisers to focus on mobile advertising in general, thus enabling the increased penetration of mobile digital reality advertising.²⁴ The mobile AR advertising market was estimated at $1.1 billion in 2016 and is expected to reach ~$20 billion in 2021.²⁵ Companies have also started using digital reality-powered kiosks in stores to market their products.²⁶
• Declining average selling price (ASP) of digital reality hardware: As many large enterprises commercialize their solutions, the ASPs of smart glasses and HMDs are forecast to decline. In terms of presentation of data (as discussed in the digital reality framework), the use of low-cost, energy-efficient micro displays specifically designed for tablets and smartphones could further improve the digital reality market growth. Declining prices of data- source hardware components, including sensors and cameras, are an added benefit.²⁷
• Improved network connectivity: Digital reality devices require high-speed connectivity to function smoothly, and connectivity providers are plugging the gaps in coverage using both wireless and wired technology, enabling them to tap into the higher-frequency spectrum. Fiber backbones are being integrated across the globe to ease bottlenecks to speed.²⁸
• Tetherless access: Thanks to improvements in underlying design in digital reality technology, a new generation of devices is emerging that offers users the benefit of a self-contained experience free of tethering wires or unwieldy battery packs. Such advances may lead to the user “wearing” a digital reality device in much the same manner as a watch or a cellphone. The increase in battery life also contributes to the unencumbered digital reality experience.²⁹
• Capital funding: There is little doubt that deep-pocket players are investing in use cases across the digital reality spectrum. There appears to be much investment activity in such areas as gaming, with one estimate suggesting nearly $10 billion in capital investment by 2021. Other areas of interest include retail showcasing, on-site assembly and safety, process manufacturing training, and travel/hospitality.³⁰

While we have discussed some of the key factors influencing the growth of the digital reality market, we should also be aware of hurdles to the adoption of digital reality applications and evaluate potential solutions.

Applying digital reality to industries

An increasing number of industries outside gaming and entertainment have started testing and adopting digital reality technologies for various applications. Some of the current and potential digital reality-related applications are described below:

Aerospace

• AR devices such as smart glasses allow engineers to provide guidance on complex tasks to maintenance workers in remote locations and at the same time view a real-time and interactive demonstration of the repair job.³⁸
• VR can help address the shortage of skilled engineers for maintenance, repair, and overall operations (MRO) by augmenting local engineering skills with the expertise of more qualified technicians from any location at any time, potentially reducing training time, improving maintenance efficiency, and enabling huge cost savings.³⁹
• Aircraft component manufacturing companies are developing MR applications, with which the position of engineers installing equipment inside aircraft fuselages can be mapped to a full-scale 3D digital model of the aircraft they are working on. This enables engineers to instantly bring up an image of an aircraft component installation and can help them ensure that they have fixed it correctly.⁴⁰

Defense

• AR is being extensively used by the military for more effective training of soldiers.⁴¹
• Smart helmets allow commanders to send maps and other information directly to the soldiers’ field of vison, enabling them to perform their missions with enhanced safety, speed, and coordination.⁴²
• MR applications in the future may involve drones small enough to fit into the palm of a soldier’s hand. The drones can then track and analyze a soldier’s movements under a training simulation and real-world combat situations.⁴³

Automotive

• AR often plays a crucial role in vehicle design. Manufacturers can use the technology to compare virtual data with the actual vehicle design, perform analysis, and, on that basis, potentially alter parts during the initial stages of design planning. In this way, the technology can provide manufacturers with an exact picture of a car’s future performance, possibly reducing time and cost of launch.⁴⁴
• Potential applications of AR, such as projecting navigation routes and distance between the car and road obstacles or pedestrians, may be displayed on car windshields instead of traditional dashboard screens, allowing the driver’s vision to stay on the road.⁴⁵
• Carmakers are also experimenting with MR technology that enables them to combine and visualize physical vehicles and computer imaging to improve design and speed up development.⁴⁶

Health care

• VR can potentially enable surgical specialists to treat patients remotely using a remote-controlled robot. Using an HMD and haptic gloves, a surgeon could virtually perform a surgery thousands of miles away.⁴⁷
• MR is being used for the preoperative preparatory phase of surgical procedures, where surgeons can plan the entire operation using 3D holograms, decide on where to make incisions, and also see the consequences of their moves.⁴⁸

Consumer products and services

• AR allows online shoppers to view products such as clothes, furniture, and other items in 3D in a real-life environment, in real time through their tablets or smartphones before purchasing.⁴⁹
• Travelers can explore scenic places, cities, or hotels in a travel destination through a VR headset.⁵⁰

Key considerations for organizations

As companies extract value from information gathered during the execution of daily tasks, sifting through voluminous amounts of data for a complex operation may soon surpass the competency of most people.⁵¹ Digital reality would thus become increasingly essential for processing this data as well as dealing with the variability of the task (when each iteration of a task is different). Digital reality technologies offer various prospects for enterprises to transform areas such as internal workforce communication and collaboration, workforce training and simulation, and customer service.⁵² The combined effect of factors, including declining hardware and software costs, increased smartphone usage, and creation of app ecosystems, have led to the proliferation of digital reality applications in both consumer and enterprise segments as discussed earlier.⁵³

With the advent of digital reality solutions, enterprises may need to upgrade their infrastructure by installing sensors and beacons, among other systems, to facilitate augmented setups. They may also need to improve connectivity—in the form of wired, wireless, and cellular—to cater to digital reality requirements in remote locations. Companies should consider investing in new tools and services such as high-definition 3D image capture, mapping equipment, and high-end gaming engines, to recreate simulations and virtual environments for digital reality interaction.⁵⁴

Additionally, change management can be essential for organizations to train their workforces and make them technology-ready. The use and adoption of such digital reality tools as smart glasses and HMDs could position organizations to take advantage of digital reality applications to increase the effectiveness of training and enable production efficiencies.⁵⁵

Simply put, organizations can use digital reality capabilities to their full potential to streamline their workflow, drive collaboration and productivity, and enhance the customer experience—and, in the process, perhaps build brand value.

Final thoughts

The application of digital reality today appears to go well beyond the consumer segment, with a reach that extends across the enterprise world. Accordingly, businesses should consider investing in infrastructure and developing new operating models to improve connectivity and fast track the adoption of digital reality technologies. Indeed, digital reality is swiftly progressing from hype to reality, as is evident from the market growth realized over the years, offering a plethora of benefits to organizations. The time seems apt for the full array of business professionals—from product planners to strategists to marketing and beyond—to understand the disruptive potential of digital reality and put the technology to use.