Digital reality in government How AR and VR can enhance government services
24 August 2018
As governments are pressured to offer more services with fewer resources, digital reality may help. Augmented and virtual technologies could transform how governments use data, help raise employee performance, and more efficiently offer improved services.
Introduction
A government agency faced a challenge. We'll call it “Agency A”—but in the era of constrained budgets, it could be any government organization. Budgetary pressures had reduced its workforce; in 2017, it had shrunk to its smallest size in more than five decades. At the same time, it was demanding more from each employee than ever before—say 3 percent more every year over the last decade. The result was that Agency A had to find a way to get more out of each employee if it was to continue to accomplish its mission.
Learn More
Read how VR learning could improve training outcomes
Explore the government and public services collection
Subscribe to receive government and public services content from Deloitte Insights
In 2017, Agency A found an innovative solution: digital reality. Digital reality is a set of capabilities—augmented reality (AR), virtual reality (VR), mixed reality, 360° video, and immersive technologies—that seamlessly present digital data to humans in natural ways (see figure 1). Once the realm of science fiction, digital reality has quickly matured from experimentation to industrial applications to user-friendly smartphone apps. Agency A developed a digital reality pilot program, using telepresence to link experts with local site technicians performing maintenance on equipment. Experts could now see exactly what local technicians saw and talk them through complicated problems. By connecting talent across the organization, Agency A reduced equipment downtime and improved on-the-job training of technicians, who could leverage digital assistance from experts to enhance their own skills. It also dramatically increased the reach of its experts, who could now assist technicians in Florida, Alaska, Maine, and Hawaii all in a single day.1
Agency A’s experience highlights some potentially good news for government: Digital reality technologies can help workers achieve performance gains once thought impossible. In the private sector, the increasing performance and falling costs of these technologies are helping many companies reach new heights in maintenance, supply chain management, and manufacturing. But government organizations can face unique challenge when adopting advanced technologies, and may need to develop their own paths forward. This article is designed to help government leaders understand whether digital reality may be right for them, and how to navigate the financial, organizational, and technological hurdles on the road to greater productivity.
Unlocking productivity through digital reality
There is only so much information that humans can ingest, process, and draw upon in their work. But in a data-driven organization, workers need to access, analyze, interpret, and apply volumes of information fast enough that it remains relevant to their needs. Digital reality could be the key to overcoming this challenge: it gives workers tools to interact with digital data in the real world. Workers can connect with each other, query information, and see, feel, and interact with data as they perform tasks. And it seems to work: Initial studies and use cases have shown that digital reality can help agencies build immersive experiences that increase productivity, improve the efficacy of training, and improve government services.2
But there is typically a difference between seeing the benefits of a new technology and applying it successfully in a large organization. How should leaders think about deploying these technologies? There are millions of tasks government agencies perform every day that could be aided by digital reality, and they tend to fall into three categories: know, connect, and learn. Understanding which tasks fall into which category can help leaders determine how best to apply digital reality to execute their mission.
Know
Digital reality can enable workers to perform standard tasks faster and more accurately by providing them with the right information at the right time. For example, digital reality can improve maintenance tasks by making manuals or product locations immediately available in an employee’s field of vision. This knowledge-assistance function is already being used in a Siemens pilot that allows maintenance personnel to access manuals via AR.3 Workers are able to pull up computer drawings and repair instructions for the exact part they are looking at, making several thousand pages of information immediately available. Similar use cases have shown that eliminating even the seemingly minor inconvenience of glancing back to a book and flipping pages resulted in a 34 percent increase in speed of accomplishing a task.4
Knowing is about accessing information. Imagine a first responder arriving at the scene of a disaster. Wearing augmented reality glasses, she can overlay the blueprints of buildings and infrastructure onto her field of vision. This could help her navigate risks and find access points. With her glasses, she can see through debris, fire, or floods and develop a safe plan to rescue stranded citizens.5 Or, take the case of a security officer at an airport, who can identify travelers as they approach check-in and assess their threat profile. With an AR view of the entrance hall, the officer can identify known threats and suspicious behavior.6 In the crowded space of a terminal, digital reality can highlight behaviors that even well-trained security personnel might miss.
Connect
Digital reality can overcome shortages of key staff or the challenge of distance by connecting resources with needs. Telepresence, for example, uses voice and video links to enable an expert to provide “see what I see” advice remotely. Immersive cloud-based programs can also seamlessly connect team members across functions, allowing for entirely new forms of service. In a notable research program, a VR computer system helped veterans tackle post-traumatic stress disorder and enhanced their job interview skills. A human resource representative named “Molly Porter,” a VR avatar, interacted with veterans in a mock interview, customizing her conversation based on answers she received from the candidates. More than 90 percent of program participants found the training useful and more than 80 percent said it gave them confidence.7
Connecting is about bridging gaps—enhancing the reach and impact of sources of knowledge. Imagine if a soldier injured during an operation didn’t have to wait for hours or even days to see a leading medical specialist. Instead, a paramedic could immediately connect with specialist doctors via VR, to get advice on the best treatment. The paramedic could show the real-time ground truth of the patient, link up vitals, and actually see what the specialist is suggesting. VR could help the paramedic eradicate the challenge of distance, and the result would be better outcomes for patients.
Learn
Digital reality can also improve the effectiveness of training and, especially for dangerous or rare tasks, make it less costly as well.8 To improve the learning experience in its training program, the US Department of Agriculture’s (USDA’s) Food Safety and Inspection Service (FSIS) began using VR to transform a traditional classroom course on public health veterinary. It devised engaging, immersive solutions for students, such as the opportunity to view an inspector’s work setting in a 360-degree VR interactive training environment.
Digital reality can also help recruitment by allowing applicants to experience a real day on the job. The same USDA division was having trouble hiring and retaining staff since some inspectors and veterinarians could not handle some of the tasks related to work in a slaughter plant. In response, FSIS created a 360-degree immersive experience to show candidates at job fairs what their work setting would look like. “About 20 to 25 percent of candidates said they would not work in a slaughter plant,” says Dean Norman, the former distance learning manager of FSIS. “That’s exactly the reaction we want. If you don’t think you can do it, it is better to know now than after we have spent the money to train and relocate you.”9
The use of technologies like VR can also help personalize training for participants. Senseye, a sensory human interface startup, is working with a defense organization as it redesigns its pilot training. The process of learning through VR can generate thousands of data points by tracking cognitive load levels, stress levels, and the ability to think and plan ahead. “We essentially built an AI model for them to know exactly what’s going on in a pilot’s brain. We know exactly what their strengths and weaknesses are. The AI will build a custom syllabus for each pilot based on what’s going on in their mind,” says David Zakariaie, Senseye’s founder.10
Potential barriers to adoption for government organizations
Digital reality is growing across government. Defense agencies have been using a combination of digital reality technologies to train soldiers and medical professionals, and the Pentagon is reportedly looking to budget as much as US$11 billion for VR, AR, and mixed-reality training systems by 2022.16 Civilian agencies such as the USDA are also using digital reality both for speeding food inspections17 and hiring the right kind of talent.18 But the experiences of early adopters in government appear to point to some key challenges for agencies. In particular, these initial explorations have highlighted four primary considerations for government organizations adopting digital reality: 1) cybersecurity and privacy, 2) legacy systems, 3) conflicting requirements, and 4) financial sustainability.
Cybersecurity and privacy
Of paramount importance to any major information system implementation is an assessment of the system’s security strengths and weaknesses. This is important for government agencies that must protect sensitive data of both the public and their own workforce.
Digital reality can also create privacy challenges in how technologies interact with users, and in what types of data the technologies interpret. First, in cases where digital reality technologies actively assess how users are reacting to new experiences, users’ eye movements, pupil dilation, and other reactions are tracked to generate data points to improve future programs. This data needs effective controls to protect the privacy of users. Second, government agencies often handle the private data of citizens. While digital reality technologies can be used to hasten processing of data and enhance citizen services, any solution dealing with such tasks should protect both the privacy of workers and of those whose data is being processed.
Balancing these cybersecurity and privacy concerns against costs is often the underlying driver in decisions on where and how to host digital reality solutions. As technology continues to evolve, organizations should monitor the development of new security standards and solutions specific to digital reality that can help them strike the right security/cost balance.
Legacy systems
Digital reality can help workers process large amounts of data through intuitive, interactive experiences. But processing such data typically means being integrated with other types of software. In government, that can mean integrating with a multitude of legacy systems. Getting legacy platforms and applications to communicate with each other is challenging enough before adding digital reality. For example, agencies can gain enhanced insights by overlaying their treasure troves of data onto AR platforms,19 but many government organizations have struggled to integrate disparate inputs to make sense of their data.20 The US Army, for example, operates many different simulators to prepare troops for a variety of different vehicles and tasks. Altogether, there are 300,000 individual items that can be used in these simulators, ranging from weapons and protective gear to aircraft controls. With many of these simulators dating back to the 1980s, it can be difficult to coordinate the simulators to give troops an accurate, complete training experience. When the army ran a simulation for its helicopter crews and anti-aircraft gunners together, it turned out that they weren’t able to operate in the same virtual space.21
Conflicting requirements
For emerging technologies like AR and VR, achieving buy-in from stakeholders can be a challenge. The reasons can range from a lack of understanding of the benefits to conflicting priorities. A survey of US Department of Defense personnel on virtual training ranks lack of understanding as the second biggest barrier to the adoption of virtual training.22 Stakeholders may misunderstand the functions of the technology, but disagreement can also come from misunderstanding the operational goals of digital reality projects. A former commanding officer of a defense agency training center explains that opposition to virtual training often emerged from miscommunication about the goals of digital reality training across different stakeholders. While the top leadership “gets” the importance of increased virtual training, there is more resistance “where folks are afraid that the concept of live, virtual, and constructive training is to take away all of the flight hours and all of the sailing hours.” He added, “Some folks may have seen it as, well, this is just a raid on flight hours, but that is not the case at all.”23
Financial sustainability
The ultimate issue that can sink efforts to update government technology solutions is the challenge of ensuring consistent project funding, from startup costs to ongoing operations. While digital reality technologies can often save money once implemented, difficulties in budgeting can cause even the best use cases to go unfunded. For example, while VR training can often save money or improve outcomes compared to traditional training, the costs of implementing such a solution may rest with a training department while the benefits may accrue to an operational department.
Creating a new reality for government work
Overcoming these potential barriers could be key for government organizations to successfully tap into digital reality technologies and increase productivity and efficiency. Luckily, possible solutions exist for each of the four obstacles (see figure 2).
Cybersecurity: Balance security with costs
Cybersecurity and privacy are nonnegotiables in digital reality solutions for government organizations. More security is always available, but generally at the cost of more money, longer development timelines, and greater complexity. Therefore, balancing these critical factors against the cost and complexity of a solution is one of the central decisions any organization considering digital reality faces.
Fortunately, precertified cloud solutions can help to address this trade-off. While security and privacy issues can impact every aspect of a solution, from hardware to user identity verification, often the most critical node is the large volumes of data stored and processed on servers. This creates one of many key decisions government leaders face, whether their security/cost balance is best met via a cloud-hosted or on-premises server setup. For example, the Federal Risk Assessment Management Program (FedRAMP) is a framework designed to help organizations managing data in cloud-based solutions assess and mitigate risks. Through the use of low-, moderate-, and high-impact levels, organizations can choose to control their data at different security baselines that correspond to relevant government information requirements. The FedRAMP classification system lays out standard requirements using auditable controls, meaning that government organizations can be sure that any certified cloud provider can meet their security needs without costly individual development.24
But cloud is not the only solution for digital reality. Some organizations may prefer an alternative, such as a hardened, local solution. Colloquially known as “on-premises,” these solutions stay within the firewalls of an organization and never traverse the larger net to connect to a cloud server. Organizations may want to assess their operational needs, risk profile, and cybersecurity strategy to determine whether on-premises installations of new technologies may be a solution.
Legacy systems: Modernize systems using API-based architecture
Integrating digital reality and modernizing legacy systems requires long-term commitment, but solutions exist to help overcome the challenges. Application programming interfaces (APIs) provide an answer to extracting key data across systems. API-based systems are a more discoverable, shareable, and usable approach to integration than legacy systems.25 APIs help modernize systems by drawing important data and applications from underlying legacy systems. Agencies can create an API infrastructure that enables the linking of data, applications, and systems even when those systems are older legacy systems not designed to work together.26 By using API-based solutions, systems can be upgraded and integrated to enable digital reality applications for government organizations.
Conflicting requirements: Run a pilot program to get buy-in from stakeholders
People are the center of any successful technology solution, and digital reality is no exception. A well-executed project plan, driven by organizational stakeholders and guided by external experts, means a far higher likelihood of success for a project. A project champion within an organization, who shares the vision for a digital reality solution, can bring together stakeholders and generate buy-in for a successful deployment. Simply passing down a policy directive from a higher authority will likely not achieve the needed consensus for digital reality to succeed.
Pilot programs can help organizations achieve real goals and demonstrate the value of digital reality. A carefully developed pilot program, with a clear goal and under the supervision of a project champion, can address system integration and user acceptance challenges that will enable digital reality to succeed.27 Organizations should begin by finding a high-value use case, starting small, iterating to solve problems, and scaling fast.28
Financial sustainability: Try small-scale projects supported by strong leaders, and track the expected ROI
Digital reality solutions require consistent project funding but can drive real returns, through improved workforce performance delivered on a reduced overall training budget. Governmental budgeting processes can pose a challenge to technology projects, especially when they involve reallocating resources across agencies, so strong department leadership may be needed to bring together relevant stakeholders. But once digital reality solutions are procured, sustainment costs can be shared across the beneficiaries of the solution. To overcome common challenges, it’s important to follow the start-small, scale-fast project framework29 and rigorously track program metrics, including a relevant ROI metric that compares both costs and impacts of digital reality solutions relative to traditional programs. Connecting an operational need to financial benefits is often helpful here: Letting internal stakeholders see how digital reality drives real savings and improved mission capacity empowers people to push the solution forward.30
Although challenges remain on the path to widespread adoption, digital reality has the potential to transform the way government uses data. With its variety of applications and ease of use, digital reality can help to improve the performance of employees in the workplace, delivering better services to citizens for less.
