Augmented reality and 3D technologies are changing how customers shop both online and in-store. How should retailers prepare?
In late November of 2019, the CEO of Levi’s boldly stated that in 10 years there would be no more sizing—soon we would replace this aging retail artifact with custom fit apparel, enabled by body scanning and made-to-order finishing.1 This potential sea change from valuing efficiency for the wholesaler toward complete customer satisfaction could have deep effects on the manufacturing and supply chain, and similar trends have been noted across many industries. For retail, it could also fundamentally shift what will be needed to merchandize products. In this new and quickly emerging world, augmented shopping has a unique and important role to play.
Explore the consumer products and retail collection
Download the Deloitte Insights and Dow Jones app
Subscribe to receive related content
Imagine a future where your customer no longer shops by perusing a catalog of products, but instead they select a product type—be it a pair of pants, a car, or a dining room table—and then interactively dial and swipe their way to achieve the perfect dimensions, color, and style of whatever they are looking for. They could then try on, try out, or interact with it in any environment they might want—all from the comfort of their own homes. This vision is what augmented reality (AR) and 3D technology are turning into reality.
It’s a revolution unfolding less like a bang and more like a slow but powerful shift in tectonic plates. However, that isn’t cause for complacency. Today, more than 1 billion smartphones and tablet devices can deliver augmented experiences,2 and within the next year, 100 million consumers are expected to shop using AR either online or in-store.3 The consumer appetite is increasing—since 2018, the number of mobile AR users has nearly doubled, with usage and popularity driven largely through social media.4 In 2020, 46 percent of retailers plan to deploy augmented or virtual reality (see sidebar, “What is augmented shopping?” for brief technology definitions).5
The trends are clear, but we are still in the early days of this revolution. Deriving value from augmented shopping today depends largely on how well the technology can represent the most meaningful product attributes for customers in context. Because those differ by retail category, we expect to see not one, but many tipping points as augmented shopping becomes the standard shopping experience. In this paper, we discuss how the technology is expected to advance for various retail categories, which challenges are decreasing, and how retailers can begin to make strategic decisions about how and where to apply augmented shopping across the customer journey for meaningful results.
Augmented reality (AR): Overlays digitally created content into the user’s real-world environment; features include transparent optics and a viewable environment in which users are aware of their surroundings and themselves.
Augmented shopping: Enables customers to engage with brands and products via digital experiences that allow them to try on, try out, interact, or personalize their product virtually; these experiences help deliver more detailed, intuitive product information than standard web experiences.
Virtual reality (VR): Creates a fully rendered digital environment that replaces the user’s real-world environment; features body- and motion-tracking capabilities.
WebAR: Refers to AR experiences delivered via a web browser, without the need to download a third-party application.
3D assets: Three-dimensional digital models of physical products that users can interact with in AR/VR experiences; the process by which products are turned into digital models is referred to as the 3D asset pipeline (for more details, see sidebar, “3D assets: A primer”).
Regardless of any possible increase in consumer expectations, Deloitte’s research has also shown that most consumers today still prioritize value, product, and convenience while making their purchasing decisions.6 They want to feel confident that they have found the right product, for the right price, and that it’s not too hard to obtain. What’s more, these attitudes seem to have held steady for generations of consumers in the United States, no matter what technology or social trends we may be in the thick of.7 This can provide an important foundation as retailers look to integrate any new technology into their customer journey.
Today’s shopping experiences are happening in many different contexts: at-home, on-the-go, and in-store. Delivered effectively, augmented shopping can help customers feel more confident in their product selection—whether through richer product information or interaction, the ability to personalize or configure unique product designs, or via the deeper emotional connection from immersive brand storytelling.8 The most successful experiences today tend to be delivered for retailers that have highly standardized product sets with high opportunities for customization (figure 1). For example, automotive retailers have a standard set of models, but the opportunity to customize and configure the models for each customer is vast. Augmented shopping provides a new opportunity for customers to see and interact with unique products like never before. The more customizable, the more valuable these experiences become to build confidence during the shopping experience.
However, the ability to deliver value with this technology also typically depends on how well it can render a product’s most important attributes in context. “When we think about AR shopping, it’s not just about seeing what something looks like in your space and seeing the size of it, it’s also about seeing the materials and the details of that product,” says Daniel Beauchamp, head of VR at Shopify. While size is typically most important for furniture, for example, smaller details—like texture or sheen—may be more important for others, like clothing. “For products where those details do matter,” continues Beauchamp, “where there is craftsmanship in terms of the materials being used, it’s really important that those materials get portrayed very realistically.” And that can be harder to do with the scale, technology, and cost challenges that retailers juggle today.
In early 2020, getting started will likely mean a commitment to experimentation—one that many companies value as a part of their brand story. For example, Bose decided it was important to deliver new ways to engage customers throughout the buying experience. “We were convinced that the technology had matured enough to deliver a decent customer experience—and I would say decent, not perfect,” says Clemens van de Ven, director of global e-commerce at Bose, “but now [the technology] is moving at a decent speed, and so we tried this to learn from it. And for the customers who have tried it, the experience adds value.”
Three augmented shopping experiences are common today: “Try On” for worn products; “Try Out” for products that can be placed in your environment; and “Interact” for interactive products, like electronics (figure 2). Configuration refers to added features that can be offered for any product type; this combines user-selected attributes and parts that complete a product, and personalization, the addition of personal touches such as monograms. Figure 2 provides a general reference for how well, as of early 2020, the technology is expected to deliver a satisfying augmented shopping experience for each product type.
To better understand how various retailers are exploring and finding value with this technology today, we looked more closely at the following product categories:
Home goods and furniture. Home goods as a category will likely see significant disruption because of the plethora of choices and the large opportunity for personalizing various colors, patterns, and styles of products. Retailers in the furniture space were one of the first product categories to find widespread and measurable success with AR. Once retailers could accurately render the most important information about the product—color, size, and fit within a given space—and allow customers to reliably visualize furniture in their room, customers bought into these experiences quickly. Returns on investment have been demonstrated with increased conversion9 and reduced returns.10 Now, it is rapidly becoming the furniture industry standard, where customers have begun to expect that retailers offer virtual product placement in their homes.
“This is where the future is going, and in three years, if you don’t have AR in furniture, you’re going to be looked at as old-school. We’re not waiting. We want to be ahead of all that.”—VP of digital marketing and omnichannel experience, furniture retailer
For example, when a national furniture retailer was looking for ways to update its e-commerce experience, they decided to integrate it with the launch of their first AR app. It allowed customers to see how a piece of furniture would fit in their home and then place orders without having to visit a physical store. In the first four months after the app’s launch, the company measured 100,000 downloads, and have since watched as their customers collectively use the AR experience 1,000 times per day on average, achieving conversion rates between 65–69 percent within the app. Reportedly, as a result of this integrated e-commerce initiative, the company saw a double-digit increase in online sales during 2017–18, and more than 20,000 deliveries scheduled online in a span of 8 months.
Beauty and makeup. Virtual makeup try-on is following quickly behind furniture in augmented shopping maturity. After all, when it comes to beauty and makeup sales, getting customers to imagine a new look or color on themselves is a critical element of the buying process. Beauty retailers are pushing hard to develop new technology that can better match color digitally. However, complicated technical challenges such as facial recognition and color fidelity haven’t stopped makeup brands from experimenting, and many are now regularly delivering valuable experiences to their customers.
“We now consider virtual make-up try on to be the base of any experience. At the end of the day the only barrier to buying [a product] is wondering what it will look like.”—Lubomira Rochet, chief digital officer, L'Oréal.11
Launched in 2019, L'Oréal’s recent AR makeup experiences have shown early results, doubling website engagement time and tripling conversion.12 Sephora has also found similar success: Between 2016–18, their Virtual Artist app saw 200 million shades tried on during the more than 8.5 million visits to the AR feature, and they continue to see repeat customer use.13 Sephora executives emphasize that with any digital effort, it is important not to introduce technology because it's cool or new. Any digital product, including augmented experiences, should pass the test of making shopping more fun and efficient for their customers.14
Automotive. Much like furniture showrooms, car dealerships don’t have the space to display their full range of products and configurations. AR sales experiences can help to fill this gap, which is why a large number of the major original equipment manufacturers (OEMs) are experimenting and offering AR experiences to their customers today. From in-store experiences like Audi City15 to Toyota’s experience allowing customers to virtually place and interact with models by clicking on a banner ad,16 these experiences are designed to help customers configure and better understand each vehicle’s parts and features, hopefully building deeper product knowledge and thus confidence in their purchase decision.
Shoes. Shoe companies are primarily using augmented shopping capabilities in two ways: personalization and virtual try-on. With the higher level of product standardization of shoes, both try-on and personalization tend to be simpler technical challenges than they are for other apparel items. Wannaby, a virtual shoe try-on app and startup, is working hard to improve the tracking and realism of virtual try-on experiences because, according to CEO Sergey Arkhangelskiy, “We believe that AR try-on can help customers to shop online and will wash away the difference between online and offline shopping.”17 However, the challenge of try-on for shoes is more than just how it looks—it’s about fit and comfort. Other app developers are experimenting with more accurate virtual size measurement by taking data points via the mobile camera on the customer’s actual foot measurements.18 Personalization is becoming more popular in general—in 2018, 26 percent of US consumers had purchased a personalized product,19 and a variety of shoe companies are now using 3D configurators to hand over part of the design process to their customers.20
Clothing. While it may be easy to imagine the benefits of virtual try-on for fashion, accurate fit and complex features, like the drape of varied fabrics, can be particularly difficult to render effectively right now. With a shorter shelf life of products and a higher demand for realism from customers, it has been hard for many clothing retailers to create an affordable 3D asset pipeline for their product lines (learn more about 3D asset creation in the next section). And of course, there is the added complication of accurately and conveniently getting either a 3D model of the customer’s body or a “good enough” approximation to deliver the information that the customer is looking for. This is why many clothing retailers have taken a variety of approaches to create augmented experiences for their customers—some choosing to avoid the more difficult technological challenges of virtual try-on, and some experimenting with workarounds or approximations.
Zara addressed this challenge by launching an in-store augmented experience. Using the Zara AR app, customers can hold up their phone to designated shop windows or sensors within the store to see models wearing a selection of outfits in clips of 6–12 seconds.21 This avoids the challenge of fitting clothing to the customer’s body, while helping the shopper see how the piece fits and moves. Gap, on the other hand, attempted to solve the problem of body fit with their online shopping experience by developing an AR app where customers could select one of five body types, and enter their height and weight to create an approximate model of themselves.22 Tilly’s, a clothing and accessory store for teens and young adults, took another approach that avoided the challenge of fit altogether, by creating an augmented experience in their store window. The company reported that the interactive in-store experience boosted customer traffic by 30 percent.23
Almost overnight, the number of customers who carry in their pockets the technology needed to view AR experiences exploded with the releases of Apple’s ARKit in 201724 and Android’s ARCore in 2018.25 Today, standards-setting organizations such as the Khronos Group are working quickly to pave a smoother path for 3D to be used portably across multiple platforms, including WebAR, the ability to view AR via the web without an app. The Khronos Group created the WebGL standard for accelerated 3D graphics on the Web, and an asset standard for 3D (called glTF), which provides an industry-wide industry foundation for any application or browser to download and render 3D products—including AR shopping.26 This will allow any browser or application to render AR and ease yet another challenge to experience delivery.
“I think retailers have enough experience with the existing technology to know they can deploy AR, and it’s proven to improve sell-through and reduce return rates. This is why there is urgency seen from retailers to get those millions of products available in 3D and seamlessly, efficiently flowing through to end-user experiences.”—Neil Trevett, president, the Khronos Group
Even with standards in place, retailers will need to find efficient and cost-effective ways to develop and use 3D models of their products—the foundation for any AR experience. To date, getting photo-realistic models has not only been technically difficult, but has been expensive, with costs ranging from US$100 to US$800 or more, depending on the product’s complexity (see sidebar, “3D assets: A primer”).27
3D models are the foundation for any AR/VR experience. There are three leading processes for developing them: photogrammetry, 3D scanning, and 3D modeling; all have pros and cons based on product attributes, retailer needs, and budgets.
Photogrammetry combines many overlapping pictures (10–250+) of the product from every angle, and then digitally reconstructs and computes the data to make one model. This works well on most products but currently has challenges with metal or glass objects that reflect or absorb light. This process can be cost-effective depending on size and complexity (for instance, professional scans are available under US$400 for a footwear product representation)—but the resulting model is less flexible for reuse than models made by an artist.28
3D scanning scans objects’ shape and measurements with millions of data points to create a dense vector point cloud. Geometry and shape can be determined but textures, colors, and lighting might not be represented that well in a photo-realistic environment. Future advancement in scanning technology with multiple scans of light, lasers, and infrared point clouds will combine with photogrammetry textures for more accurate color product representations.
3D modeling refers to the manual creation of a model by a 3D artist. This is usually the highest quality, but it can be time-consuming and expensive (for instance, costs frequently exceed US$750 for a footwear product representation).29
However, many retailers have found a surprise benefit to investing in 3D models of their products: significant cost reductions for product catalogs and prototyping, compared to using physical models and photography. “It’s been a pretty big unlock in terms of imaging our catalog, which is extensive,” says Jane Rawnsley, head of Imagery Creative at Wayfair. “I don’t know that we would have invested in [3D imagery] the same way if we continued creating our catalog physically only.”
When creating a catalog using traditional methods, the photography, warehousing, and storing of products can be expensive, time-consuming, and labor-intensive. By experimenting with photo-realistic 3D models to create 2D images, companies such as Wayfair have found that they are able to both reduce the overhead associated with traditional catalog development methods, while creating the models they need to deliver augmented shopping experiences for their customers. Using similar processes, Coca-Cola transformed its traditional prototyping process for package design, reportedly saving 33 percent of the cost of a project’s visualization phase.30 Elizabeth Barelli has seen this from her perspective as senior product marketer for AR and 3D at Adobe, stating, “Businesses are quietly revolutionizing the way they produce because of 3D technology, and that can happen at different levels of the production pipeline, before and after production.”
“If you zoom out a little bit, this is visual computing, and it’s all driven by 3D content, which will show up in multiple different ways. It’s going to take a few years, but we will get to the point where you cannot merchandize a product without it.” S—Shrenik Sadalgi, head of Next (Wayfair's in-house R&D lab) and chair of the Khronos Group’s 3D Commerce Working Group
With so many retail companies feeling more pressure than ever to demonstrate profitable growth, it can be difficult to make deep investments into what now feels like the long game of an emerging technology. Augmented shopping depends on a mix of established and emerging technologies today, but the trends seem to demonstrate that 3D technologies are being adopted by industry and consumers alike. A few product-centered use cases have already tipped and now demonstrate clear return on investment.
“People ask about what is going to be the big leap. I think actually it’s going to be a set of incremental forward steps that brings us towards the end state where AR [hardware] will replace mobile phones, and it will have a much bigger impact than we imagine.”—Neil Trevett, president, the Khronos Group
The future of shopping begins now. While this technology is developing rapidly, it's not something you should wait for and then plug and play. As you get started on this path, it’s important to consider some key principles:
Given the complexity of technologies behind augmented shopping, companies shouldn’t delay exploring its benefits and building their knowledge in this area. If you are not discovering the many ways AR and 3D technology can change retail today, you may find yourself lagging before you know it.
Digital reality represents the next digital transformation. It changes how we engage with technology, through augmented-, virtual-, and mixed-reality, 360-degree video, and immersive experiences that are at once intuitive and data-rich, and which put the human user at the center of design.
Learn more
