Augmented reality and 3D technologies are changing how customers shop both online and in-store. Deloitte's Allan Cook, Shopify's Daniel Beauchamp, and Wayfair's Shrenik Sadalgi give host Tanya Ott a tour of the future.
Shrenik Sadalgi: Actually, you know what? Maybe.
Tanya Ott: It’s a familiar scene. You’re shopping for something online and you find something you like, but you’re just not sure how it will look in your space.
See more from the Insights In Depth podcast series
Explore the Consumer products & retail collection
Learn about Deloitte’s services
Go straight to smart. Get the Deloitte Insights app.
Daniel: Oh, it has to be this one right here. What do you think of that?
Tanya: When we buy something online, we take a leap of faith that it’s going to fit and it will look good.
Shrenik: That’s a good color. But that one really works in our space. I can clearly see it in the background. It really matches the rest of the space. And it fits well too.
Daniel: It comes in the blue or the green? Here, let me just toggle between them. So, this is green and this would be blue. I think, the green’s pretty nice. What do you think?
Shrenik: I think the green really works.
Tanya: Size, color, fit—it doesn’t have to be a guessing game anymore.
Daniel: Let’s get it.
Shrenik: It’s only $200. Let’s go for it. I see a buy button right there. It looks like it comes in two days, so let’s do it.
Daniel: Can’t wait!
Tanya: Today, we’re going to tell you all about the technologies that take the guessing out of buying things online.
Tanya: I’m Tanya Ott, and this is Insights In Depth, where we take a deep dive into the issues that matter to business today. And those guys shopping for a couch? Those are our guides today.
Daniel: I’m Daniel Beauchamp. I head up AR and VR at Shopify, so my team looks at how these new spatial technologies will impact commerce and we look at bringing that to over a million merchants on our platform.
Shrenik: Hi, my name is Shrenik Sadalgi and I run a small team at Wayfair called Wayfair Next. Our job is to figure out what’s next in home and retail. And what that means is trying to look at what new technologies are emerging on the horizon and figure out what would be useful to improve our customer experience and solve customer problems and make it happen. It’s a very small investment at Wayfair. Think of it as a small R&D lab or a skunk works lab and it’s called Wayfair Next.
Allan: Allan Cook. I am one of the leaders of the Deloitte Digital Reality practice. So that’s all things AR, VR, MR, 360, and spatial. We’ve been focusing on this area as one of our next generation of exponential technologies, which we see the future of a lot of our business is now doubling down and investing in. We have over 100 dedicated full-time employees now working on this and are starting to see significant growth amongst our clients across the industry, but particularly in retail.
Tanya: We have just met, but I’m going to tell you something very personal about my relationship with my husband. Now, we’ve been married for like 29 years. Together, longer than that. And we get along really well. We love spending time with each other. But there’s one area in our life where we have a ton of conflict and that is home decorating. He actually has opinions and I do, too. And they rarely align. So we have had some knock-down, drag-out arguments about how high or how low to hang art on the walls, how big or how small we can go with a piece of furniture in the room, what size the rug should be. It is basically crazy, stupid. It sounds, though, like you guys have spent so much time working on this and you might have the solution for us.
Shrenik: It seems like that’s a problem that everyone has. That’s a common theme in everyone’s home, the discussion about what to buy. We’re hoping that we give you a very objective view of what something would look like in your space. That’s one of the reasons why augmented shopping works. It provides information in context, so you can see this chair literally in your space of the right size and you are able to make a much better decision about whether that looks correct or not for you.
Tanya: I want to dig deep into this, but before we do, it’s probably important for us to define some terms for our listeners who may not be familiar with it all. Allan, can you explain for us the difference between 3D, like a 3D model, and augmented reality? We’re going to call that AR and virtual reality, VR.
Allan: Absolutely. Virtual reality is when your entire environment is either computer-created or it’s extreme 360 video. It’s the more heavy-set glasses where everything you see is being generated by the computer. You can move around in these spaces and you have 360 degrees of freedom, so as you move your head your perspective changes. And it’s a technology that’s actually been around for a long time.
Augmented reality or mixed reality, on the other hand, you can still see the environment that you’re in. This then allows you to actually place these objects—the 3D objects—into the environment. And there have been some popular app games which allow you to place the 3D monsters and characters into the environment where you were trying to catch them. AR and MR are really beginning to dominate right now, particularly in this space and across the enterprise side of the business.
360 video, on the other hand, merely is just trying to capture spaces in 360, so you’re not having to create models of them. You can have a streaming environment, or you can have a fixed environment if you want to do, say, a tour of a showcase home to see all the furniture in situ.
And then spatial is you don’t have to be using headsets. There are all kinds of other sorts of ways to actually be seeing the technology. A lot of what we’re talking about right now, particularly in retail, allows you to have an AR experience through your cell phone or your smartphone and the like.
Tanya: That’s a fantastic explanation. Daniel, having that AR experience through your cell phone, one of the things I found really interesting is the example that you use, which is a stroller. Can you explain exactly like how that would work?
Daniel: The stroller is one of my favorite examples for augmented reality. Picture yourself shopping online for the perfect stroller, and you land on a beautiful product page and you see all these photos and you might say wow, this stroller looks really nice, but will it fit through my front door? Will it fit in my car? And you’re scrolling through the site looking for dimensions and getting out measuring tapes and all this stuff. That’s a cumbersome process. So with augmented reality, what you’re able to do is right there from that online store be able to—if there’s a 3D model available of that stroller—tap on it and then your phone camera will turn on and what you’ll be seeing is essentially that 3D model of the stroller placed in your environment.
And AR technologies on smartphones today are good enough that the scale will be one-to-one with the real thing. A lot of times the accuracy is down to the millimeter, so you can be really confident that what you see is what you’re going to get. And then you can actually just walk around your house and say, oh, what does the stroller look like here and how big [is] that compared to my door frame or my hallway?
One of my favorite examples with that stroller is, like you mentioned, inside the trunk of your car. So if the merchant has a 3D model of the stroller folded up, you could go to your car, pop open the trunk, and then from that same website you can, just like magic, beam that stroller right into your trunk, move it around and see how much space it takes, giving you way more confidence when shopping online than you’d ever get from just a list of dimensions and product photos.
Tanya: Shrenik, Daniel mentions using your web browser, and I know that when you talk to folks about this, you often ask them how many of you prefer shopping on an app or how many prefer shopping on a web browser on your smartphone. Give us a sense of what you hear from that and why that’s important in this experience.
Shrenik: A lot of people discover brands, especially brands like Wayfair, through the web browser. [When] you think of ads, it’s all on your browser. So, the first step that people take to your retail experience is through a website. That’s an important platform to have the best experience on. Then once you start using the website and you want to do something more serious, usually people move over to the app because there’s a lot more functionality on the app. But slowly over the last few years, there are very, very powerful web experiences and app experiences that look pretty much the same. And that’s true for Wayfair as well. We do provide the augmented reality experience on the website as well. If you’re browsing Wayfair on a mobile browser, you can pull up and see a couch or chair in the context of your space at the right size and make a decision.
Tanya: I would imagine there are also probably a lot of people like me who are like, I have too many apps! I have to scroll across the face of my phone and I get lost in all the apps. So being able to do it in a web browser is maybe a little bit easier. So you don’t have to have an app for every single different company that you might buy from.
Daniel: Just to add in there, so for Shopify—we aren’t a marketplace, we are the platform that powers over a million independent merchants out there. Those merchants, they might not even have the resources to create an app. You have Wayfair, that can have Wayfair Next and a whole R&D team, but if you’re a small mom-and-pop shop starting to sell online, you don’t have the budget to have a whole R&D team to create AR apps with AR functionality. So having an easy way to be able to host a 3D model on the web and launch an AR experience right from the web without having to develop a native app is huge for a lot of merchants.
Shrenik: I want to echo what Dan’s saying, too. The industry is definitely moving toward a point where all you need now is a representation of the product and that’s all. The OEM [original equipment manufacturer] or anybody else is going to power the experience for you. It’s going to be all about the content very soon. That’s an important evolution in this industry. It was all about the technology before, and now it’s going to be all about the content.
Allan: Just piling on, time is one of our scarcest and most precious resources we have right now. And using these kinds of technologies just save us a ton of time. The example that we’ve been using—in the past you would have gone to the store, tried out multiple things, perhaps brought the items home to you. If it didn’t work, return them, trying something else again, and going back to your analogy of the beginning, creating all kinds of stress in marriages at this point about the stupid decisions that one spouse is making without being able to talk to the other. This type of technology is saving time, energy, and just allows you to try so many options in the privacy of your own home before you have to pull the trigger.
Tanya: Allan, one of the industries that you’ve been looking at is leveraging AR in the automotive industry. What does that look like?
Allan: One of the ways is you just get to see how that car looks in situ. I can actually look at the car that I’ve designed using a traditional web browser and then downloaded into my phone. Now walk around my driveway looking at it. I can see where that can fit into the garage. But one of the huge things that we’re working with a lot of automotive clients on right now is, once you create that 3D model during the design process, you can keep it all the way through manufacturing, testing, marketing, distribution, and out to the consumer. So, there’s a significant cost savings. In the past, they would have to literally build physical models at each stage and go back to check to see that the original vision was being maintained. By having that one 3D model created at the front end, you save a lot of money and you keep the original kind of genius, the original design, the original ideals kept all the way through the process. It’s just fun to be able to see all these different options. We want to see everything in 3D. It’s how we see the real world. We’ve adapted by having to look at these things in 2D, but being able to see them in 3D is just the natural, better way and you build a better emotional connection to the things that you’re looking at.
Shrenik: One of the things that Allan talks about, I guess, is that it’s not just about the practicality of AR. It’s an experience. You can actually create a brand experience around this. It doesn’t have to be only a practical thing like, just take this out in this space. It’s also about, can I create like a brand experience? You can come up with fun stuff for your customers to experience.
Tanya: Allan, when you were talking about the automotive industry, you said very quickly that you could see the fit for you and what it would look like. One of the areas where I think this is really interesting and most consumers may see some of it faster is in fashion and accessories. There’s been some experimentation around that, like trying on shoes using AR. Talk to me a little bit about the shoes, because shoes are so important.
Allan: Absolutely. Literally, I was trying on shoes yesterday, and I did it in the traditional way that I went down to the store, sat there probably for a good hour and a half, going through various different fits and looks and just trying to see it myself. Again, the perspective is you looking down at my feet. But what you can actually do with the AR is sit literally in the comfort of your own [home] and go through so many different looks and feels just because I can quickly go through them and think, do I like this? How do I look? But also, once I captured my foot in the shoe, I can then turn it around in 3D, so I’m not trying to catch how does it look from one angle or another angle?
What we’re finding right now, particularly with fashion, is hard objects are easier to be able to do an AR—so, something like a shoe or potentially a watch or something like that. The next generation of what we’re really start[ing] to look at is how can you make fabric look authentic? Fabric is obviously really challenging, not only because there’s so many different types of fabric, but the way that it flows and fits onto your body changes by the person. How it catches the light is just challenging. We’re also still struggling a little bit with jewelry and the like, because whilst it’s a hard substance, [it’s difficult] just getting that reflection. Shiny is challenging today, trying to capture how a gem catches the light.
That being said, there have been huge moves in technology and just as the processing powers have kind of improved, I expect that we’ll be able to see these magic mirrors in stores so you just go in there, at least get that first impression of how clothing looks. And I know that already a lot of makeup companies are allowing you to try on all the different shades of lipstick. That, as you can imagine, is a real pain. You put one set of lipstick on, you’re going to have to wipe it off. You put the next set of lipstick on. It’s taking you some minute to try each one. With these kind of mirrors, which allow you this augmented reality, you can just fly through all the different shades and all the different types of makeup that they have.
Tanya: Daniel, last year the CEO of Levi’s made a really bold statement. He said that in 10 years, there was going to be no more sizing in fashion.1 Instead we’d be using body scanning to make custom-fit clothes. You know and some of that may be using this technology. So my question is, 10 years?
Daniel: It could very well be. I mean, there already are a number of stores out there that do custom-fit clothing. Some have experimented with you wearing a skin-tight suit with a bunch of markers on you and it tries to make a 3D model of your body to be able to create custom-form clothing. It’s a really interesting problem, but 3D avatars of yourself will be important even if it doesn’t lead to custom-made clothing. It could be something that helps with the online try-on process. You could imagine in the future where you have a 3D scan of yourself that knows your dimensions, and when you’re shopping on a particular store, it asks you for permission to use your 3D avatar in order to digitally try on clothes. And like Allan was saying, there will come a time when technology is there that can simulate cloth and fabric and the physics and movement of these materials in real time. You would be able to see what you would look like. And I can imagine it’s something a lot of people would want because that is the number-one problem when shopping online for clothes: Is this thing going to fit? I don’t want to go through the whole hassle of ordering it and then having to return it.
Tanya: Yeah. What a pain, right?
Allan: Jumping in on that, there are 3D scanners already for the body type. I was at a gym recently where I went behind a closed curtain. I stood on the stand. A camera rotated around me and built a 3D model of myself. In that case, it was actually looking at like my body fat, my proportions. But they’ve got that 3D model of me. And interestingly, in my head, I have this 3D model of me of what I look like when I’m 25, and the 25 years that have passed have not be[en] kind, necessarily. And to actually have this real 3D model of what I look like now really helps, because, again, when I look at things online, I see the good-looking 20-something models and think, I would look amazing like that—if I had that build, if I had that height, if I was still in my 20s. And of course, my ego does it. It then comes home and of course doesn’t always quite work out so well. So, actually having these 3D models of ourselves is going to really help you to see truly how these things look. Save time, save money. And actually drive me probably to be buying more of these items.
Tanya: You are speaking to my life there.
Allan: I’m also seeing that coming out of this is once you know your body type, once that 3D model is there, it’s going to start using the analytics recommendation engine to say, people who have a similar body type to you also like these items of clothing. Again, what we actually are finding at that point is people who have my height, my build, my complexion, all of these things will just become a part of that recommendation engine that will help me have better options closer to what looks for me.
Tanya: We’ve been talking a lot about the consumer experience and what this technology offers the consumer. I want to bridge that over to the business side of things, though. What do we know about how these 3D or AR experiences translate into sales?
Daniel: I just want [to] jump in here and mention one thing. It’s that 3D models don’t require an AR or VR to be worthwhile for the consumer. Imagine if you don’t have your phone on you and you’re just browsing on your laptop and you go to a retailer that you like and they have a 3D model. Well, that 3D model, even though you can’t use your laptop to view it in your space, being able to just rotate it around and see it from any angle is immensely valuable. You’ve probably been in the situation before where you’re on a product page and you see there [are] three product photos, but none of them are from the angle you want. You really want to know, what does this thing look like from below or from behind? And there’s no product photo for it. If it’s a 3D model, it’s a model of the entire product. And you’re free to rotate it. You’re free to zoom in. You’re free to see it from any angle you want, so giving you more confidence there. What we’re seeing is that merchants on our platform who have been using 3D, their shoppers are up to 2 ½ times more likely to convert when they’ve interacted with 3D models, which is absolutely huge.
Allan: So these AR, VR technologies are driving additional sales, but one of the really big savings that we’re learning about is the significant drop in returns, because I’ve now seen how that piece of furniture looks in my living room. I’ve got my spouse to agree that it looks good and that we should move ahead. I know that it will fit. I’ve kind of seen how the watch looks on my hand or how the shoes look on my feet. So, again, I’ve had that first impression that normally I wouldn’t get until the object has been delivered to my doorstep and I go, oh, it wasn’t the right shade. It wasn’t the right fit.
Tanya: That’s interesting.
Shrenik: Just playing devil’s advocate here: One of the things that we need to think about is that the numbers today are—it’s a little early in the game in a sense, because if you think about traditionally how people interact with cell phones or data or information on your phone, usually on a website you’re used to seeing images, but now you have this camera experience that’s open and you can see this product in your camera, but it’s very tough for a user to actually trust that that’s the right size. How do you interact with it? What are the kind of user experiences that all of the other retailers are coming up with that help make that interaction frictionless? How can you convince the user that that product that’s virtually there is the right size? There’s a little bit of a hump that we need to get over where this will become a requirement where people are going to start expecting this experience. At that point in time, the numbers or how it would affect sales, that would be a lot more clear. I think it’s a new paradigm for interaction. Things like Pokemon Go and some of these other games will actually help users understand what to think of augmented reality or what to think of this content that’s showing up in your space.
Daniel: The impact of AR goes beyond just increasing sales, as well as decreasing returns. Going back to what Allan was saying before about using AR and 3D in the manufacturing process, cutting down on the feedback cycle, the amount of time it takes to develop their prototype of a product and then you have to go get it built and then you realize, oh, shoot, we should change this and this. And then you have to remanufacture it. Being able to use 3D, if it can cut down a ton of time and a ton of cost of that back-and-forth of manufacturing something, that is something that has a direct impact on business.
Allan: I don’t think we know just how huge the impact is going to be. I use the analogy of the smartphone. Ten years ago, if I was to say that you were going to touch your smartphone a hundred times during the course of the day, you would think I was nuts. You were maybe getting a few telephone calls, but you were still paying roaming charges so you would normally say, let me call you back from a landline. I remember having, I think it was less than 50 texts per month, and feeling that that was excessive. Nowadays, I read a statistic that we touch our cellphone over 2,600 times per day on average. It’s the very first thing we pick up in the morning. It’s the very last thing we look at each evening. A mere 10-ish years ago, we had no idea just how impactful that device would be for us. And we’re right at the very beginning of how we’re starting to use AR, VR, spatial technology. Over the next 10 years, it’s going to become so ubiquitous and used in ways that we can’t even begin to comprehend.
Tanya: Obviously, the space is evolving really quickly and I’m wondering what’s happening on the standards side of things. Do we have a VHS versus beta situation right now, or are companies pretty aligned?
Shrenik: We definitely do. Everyone is trying to solve this problem in their own little ways, and so we’re having multiple silos. Take the example of digital pictures, when that became more common. So you had jpeg. You had png. You had all these different formats that are all just slightly different. We’re facing the same situation today with virtual products or 3D content that represent virtual products. We’ve established a standards group. It’s called 3D Commerce. And the aim of that group is to create the JPEG for 3D. We want to basically build one representation off the virtual product that everybody can follow and also teach people how to do it at the same time.
Tanya: Somewhere someone’s listening to our conversation here and they’re thinking, I can totally see how I could use this in my business; but they don’t necessarily know how to start. What do they do? What questions should they be asking themselves as they think about getting started in this?
Daniel: There’s a few ways to get a 3D model made. One of the ways is a technique called photogrammetry. That is just a fancy term for, you take a camera, you take a bunch of photos of your product from different angles—you might take 200, 300, or sometimes even a thousand, depending on the complexity of the product—and then there’s algorithms and computer software out there, some of them free, some of them paid, that will take those photos and try to reconstruct a 3D model based on those photos. The challenge is, it does require a little bit of expertise. It’s not something as simple as saying, I just snapped five pictures and I’m good to go. There is a little bit of cleanup that has to be done to make sure that the models are the right size and don’t have weird textures or weird glitches all over them. It can be tricky if you are, let’s say that brick-and-mortar merchant just starting off and you don’t know much about 3D at all. It is quite challenging to go off and to do something like that.
There are also 3D scanners out there. There are apps that you can download for your smartphone where you can just kind of quickly, almost like you’re taking a video of the product, and then it tries to do that 3D creation. The problem, I would say, for commerce is it only gets about 80% there. The model will look okay, but not good enough to be a perfect representation that you would be proud to host on your product page and say this is what the product looks like. So that technology, I’m hoping over the next five years, will get a lot better and easier for smartphones so that people can just at home create a 3D model of any product that they wish to sell.
Then the third way of getting stuff modeled is actually using an artist that will use reference photography to create a 3D model from those photos. For our merchants on Shopify, we actually have a way to connect them with partners because there are lots of companies throughout the world that have large networks of artists that from those photos can create those 3D models in a quick and cost-effective way. For our merchants, we normally suggest [they] go that route. You only need to take a few photos. Maybe one from the front, from the side, from the top. Provide a few dimensions, and then a week or so later, you’ll get back an amazing 3D model that was handcrafted.
Allan: We’re working with multiple clients right now to help them build out that strategy on where they should be playing in particular marketplaces and how they can win in those marketplaces. We help them to identify what are the best use cases for them to invest in and look at, both based on what other companies are doing in that space, as well as just helping them to try to be creative about how they can use this new technology, help them and build out the ROI [return on investment] to show to their management that the investment will be worthwhile, doing prototyping, and then really helping them to expand rapidly.
One area is really interesting that we’re just beginning to look at is some of the ethical implications of all of these technologies. Right now, our smartphones already know a tremendous amount of information about ourselves, probably more than we’re aware. This next generation of technologies is going to be measuring our biometrics to actually understand when we have an emotional response. One of the things we’re really beginning to talk about—in fact I gave a presentation at the United Nations last year about the ethical implications on exponential technologies—is how we can ensure that we’re using all of this tech in the right way and in a positive way and in a way that we all feel comfortable with it. But the data it’s going to be tracking on us is just going to be absolutely vast and at a level that we’ve never known and that we’re not even aware about ourselves.
Shrenik: A lot of that responsibility would fall on the shoulders of the OEMs. Hopefully, we will have a structure in place where the technology providers actually have a strict enforcement of policies that the app builders like retailers will have [to] follow. There’s going to be a lot of discussion and a lot of collaboration there that’s going to be needed. But it’s a very good point. As soon as you open up the camera and you’re looking at your space, there’s a can of worms there.
Tanya: I would imagine when people use this technology for the first time it can really kind of blow them away all things that they’re able to do with it. What kind of reactions do you see from folks?
Daniel: When I show people AR for the first time, they think I’m a wizard. It’s like magic. This product appears in front of them and they look down at the phone screen and they look up and they look down and they look up and they’re like their brain is breaking at like, why is there a product on the video feed on my phone? But there’s nothing in front of them. You see them try to understand for a second and they’re just like, my goodness, this is possible today? It’s stuff that they thought was maybe sci fi and this is running on my phone. I launch an AR experience from a website and their mind is completely blown.
Allan: I love it. I get my client to giggle the first time that they see the whole VR and they are saying it really is magical. I love that analogy about being a wizard. It’s something that people just can’t possibly believe that isn’t science fiction, it’s science now.
Daniel: The biggest challenge with AR and VR is, you have to see it to believe it. So even listening to a podcast like this, if you haven’t tried it, you might have some expectation of what it’s like, but really you have no idea how amazing it is until you try it. It’s about getting people inside VR headsets or getting people to try out AR experiences to really see the potential that this technology will have in the future.
Tanya: Daniel Beauchamp heads up augmented and virtual reality at the e-commerce company Shopify. Shrenik Sadalgi runs a small R&D team at the online furniture and housewares company Wayfair. And Allan Cook leads the digital reality practice at Deloitte Consulting LLP, where he works with a variety of businesses looking to implement virtual reality, augmented reality, mixed reality, and 360° immersive experience.
I know—It’s a lot to take in, all this technology. You probably want to learn more and you’re in luck. We’ve got plenty of resources on our website deloitte.com/insights.
We’re on Twitter at @DeloitteInsight. I’m on Twitter at @tanyaott1. I am Tanya Ott. Have a great day!
This podcast is produced by Deloitte. The views and opinions expressed by podcast speakers and guests are solely their own and do not reflect the opinions of Deloitte. This podcast provides general information only and is not intended to constitute advice or services of any kind. For additional information about Deloitte, go to Deloitte.com/about.
