Combining Marketing Mix Modeling and the Conversion Lift Experiment

Alshaya Group/H&M Business Case

Alshaya is a company that engages in the operation of franchises in various consumer goods sectors, including fashion, health and beauty, hospitality and leisure. Among the franchises it manages are brands such as Starbucks, Mothercare and H&M.

Alshaya manages 30 H&M stores and the H&M e-commerce division in the United Arab Emirates (UAE), where it is also responsible for H&M’s marketing strategy. In particular, Alshaya engaged Deloitte to conduct a study with the aim of measuring the effectiveness of its advertising activity in both online media (Paid Search, Facebook, Instagram, GDN, Snapchat, Criteo, Affiliate Marketing and YouTube) and offline media (magazines, outdoor advertising, radio and SMS) for the H&M brand in the United Arab Emirates.

In order to achieve this objective, Deloitte used a holistic methodology, namely Marketing Mix Modeling (MMM), which consists of building machine learning models that analyse the time series of one or more KPIs of interest. Furthermore, this methodology allows a second stage of analysis, “media mix optimisation”, which, starting with a given budget, can be used to guarantee the best possible allocation of resources.

In this study, Deloitte also analysed the result obtained from an alternative measurement approach, namely Facebook's conversion lift experiment (CLE), a methodology that facilitates the measurement of incremental business generated by advertising.

Lastly, the results of the two methodologies, MMM and CLE, were compared in order to achieve a unified vision that best matched ground truth, and, in addition, a quantitative method was proposed that allows MMM to be calibrated using CLE in cases in which major differences are observed between these methods.

MMM and CLE: two complementary methodologies

What is MMM and what is CLE? What are the differences between them, and how can they be combined? Although the two methodologies have a common objective, namely, to measure the return on advertising spend (ROAS), they do it using different approaches.

On the one hand, MMM uses historical time series data to model the results of sales (or other KPIs) based on marketing and control variables such as climate, seasonality, competition, etc. Using these variables, metrics such as return on advertising spend (ROAS) and spend optimisation are obtained. In short, MMM can be used to provide answers to strategic questions.

On the other hand, CLE is fed with more granular data, it tests a specific hypothesis and guarantees highly accurate measurements. It is the most suitable approach for measuring the causal increase in conversions due to advertising, but it works within a very specific, delimited sphere (in our case, exposure to Facebook), offering no visibility of the other drivers moving the business.

To summarise, the advertising industry is faced with the great challenge of finding a convergence between the two methodologies, MMM and CLE, that will enable it to combine the best elements of both options and achieve a new unified vision of measurement.

Results of the Alshaya/H&M business case

The results provided by the study offered Alshaya and H&M complete visibility of a series of issues that had previously been difficult for them to evaluate due to the complexity of the environment and the simultaneous presence of interacting variables.

After applying advanced measurement algorithms (Prophet, Bayesian models, etc.), the following findings were obtained with respect to H&M in the UAE:

• Baseline and price play an important role, especially in the offline channel.
• Factors such as advertising, promotional activities and special launches have a greater impact in the online channel.
• Online advertising contributes significantly to offline sales, in addition to its direct effect on the online channel.
• Positive returns on all advertising spends, with Facebook standing out as the medium that generated the most sales.
• On comparing the figures for both 2018 and 2019, we observed that Facebook ROAS increased by more than 20% in 2019 due to the improvement in the strategy implemented by the client.
• The results obtained from the two measurements, MMM and CLE, were in line with each other and, therefore, it was not necessary to apply any calibration method.

In short, Deloitte recommends working with MMM and CLE, integrating and calibrating the two methodologies when this is necessary. These measurement methods are really useful in helping marketing teams to decide on how to invest their budgets, using data instead of mere intuition. Moreover, this combination of approaches is key to generating greater confidence in the measurement results and avoiding possible measurement bias.

Such a fragmented advertising scenario (with a wide range of media, devices, platforms, formats, audiences, etc.) makes the development of robust measurement models an even greater challenge. The fusion of different methodologies, combining both strategic and tactical vision, would appear to be the only way to continue to guarantee the best results.

If you'd like to visit the spanish version of this article  click here