Best practices for designing an advanced analytics .... Similar to most forms of analytics and tools, there is no one-â
Econometric Modeling
BEHIND THE BUZZ – WHAT IS ECONOMETRIC MODELING AND HOW IS IT DRIVING TODAY'S MARKETING DECISIONS?
December 2015
WHY DOES ECONOMETRIC MODELING MATTER?
What’s all the buzz about and why should I pay attention
WHAT IS ECONOMETRIC MODELING?
A topline look at how econometric modeling works
TYPES OF MODELS AND THEIR USES
How and when is it appropriate to use MMM, Brand Equity Modeling and Long Term Modeling?
DEVELOPING AN ANALYTICS ROADMAP
How to make the most of the data available now while also planning for the future
GETTING STARTED – Best practices for designing an advanced analytics 3 GUIDING PRINCIPLES strategy KEY TAKEAWAYS
Key takeaways for brands considering econometric modeling
WHY DOES ECONOMETRIC MODELING MATTER? PRECISE BRAND-‐SPECIFIC LEARNINGS AT YOUR FINGERTIPS How should we allocate and optimize our marketing spend across all products and channels? How can we forecast how a budget change will impact our sales in one year? Five years? Ten years? The data shows us that awareness behind our product is increasing – why is this not translating to sales? As marketers, these are some of the questions we are faced with as we grapple to understand how to optimize marketing spend. While we often work against assumptions such as industry norms or brand comparisons within a category to determine how to allocate budget or build awareness, looking at historic and competitive data is not always enough. Today, more is being demanded of marketers and with that comes the pressure to precisely quantify the impact of marketing decisions. Econometric modeling is key to finding the answers to these questions, particularly when brand-‐ specific, precise recommendations and results are required. This Spotlight On goes behind the scenes to discover how econometric modeling works, how to choose the appropriate model based on common questions, and how to quickly start leveraging analytics for advantage.
DATA CONNECTIVITY WILL WHAT IS SAVE US ALL ECONOMETRIC MODELING? WHAT IS ECONOMETRIC MODELING? ‘Econometric modeling’ is a term that has become so prevalent, it’s beginning to sound like a buzzword, or industry jargon, used a to make learnings and results seem more scientific. The fact is econometric modeling sounds scientific because it is. Econometric Modeling leverages the variation in granular data to statistically tease apart the impacts of marketing activities on the KPI of interest. Today, various types of regression analysis are implemented in econometric modeling within the industry, including: multivariate, time series, vector auto regression and Bayesian methods. HOW DOES BASIC REGRESSION ANALYSIS WORK? Generally, we know that media over and under delivers in certain markets, and the amount of over/under-‐ delivery varies when examined on a weekly level. Therefore, when looking at a media plan on a DMA level, there is a significant amount of variation in the executed data. Since there are 210 DMAs, that translates to 210 data points per week. Most models are built with three years (166 weeks) of data, resulting in 32,760 data points per variable. Therefore, if TV, print, radio, OOH and banner were all part of the media plan, this would provide up to five variables or 163,800 data points. And that is just the media portion of the puzzle! Now, add to this the impact of price, various pricing promotions (ex: buy one get one vs. 5% off), distribution, macroeconomic factors (ex: unemployment, weather), competitive activity and category trends. All of these variables highly influence brand performance and have significant variation across markets and time. As such, they are necessary to account for in the model; and with them, we’ve more than doubled the number of data points. So we’ve established that there is not only a lot of data available to marketers, but variations in that data that must be carefully considered. Now the data must be measured. Whatever the metric you are using to measure the data against is known as the dependent variable and should be associated with the brand’s KPI. While this variable is often actual sales, clicks or conversion, it will need to be determined by what data is available, at what granularity and of course, based on the brand KPIs. Once the dependent variable is determined, all of the other variables (independent variables) – TV, print, banner, distribution, price, etc. – are measured against it using historical data. Due to the variation in data trends across the three years and 210 DMAs, the model will establish correlations between the independent variables and the dependent variable. For example, if there was a price change in New York that coincides with a spike in sales vs. no price change in Boston and no spike in sales, we can measure the impact of that price change on sales. Similarly, if TV went dark for a brand for the first time in certain markets and after some time those markets experience a decrease in sales, we can measure the impact of going dark. While these examples are a bit over-‐simplistic, they demonstrate the ultimate goal of econometric modeling: to isolate the impact of each factor that would influence sales while holding all other factors constant. It is the depth (210 DMAs) and breath (166 weeks) of data, and the variation of that data across DMAs and weeks that enables this methodology to work, delivering increased ROI for marketers that are leveraging to their advantage.
TYPES OF MODELS W ILL DATA CONNECTIVITY AND UU SES SAVE S ALL EACH MODEL SERVES A SPECIFIC PURPOSE
Similar to most forms of analytics and tools, there is no one-‐size-‐fits-‐all solution when it comes to econometric models. Models are to be chosen based on the brand KPIs, the key questions at hand and the data available to answer those questions. In the pages that follow, we have mapped out the most common forms of econometric modeling, the key questions they should be used to answer, how these analyses have proved useful to clients, and finally the methodologies and data needs associated with each. A summary cheat-‐sheet follows below.
DID YOU KNOW?
Econometrics methods are based on aggregated, macro-‐level, time series data; whereas, digital attribution methods are based on micro-‐level, granular, cookie data. Therefore, econometrics models capture major trends across time versus an individual's exposure and resulting action. While both methods are used to make predictions for future investment, econometrics is best applied to broad budget and investment decisions (e.g., TV, radio, digital), whereas digital attribution is applied to more detailed online tactics investment decisions (campaigns, partner, placements, AdWord groups etc.). Given their complementary nature, the output is strongest when used simultaneously and in conjunction with each other.
TYPES OF MODELS W ILL DATA CONNECTIVITY AND UU SES SAVE S ALL MODEL TYPE 1: MARKETING MIX MODELING (MMM)
Key questions answered: ü How should the budget be optimally allocated? Across brands and channels? ü At what point is media hitting diminished returns? ü What is the impact (short term ROI and/or volume impact) of marketing vehicles? The Marketing Mix Modeling (MMM) is likely one of the most common forms of econometric modeling leveraged by marketers today. It provides a top-‐down, high level view of how each marketing channel is performing, the relationship/synergies between channels, and where each channel falls on the diminished returns curve. These results can be used for optimal budget-‐setting, forecasting and scenario planning. MEC MMM techniques can additionally measure the impact of reach and frequency within a specific decay period (measured by adstock) in order to maximize MMM results within media plans. MMM most commonly leverages some variant of regression-‐based methodologies as described in the “What is Econometric Modeling?” section. Data required for MMM: Two-‐three years of sales, media by channel, in-‐ store data – all by week and by DMA.
DID YOU KNOW?
MEC’s proprietary Admodel leverages 35+ years of econometric modeling results and expertise to measure beyond the standard GRP level. MMMs built in-‐house at MEC break out effective frequency, recency and decay as well as frequency within a recency period. These results enable more precise inputs and specifications into a marketers media plan.
MEC’S PROPRIETARY MMM PROCESS
TYPES OF MODELS W ILL DATA CONNECTIVITY AND UU SES SAVE S ALL MODEL TYPE 2: BRAND EQUITY MODELING
Key questions answered: ü What should the awareness goals be to achieve a volume/profit growth of x%? ü Brand tracking metrics are showing that consumers are aware of the benefits this brand has to offer but this is not converting into sales. Where/how are we losing consumers in their purchase journey? ü How to connect the fast moving metrics (ex: awareness, consideration) to slower moving analytics (MMM, digital analytics)? While MMM measures marketing activities against a single specific consumer behavior (ex: sales), Brand Equity Modeling measures the impact of marketing activities against multiple consumer outcomes. These outcomes can vary depending on the KPIs of the brand but are traditionally: Awareness, Consideration, Purchase Intent and Sales. In short, Brand Equity Modeling measures the impact of marketing through the full purchase loop; along the passive stage, through the trigger and into the active stage. These analyses can help brands understand where and why they are losing and/or gaining consumers through their purchase journey. In addition, it can help connect brand equity metrics with sales behavior. For example, if a brand had previously measured that when a TV creative drives 10% awareness, this results in a 1% lift in sales, this 10:1 ratio would be a good benchmark when creating awareness goals for a campaign that is also working against specific sales goals.
TYPES OF MODELS W ILL DATA CONNECTIVITY AND UU SES SAVE S ALL MORE ON BRAND EQUITY MODELING…
While MMM helps answer short term budget allocation and scenario planning questions, Brand Equity Modeling is crucial when trying to understand the evolution of brand equity over time as well as how consumers are interacting with the brand. Leveraging these results, MEC can plan against awareness more effectively by knowing how it will translate into sales. Simultaneous equations or vector autoregression tend to be the best methodologies for Brand Equity modeling since these methodologies enable modeling with multiple dependent variables. Data Needs: All data required for MMM as well as weekly brand equity tracking data – preferably at a local level, but national data will do for Brand Equity Modeling.
MODEL 3: LONG TERM MODELING
Key questions answered: ü What is the long term ROI for marketing activities? ü How does the long term effectiveness differ from the short term? ü Which vehicles are doing more for driving long term ROI and thereby driving brand equity? Long term modeling is exactly what it sounds like. It isolates the impact of marketing vehicles in the long term vs. MMM which does so in the short term (two-‐three years). Long term modeling examines how base sales are changing over time and quantifies the impact of advertising on that changing base. For example, as a general guideline in CPG the long term ROIs for TV and print are approximately double the short term ROI; however, the long term ROI for in-‐store promotions is the same or less than the short term ROI. This is because while media drives brand equity and loyalty (as seen in base sales increases), in-‐store promotions have a tendency to make consumers dependent on discounted products. These results came about through a long term model but are generalized across the CPG industry. To understand the true and lasting impact of media on a brand vs. in-‐ store promotions, long term models should be built for each specific brand or portfolio.
DEVELOPING AN WILL DATA CONNECTIVITY ANALYTICS SAVE R UOADMAP S ALL THE IMPORTANCE OF DEVELOPING A ROADMAP
As described above, there are several different kinds of modeling techniques that each work to solve individual tactical, brand or portfolio questions. In some cases, brands can leverage different types of analysis to answer short and long term questions over a longer period of time. ACTIONABILITY IN PLANNING When choosing a modeling methodology, it is just as important to understand if the results of the model will be actionable. The modeling results should be available to make changes as necessary when the market shifts, media consumption patterns adjust and rates/buying patterns fluctuate. In addition to this, there are often changes to budgets and priorities which must be accounted for in the media plans. Note that if the model results are not accessible to planning teams, it is more challenging to leverage the results for any last minute adjustments. In the example below, the client has all of the sales and media data needed to conduct an MMM in year 1, but not for brand equity or long term modeling. Therefore, in year 1, this brand builds and leverages the results from an MMM but invests in collecting brand equity data and continues to store its historical data. Then, in years 2 and 3, the brand can continue to leverage results from MMM while also benefiting from Brand Equity Analysis (in year 2) and Long Term Modeling (in year 3). As such, the brand is continuing to improve, evolve and stay ahead of the competition.
Non-‐econometric methods should also be considered when developing an analytics roadmap. For example, even though digital attribution and agent based modeling are not econometric methods, they should also be included if needed when developing a complete analytics roadmap.
3 GUIDING PRINCIPLES TO GET STARTED
DEFINE SHORT AND LONG TERM KPIS THAT MATTER Analytics are only as useful as the questions they are set up to answer. Understand where your business is now (Don’t know? we can help by looking at key metrics – sales, awareness, favorability) and what the priority KPIs are in the short vs. the long term. Then evaluate what data and analytics are needed to help drive the business further.
DEVELOP AN ANALYTICS PLAN IN LINE WITH KPIS All modeling approaches are not created equal. Some are more appropriate for specific brands, while others are not. This will depend on brand, category, competitive influences and data availability. Most importantly, the questions and KPIs from step 2 must be matched with the appropriate analytics tool with which they can be answered. This can then be developed into a long term roadmap of what models are currently or will eventually be essential for your business. Ensure that the key questions identified at the beginning of the project are answered with the approaches identified within the action plan. At MEC, this action plan is integrated into the results of every analytics project undertaken, and the results have been proven out across clients.
1
2
3
INVEST IN BEST IN CLASS DATA COLLECTION Getting best in class analytics results are entirely dependent on having best in class data. This includes tracking brand equity metrics on a granular level (if possible) and developing methodologies to collect and store sales data if it does not already exist. This data collection process begins with investigating what data is available to your brand vs. what MEC has in-‐house or can collect and store on y0ur behalf. Collecting this data now is an investment in the brand and will help with strategic decisions that need to be made in the long term for future growth.
MODELS CAN BE POWER TOOLS THAT DRIVE BUSINESSES FORWARD
Making smarter marketing decisions and having a long term analytics plan is no longer a nice-‐to-‐have – it’s essential to maintaining brand health and keeping up with the competition. Econometric modeling can improve budget setting, optimize media plans, help track and set short term and long term goals as well as enable a better understanding of the consumer journey. Leveraging these insights is becoming a matter of course in today’s dynamic marketplace.
THERE IS NO ONE-‐ SIZE FITS-‐ALL ECONOMETRIC SOLUTION
Models are custom made for each business and brand for a reason: They deliver precise results that cater specifically to the individual brand or group of brands being analyzed. The specific models built should be customized to the brands’ KPIs, key challenges and data availability. This holds true for econometric modeling, digital attribution and agent-‐based modeling.
SET KPIS THAT MATTER, THEN DEVELOP A DATA COLLECTION AND ANALYTICS ROADMAP AROUND THEM ENSURE ANALYTICS RESULTS CAN BE ACTIONABLE IN MEDIA PLANNING
At the center of marketing analytics is the brand. Without understanding the important brand KPIs and catering the analytics to help drive those KPIs, the results are a mute. Once the KPIs are clearly defined, then they can be matched to appropriate analytic methods and tools that will help drive those metrics. If this is a long term solution, data can start being collected; otherwise, once there is an agreed upon methodology, modeling can begin right away. When analytics are done as a theoretical exercise, the results sit on a dust-‐covered shelf and no one benefits. Similarly, understanding the ROI and contribution of marketing activities does not benefit brands as much as the actionable recommendations that come from those learnings.
For questions or to request more information, please contact your MEC A&I or Account Lead.
www.mecglobal.com
