Building and Validating Media Mix Models
In e-commerce, measuring and understanding marketing effectiveness holistically across channels is essential to the success of a company. Doing this properly is complex as e-commerce companies may advertise on multiple platforms, both online or out of home, or work with multiple agencies over time. Further, with increasing concerns for customer privacy and data regulation, attaining this holistic understanding has become more difficult as advertising platforms are, rightly, less inclined or able to share customer level data. While all businesses have varying objectives, the underlying goal for most marketers is the same: understand the true return on investment comparability for all marketing channels. Typically this is achieved through some combination of click attribution, experimentation, and media mix models (MMMs). Each have their own strengths and weaknesses for understanding ground truth and cross-channel effectiveness, which we will expand on. This paper will offer a case for marrying two approaches, time series MMMs and experimentation, to provide a validation methodology that can yield a higher degree of accuracy than relying on any one approach in isolation. We will show that in particular, building an MMM alone, without validating the model against ground truth results from experiments, can lead to incorrect conclusions. In the example provided, three models built from the same dataset and slightly different assumptions conclude that a different marketing channel has the lowest cost per acquisition. Ultimately the goal of this paper is to offer a solution that can help advance the precision of cross-channel measurement throughout the marketing science community. Industry innovation and standardization requires cooperation and willingness to test new methodologies. We hope this white paper provides a meaningful step in that direction.
Link to full paper and companion code
-- Erica Mason