Dentsu Media UK chief Jenny Bullis: Measurement is a ‘hot mess’
Attribution is wrong and brands shouldn’t do it, Dentsu Media’s UK and Ireland CEO has warned.
Jenny Bullis, who joined the agency group in June from Meta, told the Media Research Group’s annual conference that new marketing mix modelling (MMM) techniques can mitigate advertisers’ “insecurity” over audience measurement, but experimentation was critical.
“I think measurement is frustrating and frankly — as my American friends say — a hot mess,” she said in a keynote speech. Bullis was particularly critical of attribution techniques that seek to credit a particular media channel or campaign for a direct impact on an advertiser’s sales.
Bullis explained: “I’ve spent a lot of money on attribution in my time. I’ve also raised a lot of money selling attribution modelling.
“And what I mean by attribution is data-driven attribution, multi-touch attribution, rules-based attribution — basically anything where digital data is pulled from standard ad platform reporting and refactored into a new calculation and weighted. It’s attribution and it’s wrong.”
The GDPR effect
GDPR restrictions, which came into force in 2016, made attribution “bad”, Bullis added, but “it is getting worse”.
“Attribution demands log-level data and this just doesn’t exist in useful volumes any more. The lack of visibility fundamentally impacts accuracy. And the issue of attribution is: there is no way to know how inaccurate it is — and that makes me very uncomfortable.”
And yet getting a business to stop modelling attribution is “really hard”, Bullis explained.
“[Attribution models] are often so established and baked into personal and team goals and bonuses,” she continued. “When I challenge marketing departments about attribution, they tell me they need something because the metric straight out of the platforms is wrong. I tell them: ‘Two wrongs don’t make a right — don’t do attribution. Instead, do experiments and do MMM.’”
Continuous experimentation
Instead, Bullis said she advocates for experiments and use of newer MMM tools as more accurate techniques for understanding marketing effectiveness.
These new tools use granular data, machine learning and AI to provide faster and more accurate insights, Bullis argued. She cited a Deloitte study showing a fourfold improvement in effectiveness and return on investment for business using continuous MMM.
“The critical concept here is incrementality. So experiments that use holdouts, that create random control, trials that analyse the direct effects of exposure or treatment, are the only way to know if advertising is actually working,” she said.
“If lift is observed, this is causation… The lift in experiments can be used to optimise many channels and they also feed that new breed of MMM, using experiment data to calibrate it and validate it. Systemised and regular experiments at the campaign and channel level can answer most of the isolated marketing effectiveness questions that I get asked.”
Referencing work by Northwestern University, Bullis said advertisers that use at least 15 experiments typically saw a 30% improvement in campaign performance for business outcomes, compared with when there were no experiments. Those that did another 15 experiments in year two saw another jump in improvement to 45% overall.
“Is it any wonder that the hyper-growth and hyper-successful businesses like Meta, Google, Amazon, Booking.com, Netflix, Next, the gaming industry, institutionally adopt experimentation into their own marketing? Whereas it’s only 6.7% for consumer packaged goods and 4% for automotive. Whose businesses are more successful and growing faster?
“If marketers do nothing else, do experiments.”
How to make marketing indispensable to the CFO? Focus on incrementality
UK ‘overly proud’ of panels
Bullis has been a long-standing proponent of Origin, Isba’s cross-media measurement initiative that is set to formally launch next year.
While the initiative has been dogged for years by disagreements over how to equitably provide TV measurement data from joint industry currencies like Barb alongside data pulled directly from Google and Facebook, The Media Leader revealed last month that a possible solution has been tabled.
However, Bullis highlighted that, as more media consumption happens online, “less media is happening in perfect and known isolation, such as linear TV through an aerial”.
“In the UK, we may be accused of being slightly overly proud of the legacy of our audience panels. We hold them as ‘best in class’ and as ‘gold standard’, and I’m sure they are when and if they are the right one to use,” she observed.
“But, at worst, we’ve become fixed to those panels and unchanging. But with media having a growing volume of ‘at source’ consumption data, this data is potentially more useful than panel. Platforms that have this, like the walled gardens, are challenged both ideologically and technically by being panel-first. Why use a panel when your own data is more granular, more accurate, it’s in greater volume and is faster?
“One of the things I love about Origin from Isba and Halo from the [World Federation of Advertisers] is it aims to marry these two factions together: the full-fat pipe of platform data at source with a keystone panel.
“This is very different to a single-source panel at the centre of a single medium. Single-source panels were the gold standard, but computer power, data access and modes of consumption challenge this — and we should too.”
Plan tabled for Barb to join Origin in hybrid reporting model
Industry endangered by ‘combative’ attitudes
Bullis has worked in the industry for over 25 years and before Dentsu led the EMEA-wide marketing science team at Meta. She has held senior roles at BT, Sainsbury’s and ITV, before taking on leadership roles at Omnicom (Annalect UK and EMEA managing director) and Group M (Essence EMEA CEO).
She added: “There have been big differences in the jobs that I’ve done and there have been deep, big differences in the culture and the purpose of each place. But there is an underlying common element of things that have shaped me and that is my passion for media planning; I am a media planner by trade. I fundamentally believe in measurement analytics.”
However, she criticised the “rapid rise of the marketing think-fluencer” that proliferates on her LinkedIn feed.
“Every day, there’s constantly people telling me what good advertising, what good measurement, what good tech looks like — like all brands, like all marketing, like all media, are the same.
“And then there is something I never thought that would ever be a debate: news as an unsafe and unsuitable advertising environment because of our political discourse. And that political discourse is overflowing into our culture, into our industry.
“I don’t know about you, but I find much of our industry is an ‘A versus B’ debate these days. It is very combative. It’s quite clickbaity. There is an absolutism of opinion and it’s nearly always with a vested interest of someone selling something.”