A measurement story from Data Stories…

Running your own consultancy is fun when you win business, and less fun when you don’t. Although with a seven-month old at home, the latter can sometimes be a blessing in disguise when you’ve overbooked yourself. Of course, losing the odd pitch or two is all part of the independent consultancy life and not something you should lose too much sleep over. One recent loss in particular has rankled however, if only because it really highlights the same old issues with the measurement of digital ad effectives that a vocal minority have been banging on about for what seems like years now:

  • The issues of non-transparent / independent measurement
  • A continued obsession with basic behavioural metrics that tell us nothing of real-world impact
  • And a focus on measuring short term outcomes rather than long term impact

To cut a long story short, a content creation agency included as part of their pitch to a global advertiser, an overview of how I / Data Stories Consulting would provide some independent market research and analytics to report on campaign effectiveness. Although part of the agency pitch, my contact would have been directly with the client and results would have bypassed any agency filter – i.e. they would have been genuinely independent.

I was delighted for the agency when their pitch won – it was well deserved, but not least because it gave me a new interesting project to sink my teeth in to. It was rather depressing therefore, to be told a couple of weeks later that the advertiser has suddenly decided that they’d like their social agency to handle all campaign measurement. It was no fault of my content agency contacts at all, but rather a reflection of the advertiser’s attitude that social was all.

As means of distribution of course social is vital, but as a means of total campaign effectiveness measurement? I doubt it. As Keller Fay tells us, 90% of brand conversations happen offline. Quite how some social listening tools that are inevitably going to report on the usual spread of clicks, likes and shares are going to report on offline conversations, or even report on total campaign brand impact and long term ROI I don’t know.

I’ve been working with numerous brands and agencies over the past few years to help them better understand how to measure the impact of digital (summary: just like you measure the impact of offline! …kind of), and have produced measurement frameworks like the below to do just that. It’s a straw man and needs adapting for specific client needs, but it’s a useful starting point:

It’s hard not to sounds like a stuck record sometimes, but there’s a reason why the IAB had to launch a national “Don’t be a Clickhead” campaign earlier this year. Clicks, likes and shares might be useful metrics to optimise campaign delivery on, but as a measures of true campaign impact they are virtually meaningless.

Measurement isn’t easy of course, and I hear plenty of excuses as to why it’s not being done properly:

  • We get CTR data in real time. How are we supposed to optimise media and creative on brand impact when survey results aren’t available for weeks after a campaign has closed? My usual answer: There are plenty of research vendors who offer real time brand uplift reporting now, enabling you to optimise performance on something far more closely related to real world impact. Attention is also a behavioural metric with some meaning – and there are plenty of scalable measurement options in this space too.
  • Measuring brand and sales uplift properly is expensive! Sometimes it is for sure, but arguably a campaign which devotes 5-10% of its budget to measurement rather than media/reach is going to generate far more strategically important learnings for the business in the long run.
  • We can’t link to any real world sales data: Again, there are big research and data vendors in this space who would say otherwise. Where sales data isn’t directly available, then there are a world of useful proxies you can lean on instead: for example, efficiency scores which allow you to assess the cost of each incremental purchase intender, or the use of geo-location data to look at store footfall uplifts.

The much publicised Ebiquity and Radiocentre Re-Evaluating Media report last year showed us the huge disparity between perception and reality when it comes to providing independent third party measurement. For example, agencies and brands rank digital video top for transparent measurement, whereas the evidence suggests that quite the opposite is true:


It’s one data point, but it certainly gives a hint as to why we must continue to bang the drum when it comes to reporting on digital impact properly. It’s difficult (sometimes), but that’s no excuse for not giving it a go. If you only get half way there you’re going to be doing your business many more favours than if you just regress to what’s easiest!

Founded by Ian Gibbs, Data Stories is an independent data consultancy focused on all things digital, advertising, media and publishing related. Drawing on over fifteen years of experience in market research, data analytics, digital advertising and strategic planning, Data Stories employs a broad data-driven skill set to help clients solve their business problems. Find out what clients have said about Data Stories here.

The Rev Ops Barometer 2019 is now available


The Colab Revenue Operations Barometer has launched! Produced in collaboration with CoLab Consulting, this study of rev ops professionals from over fifty leading publishers is essential reading for media owners and ad tech vendors alike.

Key business issues and priorities for Rev Ops Leaders and Laggards are covered in depth, along with insight in to how ad tech vendors should position their solutions for different segments of the publisher market.

Order your copy here! https://lnkd.in/eA_cBih

Award winning end to 2018 for Data Stories



I’m delighted to announce that three client projects have been recognised with industry honours this year. Each one has been a pleasure to work on. If you’d like to learn more about how Data Stories can help harness award winning research for your business then don’t hesitate to get in touch.

The time spent with social video ads matters more than ever: three to four seconds is optimum for driving purchase intent

social blog

This article was first published on ondeviceresearch.com. On Device Research are a mobile surveying and digital ad effectiveness research company with whom I work in my capacity as an independent data consultant. To find out more about On Device please get in touch.

On Device Research has today released findings from its social media effectiveness measurement solution that reveal the optimal video ad exposure time on social platforms is three to four seconds: a time frame in which both Brand Consideration and Purchase Intent are maximised, increasing by +4% and +6% respectively.

“Obviously we stopped wasting money on 30-second ads, and we’re designing ads to work in two seconds.”
Marc Pritchard, Chief Brand Officer, P&G

It was little over a year ago that Marc Pritchard, Chief Brand Officer of the world’s biggest advertiser, made a rallying cry at DMEXCO for the digital ad industry to create the next generation of video ads. Citing the fact that only 20% of video ads are viewed for longer than two seconds, the need to drive brand impact in an ever shorter time frame in a medium where consumer attention is scarce, is more important than ever.

One year on, and having tested several video ads running in social platforms across 3,000 survey responses, On Device Research has seen a trend develop that highlights that while video ads in social environments can indeed have impact after just one to two seconds of viewing time, optimum brand impact is seen at just three to four seconds. At the three to four second mark, a +4% increase in brand consideration and a +6% increases in purchase intent has been recorded on average.

While previous industry research (such as this 2016 study) points to shorter video ads having an impact on smartphone, perceptions around the definition of “short” will increasingly fall in line with Marc Pritchard’s two-second claim if the trend seen in On Device’s data continues.

alt text

It’s clear from the ads tested, that creative design is catching up with the reality around the attention consumers have for online video ads. With On Device Research’s Guide to Retail Marketing Effectiveness revealing that four fifths of consumers always or very often skip through video ads in social platforms, the importance of having immediate impact is again highlighted. Early and prominent branding and product shots are both commonalities of successful social video ads.

The debate around the role of social platforms such as Facebook and Instagram in the marketing mix, continues to rage on, as highlighted by the recent high profile disagreements between Mark Ritson and Gary Vaynerchuk. Either way, what is clear is that social video can drive brand impact throughout the branding funnel and that the time we spend with ads matters more than ever in terms of maximising impact.

The proven unconscious impact of digital advertising

On Device Research has released a report that highlights the unconscious impact of digital advertising. The findings reveal that consumers who have been exposed to digital ads but who don’t actively recall having seen them, still record uplifts across a range of brand metrics: from an average increase of +10.2% in Unaided Brand Awareness to +1.6% in Purchase Intent. In other words, digital advertising works, even though you may not know or remember having seen it.

The unconscious effect that advertising has on consumer attitudes and behaviours is a well-established assumption of marketing effectiveness. Daniel Kahneman’s often quoted book Thinking Fast and Slow highlights the role that the brain’s System 1 performs in driving fast automatic decisions at the unconscious level (unlike System 2, which focuses on conscious and rational decision making). Where advertising is concerned, it is System 1 processing that creates emotional brand associations and drives long term brand preferences.

An analysis of ten digital ad impact studies across seven advertiser categories and 3,000 consumer responses has enabled us to shed light on this issue. By tagging campaign creative and visualising digital exposure on our consumer panel we can compare brand metric scores among those who we know have seen an ad but don’t recall doing so, and those who we know have seen an ad and do recall doing so.

Our key findings are as follows:

1. Our memories are not perfect: Most people don’t recall the ads they have been exposed to. In fact 76% of people who we know have been exposed to test ads, claim that they haven’t seen them in the last 30 days. Brand impact therefore happens for most people passively, highlighting how important it is to measure alongside ad recall.

alt text

2. Digital ads have impact passively: Those who don’t actively recall seeing test ads still drive brand metric uplifts throughout the purchase funnel vs those who we know definitely haven’t seen test ads. On average we see uplifts of +10.2% increase in Unaided Brand Awareness; +5.9% in Top of Mind Awareness; +2.3% in Brand Consideration; and +1.6% in Purchase Intent.

alt text

3. Significant brand multiplier for actively recalled ads: Despite the fact that brand impact happens for most people at the passive level, when ads do cut through and are actively recalled they can drive a significantly higher impact. Amongst the 24% of our sample who actively recalled seeing the tests ads that they were exposed to, they drove shifts in awareness metrics three times higher than the passive exposure group; along with six times greater shifts in Brand Consideration and seventeen times greater shifts in Purchase Intent. Indicative analysis reveals that those who actively recall ads are far more likely to be existing brand customers, so while ad cut through matters, it is likely to be your existing customers who will be most primed to recall your ads.

4. Passive ad measurement matters: Much of ad measurement still relies on the imperfect human memory to record ad exposure. Many brand tracking studies and ad impact measurement solutions determine who has and who hasn’t seen advertising through simple ad recall questions which ignore the effect that advertising has on those who do not actively recall having seen an ad. To ignore the impact on those who do not actively recall ads results in an under-estimation of total campaign brand impact. Passive measurement should not just be confined to the digital space either. ACR (audio content recognition) technology can be used to measure TV and radio exposure, while geo-location data enables us to track exposure to out-of-home media.