Engagement: Are we there yet?

Becky McQuade (BSkyB) and Anne Mollen (Cranfield School of Management) think we might be…

Pure play content sites, like those in BSkyB’s stable, face a number of substantive challenges within the online display advertising trading environment. In addition to the commoditised attractions of ad networks, content sites are finding marketing resource being leached to social media – acknowledged as being increasingly central in people’s daily lives.

The rationale for this advertising shift is not only the consequence of the diversion of consumer attention to these communities but also the result of being judged against what we might call performance + metrics (conventional metrics augmented by ‘cost per engagement’ models that use a confection of behavioural algorithms that are assumed to deliver a better prediction of commercial return) and revised targeting strategies that infer that ‘internet activists’, those that tend to patronise social media, are inherently more valuable, by virtue of being more ‘engaged’, therefore more ‘loyal’ and more responsive to advertising.

The consequences of such thinking can be seen in the IAB ad revenue figures (First Half: 2011): while online ad spend had increased by 14% year on year against a media industry increase of 1.4%, social media spend had increased by an estimated 60%.

All online media players, desirous of being seen as the most ‘engaging’ of their genre in a context where consumers are exposed to an estimated 22 online ads a day*, are hampered by the fact that they are being judged against engagement metrics that, despite their notional arithmetical complexity and seemingly intuitive robustness, have not been empirically tested, not least because ‘engagement’ itself has no agreed working definition or operational parameters (it is what I essentially say it to mean), which has led some media commentators to dismiss it as a ‘vanity metric’.

BSkyB, in a joint project with Cranfield School of Management, therefore had three objectives:

1. To establish whether engagement was a viable metric (could it be defined and therefore operationalised?) and whether it was a robust predictor of commercial return.
2. To examine whether those behavioural measures were reliable proxies for the mental state of ‘engagement’.
3. To interrogate whether certain consumer archetypes (internet activists) are more likely to be ‘engaged’ than more ‘passive’ consumers.

The definition, operational parameters of ‘engagement’ and scale items were supplied by Cranfield School of Management. Online engagement is defined as “a cognitive and affective commitment to an active relationship with a brand, as represented by a website or computer-mediated entities designed to communicate brand value, at a point in time” (abbreviated definition).

The results of the study support our hypothesis that ‘engagement’ would show a strong positive correlation with key commercial drivers.

With regard to advertising effectiveness, we found that increased levels of engagement lead to higher advertising cut-through: recall of advertising was significantly higher for the more engaged respondents, with a 32% differential between mean engagement scores between those who could recall the specific creative and those who could not. We also found significant positive correlation between engaged users and agreement with some campaign characteristics, such as the ad “looked good” or was “eye-catching”.

Engaged users were more likely to associate the advertised brand strapline with the correct brand – in the Sky Entertainment study that equated to an 18% differential between engaged and non-engaged users. More engaged users were also more likely to have favourable opinions about the tested brand: with a 51% and 31% differential in mean engagement scores (Sky Entertainment Study & Netmums* Study, respectively) between those who responded very favourably and those who responded most unfavourably.

In the Netmums case study, we found that ‘engagement’ showed a strong positive relationship with key marketing metrics: Site Trust, Site Satisfaction, Site Loyalty, and Advertising Responsiveness. Additionally, engagement was shown to be a predictor of the Net Promoter Score metric.

Our findings also suggest while our experiential measure of engagement can be supported, the behavioural proxies currently used for the metric are flawed.

While there was significant positive correlation with such measures as ‘time spent’ and ‘visit frequency and recency’, the correlation strength was low. If we take one measure ‘time spent’, it is true that light users tend to be less engaged, and medium and heavy users more engaged.

However, if we look at the data we can see that for the majority of respondents who were medium users, while 51% were engaged, a substantial proportion (38%) registered low or very low engagement scores. Thus, time spent is not a particularly valuable discriminator of engagement.

We also found no correlation between a behavioural predisposition to ‘activism’ and engagement and no significant relationship with commercial drivers such as site trust, site loyalty, or advertising responsiveness. Thus, a propensity to certain behaviours does not in itself guarantee engagement or outcomes such as advertising responsiveness.

Our findings suggest that engagement is both a viable and worthy metric. Our study also points out that what we as media researchers feel is ‘intuitively’ right, such as ‘time spent on site’ or the typology of ‘internet activism’, is not necessarily empirically sustainable, to the extent that the current engagement metrics, and the media planning associated with such metrics, need to be far more rigorously challenged.

For more, please contact Becky McQuade and Anne Mollen

*comScore Ad Metrics
**Netmums, at the time of the study a partner site of Sky