Embrace contradictions in data to get the most valuable insights

Opinion

If we don’t reveal and accept contradictions in data, are we truly demonstrating an understanding of real life?

Some people shy away from conflicting information and opinion. They immediately want to refute it, to uphold one piece of data as superior to another.

This is one of the points raised in an important new paper from the MRS: The Rise of the Insight Alchemist. The paper advocates a data agnostic mindset, acknowledging that all data is imperfect so we should use a range of sources.

I have always found conflicting data and opinion fascinating. Any criticism that comes my way my first reaction is always – well, that’s an interesting bit of data with a different perspective from my own –  I wonder what I can learn from it?

This is not a common reaction. Most people prefer the simplicity of a singular truth that supports their existing data and opinions.

In my experience, contradictions in data are not welcomed. A first reaction is to elevate the status of one source over another so as to be able to dismiss a piece of data and land on the magic answer.

However, since we live in a world of ever expanding data, we are more likely to encounter contradictions, so this practice is no longer realistic. Access to data is spread across a broader range of disciplines, more non-research practitioners are using data. This makes the challenge tougher as data is being used by people with less experience and confidence.

Now, of course, data contradictions can be confusing and potentially undermining. Clients want clarity and confidence to aid their decision-making, contradictions appear at first glance to be unhelpful. It can be tempting to downplay them and only share part of the story that makes sense at a surface level.

I believe shining a light on data contradictions can be revealing and incredibly useful if you know how to navigate them.

Data, whether it is self-reported via surveys, or behavioural data collected by technology, has a clear purpose, and that is to provide understanding of people. Since humans are innately complicated and contradictory creatures, it’s inevitable that we will encounter inconsistencies. If we don’t reveal and accept contradictions in data, are we truly demonstrating an understanding of real life?

Contradictions relating to context

In real life, two supposedly opposing things can be true. For example, I might claim to be happy with my standard of living but still want a pay rise. This is because, when I think about my standard of living, I am comparing myself to the population at large. However, when you ask about my salary I will compare myself to other people in my profession. It’s important to remember all data is collected in context even when it is not stated.

So how do we navigate these contradictions? Here are some things that I have learnt and some perspectives from a few other people who know a thing or two about research.

Contradictions between attitude and behaviour

This is one you might relate to if you are an advocate of small businesses. I might say I dislike a certain retailer but still shop there. The reason for my difference in attitude and behaviour could be down to convenience, cost or habit. Understanding the competitive context and what is driving consumer attitudes is important and will reveal to what extent disliking the retailer is an issue.

With the rise of behavioural data this is a key area which serves to underline the importance of using both survey and behavioural data sources.

Karen Fraser, senior director at Ipsos, agrees, saying: “We recognise the potential for multiple datasets to produce a more profound predictor of human behaviour.”

However, it’s not just about using multiple datasets or suppliers it needs to extend to team culture, a point that PJ De Queker, director of consumer research at Bose Corporation supports. Queker says: “I always encourage market research teams to tackle contradictions head on…. For example, reasons for buying a product can be different from reasons for using a product –  and only contradictory data can open your eyes to this gap to understand the differences.”

Trends and countertrends

This is one of my favourite areas, for every trend there is often a counter trend. For example, according to a new report by the7stars, we are witnessing a rising tide of consumer resilience. The media agency’s report finds that “Having navigated a pandemic, the optimism of many Brits has been replaced by resilience, with 64% of Brits agreeing that, ‘in difficult circumstances I keep calm and carry on’”.

I believe this can sit comfortably alongside all the data that says there is an increase in self-reported mental health issues. If we wanted to scrutinise these two perspectives further, it would be interesting to establish whether these are different audiences, or if self reporting is also driving up commitment to self help and perceptions of being able to bounce back.

Contradictions between how we present ourselves to others vs our genuine selves

An interesting contradiction was recently highlighted by Ipsos in its paper Addressing the sustainability say do gap. Ipsos found that “72% of individuals across 30 countries believe that they will be failing future generations if they do not take personal action to combat climate change.” At the same time, Ipsos reported a “stagnated behaviour change in real life, a “say-do gap” when it comes to sustainability”. Concern about this issue more generally is not always being followed up with genuine individual action.

Nuances in data

Sometimes a contradiction emerges because there is a subtle difference in meaning. This often happens when we explore concepts.

For example, we could see that consumers claim to distrust newspapers in general. However, when you ask them about the newspaper they read, they will say they really trust it. This reveals a subtle nuance and demonstrates that trust is a relative concept. It also underlines the importance of asking about people’s experiences as well as their overall perceptions.

Contradictions between sources

One of the most common things I have encountered is frustration from media planners that they don’t know which media channel data to ‘believe’. At first glance all channels would appear to be suggesting they are the best solution, apparently contradicting each other.

All too often though this data is not studied in detail. I believe the trade bodies each make a strong case for their varying roles; the data just needs to be looked at carefully. Whilst at Magnetic, representing magazines, I often quoted what might be considered competing research from the IAB, Newsworks and Thinkbox.

Justin Sampson, CEO at BARB agrees, saying that “When faced with contradiction, the line of least resistance is to opt for the explanation that chimes with the popular narrative. This approach too often reinforces a prejudice for what’s shiny and new, and overlooks more established —  and still important —  options and behaviours.”

I would add that ‘new’ can often mean a report that has been most recently released. Studies that were published a few years ago still hold incredible value. This was highlighted by the brilliant Radiocentre ‘Re-evaluating’ media’ study that referenced a body of ROI research spanning the last 10 years.

When you explore contractions you get to the most interesting insights. Exploring nuances in data requires experience and natural curiosity. It’s important to ask questions and be comfortable that you don’t always know the answer straight away. Sometimes it will involve going back and recutting the data or looking to another source.

The best insight practitioners build up a body of work they can add to with the aim of continually expanding their understanding. In this way you become cognisant of broader underlying trends in a particular field and confidence grows.

Data contradictions are everywhere. It’s important to work with a range of sources, whether it’s desk research sources or qualitative information alongside what is more obviously perceived as ‘data’.

So let’s get comfortable with data contradictions and use them to develop hypotheses so we can test and learn.

Anna Sampson is the founder of Anna Sampson Consulting and was previously insight and strategy director at magazine marketing body Magnetic.