When quantity is less than enough

Thinking Big Data will give you ‘the full, complete and true picture’ is a fallacy says Steve Smith, head of thought leadership research at Starcom MediaVest.

‘Big data’ is, well, big. Barely a few days go by without hearing or reading about it.

For social and market researchers alike, some of the excitement around big data is its coverage of a huge breadth of population together with a lot more information about an individual – their habits, what they buy, where they go, what they do.

On the one hand this is because more technology is in place to track people’s behaviour. On the other, a lot of people’s behaviour is online, which makes that tracking easier.

But despite the apparent benefits of big data to social and market researchers, big data is still quant. And with this are all the caveats that quantitative research and data bring. Don’t get me wrong. I use quantitative data. Heck, a lot of the research I do is quant. But knowing these caveats helps make a good quant researcher.

It appears to me that delving beneath some of the drive to big data is the belief that somehow we can get ‘the full, complete and true picture’ about what’s going on. The thinking goes something like, ‘If only we had enough data, then we could thoroughly understand people and uncover what makes them tick.’

This fallacy of ‘a full picture’ is a lot like the fable Jean Baudrillard wrote about in Simulacra and Simulation. In it, a great empire creates a map that is so detailed that it is as large as the Empire itself – it is fully complete. This is what some people believe big data will provide – a fully complete representation.

Baudrillard goes further and argues that this representation became the reality over time. This is what we have tended to do with quantitative data. Forgetting its limitations and caveats, we use quantitative data to create a reality made up of conclusions drawn from numbers and statistical models that at best underplay people’s lived experiences and tell us little about why people do what they do. But I’m getting ahead of myself.

This promise of a full and complete picture is understandably alluring. If we could somehow create this picture, then we can also create the absolute best customer experiences, products and services. And who doesn’t want to do that?

But of course, there is the obvious fact that there is always more to uncover and understand. Apart from this, there is also the erroneous belief that ‘cold, hard’ numbers can be truly objective. In the pursuit to legitimise social and market research as ‘sciences’ we have taken on the flawed belief that the researcher can stand outside of world they are studying and clinically observe it – the so called ‘God-like view’.

However, no knowledge exactly represents what it seeks to understand. Things are always mediated and they are always up for grabs. There’s always one way to tell a story. Your story about what happened in that meeting will be different from mine, even though we were both there, because of how we perceive things differently. And how many arguments have a couple had after a party, disagreeing about what ‘he said / she said’?

The same goes for quantitative (and qualitative) research. What we ‘find out’ from it is merely a version of what we seek to understand. We create stories about things that can be told in more than one way. A pragmatic solution is to create accounts that are persuasive, whilst accepting that there are things that might be told differently and that might have been misinterpreted or overlooked.

Another thing is that, whether we like it or not, we always take our values and ways of seeing the world into what we study. For example the questions we ask, how we analyse results, and how we present our findings.

And how many of us can truly say that we create hypotheses and then don’t try to ‘prove’ them rather than seek to disprove them (the true scientific approach approved by Karl Popper)? This doesn’t mean we shouldn’t do quantitative research or even use big data. Rather, it means we need to be more reflective about our own values, how we use quantitative data and what it seems to tell us.

So, let’s accept that everything is mediated. But a particular problem with things mediated through numbers is that numbers aren’t good at uncovering the meanings people attach to their actions – their motivations, justifications and intentions – the ‘why’ question.

Which brings me to qualitative data. Qualitative research, particularly ethnography, can be especially good at providing a ‘thick description’ of people’s everyday lives that quantitative research cannot provide.

Through qualitative techniques, researchers are able to uncover rich, underlying data about meanings and intentions they can then use to contextualise and help explain behaviours they observe. Put succinctly, quantitative data can tell us what happened, but it’s not good at telling us why it happened.

As an example, think of the last time someone smiled at you. Quantitative research will tell us that someone smiled. It may even tell us a reason according to a set of pre-coded categories. But it won’t tell us why they smiled in a person’s own words – what was intended by that smile because it misses the underlying understanding that is vital to more effective insights and outcomes.

Another result of the ‘God-like view’ is that it helps create and maintain an unhelpful distance from the people we are researching. It’s like standing at the top of a mountain and then trying to describe what we see in intricate detail.

It’s impossible because we are only able to create broad brushstrokes because we aren’t ‘up close’ and involved. No surprise then that we start using homogeneous terms like ‘consumers’ or ‘users’ to describe people, which doesn’t give justice to their complex lives, behaviours, and motivations.

As an example, I recently interviewed a few people about their home decorating projects. Sounds quite dull and uninteresting, perhaps, but during one of these interviews, a woman began to hesitatingly explain that she had only recently begun decorating her home because of the passing away of her mother.

Visibly upset, she started describing how she had been seeing a bereavement counsellor who had urged her to do something she would find fun and rewarding and would keep her busy. Decorating was her answer.

This rich data was unexpected, and was only uncovered through careful questioning and probing. Qualitative research can help us to get close and personal to people we are researching when we delve in deep to really explore the textures of their lives.

It helps create empathy, which in turn can create richer, deeper, and more insightful and explanatory data. There is no way even a carefully pre-coded quantitative questionnaire can do this.

None of this means we shouldn’t do quantitative research or use big data to understand people’s behaviour. We just need to understand their limitations and use them for the things that they are good for, and then dive beneath the surface to get a better understanding of people through qualitative data to explore the ‘why’ questions.

Which leaves me to wonder how much business has been lost, pitches un-won and devices unsold because we have refused to delve beneath quantitative data and so missed out on the rich insights we could have otherwise gleaned?