Data is invaluable to advertisers – but are we in danger of having too much? ZenithOptimedia’s Richard Shotton and Richard Clay find out.
The rise in the variety of data available to advertisers has been one of the most exciting changes in the industry over the last few years. Click data, search patterns, tweets: all can be used to better understand consumer behaviour. This is a great opportunity for insight generation as it stops us relying on just one methodology.
Since every approach has its own flaws it’s importance to use a variety of sources. Where multiple methodologies point to the same conclusion, greater credibility can be given to the findings.
However, whilst relying on a single source of data is dangerous, an excessive variety can be problematic. Paul Slovic, currently professor of psychology at the University of Oregon, ran an ingenious experiment with professional horse-racing handicap setters investigating this phenomenon.
The handicappers were given a list of 88 variables that were useful in predicting a horse’s performance. They then predicted the outcome of races using either 5, 10, 20, 30 or 40 of the variables.
The results were illuminating. Accuracy was the same regardless of the number of variables used. However, over-confidence grew as more data was harnessed. Experts over-estimated the importance of factors that had a limited value. It was only when five data points were used that accuracy and confidence were well calibrated.
If this were a rogue finding then it wouldn’t justify much concern. However, Slovic’s results have been repeatedly validated.
Stuart Oskamp, psychology professor at Claremont Graduate University undertook a similar experiment with thirty-two clinical psychologists. They were given a case history split into four sections, each one dealing with a successive chronological period of a patient’s life.
After the psychologists read each section they answered twenty-five questions, to which the answers were already known, about the patient. As with Slovic’s experiment increased information led to a significant increase in confidence but a negligible increase in accuracy.
One marketer recently described his feelings towards ‘Big Data’ through the analogy of “being like a kid in a sweet shop”. This unbounded enthusiasm is dangerous. Advertisers should avoid incorporating data into the planning process just because it exists.
Instead, as much time, energy and effort should be invested in choosing which data sets are the most valuable as in interpreting the data. Advertisers who resist this painful cull and gorge on data might, like the proverbial kid in the sweet shop, end up feeling rather sick.
