Frustrated by the accuracy – or inaccuracy – of prediction forecasts, Richard Shotton is taking matters into his own hands…
Last month, like last January and the one before, the marketing press was over-flowing with bold predictions. But how far should we trust these forecasts?
An experiment by Philip Tetlock, from the University of Pennsylvania, suggests caution. Over a 20 year period he analysed 82,361 forecasts from 284 experts and found that the predictions were just as likely to be wrong as right. In his memorable words, the average pundit fared no better than ‘a dart-throwing monkey’.
Worst of all Tetlock found that the forecasters favoured by the press were less accurate than average. Media outlets preferred pundits who had a single, big idea and then twisted messy reality to fit that simple but compelling story.
He termed these forecasters “hedgehogs”, borrowing from a phrase originally coined by Isaiah Berlin. His data suggest that in contrast “foxes”, those who drew inspiration from many sources and accepted the uncertainty of predictions, were far more likely to be accurate.
[advert position=”left”]
Unfortunately, our industry favours “hedgehogs” over “foxes”. Perhaps this explains why media and marketing predictions are so often wrong. History is littered with poor predictions from impressive industry figures:
For example, Darryl Zanuck, the legendary producer at 20th Century Fox, said in 1946 “Television won’t be able to hold on to any market it captures after the first six months. People will soon get tired of staring at a plywood box every night.”
Nor have our predictions haven’t improved with time.
Steve Chen, co-founder of YouTube, talked down his company’s future in 2005 when he famously said: “There’s just not that many videos I want to watch.”
In 2007 Steve Ballmer, Microsoft CEO, surpassed himself by saying that: “There’s no chance that the iPhone is going to get any significant market share.”
So what can we do?
I’d like to make a proposal: any prediction in the trade press has to be accompanied by the pundit’s attempts from last year.
There would be two benefits. First, by publicising inaccurate historic predictions it would puncture our collective over-confidence and demonstrate that the uncertainty of “foxes” is justifiable. Second, it changes the pundit’s incentives: from just being interesting to also being accurate.
Hopefully, if we adopted the suggestion predictions in 2016 might be more accurate than those from 2015.
Richard Shotton is head of insight at ZenithOptimedia
Twitter: @rshotton
