Big Data: lipstick-on-a-pig, or genuine insight?

Beneath all the buzzwords and attractive visualisations, IMGROUP’s Paul Collins wonders whether big data can actually provide actionable and focused insight.

How many times must I have heard someone in IT say that their organisation needed a Big Data solution, or rave about some great piece of new technology they want which can analyse social media and provide amazing dashboards in a flash.

Sometimes I find myself thinking that if everyone else is going down that path (although actually they aren’t, despite what suppliers may want you to think), maybe I should too. Perhaps you feel under the same pressure, although you might wonder how your organisation would cope with a Big Data solution, when in your opinion IT already struggle with the high volume of smaller data and research requests.

There is another way to obtain useful insight, a way which might not require such a massive investment in new technology. Lots of organisations already collect more data than you can shake a stick at; some may be sitting on a shelf gathering dust, the rest may just be in need of a tune-up.

Putting aside the Big Data buzzwords and the massive workload adapting to Big Data usually entails, it’s really all about making use of what you already have (or can easily get) in smarter ways. It’s about focusing on converting data into insight which you can use to drive the business forward, and having a smart, cost-effective, long-term data strategy which uses the right data in the right way for maximum impact.

So the discussion about data is moving on towards smart and actionable insight, and the next wave of sales pitches will soon sweep up and down the land. They’ll tell me that in addition to a Big Data solution, I must have, courtesy of so-called data visualisation and insight specialists, shiny new data analysis and visualisation toys.

Naturally, these toys look great in demonstrations, and will have execs and IT departments drooling over the glitzy visualisations and brilliant dashboards; they will want them, and they will want them now.

But what’s the old saying? Let the buyer beware.

These tools are meant to help the user understand the past, present and future, track against KPIs, and enable the user to ask questions about and make decisions based on the insight gleaned.

To answer these questions and deliver the corresponding dashboard requirements, I need to have confidence that the data being fed to these tools is both accurate and complete, since the visualisations will have no real value if the data is unreliable or inaccurate.

Many of these new wave suppliers focus on how to make attractive visualisations; they’re not so good at delivering the data sourcing effectively, or combining and maintaining its quality on the long-term basis required to produce useful insight and facilitate effective decision making.

If my data provision is not managed properly, all the above is so much lipstick on the proverbial pig. It may look slightly prettier, but it’s still a pig.

In terms of providing actionable insight to help me run the business, all I’ll have is a prettier way of looking at unreliable information. This will lead me to make decisions based on unreliable information, which will be at best less than optimal, at worst unsound and costly.

I need to qualify, identify, clean up and transform what data I already have. By adopting a purely IT approach (i.e. looking at the latest and greatest platforms, technologies, tools and, yes, Big Data solutions) one stands a high risk of deciding on the answer before hearing the question.

The IT perspective is important, but it is critical not to put the cart before the horse. The main drivers of any change should be the business imperatives and questions that need answering – i.e. what data and insight is needed by whom to help them to run their part of the organisation? For example, what information do sales and marketing teams mean when they say “if we only had this data against these KPIs, we’d be able to prevent churn, increase acquisition, reduce marketing spend,” etc.? Consider the same question for operations, accounts and so on.

In any organisation, the likelihood is that most of the required data is already around somewhere, and that much of the rest will be available externally, sometimes free of charge, sometimes not. The problem is normally that no-one has either:

– Worked out what data is needed, by whom
– Compared that with what you have and could have
– Worked out how to combine and provide it

I need someone to identify the data needed to feed my KPIs, and then determine the most effective base data model needed to supply not only the core KPIs, but also ad hoc requests, which will undoubtedly follow once decision makers realise what valuable insight is available when the data is stored and provided efficiently. This is not a Big Data solution; this is a focused data insight solution.

Once this exercise is completed – which, if done effectively, will take a matter of weeks rather than months – I can then determine whether I need a focused data solution, or I do in fact need a Big Data solution or some other flashy gadget.

If it turns out that I do need one of these three things, the decision to proceed can be made with the full blessing and support of the executive business stakeholders, because if done properly, the business case for this new capital investment will already be there.

So, before putting lipstick on the pig, or chucking out the pig and replacing it with a Big Data white elephant, take take a step back to consider what insight you need for the day-to-day running of a business. Have a look and see whether you’ve already got it.

You may just need some help in joining the data sources together into an insight-focused data model that will provide the base for great looking, valuable insight with which to drive your business forwards.

Paul Collins is head of CME and all types of data consultancy at IMGROUP