Pick up any business publication and you will read articles about the impact of big data and how it is changing the world in which we live, making our world more certain and providing value to consumers that we never thought possible.
While there is a lot being written about big data, finding practical examples is more difficult. The reality, however, is that Big Data is around us, every day. Type a search term into Google, navigate using mapping software that has traffic patterns enabled or make use of Apple iTunes Genius mix, and you are the beneficiary of Big Data. Big data has three distinct attributes that differentiates it from other forms of data: 1) The data is more diverse, consisting of transaction data, free format text and is often sourced from multiple locations. 2) The data has many more diverse data attributes to describe a particular scenario and these change at a rapid rate. 3) the data typically has much bigger volumes than is the case with simple transactional data.
In their book “Big Data: A Revolution That Will Transform How We Live, Work, and Think”, Viktor Mayer-Schonberger and Kenneth Cukier make the very profound point that in a world where “n=all, correlation is all that matters and causation becomes irrelevant”. What this means in the real world, is that if the dataset you are working with represents every possible scenario, then the correlations will tell the truth. In the majority of cases, this is a very good outcome in that real life behaviour can be very accurately predicted. As far back as 1990, Prof Orley Ashenfelter put together a formula for predicting which year of wine would produce the best vintage. While disregarded and ridiculed by wine connoisseurs at the time, his formula has stood the test of time. As Chris Anderson wrote in his article “The end of theory; The data deluge makes the scientific method obsolete”, There is now a better way. Petabytes allow us to say “Correlation is enough” We can stop looking for models. We can analyse the data without hypothesis about what it might show.
As a company focused on the domain of Customer Experience Management and Customer Engagement, inQuba’s interest in big data is understanding how it can be used to monitor and manage customer experience. It turns out that the notion of “n=all” is highly applicable to these domains. While traditional research companies are focused on collecting datasets through expensive mechanisms that result in small datasets, we have developed approaches and techniques for dramatically increasing the datasets sizes and equally dramatically reducing the cost of data acquisition. What this means is that we are able to collect up to 3 times more data per month, every month for a year for about the same price a traditional research company can conduct a single phone interview survey study that can only be conducted once a year. While we are not quite in the petabyte domain, this does mean we can collect up to 10 000 data items of feedback from our customer’s customers. The data is available realtime and the data is processed and made available immediately. In the past, traditional research companies have developed complicated models to explain their research results, necessitated by their very small datasets. We are finding that simple correlation models are incredibly powerful in explaining the correlation between factors such as customer experience and revenue. The energy can then go into fixing factors where the correlations show poor customer experience and poor revenue. Equally the data correlations can be used to establish which customers will be interested in which products and when they are likely to buy. In the business world, correlations can instantly show the route to the potential revenue flows. The days of producing causation models that need months of academic debate are simply no longer relevant and therefore companies need to discard these models in the interests of driving revenue.
Big data is sure to change every aspect of life and even more so in the world of customer experience management and engagement as well as customer research.
