Big Data: what is it and what do we do with it?

big-data 2

We’ve all heard people talking about ‘Big Data’. Some are excited by it and some are confused by it, but what exactly is it? Wikipedia defines it as ‘a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications’. If you’re still confused, or even more excited, read on to find out what Inspired Selection heard about at the @urbanonetwork event, with speaker Francine Bennett @fhr from @MastadonC.

Up until recently, we have been used to receiving and understanding data in a structured form, in spreadsheets or SQL. However, we now live in a world where we receive data from a diverse range of sources and often in a constant stream. In other words, imagine that businesses have gone from reviewing all data in an Excel sheet to receiving it as a Facebook Homepage, with links to other sites and images connected to comments from other social networks. Businesses might have access to what people are buying, what they’ve looked at but discarded, what their shopping history is and how they’re shopping. Big Data is the idea that we have so much data, from so many sources, that we need new ways of looking at it to be able to discover things about the data subjects (usually consumers) and to predict what they might do next.

Francine introduced some of the new ways, including software such as Hadoop, advanced analytics, text mining, machine learning and network analysis all of which allow the business or analyst to view the combined data sets from a new angle and derive findings. Different publishers may be using or considering different methods but many of them will be thinking about how to use sales data to inform product development or sales strategies. What do we know about buying behaviour to make us sell more? In an industry where more and more, we’re seeing a ‘consumer as king’ market, it is crucial that publishers utilise the data they have at their fingertips wisely.

We’re in an environment where lots more data exists and where storing and analysing it is cheaper. This will inevitably give rise to new opportunities for insight and revenue growth but when handling consumer data we must use it cleverly and with their best interests at heart.