Journey to the centre of big data

In a general context big data is an aggregation of data sets that are so large and complex that it becomes difficult to process using readily available database management tools or traditional data processing applications. This challenge also contains an opportunity for commercial organisations that are equipped to find ways to use it - or elements of it - to inform and enhance revenue drivers (see 'Puzzling out big data', E&T Vol 7 Issue 12). The term 'big data' is partly a recognition of the growing relative weight and importance of unstructured data not amenable to conventional database analytics and reporting tools and techniques. Above all, though, it embodies an ambition to extract value from data, particularly for sales, marketing, and customer relations.