‘Big data is suddenly everywhere’ – we all know it, it’s why we read this very site. Be it as a member of the converted, or someone engaging the topic for the very first time, you just can’t deny that big data is everywhere. Furthermore, the value of big data is often highlighted as a ‘silver bullet’, untapped potential in a world full of questions. In the following article, Gary Marcus and Ernest Davis of The New York Times brush the hype away and take a moment to look at some of the problems of big data. They ask: ‘is big data really all it’s cracked up to be’?
—
‘By combining the power of modern computing with the plentiful data of the digital era, it promises to solve virtually any problem — crime, public health, the evolution of grammar, the perils of dating — just by crunching the numbers.’ A bold claim, but certainly not one you’ll find many people arguing against. If anything, we seem so overwhelmed by data that it’s just the natural reaction. There’s so much of it out there, it can only be a good thing. We know we need to utilise more data, we know it will mean better decisions, but it would almost seem as though the debate stops there.
In a move to reignite the debate, Gary Marcus and Ernest Davis draw our attention to some of the flaws of big data. The first of their nine problems is the fact that data alone ‘never tells us which correlations [between data sets] are meaningful’. Its one thing to have data, it’s another to understand it and draw relevant conclusions from it. Something you cannot do with data alone!
Without spoiling all the authors’ problems with data, I will highlight a further issue that I feel you, our readers will appreciate. They make it very clear that ‘big data is prone to giving scientific-sounding solutions to hopelessly imprecise questions. In the past few months, for instance, there have been two separate attempts to rank people in terms of their “historical importance” or “cultural contributions,” based on data drawn from Wikipedia. This is not such a far away question from ‘which is the best charity?’ or more specifically still, ‘where would I get the best social return on investment?’
The article reminds us that there is still a great deal of debate to be had. Despite its promise of better decisions and better philanthropic giving, ‘big data can reduce anything to a single number, but you shouldn’t be fooled by the appearance of exactitude.’
Be sure to follow our friends at The New York Times, Gary Marcus and please let us know your opinions below, or tweet us, @MarketsForGood.