….Although the Big Data Initiative only includes scientific agencies, it turns out that the humanities have been involved for just as long. In 2009, when Data.Gov was being launched, the first Digging into Data Challenge was held. Sponsored by the NEH and comparable organizations in the UK and Canada, the eight winning Digging into Data teams used high-performance computing stacks to analyze everything from Enlightenment correspondence networks to the rise of the railroad in 19th-century America to “structural analysis of large amounts of musical information.” This humanities-based exploration of big data was so successful that the competition happened again in 2011 and 2013 and added, in the meantime, six more funders and one more nation, The Netherlands (hup Holland!).
Why the sudden interest in Big Data on the part of the federal government? There are certainly many reasons, including Obama’s adept use of data mining in both his 2008 and 2012 campaigns. This is a man that knows the power of large-scale data.
But perhaps we could point to a 2008 article in WIRED—”The End of Theory“—as representative of this zeitgeist as well. In it, Chris Anderson, then editor in chief for WIRED, writes about the role of Big Data for every field of knowledge:
“The scientific method is built around testable hypotheses. These models, for the most part, are systems visualized in the minds of scientists. The models are then tested, and experiments confirm or falsify theoretical models of how the world works. This is the way science has worked for hundreds of years. […] But faced with massive data, this approach to science—hypothesize, model, test—is becoming obsolete. […] There is now a better way. Petabytes allow us to say: ‘Correlation is enough.’ We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.”
What Anderson is suggesting is a vision of data in the mold of Hari Seldon or pointy-haired bosses: the idea that data will save us.
Red Herring #1
And this, finally, is the first red herring of big data: the belief that data on its own is enough to simply answer any and all of our questions. To think that running the numbers is enough. To mistakenly believe that the data is not the means but the end. But this is false….