"It is difficult to make predictions, especially about the future"
Among others, variations on this quote have been attributed to Yogi Berra, Dan Quayle, Niels Bohr, Albert Einstein and Confucius. Perhaps that's because it's so obviously true.
Still, people looking at how to predict the future can sometimes do so quite accurately. Others, not so much.
"The Americans have need of the telephone, but we do not. We have plenty of messenger boys."
Sir William Preece, chief engineer of the British Post Office, 1876
One key difference is context. Sir William was stuck in his own present context and clearly had trouble envisioning a new one. That new reality was completely different. But to imagine it, to predict it, he would have had to question the need for all those messenger boys. I wonder what he would have said about smart phones.
Some predictions, made in context, can be pretty safely trusted - Moore's Law for example has held up well as a predictor of microprocessor transistor counts for more than 45 years. In the case of Big Data the context is still being worked out.
Last week, Information Week published some excerpts from a recent survey done by the Pew Internet & American Life Project. The study, The Future of Big Data, asked 1000 internet experts for their opinions on just where Big Data is headed. What should we expect in 2020? Responses were very varied and in some instances very scary.
The introduction sets the context:
"We swim in a sea of data … and the sea level is rising rapidly."
And lists the interested parties:
"Government leaders, scientists, corporate leaders, health officials, and education specialists are anxious to see if new kinds of analysis of large data sets can yield insights into how people behave, what they might buy, and how they might respond to new products, services, and public policy programs."
So, unlike Sir William, we have some idea of what we're looking at. Then the experts diverge.
Start with McKinsey research that predicts, "five new kinds of value might come from abundant data:"
"1) creating transparency in organizational activities that can be used to increase efficiency;
2) enabling more thorough analysis of employee and systems performances in ways that allow experiments and feedback;
3) segmenting populations in order to customize actions;
4) replacing/supporting human decision making with automated algorithms;
5) innovating new business models, products, and services."
Overall, however, the Pew research was "decidedly split" with 53% seeing a positive Big Data future that will "enable new understanding of the world." And agree that "Overall, the rise of Big Data is a huge positive for society in nearly all respects.”
On the other hand, 39% view Big Data negatively, agreeing, " the rise of Big Data is a big negative for society in nearly all respects." And believe that "The existence of huge data sets for analysis will engender false confidence in our predictive powers and will lead many to make significant and hurtful mistakes."
The report is fascinating and worth a full read. If nothing else it provides some thoughtful views that will help to refine the Big Data context. I believe that the most likely outcome is a mix of good and bad results, as did a few of the respondents who came down on both sides.
"Humans consistently seem to think they know more than they actually know in retrospect. Our understanding of technological effects, for example, lags by many decades the inexorable effects of implementation.
So the best intentioned of humans will try to use Big Data to solve Big Problems, but are unlikely to do well at it. "
BUT...
"There are a few bright spots on the horizon. When crowds of people work openly with one another around real data, they can make real progress."
Check back in 2020.
Comments