Cloud Matters

Big Data Predictions - Truth and Context

 

"It is difficult to make predictions, especially about the future" 

Among others, variations on this quote have been attributed to Yogi Berra, Dan Quayle, Niels Bohr, Albert Einstein and Confucius.  Perhaps that's because it's so obviously true.   

Still, people looking at how to predict the future  can sometimes do so quite accurately.  Others, not so much.

"The Americans have need of the telephone, but we do not. We have plenty of messenger boys."

Sir William Preece, chief engineer of the British Post Office, 1876 

One key difference is context.  Sir William was stuck in his own present context and clearly had trouble envisioning a new one. That new reality was completely different.  But to imagine it, to predict it, he would have had to question the need for all those messenger boys.  I wonder what he would have said about smart phones. 

Some predictions, made in context, can be pretty safely trusted - Moore's Law for example has held up well as a predictor of microprocessor transistor counts for more than 45 years.  In the case of Big Data the context is still being worked out.  

Last week, Information Week published some excerpts from a recent survey done by the Pew Internet & American Life Project.  The study, The Future of Big Data, asked 1000 internet experts for their opinions on just where Big Data is headed. What should we expect in 2020?  Responses were very varied and in some instances very scary.

The introduction sets the context:

"We swim in a sea of data … and the sea level is rising rapidly."  

And lists the interested parties:

"Government leaders, scientists, corporate leaders, health officials, and education specialists are anxious to see if new kinds of analysis of large data sets can yield insights into how people behave, what they might buy, and how they might respond to new products, services, and public policy programs."

So, unlike Sir William, we have some idea of what we're looking at.  Then the experts diverge.  

Start with McKinsey research that predicts, "five new kinds of value might come from abundant data:" 

"1) creating transparency in organizational activities that can be used to increase efficiency; 

2) enabling more thorough analysis of employee and systems performances in ways that allow experiments and feedback; 

3) segmenting populations in order to customize actions; 

4) replacing/supporting human decision making with automated algorithms;

5) innovating new business models, products, and services."

Overall, however, the Pew research was "decidedly split" with 53% seeing a positive Big Data future that will "enable new understanding of the world." And agree that "Overall, the rise of Big Data is a huge positive for society in nearly all respects.”

On the other hand, 39% view Big Data negatively, agreeing, " the rise of Big Data is a big negative for society in nearly all respects."  And believe that "The existence of huge data sets for analysis will engender false confidence in our predictive powers and will lead many to make significant and hurtful mistakes."

The report is fascinating and worth a full read.  If nothing else it provides some thoughtful views that will help to refine the Big Data context.  I believe that the most likely outcome is a mix of good and bad results, as did a few of the respondents who came down on both sides.

"Humans consistently seem to think they know more than they actually know in retrospect. Our understanding of technological effects, for example, lags by many decades the inexorable effects of implementation. 

So the best intentioned of humans will try to use Big Data to solve Big Problems, but are unlikely to do well at it. "

BUT...

"There are a few bright spots on the horizon. When crowds of people work openly with one another around real data, they can make real progress."

Check back in 2020.

 

 

 

 

Independence Day Redux

Three years ago cloud IT was still an emerging concept.   Intriguing.  

Two years ago the promise of cloud was better defined and many had already made a successfull start.  But new understanding was still unfolding.  

Now, Big Data is emerging very fast and in many ways enabled by and enabling of cloud.

( Check out Dan Hushon's blog for some perspectives on the meeting between the two.  And of course as always, Chuck's blog.)

So, I went back two years to reread my own perspective at the time, looking at independence as a cloud benefit.  Didn't see Big Data coming then and it'll be interesting to look two years from at what we aren't seeing today.  

I repost here:

Why is Private Cloud like Independence Day? 

My holiday ramble. 

Maybe this is a stretch but I'm entering a long weekend celebrating the 4th of July here in the US and a wandering notion struck me that there are similarities between the underlying principles of Private Cloud and the principles of independence. 

Bear with.

Start with the definition of independence: 1 : not dependent: as (1) : not subject to control by others : self-governing. Synonyms include: freedom, sovereignty, autonomy, self-rule. 

Then listen to some of the reasons given by those moving to the cloud - beyond the financial ones - and you hear a lot about freedom - freedom of choice in equipment, location, provider, service levels, data movement...  

Call it freedom.  Call it independence. Call it flexibility. A central idea of the cloud is freedom from the tyranny of centrally controlled IT.  That concept of centrally managed data centers - think the British Empire - was justified for purposes of standardization and control and security and because the technology for viable alternatives didn't exist. 

Now it does. 

With cloud, that rigidity is gone.  But with freedom comes responsibility and, sometimes, more than a little confusion and hype. 

However, I think the bigger implications are still unfolding.  They aren't technical either. They are more profound in regard to how we organize businesses, how we communicate and even how we look at the modern world.  We just don't understand them all yet. Can't even imagine.

The original section of the old house I live in was built in 1756.  Thomas Jefferson was just 13 and all of the best wide pine boards ("the kings lumber") were still being sent off to England - well not all, some are still in my living room floor.  

Twenty years later came the American Revolution.  Independence. 

So, what would the builder of my house, Josiah Hersey, think of where we are now?

How could he have imagined what the future of an independent America would be? 

How did he imagine the future?

How well do we?

 Too deep for a holiday weekend?

How about some fireworks instead?

Happy 4th!