There are some fundamental things to know about Big Data. Start with the ambiguous term itself. It’s important because it is affecting each of us already and the impacts are rapidly increasing.
I wrote a piece for “C-Suite Insider” that does a high level Big Data intro. This post is a follow-on to that.
There are multiple elements that create the value of Big Data, that make it “Big” in scope as well as importance. Start with a collection of data sources and technologies that enable new measures of predictive analytic insight. Use all of this to gain techniques that enable rapid decision-making leading to competitive advantage.
A collection of data sources…
In addition to the immense size of data sets, the new technology takes advantage data types beyond traditional relational and transactional databases. This incorporates semi-structured data from social media and unstructured data such as text, photos and video. It is this variety of sources, taken together with data sets that were previously neglected that are providing brand new value and insights.
A murder study applied standard health tracking methods to data on Newark N.J. murders and developed their conclusion that the spread of homicide can be predicted. It spreads in the same way diseases do.
Santa used data from “millions of sources worldwide”, including report cards, detention records, chore charts and text messages to determine who was naughty or nice. Suspend your disbelief.
Big Data is being employed to make Academy awards predictions which are based upon other awards already given, historical trends, Las Vegas bets and more with weekly updates to keep the predictions on track right through February 24th - when Argo will win best picture. Watch.
What are the Big Data business implications?
By most indications, impacts will be felt across all industries. Many are well on their way, particularly insurance, finance, retail and manufacturing. It is being used to detect fraud, analyze customer satisfaction, or enhance risk analysis for banks. A McKinsey study estimates the impacts of data analytics on US health care could be $300 billion in savings.
A collection of technologies
A variety of hardware and software technologies have developed to manage Big Data. They provide capabilities that range from reporting, to data mining to predictive analytics. Many are also in an early phase of maturity. You can find a good high-level review of the technologies here at Wikbon.org.
The technologies include several elements that, for the first time, are capable of creating ways of managing and analyzing extremely large data sets that are spread across parallel computation and storage nodes. What makes this different is the capability to use relatively inexpensive hardware to examine ever more massive amounts of data of various types, including structured, semi-structured and unstructured data.
There are many familiar company names and quite a few new ones in the mix. Forbes created a big data graphic that is useful in sorting through the various companies and technologies involved but keep in mind that it’s a rapidly changing landscape. Also, while the technology is interesting and essential to perform the function, it is the output and insight that’s getting all the business attention.
A new means of predictive insight and decisions through analytics
One very significant difference in this new digital landscape is that it looks forward more than back. It can also focus much more specifically on an intended target.
On-line advertising is an example most of us will recognize. It used to be that a mass-market campaign would repeat over an over to get the message out. They would print or broadcast and hope to hit a target audience. Now, in the digital landscape, they can target specific individuals and precisely measure impacts. They track your “digital exhaust” and on-line habits and use sophisticated algorithms to customize offers that suit you. With mobile devices gaining massive traction globally, it’s very big business. For example, ecommerce transactions are expected to grow from $30 billion in 2012 to more than $320 billion in 2016.
Several dramatic examples of predictive analytics have emerged in health care – some based upon data that was previously discarded.
MIT researchers used data mining and machine learning to apply previously missed data from EKGs to identify 70 percent of the patients most likely to have repeat heart attacks.
University of Ontario researchers on a project called Project Artemis used premature baby heart monitoring data to create an early-warning system that predicts infections a day earlier than they become symptomatic.
Much of this is still experimental or early stage. As the level of sophistication improves you will see these kinds of applications expand and proliferate. And so will the attendant issues of management, security, privacy and ethics.
A competitive advantage
More than ever this New World of Data means information is currency. With potentially significant competitive advantages to be gained, some sectors will gain more than others. In early stages at least, look for advances in tech/IT companies, retail, finance and government.
McKinsey believes that a “retailer embracing big data has the potential to increase operating margin by more than 60%.” (I’m skeptical.) And “ forward thinking leaders can begin to aggressively build their organizations’ big data capabilities” that will “…confer enhanced competitive advantage over the long term…”
Some other sectors such as health care and education could see great value gains but need first to invest in the supporting infrastructure and data-driven capabilities.
Several issues will have to be addressed to capture the full potential of Big Data. Policies related to privacy, security, intellectual property, and even liability will need to be understood and confronted. Organizations need not only to put the right talent (in short supply and very expensive) and technology in place but will also need to structure workflows and incentives to optimize the use of big data.
It is an exciting and rapidly changing arena with unforeseen implications in social, civic and economic domains.
So here are five we briefly touched:
- a collection of data sources
- a collection of technologies
- new measures of predictive analytic insight
- new techniques for rapid decision-making
- new means to gain competitive advantage
Here are five we haven’t touched yet:
- security
- privacy
- data classification
- data science resources
- Big Data skeptics
Plenty to examine in future discussions.
Comments