What’s the big deal with Big Data?

Big Data is the technology focus of the year, and Hadoop is the software that is gathering all the attention of investors and startups. How much is hype and how much is ‘wave of the future’? There is always some amount of hype around new ideas, but I’ll argue Big Data has its value based in diversity.

Diversity of locations

Those who argue that Big Data is just hype (and there are a few) ignore the argument that there is a new diversity of data that has never existed before. The diversity comes from the combination of internal and external (and cloud), mobile, plus the explosion of personal data from social networks. 800M people have put their preferences and relationships on Facebook, but there are many, many other places where data is now available that was never reachable before.

Diversity of users

Big Data has benefited large organizations that could afford hardware and quantitative analysts for many years, but has been out of the reach of small to medium companies. The low cost of storage and open source software has made this technology available to a broad spectrum of companies. It will only get cheaper, meaning the adoption will increase.

Diversity of data sources

The proliferation of web-enabled devices means that satellite, RFID, machine-generated and many other sources of data are in the mix along with consumer information. This data, taken at the right volume and velocity, demonstrates patterns that are useful for predicting opportunities and problems in supply chains, contagious diseases, drug therapies, and a host of other meaningful ways. It opens the door to ideas and solutions that weren’t imaginable a short time ago.

Data traction

Big Data without context or purpose isn’t all that useful, and that’s where process management comes in. In Clay Richardson’s words, Big Data needs Big Process. In his post, Big Data Ain’t Worth Diddly Without Big Process, he argues that ‘Big Process’ is a move from silo’d approaches to BPM and process improvement to more holistic approaches that have a chance to effect business transformation.

Companies have long brought together subject matter experts and mapped how things should be done, but we’ve never really been disciplined about applying the wealth of available data in a systematic way that can revolutionize business. You can’t improve what you can’t measure, and Big Data is all about having the right, timely information to measure meaningful things and affect change.

That means process becomes less about rigid flows and more about an organization being able to use data in specific ways to improve product development, strengthen marketing and sales, and create better customer service. Big Data used well drives organizational change, and that change needs to be wisely managed as an agile process infrastructure.

The companies that don’t put their ear to the tracks by managing Big Data are the ones most likely to be disrupted. And there is lots of disruption ahead.


Categories: Data Analytics / Big Data, Disciplines, Featured, Strategy, Tech Strategy

Author:Chris Taylor

Reimagining the way work is done through big data, analytics, and event processing. There's no end to what we can change and improve. I wear myself out...

Subscribe to the blog

Subscribe and receive an email when new articles are published

6 Comments on “What’s the big deal with Big Data?”

  1. April 16, 2012 at 5:03 am #

    Chris, well put. I recently blogged about another aspect of big data (http://unapage.com/graphene-the-new-silicon/), which is the big computing power that is necessary to analyze it. Companies like Google are leading the field in using AI to heuristically mine big data to deliver results people can use.

    I believe that business leaders will increasingly be called upon to select the “right” variables to watch because big data will be much more available to those who need it.

  2. April 16, 2012 at 7:35 am #

    Even beyond the right variables, Big Data can’t be completely useful as a standalone solution. It needs to be part of the wiring that also includes standard databases and other tools. The company that will dominate the space will have an integrated solution. Read more at Forbes.

  3. May 11, 2012 at 9:04 am #

    Reblogged this on Fabless Labs.

  4. Jeanne Roué-Taylor
    May 13, 2012 at 10:31 pm #

    Reblogged this on Fabless Labs.


  1. What's the big deal with Big Data? | BPM For Real - April 15, 2012

    […] for more info: What's the big deal with Big Data? | BPM For Real […]

  2. Disruption: the Facebook and Big Data conspiracy | BPM For Real - May 8, 2012

    […] in turn means complexity needs to happen behind the scenes with heavy reliance on data analytics (Big Data) atop as much relevant data (like Facebook) as possible. The result is the right offer to the right […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: