Big data means everything and nothing (it depends who you ask)

arm wrestlingIt seems big data means something different to everyone. In the great debate/hype about big data, there’s no lack of opinion on the topic and it seems to mostly depend on an individual’s product, skill set and business challenges. This ambiguity shares a great deal of the blame for why the term is often polarizing and why there’s a fair amount of cynicism in corners of the marketplace. Just for fun, let’s take a look at some of the points of contention.

  1. Big data isn’t anything new – This is a very legitimate argument for why big data doesn’t deserve quite so much hype. You’ll hear this argument mostly from the companies that have been solving problems and earning a living with vast amounts of data for decades. There are exceptional examples of this like Nielsen, the company that started off rating the advertising value of media and morphed into consumer preference and pattern juggernaut.
  2. Big data is really about small data – Also a legitimate argument against some of the hype. Companies that crunch data sets, small or large, often find that the pattern exists in just one variable, like the way preferences for wine often come down to our tolerance for acidity. Some of what’s called big data isn’t big when the results come in, but it often takes large data sets to prove that a small amount of data matters…a big data paradox.
  3. Big data is about the right algorithm, not more data – Like the other two points, this is also mostly true. The shows up in the crowdsourced contest Netflix used to improve on the company’s Cinematch predictive powers, which became about tiny tweeks to algorithms to raise results by .01%. There was no human X factor that solved the problem. This argument pits the traditional quantitative analysts against the new breed of data scientists. You could say it is also the fight between math and science and causation and correlation. This is a fascinating debate and I suspect both sides are right in differing circumstances.
  4. The 3 V’s (volume, velocity and variety) aren’t enough – Coming up with a new V for the description of big data is now the object of derision. “How many V’s do you have?” comes up often as an easy way to understand someone’s perspective on the topic but has also reached the point of silliness. Gartner’s Doug Laney came up with the 3 V’s back in 2001 and the debate has raged ever since around value,
  5. Big data is creepy – This one really depends on the definition of creepy. People with Rain Man-like capabilities have always been able to mentally process exceptional amounts of data and that ability could be used to cheat, manipulate and get ahead. Just because we’re able to see more complex patterns in ever more data doesn’t make big data itself creepy. Its use, just like before computerization, is what can be creepy.

At the end of the day, big data is going to continue to be a topic of intense debate because so much of what we do is affected by someone’s ability to gather, analyze and then predict who we are from our patterns, even the non-transactional ones like social media use. Enterprises can’t afford to ignore the technology that their competitors are using to better understand customers and be more efficient in their operations.

Join me in Las Vegas

The big data debate is an excellent example of the value of tapping a wide variety of experts to wade through the hype and find value. I’ll be hosting the Big Data Workshop at InterOp in Las Vegas on May 7th and invite you to join us to hear from a broad variety of sources like TIBCO, IBM, Datameer, Fabless Labs, QLogic, HP, Dell, Talend, and the editors of Big Data Republic and Venture Beat. Rarely will you find such an opportunity to hear a full day of so many valuable perspectives.

Receive 25% off the onsite price of a Conference Package or register for a Free Expo Pass with Priority Code “DISPEAKER“.

Screen Shot 2013-04-29 at 9.16.48 AM



Categories: Data Analytics / Big Data

Author:Chris Taylor

Reimagining the way work is done through big data, analytics, and event processing. There's no end to what we can change and improve. I wear myself out...

Subscribe to the blog

Subscribe and receive an email when new articles are published

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: