Hadoop: A race car without wheels?

Hadoop without infrastructureI had a chance to both present on Big Data this week and to sit in on several other talks on the topic. There was a clear theme that emerged…Hadoop and other Big Data-specific tools are just the tip of the iceberg. By themselves, they solve nothing.

Part of the confusion arises from vendor marketing and analyst hype, and part comes from the ‘bright, shiny object’ problem that we all suffer from as human beings. We like the new stuff. Beyond our fascination with all things new, not as many people know and understand the newest technology, making expertise a personal differentiator. It’s good for our careers.

Success matters more than knowledge

Race Team

But so is success. The reality is that without a great deal of the ‘old stuff’, the things that most companies still struggle to get right, Big Data is a research project and not an operational possibility. Insight is fantastic and Big Data technologies certainly help deliver insight, but only when the right data is available and only when the right sense and respond systems can act when it matters most.

Big Data without the things that have always been important…well integrated infrastructure as a great example, is like a car sitting on blocks: All potential, not much reality. Big Data technology is just part of an ecosystem that touches broader capabilities that extend from networking to process.

We need to stop thinking of Big Data as a set of ‘new’ technology tools and see it as a need to handle the rapidly increasing data that drives risk, opportunity and efficiency. That will put us on the right course for choosing how to approach the challenge and keep us from having a ‘race car on blocks.’

Really, really good filter

The other key point came from a conversation with a clever European architect who kept asking me the question, “But how does it scale?” I couldn’t figure out what he meant at first and thought it was a language barrier. Then I realized he meant, “How does Big Data scale?” He was looking at the wrong way…Big Data isn’t a scaling problem. The problem is how to pare the data down, find the key insights and not how to move more data through your systems. Big Data isn’t about transactional volumes. Big Data is about finding just the information that matters and then making the most of it.

Big Data is about having a really, really good filter that makes the most of what passes through.

Advertisements

Tags: , ,

Categories: Data Analytics / Big Data

Author:Chris Taylor

Reimagining the way work is done through big data, analytics, and event processing. There's no end to what we can change and improve. I wear myself out...

Subscribe to the blog

Subscribe and receive an email when new articles are published

2 Comments on “Hadoop: A race car without wheels?”

  1. January 14, 2013 at 6:59 am #

    “The problem is how to pare the data down, find the key insights and not how to move more data through your systems.”

Trackbacks/Pingbacks

  1. Big Data myths and real decisions | Successful Workplace - March 10, 2013

    […] even the smaller part. Big Data is actually an ecosystem of technologies that go well beyond the Hadoop platform and well beyond just […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: