Big Data is so much bigger than that

The following is a guest blog by Simon Hall from Newbury, England. In Simon’s words, “Sales and selling have been my life. I’ve never been more excited than I am now about what the future of technology and business can be.”

We’ve all heard Big Data can help us to find the proverbial needle in the haystack. That’s the promise that we hear about most often — that intelligence comes from a wide variety of high-velocity, unformatted data that can be crunched in parallel by distributing its storage and processing across a whole bunch of computers. If you agree this is Big Data in a nutshell, you’re very wrong. Big Data is so very much bigger than that.

Just to get the ball rolling, Big Data isn’t merely about distributed storage in databases. Well done, it is real time information that can be processed at thousands of times faster than with a traditional database. That doesn’t get talked much but matters enormously. Batch processing to get information is very 2009.

Why don’t we take a look at what makes Big Data so much more than a haystack analogy.

Machine data never sleeps

More voluminous and faster than traditional sources, machine data comes from automatic ‘scrapes’ of websites, from servers, from applications as they run, from network devices and from mobile devices (not just phones but things like UPS handhelds). These are the truly biggest of all big data sources as they run 24×7, every day, and churn out massive amounts of information. Big Data never sleeps.

This is really, really valuable information that can’t be followed and digested fast enough. It takes automation to make sense of it. Companies that invest in paying attention can lower their security risks and costs, improve their selling opportunities and customer/partner service, and gain an understanding of why something happens, not just when it happens. Once someone knows why, they can start to predict what will happen. Predicting is so much more valuable than find out later.

Before Big Data capabilities, many non-human sources of information were a record to be checked when something went wrong or to be analyzed as time permitted at some point in the future. Considering how much information is in those records, what a shame.

When you can see it happen in real time, instead, you can do something about it. Big Data is actionable.

Machine data at work

In the moment:

  • How important is it to know that a bank’s customers are having trouble logging in?
  • What if a global retailer’s online transactions were not completing?
  • Where would a package service find out about a pattern of overdue deliveries?
  • Who would want to know that beer deliveries are late on a holiday weekend?
  • When would someone discover that sales promotions aren’t working?

In the future:

  • Who needs to know that denial of service attacks will come from a particular region?
  • How would we benefit from knowing when a heart attack will occur?
  • What if the studio knew exactly what movie would perform the best?
  • When would we want to know the next natural disaster’s location and timing?

Waiting for people to spot trends or react to high volumes of information is a really poor option. Waiting for historical data before reacting to challenges or opportunities is another really poor option.

I’ll go out on a limb and say these are the current options for companies waiting to go out of business.


Tags: , ,

Categories: Data Analytics / Big Data, Featured, Real-time

Author:Jeanne Roué-Taylor

I'm fascinated by disruptive technology and its impact on our world. I manage sales operations for an excellent startup with a unique team of highly experienced data scientists.

Subscribe to the blog

Subscribe and receive an email when new articles are published

One Comment on “Big Data is so much bigger than that”

  1. Simon, you could not be more right. Your thoughts and expression of BigData “Calls” are spot on. Im a 50 yr old who has risen thru the ranks from Trainee Programmer programming in Fortran IV in 83 and then forcefully moved to Management in IT becoming a VP of a few global companies. IN last few years I was so bored of my Mgmt job that I started coding and put up my website in java, then Android App, now I own some servers in EC2 running hadoop and offlate installed H on my laptop too. IM so excited about prospects of BigData that I cant even sleep ! Keep writing and inspiring

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: