Big data should lead to big dreams

Dream bigOrganizations are collecting more and more data every single minute of every day. It’s no secret that Facebook, for example, processes 2.5 billion pieces of content and over 500 terabytes of data each day, pulling in 2.7 billion Likes and 300 million photos per diem. Facebook also scans a whopping 105 terabytes of data each half hour.

Of course Big Data is nothing new. It was relatively enormous all the way back to the 1960’s when NASA took a shot at the moon. It shouldn’t have been a surprise when Google gave everyone the proverbial heart attack when they revealed they churn almost the entire Internet every couple of days to meet our search needs.

Despite the hype, Big Data didn’t happen all at once…data just got bigger.

So we know that companies manage, manipulate and extrapolate information from ever larger amounts of data. But what we don’t know is whether they are asking anything bigger from it all. They should be.

Limitless information, limited imagination

A customer-focused business with Big Data in its grasp has an unparalleled source of knowledge from an increasing number of sources now; mobile data, social data, transactional data, locational data, financial data, family data, medical data, carbon footprint and consumption data. We even have data about data in the form of log data, as Tesla showed us in rebutting the NY Times.

What’s more, a similar increase of that information is being collected in real-time with lots of integration challenges. (But often stored very traditionally and processed in batch. What use is that for operational decisions?)

But with all this information at hand I’m seeing a worrying trend of organizations still asking the same questions of it and receiving the same answers as before, just with a little bit more data support behind it. When you combine social + mobile + medical + financial + family you get quite a bit more than just demographic segmentation. Too often, that’s the apparent limit to where current thinking goes in customer service and marketing.

Limitless information, limited processing

There is another angle I want to work on here, and something I’ve mentioned aplenty elsewhere. This amount of data requires a modicum of processing power. Whilst you can move it all to the Cloud and leave it to another provider to churn the numbers for you there’s nothing to stop you from applying a little distributed magic yourself and using the idle processors sitting on every desk in the organization. In fact, what if AT&T managed to work out how to use idle clock time in every one of its smartphone to process its own data from its customers ?

SETI@Home famously did this by allowing 3 million users to assign their PCs and PlayStations to solve computational data from radio telescopes. It’s not such a far fetched notion for a business such as mobile provider or a bank to do exactly that via an app.

Limitless information, unlimited possibilities

Big Data actually demands of us big questions. It demands us to think bigger than what we’re currently doing. TIBCO CTO Matt Quinn put it well when he said we should be asking ourselves,

What’s the No. 1 question that you’ve wanted to have answered but you were always told it was impossible? Start from there. Don’t start with what data you have, start with the important question.

The term Big Data may just be part of the industrial hype cycle since it’s all about the Data, but the Big part should be a reminder to dream big.

Tags: ,

Categories: Data Analytics / Big Data, Real-time

Author:Theo Priestley

"I had more creative ideas from Theo in 6 months than I have had in 6 years from most people." Theo Priestley is one of the most recognised independent technology industry influencers and evangelists, ranking in the Top 100 thought leaders across Virtual/ Augmented Reality, FinTech, Artificial Intelligence, Big Data, Internet of Things and future trends. Theo has written insights for Forbes, Wired, The European Magazine, Venturebeat to name a few, and has been interviewed for many online publications including the BBC on his thoughts on technology and the future. A regular paid keynote speaker and panelist at conferences and events, Theo is engaged for his forthright views and isn't afraid to challenge conventional thinking and the marketing hype surrounding the industry when presenting, never pulling punches to get the message across on how technology can be applied to improve business and the customer experience. He has also successfully organised and run TEDx and Ignite events. Highly active across social networks, he sits in the Top 1% for social media engagement on Kred and Klout and is constantly sharing articles and his analysis that he feels his audience would be interested in. Theo is also active in the startup community, mentoring within UK and US accelerators and sits on a number of advisory boards. Former VP and Chief Technology Evangelist at a Top 25 European enterprise software company with a career spanning both innovation strategy and delivery of software and business change in Financial Services, and as an independent technology industry analyst. Follow Theo on Twitter @tprstly or connect here directly for constant insights on tech and marketing trends. • Top 1% Influencer on Kred (915) • Top 1% Influencer on Klout (70+) • 12,000+ Followers on LinkedIn • 13,000+ Followers on Twitter • Recognised Top Influencer in AI, Virtual/ Augmented Reality, Fintech, IOT and Wearable Tech, Big Data and Analytics.

Subscribe to the blog

Subscribe and receive an email when new articles are published

One Comment on “Big data should lead to big dreams”

  1. Azana Baksh
    February 25, 2013 at 12:12 pm #

    Good insight Theo. We are seeing an increase in businesses seeking specialized skills to help address challenges that arose with the era of big data. The HPCC Systems platform from LexisNexis helps to fill this gap by allowing data analysts themselves to own the complete data lifecycle. Designed by data scientists, ECL is a declarative programming language used to express data algorithms across the entire HPCC platform. Their built-in analytics libraries for Machine Learning and BI integration provide a complete integrated solution from data ingestion and data processing to data delivery. More at

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: