Mixpanel, a startup founded by former Y Combinator alums Tim Trefren and Suhail Doshi put together an April Fools joke video yesterday around the concept of ‘bigger data’ that made TechCrunch’s shenanigan summary ‘April Fools 2013: The Ultimate Round-Up‘. Whether it was meant to be funny or not, it brings up a point that gets lost in the hype around big data: Big data is going to grow exponentially over the next two to three years for four very straight-forward reasons:
1. Data has more value – As organizations learn the lesson of holding onto data that may not seem so valuable right now, the amount of data being stored naturally increases. Nature abhors this surplus of data and new purposes are inevitably found, some that even have value :-).
2. Internet of Things – OK, this one isn’t a shocker but it has to factor into any top 5 list. If Gartner is to be believed, 50 billion sensors will join the 2 billion connected users to create far and away more data than we have now. A 25x increase in connected ‘things’ is quite a change, but when those 25x things are automated, 24×7 sensors, we’re talking about data on a whole new scale.
3. New data – Connection, storage and analytics against types of data that weren’t held onto in the past simply because of sheer volume or velocity are now in play, with the two greatest examples being location and video data. Constantly-changing data presents challenges but enormous opportunities and pushes the limits of analytics around streaming data. As stated before, nature abhors a surplus of data, and this will be no exception.
4. Social data – When we see Amazon buying Goodreads, we get a glimpse into the future of fully social commerce. As much as it may feel social has run its hype cycle, it has yet to be deployed to most enterprises and yet to become a measurable part of most commercial opportunities. When the world catches on and catches up, social data will grow exponentially from just Facebook, Twitter and a handful of other big players.
Keep in mind that each of these reasons creates a cycle of ever-increasing data. Nature also abhors a vacuum and an advance in big data volumes or technologies in one area will have an impact on the others. This cycle will continue to push the boundaries of what we can handle.
For companies that realize they need to gain big data capabilities but haven’t started, this is your wake up call. But more importantly, if you don’t have the ability to connect and begin to manage data at today’s scale, how do you expect to keep up with what’s certainly coming?
Integration of systems is the new black. If you’re not architected to make the most of data today, it’s only going to get worse.
5. It was just awful as an April Fools joke – See for yourself: