If you haven’t followed the Tesla vs. NY Times story, you’ve missed the first mainstream accounting of machine data. The venerable NY Times was somehow caught off guard by the fact that Tesla was collecting data from their vehicle that could be analyzed and produced on demand. What this story didn’t tell us is that Tesla has access to data as it is generated, not just when they need to defend the performance of their car.
We’re seeing a glimpse into what’s already here but few realize: real-time data, especially for security, is the differentiator for most major companies today.
Internet of Things
The Internet of Things refers to the connection of sensors to the Internet that can bring far more data than human were ever able to generate. To get an idea of scale, Gartner predicts that the Internet of Things trend will bring 50 billion new sensors online over the next five years while the Internet of Humans moves from 2 billion to just 3 billion. Humans are being outpaced significantly by the growth of machine data. Machine data creates both new challenges and new opportunities in security. Most importantly, it creates extensive logs.
Those logs have incredibly high value in the right hands. Just this week, San Francisco hosted the 2013 RSA Conference where thousands of cryptology and security professionals came together to see the latest in technology that protects the information systems of the world. Monitoring and responding to log data is a significant part of making machine data useful and was a major focus of the conference.
Trends from RSA
In a great write up by Chima Njaka, three key trends were highlighted from the event. They were:
1. Advanced persistent threats – There is a powerful need to have a machine layer of defense against known threats
2. Bring Your Own Device – This trend struggles with the need to support disparate devices with a single security policy
3. Well-known vulnerabilities – Gartner predicts that 80% of successful attacks are against well-known vulnerabilities
Machine data and its log data outputs are a key component of dealing with the rise of some of the most interesting and urgent kinds of Big Data. The challenge of have so much data at our fingertips is in keeping it out of the hands of the wrong people and yet still available to our own organizations. The key questions are:
- Do you have one central solution for tracking and analyzing all log data in real-time?
- Are you able to monitor your business processes and not just your network?
- Do you monitor identities and how they match to applications instead of just access points?
The biggest challenge of machine data security is seeing it happen in real-time, before a credit card fraud happens. This often means being able to correlate multiple events seen only in log data at the same time. This is the new frontier of security and a wide open place for the bad guys and the good guys.