Privacy rules are the first casualty of Big Data

Data privacyIn our haste to study larger and larger amounts of data and find information, there’s a point getting lost in the excitement...those are people who often haven’t given their permission for their data to be used for just any purpose.

This isn’t a small problem or isolated problem. The use of consumer data to understand the marketplace is one thing, but we’ve gone beyond the question of what’s selling well.

The volume of data we’re able to see today means we can see where, when and how each and every consumer made a purchase. The data about the purchase has become as important as the purchase itself and less likely to be scrutinized for privacy than ever before.

The gotchas

And knowing that we can be knowing so much, there are new levels of responsibility for those who collect, hold and use all of that data. Those folks are operating in a world where the rules were set up for completely different times (the beginning of computerization forty years ago). Back in those days, data was extremely transactional, relatively small and always collected for very, very specific reasons. It was collected, stored and used in separate data repositories that were only occasionally cross-referenced in a way that drew a bigger, more personal picture. And very significantly, data was easily protected on the server before there was such a thing as a firewall or the Internet.

The old rules still exist but those simpler times are over.

Data governance

The biggest challenge of Big Data isn’t the storage or processing. The single biggest challenge, a challenge many don’t see or aren’t talking about, is the need to govern data in ways that protect the individual from manipulation, fraud or worse. That takes a combination of things that readily available but underused like access security, log data management, managed file transfer, and people and processes that are up to the task.

Where do you think this is headed? Will we need new laws or can we trust that Big Data holders will act responsibly?



Categories: Data Analytics / Big Data, Privacy

Author:Chris Taylor

Reimagining the way work is done through big data, analytics, and event processing. There's no end to what we can change and improve. I wear myself out...

Subscribe to the blog

Subscribe and receive an email when new articles are published

2 Comments on “Privacy rules are the first casualty of Big Data”

  1. Rob Mian
    March 5, 2013 at 9:05 am #

    Excellent point, Chris. The privacy implications are staggering and governance will be very complex. The Big Data holders will continue to push the boundaries until they cross a line (eg. Google Maps data collection). Once the public understands the implications they will have to make interesting choices.

    • March 5, 2013 at 9:46 am #

      Thanks, Rob. I agree. There are multiple things going on at the same time…the vendors are pushing the boundaries, the public is growing more accepting of less privacy and governments are on the sidelines watching. There will be a point where this equilibrium stops working.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: