While You Slept Last Night…Big Data, Privacy and the Public Square

While you slept last night, the world around you continued to amass information about everything, including you. Banks processed your purchases. Your credit information was updated and redistributed. The electric company recorded your power usage. As Malte Spitz showed us in his TED piece, your cell phone on your nightstand dutifully reported your location.

We’ve more or less accepted this as part of our technological life. The alternative is to live ‘off the grid’ and outside the conversation. Few are willing to do that.

But there’s more than that going on. When you went for coffee this morning the camera on the police cruiser’s dashboard watched you cross the street. The ATM caught a glimpse as you passed by. When you staggered into Starbucks, that camera on the corner bank relentlessly recorded your moves.

Really Big Data

Do you care? You should because video is on the verge of being data in the same way that conversations and transactions already are. As video-as-data becomes economically and technologically feasible, no human eye needs to watch a screen to know what happened and the level of detail (data) that can be taken away from video goes up exponentially. It becomes frames per second times screen resolution. It makes today’s big data tomorrow’s tiny data.

In the past, analysis of video took enormous amounts of manpower to organize, execute and analyze. We’ve broken through that barrier and cheap, high-speed storage combined with analytical über-processing make video the new voice.

Imagine the crowd walking through Times Square. Video-as-data means easily matching the faces in the crowd in one way or another, with Facebook as the easy but not only choice. We can see what they’re doing and whom they’re doing it with. We can read gestures, emotions and body language. Given enough angles, we know what draws their attention and for how long.

Permanent record

And it isn’t just today’s video. We can go back and process every video of Times Square ever taken and know how many times someone passed through, and with whom.

Going further, we can choose an individual and see a digital footprint of every video they’ve ever appeared in. We can combine that with cellular data to fill in the blanks, and transactional data to understand their financial interactions. Our ability to correlate all of that information redefines the public square and privacy in a way that most people probably don’t realize.

This makes laughable the idea of an implanted chip as the way to monitor a population. We’ve implanted that chip in our phones, and in video, and in nearly every way we interact with the world. Even paranoids are right sometimes.

Database of Ruin

There are plenty of benefits and problems with this. We can intervene as a medical crisis unfolds and we can keep the population safer from bad people doing bad things. But we can know too much, too. Paul Ohm describes this as a Database of Ruin, “…these databases will grow to connect every individual to at least one closely guarded secret.”

Does privacy need to be a casualty of Big Data? I recently spoke with author, analyst and serial entrepreneur Alistair Croll, who believes that big data is our generation’s civil rights issue. “Ten years ago we said, ‘don’t put your name on the internet.’ Society is a moving target and we’re adapting to it, adjusting what’s appropriate for convenience and comfort.” How will we fully adjust to the new definition of the public square where every moment can be captured, analyzed and acted upon?

Croll believes that people who know about us are going to have to hide what they know to stay out of trouble. In the past, we had crude targeting that when someone marketed to us, we didn’t find it creepy. They were using such blunt instruments we didn’t recognize that we’ve been identified or that someone has peered into us.

Croll continues, “What’s happening now is more surgical. We know they know and wonder what else they know and aren’t saying. Our laws going back to the Magna Carta and before are aimed at a physical world and not a digital one. We don’t have laws for this new, second life and we’re going to scramble to write them.”

Cover your tracks?

This new, second life means we’ll need to think about what we do on a personal level and understand the implications. Rethink your expectation of anonymity. Consider paying in cash for embarrassing purchases and other similar steps. Acknowledge that we own part of the problem.

For companies needing to compete with increasing information and more powerful analytics, there are things to be doing right now. Being creepy is not a good strategy, nor is ignoring the opportunity to use some of what’s out there in the right ways. Finding the balance will mean doing several things very well:

  • Strategy – There needs to be a clear strategy for staying clear of controversy
  • Process – Delineating what to capture, how to analyze and where to use information. Methods must be spelled out and boundaries established.
  • Vigilance – Systems provide us with information on who and how data was used. It would be a good idea to make sure the right people are seeing the information for the appropriate reasons.
  • Infrastructure – You almost can’t have enough for what’s coming.
  • Analysis – Be really, really good at this part before you make an ugly mistake
  • Execution – Know what to do automatically and what needs to be thought through some more

For governments, there’s a clear need to protect freedoms and allow commerce at the same time. There needs to be a balanced approach that errs on the side of society.

We’ll get through this…we always do. There will be abuses and incidents that will generate new laws and human behaviors and then we’ll settle in with a new definition of the public square. That is, until something else comes along.

This article was first posted on Forbes Magazine.


Tags: , ,

Categories: Data Analytics / Big Data, Privacy

Author:Chris Taylor

Reimagining the way work is done through big data, analytics, and event processing. There's no end to what we can change and improve. I wear myself out...

Subscribe to the blog

Subscribe and receive an email when new articles are published


  1. Facial recognition creeps quietly into our personal space | Successful Workplace - April 14, 2013

    […] you add that information to the rapidly increasing ability to do the same with video, the center of Croll’s venn diagram begins to grow larger and larger. Perhaps most scary, […]

  2. Boston Marathan bombing shows where lack of privacy is a good thing | Successful Workplace - April 19, 2013

    […] with a focus on what happens when video can be ‘processed’ as data. In the article While You Slept Last Night…Big Data, Privacy and the Public Square, we warned that even old video has the same potential to disrupt privacy as something shot […]

  3. 60 Minutes says, “Say goodbye to anonymity” | Successful Workplace - May 20, 2013

    […] case you missed 60 Minutes on CBS last night, there’s a new challenge to privacy that is coming faster than people realize and was made more urgent by the terror attack in Boston a […]

  4. A guy just can’t commit a crime nowadays | Successful Workplace - July 30, 2013

    […] where does this take us? How much privacy is lost in the use of technology to solve crimes? Do we need to intentionally silo data to prevent […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: