Big Data without process is creepy

We used to know the purpose of information before we stored it. In fact, the mere act of defining the storage and retrieval patterns was what made it ‘data’. Going one step further, anything we didn’t store was by definition not usable.

Those days are coming to an end. Unstructured information is a fact of life, the Cloud is a place to put nearly everything, the high volume and velocity of data keeps us from planning too far ahead, and the ‘Internet of things’ means we have plenty of information to see that isn’t ‘ours’ to create, modify or delete.

We’ve entered the era where we can easily grab large amounts of unstructured information before we know what to do with it, or as Alistair Croll says in Big data is our generation’s civil rights issue, and we don’t know it, “…collect first and ask questions later.” He makes great points about the thin line between personalization and discrimination.

As Croll points out, simple things like our music tastes can be used to infer our gender, religion and race. It can go even further, as in the now-famous story by Forbes’ Kashmir Hill of Target’s study of what women buy when pregnant, “How Target Figured Out A Teen Girl Was Pregnant Before Her Father Did“. We know humans are fairly predictable, and as Hill points out, enough data makes our predictability scarily accurate.

When we’re highly predictable, we can be segmented by those who are more predictable than others. We can be targeted for our likelihood to behave a certain way, and this comes with some risk depending the who is calling the shots and what is on the agenda.

De-creeping Big Data

So how to limit this problem? How do we de-creep the use of Big Data? The simplest and most effective way isn’t anything new. It is to build in safeguards for who can see and analyze data and to make the process for its use transparent and managed. This means log files for access and analytics (easily done), it means rules that watch those logs and it means business process management for the way work is done.

I’ll hazard a guess that if industry doesn’t come to this conclusion on its own, high-profile misuse of Big Data will bring on regulation and compliance that will take the fun out of something powerful and exciting.


Tags: , , ,

Categories: BPM, Data Analytics / Big Data, Featured, Process Management

Author:Chris Taylor

Reimagining the way work is done through big data, analytics, and event processing. There's no end to what we can change and improve. I wear myself out...

Subscribe to the blog

Subscribe and receive an email when new articles are published

3 Comments on “Big Data without process is creepy”

  1. Nanda S
    August 4, 2012 at 6:36 pm #

    I agree, if this is about catching the ‘Big Data fever’!!
    Usually organizations look for such strategy only to augment their Business strategy – I.e., further refine an existing process by feeding in the result of ‘Big Data’ initiative.

    With that, it’s extremely important for organizations to structure their processes using BPM methodology for multiple benefits such as Operational efficiency, Governance, visibility and agility, etc.

    Bottomline, structured processes following a well defined and comprehensive methodology is a must for organizations to stay ahead of the curve in this competitive and regulatory world!!

  2. August 5, 2012 at 12:03 pm #

    Chris. Interesting observations and conclusions! I agree with your suggestions for safeguards we’ll need to put in place, as a starting point. I assume you’d agree that numerous new challenges will no doubt appear and we’ll need to be prepared to adapt and adopt whatever’s necessary to keep us safe!

    From the perspective of a career IT professional who loves this “stuff,” and a person with a lifelong penchant for privacy. . . it’s creepy!

  3. August 6, 2012 at 5:46 am #

    There will be safeguards put in place, regardless of positive usage. Reason being, you just cannot trust people will never step out-of-line. The government cannot risk that kind of abuse. I bet they will end up being reactive, instead of proactive, but that’s another discussion.
    That said, it is an exciting time for marketing and IT. There is a lot of work which should be done to get proper systems in place to use all the data which is available. I have been paying attention to the companies utilizing predictive algorithm to help with matters of supply chain. I look forward to the evolution of predictive modeling to be used to expand other areas of business communication.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: