The flaw in the data is you

Deloitte recently published A Delicate Balance, Organizational Barriers to Evidence Based Management.  A surprising conclusion was that the biggest barrier to success with data analytics projects is the behavior of the humans and not the quality, volume or reliability of data. Regardless of how difficult data science can be, success depends on the usual thing…us. We stand in the way of ourselves.

Deloitte’s report points to four common failure areas for Big Data and analytics projects:

Data’s role

There’s an enormous gulf  between completely automated algorithms like the Netflix movie recommendation engine and purely human judgement, like what a physician provides in the hectic emergency room. Within that broad spectrum lie a thousand possible combinations of human judgement and machine responses. Layering in the appropriate amount of analytics alongside business process and decision points is a complex recipe and getting it wrong can be disaster for a business or a patient.

Organizational dynamics

If the role of analytics is poorly understood, there is high likelihood that efforts will be met with irrational enthusiasm for all things analytical or the opposite…skepticism from those either lacking understanding or feeling threatened by a new paradigm. There needs to be a process for organizational change.

Historical bias

The more we value strong judgement and those we believe demonstrate it, the harder it is to accept that there can be a better way to make decisions. We believe what we believe and convincing us otherwise is nearly impossible. After all, didn’t we get here without analytics?

Seeking perfection

Equally challenging is the belief that the best data put through the perfect model will achieve an ideal outcome. This is also the best formula for organizational paralysis as we strive to reach an unattainable level of data cleanliness and model perfection. Perfect is the enemy of good enough when it has a negative impact on revenue or cost.

In the end, the answer comes down to how the organization establishes goals, educates stakeholders, overcomes understandable bias and stays humble about expectations.  It comes down to a balance of process and technology. Funny enough, it always does.

Advertisements

Tags: , , , ,

Categories: Data Analytics / Big Data

Author:Tom Molyneux

A business process strategist with a focus on real-time event management.

Subscribe to the blog

Subscribe and receive an email when new articles are published

4 Comments on “The flaw in the data is you”

  1. June 8, 2012 at 4:25 am #

    Tom, big data has been such a hot topic lately that, as I read my current book (The Black Swan), I am struck by the inherent flaw in using historical data to predict the future. No matter how how good it is, it cannot predict outlying (or Black Swan) events. I even felt compelled to blog about it myself (http://unapage.com/projection-protection).

  2. June 8, 2012 at 12:33 pm #

    Tom, thanks for your comment. The Black Swan is a great book and points out the flaws of of using historic data when their can be very large outliers – especially if those outliers can have catastrophic (a stock market crash) or very positive (investing in Microsoft in 1985) impacts. A lot of this can come from not having enough data and not sampling a big enough universe. The black swan reference (for those who haven’t read the book) was to the possibility of a black swan existing – the concept that no matter how many white swans you saw, you could never say there wasn’t a black swan.

    It turns out there are indeed black swans, native to Australia. Europeans had just never encountered them.

    Strangely enough, when I was reading the book several years ago, moved near Newport Harbor. Our harbor actually had a resident black swan named Rupert. If that wasn’t an improbable enough, before I completed the book, Rupert was run over by the harbor patrol (they were on an emergency call), so I now live in a world with no black swans.

  3. Jeanne Roué-Taylor
    June 8, 2012 at 8:07 pm #

    Reblogged this on Fabless Labs.

  4. June 10, 2012 at 2:08 pm #

    People reading this might like to see some notes on the psychological aspects of leadership here http://www.duxinaroe.com/1/post/2012/06/the-human-cost-of-control-pt3.html – the 3rd in a series of notes expanding on ‘The Darker Side of Lean’ by Darius Mehri.

    & I agree – The Black Swan was a great way to show the folly of controlling that which can be controlled rather than developing the capacity to respond to tail events as defined by the Gaussian curve.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: