The deceptive seduction of Big Data

Everyone wants to dance with Big Data. We are in love with its velocity, volume, and variability. It is an exciting partner that hypnotizes us with the allure of all of its possibilities. The market lavishes money on it and everyone wants to say they know it.

The reality is a little more mundane. Often, it has 5 o’clock shadow and wears grandpa shoes.

Big Data is far more often about decision made in the many ‘normal’ moments and much less about eureka moments delivered with trumpet flourishes. The dimensions of decision making are nuanced and complex. To make very good real-time decisions, arguably the most valuable part of Big Data, requires a very good understanding of…ready for this?…business requirements and process.

We live in an age where the mantra is better and faster, but without understanding the implications, simply pursuing velocity or volume isn’t the answer. Like driving a car, the faster you go, the faster you must make decisions. But if speed kills, faster isn’t better. The best pattern for any decision is found at the intersection the right speed, the right amount of information, and the right context.

Not just fast

The best enterprises make good decisions, not just fast decisions. Getting there isn’t simple…there’s a point where the decision needs to be made on the right amount of information at just the right time. Made early, the decision lacks key data. Made late, the moment to decide may be over. Just like the speed decision, the ‘just enough information’ decision is a tuning exercise and not a one-size-fits-all.

Not just fully informed 

Many organizations are still depending on traditional business intelligence (BI), which isn’t about having the right decision in the right moment. Instead, it is often about understanding something that has already happened and is over-the-shoulder. BI is only marginally helpful and is losing relevance in real-time decision making.

Context matters

Decision making is also about the impact from getting it wrong or not making the decision at all. Clearly, making a large trade that has high risk and reward can be put off or cancelled, but decisions in an emergency room can’t. Making the wrong choice when free climbing a mountain has very negative implications.

Kinds of decisions

Gartner’s W. Roy Schulte describes three kinds of decisions in Approach Operational Decisions Differently From Tactical or Strategic Decisions:

  • Strategic decisions are made in months to years. Should we acquire a company? Should we expand into South America?
  • Tactical decisions are made in days to weeks. What customers should we pursue? What features will go into our next product?
  • Operational decisions are made when the speed and amount of data align. How should we respond to a customer service request? Is a credit card being used fraudulently?

Operational decisions are the ones that are being driven by technology that goes into decision support systems that rely on process. A good decision support system connects to the many sources of information that go into a broad range of decision and also creates a context (time, place, level, business unit) that is relevant for making consistently wise choices. The best systems are about environmental and situational awareness and that’s being done more and more often through event processing.

As the human ability to decide is mimicked more and more by technology and less by humans. This is driven by cost, accuracy and the speed or complexity of information that goes into a decision. The human brain still soundly trounces the best automation, but that gap is closing.

Flexible and adaptable

Many decisions spawn processes or other decisions, and any system needs to be able to flow back into itself or kick off an external set of activities. Rigid systems don’t do this well and are impossible to maintain. The best systems being developed now have user interfaces that blend process, analytics and information so that you can change things on the fly. Anyone playing in what most call the real-time space need to have event processing as a core capability.

Done well, you have a system that helps you know how much data is just enough to make the fastest, best, most contextual decision possible.

This article was inspired by a presentation by Gartner Distinguished Analyst W. Roy Schulte at TIBCO’s TUCON 2012.

Advertisements

Tags: ,

Categories: Data Analytics / Big Data, Patterns / Rules / Events

Author:Chris Taylor

Reimagining the way work is done through big data, analytics, and event processing. There's no end to what we can change and improve. I wear myself out...

Subscribe to the blog

Subscribe and receive an email when new articles are published

Trackbacks/Pingbacks

  1. What Will Happen With Business Intelligence Tomorrow? | Wisdom Health Prosperity - October 2, 2012

    […] The ERP Acting As A Decision Support System!Making Business Decisions with SharePoint and CloudShareThe deceptive seduction of Big Data noCon(document).ready(function(){ noCon("#dropmenu ul").css({display: "none"}); // For 1 Level […]

  2. Get your organization to think and act like a fighter pilot | VentureBeat - February 6, 2013

    […] shortens your own OODA loop so that you can ‘turn inside’ the adversary’s process. Schulte talked about OODA Loops at TUCON 2012, a gathering of TIBCO customers and partners last September in Las […]

  3. Get your organization to think and act like a fighter pilot | CrowdBacon – The Startup Blog - February 6, 2013

    […] shortens your own OODA loop so that you can ‘turn inside’ the adversary’s process. Schulte talked about OODA Loops at TUCON 2012, a gathering of TIBCO customers and partners last September in Las […]

  4. Get your organization to think and act like a fighter pilot | 5 For Business - February 6, 2013

    […] shortens your own OODA loop so that you can ‘turn inside’ the adversary’s process. Schulte talked about OODA Loops at TUCON 2012, a gathering of TIBCO customers and partners last September in Las […]

  5. The Capitals™ – Capitalists' Magazine | 資本家札記 | Get your organization to think and act like a fighter pilot - February 6, 2013

    […] shortens your own OODA loop so that you can ‘turn inside’ the adversary’s process. Schulte talked about OODA Loops at TUCON 2012, a gathering of TIBCO customers and partners last September in Las […]

  6. Get your organization to think and act like a fighter pilot | Successful Workplace - February 9, 2013

    […] shortens your own OODA loop so that you can ‘turn inside’ the adversary’s process. Schulte talked about OODA Loops at TUCON 2012, a gathering of TIBCO customers and partners last September in Las […]

  7. San Valentín te llevará hasta tu “Grey”….solo tienes que ¡pintarte las uñas! | essie blog - February 14, 2013

    […] : Proidee, Suscessfulworkplace, […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: