Big Data ends the era of hunch and extrapolation

Focus Group RoomI made great money in college recruiting people for focus groups. Those were tough days for students in a state (New York) where the minimum wage was only $3.35 an hour. Instead, I was making $25 an hour working in a phone bank that called and convinced specific demographic groups to spend a few hours at a marketing company as part of a focus group.

I could recruit 30-year-old stay at home moms or 40-year-olds making between 55 and 65K per year…you name it, I found it.


This job so paid well because it was critically important and cost intensive to test products and services through focus groups. It took just the right mix to figure out how to sell to the general population with the right message for various audiences. Rather than hunch, which isn’t the most reliable plan, these focus groups were key to extrapolation, the best process we had for knowing the world before data became large and ubiquitous.

What’s truly changed is the amount of information that we now have on every imaginable demographic. Today’s systems track and store every purchase by every consumer in a way that turns us from extrapolation to intimate knowledge instead. Look no further than Netflix’s House of Cards, the very popular television program that was constructed using analytics of the massive amount of consumer patterns that Netflix has at their disposal. They didn’t need to focus group the concept to know it would work…the data told them it would work. And it did.


Beyond ending extrapolation, Big Data provides an enormous incentive for companies to push products and services out more quickly. Manufacturers can measure and iterate product development based on a stream of feedback that can now be captured and analyzed very quickly. Big Data has, effectively, lowered the upfront cost of product and service development and at the same time, increased the likelihood for longer-term success. This pattern changes the foundation of business models and allows more players to enter markets that have higher risk and reward.

Process change

Anything that changes the business model also changes business processes, meaning that Big Data kicks off a corporate-wide remodeling project, in effect. Companies that don’t remodel are likely to fall behind quickly as their competition iterates its way to better sales and lower costs. This isn’t some farfetched prediction. It is happening slowly across the marketplace in manufacturing, retail, healthcare, travel, and others and is gathering steam as the stories emerge.

The value of Big Data is found in these use cases, not necessarily in the “amazing insights” that are far more fuzzy and a lot less measurable.


Tags: ,

Categories: Data Analytics / Big Data

Author:Chris Taylor

Reimagining the way work is done through big data, analytics, and event processing. There's no end to what we can change and improve. I wear myself out...

Subscribe to the blog

Subscribe and receive an email when new articles are published

3 Comments on “Big Data ends the era of hunch and extrapolation”

  1. John G Tesmer
    May 15, 2013 at 5:38 am #

    I disagree that there are no more hunches. The hunches just move up the value stream. Someone had to pick the variables that are included in the model. If Netflix’s model didn’t include the right variables then it would still be wrong, no matter how fancy or how much data they included.

    • September 11, 2013 at 9:42 pm #

      Ah, touché. Indeed, there are hunches, but we prove them out with data, no?


  1. Big Data debunks Bigfoot | Successful Workplace - September 11, 2013

    […] shift from extrapolation to understanding. We shift from folklore to digital evidence. The era of hunch comes to a […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: