Big Data ends the era of hunch and extrapolation

I made great money in college recruiting people for focus groups. Those were tough days for students in a state (New York) where the minimum wage was only $3.35 an hour. Instead, I was making $25 an hour working in a phone bank that called and convinced specific demographic groups to spend a few hours at a marketing company as part of a focus group.

I could recruit 30-year-old stay at home moms or 40-year-olds making between 55 and 65K per year…you name it, I found it.

Extrapolation

This job so paid well because it was critically important and cost intensive to test products and services through focus groups. It took just the right mix to figure out how to sell to the general population with the right message for various audiences. Rather than hunch, which isn’t the most reliable plan, these focus groups were key to extrapolation, the best process we had for knowing the world before data became large and ubiquitous.

What’s truly changed is the amount of information that we now have on every imaginable demographic. Today’s systems track and store every purchase by every consumer in a way that turns us from extrapolation to intimate knowledge instead. Look no further than Netflix’s House of Cards, the very popular television program that was constructed using analytics of the massive amount of consumer patterns that Netflix has at their disposal. They didn’t need to focus group the concept to know it would work…the data told them it would work. And it did.

Iteration

Beyond ending extrapolation, Big Data provides an enormous incentive for companies to push products and services out more quickly. Manufacturers can measure and iterate product development based on a stream of feedback that can now be captured and analyzed very quickly. Big Data has, effectively, lowered the upfront cost of product and service development and at the same time, increased the likelihood for longer-term success. This pattern changes the foundation of business models and allows more players to enter markets that have higher risk and reward.

Process change

Anything that changes the business model also changes business processes, meaning that Big Data kicks off a corporate-wide remodeling project, in effect. Companies that don’t remodel are likely to fall behind quickly as their competition iterates its way to better sales and lower costs. This isn’t some farfetched prediction. It is happening slowly across the marketplace in manufacturing, retail, healthcare, travel, and others and is gathering steam as the stories emerge.

The value of Big Data is found in these use cases, not necessarily in the “amazing insights” that are far more fuzzy and a lot less measurable.