Harvard Business Review published a really interesting interview with Nate Silver this week. Silver was asked about how to approach analytics based on several different points of entry to the field. One of the best questions involved how an organization should deploy ‘stat heads’, Silver’s term for data analytics experts. The interviewer wanted to know if Silver felt they should be a department that serves the company or if stat heads should be embedded in the business.
I think you want to integrate it as much as possible. That means that they’re going to have some business skills, too, right? And learn that presenting their work is important. But you need it to be integrated into the fabric of the organization.
You’ve seen this shift in baseball teams, for example, where it used to be that you’d hire an analyst to check that box and have them compartmentalize. That doesn’t accomplish much at all.
Integrate, don’t segregate
Silver makes a great point about not partitioning off the data science people. There are several reasons to do this:
- Without deep knowledge of the business, it’s very hard for a data type to know what inputs matter and what outputs make sense. It isn’t just a matter of, “Well, what does the data tell us.” If it was that simple, we’d simply turn the machines loose.
- The business people need to see how the data is accumulated. There is a massively important part of Big Data that people miss…where did the data come from? When did it come? How was it refreshed? How clean was it? These questions are enormous in figuring out what to trust, what to toss, and what to change. The data’s “back story” is full of nuance for the organization.
- All models are flawed and all data is imperfect. If both parties know that going in and work together, the models improve and the data gets better. That’s a very tough thing to accomplish in silo’d workplaces.
- More important than more data is having “all data“. That involves not just the analysis of massive data sets, but blending data from as many sources as possible. This approach is very difficult without keen business sense that is probably only available outside of the data scientist’s area of expertise.
Teams need to be blended. Data science is not something done in a dark room by data nerds. Besides, there are so few solid data scientists that you can’t expect them to do the work for the many. It doesn’t scale.
We’ll be in New York next week for Interop and hope to see you at the Big Data Workshop. We’ll be talking about the ecosystem of Big Data and how value is derived beyond Big Data’s enormous hype.