In benchmarking, you are the output

I wanted to tackle another benchmarking myth in this post. I’ve touched on it in previous posts in this series, but wanted to address it specifically. It is the myth of getting EXACTLY comparable data within peer groups.

Why is it a myth?

You can’t achieve the elusive “apples-to-apples” comparison when humans are involved, ever. Unless you are comparing very transactional data that is created by machines, you will always have issues with gauge R&R. Even if the data is machine-generated, you’ll probably still have these issues. Gauge R&R describes the impact of humans on data repeatability and reliability. In other words, you can have the exact same person report data using the exact same survey from the exact same process and you will get different results.

You shouldn’t try to overcome it

It is probably theoretically possible to overcome this issue, but you can’t afford it. The amount of cost and time (duration) it would take to overcome this issue would grind your business to a halt.

Your time is too valuable and you are too smart to focus too much on this part of the benchmarking exercise. Don’t get me wrong, I’m not saying ignore issues around validity, comparability, reliability, etc. (see my post on getting valid benchmarking data). I’m just saying spend your time more wisely.

What is the solution?

With benchmarking, learning is the outcome, not the number. The same goes for you. Your time is best spent learning about the data, understanding what the benchmarking, data, reliabilitydifferences are between your data and other peer groups, and making assumptions based on that understanding. Your company doesn’t pay you to find comparable data streams, read reports, and execute click the appropriate button. Or, if they do, a machine will probably be doing that job pretty soon.

You are valuable to your company because you understand your business, the marketplace, and how to achieve the goals (performance) of the organization in better, faster, and cheaper ways. As with everything, the more you engage in the benchmarking process, the more you are going to get out of it. Don’t make is a part of your “night job”, make it a key piece to your “day job”.

You make the benchmark the benchmark

At the end of the day, the output of benchmarking isn’t valuable, what you learn from the process and able to do for your organization with that output is valuable. You are the benchmark, not the report. There are ways to be more efficient and effective in benchmarking (like using third parties that specialize in benchmarking), but don’t try to commoditize the benchmarking process or output too much. You get too far away from the real value of benchmarking.


Categories: Benchmarking, Continuous Improvement

Subscribe to the blog

Subscribe and receive an email when new articles are published


  1. Using Benchmarking Vendors (Part 2 of 2) | Successful Workplace - April 20, 2013

    […] Comparability – If you are conducting a predominantly metric benchmarking project, ask about comparability. If they tell you they have data that is truly “apples to apples”, they are lying. Dig deeper. There is no other organization just like you, so there isn’t an exact data match. All data will have some variation in it. Besides, it isn’t about finding an exact comparison, it’s about understanding the data as I discussed in this post. […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: