We were recently working with a client on their Balanced Scorecard report and noticed something very interesting. While they had 6 years of data recorded, they only wanted to show the last two. Being a little bit of a performance geek, I had to ask why they were hiding those other 4 years, especially as the trend represented by the previous two years did not match the longer trend.

Two example charts are below, the first represents only the previous two years, while the second represents the entire data set.

Simply looking at the first chart, with no other context, shows that significant upward progress has been made.

The second chart tells a very different story though. It shows that there was a steady deterioration from 2005 to 2009, with a nice jump upward in 2010.

Being a performance geek, I would feel cheated if someone did not disclose a longer trend to compare recent progress against.

But this story is not complete. In discussing the scenario with the measure owner, she mentioned that the formula had changed and because of the difference, they did not want to show data that was calculated with the old formula.

Touché- but my inner geek is still not happy.

Changing a formula is no reason to forget past data. I mean think of the precedence. If you don't like past data, just "change the formula", and it will all disappear. In my mind, if the data was related, there should be some note of the switch, with at least a reference to past performance.

This chart is more honest and transparent to the outsider with the story being something like "We used an old formula (abc), had a downward trend for many years. To change behavior we decided that a new measure formula would be more appropriate (xyz). That chart represents both the old data and the new data, with the green representing the old, and the blue representing the new. While they are not measuring the exact same information, we believe that the new formula is a better indicator/motivator of performance."

In the words of Sherlock Holmes,

"It seemed to me that a careful examination of the room and the lawn might possibly reveal some traces of this mysterious individual. You know my methods, Watson. There was not one of them which I did not apply to the inquiry. And it ended by my discovering traces, but very different ones from those which I had expected."

Our job, as performance management experts, is to really understand the story behind the numbers. We need to collect and present all the "clues" to those in leadership roles, so they can make fully informed decisions. If the formula changes, so be it, but it's always best to show the full story.

• Balanced Scorecard Measurement + Control Charting Theory (November 20, 2010)