Our Misguided Trust in Quantitative Measurement

The belief that detailed quantitative measurement will make performance easier to evaluate, manage efficiently, and improve has survived repeated failures of the doctrine. In fact, the failures serve only to bring forth calls for more of the treatment. Only very rarely is it admitted that quantification doesn’t work and should be scrapped.

Quantitative measurement has gripped the management of everything in the last 60 years. It is now so firmly entrenched that nothing could shake it loose. Going back to the old methods of subjective, non-quantitative evaluation – even in small ways – is virtually unthinkable. For that reason, history professor Jerry Z. Muller, in his very important book The Tyranny of Metrics, says “Because belief in its efficacy seems to outlast evidence that it frequently doesn’t work, metric fixation has elements of a cult.”

Throughout society, the reliance on fixed beliefs has distorted outcomes and misallocated resources. But the best example of how this type of overreliance led to a horrendous result occurred over 60 years ago, among the followers of an obscure cult.

In a 2010 Wired article, writer Jonah Lehrer retold psychologist Leon Festinger’s account of what happened after a cult whose leader predicted the end of the world on December 20, 1954, realized that it hadn’t occurred. Festinger was the originator of the concept of cognitive dissonance. He was curious what would transpire when the cult discovered that the belief they had committed their lives to turned out to be false. He infiltrated the cult in order to observe.

The cult leader, Dorothy Martin, to whom Festinger gave the pseudonym Marion Keech, had been in constant contact with aliens of the planet Clarion. She told her cult members that these aliens would pick up her cult in a flying saucer at 12:01 AM on that date just before the world was destroyed by massive flooding. In Lehrer’s retelling:

On the night of December 20, Keech's followers gathered in her home and waited for instructions from the aliens. Midnight approached. When the clock read 12:01 and there were still no aliens, the cultists began to worry. A few began to cry. The aliens had let them down. But then Keech received a new telegram from outer space, which she quickly transcribed on her notepad. "This little group sitting all night long had spread so much light," the aliens told her, "that god saved the world from destruction. Not since the beginning of time upon this Earth has there been such a force of Good and light as now floods this room." In other words, it was their stubborn faith that had prevented the apocalypse. Although Keech's predictions had been falsified, the group was now more convinced than ever that the aliens were real. They began proselytizing to others, sending out press releases and recruiting new believers. This is how they reacted to the dissonance of being wrong: by becoming even more certain that they were right.

This analogy may seem extreme, but it is not.

Imposing a strict system of quantified metrics to evaluate and reward performance has serious deleterious unintended consequences. It induces gaming of the system, a kind of rent-seeking behavior that adds nothing to productivity and often detracts. It siphons attention toward goals whose achievement can be measured and away from goals whose achievement is difficult or impossible to measure but may be of greater importance. It brings forth a large bureaucracy of data-recording and analysis, performance measurement and evaluation, thus undermining one of the main goals of quantitative measurement, fiscal efficiency. It poisons employees’ desire to do their job by substituting external and often arbitrary-seeming requirements for internal motivation.