April 13, 2010
Failure to acknowledge the decision-making context
Mauboussin, who also teaches at the Columbia Business School, asked a group of students to write down the last four digits of their phone number, and then he asked them to estimate the number of doctors in Manhattan. The answers revealed a strong correlation – those with lower phone numbers estimated fewer doctors, and vice versa.
Along the same lines, home appraisers in Tucson were asked to appraise a house. They were also shown the listing price, and the researchers found that a higher listing price led to a higher appraisal – despite the fact that 80% of the appraisers contended that the listing price had not swayed their judgment at all.
Those examples illustrate the power of the context in which a decision is made, and how subtle changes in the decision-making environment can lead to costly errors.
“Be extraordinarily mindful of what is going on around you when you make decisions,” Mauboussin warned. “We are not as objective and rational as we believe.”
Recognize that you are creating a decision-making context for those around you. “The tone you set will by actions and words will shape those decisions,” Mauboussin said. “Some environments are conducive to great decision making and others much less so.”
Undue belief in experts
“There is a role for experts in decision-making and, in some cases, they are absolutely vital,” Mauboussin said. “That said, we pay way too much attention to people in pin-striped suits and lab coats, and to the talking heads on television.”
Experts tend do well with rules-based problems that have limited degrees of freedom, according to Mauboussin. The video-rental system Cinematch, created by Netflix, is a prime example of the power of expert analysis. By matching user preferences to movie titles, Netflix was able to get people to rent from their full inventory of movies, instead of only new releases.
Experts typically fail at problems with high degrees of freedom that are probabilistic. Phil Tetlock’s 2007 book, Expert Political Judgment, surveyed 300 political experts and measured the accuracy of their nearly 30,000 predictions over a 15-year period.
The experts were dismal forecasters. “Experts are notoriously bad at predicting political and economic outcomes,” Mauboussin said.
Especially disconcerting was the revelation that the more frequently a pundit was mentioned, the worse their predictions. Those on whom we are most likely to rely are the least likely to offer accurate forecasts.
The alternative is to rely on ordinary people. Jim Surowiecki, who wrote The Wisdom of the Crowds in 2004, has done some of the most prominent research illustrating how average people can be better at forecasting than the so-called experts. Surowiecki worked with the electronics retailer Best Buy and found that a group of their relatively uninformed employees could more accurately predict key variables (such as holiday-season sales) than the experts Best Buy was employing.
Even though crowds have proven remarkably good at providing vital information, Surowiecki said recently that firms such as Best Buy still have fundamental misgivings when it comes to throwing problems to the crowds. Management still doesn’t trust them.
For crowd-based forecasting to work, Mauboussin said three conditions must be met: the participants must be diverse, there must be a properly functioning mechanism for aggregating their individual inputs, and there must be a system for rewarding accuracy. The most likely of those conditions to be violated, Mauboussin said, is diversity because, as humans, we are naturally social and imitative.
The fallibility of experts has occasionally been brought to light by the power of crowds. In 1870, a German scientist was studying the nutritional value of green vegetables and misplaced a decimal point and overstated the iron content of spinach. In 1920s, when the movie house Paramount decided to help Americans improve their health and to get people to eat from cans, it created the character Popeye, whose power was enhanced by eating spinach. That myth that spinach has more iron, having been propagated by crowds for nearly 150 years, still prevails today.
Display article as PDF for printing.
Would you like to send this article to a friend?
Remember, if you have a question or comment, send it to .
