Understanding Our Dismal Ability to Forecast

Despite its flaws, Noise, the blockbuster from Daniel Kahneman, Oliver Sibony, and Cass Sunstein, is an important book that both informs and entertains.

The authors are at their best when describing the shocking noisiness of human decision making, not just among different observers in the same circumstances, but even by single observers in the same circumstances. Different judges, for example, hand down radically different sentences, on average, for the same cases; most amazing of all, sentences imposed by the same judge in nearly identical cases will vary from occasion to occasion and even by time of day.

Pro tip: avoid being sentenced more than two hours after the judge’s last meal.

The author engagingly describe the same inter-and intra-observer variance in multiple domains: hiring decisions; fingerprint interpretation; diagnoses by a wide range of medical specialists, most disturbingly in psychiatry. (Which won’t surprise any health care professional; after beginning his career as a ground-breaking child neurologist, Sigmund Freud spent the rest of his life in the land of untestable hypotheses.)

The authors intersperse these examples with a theoretical framework that breaks down noise into three domains. In the setting of judges’ sentences, for example, they are:

  • level noise: the variance of the average sentences handed down by different judges across all cases, i.e., “bleeding heart” versus “hanging” judges;
  • stable pattern noise: sentencing variance among cases according to the type of crime. I.e., some judges are hardest on drug cases, others on white collar crime; and
  • occasion noise: sentencing variance at different times by the same judge the same type of case.