Daniel Kahneman and Richard Thaler won Nobel prizes for their work in behavioral science, propelling that discipline to the forefront of the advisory profession. But new research shows that behavioral science produces results that are no better than a simple model, and mutual funds based on it are no better than an index fund.
Behavioral science is the study of human behavior through systematic experimentation and observation. Behavioral scientists do the following: study why people sometimes behave in ways that may not maximize their own well-being, such as making choices in the present that do not maximize their happiness in the future; examine how seemingly arbitrary contextual factors influence people’s decisions, beliefs and attitudes; test how different incentives affect people’s motivation and behavior; analyze how people judge others’ traits and characteristics based on features of their face or voice; investigate how consumers can be encouraged to make, avoid or change spending decisions; and design policy interventions that can help people make choices they would personally recognize as optimal in the long run. Among the disciplines that fall under the broad label of behavioral science are anthropology, cognitive psychology, consumer behavior, social psychology, sociology and behavioral economics/finance.
Since we expect experts in any field to make accurate predictions within their domain of expertise and thus rely on them when making decisions, an important question is: How accurate are the forecasts of behavioralists? Dillon Bowen sought the answer to that question in his August 2022 study, “Simple Models Predict Behavior at Least as Well as Behavioral Scientists.” He analyzed data from five studies in which 640 professional behavioral scientists predicted the results of one or more behavioral science experiments. He compared the behavioral scientists’ predictions to random chance, linear models and simple heuristics like “behavioral interventions have no effect” and “all published psychology research is false.” The five studies included:
-
An exercise study that examined 53 behavioral nudges to encourage 24-Hour Fitness customers to exercise more. Ninety practitioners from behavioral science companies were then asked to predict how effective each nudge would be.
-
A flu study that used 22 text-message treatments to encourage Walmart customers to get a flu vaccine. Twenty-four professors and graduate students, most affiliated with top 10 business schools, were then asked to predict how effective each treatment would be.
-
A study that assembled data from 126 randomized control trials (RCTs) from two of the largest nudge units in the United States. The researchers measured the RCTs’ effectiveness as the percentage point increase in adopting a target behavior compared to a control condition. Two hundred thirty-seven behavioral scientists from academia, nonprofits, government agencies and nudge units were then asked to estimate the effectiveness of 14 randomly selected RCTs.
-
An experimental study that measured how much effort participants exerted in a key-pressing task in 18 experimental conditions. Participants scored points for alternating between pressing “a” and “b” for 10 minutes (participants earned one point each time they pressed “a” then “b”). The experimental conditions were monetary and nonmonetary incentives to score points, such as piece-rate payments, time-delayed payments and peer comparisons. The researchers measured the effort participants exerted in each condition as the number of points they scored. Two hundred thirteen academic economists were then asked to predict the results.
-
A study that attempted to reproduce the results of psychology studies. The researchers defined a replication as successful if it obtained a p-value of less than .05 and the estimated effect’s direction matched the original experiment. They then asked 76 psychology professors and graduate students to predict how likely 44 of the studies were to replicate successfully.
Following is a summary of his findings:
- Behavioral scientists were consistently no better than, and often worse than, simple heuristics and models. For example, the behavioral scientists’ exercise study predictions were significantly worse than the null model.
- Behavioral scientists’ predictions were not only noisy but also biased, overestimating the power of behavioral predictions. For example, their exercise study predictions estimated that the average behavioral nudge would increase exercise by 2.5 gym visits every week, but the results suggested that the average nudge increased exercise by only one gym visit every six weeks. This bias was statistically significant. The behavioral scientists also estimated that the average text-message treatment would increase vaccination rates by 4.5 people per hundred, but the results suggested that the average treatment increased vaccination rates by only 2.1 people per hundred. This bias was also statistically significant. Overestimates of results were also found in the RCT and the effort studies.
- Behavioral scientists’ predictions systematically overestimated how well their discipline “works” – overestimating the effectiveness of behavioral interventions, the impact of psychological phenomena like time discounting, and the replicability of published psychology research. For example, psychologists estimated that the average published psychology study had a 54% chance of replicating, but the study results suggested that the average study had only a 32% chance of replicating. This bias was statistically significant.
- Even the most experienced behavioral scientists overestimated nudges’ effectiveness.
- Behavioral scientists do learn from experience. However, the results suggest that even the most experienced behavioral scientists overestimate the effectiveness of nudges.
Why is it so hard for behavioral scientists to outperform simple models? One possible answer is that behavioralists are subject to the same biases their subjects are, including the all-too-human trait of overconfidence. For example, they overestimate the effectiveness of nudges, believing that behavioral science works better than it actually does. In the exercise study, behavioral scientists significantly overestimated the effectiveness of all 53 treatments even after correcting for multiple testing; they overestimated the impact of psychological phenomena in the effort study; and they overestimated the impact of the replicability of published behavioral science research.
Bowen stated that another explanation is that “behavioral scientists are selectively exposed to research that finds large and statistically significant effects. Behavioral science journals and conferences are more likely to accept papers with significant results. Therefore, most of the literature behavioral scientists read promotes the idea that behavioral interventions are effective and psychological phenomena substantially influence behavior. However, published behavioral science research often fails to replicate.”
A third explanation is that “behavioral scientists might be susceptible to motivated reasoning. As behavioral scientists, we want to believe that our work is meaningful, effective, and true. Motivated reasoning may also drive selective exposure. We want to believe our work is effective, so we disproportionately read about behavioral science experiments that worked.”
The specific field of behavioral finance is the study of human behavior and how that behavior leads to investment errors, including the mispricing of assets.
Behavioral finance
If behavioral finance is to have merit as an alternative investment strategy, one should be able to observe managers who have successfully utilized its theories and produced abnormal returns. The authors of the 2008 study, “Behavioral Finance: Are the Disciples Profiting from the Doctrine?” identified 16 self-proclaimed or media-identified behavioral mutual funds that practice some form of behavioral finance in their investment strategies. The authors, Colby Wright, Prithviraj Banerjee and Vaneesha R Boney, analyzed behavioral funds to determine whether they successfully attract investment dollars and if their strategies earn abnormal returns. Following is a summary of their findings:
- Behavioral funds successfully attracted investment dollars at a significantly greater rate than the index and matched actively managed nonbehavioral funds. Investors apparently believed that the pricing errors were persistently exploitable.
- While the funds did outperform S&P 500 Index funds, the explanation for the outperformance was that they had significant exposure to value stocks. After adjusting for risk, they did not earn abnormal returns.
- Behavioral mutual funds were tantamount to value investing and not much more.
Investor takeaways
While behavioral scientists’ predictions regularly inform academia, public policy, business decisions and investment strategies, unfortunately the evidence demonstrates that behavioral scientists exaggerate the effectiveness of their work. That creates problems because, by overestimating the effectiveness of nudges, they may drain support and resources from potentially more impactful solutions. For example, accurately disclosing the impact of nudges could shift support toward carbon taxes and away from a nudge solution. Actively managed funds using behavioral finance strategies have been unable to exploit their findings by generating alpha.
Larry Swedroe is the chief research officer of Buckingham Wealth Partners and Buckingham Strategic Wealth.
For informational and educational purposes only and should not be construed as specific investment, accounting, legal, or tax advice. Certain information is based upon third party data and my become outdated or otherwise superseded without notice. Third party information is deemed to be reliable, but its accuracy and completeness cannot be guaranteed. By clicking on any of the links above, you acknowledge that they are solely for your convenience, and do not necessarily imply any affiliations, sponsorships, endorsements or representations whatsoever by us regarding third-party websites. We are not responsible for the content, availability or privacy policies of these sites, and shall not be responsible or liable for any information, opinions, advice, products or services available on or through them. The opinions expressed by featured authors are their own and may not accurately reflect those of Buckingham Strategic Wealth® or Buckingham Strategic Partners®, collectively Buckingham Wealth Partners. LSR-22-385
Read more articles by Larry Swedroe
Daniel Kahneman and Richard Thaler won Nobel prizes for their work in behavioral science, propelling that discipline to the forefront of the advisory profession. But new research shows that behavioral science produces results that are no better than a simple model, and mutual funds based on it are no better than an index fund.
