Is the market efficient? Of course not – not exactly, or not even close, depending on your point of view. However, the efficient market hypothesis (EMH), while self-evidently not quite correct, has remained surprisingly resistant to overturning. The reason is that, as MIT professor Andrew W. Lo says repeatedly in his new book, Adaptive Markets, “it takes a theory to beat a theory.” And, up to this point, there has been no alternative theory that can substitute for the EMH if the latter is found wanting.

In the modern study of capital markets there have been two kinds of innovation: (1) insights that contribute to our ability to make better judgments, as found in the brilliant work of Fischer Black, Peter Bernstein, Marty Leibowitz, and many others; and (2) true hypotheses and theories, which are testable and, as I’ll explain shortly, falsifiable. The latter are few and far between. Andrew Lo’s book aims to present a new theory of capital markets, but, while it does not really do so, it is full of brilliant insights into behavior, evolution and the ways in which these factors help us to better understand how capital markets work.

What is a hypothesis?

Karl Popper, the great 20th century philosopher of science, said that a hypothesis is a statement, intended to explain a set of observations, which can be falsified. It only takes one exception to a supposedly universal rule to prove it wrong: the existence of one black swan, Popper said, proves that the rule “all swans are white” is incorrect.[1]

By this standard, Lo’s adaptive markets hypothesis (AMH), despite its name, is not a theory or hypothesis. It is a set of observations about human nature and, by extension, the behavior of markets. Lo delivers a detailed and thoughtful critique of the EMH in particular and modern finance in general, but no new theory and no revolution in financial thinking. Along the way, however, he introduces a great deal of challenging and informative material, and for that reason the book is worth reading.

The EMH in context

The EMH, in contrast, is a proper hypothesis or theory. It is a theory of price. It says that the price of a security reflects all available information. (If this is true, you cannot beat the market on a risk-adjusted basis except through pure luck, but that is a consequence of the EMH, not an essential element of it.) If all available information can be reduced to a set of period-by-period – say, annual – cash-flow forecasts, then the price of the security is the discounted present value of those cash flows, where the discount rate reflects the amount of risk inherent in the forecasts.

The EMH says that the market performs this task correctly. It does not say that every individual does it properly, only that in aggregate the answer is correct: “the price is right.”

The EMH can be falsified by finding even one example of an abnormal profit opportunity in the market that is not arbitraged away. There are so many such examples that no one considers the EMH to be literally true any more.

Spoiler alert: Here’s the adaptive markets hypothesis

The only thing that is lacking, then, is an alternative to the EMH. It is not good enough to say “the market is not efficient.” How, then, are security prices formed if not by market participants collecting all available information and agreeing on a price so they can trade? Do they ignore certain types of information? Do they disagree on the importance of each piece? Of course they do, but what is the mechanism?

Here is Lo’s presentation of the AMH:

The basic idea can be summarized in just five key principles:

  1. We are neither always rational nor irrational, but we are biological entities whose features and behaviors are shaped by the forces of evolution.
  1. We display behavioral biases and make apparently suboptimal decisions, but we can learn from past experience and revise our heuristics in response to negative feedback.
  1. We have the capacity for abstract thinking… predictions…based on past experience; and preparation for changes in our environment. This is evolution at the speed of thought, which is different from but related to biological evolution.
  1. Financial market dynamics are driven by our interactions as we behave, learn, and adapt to each other, and to the social, cultural, political, economic, and natural environments in which we live.
  1. Survival is the ultimate force driving competition, innovation, and adaptation.