John Cassidy’s 2009 book, “How Markets Fail,” drives the final nail in the coffin of the Efficient Market Hypothesis.
Well, perhaps the penultimate nail – as I’ll explain later. It is the most compelling argument I have read that we need a new and improved theory of markets, a theory that subsumes the efficient market hypothesis, much as Einstein’s relativity theory subsumed Newtonian physics.
As a bonus, Cassidy’s book is an excellent layman’s introduction to economics. It begins by tracing the history of economic thought from Adam Smith through Friedrich Hayek to Arrow-Debreu, culminating in the perfection of the idea that the collective result of individuals pursuing their self-interest is that social welfare is maximized – what Cassidy calls “Utopian Economics.”
An historic interchange
The book begins with an historic exchange that deserves reproduction here at length. It took place on October 23, 2008, in the chambers of the U.S. House of Representatives, between Henry Waxman, chairman of the House Committee on Oversight and Government Reform, and Alan Greenspan, former chairman of the Federal Reserve. Cassidy describes its climax thus:
Waxman returned to his notes and started reading again. “‘I do have an ideology. My judgment is that free, competitive markets are by far the unrivaled way to organize economies. We have tried regulations. None meaningfully worked.’” “That was your quote,” [Waxman] said. “You had the authority to prevent irresponsible lending practices that led to the subprime mortgage crisis. You were advised to do so by many others. Now our whole economy is paying the price. Do you feel that your ideology pushed you to make decisions that you wish you had not made?”
[Greenspan responded:] “To exist, you need an ideology. The question is whether it is accurate or not. What I am saying to you is, yes, I found a flaw. I don’t know how significant or permanent it is, but I have been very distressed by that fact.”
… but [Waxman] wasn’t finished. “In other words, you found that your view of the world, your ideology, was not right,” he said. “It was not working?”
“Precisely,” Greenspan replied. “That’s precisely the reason I was shocked. Because I had been going for 40 years, or more, with very considerable evidence that it was working exceptionally well.”
This is the stuff of which Shakespearean tragedies are made. Greenspan was the ultimate empiricist. He was thus, in the end, one of the very few to respond publicly to the challenge of his intellectual rival John Maynard Keynes: “When the facts change, I change my mind – what do you do, sir?”
It is easy for people to forget that we are now seeing the obvious flaws in “utopian economics” only in retrospect. Until the financial crisis of 2007-2008, the vast majority of economists and political thinkers were converts to what is now being referred to – pejoratively – as “fundamentalist” free-market theory. It took a new and unprecedented experience for many to change their minds. Having finally seen the truth from which we were blinded, we should remember an essential part of Greenspan’s words: “I found a flaw. I don’t know how significant or permanent it is…”
Reality-based economics
In his section titled “Reality-Based Economics,” Cassidy makes a compelling case that utopian economics are not real economics.
Those who have taught a course in environmental economics, as I did recently, know how flawed the classical models of economics can be. You teach environmental economics by first explaining the classical models of economics, then showing why they don’t work in many – perhaps most – real-world situations. You then proceed to add Rube Goldberg contraptions to patch them up, so that they can be applied anyway. It makes you wonder, “Do we really need these models?”
I came upon Cassidy’s book while I was already teaching such a course. A chapter in the book that makes the transition from the history of the development of classical economic models to the flaws that almost totally invalidate them – it focuses on polar bears and global climate change – was so readable and so elegant an encapsulation of the basic problem of environmental economics that I immediately assigned it to my class.
Assumptions of the utopian model
Here is a brief aside on classical economics assumptions that are often violated in the real world.
In classical economic theories, traded goods are tacitly assumed to be “rival” and “excludable.” Suppose I buy a sausage. If I consume it, only I can consume it. A rival who might like to consume it can’t consume it also. Thus, it is a “rival” good. It is also “excludable” because in the act of buying it, even if I don’t eat it right away, I can cause anyone else who might like to have it to be excluded from having it (thanks of course to government intervention, which protects me against theft). Classical economists also assume that buyers have a perfect and complete understanding of whatever good they are buying.
It’s only under these assumptions that Adam Smith’s magical invisible hand works, allowing markets to self-regulate. In Smith’s time, the economy was simpler, and the vast majority of goods were rival and excludable. Yet even then, as Harvard economist Dani Rodrik points out, Smith believed financial transactions needed to be regulated or they “might endanger the security of the whole society.”
Today, most goods don’t fit the model’s assumptions. To explain why Facebook rapidly took over the business of social networking from seemingly equally well-featured competitors, we need to invent strange terminology like “network externalities,” which are defined as the change in value of a good depending on how many other people use it. In the field of finance in particular, the assumptions don’t fit. For example, people whose pension funds acquired collateralized debt obligations didn’t know the names of investments being bought on their behalf, much less what the investments actually were.
Rational and irrational irrationality
A fundamental assumption of equilibrium economics and efficient markets, beyond those mentioned above, is that economic agents are “rational” – which is to say, their goal is their own self-interest and they know the best way to pursue it. Under these assumptions, equilibrium economics shows that the invisible hand – operating now through mathematics – makes markets efficient and causes individuals’ pursuit of self-interest to combine in such a way as to create the maximum benefit for society.
Cassidy covers, of course, the studies grouped under the banner of behavioral economics that show that people are not necessarily rational in the pursuit of their self-interest. For example, they behave inconsistently when making risky bets, depending on how the bets are presented.
Much more interesting for me, and in my opinion for the economics profession as a whole, is what Cassidy calls “rational irrationality.” These are situations in which individually rational decisions, made in individuals’ self-interest, combine – not to produce the collectively optimal result, as in equilibrium economics – but to produce utterly irrational results, sometimes the worst ones possible.
This has been known for a long time. When a good is non-rival – for example national defense, which benefits everyone in a country if it benefits anyone – then it is rational for each individual not to pay for it. Everyone will wait for someone else to pay in the hopes of free-riding. But all that rational non-payment means the nation is irrationally defenseless. Hence, the government must pay for national defense using coerced taxpayer dollars. Free-market fundamentalists seem to believe that national defense needs to be paid for by the government, but little if anything else does, failing to acknowledge that many other important goods and services meet the description of being non-rival also.
The Prisoners Dilemma
A classic, simple case of rational irrationality is called the Prisoners Dilemma, because one of its versions involves two prisoners. I invite you to take a few minutes to join me for a walk through an example that is not only instructive but fun.
Please begin if you are online by clicking here. Watch the beginning of the video, then pause it at 1:40 (one minute and forty seconds).
For those who are not currently online, I’ll explain the situation. It is a British game show called Split or Steal. A young woman and a man are competing. Each can choose “Split” or “Steal.” If they both choose Split, they split the prize of £100,150. If one chooses Split and one chooses Steal, the one who chooses Steal goes home with the whole £100,150. But if they both choose Steal they both leave with nothing.
Now consider the following diagram:
Choosing Steal is a “dominant” strategy for him – that is, it does at least as well as choosing Split, no matter what choice she makes. If she chooses Split, he wins the whole £100,150; if she chooses Steal, he wins nothing, no worse than if he had chosen Split.
Similarly, choosing Steal is the dominant strategy for her too. Hence, it is individually rational to choose Steal, but the end result is the worst possible for both of them: They both leave with nothing.
That, however, is without the possibility of negotiation. Continue watching the video to see what can happen if negotiation is possible.
What if rational irrationality is more common than rational rationality?
The economics profession has mostly considered this kind of result a rare curiosity. Only in certain niche fields, like environmental economics, it has been thought, may the potential for lose-lose results be more common. (The most-frequently cited paper on environmental economics is “The Tragedy of the Commons,” written in 1968 by Garrett Hardin, who was not even an economist but a sociologist.) Rational irrationality is still considered rare, however, in most economic situations.
But what if such results are not rare at all? What if all market activities have a game-theoretic component that can occasionally – perhaps frequently – result in collectively irrational results? We think of bubbles, crashes, and even bank runs (“panics”) as irrational; but what if the actions that cause them are all individually rational? Bank runs, for example, can be caused by the perfectly rational decisions of depositors to preserve their own wealth. This, I believe, is the challenge that the economics profession faces now.
What do irrational irrationality and rational irrationality have to do with each other?
If there is a weakness in Cassidy’s coverage, it is that he makes no distinction between the irrationalities of behavioral economics and the collective rational irrationality of game-theoretic situations. Frankly, I don’t know what the connection is. I find behavioral economics to be little more than a collection of interesting – and sometimes useful – anecdotes that fall well short of adding up to a theory. “Rational irrationality,” on the other hand, has great potential.
A collection of bidders bid up a price so it is considerably higher than its fair valuation, playing a game of chicken to see who can bail out just in time before the crash. What do they have to lose? Even if they miss the peak, there may still be a reasonably good chance they’ll sell out at a price well above fair value. Modern portfolio theory doesn’t even begin to think about such questions.
Preserve and teach the efficient market model!
Why did I say that Cassidy hasn’t quite driven the final nail in the coffin of the efficient market hypothesis?
Jettisoning that hypothesis now would be even worse than jettisoning classical physics would have been around 1900, when experiments were getting results that couldn’t be explained by Newtonian physics (only to be later explained by the theory of relativity). To do away with the efficient market theory, we must first understand it to the fullest extent of its limitations.
The problem is that most people still haven’t grasped the main tenets of efficient market theory, which are substantially correct. They still need to learn them. If people hadn’t understood, or hadn’t believed in, conservation of mass and conservation of energy in 1900, it would have been dumb to throw out theories that taught them – even if the theories appeared to be not quite right.
In the huge online community of day-traders, for example, there is a saying, “The trend is your friend.” They seem not to know or understand that there is a reason why trends in securities prices are generally not predictive. Most people may see the warning but don’t really believe that past returns are very poor guides to future returns. As a matter of basic financial education – so that people are not overly naïve investors, who pay too much for too little – we need to redouble our efforts to teach these basic facts.
For that matter, perhaps if people had better understood the efficient market hypothesis in 2004-2007 – and its corollary, the random walk model – they would have realized that the fact that housing prices had been rising strongly for several years was no reason for them to keep going up.
If a new theory can explain why occasionally there is momentum in securities prices, but often with an unpredictably timed precipice lurking ahead, so much the better. That is a pursuit for trail-blazing theorists. In the meantime, we need to help people to understand the implications of the efficient market model much better than they do.
Michael Edesess is an accomplished mathematician and economist with experience in the investment, energy, environment, and sustainable development fields. He is a Visiting Fellow at the Hong Kong Advanced Institute for Cross-Disciplinary Studies, as well as a partner and chief investment officer of Denver-based Fair Advisors. In 2007, he authored a book about the investment services industry titled The Big Investment Lie, published by Berrett-Koehler.
Read more articles by Michael Edesess