Random Walk Part 4 – Can We Beat a Radically Random Stock Market?

This is the final article of my four-part series into the fallacy of the random-walk paradigm. In Part 1 and Part 2, I showed that asset prices do not follow a tidy bell curve and instead are radically random. In Part 3, I demonstrated that many bad risk management practices are the direct results of equating volatility to risk. In this article, I offer a probability-based framework that captures the true nature of investment reward and risk.

The Efficient Market Hypothesis (EMH) argues that the market is hard to beat because very few people could make better forecasts than the collective market wisdom, which instantly discounts all available information. My new reward-risk framework reveals a little-known secret that market gains and losses have very different distribution profiles. We can beat the S&P 500 not by making better forecasts, but by exploiting the dual personality of Mr. Market.

The random walk theory has been the core of modern finance since Louis Bachelier wrote his 1900 PhD thesis. Economists define reward as the mean return (expected value) and risk as the standard deviation (volatility) of the returns. These mathematical terms may be convenient for academics in formulating their economic theories, but make no sense to the average investor. Reward has a positive overtone but mean could be negative. Risk has a negative undertone but standard deviation weighs gains and losses equally. Investors view reward and risk as two sides of the same coin – reward comes from gains and risk comes from losses. My reward-risk framework quantifies this subtle diametrical symmetry.

The random-walk definitions of investment reward and risk

Modern finance adopted the mean-variance paradigm to frame reward and risk. Appendix A presents the mathematical definitions. Figure 1 illustrates the reward and risk graphically with the annual returns of the S&P 500 from 1928 to 2016 (data sources: MetaStock and Yahoo Finance). The dark blue curve is the random walk probability density function (PDF). Reward (mean or expected value) is computed by integrating the total area under the PDF curve using equation A1 in Appendix A. Risk (the square root of variance), computed with equation A2, is one-half of the width of the light blue central region bounded by ± one standard deviation. The random walk PDF roughly matches the data (the jagged gray area) in the central region except near the peak. Beyond ± one standard deviations, data reside mostly above the PDF curve.

Figure 2 compares the random walk PDFs (blue curves) to the actual S&P 500 returns (gray areas) in a one-, five- and 10-year horizon. The peaks of the PDFs denote the means. The red arrows signify ± one standard deviations. Random walk's notions of mean and volatility bear no resemblance to actual returns and risks in the real world. The longer the return horizons are, the larger the gaps. This is why so many conventional risk management practices derived from the mean-variance paradigm broke down during financial crises. The academics' bell curve paradigm offers investors no protection against financial market risks.