“Determinants of Portfolio Performance,” the seminal 1986 paper on asset allocation by Gary P. Brinson, Randolph Hood, and Gilbert L. Beebower (BHB), begins by citing a study showing that most corporate pension funds focus their attention solely on the problem of manager selection. BHB’s research presents evidence, however, that 93.6% of total return variation is due not to manager selection, but to asset allocation. Often, the takeaway is that the variation in performance across investment funds is 93.6% attributable to asset allocation, and only marginally to manager decisions such as security selection and market timing.
But that’s not what it says at all.
The race is to the swift – or is it?
Let’s suppose you are doing research on the performance of 100 racers who run the Boston marathon – a hilly course over which lap times change from mile to mile. The runners all do their own stretching exercises before running, and they drink the energy drink and eat the granola bar that are handed out by the race sponsors.
You want to know how much the energy drink and the granola bar contribute to their performance. So here is what you do.
You have each runner run the course four times. The first time it is run, each runner does their stretching exercises and drinks the energy drink and eats the granola bar, and you record each runner’s times over the miles of the course and call them data series A.
The second time the runner stretches and only drinks the energy drink, and the recorded times are data series B. The third time the runner stretches and only eats the granola bar and the times are data series C, and the fourth time the runner only stretches and the times are data series D.
Then following BHB, you compute the correlation coefficient of D against A, C against A, and B against A (BHB call them “regressions”), and square each correlation coefficient to get an R-squared. The results are shown in Table 1.
Table 1. R-squareds of correlations of 26-mile sequences of running times
|
Only energy drink (sequence D), against both energy drink and granola bar (sequence A)
|
97.8%
|
|
Only granola bar (C ), against both (A)
|
95.3%
|
|
Only stretching (B), against both (A)
|
93.6%
|
The conclusion is that 93.6% of the variation in running times is explained by stretching, and only a relatively tiny percentage by the energy drink and the granola bar.
What does the variation over the miles of the course have to do with this?
You thought we were going to do something to measure how the runners’ times for completing the marathon varied depending on whether they drank the energy drink, ate the granola bar, or only stretched. But it seems we did something different.
For some reason we focused on the variations within the sequences of running times over the 26 miles of the course, not the total times. And for some reason we computed the correlations between those sequences.
Of course, all of the sequences are going to have high correlations to each other. The variations are because some miles are uphill and some miles are downhill, not because of whether you drank the energy drink or ate the energy bar or stretched. You could have picked any of the combinations and it would “explain” more than 90% of the variation. Every time you run the course your mile-by-mile times are going to correlate closely, no matter what else you did.
Is there any reason to conclude from these results that the most important thing to do is to stretch – and that it is much more important than whether you have an energy drink or a granola bar? Of course not.
Asset allocation studies: same absurdity
In the case of asset allocation, the conclusion of the BHB study has essentially zero to do with the study’s so-often-repeated statistical result. The conclusion is right, in a general way, for other reasons – as I will explain later – not because BHB’s research shows that it’s right or that it quantifies it, which it does not.
To imagine that the BHB study quantified the importance of asset allocation relative to other factors is as silly as imagining that the marathon research described above quantified the relative importance of stretching, eating granola bars, or drinking energy drinks.
Does it even measure the contribution to variability?
Idzorek goes on to say, however, “It is possible to have a high R-squared, indicating that the return variations in the asset-class factors did a good job of explaining the return variations of the fund in question.”
Does it really indicate that? Let’s go back to the marathon example. Suppose that the runs that were preceded only by stretching had a 99.9% R-squared with the runs that were preceded by eating and drinking also.
Does that mean that stretching “explains” the variability of the one-mile running times? No it doesn’t; the hilliness of the course explains that variability. So how can the R-squareds be said to help explain where the variability comes from, and by what percentages?
This mistake, too, was later identified by Idzorek himself in a 2010 article, in collaboration with three other authors, James X. Xiong, Roger G. Ibbotson, and Peng Cheng. Xiong et al. point out that, like the variability of mile-by-mile race times over the hilly Boston marathon course, the variability of quarterly or monthly returns over time is due mostly to the variability in the market itself, not to asset allocation policy, market timing, or security selection.
The end result is that the 93.6% number means nothing at all. It doesn’t signify the differences of performance from fund to fund as a result of differing asset allocation policies, and it doesn’t signify the percentage of variation of performance from fund to fund as a result of asset allocation policy – neither of which BHB intended it to do, nor claimed that it did. But the 93.6% number doesn’t even signify the percentage of variation in returns across time as a result of asset allocation policy, as BHB did conclude that it did.
Is there a “right” way to get the answers?
So far I have shown that the BHB results answer no meaningful question. Therefore, the 93.6% number they obtained (91.5% in a follow-up article in 1991) should be ignored.
The real question, of course, is how relatively important is long-term asset allocation policy on the one hand, and active management (i.e. market timing and security selection) on the other? This is the question BHB set out to answer. In a follow-up article in the Financial Analysts Journal, one of the original authors of the BHB study, L. Randolph Hood, explains:
In the early 1980s, Gary Brinson and I were wondering why our institutional pension clients spent so much time and effort in manager searches and so little time in reviewing their asset allocation policies. It was not as if all our clients had identical risk tolerances, liability streams, and funding policies. In discussions with the clients, we discovered that they had a firm belief that manager selection was important (and it is) because they could quantify the benefits of superior management. They could not, however, or perhaps did not wish to, quantify the contributory effects of their allocation policies on the returns to their funds.
The question would, however, appear to have been answered by the paper referred to above by Xiong et al., which is one of the most recent contributions to the academic-journalistic literature. The title of that paper is, “The Equal Importance of Asset Allocation and Active Management.” And so, it would seem, they have answered the question.
And to practitioners of the mindless “evidence-based,” regression-obsessed approach that is prevalent in the academic literature, it would indeed seem that they had answered the question.
Instead of running a time-series “regression” as BHB did, Xiong et al. ran a cross-sectional “regression.”1 For their data, they used the 120 monthly returns from May 1999 to April 2009 for 4,641 U.S. equity funds. By means of a style analysis, they attributed an asset allocation policy to each of these funds. Then they calculated, for each month for each fund, what its return would have been if it had merely invested in index funds in proportion to its policy mix.
Then they regressed the actual returns in each month across funds against their policy returns by calculating a correlation coefficient (or R-squared).
The result was that the R-squareds fluctuated wildly from month to month, in a range from 0 to 0.90. However, because their average was about 0.40, Xiong et al. concluded – as per their title – that asset allocation and active management were about equally “important.”
What does “important” mean?
The original question was, how relatively “important” are long-term asset allocation policy on the one hand and active management on the other?
What could be meant here by important? Presumably, it means how relatively important it is to monitor and control asset allocation on the one hand, and manager selection on the other.
In an earlier article, I invoked a game called the “Red Bean Game” that the statistician and efficiency expert W. Edwards Deming used to illustrate the travesty of a manager trying to make workers produce more products of high quality, in a situation where quality actually varies uncontrollably at random. Deming was trying to get managers to focus on the controllable variables, instead of wasting time and effort trying to control the uncontrollable ones.
Deming’s lessons were absorbed by the Japanese automotive industry, to magnificent effect, but they have not been absorbed by the investment industry. Investment manager performance can be shown to be mostly inherently uncontrollable. For example, managers who are fired for poor performance subsequently do equally as well – or as poorly – as those who are retained for good performance. Except for fees and level of risk – for which asset allocation is a proxy – manager performance cannot be shown to correlate with any known control variables.
Therefore, the relative importance of asset allocation – in terms of the only thing that matters, controllable variables – is much greater than that of manager selection. Xiong et al.’s study does not tell you that; it just tells you the results of mindlessly obtaining R-squareds on a historical data set.
As I implied before, the apparent intent of these data studies is to obtain so-called “evidence-based” results that are totally objective, by letting the data speak for itself through statistics – without biasing the results by contaminating them with either common sense, or theoretical analysis, or simple logic.
If this is the purpose of academic research in finance, then it is in need of a major overhaul.
The only asset allocation paper you’ll ever need
There is one paper that gets at the truth of asset allocation very well and clearly. It is Vanguard’s 2007 paper, “The Asset Allocation Debate.” This is the only paper anyone needs to read.
The Vanguard paper does cite the literature in the academic journals on asset allocation, beginning with the BHB study. A reader might think that the Vanguard paper was therefore a compilation drawn from all the wisdom produced by the academic studies, much as a paper explaining quantum theory would be drawn from all the learned scientific research performed by the creators of quantum theory.
Vanguard’s paper rehearses the well-known fact that “Broadly diversified portfolios with limited market timing tend to move in tandem with broad financial markets over time, resulting in high time-series R2s,” but also that, “Despite this co-movement, active management creates significant performance dispersion across portfolios, resulting in low R2s across funds’ actual and policy returns.” But it points out that results of active management are, “less stable and less predictable … over time” – and, hence, less controllable. It concludes that ,“Unless there is a strong belief in the ability to select active managers who will deliver higher risk-adjusted net returns, investors’ focus should be on the asset allocation choice…”
Citing the academic studies does, however – in the way of things – tend to lend added credibility to the Vanguard paper. This is too bad. Do we really need a steady stream of academic papers filled with mathematics that are made to appear more complicated than they really are in order to make common-sense conclusions credible?
As I said before, academic research in finance – and the willingness and, even, eagerness of practitioners to cite it, and, all too often, to misinterpret it – is in need of a major overhaul.
Economist and mathematician Michael Edesess is adjunct associate professor and visiting faculty at the Hong Kong University of Science and Technology, chief investment strategist of Compendium Finance, adviser to mobile financial planning software company Plynty, and a research associate of the Edhec-Risk Institute. In 2007, he authored a book about the investment services industry titled The Big Investment Lie, published by Berrett-Koehler. His new book, The Three Simple Rules of Investing, co-authored with Kwok L. Tsui, Carol Fabbri and George Peacock, was published by Berrett-Koehler in June 2014.
1 I placed “regression” in quotation marks because in both cases, BHB and Xiong et al., the authors’ “regression” consists merely of calculating a simple correlation coefficient, something that is taught in the early years of high school. The results of such a “regression” are sufficiently conveyed by viewing the points on an X-Y scatter graph. (R-squared is the square of the correlation coefficient.)
Read more articles by Michael Edesess
