The popularity of the endowment model among advisors has been driven by the belief that hedge funds have produced positive risk-adjusted returns. But the basis for that notion has been statistics gleaned from hedge fund databases, and new research shows returns from those databases are even more upwardly biased than previously thought; the supposed alpha never really existed.
For years, hedge fund databases have contained only data selectively and voluntarily reported by certain funds. Those who research and compile these data have trumped up feeble explanations of why this does not bias the databases, though it clearly does. The uncritical financial media have robotically reported the database statistics as if they were facts.
Now, at last, someone has constructed a database that does not have this self-selection bias, and – surprise, surprise – it reveals that hedge funds on average perform no better than mutual funds. Perhaps worse.
Registered funds-of-funds
In a study titled, “Out of the dark: Hedge fund reporting biases and commercial databases,” Adam L. Aiken of Quinnipiac University, Christopher P. Clifford of the University of Kentucky, and Jesse Ellis of the University of Alabama assessed the performance of hedge funds using a unique data source that is untainted by the self-selection bias afflicting other hedge fund databases.
Registered funds of hedge funds are funds-of-funds that register with the U.S. Securities and Exchange Commission (SEC) under the Securities Act of 1933 and the Investment Company Act of 1940. Registration allows a fund-of-funds to escape certain constraints. For example, while managers of unregistered funds are not allowed to publicly market their funds, registered funds-of-funds can advertise the same way mutual funds do. Furthermore, managers of registered funds-of-funds can give their investors the comfort of knowing that they are more transparent than unregistered funds, reducing the possibility of fraud.
Registration does impose a reporting burden on the funds-of-funds. They must report each quarter their holdings and the cost and market value of each holding. These reported data make it possible to calculate quarterly returns on each of the hedge funds held by the registered funds-of-funds.
By exploiting these data, Aiken, Clifford and Ellis captured more than 10,000 quarterly returns for 1,445 hedge funds over the period 2004-2009. They found that about half of the fund-quarters are also in at least one of the commercial databases – Lipper TASS, Hedge Fund Research, BarclayHedge, Morningstar, or Eureka Hedge. Hence, it is possible to compare a large sample of hedge funds that report to a hedge fund database with a large sample of funds that do not report their data.
Hedge fund reporting biases
It has long been known that hedge fund databases are riddled with bias. In an extensive 2005 paper on hedge funds, academicians Vikas Agarwal and Narayan Y. Naik listed the sources of bias, virtually all of which skew average risk-adjusted returns upwards. I can summarize them as follows.
Data accuracy and measurement bias. According to Agarwal and Naik, “The organizational structure of hedge funds, as private and often offshore vehicles, makes data collection a much more difficult task, amplifying the impact of performance measurement biases.” Auditing makes a clear difference in data quality, Agarwal and Naik said, but only 40% of hedge funds are audited. Hence, data reported by hedge funds to databases may not be reliable.
Survivorship bias. Hedge funds have a high attrition rate. About 15% of hedge funds disappear each year (and many more did following the 2008 financial crisis). Hedge fund databases contain only the survivors, which are, of course, the better performers. Agarwal and Naik cite estimates saying that average returns in hedge fund databases are about 3% too high because of survivorship bias.
Backfill bias. Many hedge fund strategies are run for a while by their managers before they are offered to investors, to see if the strategy seems to work. If the manager then decides to offer it to investors and to report the data to a commercial database or databases, the returns from the period before it was available to investors are backfilled into the database. Obviously this represents a bias, because the backfilled returns for other strategies that did not work, and thus were not offered to investors, are not in the database. In their 2011 study of hedge funds, Roger Ibbotson, Peng Chen, and Kevin Zhu estimated that adjusting for both survivorship and backfill bias causes the average net return for their equally-weighted sample of hedge funds to plummet from 14.26% in the database to 7.63% – almost a seven percent reduction, bringing the average performance below the S&P 500 return of 8.04% during the period that they measure.
Stale price bias. According to Agarwal and Naik, “Hedge funds invest in relatively illiquid securities, for which market prices may not be always readily available. In order to report returns at all dates, the last price of the security is often used. This is referred to as stale price bias.” Stale price bias poses two problems: (1) volatility will be underestimated; and (2) the degree of correlation with market indices will be underestimated, and therefore market index coefficients, such as beta, will be underestimated. When beta is underestimated, alpha – risk-adjusted return – is overestimated. Database researchers have responded by regressing hedge fund returns against market returns for both contemporaneous and prior time periods, but this is a very poor solution. Since it is not known how stale the prices are, it is not known against exactly which earlier time period the returns should be regressed. Unfortunately, this bias is present in Aiken, Clifford, and Ellis’s study also, with the likely effect of overestimating alpha. Hence, the real average hedge fund alpha may well be strongly negative.
And last but not least:
Self-selection bias. Concerning this bias, Agarwal and Naik explain:
Self-selection bias may arise if only funds with good performance chose to be included in a database. This can lead to an upward bias in the reported historical performance of hedge funds. However, this upward bias is limited, as funds with good performance may sometimes choose not to publish their performance as they may have reached their goal in terms of assets under management or their target size and thus, may not wish to attract more investors. Fung and Hsieh1 [among others] conclude that this bias is negligible based on such reasoning.
The conclusion that the database bias introduced by non-reporting poorly performing funds is entirely counterbalanced by non-reporting winning funds is highly speculative. In an investment sector in which 15% of funds fail annually, there will be a lot of funds that don’t publicize their performance. As a case in point, Long-Term Capital Management’s high returns in 1994-1997 were in the databases, but its huge loss in 1998 was not. There has never been, in point of fact, any evidence that hedge funds that did not report to the databases, due to inferior returns, were offset by hedge funds that did not report due to superior returns.
1. W. Fung and D. A. Hsieh, “Performance characteristics of hedge funds and CTA funds: Natural versus spurious biases,” Journal of Financial and Quantitative Analysis, vol. 35, no. 3, pp. 291–307, 2000.
Other problems with the databases
Aiken, Clifford and Ellis record a myriad of other biases that could affect the databases. It is worth quoting them on this observation at length. As they point out:
Funds have some discretion as to the timing of their reports to the databases. … In most cases, the fund has up to 3 months to file its monthly return. … A fund with poor performance in a given month may have the incentive to delay reporting, increase the fund’s risk, and hope for a better outcome in the next month. If the strategy works, both monthly returns are listed. If it does not work, the fund never reports either return to the database. A similar version of this bias occurs when a previously delisted fund is allowed to rejoin the live funds file. If a fund is willing to fill all gaps in its time-series of returns, it is allowed to rejoin the database. As it is likely that only funds that performed well during their delisting period will re-list, these features of the commercial data will impart a further upward bias on the return data.
Second, there is evidence that funds retroactively revise or completely remove returns from databases, creating scope for a deleted/revised history bias. For example, we downloaded the BarclayHedge database in both August 2008 and August 2010. Looking at both the live and graveyard sections of the 2010 data, we notice 67 hedge funds that were present in the 2008 data but have been completely deleted in 2010. We find deleted funds in Lipper TASS as well. Conversations with BarclayHedge revealed that funds may ask the databases to completely remove their return history from the database. If it were the case that poor return histories are more likely to be deleted, then the observed returns in a database are biased.
The authors further cite a recent working paper that found that nearly 40% of hedge funds either deleted, revised, or added returns to their historical record. “These revisions appear to be non-random and predict subsequent underperformance on the part of the revised funds.”
Remember that the business purpose of hedge funds is to cunningly exploit arbitrage opportunities in the financial markets. Should it surprise us if they apply their skills also to finding arbitrage opportunities in the database reporting procedures – opportunities to make their risk-adjusted returns appear in a better light? If the hedge funds are indeed inclined to seek out and take advantage of database arbitrage opportunities, the rules for reporting or not reporting to the databases make it easy to do so.
Previous results of hedge fund studies, and Aiken, Clifford and Ellis’s findings
The received wisdom is that hedge funds outperform other investments, such as mutual funds, on a risk-adjusted basis. This wisdom has been buttressed by studies using the hedge fund databases that find abnormally positive risk-adjusted returns. For example, the Ibbotson et al. study cited above, while noting that the average hedge fund underperformed the S&P 500 on a risk-unadjusted basis, found an average annual alpha for hedge funds of about 3%. (Some studies using different methodologies, however, have obtained different results.)
All of these studies suffer from the hedge fund database problems described above. They have attempted to correct for some of these biases – in some cases, using credible methodologies – but there has been no good way to correct for the self-selection bias.
Aiken, Clifford and Ellis, using the data in their registered funds-of-funds database, obtained very different results. They compute alphas using three different factor models. In all cases, they obtain alphas that are essentially equal to zero. When they divided the funds in their database between the roughly half that are also in the commercial research databases, and those that are not, they found that the annual average return for the funds in the commercial databases exceeded by four percentage points the average return for those that are not. In other words, self-selection biases the average return upward by at least 4% – more than enough to completely eliminate the 3% alpha found by the Ibbotson study.
Why has the faulty wisdom been so credulously embraced?
How to explain the credulity of a statistics-greedy financial media, the gullible public, and even the supposedly skeptical academic community, all of whom accepted hedge fund alpha findings that were based on dreadfully contaminated databases? There are several important points to consider.
First are two common fallacies, which mistake a truth about a part of the whole for a truth about the whole.
Everybody knows that David Swensen of Yale secured market-beating returns for the Yale endowment fund through a strategy that, among other things, included substantial hedge fund investments. This has produced the impression that all hedge funds perform well, or at least that the average hedge fund performs very well.
Everybody knows, also, that only rich people can invest in hedge funds, and rich people are rich because, we assume, they are smart about money; ergo, what is true about how well rich people make their money must be true about how well they invest their money. Therefore, hedge funds must be a superior investment.
In addition, there is a tendency to say that, when good data are lacking, we have to do the best with the data we have. Academia is not content to say simply that the database is too contaminated to helpfully use. The contamination gets swept aside and forgotten in the name of supposed pragmatism; only the results are published and discussed.
A third and potentially most pivotal cause, however, is symptomatic of a major problem of our day. In an interview since his recent release from prison, Jack Abramoff, the former lobbyist, explained his former power to affect legislation, saying, “The minute I would go to staff and say to them, you know, in a couple of years why don’t you come work for me, at that moment I owned that office … that person worked for me, because they were thinking about their future employment.”
Most people, especially ambitious people, don’t want to cut themselves off from a potential path to riches. Employment by a hedge fund is a path to riches because of the high fees they charge. Perhaps this insidious incentive structure has steered researchers, financial journalists, and many others onto the mindless bandwagon of self-deceptive belief that hedge funds in general outperform other investments.
Thanks to the work of Aiken, Clifford and Ellis, we finally have clear evidence that this bandwagon is heading in the wrong direction.
Michael Edesess is an accomplished mathematician and economist with experience in the investment, energy, environment and sustainable development fields. He is a Visiting Fellow at the Hong Kong Advanced Institute for Cross-Disciplinary Studies, as well as a partner and chief investment officer of Denver-based Fair Advisors. In 2007, he authored a book about the investment services industry titled The Big Investment Lie, published by Berrett-Koehler.
Read more articles by Michael Edesess
