Advisor Perspectives welcomes guest contributions. The views presented here do not necessarily represent those of Advisor Perspectives.
In recent years there has been a switch from active management to passive index funds by both institutional and retail investors. One justification of this trend is even if some active managers can persistently add value to investment outcomes, investors are unable to distinguish between active managers with such skills and those without. There are at least three approaches that have been demonstrated effective in identifying skill in asset managers, offering investors the likelihood of superior future performance. All three methods can be implemented by advisors or consultants as they do not require “inside the manager only” data. Unlike conventional analyses, long track records are not required. The three differ on what information the investor must have to carry out the analysis.
If markets are very efficient. there should be no persistence patterns in active management returns. While there are innumerable studies showing markets are somewhat efficient, many academic studies of mutual funds show that persistence of superior or inferior performance does exist. At least seven such studies were published in the 1990s alone, with Hendricks, Patel and Zeckhauser (1993) being one of the best known. The results of these studies have a great deal of commonality. All show that statistically significant persistence of returns is relatively short lived, over horizons of one to two years. Given these relatively short-lived effects, it is possible that other sources rather than skill accounted for the observed persistence. One alternative explanation is “mis-classification” where a fund performs differently from peers because it is being compared to the wrong peer group. Another is liquidity effects where a fund with good performance often has large inflows of new investor capital. If such a fund invests in the same securities, it drives up the value of its existing holdings, perpetuating temporarily positive returns.
The question remains as to how to identify real investment skill, which is likely to have long-term positive benefits for investors.
Forecasting precision-weighted excess returns
Our first approach is called “Precision Weighted Excess Returns” (PWER) and was proposed by Bolster, diBartolomeo, and Warrick (FMA Annual Meeting, 2006). The only data requirements for this approach are historical returns for the fund being analyzed and a database of return history of other funds to which the subject fund will be compared. The PWER process involves statistical enhancements to increase the precision and accuracy of the traditional evaluation of investment performance. Such PWER scores have been available for U.S. mutual funds for years and have demonstrated statistically significant predictive power that seems related to true skill.
The motivation for creating PWER was that traditional evaluation of manager performance has material flaws. Much manager evaluation occurs relative to benchmarks that are often not suitable for the manager’s investment approach. The evaluation of past performance is based on standardized periods (e.g., five years) rather than periods that are relevant to the manager in question. Many evaluation measures, such as the Sharpe or information ratio, do not correspond to meaningful economic utility. The statistical significance of ex-post performance is measured crudely, excluding the context of whether the fund exists among a tightly bunched set of peers or a widely dispersed set. This is a critical requirement in examining the “luck versus skill” issue.
The PWER method starts with classification. We want to make sure each fund is being measured against the right benchmark and the right peers. We use an augmented method of returns-based style analysis (Sharpe, 1992) for this purpose. However, we must account for the risk of “gamed” peer comparisons. diBartolomeo and Witkowski (1997) showed that many funds try to gain marketing advantages by misleading investors on comparisons to peers pursuing the idea that “The best way to win a contest for the largest tomato is to paint a cantaloupe red and hope the judges don’t notice”.
The second step in PWER analysis is process control. We want to evaluate each manager or fund over the time period that is best for that fund. Practitioner tradition in the investment industry is to evaluate active manager track records over a long period of at least three to five years. Some organizations argue that performance must be evaluated “over a full market cycle,” although with no formal definition of such a period. Almost all studies refute this. They find no evidence that long-term past performance is predictive of future performance. If there is any meaning to past performance at all, its short-lived, perhaps the last year or two.
What time portion of a track record do we need to evaluate as part of our monitoring of fund managers? We need is a procedure to draw the line between getting enough meaningful data within a manager’s record and dropping old, stale data that is no longer relevant. To solve this question, we utilize a well know statistical procedure from operations research known as CUSUM (Paige, 1954), which is formally known as a “backward looking sequential probability ratio test.” It was created to detect small shifts in the quality of a repeating process (e.g. an assembly line). It has many desirable statistical properties and is much more effective than common procedures such as moving averages.
The use of CUSUM to monitor active managers started with the IBM pension fund as described in Philips, Stein and Yashchin (2003). Their CUSUM procedure classified fund managers into three categories: good, we don’t know, and bad. Obviously, manager or fund decisions would be reviewed whenever a class boundary was crossed. The PWER application of CUSUM is somewhat different. Instead, we look backward in time and ask ourselves: “What is the past moment time T we can identify such that active performance of the fund before and after time T is the most statistically significantly different?” Once that past turning point in time is found, we can ignore results before T. We evaluate whether active performance from time T to the present day is improving or declining.
Our final step is to directly quantify past performance using a return measure such as alpha, not the Sharpe or Information Ratio. Many performance measures are not congruent to adding value for investors, as noted by deGroot and Plantinga (2001). We can simply consider a manager that adds exactly one basis point of return in every time period. The information ratio is infinite, but very little investor wealth is added. Instead, we choose to measure excess return which directly measures added value for investors.
To maximally exploit our information about manager performance, we need to separate skillful managers from the merely lucky. We need to adjust for the fact that if manager returns are widely dispersed within a peer group, it is easier to have a high excess return. If the dispersion of returns is low, it is more meaningful to show significant outperformance (the manager is either skillful or cheating). To weigh the competing concepts of skill and luck we utilize Bayes’s theorem in a fashion similar to Shanken and Jones (FMA Annual Meeting, 2004). This calculation forms a weighted average of past active performance of the subject fund and the average active performance of the peer funds as the estimate of future excess returns of the subject fund. In the original BDW (2006) study, PWER was tested on three large data sets: U.S. funds, international funds, and hedge funds. It was found was found that while fund returns showed about a 20% correlation between the past and present, the PWER values were correlated at over 40%. This highly significant improvement arises from the PWER method identifying skill among fund managers. Similar results have been found in repeat analyses in the intervening period.
PODS: Did fund managers make good choices given what they were allowed to do?
Our second method to identify skill is “Portfolio Opportunity Distributions” (PODs) as first proposed by Surz (1994). In PODs, we need the historical record of a manager’s performance and a description of the limitations that were imposed on the active portfolio (security universe, benchmark, position size limits, liquidity constraints). Using a form of Monte Carlos simulation, we can create a broad range of alternative portfolios whose returns could have arisen had the manager made different choices but under the same constraints. By comparing the realized performance of the subject to the distribution of “what if the manager had done things differently” returns, we can quicky make statistically significant evaluations of whether a fund manager made good choices.
Our implementation of the PODs technique includes a number of mechanical improvements to the simulations. Simulated portfolios can be weighted equally, by capitalization or by square room of capitalization. The number of securities to be included can have an upper bound. Security position sizes can be subjected to constraints either in absolute terms or active weight relative to benchmark. We can also combine PODs with “bootstrap” time series simulations (see An Optimized Approach to Scenario Driven Risk Simulations (northinfo.com)) to overcome the optimistic bias pointed out by Bailey, Borwein, dePrado and Zhu (2012).
Were individual portfolio positions optimal? The EIC Method
Positions in an active portfolio must reflect manager skill in forecasting returns and in constructing portfolios that can benefit from good forecasts. To assess this skill at the individual portfolio position level we can use the method known as the “Effective Information Coefficient” (diBartolomeo, 2008), wherein we use risk models to estimate the implied security return values an active manager must have believed at each moment in time. By correlating the implied returns of individual securities with the subsequent realized returns we can evaluate skill inclusive of all constraints. Multiplying through by the cross-sectional volatility (variety) of security returns yields the expected excess returns for the portfolio. Given that the EIC analysis occurs at the security level (i.e. very large sample size compared to portfolio returns) it achieves statistically significant results quickly but does require full transparency of past and present portfolio holdings.
A fund manager knows their own expectations about expected returns and their constraints so they can separately calculate an Information Coefficient (see Grinold, 1989) describing the quality of their return forecasts. They can also calculate a Transfer Coefficient as defined in Clarke, DeSilva and Thorley (2002). If you assume you know the covariance of the securities (obtained from commercially available models) then an outside observer (investor, consultant) can compute EIC without knowing IC or TC separately. The final required input is how averse the investor is to active risk which you can infer from the portfolio tracking error based on the GLUM concept of Rubinstein (1976).
Dan diBartolomeo is the president and founder of Northfield Information Services.
Read more articles by Dan diBartolomeo
Advisor Perspectives welcomes guest contributions. The views presented here do not necessarily represent those of Advisor Perspectives.
