The Art and Science of Performance Benchmarking

This benchmarking indicates a huge shortfall and says we and our clients should avoid these types of funds. But is that a valid conclusion? I agree with Morningstar’s John Rekenthaler that it’s not. Has the Bank of America consistently avoided international stocks and only recommended large-cap U.S. stocks (S&P 500) over this timeframe? Does it manage portfolios for no fees? As Rekenthaler put it, the bank is using “perfect hindsight” in selecting indexes to benchmark against to show a point it wants to make. He does his own analysis by benchmarking against the actual asset allocation of these funds, throwing in fees, and concluded “the funds look fine when compared with the existing alternatives.”

Selecting the right benchmarks

While I happen to agree with Rekenthaler’ s benchmarking much more than Bank of America’s, his isn’t perfect either. Admittedly, I’ve never done a perfect benchmarking. That’s because, when it comes to benchmarking, there is judgment involved. There are over three million indexes and those numbers are increasing constantly.

There is clearly crooked benchmarking. The most common I see is comparing a portfolio’s total return to the S&P 500 index. Any index includes only the part of the return that comes from price appreciation. It strips out the dividend portion. Comparing a U.S. stock portfolio to the raw S&P 500 index is comparing the total return of the total U.S. stock market to part of the return of part of the U.S. stock market. I once spoke with behavioral economist Dan Ariely on this subject. He noted that it’s human to pick a benchmark that will make us look good. But using the raw price of only the S&P 500 index goes past being human and into the realm of being evil. Still, I see it often.

Building a custom benchmark using some of these indexes involves a ton of judgement. How granular should the benchmarks be? Should we benchmark a small-cap-value portfolio against a small-cap-value index? Should we compare an alternative market-neutral fund against an index of all market-neutral funds? If we benchmark at the most granular level, it becomes useless. Using literally an “apple to apple” example, we would benchmark Apple stock against Apple stock.

Beyond that, when a portfolio shifts asset allocations over time, should we shift the weightings of the benchmarks? Doing so might reward the manager for performance chasing.

How I benchmark

I’m a believer that broader is better. The more granular the benchmark, the more likely the portfolio will perform exactly like the benchmark, which renders the performance analysis close to worthless. Fees should be included in the benchmark but at the lowest level. I use the following five benchmarks:

1. U.S. stocks – Vanguard Total Stock Index (VTI)
2. International stocks – Vanguard Total International Stock Index (VXUS)
3. U.S. Bonds – Vanguard Total bond Market Index (BND)
4. Real estate – Vanguard REIT Index (VNQ)
5. Cash – Vanguard Federal Money Market Fund (VMFXX)

I use Vanguard because these funds have been around the longest (giving more data to benchmark) and are among those with the lowest fees. These are returns that could have been achieved after considering costs. However, these benchmarks are far from perfect. For example, a small-cap-value fund has significantly more risk than the broad market. Investment-grade corporate bond funds have more risk than funds that are mostly backed by the U.S. government. I would point out to the client that I haven’t adjusted for these risk differentials.

There is far more judgment involved in private investments (which I typically recommend against holding) such as hedge funds or private equity funds. Here I look at the underlying investments (or stated objective) but then add an illiquidity premium, which can be as high as 4%, depending upon the terms of the investment. Even if a fund says it provides quarterly liquidity, if it has the right to gate redemptions, I will pump up the benchmark. Liquid alternative assets are even harder. For example, a market-neutral fund should be benchmarked to the risk-free rate; I’d use cash but note the fund has far more risk.

The benchmarks I’m suggesting are extremely subjective. My main goal is to help you think about the appropriate benchmarks. The more difficult it is to determine the appropriate benchmark, the less likely the holding belongs in a portfolio. If you can’t explain the fund to your client, you should rethink using it.

Using any hindsight in weighting the benchmarks creates a bias. I typically use the average portfolio weighting over the longest period my client has data. For example, if my client has four years of data, I take the average weighting of those four years. If weightings changed significantly over those years, the impact of that market timing will be reflected in the analysis.

You can’t eliminate bias

Bias is part of human nature, though we can work hard to minimize it in the subjective art and science of picking benchmarks.

We would have an incentive to use higher performing benchmarks if we were analyzing someone else’s portfolio, especially if we had something to gain by showing underperformance. But if it’s a portfolio we recommended, just the opposite is true.

To minimize bias, I suggest the following:

1. Select benchmarks independent of selecting of money managers.
2. Select benchmarks ahead of time.
3. Benchmarks must be consistent over time.
4. Broader is better (total stock market index or total international stock market index).
5. Match characteristics rather than using a narrow focus (market neutral, managed futures).
6. Weighting should be the average over the time period to separate alpha caused by:

• • Manager
  • Timing

Summing it up

Because our clients deserve to know how their portfolio has performed relative to reasonable benchmarks, there must be a high dose of science thrown in along with the art. The average return of the U.S. stock market over any period of time is a cap-weighted total-stock-market fund less the lowest fee. Some point to a Wall Street Journal article noting it’s not quite technically true because of IPOs, stock repurchases, and the fact that indexes are reconstituted. I disagree; the first two are minor while total-stock index funds have very little reconstitution.

Even portfolios I’ve advised on typically underperform the benchmark. Though I usually recommend buying the benchmarks, there are often holdings I don’t sell due to tax consequences. It’s hard to beat appropriate benchmarks.

Allan Roth is the founder of Wealth Logic, LLC, a Colorado-based fee-only registered investment advisory firm. He has been working in the investment world of corporate finance for over 25 years. Allan has served as corporate finance officer of two multi-billion-dollar companies and has consulted with many others while at McKinsey & Company.

Read more articles by Allan Roth