The Wall Street Journal says that funds given a top “star” rating by Morningstar won’t be top performers. But the Journal’s findings are neither new nor as conclusive as its article states.
The Wall Street Journal published its findings in a long investigative article, “The Morningstar Mirage,” which appeared on October 25, 2017. Morningstar responded with articles by three of its executives – Don Phillips, Kunal Kapoor and Jeffrey Ptak.
The Journal’s central claims were that a high percentage of top-rated (five-star) funds don’t maintain their top ratings over time, but that achieving that five-star rating attracts significant flows from investors. As a result, it claims, investors have suffered by choosing top-rated funds.
Morningstar’s response centered on the fact that, as the Journal’s findings demonstrated, five-star funds maintained higher ratings than four-star funds over the time period studied. Similarly, four-star funds maintained higher ratings than three-star funds, and so on. Thus, choosing a higher rated fund gave investors an advantage.
The performance deterioration of top-rated funds has been known as far back as 1999, in a study by Matthew Morey and Christopher Blake in Morningstar Ratings and Mutual Fund Performance and by Morey in 2003 in “Kiss of Death: A 5-Star Morningstar Mutual Fund Rating?” The fact that asset flows follow top-rated funds was documented in 2001 by Diane Del Guercio and Paula Tkac in “Star Power: The Effect of Morningstar Ratings on Mutual Fund Flow.” Several studies have been published since these papers confirming, updating and expanding their findings.
The Wall Street Journal failed by not acknowledging this prior research. It led its readers to believe that it had discovered previously undocumented relationships.
But the core issue is whether Morningstar’s “star” ratings are a reliable guide for investors and advisors. On this point, Morningstar has consistently maintained that its ratings are “moderately predictive” of future performance and are a “starting point” for future research. (It makes stronger claims with regard to its analyst ratings, which I will come back to later.)
The Wall Street Journal acknowledged that it was aware of the extent to which Morningstar claimed its star ratings had predictive power.
Given the modest claims Morningstar makes and has made with regard to its star ratings, its response to The Wall Street Journal’s claims is a sound defense. If it is a battle words over the semantic issue of the meaning of “moderately predictive” power, Morningstar is the victor.
The Journal’s is also at fault for injecting a substantial amount of anecdotal evidence into its reporting, in the context of interviews with advisors and investors who relied on star ratings but were disappointed with the results. I know of no reliable study that shows that advisors consistently rely on star ratings as a key determinant in fund selection. The Wall Street Journal should have stuck to its statistical analysis.
Indeed, our own research, conducted as part of a focus group with 30 large RIAs, showed that star ratings are rarely relied on as a key determinant in selecting actively managed U.S. equity funds.
A better methodology
Before I absolve Morningstar of all guilt, we need to recognize that The Wall Street Journal based its results on the subsequent star rating of funds, not on their subsequent performance. Thus, it was using the subsequent star rating as a proxy for subsequent performance.
Also, it grouped all funds with the same star rating together. Thus, the only way to obtain the benefit of five-star funds outperforming four-star funds is to buy all the funds with five stars, which is obviously impractical.
A better question to ask is the following: What is the probability that a randomly selected five-star fund will outperform a randomly selected four-star fund, where performance is measured based on raw returns or risk-adjusted returns?
This is a key question, and one which we studied in a 2009 article, “Morningstar Ratings Fail over a Full Market Cycle.” We measured the star ratings across five categories: U.S. equities, international equities, balanced funds, taxable bonds and municipal bonds. We measured the probability that a one-star increase in ratings (e.g., from a four-star to a five-star fund) would result in better performance.
The average probability of improvement over all five categories was 50.6%, barely better than flipping a coin.
That might still fit within Morningstar’s definition of “moderately predictive,” but it’s a better methodology than what The Wall Street Journal used.
What about Morningstar’s analyst ratings?
Morningstar makes stronger claims about the predictive power of its analyst ratings than its star ratings. It describes them as “forward-looking” with a better ability to identify risk-adjusted outperformance, as compared to star ratings.
The Wall Street Journal found the same pattern with analyst ratings as it did with star ratings – the performance of top-rated funds tended to converge with those lower-rated funds. Based on this finding, it discounted the predictive value of analyst ratings.
But, as with its study of star ratings, The Wall Street Journal’s methodology was imperfect.
In 2014, we did our own study of the first “vintage” of analyst ratings, and published our findings in an article, “A First Look at Morningstar's Analyst Ratings.” We used a similar methodology to our study of star ratings, but asked a slightly different question: Is a randomly chosen fund based on analyst rating likely to have a greater alpha or excess return than a randomly chosen fund based on expense ratio? In other words, how does a strategy of selecting funds based on analyst rating compare to the naïve strategy of selecting funds purely on the basis of expense ratio?
Our results showed that the analyst ratings had little predictive power when compared to selecting a fund based on expense ratio. Indeed, an advisor would be better off selecting the lowest cost funds than selecting gold funds, based on either alpha or excess return.
Our results were inconclusive, because they were not conducted over a full market cycle. The value of active managers and of a rating system to identify future outperforming active managers is greatest under adverse market conditions. The star ratings have been available only since 2011 and U.S. equities have not had a bear market since then.
Implications for advisors
It’s incredibly difficult (but not necessarily impossible) to identify active managers that will outperform the market on a risk-adjusted basis. That assertion has been proven by countless academic studies.
The Wall Street Journal added nothing new to this field of study.
But its reporting is an important reminder that star ratings are only “moderately predictive” of future performance. Advisors should recognize the extremely modest value of something that is moderately predictive. Advisors should apply this familiar admonition to star ratings: Past performance is no guarantee of future results.
Advisors should withhold judgement of the analyst ratings until they have endured a full market cycle.
As I have written previously, the single most reliable guide to selecting actively managed funds that will generate risk-adjusted outperformance is low costs.
Read more articles by Robert Huebscher
