Mutual fund performance and survivorship bias

Mutual fund performance

As we have noted in previous Mathematical Investor blogs (see this blog for instance), surprisingly few mutual funds beat their respective benchmark (typically some market index). Even fewer consistently beat their benchmark year after year.

A new report from S&P Dow Jones sheds light on this phenomenon. It tabulates, for each year from 2001 through 2017, the percentage of mutual funds in various categories that are out-performed by their respective benchmarks. Here is a brief summary of this performance data.

Table 1: Percentages of U.S. mutual funds beaten by their benchmark

Note that in each category, one would do better over the seven-year period by simply buying a low-fee index mutual fund or exchange-traded fund (ETF) that tracks the corresponding benchmark. We should emphasize that the table above only includes results for six categories of funds, all of which deal primarily if not exclusively in U.S. securities. But the results are pretty much the same in other categories as well — see the S&P Dow Jones report for additional data.

Persistence data

S&P Dow Jones has also compiled some “persistence” data, namely results on the fraction of high-performance mutual funds that are still doing well one year or more later. Here is a brief summary of this data, from a separate 2018 S&P Dow Jones report.

Table 2: U.S. equity funds: 12-month persistence statistics
(Apr 2015 thru Mar 2016 versus Apr 2016 thru Mar 2017 and Apr 2017 thru Mar 2018)

These data are even more dismal than Table 1. Note that for mid-cap U.S. equity funds, not a single one of the top quartile funds for the 12-month period ending March 2016 remained in the top quartile for the 12-month period ending March 2018.

Table 2 focused on 12-month performance data. Do funds do any better over, say, a three-year performance window? Table 3 has some persistence data for consecutive nonoverlapping 36-month performance windows. In particular, the table compares three-year performance for the period ending March 2015 with three-year performance for the period ending March 2018.

Table 3: U.S. equity funds: 36-month persistence statistics
(Apr 2013 thru Mar 2015 versus Apr 2016 thru Mar 2018)

One somewhat bright spot here is that for large-cap stocks, 33.83% of the funds in the first quartile in the earlier period retained their first quartile ranking for the second. But none of the rest of this data shows much deviation from random chance.

Additional persistence data is available in the 2018 S&P Dow Jones persistence report.

Survivorship bias

The data in Table 3 are particularly useful because they include percentages for mutual funds that were merged, liquidated or significantly redirected in investment style. Needless to say, few if any mutual funds are merged, liquidated or are fundamentally redirected in investment style if they are performing in the top quartile or even the top half of similar-category funds. Thus we should consider those percentages to be added to the 3rd or 4th quartile statistics.

If this is done, then the percentages are even less favorable to the hypothesis that the funds are exhibiting significant skill. Perhaps a few funds here and there are consistently performing above-market, but this is not seen in the overall statistics.

These data highlight the issue of survivorship bias, namely the statistical distortion that can result if one focuses only on data that survives for the full period under study, and ignores other data (which are typically failures). The closely related problem of survivorship bias under multiple testing is at the heart of the phenomenon of backtest overfitting, wherein those designing a new investment strategy or mutual fund analyze thousands, millions or even billions of strategy or allocation variations, and then only publish or implement the one with the best results, based on backtest metrics.

As we have emphasized before (see previous blogs A, B, C, D and E), backtest overfitting, and the more general survivorship bias under multiple testing, are unfortunately endemic in the field of finance. They are arguably the principal reason why newly fielded strategies and funds, which typically look great on paper based on backtests, often fall flat or even result in disastrous losses when fielded.

Realities of today’s high-tech markets

More generally, the results above underscore a point that we have emphasized before (see also this blog): The easily mined gold is long gone.

After all, today’s markets by definition incorporate the collective judgments of many thousands of highly trained market analysts worldwide. Many of these investors are high-tech quant operations that utilize sophisticated mathematical algorithms and large datasets, typically running on powerful supercomputers, to ferret out any actionable regularities or correlations, and which employ high-frequency trading algorithms to act on these phenomena in real time. The competitive efforts of these many players largely cancel each other out, leaving a time series that is little more than a random walk.

Thus, we should not be surprised that many mutual funds have difficulty achieving consistently above-market-index results — they are betting against a time series that is (almost) entirely random noise.

There are opportunities for true positive-alpha investing. But, as we have emphasized before, these typically require advanced algorithms, high-tech machine learning techniques and access to big data. In other words, go high-tech, with big data, or leave the market to those who do.

It does appear that more individual investors, at least, are following the advice of Vanguard founder Jack Bogle and author Charles D. Ellis, who recommend leaving relatively high-cost, actively managed funds and migrating to low-cost index funds.

One question is whether this trend to passive, index-based investment, at least for individual investors, will be accompanied by fundamental changes in behavior, such as more consistent saving, less panic selling and fewer attempts to time the market. So far the results are not particularly encouraging. The other question is whether the shift to passive investing will eventually result in significant distortions in the market, e.g., the blind leading the blind. At the present time, however, with only 20% of global assets in passive instruments, this does not appear to be a major issue.