Do new backtested index ETFs outperform the market?

Many investors, individual and institutional, have come to the conclusion that index-linked investments are a rational and, in the long term, profitable investment strategy.

It is certainly true that many individual investors could do far worse that merely investing, say, in a S&P500 index fund or exchange-traded fund (ETF). As we described in a previous Mathematical Investor blog, the typical U.S. equity investor has significantly underperformed the S&P500, with similarly dismal results in other asset categories. In particular, the average equity investor has a 20-year return of 4.25% per annum, compared with a 8.21% average return of the S&P500, for a gap of 3.96% per annum.

There are indications that at least some individual investors have finally got the message that indexed investing is a good idea for at least part of their savings, as indexed funds and ETFs have seen a significant influx. This is in spite of some fairly well-known difficulties with the management of index funds, including the question of whether they adhere tightly enough to the index, and whether they can avoid index front-running, when market arbitrageurs anticipate index fund rebalancing.  A dozen years ago, nearly 30% of the value of the Canadian TSX index sat in one stock (Nortel), so that one had to decide whether to track with or without Nortel. Similar issues arose with Apple’s dominance in the Nasdaq-100 index, which required the index to be rebalanced in 2011.

Even large institutional investors, such as the California Public Employees’ Retirement System (CalPERS), the largest public employee retirement fund in the U.S., is now moving to a greater level of indexed investments. The fund currently has 35% in “passive” investments, and that fraction is slated to rise.

In the wake of this new-found popularity of index-linked investment, a wave of new exchange-traded funds (ETFs) has hit the market, many of them based on newly minted indices. Whereas in 2000, approximately 50 new ETFs and the same number of indices were introduced, in 2011 those figures had risen to nearly 300 and 250, respectively. In July 2012, there were over 1400 U.S.-listed ETFs tracking over 1000 indices, according to a Vanguard report.

How can there be so many indices? In part, there are so many indices because the conventional market-capitalization index schemes used, for example, in the S&P500 index, have been augmented with other approaches: price-based weighting, characteristic-based weighting, and other schemes (with “other” now the most common new index approach). What’s more, in recent years the time gap between when an index is introduced and when it is adopted by a commercial index fund or ETF has been dropping, from three years in 2000 to 77 days in 2011, thus suggesting that the index creation is linked to the establishment of the new ETF.

This evidence suggests to us that the creation and proliferation of new indexed ETFs may be susceptible to statistical overfitting, in the same way that the creation of new investment strategies are susceptible to overfitting, as we documented in our recent study Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance, and for a similar reason: investors are not clearly told how many different variations of an index were tried but then discarded, before one that appeared to perform well in recent years was selected.

Indeed, it appears, based on data from a 2012 Vanguard report, that many newly minted index ETFs have not performed as well as they did in backtests done before the release of the ETF. In particular, Vanguard analysts found that the average new index fund outperformed the broad U.S. market by an annualized rate of 10.29% in the 60 months before launch, but then underperformed the broad U.S. market by an annualized rate of -0.99% in the 60 months after launch.

Additionally, the Vanguard report observes, “back-tested performance does not appear to persist, on average, past the live-index date, … possibly because benchmarks are often being chosen for new products based on their attractive past performance history.”

The Vanguard report concludes its analysis by saying

Overall, it appears that index creation activity has been transformed from an exercise of providing investable benchmarks for different asset-class segments to one of providing ETFs a way to market and promote new products with ready-made indexes that might jump-start the acceptance and viability of new offerings.

Comments are closed.