The importance of actively managed mutual funds in the financial sector has led to substantial research focused on their performance. The overwhelming evidence is that it’s very difficult, if not impossible, to identify ahead of time the shrinking percentage of active managers who will outperform in the future. Among the reasons for the difficult nature of this task is that so few managers succeed, and that it’s difficult to separate skill from luck.

Consider the findings from an August 2016 research paper, “Mutual Fund Performance through a Five-Factor Lens,” by Philipp Meyer-Brauns of Dimensional Fund Advisors (DFA). His sample contained 3,870 active mutual funds over the 32-year period from 1984 to 2015.

Benchmarking the active funds’ returns against the new Fama-French five-factor model (which adds profitability and investment to beta, size and value), Meyer-Brauns found an average negative monthly alpha of -0.06 % (with a t-statistic, a measure of statistical significance, of 2.3). He also found that about 2.4 % of the active funds had alpha t-statistics of 2 or greater, which is slightly less than what we would expect from chance (2.9 %).

He concluded: “There is strong evidence that the vast majority of active managers are unable to produce excess returns that cover their costs.” He added that “funds do about as well as would be expected from extremely lucky funds in a zero-alpha world. This means that ex-ante, investors could not have expected any outperformance from these top performers.”

Meyer-Brauns’ findings are consistent with the overwhelming evidence that, when it comes to active managers, past performance is not predictive of future results. For example, studies on Morningstar’s star rating system have found that, while the lowest-ranked mutual funds continue to underperform (partly because they have high expense ratios) and remain at only one star, five-star funds don’t continue to outperform and their future returns are not statistically different than lower-ranked three- and four-star funds. Despite compelling evidence that the star rating system has no predictive value, it still exists as a measure of mutual fund ability, and mutual fund flows are strongly impacted by changes in the ratings — upgrades lead to strong inflows and downgrades to large outflows.

Timothy Marlo and Jeffrey Stark contribute to the literature with their September 2016 study, Ability to Capture Up Market Returns and Avoid Down Market Losses: The Upside and Downside Capture Ratios. In addition to its star rating system, Morningstar provides two other measures of fund performance: upside and downside capture ratios. Marlo and Stark used these metrics to determine whether mutual fund performance manifests itself in a fund’s ability to outperform in up markets or to reduce underperformance in down markets.

Their study covered the period from 2003 through 2015. The following is a summary of their findings:

There’s very little evidence that upside and downside capture ratios predict future fund performance.

• The upside capture ratio fails to provide any information on subsequent four-factor (market beta, size, value and momentum) alpha performance.

• The downside capture ratio also fails to provide information on future performance.

• Funds do not successfully outperform in up markets and minimize risks in down markets.

• Funds with a large upside capture ratio have significantly negative performances in the next down market, and funds with low downside capture ratios have significantly lower returns over the next up market — highlighting the risks of using capture ratios as a way to make investment decisions.

• Even funds with the ideal relationship between their upside and downside capture ratios fail to outperform in subsequent months, lending further support to the finding that mutual funds do not possess the ability to generate large returns in an up market and minimize downside risk in declining markets.

Despite the inability of the upside and downside capture ratios to explain future mutual fund performance, Morningstar continues to provide them to investors, and there’s a significant association with both ratios and subsequent fund flows. Specifically, Marlo and Stark found a stronger response to the upside capture ratio if the current market state is up, and a stronger response to the downside capture ratio if the current market state is down.

In the end, Marlo and Stark concluded: “The results of this analysis suggest that investors display a strong response to both Upside and Downside Capture Ratios when deciding which mutual funds to allocate new money to.” Again, we see that investors continue to base their investment decisions on measures of fund performance that provide no information regarding fund ability.

Sadly, many investors are engaged in what Albert Einstein described as the definition of insanity: doing the same thing over and over again and expecting a different outcome. But, now that you have the evidence, hopefully you’ll avoid this mistake.