A cura di Elisabetta Basilico, PhD, CFA
Quantifying Backtest Overfitting in Alternative Beta Strategies
- Annti Suhonen, Matthias Lennkh, and Fabrice Perez
- Journal of Portfolio Management, Winter 2017
- A version of this paper can be found here
- Want to read our summaries of academic finance papers? Check out our Academic Research Insight category.
What are the Research Questions?
- Does persistence (out of sample performance) exist for alternative beta strategies sponsored by investment banks?
- Does adding complexity to a strategy increase the risk of backtesting overfitting?
- Do the strategies capture the factor exposure they seek to exploit? And, does the exposure remain consistent between backtesting and live periods?
What are the Academic Insights?
By analyzing a proprietary dataset of around 215 individual strategy indices across 17 sponsoring investment banks with both backtest and live data (the minimum length of the backtest is 3.3 years while the average is 10.7 years; the minimum length for the live period is 0.44 years while the average is 4.6 years), the authors find the following:
- Most of the strategies exhibit a substantial reduction in Sharpe ratios (SR) in the live period. This reduction is due to a falling in excess returns. The haircut is at 50% or more for all asset classes and at 60% or more for all strategy groups. Only 18 of 215 strategies ( or 8.4% of the sample) had a live SR equal to or greater than the backtested one. Also, 65 strategies (or 30% of the sample) had a negative SR during the live period. (similar to the analysis in “Chasing Returns and Avoiding Spaghetti Against the Wall Fund Companies“)
- YES, more complex trading strategies are more liable to overfitting, and deteriorated performance. After controlling for fixed effects, the most complex strategies suffer from an SR haircut that this more than 30% compared to simpler strategies. (see here for the reasons why)
- The equity value strategies display a positive and significant exposure to the value factor, which becomes insignificant and negative in the live period. Differently, the equity volatility strategy, the fixed income curve strategy, and FX carry strategy show consistent exposures both in the backtest and live periods.
Why does it matter?
This study contributes to an emerging body of research on backtest overfitting in alternative beta strategies. In fact, it focuses on a very particular dataset: strategies sold by investment banks. The findings highlight the importance of due diligence on quantitative strategies (which we also highlighted in the past here) and suggest that backtested performance need to be taken with a grain of salt when selecting a strategy. The authors conclude by suggesting two further readings: Principal for Financial Benchmarks (IOSCO, 2013) and Financial Benchmarks: Thematic Review of Oversights and Control (FCA, 2015).
The Most Important Chart from the Paper:
Continua a leggere su AlphaArchitect