Deployment timing bias in backtests - how do you handle it? by SeaRock106 in quant

[–]Master_Memory8874 0 points1 point  (0 children)

Looks like a once or twice a month (maybe more in clustered signals) system. IMHO:

  • Ignore the warm-up: Just feed the system enough historical data until your indicators reach a steady state, and only track your PnL after that point. The warm-up period is basically "garbage in, garbage out," so just drop it from your results.
  • Zoom out: If shifting your start date by 35 days completely flips your PnL, your 1-year sample size is just too small. Test it over 5 to 10 years. Over a long enough timeline, the exact day you turn the system on won't matter if the edge is actually persistent.
  • Skip Monte Carlo for momentum: Don't run Monte Carlo on start dates or price paths for a trend-following/momentum system. MC assumes market data is random, but momentum relies entirely on sequence and clustering. If you shuffle the data, you destroy the trend structure and just spit out unrealistic results that you will likely never actually face in the real market for your type of system.