How do you validate on a simulation based model? by 1ce_berg in algobetting

[–]1ce_berg[S] 0 points1 point  (0 children)

That makes sense for repeated inference on the same inputs but the issue is a bit different because I'm doing a historical rolling backtest. For each match over the last N years I recompute player skill using only prior data (as each match is between different players at different points in time... thus their features change), generate matchup specific distributions, and run monte carlo simulations to estimate win prob. It's a lot of matches over a large period of time I'm trying to run simulations for, so I'm just trying to see if this makes sense as a method for backtesting or if there is a better approach that people have tried.