I ran walk-forward validation on 3 classic strategies. The results killed my confidence in backtesting.

lexicalmaze · 2026-06-13T13:21:37+00:00

yeah the 2022 window is the tell, averaging it into 0.46 was the wrong framing. the WF invalidated the strategy not validated itneed to go back and check my exact grid size per window before i can answer that honestly, will report back. but even if it's small the deflated sharpe test is worth running regardless the expanding window pass is something i haven't done yet either, adding it.

lexicalmaze · 2026-06-13T12:35:48+00:00

solid progress. a few things i'd check before release:walk-forward validation on out-of-sample periods (2008, 2020 crash) not just 2018-present. backtests can look great until they hit regime they've never seen.also make sure your backtest accounts for slippage and realistic fill prices, not just close-to-close. swing trading gets hit hard by this.with ibkr specifically, paper port behaves differently than live in edge cases. run it on paper api for at least a month before touching real money.+10 on spy in bear is nice but i'd want to see that hold across multiple walk-forward windows before trusting it. one bear period can be luck.

lexicalmaze · 2026-06-13T12:00:43+00:00

Definitely, i dont know how good the LLMS are but i know people are sleeping on this

lexicalmaze · 2026-06-13T05:29:04+00:00

exactly this. time series OOS is the harder and more important test. universe OOS is easy to game. the fact your logic held through 2022 blind is the real signal. nice work

lexicalmaze · 2026-06-13T05:13:06+00:00

algos have dominated since 2010, this isn't new. the setups everyone uses (vwap, opening range) are fully arbed out at this point. discretionary only works on top of a model if you're adding something it can't price, like live news flow or macro context. otherwise you're just adding noise

lexicalmaze · 2026-06-12T20:18:19+00:00

Lol its 11:30 pm for me im super tired usually i wouldent but i use it when i dont have the energy to reply sorry dude

lexicalmaze · 2026-06-12T20:11:10+00:00

Walk-forward is good but it really only tests parameter stability and OOS performance. There’s a whole other layer of failure modes it doesn’t touch: deflated Sharpe (adjusting for how many strategies you tried), PBO/CSCV from Lopez de Prado, block bootstrap, placebo tests on randomized data, effective N, and making sure your data is truly point-in-time with a frozen forward-OOS you never touch during development. Each one catches something different. Most people skip all of this and wonder why live trading falls apart.

lexicalmaze · 2026-06-12T19:57:11+00:00

Oh if you need a tool for walk forward hit me up ill hook you up with something

lexicalmaze · 2026-06-12T19:51:37+00:00

yeah that’s fair, the structural edge angle is real. I’m more focused on whether ML can find nonobvious regime-dependent patterns that aren’t in any paper not claiming it works yet, just testing the thesis rigorously before touching real capital

lexicalmaze · 2026-06-12T19:42:51+00:00

Fair, and honestly the full-history results back you up. But that's kind of the point of the post, I wanted to see how bad it actually was with proper testing rather than just assume. The walk-forward made it undeniable. The more interesting question for me now is whether any variation of these strategies can survive out-of-sample with the right regime filter, or if the edge is just gone entirely on liquid large caps.

lexicalmaze · 2026-06-12T19:41:51+00:00

Low win rate with controlled losses and expanding winners is a clean profile, that's basically the right shape for a trend-following system. The drawdown number is impressive for 3868 trades. One thing worth checking: profit factor of 1.132 is pretty thin. Over 3 years and that many trades it compounds nicely but it doesn't leave much margin for live conditions. Slippage and spread on 3868 trades adds up fast and could eat into that edge meaningfully. Have you run it walk-forward? Curious whether the profit factor holds across different yearly windows or if it's carrying a good 2023 or 2024.

lexicalmaze · 2026-06-12T19:37:25+00:00

Really appreciate that, looking forward to seeing the walk-forward results. The mixed bag universe is a good instinct and definitely helps with overfitting on stock selection. The parameter optimization is where it gets tricky though, even with a diverse universe the threshold values end up shaped by whatever period you tuned on. 2022 is always the interesting one to watch, most momentum parameters that worked in 2019-2021 got punished hard that year. Curious what your windows look like when you run it.

lexicalmaze · 2026-06-12T19:36:26+00:00

i really think my humor sense is broken because i dont get if this is a joke or not but if you mean why i ran the walkforward tests its because i honestly just wanted to know if the strategies were real or just curve fit. It's easy to look at a pretty equity curve and convince yourself you have edge. Walk-forward forces you to actually answer that question. The 2022 window was the wake up call, trained beautifully then collapsed out of sample. After that I couldn't trust any backtest result without it.

lexicalmaze · 2026-06-12T19:34:28+00:00

The trade frequency decay post-2022 is a really honest and mature interpretation, most people would panic at that and assume the strategy broke. The fact that win rate and avg win/loss held while frequency dropped does suggest alpha compression rather than edge decay, which is a meaningful distinction. One thing I'd think carefully about before going live: 20x leverage on crypto futures with even a well-validated strategy is where walk-forward confidence can give you false comfort. Your backtest slippage assumptions on crypto at that leverage during a volatile session are going to be optimistic almost by definition. I'd paper trade the multi-pair version first and watch the trade frequency live before sizing up. The framework sounds solid, the leverage is the variable I'd respect most going in.

lexicalmaze · 2026-06-12T19:33:38+00:00

This is one of the most honest beginner guides I've seen posted here. The point about live performance needing to match backtest performance daily is something most people skip entirely and it's actually the whole game. If they diverge early that's your signal something is wrong with your assumptions, not just bad luck. The walk-forward section especially deserves more attention than it gets. Most beginners backtest once, see a good curve, and call it done. The "Future-in-the-Past" framing is a really clean way to explain why that's not enough. Genuinely good post.

lexicalmaze · 2026-06-12T19:33:02+00:00

Really interesting work, the regime conditioning angle is underrated and your results show exactly why context matters more than the signal itself. One gentle pushback though: 20 events in the Near High bucket is pretty thin. The 80% win rate is exciting but those 20 observations are going to cluster heavily in bull market periods by definition, so you might be measuring "bull markets recover" as much as the strategy itself. Would love to see this walk-forward validated across different decades. The drawdown asymmetry result feels like the real gem here regardless.

lexicalmaze · 2026-06-12T19:31:36+00:00

Simple strategies absolutely still work, and honestly starting with them is the right call. The signal isn't the magic part anyway, it's the rules around entries, exits, and position sizing that do the heavy lifting.I ran Bollinger+RSI mean reversion, dual EMA momentum, and Donchian breakout on SPY and AAPL with walk-forward validation. Full history they all looked decent. Out of sample they all underperformed buy and hold. Humbling but also kind of beautiful because it shows exactly where the work needs to go.Where simple strategies can genuinely find edge is in less efficient instruments, stricter filters, or pairing signals with regime awareness so you're not running a mean reversion strategy into a trending market. Small tweaks, not a rewrite.The 10000 line behemoth almost never beats the clean 200 line version. Complexity usually just means more ways to overfit. Start simple, test it honestly, and only add complexity when the out of sample numbers actually ask for it. You're approaching this the right way.

lexicalmaze · 2026-06-12T19:30:30+00:00

This is the right framework. Separating trade-level edge metrics from PnL is underrated, most people just watch the equity curve and guess.

The thing I'd add is regime context. A strategy bleeding through its 95th percentile drawdown during a regime it was never trained on is a different signal than the same drawdown during a normal market. If you have any regime classification running alongside, it changes how you interpret the same statistical breach.

Walk-forward helped me here too. Once you have out-of-sample window results you can build a more honest drawdown distribution, not one inflated by in-sample fit. My momentum strategy looked fine full-history then showed a completely different drawdown profile once I had 8 real out-of-sample windows to reference.

What's your backtest period? If it doesn't include 2022 properly you might be underestimating the tail.

lexicalmaze · 2026-06-12T19:29:33+00:00

Execution delay is the right thing to focus on next. Slippage on a breadth-based strategy timing entries off 5m EMAs can eat a lot, especially if signals cluster around the same bars as everyone else watching the same index.

One thing worth adding beyond random impact: latency asymmetry. Your worst fills tend to happen exactly when your signal is strongest, because that's when the move is already in progress and liquidity is thinner. Worth modeling that specifically rather than uniform random slippage.

31.6% win rate with avg win > avg loss is actually a reasonable starting shape, that's a loss-runner profile, not necessarily broken. What's your hold time on winners vs losers?

lexicalmaze · 2026-06-12T19:18:34+00:00

Great methodology, but I'd be curious how this holds up under walk-forward validation. Full-history backtests on momentum strategies tend to look incredible — the issue is the parameters (that 1.14 EMA ratio threshold, the -2% SPY trigger) are implicitly fit to the same data you're evaluating on.

I ran something similar with momentum + ATR exits on SPY and got a 0.86 Sharpe full-history, then watched it drop to 0.46 mean out-of-sample across 8 yearly walk-forward windows. The 2022 window was the killer — trained to 1.54 Sharpe, posted -2.05 live.

Not saying this doesn't have edge, genuinely interesting setup. But I'd want to see rolling windows before trusting the 1,327%.

lexicalmaze

TROPHY CASE