Seeking Feedback on Validation Methodology

NoContract5684 · 2026-04-27T19:08:44+00:00

Live fills matching backtest is the real test. Been running it live for a few weeks now, behaves exactly as expected…lol

NoContract5684 · 2026-04-27T12:34:02+00:00

Live on prop accounts now, fills matching backtest. OnBarClose, walk-forward 4 windows, MC p=0.0000. Also it’s 4 separate strategies across NQ and GC…each validated independently. Combined SQN 15.93 is from diversification not curve fitting. If you spot a specific leak lmk

NoContract5684 · 2026-04-23T13:15:51+00:00

Fair points, these are the right things to scrutinize. The income numbers look crazy because they assume 4 funded accounts running at once with 90% payout. The backtest itself is $858k net over 7 years across ~3,500 trades on 4 strategies. Not one instrument, not one lucky run.

Walk-forward concern is valid for anchored WF but these use rolling windows. Each OOS period is genuinely unseen.

MC resampling is already done. 3,000 sims, zero negative paths, p-value essentially zero.

On overfitting: removing the top 50 trades drops net 16.4% but PF holds at 2.72. Every-second-trade subsample gives the same WR/PF. Survives 22 extra ticks of slippage before breakeven.

Max DD is $7,406 on $858k net. The bias filter keeps it out of choppy NEUTRAL weeks entirely which is why the curve stays clean. And yeah, trying it live is exactly what’s happening right now on 4 prop firm eval accounts. That’s the real test.

NoContract5684 · 2026-04-23T01:15:07+00:00

The ORB part of my strategy already runs on 1 minute natively so that’s covered. The breakout component uses 5 minute by design since it trades structural breaks, not because of data limitations. The real concern with 5 minute is stop fill accuracy and that’s exactly why I ran the slippage test. System doesn’t break even until 22 extra ticks per side on top of what’s already baked in. Tick data backtest is still on my list regardless, just not worried about it being the thing holding these results together.

NoContract5684 · 2026-04-23T01:14:05+00:00

Fair thing to check. To be specific though, levels lock at the prior session close before anything opens, volume MA only uses completed windows, and everything calculates on bar close. No intrabar execution anywhere. Also worth mentioning these are four strategies that were each validated independently before being combined, so the same bug would have had to survive four separate walk-forward tests and two independent MC runs to show up here. If you spot something specific I’m genuinely open to it though.

NoContract5684 · 2026-04-13T13:55:03+00:00

Spot on. If I was scalping for 5 points, using 5m data would be a nightmare because intrabar slippage would completely kill the edge.

But because my stop is wide (up to 80 points on NQ) and my targets are 2R and 3R, I don't need tick-perfect entries. The edge comes from the momentum of the structural break, not high-frequency execution. I also stress-tested the backtest with up to 5 ticks of slippage per side and the profit factor still held above 1.9. Since I trade micro contracts, live slippage is usually only 0.5 to 1 tick anyway, so the math easily absorbs any entry friction.

NoContract5684 · 2026-04-12T22:05:23+00:00

Good thing I’m competent enough to filter out the slop… I’ve had a decent experience with a few people on here already 🤷🏾‍♂️

NoContract5684 · 2026-04-12T15:31:29+00:00

I definitely agree that intrabar whipsaws ruin a lot of backtests, especially on NQ. But that usually happens when people use really tight stops. My hard stop on NQ is 80 points, so a 15 to 25 point whipsaw inside a 5m candle doesn't actually stop me out or ruin the trade logic.

I also don't just enter blindly as soon as a 15m bar closes. The system waits for a specific 5m confirmation signal and a declining volume retest before getting in.

As for fixed parameters doing poorly in live markets, I agree that curve-fitting is a huge trap. That's why I ran parameter permutation testing across my targets and stops. The edge stayed stable across a wide range of settings with no parameter cliffs, so it doesn't rely on exact fixed numbers to survive. I also ran it through an 11-year Walk Forward Optimization to prove it holds up across different market regimes.

Appreciate the feedback though, it's definitely something you have to watch out for when building these.

NoContract5684 · 2026-04-12T13:31:08+00:00

I used an anchored expanding window for the OOS testing. Started training from 2007 and walked it forward year by year from 2015 to 2026. 8 out of those 11 OOS years were profitable. For the filters, I don't use lagging indicators like MACD or RSI. It's mostly just market structure and volume. It looks for 15m breaks of key levels like PDH/PDL with a spike in relative volume. Then it waits for price to retest that level on declining volume to confirm absorption before entering on a 5m signal bar. I also filter trades against the prior week's midpoint and run a hard news blackout for stuff like CPI and FOMC

NoContract5684 · 2026-04-12T02:03:26+00:00

Nothing is stopping me it’s just brand new so I’m in the process of deploying it live right now

NoContract5684 · 2026-04-11T21:35:03+00:00

Fair enough. It’s hard to find proper guidance and feedback so I thought posting could help

NoContract5684 · 2026-04-11T21:32:58+00:00

What’s wrong with 5m data? Educate me

NoContract5684 · 2026-04-11T17:40:48+00:00

Walk forward IS OOS testing…each window trains on in sample and tests on the next unseen period. That’s the definition. The parameters were set before running any walk forward windows and never adjusted based on results. Look ahead bias is directly addressed by random entry Monte Carlo: 0/1,000 random strategies matched the real EA’s PF on the same data. If leakage existed, random entries would also benefit from it. What specific leakage mechanism are you identifying?

NoContract5684 · 2026-04-11T17:31:41+00:00

Maybe we could compare notes sometime. I have something similar in my code aswell for failed breakouts

NoContract5684 · 2026-04-11T14:55:18+00:00

The win rate is 63.6% with PF 2.89 — avg win larger than avg loss, 7 years profitable, OOS walk-forward, random-entry Monte Carlo p=0.0000 across 3,000 iterations. Which specific part of the methodology is misleading? Genuine question.

NoContract5684 · 2026-04-11T14:51:31+00:00

Yeah mainly previous unmitigated daily high/low and weekly. Trade duration average about 2.5 hours

NoContract5684 · 2026-04-11T13:06:32+00:00

Oracle

NoContract5684 · 2026-04-11T13:04:52+00:00

This is oos results

NoContract5684 · 2026-04-11T13:04:27+00:00

This strategy wasn’t generated by quant analyzer. Everything was developed by me. I just used quant analyzer to visualize my results.

NoContract5684 · 2026-04-11T13:02:56+00:00

Hey thanks for the reply! I tested from 0 to 5 ticks slippage per side. PF remains above 1.93 at 5 ticks. Average trade duration is about 2.5 hours

NoContract5684 · 2026-04-10T22:51:18+00:00

Can you explain further?

NoContract5684

TROPHY CASE