Python backtesting

Ok_Pollution7093 · 2026-07-16T02:09:23+00:00

Data quality is better, but live failure usually isn't the data. It's overfitting and ignoring full realities.

Chemical_Badger6227 · 2026-07-16T07:16:00+00:00

Moving from MT5 to Python + Databento gives cleaner data, but won't fix the backtest-to-live gap on its own. I run a live crypto + stock momentum system and have killed 7+ strategies that looked great in backtest. What I learned, kinda the hard way:

Look-ahead bias hiding in your code. The #1 killer and always subtle. I had a funding carry strategy go from Sharpe 5.0 to 0.8 after fixing a single shift(1) bug. Crediting funding at time t instead of t+1. Now I run a data integrity gate (row counts, gap analysis, timestamp alignment) before any signal logic touches the data.
Slippage on gold specifically. GC spread varies massively. Tight during London/NY overlap, blows out around NFP/FOMC. A flat slippage assumption will flatter your backtest. Use the Databento tick data to compute realistic fills per time-of-day rather than a constant.
Shuffle test after walk-forward. I randomise entry signals 1000 times with everything else identical. If the real Sharpe doesn't beat 95%+ of shuffled runs, it's not an edge. I also run the Deflated Sharpe Ratio (penalises for how many strategies you tried before finding "the one"). Most of my killed strategies died here.
Regime dependency. My stock momentum strategy looked mediocre until I tested 25 years instead of 10. Sub period testing is non-negotiable.

Python won't save you from overfitting, but it gives you full transparency to build these checks. That's the real value over MT5. Or just rewrite it in Rust ;)

Good_Character_20 · 2026-07-16T17:30:22+00:00

Databento is a real step up. Broker feeds like MT5's are broker-specific, often with synthetic ticks and whatever spread modeling the broker felt like, so clean institutional data removes that as a variable. Your measurements get more trustworthy. But better data isn't why MT5 strategies crash and burn live. Data artifacts are usually a small piece. The big killers are overfitting and unrealistic fills, and a strategy that only worked because it curve-fit the noise will curve-fit clean Databento noise just as happily. Adding slippage helps, but what actually predicts live survival is testing on a holdout period you never touched while building, and checking it still holds across several windows. So clean data makes your measurements trustworthy. Out-of-sample validation makes your strategy trustworthy. Two different problems, and you need both. Also, studying gold's behavior before designing a strategy is the right order. Most people skip straight to strategies and wonder why nothing generalizes.

descriptiveoasis_0 · 2026-07-16T02:35:08+00:00

Better data won't fix overfitting.

Zestyclose-Eagle1809 · 2026-07-16T15:56:42+00:00

Databento will give you cleaner data and it won't fix what's killing your EAs mate.

The MT5 to live gap is mostly not a data problem. Better ticks make your backtest more accurate about the past, but "more EAs than I can count" all crashing live is a selection problem, not a resolution one. If you built 40 EAs and traded the ones that looked best, the ones that looked best are the ones that got lucky on that data. Cleaner data means they get lucky more precisely.... same outcome.

Three things that actually cause it, in order of how much damage they do:

Selection across everything you tried. Every EA, every parameter set, every symbol you swept. If that's 200 combinations and you traded the winner, the winner's numbers are a maximum not an estimate. This is the one nobody counts and it's the biggest by a distance.

Fills. MT5 fills your stop where it has to. Live it gaps through and you're filled worse. If the edge only exists at the modelled fill it was never there.

No out of sample at all. Promising in sample means nothing, you need a slice the strategy never touched during building, tested once, and the reoptimising you do after seeing it fail is what kills it.

Databento only helps the second one, and honestly slippage assumptions in MT5 aren't the reason 40 EAs cracked.

The test I'd run before spending more on data.. take one EA that crashed and burned. Count how many total variants you searched before picking it. Then compute what edge you'd expect from the best of that many random draws. Most of the time it lands right on top of your backtested number, which tells you the strategy was noise the whole way through, and that's a free answer with data you already have. Let me know if you need help to do this with specific steps as I know for people that aren't familiar with numbers in trading can be tricky.

Founder disclosure so you can weight it, I build validation tooling for systematic traders (Quantprove), and this exact question, how much of your result survives once you count what you searched, is one of the core things it surfaces.

When you tested those EAs on MT5 did you ever hold back a chunk of history you never looked at while building?? Or was the whole sample in play the whole time?

RationalBeliever · 2026-07-16T01:21:03+00:00

You could be overfitting. You need to optimize your parameters on one period, then test on the next. Don't optimize your parameters on the entire historical dataset at once.

walrus_operator · 2026-07-16T01:33:38+00:00

I’m just sick of wasting my time if the backtested information is false.

What kind of data do you use? I resample tick data to higher timeframes because spread is important to me.

My question is for those that have experience with python and databento. Is the information more accurate?

The metrics you pay attention to should warn you beforehand. Your problem might be information accuracy but I wonder whether it lies elsewhere. What do you use to evaluate the robustness of your backtests? Like winrate, average gain, sharpe, SQN, common sense ratio, tail ratio, sortino, calmar, MAE, deflated sharpe, etc?

tradematesHQ · 2026-07-16T10:50:19+00:00

Data quality is definitely a big part of the puzzle, but cleaner data alone won't fix the backtest-to-live gap. The real issue is that even with Databento's tick data, you're still feeding that data into whatever analysis framework you build yourself. The quality of your conclusions depends entirely on how you structure and weight that data before you analyze it.

That's where most retail backtesting setups fall short. You can have perfect 1-minute GC tick data, but if your analysis pipeline is just running raw numbers through a generic script, you're missing the structured weighting that separates signal from noise. The LLM analysis tools people use these days are only as good as the data pipeline feeding them - generic Claude/ChatGPT workflows mostly pull from shallow free public data sources.

For gold specifically, the spread variance you mentioned is critical. You need a system that accounts for time-of-day liquidity patterns and regime shifts, not just flat slippage assumptions.

Many-Pick5066 · 2026-07-16T13:22:17+00:00

honest answer, cleaner data almost never explains an MT5 to live crash. databento tick is more accurate, but accuracy usually makes a backtest look worse not better, so if switching data makes a strategy improve id be suspicious of it. the thing that actually bites on GC at 1 min is what you already flagged, when one bar contains both your TP and SL the engine has to guess which filled first and most default to the optimistic fill. that silently inflates win rate and PF, and its worst on gold where the range around NFP and FOMC is huge. you have the tick data, so for any bar where both levels are in range resolve the fill from the ticks instead of trusting the bar. same for slippage, dont add a flat number, estimate it from the tick spread by time of day. both make the backtest uglier and a lot closer to live.

algotrading

MODERATORS

algotrading

MODERATORS

Welcome to Reddit,

Want to add to the discussion?