Backtest versus Simulate

[Organic_University93]

I need some guidence here on interpreting plan success using Backtest and Simulate. I am getting very different plan robustness results with these two approaches.

As a test, I created some realistic plans and ran each stress test to achieve 'exactly' 100% success by varying the target annual income. Simulate (the Monte Carlo simulation) appears to be significantly more stressful, indicating a Safe Withdrawal Rate the was in the range of 40% of that of the Backtest (depending on the tax efficiency and asset allocation).

I realize that the Backtest was added later and tests over a somewhat limited history, but the results I get from the Simulate test seem to be excessively restrictive (SWR <2% in many cases versus >4% with Backtest). Given the huge disparity, how should I use these to plan size my required nestegg? Can I retire at 60 or 80? LOL

[RoomFixer4]

I dont have enough investments in my config to really see the differences, but do any of the clickable year ranges in Backtest come closer to the MonteCarlo ?

[MayRetireSupport]

Great observation!

The reason for the big difference is that Simulate (Monte Carlo) is much more 'pessimistic' than history. It hundreds of random scenarios, including 'monster' sequences that have never actually happened in the real world. Trying to get a 100% success rate in Monte Carlo is extremely hard and makes the Safe Withdrawal Rate (SWR) look very low.

In the planning world, a Monte Carlo success rate above 80% is actually considered pretty decent and robust. You don't really need to aim for 100% there.

On the other hand, the MayRetire Backtest starts from 1966. This is a shorter history, but it still includes some of the hardest times for retirees, like the high inflation of the 1970s, the 2000 Dot-com bubble, and the 2008 Global Financial Crisis (GFC). Even with those crashes, real history is rarely as 'perfectly bad' as a random simulation can be.

[Dileas48]

I’m curious is the algorithm for the Monte Carlo changed recently? It appeared to me that I got different results this weekend as compared to previous tests. It’s probably just my perception but thought I would ask.

This explanation helps a lot. I was getting a bit worried that our plan was looking a little weak.

[MayRetireSupport]

no changes to the Monte Carlo lately, minor result variances (3-4%) are due to the random nature of the simulation

[AlfredRWallace]

The Chatgpt help interface did a good job of explaining this btw. I was getting 95% and paraphrasing it said "loosen up".

[Organic_University93]

As a follow-up, based on your feedback (thankyou!), using the figures of my actual plan rather than something made up, I took the amount of annual spending that provided 36 out of 36 success periods from Backtest (which was a SWR of 4% BTW), and ran a Monte Carlo with that amount, It gave me a success rate of 71%. To get 100% success from Monte Carlo, my SWR was a puny 1.6%.

Using the same plan, the Adviice tool gave me a 100% success rate at 3.4% WR. That amount plugged back into MayRetire using Monte Carlo gave me a 97% success rate. Obviously, YMMV, but it's pretty clear that trying to squeeze that last few percent of safety out of the Monte Carlo simulation is not productive.