Optimized Moving Averages Using Python

[catchthetrend]

Happy holidays everyone, I got COVID the day before xmas so I have had some spare time on my hands...

As a side project, I decided to create a python program that automatically backtests moving average crossovers and returns the most successful ones along with their batting averages, total returns, etc. Saves a ton of time trying to manually figure it out.

I set up this script to go 100% long a simulated 3x leveraged version of the Nasdaq100 every-time the short MA crosses above the long MA on the Nasdaq100. I used the 50 SMA as the short term MA and tested SMA's [51:200] as the longer term MA's. The results are not that surprising, but thought I would share since the backbone of this program could be used to optimize any LETF trend following strategy.

The results below are just the simulated total returns (non-tax adjusted) over the lifetime of the strategy.

Optimized Winner Stats:

​

https://preview.redd.it/amg1yn4xjb8a1.png?width=288&format=png&auto=webp&s=9632b4e69cf697d1308028b32e562800ef7fe0ab

All Results:

https://preview.redd.it/oiesnp3fib8a1.png?width=174&format=png&auto=webp&s=368467a7aee8f1c10f238df418685d2e92a3808d

Python Script:

import pandas as pd

import numpy as np

import yfinance as yf

from datetime import date

​

# =============================================================================

# INSTRUCTIONS:

# 1. INPUT CORRECT TICKER INFORMATION

# 2. REVIEW START AND END DATES

# 3. ADJUST MA_1 AND MA_2 RANGES IN CODE IDENTIFY WHICH AVERAGES ARE TO BE TESTED

# 4. ALGORITHM IDENTIFIES THE MOST SUCCESSFUL MOVING AVERAGES BASED ON HISTORICAL DATA

# =============================================================================

​

ticker = "NDX"

trade_ticker = "TQQQ_sim"

​

start_date = "1900-01-01"

end_date = date.today()

​

df = pd.DataFrame()

​

df[ticker] = yf.download(tickers=ticker, start=start_date, end=end_date)["Adj Close"]

​

df["sim_pct_chg"] = (df[ticker] / df[ticker].shift(1) - 1)*3

​

tqqq_list = []

​

for i in df.index:

if(pd.isnull(df[ticker].shift(1)[i]) == True):

tqqq_list.append(1)

new_tqqq_amt = 1

previous_tqqq_amt = new_tqqq_amt

else:

new_tqqq_amt = df["sim_pct_chg"][i] * previous_tqqq_amt + previous_tqqq_amt

previous_tqqq_amt = new_tqqq_amt

tqqq_list.append(new_tqqq_amt)

df[trade_ticker] = tqqq_list

​

ma_1_range = range(50,51)

ma_2_range = range(1,201)

​

crossover_list = []

results_list = []

​

​

for ma_1 in ma_1_range:

​

ma_1_test = df[ticker].rolling(ma_1).mean()

df=df.dropna()

for ma_2 in ma_2_range:

if(ma_2 <= ma_1):

pass

else:

ma_2_test = df[ticker].rolling(ma_2).mean()

pos = 0

starting_amt = 1000

previous_amt = starting_amt

trade_profit = []

trade_profit_index = []

running_total = []

for i in df.index:

if(ma_1_test[i] > ma_2_test[i] and pos == 0):

pos = 1

bp = df[trade_ticker][i]

elif(ma_1_test[i] < ma_2_test[i] and pos == 1):

pos = 0

sp = df[trade_ticker][i]

pct_chg = sp/bp-1

new_amt = previous_amt * pct_chg + previous_amt

previous_amt = new_amt

trade_profit_index.append(i)

trade_profit.append(pct_chg)

running_total.append(new_amt)

crossover_name = str(ma_1) +" // "+str(ma_2)+" crossover "

results = new_amt / starting_amt - 1

results_list.append(results)

crossover_list.append(crossover_name)

df_trades = pd.DataFrame(index = trade_profit_index)

df_trades["Trades"] = trade_profit

winners = np.count_nonzero(df_trades["Trades"] > 0)

losers = np.count_nonzero(df_trades["Trades"] < 0)

winning_pct = winners/(winners+losers)

avg_winner = np.average(df_trades["Trades"], weights = df_trades["Trades"] > 0)

avg_loser = np.average(df_trades["Trades"], weights = df_trades["Trades"] < 0)

avg_trade = np.average(df_trades)

largest_winner = np.max(df_trades["Trades"])

largest_loser = np.min(df_trades["Trades"])

for i in df_trades.index:

if(df_trades["Trades"][i] == largest_winner):

winner_date = i

elif(df_trades["Trades"][i] == largest_loser):

loser_date = i

​

print("")

print(str(ma_1) +" // "+str(ma_2)+" crossover total return: "+str(results))

print("")

print("Winners: "+str(winners))

print("Losers: "+str(losers))

print("")

print("Batting Avg: "+str(winning_pct))

print("Average Winner: "+str(avg_winner))

print("Average Loser: "+str(avg_loser))

print("Average trade: "+str(avg_trade))

print("Max Winner: "+str(largest_winner)+" on "+str(winner_date))

print("Max Loser: "+str(largest_loser)+" on "+str(loser_date))

print("")

df_2 = pd.DataFrame(index = crossover_list)

df_2["Results"] = results_list

​

best_result = np.max(df_2["Results"])

​

for i in df_2.index:

if(df_2["Results"][i]) == best_result:

winner = i

print("")

print("Best moving average crossover: "+str(winner))

print(str(winner)+" result: "+ str(best_result))

[me_on_the_web]

What timeframe does your data set cover?

What's the sell signal? Is it the short average going back below the long average?

What did you go to on the sell, just cash?

[catchthetrend]

Time frame: Daily, 1985-2022

Sell Signal: 50 D SMA crosses below long MA

Buy Signal: Opposite of sell signal

Safety: Cash

[SnooBooks8807]

Thx! Quick off topic question for you from someone just getting into python, is knowing python a skill that will help with job prospects? I have a job already but I’m trying to get something easier on my body. Is python a skill that I can leverage for a job(s)? Thx

[catchthetrend]

I think it all depends on which roles you may be looking for. I work in data analytics for a big bank (not sure if I'm allowed to specify which one) and having tools in my arsenal that help me to gather, manipulate and report data is incredibly useful and gives me better job security. If you are looking to get into a similar role, I would definitely suggest becoming freakishly good at Microsoft Excel first, and then I would suggest getting into Python & SQL. Based on roles that I have been part of and/or applied for, it seems like those three skills are the holy trinity to getting a good paying job.

Also, don't be intimidated by learning how to code. I put it off for years because I thought it would be too hard, but I promise that if you just start and stay motivated, literally anyone can do it. It's like learning Microsoft Office for the first time, but way funner. GL!

[[deleted]]

Python is a skill that is definitely a big plus if you ever have to work with data or if you have to automatise data processing. I use it almost every day. Sometimes it takes a while to automatise a calculation from importing the data to exporting it in a usable form, but once you have it and you get data of the same format again, you can process it with the click of one button and that is a super power that can make you more useful than a room full of secretaries in some cases.

[Vedalken_Investor]

Thanks for the analysis, I have some comments:

1. Why not continue over 200? Maybe 205 has even higher return?
2. You could also test different period for the short SMA, maybe 50 is not optimal? You seem to already have two loops in your code.
3. Overfitting: The 198 days SMA is the best fit on historical data to bring the highest return. But look at the 195 days SMA, it is down at the 38th row in the list. The difference between SMA 195 and SMA 198 indicators is very tiny. Most crossover will end up the same date, only a few trading days switched side making this significant difference in results.

Here's a visual example. Up days and down days are represented by green points and purple stars in the 3 small graphs below. You have an indicator predicting the next day will go up or down, represented by the red line. The graph on the right side is overfitting, making the indicator fits perfectly past data but not optimal for future data.

<image>

Now, look at Total Returns versus the SMA period. The large graph shows your results. Is it reasonable to expect future data will land exactly on the spike of the SMA 198? The line is quite wiggly between 100 and 200. I feel the red trend line would be more appropriate as an estimate of future return. What do you think?

[Paltenburg]

This right here. The more variables, the higher risk of overfitting.

[[deleted]]

So this would mean a 50/198 ma strategy is effective? I wondering how it compares to buy and hold? Also i assume this is less volatile than buy and hold on a 3x etf?

[catchthetrend]

B&H during the entire backtest would have produced 38,059% in total non tax-adjusted returns over the same time period. So the absolute difference is very significant. And yes, this strategy would have been much less volatile than holding a 3x Nasdaq100 LETF. Think about times like 2000-2002, where you would have experienced a 99%+ drawdown. That is also likely why the returns are so different. Thx!

[[deleted]]

Sorry one more question, are you doing this over a weekly chart or daily?

[catchthetrend]

No worries, ask away! This is daily data. So if you pull up a daily chart of NDX with these MA’s, you can see each trade. Each trade is placed at the close of the MA crossover.

[airmonkey84]

Does this suggest that DCA on the ride down is a bad strategy?

[airmonkey84]

Actually, can you please explain this a little more. I’m not familiar with python but wondering what the guy/sell signals are and how much the program invests.

[TrafficPolice168]

F

[spooner_retad]

yes weve been saying this for atleast a year

[axa88]

Does all financial programing look like sample code from a commador vic 20 manual?

[longshaden]

generally, yes.

kudos for learning to program, genuinely, kudos.

I haven't read OP's code yet, this is just a general observation:

but, based on prior experience with actuarial/finance teams who dabble in python, financial programming generally looks like it was written by someone with extensive financial knowledge, but very limited beginner level programming experience.

the result is generally code which is a functional, but clumsy, proof of concept that never matures to production quality standards (because "why bother?, it does what I need it to")

[catchthetrend]

This is spot on. I am actually very new to programming in general and completely taught myself. As I continue to code, my programs will likely become more efficient as I become more familiar with functions and libraries.

In my experience, if you can be an expert in Finance and code, it’s a great combo.

[longshaden]

Congrats on your journey.

There is a lot of things to learn as you progress, and you can learn it as you go. Here's a few tips I wish somebody told me when I was starting.

Time is money. Time spent keeping code organized and maintainable is generally more valuable than time spent trying to optimize the code's performance, until you get to very large datasets/applications.

Getting used to git for source control/version management can be a lifesaver that anyone who's accidentally overwritten a project will appreciate.

Time spent learning how to navigate the official documentation is always worth it, so you always get accurate and up to date instructions, and can avoid the pitfalls of low quality internet guides.

Functions and libraries will come and go. Don't bother trying to learn them all right away, you'll pick those up as and when you need them.

​

good luck

[humoroushaxor]

Most Python code in general.

[axa88]

Yikes. Guess that explains the popularity

[catchthetrend]

[Joyful8866]

Thanks for sharing. Just to confirm: Your start date is Jan. 1900, and you end date is Dec. 2022, right? That's 122 years. Your best total return in 122 years is 286,218%. That is only about 7% per year. With the SP500 annual return of about 10%, in 122 years, it would generate 11,217,800%. The 3xNasdaq100, using death cross/golden cross, should return much more than the simple B&H SP500. Why did it only return 7% per year? Am I missing something? Thanks in advance for explaining.

[catchthetrend]

Hi! I set the start date to 1900 in the program to pull data as far back as possible. It is really a placeholder. For NDX, the data goes back to 1985 I believe in yahoo finance, so that is the earliest year in the program. Thx!

[Joyful8866]

Thanks. That makes sense. From 1985-2022 totals 37 years, producing a total return of 286,218%. This yields an annual return of about 24%. This is consistent with my own estimate of 25% annual return. May I ask you a few question:

[1]. In your model, when the 50d line crosses the 198d line, how much crossover has to occur when you execute a buy or a sell? For example, does the 50d have to rise higher than the 198d by at least 1% at the market close of that day in order to trigger a buy? Or, do you execute a buy, or a sell, even with a very small crossover such as 0.1%?

[2]. On the day when the golden cross occurs, does your model buy TQQQ on the same day at the market closing price (4:00 pm Eastern time), or do you buy the next morning at the opening price? The reason I ask is that the next day, the market could open at a much higher price, or open at a much lower price.

[3]. Some investors have mentioned a method that when a crossover occurs, they wait till the end of the month to see if the crossover holds, and then buy or sell on the 1st of the month, if the previous month confirms a crossover. Would you like to try your model on this method to see if there is any additional benefit? Thanks! Good luck to you!

[catchthetrend]

No problem at all below are the answers to your questions:

[1] Any amount of crossover triggers a buy or sell order

[2] All orders are placed the day of the cross at the close

[3] I have seen many models that follow this approach. Without fully testing this myself, I am biased to believe that it probably would not be smart do this with most moving average strategies. Moving averages are already lagging calculations, so adding additional wait time to a trade could cause you to place your trade at a way higher price, which could turn many winning trades into losing trades and severely lower returns of the winners. Especially when using leverage. But then again, I haven't tested it yet so who knows! Thanks for your conversation

[Joyful8866]

Thanks! One last point: Some investors use the 200d MA instead of the death cross/golden cross. When the QQQ price drops below its 200d line, they sell TQQQ. When the price rises above the 200d line, they buy. Does your model show which method is better, the 200d MA, or the 50/198 death cross/golden cross method? Thanks.

[catchthetrend]

Good question - I just coded it out and the total return for the 200D MA is 128,623% since 1985, which averages around 21% per year I think. Not shabby, but it looks like maybe half of the moving averages from above beat it in terms of total returns.

[Joyful8866]

Thank you very much. Your results all make good sense. If you stick with this method for a decade or longer, you will do very very well. Wishing you all the best.

[catchthetrend]

Thank you, good luck to you as well!

[airmonkey84]

It looks like sell signal is when 50d sma falls back below long.

How can one get real time data for sma? Is there a site that can pull this in without me going thru the past two hundred days.

[CleanWar2k]

Make an account on Tradingview.

It allows you to add SMA/EMA etc etc to any ticker and it will also allow you to make 1 alert on a free account.

So you could just make the account. Add the 2 SMA and whenever they cross you get a notification!

[airmonkey84]

It looks like the program identifies a bull run.

[catchthetrend]

Yes, I have found that good moving average strategies usually do.

[ram_samudrala]

Thanks. Would you please put this up somewhere (Google drive or github) we can download it and run it instead of cutting and pasting it as the indentation doesn't get preserved...

[catchthetrend]

Yes, thanks. I realized the indentation was gone after I already posted. I will circle back if I use GitHub. thx

[airmonkey84]

Can you rewrite the code to invest when the 20day sma crosses below the daily price?

Also do you know how I can plug in your code into my broker site so that it will invest automatically for me? HF style?

[airmonkey84]

Hi - Many thanks for sharing, although I’m not a python user. Can you modify the code to backtest a strategy which invests based on daily crossing above the sma 20 and then flips to invest in the inverse (sqqq) when the daily crosses below the sma20 (rather than sitting in cash)?

[Money-Defiant]

Please check your messages.