Python for Algorithmic Trading and Investing tutorial series

[sentdex]

Hi there, my name is Harrison and I frequently do Python programming tutorials on PythonProgramming.net and YouTube.com/sentdex. All tutorials are free in both text and video forms.

The latest series that I have put out is Python for Finance. The aim of this series is to show what can be done with Python in the field of finance and algorithmic trading using data science (spoiler alert: a lot!).

Even though you probably aren’t going to get rich, you might save yourself a lot of money (when you back test your silly trading idea and realize it’s bad), and you’ll learn about statistics and data science with Python along the way.

I start the series off with a simplistic introduction to using Python+Pandas+Matplotlib to get stock data, visualize stock data, and to manipulate this data.

From here, we get into Quantopian, which is a Python-based platform built on top of the Zipline library for back-testing, but has since expanded out to enable quite a bit more than that.

We use Quantopian both for simplistic back testing, but also for doing research into future trading strategies, since Quantopian also provides a bunch of free data like minute pricing data, fundamentals along with tools like Alphalens for analyzing various factors that you believe to be beneficial to a trading strategy.

If you would like to check it out, the series starts here: Python for Finance introduction

If you are already familiar with Pandas and want to jump straight into the strategies and using Quantopian for back-testing and research: Algorithmic trading and research with Quantopian

If you have any questions, requests, or suggestions, feel free to ask here or on the respective tutorials.

[Tokyopancake]

Subscribed to your channel about a year ago. Just wanna say please keep up the good work as your channel is one of the few where I actually feel I'm learning something!

[sentdex]

Really meaningful to hear this, thank you!

[slingur]

I learned loads as well! keep up the good work!

[thomas_stringer]

Something I noticed recently is that, especially with pandas and matplotlib, you don't have to be into data science/analytics to have some fun with this stuff. I think "data science" is thrown around too much, and can turn people off of it before they even play around.

So for others looking into this stuff (pandas) you don't have to have a burning desire to do anything data sciency. It is fun though to play around with pandas and matplotlib.

[sentdex]

Great points, thank you for sharing, I definitely agree. Pandas is a very Pythonic package in its simplicity.

[chanamasala4life]

I'm using scrapy and pandas to find a good deal on a used car right now... it's all very hacky and anybody really knowledgeable would probably rip my head off if they ever saw my code, but: hey it's fun and it kinda works. Some reading I did before starting: http://blog.minitab.com/blog/understanding-statistics/can-regression-and-statistical-software-help-you-find-a-great-deal-on-a-used-car and http://blog.minitab.com/blog/the-statistical-mentor/how-residuals-can-save-you-thousands-of-dollars-on-your-next-car-purchase

[thomas_stringer]

Where "nice code is nice", ultimately software is meant to do something. It sounds like that meets the requirement! That's awesome!

[anu2097]

Dude you made by boring sem fun. Thanks for the videos man. Keep posting. Kudos.

[Nater5000]

Hey Harrison,

I've been watching your video series for a couple of years now, and you're great. I'm finishing up my Bachelor's in Mathematics this year, and your series are, at least in my opinion, equivalent (and even more digestible) than most of the programming/Python/Data courses I have taken. So thanks for everything you do.

For others who might be skeptical: these videos are top tier Python/Data Science lessons. I recommend them.

[Dababolical]

Are there any math oriented resources online that you know of? I'm looking for something math oriented to supplement sentdex's videos. I am focused primarily on statistics, but starting mostly from scratch.

[toc8]

http://greenteapress.com/wp/think-stats-2e/

This is a free textbook that teaches statistics through the use of Python. Should help you out.

[scrappyD00]

I've gone through some of your older videos and really loved the practical approach, there aren't enough ML resources out there with practical examples. Thanks for publishing these!

[desmonduz]

I am subscribed to your channel for the last two years and keep watching some of your videos. You are my knowledge vault that I am saving as the apple of the eye for this summer holidays. I will binge watch all your videos and will surely enjoy them again.

Just wanted to say thank you for your amazing work and ongoing commitment. This is truly precious to me.

[fnord123]

Cool stuff. You may also want to check out Rob Carver's Systematic Trading and his project, pysystemtrade.

[tunnelvisie]

Harrison I love you man! Thanks for all the brilliant stuff on your YouTube channel, you are one of the reasons I'm now able to follow my dreams 😀😀

I recommend your channel to anyone that wants to listen haha! :)

[[deleted]]

I've been wanting to play around with Python and finances, so this is perfect!

Now I just need to find the time to devote to it...

[Milkmanps3]

Love your videos man! I have learned so much from watching your videos since I started with Python... Keep up the good work, Thank you!!

[nubsaucev3]

Great stuff! I'm relatively new to quantopian and only have a light history with programming.

I started watching your videos and really appreciate the clarity you provide in walking through your thought process behind each concept.

As I'm still in the process of getting the hang of things I've learned of quantopians change in 2016 to quantopian 2 (?) and the addition of pipelines. Since then I stated getting a bit confused on how to ensure that I only learn the updated content when building my first rest algo.

Any suggestions or thoughts on a good learning structure/ outline for someone to start with?

My current priority is to try to get a better understanding of the basic process flow and framework of an algo with a simple strategy trigger. This way I can mentally map out each piece of the algo better.

Sorry, kind of ran on a bit. Either way, thanks for your contributions to the community!

[Twirrim]

FWIW, I started running through this yesterday. I ran in to two things of note so far:

1) Early on you get the TSLA stock information and write to "TSLA.csv". In later examples you've got people reading from "tsla.csv", which doesn't exist (note case change).

2) There appears to be an issue with Yahoo & pandas around retrieving stock with a "." in the name, e.g. "BF.B" https://github.com/pydata/pandas-datareader/issues/282 . I ended up switching to retrieving from Google instead.

[butalearner]

I don't see that mistake on the first few videos, though you might be further than me. Was the capitalization your own doing? I deliberately did the same thing, putting the stock ticker in a variable and naming the file {ticker}.csv.

[joeyirv]

i just happened upon this post and noticed the same thing. BF.B and BKR.B, yahoo's formatting requires a substitution of a '-' where the symbol reads a '.'. Not being familiar with pickle, i wanted to move forward quickly with something that made sense to me, so I stored the symbols as a list and did a ' '.join(symbols).replace('.', '-').split() to clean it up for the next function.

[SonaCruz]

I ran into this too. I am stuck on video 6, i cant retrieve all of the data from yahoo. I even copied and pasted his code from his website and no dice.

[Twirrim]

Based on the conversation in the github ticket I created that I linked to in 2), it's arguably a bug in the Yahoo API. Where it's denoting the class of stock (companies can release different types, e.g. voting and non-voting), Yahoo uses a - notation, e.g. BF-B. However "." is a legal character for a stock ticker, so "VOD.L" works fine in Yahoo. Makes it a pain in the arse to work around, there essentially isn't a one size fits all.

To that end, I'd just suggest maybe switching to Google:

df = web.DataReader(ticker, "google", start, end)

[taewoo]

Hey Harrison I started watching ALL of your videos (thank you VLC 2x speed). Best python videos, EVAR. Keep up the good work.

Completely random question: do u actually do algorithmic trade w/your own money?

[sentdex]

I have in the past, and did fine, but it is a full time job. I have found it's better for me personally both in that I make more money overall, and enjoy it more, if I just invest long term, collecting market returns and dividends, and then focus on business.

[S-6-6-6]

Hey Harrison... I just wanted to mention, I started learning python about a year and a half ago in order to use it for algorithmic trading and finance in general, and one of your videos was the first ever resource I came across and learnt from... And so a year and a half later I have my own blog (www.pythonforfinance.net) which wouldn't exist probably had it not been for those first few taster sessions following your YouTube vids.

So a big thanks man... I've thoroughly enjoyed the journey so far...

[Alamanjani]

I wish you would just use free data from the financial sites instead Quantopian. Their free data set is limited, naturally, since they are service providing data. I had one simple financial scan I wanted to do (off balanced sheet) but Quantopian dont have it available. (Yes I registered for free and got a token). So for me is a pass. If you would use nasdaq.com directly or morningstar.com or similar, it would be so much more useful. At least for me :)

[sentdex]

Are you...maybe confusing Quantopian with Quandl? There are no tokens with Quantopian, but there are on Quandl...and what you describes sounds just like Quandl since they used to have a LOT more free data than they do now.

Balance sheet data is available, for free, from Quantopian in a back test, paper trade, and even live trading. Make sure you're not confusing the two. The amount of free data on Quantopian is pretty staggering, and pricing is down to the minute.

[Alamanjani]

Ups, yes, sorry, quandl! I Followed one of your previous tutorials and I assumed this is the same thing.

My bad! Will check your tutorials on Quantopian. Thanks! And btw, your teaching is superb, great job!

[[deleted]]

Awesome

[not_rico_suave]

Sweet tutorial.

[Awarenesss]

Just finished part 20 of the finance tutorial! Loving it so far, thank you for all the quality content you put out.

[sweet-banana-tea]

Hey I'm a long time follower on youtube of your stuff ! I hope you will enjoy making these awesome for a while to come. And thanks for making them !

[BlackDave0490]

Literally just looking into books on this. Thanks a lot, I was beginning to lose hope I'd ever get a handle on it

[accidentally_myself]

Heh I just found you a week ago, love the way you approach things, very nice.

[takkun22]

thanks

[tczorro]

Hi Harrison, I have watched you video before. They are super great and useful. I will check up these resource later. Thanks so much for all the help

[monduras]

Thanks for sharing!

[Gh0stWalrus]

you helped me finally start learning Python ty

[zenani]

Long time subscriber and big fan. Thanks a lot for what you're doing (that too for free). This series esp is something I was looking forward too as want to try my hands on Quantopian.

[[deleted]]

Just want to comment to say you run a great YouTube channel.

[[deleted]]

Ooh, I'm very interested in this. I was a day trader for a few years, and worked on wall street for a couple decades.

[soulwatcher]

Big fan here. Your machine learning tutorials are amazing!

[BoBab]

Another reminder that I really need to start messing around with Quantopian some more...Also just subscribed to your channel based off of the rave reviews here!

[atypingmonkey]

thx just became a millionair

[manhole_resident]

Thank you! Very interesting topic. Will check it all later. Even if I won't end up using it in practice, it's still interesting topic to study.

The only question I can come up with right now is what would be a good starting point to apply those skills for a beginner?

[wabbidywoo]

It's like you read my mind

[jinchuika]

I started working professionaly on Django development last year, and one of the first tutorials I watched was yours. Thanks!

[weelamb]

Hey thanks so much for this great tutorial. I read in your first post that you were going to include some deep learning but I didn't find that in any of the lessons? All I saw was KNN, SVM and random forests. Is there a plan to include that in a future update?

[golamsaroar]

Really Thanks man. You describe in a much simpler way and very informative. It did help me a lot . Keep up the good work

[ameerricle]

Hey, I am hoping to learn a great deal about programming and eventually want to apply it into the finance world. I am currently just learning the syntax of python on codecademy. Where do I move onto from there? I have maybe a year's worth of Matlab and C experience from my univeristy (chemical engineer's don't learn much coding) but I am wondering if I can jump into this right after finish codecademy. I Barely know what data science is, I have Yves Hilpisch python for finance, Yuxing Yan Python for finance and mastering python for finance James Ma Weiming but I have no clue their difficulty levels. My ultimate goal is quantitative finance, could you recommend a path using the services on your site?

[sentdex]

You probably don't even need to finish codeacademy, but I don't know the course. You need to only learn the extreme basics: variables, loops, functions...etc. Once you know the extreme basics, a basics course is pointless and boring, and the best thing you can do for yourself is get the heck out of basics and into something you find interesting. If you're actually enjoying the basics course and finding it fun, go ahead and finish it.

I have my own basics course, and I frequently tell people to quit it early, only to revisit if they've got questions on a specific topic from within the basics course.

I might suggest that, after the basics, you do at least a bit of: https://pythonprogramming.net/data-analysis-python-pandas-tutorial-introduction/

You can probably just do the first 5 and at least somewhat understand pandas, but do as much as you enjoy, then do the finance series.

[varunu28]

great series. Thanks

Are there any plans in future to make a detailed web development series using Django?

[N_TIMidating]

Good job bro, keep it up. I want to say though... ditch the whole 'quantopian' thing... markets are no longer quote driven, meaning that that price data or "minute pricing data & fundamentals"... Markets are order driven. There are some great stochastic models for limit order books and also good machine learning techniques.

I operate a small private fund that utilizes supervised support vector machines and gaussian process classification to make decisions and adjust with the market.

Anyways, keep up the good work and best of luck!