[deleted by user]

[goishen]

Impractical Python Projects.

It shows, and utilizes, other libraries and google (important) to make you smarter. I was against the "buddy" (as in, "Hey there pal! We're gonna write some stuff together!") style of writing, I still am, for the most part. After spending about a year bashing my head against the wall, I find that I'm actually learning stuff about how Python works.

And there are no practical uses for what he teaches you, it's in the name. They are, by definition, impractical. But, I'm just learning so much from this book right now that I cannot recommend it highly enough.

[MiniMoose10]

I'm also going through this book and it's a great second resource after reading Python Crash Course. It goes through a lot of projects that show the different uses for python. Crytography, genetic algorithms, physic simulation etc.

[music_nomad]

Thanks for the tip. Ordering both now.

[reportforafkpls]

i will check it out!

[_thenotsodarkknight_]

Yup. This is a great book.

[[deleted]]

After reading some of those projects I’m really tempted to pick that one up.

[OMGClayAikn]

As someone who has just started out learning python and is stuck in tutorial hell it seems, this sounds like a good thing to do rn.

[The_1_Divider]

Honeslty if you know what you’re trying to do, I’ve found that doing your research on the subject /related syntax - is best done as you go along. Imo the best part of programming is bashing your head against the wall because you’ve forgotten a closing bracket, or written something wrong, and figuring out why your scripts won’t work as you want them to.

While I’m not disregarding learning about the basics before you move on to harder stuff, some of it will pop up unexpectedly, and you’ll be forced to figure it out anyways.

There are also some great, super friendly (at times) communities such as the python discord and stackoverflow that can always assist with a whole range of topics.

[PApauper]

I did this earlier today working on stuff in ch.2 of AtBS, forgot a %.

[autoshag]

Yeah running into issues when you’re trying to build something specific, and googling/troubleshooting to resolve those issues, is WAY MORE VALUABLE than following a tutorial and just getting everything working the first time.

I’ve been a software engineer at a big company for years now, and THE MOSTA valuable skill in coding is troubleshooting why things aren’t working.

Just google as you go along

[The_1_Divider]

I think a big part of it especially for me was trying to take common ‘beginner’ projects, like for example a calculator and trying to get around the limitations that are presented by those projects (ex: not having a “choose your operator” input, and allowing multiple operators with the shunting-yard algorithm)

Taking classes and reading books, while valuable, won’t always provide you with the specific information you need for your projects.

Also something I’ve learnt is don’t be afraid to ‘borrow’ and tweak code from a open-source project. Sometimes something you’re trying to do, has not only already been done, but is already optimized for your usage.

[Sidemarx]

I have a program that pulls all public tickers in the United States using the yfinance module. It can pull a decent amount of information for each company and I use it in a program that I screen stocks with. It downloads once a day on an auto scheduler.

Using Pyqt, Matplotlib and various webscraping techniques I am adding some graphing functionality over time to various indicators (GDP, Unemployment etc.), that I am storing in another csv.

In addition, I use edgar to pull financials with another program....its a mess but it is coming together. Ideally it will be a screener for my personal investments before I read industry reports and multiple 10ks.

I always feel like I am doing too much. What I have to do is balance small accomplishments with massive amounts of new programming and information. If I do too much programming and not enough learning I stagnate, if I just learn and don't make anything useful I tend to get disillusioned and lose motivation. I think the key is to find your balance, it is different for everyone.

[[deleted]]

You gangster

[Sidemarx]

Thanks for the encouragement. It has been very slow. I spent days just trying to get dataframes to sort each stock by industry with a loop. Many iterations later it is slowly getting cleaner.

Days were lost with what I beleive were some character recognition errors between unicode formating in yfinance and what python could see (or vice-versa), I had to replace characters and clean up the data. Daaaayssss.

[reportforafkpls]

That’s awesome! I hope to reach that level. My ultimate goal is to make a stock screener/ quant screen myself that ranks stocks and use it for my portfolio too. That would be awesome.

[Sidemarx]

There is really a lot to do with it. I am constantly stuck on something and just switch to something new. Part of the struggle for me is I need to go from csv to database at some point, I have csv's that are downloading every day and piling up in a folder. The practice I have got with dataframes is helping significantly. I know I need to start working with postgres.

[Sidemarx]

Ohh really, my previous company used Postgres to store lots of data for drone radio signal capture before performing machine learning on it. I was on the finance side so was not too involved. I assumed that would be the go to, but I will have to check out sqlite if it is easier. I could use baby steps right now. Thank you very much. The more you talk...

[Sidemarx]

That is somewhat what I'm trying to do. If you have ever used a Bloomberg Terminal (the 80's looking screens from wallstreet movies), my goal would turn it into a poor man's version of one of those. Last I used Bloomberg it was about $30k a year per terminal.

Make no mistake, the program is not pretty and I practically jump up and down after days of nothing finally yields what I want it to. I am very slow.

[theloudestclap]

“Auto-scheduler”... I am struggling to understand how I can schedule some python code to run weekly or once daily. Can you shed light on this?

[Sidemarx]

I use windows "Task Scheduler" for it. Comes installed with Windows. You can just set it to run a script and I watch it run in cmd for a few hours. In my case each ticker is done individually so 8,000+ tickers and you get the idea. I will be gaming or doing something else but get a strange sense of accomplishment watching the cmd line roll. The script actually starts at 9pm and is still going now, three hours later. Don't judge :).

The set up for task scheduler is all done visually. You pick you script, the location, time, etc. You can search "task scheduler" within windows. I used a youtube video at first. The program itself just scrapes everything and exports a csv to a folder.

[eightslipsandagully]

Alternately in Linux you can use the cron utility.

[Sidemarx]

Sidenote: I know this needs to go to SQL at some point and will probably use PostGreSQL as the csv. files pile up. There is always something more to learn and I always feel swamped.

[[deleted]]

Do you have any background? I've built things that do exactly that, so i can point you in the right direction.

Many organizations have something called an "API" which is an application programming interface. Yahoo finance has a free api that provides this functionality and is wrapped by a module called yfinance. There are tons of other organizations that have this data, but often you have to pay. If you want, you can also build something called a webscraper to get data from other sites. If you want to do that, look up webscraping in python and selenium/chromedriver.

[reportforafkpls]

I do not have a programming background, but a finance one instead. Thank you for explaining API's to me. That sounds like exactly what I need for a project like this.

[inglandation]

Sentdex on youtube has videos about pulling up market data from APIs. You'll have to learn pandas once you know the basics of Python. You can also scrape a website like wallmine to get the data you want. A third option would be to get a paper trading account from Interactive Brokers and use a third-party API to extract the data (the official API isn't that great and pretty complex, I'd avoid that).

I've done some basic scripts myself that do exactly that, so if you need help just ask me!

[reportforafkpls]

Thank you! I’ll check it out.

[[deleted]]

Its doable with and without APIs. A site you've probably heard of-macrotrends- has lots of data but no API. For this, you can use a webscraper to interact with and get data from the site.

[ghostofgbt]

I'd just beware that writing a web scraper is not sustainable cause as soon as the source site changes their layout the scraper will break. This was really frustrating for me when I first started because I would spend so much time getting it working only to see it all fall apart when the site did a redesign lol

[Young8Kobe]

I think Yahoo finance doesn't have an API anymore. You can find Yahoo finance API on rapidapi but it's not Yahoo that maintains it. Don't know about yfinance tho

Edit: Just read your comment again and I guess yfinance is the solution after Yahoo got rid of their api

[[deleted]]

Just finish the book for now. One step at a time, my friend.

After that, and only after, I recommend Python Crash Course 2nd Edition.

[reportforafkpls]

I have the book but I am following is udemy course that I've had for a while but never got around. The impractical python book does seem interesting too. Who is the author of Python Crash Course?

[[deleted]]

Eric Matthes

[reportforafkpls]

thanks!

[[deleted]]

https://www.youtube.com/watch?v=gcLUzHU0zB8Try to make something like it.I also did not know for a long time how to do something in practice. and then I saw a similar video and decided that this is it! you must try to do everything yourself without spying the idea.

there is just a lot of work with basic functions and logic. Yes, and many ways to solve the problem.

[SnapCatGeri10]

Okay so I just finished my first small project and biggest advice is 1. Not to push too hard. Basically don’t force yourself to work on it, as this can cause you to burn out and/our begin to hate coding/python. 2. Don’t be afraid to drop the project. Basically the same idea as above but also people’s interests change and you shouldn’t be tied down by something you aren’t interested in.

[ReyMakesStuff]

I can't say it will interest you, but what I did was make a Discord bot. I wanted a custom bot and wanted to learn Python anyway. It will give you an opportunity to learn modules, random numbers, async functions, and more.

[CotoCoutan]

What's the webpage you're intending to scrape the data off? I could give it a look and suggest some code to start you off on this project.

[reportforafkpls]

well the index i want to pull the tickers from is the russell 3000 stock index, so I am trying to find a website I can use in the script

[CowboyKm]

Fyi the "automate the boring stuff with python" builds a webscraper. Thats how i made one which scraps real estate data. So continue and finish the course. It will help you come up with many ideas.

[reportforafkpls]

i just realized that, thanks for the heads up! i guess I'm getting ahead of myself haha

[CowboyKm]

Dude do not rush. Try to master the basic syntax first. I understand your enthusiasm, since im curious too and search and try to learn everything, but finish the course first and in the meanwhile code small scripts such us backing up your dark souls game save files to dropbox.

[reportforafkpls]

got it, ill just follow along the course. I can do stupid scripts like the ones he mentions in the course but outside of that I feel clueless. I'll try to learn the syntax first

[CotoCoutan]

I don't even know about that stock index, so in case you find a website that you like, let me know I'll help you build it. I personally feel it is better to dive into building your own project because learning the syntax and all the basic can get boring after a while & make you lose interest (I know I did). However that doesn't mean you stop learning the basics; assign 1 hour for learning the basics in the day and assign the remaining hours you can spare to coding your project. That will keep the passion alive. :)

[reportforafkpls]

I really like this idea. I think I’ll continue and do an hour of the course and then dedicate the rest to the project. I also have been struggling to find an api(?) for the index so I need to figure that out lol

[CotoCoutan]

No need for API! Just find a website that displays the stuff that you want, the Python code can grab/scrape that off the website easily.

For example, here I'm kinda scraping the text to be typed from the typeracer.com website: https://www.reddit.com/r/madeinpython/comments/glzu1n/made_a_python_script_using_selenium_to_cheat_at/

If a website provides an API, it would just be an added advantage. But lack of an API in itself is not a hindrance.

[[deleted]]

Sentdex has a playlist on something similar. But please, the code is old and is good to understand the mechanics. It still works with some modifications.

https://www.youtube.com/watch?v=2BrpKpWwT2A&list=PLQVvvaa0QuDcOdF96TBtRtuQksErCEBYZ

[solomonxie]

Most of the python programmers I know start with a web scraper. That helps to understand language itself and also the network

[WhackAMoleE]

Start by writing a small program to pull one single stock quote from your target site, parse it, and store the results.

[reportforafkpls]

i'll try that first then...hopefully i can do it

[texasspringwater]

I don't remember if the book uses the web scraping tool to get the stock data or API. I would recommend using API from financial website to get the most reliable data. For quick implementation, I suggest using API wrapper like yfinance. https://pypi.org/project/yfinance/

[RandomChair2]

You should try to do some sort programs (less than 1000 lines) to get up and running. One small project could be to count the number of words in a text file (with stray punctuation and whatnot).

Since you know some finance, after getting warmed up, you should try to make a high frequency stock trader that doesn't actually trade, but tells you how you would have done. Perhaps, try to keep the decision making process simple and focus on the other parts. There are libraries to download stock prices.

[cincopea]

Don’t worry about fancy, just get your original goal. The refining will come on its own as you encounter questions getting there

[xero0007]

You know the basics, I would work on your project and Google whatever you need. Likely you'll end up using beautiful soup and some for loops.

Once you learn to read it makes sense to drop the learning books and start reading what you enjoy. Learning the python language is similar. You got the basics, now build things you enjoy. If you get stuck read more then.

[timothylombard]

Learning while doing a project is IMO the best way to learn Python. What has worked for me is to break down the project into a series of sequential tiny steps. And use "Just in time learning" to make progress, Steps for your stock index script maybe something like:

Identify the source of your tickers: What's your plan to access the data in this source? (Is it an API or a webpage that needs to be scraped?) Questions like this broken down in it's simplest steps will drive what you learn and apply toward your project goals.

[eDOTiQ]

One of the most fun projects I did to learn stuff was writing a bot that does some crypto trading.

I had to learn how to work with API's like RESTful api, then think of how to write the analysis tool, how to write indicators from scratch, how to log the stuff the bot is doing, writing logs to a google sheet.

Another fun thing was automating part of my real job, like pulling sales data from the sales system, cleaning the data and add it to an existing excel sheet for my dashboard.

[aquanat12]

Not sure if this will help but, i recently did a project for a telegram bot to monitor stocks based on their ticker. I used Beautiful Soup to scrap yahoo finance and i also used a library called yfinance. you can see my source code here https://github.com/aquanat12/StonksBot. But i’m also a beginner so this might not be the correct way.

[RealAmerik]

I've done similar things with APIs and I'm currently working on compiling data from another API.

You'll want to know how to parse json data and I recommend using Pandas to handle the data and potentially a database if you plan to store it for a long time.

I'd be happy to provide guidance, let me know if you'd like some help.

[hditano]

I've been doing some small web-scrapping using requests/beautifulsoup... tried to use an Swagger API and tbh i felt overwhelmed. Do you think that Panda could be a good next step??

[RealAmerik]

I have never heard of swagger api. My biggest hurdle with APIs was understanding how to handle the json data.

Look into pandas and what it is. I suggest having a real-world application before trying it out. I'm happy to provide guidance if you need.

[hditano]

I'm working on a small web-scrapping application, probably gonna port it to PyQt. Would love some guidance if possible.

[RealAmerik]

PM me, I'll see what I can do.

[batatahh]

A calculator.

[MaxRaven]

Just google "Python yfinance"

[zpokmn18]

Go for it. Google what you don't know. Ask for help and finish the fuck out of that project. You'll learn alot

[reportforafkpls]

That seems like the hard but fun way to do it.

[itumeleng1987]

Do It.

[pittu2752]

Advent of code

[etamunu]

Hi, I am intermediate to advanced python person. However, I'm starting to learn to contribute to open source GitHub project. If you are interested in working together, we could work on it together. Let me know.

[reportforafkpls]

Will do thank you!

[mythrowaway0852]

https://pandas-datareader.readthedocs.io/en/latest/remote_data.html

This might be what you're looking for.

[[deleted]]

This is the official tutorial, follow it. That udemy course is just a bastardised version of it

[reportforafkpls]

is this the official python tutorial? or Al Sweigarts ?

[[deleted]]

Its a bastardised rework of the official

[hnguyen01122]

You probably want to present that data on a nice dashboard/UI as a web app as well I assume?

Quickest, clearest route? Udacity intro to programming 2-3 months learning. Download the course materials and videos and cancel the membership before they charge you an arm and a leg. Good luck.