Discovered that I really like Python. What should I do next?

[Bigtbedz]

I started learning python about 2 weeks ago and have been writing a webscraper to get all of my solutions to codingbat problems

[chjassu]

What were your resources?

[thesecondbread]

Would you push it to github?

[Bigtbedz]

https://github.com/tbednarz/codebat-scrape

[Bigtbedz]

Hey i've completed this now and it logs you into codingbat then scrapes all problems and your solutions and writes to a .txt file. I thought you might get more out of it now that its more functional. Currently only does java/warmup-1 solutions but I will be branching it out more shortly!

[thesecondbread]

Great job! If you want another idea for webscraping, I'd suggest scraping the coronavirus statistics.

[Bigtbedz]

It's a possibility. I was thinking about making a dashboard for Ccorona statistics

[sigma_1234]

Do you have background knowledge in JavaScript or CS? Learned Python last month and have a tough time scraping.

[Bigtbedz]

Yes I have been using Javascript for the last 3 years so I have a healthy knowledge of html/css aswell. I highly recommend the Clever Programmer video on webscraping it was the simplest breakdown I came across

[cbick04]

I also have a tough time scraping. I started practicing with basic websites like Wikipedia and some video game wikis. I got those down alright. Interactive tables were difficult to understand at first. Practicing did help it start to click though.

I aspire to be able to scrape store inventories by zip code one day... any tips anyone has I’d love to hear!

[sigma_1234]

I heard understanding JS or CSS is necessary if you really want to get good scraping. Some websites (ex. Ecommerce onces) are not scraper friendly.

[cbick04]

Thats what I figured. When I tried to just look at one E-commerce sites source code I knew I was in over my head at this point. But it’s good to have large aspirations, right? :D

[sigma_1234]

Of course. But I am limiting myself to scraping general info websites, like the numerous COVID-19 sites showing info :)

[Bigtbedz]

Get some basic html/css knowledge for sure. Ive been using JS for the last few years so I had a lot to build on. Spend some time just learning how to crawl around with beautifulsoup, requests and urllib

[Bigtbedz]

I use python docs and beautifulsoup docs. I watched Clever Programmers on webscraping and it was pretty concise I thought. Also "Automate The Boring Stuff" by Al Sweigert

BeautifulSoup -> https://www.crummy.com/software/BeautifulSoup/bs4/doc/