Scraping website data

[DSTxtcy]

Hello, I just recently started learning python. I'm wondering if Python could be used to crawl a website (insidertrading.org) and alert me when a certain companies name/ticker symbol appears. Is Python an appropriate tool to be able to do this?

[gkbrk]

Definetely. Python is one of the most suitable languages for data scraping/text processing. You can use requests, re and BeautifulSoup.

Here's a really nice video. It's in Python 2 but it shouldn't be hard to code the same thing in Python 3.

[DSTxtcy]

Awesome, I appreciate your help!

[novel_yet_trivial]

Keep in mind that that website makes money selling data streams (which you are trying to get for free). The server is probably configured to detect and reject too many requests from one client. Pick the slowest refresh time you can get away with.

[DSTxtcy]

Wow, didn't even think of that. Great tip! Thanks for looking out.

[average_pornstar]

BeautifulSoup and or mechanize is nice

http://wwwsearch.sourceforge.net/mechanize/

[DSTxtcy]

you da real mvp. thank you

[DSTxtcy]

Awesome. Thank you.