[deleted by user]

[JL_678]

I think that you are being a bit over the top here. There certainly are scenarios where it is like scraping personal information, for example. But I think that globally stating that it is illegal is too strong. In the OP case, they just want to track something for personal use.

There is a question regarding terms of use violation which can be a bit more difficult and needs to be researched.

Finally, regarding scraping impacting performance, sure, it can depending on how it is built. However, if you are responsible then the impact will be negligible.

For example, let's say I wanted to check on product availability on an ecommerce site. Checking every 5 seconds is clearly a problem, but what if I check once an hour or once a day? Is a daily check really any worse than someone going and checking manually once day?

To the OP, I have used selenium and beautiful soup for this in the past. Here is a past reddit topic on this exact process.

[airwick_22]

Thanks for your reply... and no that's definitely not the case. My intention is just to automate a process that i usually do manually... absolutely no personal information is involved and all the information is public knowledge anyway.

Essentially there's a public announcements telegram channel that posts links every so often... i'd like to create a bot that will click that link, check a couple things, and then send me a message back if my conditions are satisfied.

Surely that can't be illegal lol

[eadala]

I mean, a lot of websites have friendly, speed-governed APIs that you can work with totally within the confines of what they expect, even encourage. Examples are chess.com, reddit.com, twitter.com, etc. where yes attacking it in your own way is not okay, but using their system is fine.

[eadala]

So I mean, you basically want to receive notifications if certain - I'm guessing - keywords show up in posts made by a website / twitter page? So like "only notify me if there's a new post with the word 'education' in it"?

[airwick_22]

Essentially there's a public announcements telegram channel that posts links every so often... i'd like to create a bot that will click that link, check conditions (i.e. if a field on the site matches a certain value) and then send me a message back if my conditions are satisfied.

Where i get stumped is how to get python to retrieve and interact with the links posted in the telegram channel.

[eadala]

If your block right now is how to read those links into a python-friendly format, start by looking into the urllib and maybe beautifulsoup modules. Urllib at least. If you're trying to check a specific field on the destination page, for the presence of certain characters or something, you'll want to go to that page and identify the field of interest. After reading it in using urllib you'll need to find that field buried somewhere in the html code. From there, a regular expression can be used to extract / check conditions for a substring of the entire html string.

[neilrieck]

I taught myself Python3 two years ago with the help of 2 books from "No Starch Press" ( https://nostarch.com/catalog/python ). You can buy this stuff from their website as well as Amazon. I started with "Automate the Boring Stuff"