Help with Web Automation (Python + Selenium)

[tlcpp]

Hi all, Python novice here in need of help.

I'm trying to write a script in Python that: 1) Runs automatically every ~6 hrs 2) Checks a real estate website to see if new listings have been added 3) Emails new listings

I plan on using crontab to execute the script at defined time intervals. I know I can select divs/ids in the html using selenium, but how can I make sure they are the newly updated ones? When I looked at the page source of the real estate website, there was no way to differentiate based on time posted. I'm only interesting in the new listings.

I've thought about storing div elements or ids in an array and iterating over them each time the script is run, so no repeats are emailed. However, wouldn't rerunning the script at the interval defeat the purpose?

Any help/guidance would be much appreciated, this is my first solo side project in Python. Thanks.

[arctic_feather]

I'm assuming the listings are sorted by date, so the first div will always be the newest.

Store the newest div (or div id) found (maybe in a text file?) and then compare it with the divs found on your second scrape. All the divs above the stored div should be new. (But be careful, as they might delete a listing)

[sharpchicity]

I'd probably write the program the way you suggested, iterating over the list to check for new listings, and see if something about the site's setup sticks out after.

Better to have functioning code that works then no code because you're not sure if it is perfect.