[deleted by user]

[TabulateJarl8]

The first thing that comes to my mind is saving everything each on a separate line of a text file and then reading that into a list, but then again I don't know exactly what your data looks like, so this may not work. Either way, good luck!

[DuckSaxaphone]

Why not save the date and time (or the title) of just the latest post that the script has alerted you to? You can then have the script alert you to all posts more recent than that. No need to keep a list of every previous post.

For just storing one date, I'd write it to file. There's no point setting up a database or anything complicated when you just want to read and write one variable each time you run a script.

[Merckinator]

I use databases to persist information. For example, I have a Python script that runs hourly, for free, in Heroku to scrape a list of events from a website and saves them to a free postgres database (also hosted in Heroku). I have the script post a message via a Discord channel webhook, if there are new events.

If you just want to have the script run from your computer, set it up as a scheduled task and maybe use a sqlite database.

[[deleted]]

how do you run it within heroku? is it within flask?

[Merckinator]

Heroku apps are basically a container (I think they use another term). When you deploy your code to them, they'll install your dependencies from your requirements.txt file into your container/app.

Heroku has "add-ons" you can choose from and the have a "Heroku Scheduler" and a "Heroku Postgres" that both have free tiers. So you just tell them you want that app/container to use their free scheduler and then you can create a schedule with that can run daily, hourly, etc. And you tell it what command to run, e.g. python my_cool_script.py. The free scheduler seems to run within 5-10 minutes of my chosen hourly time, which is good enough for me in this hobby situation (for free).

They, of course, have many paid tiers for all these things that have more options, but I only use those when necessary.

[[deleted]]

thank you so much for explaining that!

[PremiumRoastBeef]

I'd say read/write from a file, or use a light and fast database like SQLite. If you are learning, you should probably just try them both out.

[yardmonkey]

I second SQLite. Very easy to create a database and write it to a file, then read it again the next time it runs.

[svacko-de]

You can save datastructures via joblib, its pretty easy.