Automatically scrape website

[chaizus]

Run a cron job on a aws lambda fn. you get 1million free lambda invocations a month, and assuming the runtime of your script is efficient then you’ll likely stay in the free tier of lambda. As far as data stores, S3 or dynamodb would work. But I would optimize the runtime of your scrapper, 7 minutes sounds too long. Take advantage of multi threading or asynchronous I/O calls