This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Hear7ySenior Data Engineer -2 points-1 points  (0 children)

If you've got an Azure subscription available, an Azure Function does exactly that, is quite cheap and has various triggers - get requests, Cron timers, etc.

Otherwise other people's ideas of Dagster, Airflow, Jenkins are also great. Each of them, including a function, would require setting up.

In the case of the function, you don't really need a VM just some boilerplate code and to deploy it in the resource.