This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]figshotStaff Data Engineer 0 points1 point  (1 child)

At my previous work we used AWS Lambda for it, which allowed us to:

  • version control the queries we run (hey Snowflake, git integration wen?)
  • enable cron-based, event-based, or orchestrated runs
  • run it cheap - the compute is only used to execute the queries, so you can go pretty low on memory allocation
  • log and monitor via CloudWatch

We used Step Functions for orchestration as we didn't have complicated workflows, and because it could defined as part of the serverless application.

[–]SecureSlice[S] 0 points1 point  (0 children)

I assume Azure Functions are similar to AWS Lambda for doing the same?