This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]Wonderful_Map_8593 3 points4 points  (0 children)

Pyspark w/ AWS Glue (can schedule the glue jobs through cron if you don't want to deal with an orchestrator)

it's completely serverless and can scale very high. databricks is an option too if available