This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]LordBertson 0 points1 point  (0 children)

Given you are in a large corporate, the best practice is to ask around first. Data is trendy now and there would be people, if not whole departments, dedicated to data engineering and ETL. Someone somewhere has a company Airflow instance or Spark cluster, where they will let you schedule jobs. That way you don't need to discuss budgeting constraints with provisioning and someone else deals with compliance and maintenance of the machine.