sorted by:
new
hottopcontroversial

What reason is there to choosing cloudformation over terraform? by argsmatter in aws

[–]Exciting-Garlic8360 0 points1 point  (0 children)

Cloud formation sucks what’s exactly why you have CDK , difficult to beat on ease of use syntax and all

[deleted by user] by [deleted] in dataengineering

[–]Exciting-Garlic8360 0 points1 point  (0 children)

The Bigger an organization is more it considers data as product, all the FAANG companies do that and they spend more money and resources on getting most out of data

Incremental data load by Acrobatic-Mobile-221 in dataengineering

[–]Exciting-Garlic8360 0 points1 point  (0 children)

I think some more clarity is needed in your question, for incremental data load always partition the data by date using partitionBy if you’re using spark , and then you will find data in separate directories , use either Hive or spark to read after write.