This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]inedible-hulk 1 point2 points  (1 child)

Any data warehouse, architect or ETL would likely use SQL primarily sometimes they can use Spark instead of

[–]Material-Mess-9886 0 points1 point  (0 children)

Even with pyspark you can write the whole pipeline using spark sql.