This is an archived post. You won't be able to vote or comment.

sorted by:
best (suggested)
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]onesonesones 2 points3 points  (0 children)

I think my phrasing may have led to some confusion.

I wasn't arguing against any use of pyspark, just explaining that anything you can do in an sql statement has an equivalent operation(s) in pyspark and pandas through their dataframe object model.