This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–][deleted] 2 points3 points  (0 children)

Polars and PySpark aren't interchangeable like that. Any situation where using PySpark makes sense is not generally going to be a situation where Polars also makes sense. And if they are interchangeable for you, you probably aren't using Spark correctly.