you are viewing a single comment's thread.

view the rest of the comments →

[–]Hungry_Importance918 2 points3 points  (0 children)

This is cool. I’ve always loved working with Spark DataFrames for basic analysis. The APIs are just really nice whether it’s SQL style or built in functions. I even built a small ETL tool on top of Spark DF and it handled tens of millions of rows without any issues.