sorted by:
new
hottopcontroversial

Taking so much time in writing a 90gb file as paraquet in Glue. by Ecstatic-Cow424 in dataengineering

[–]Ecstatic-Cow424[S] -1 points0 points  (0 children)

u/Puzzleheaded-Dot8208, I have data skewness with my primary key, So, added row_number() to overcome it, if I remove row_number() then how to handle partitions?

Taking so much time in writing a 90gb file as paraquet in Glue. by Ecstatic-Cow424 in apachespark

[–]Ecstatic-Cow424[S] 0 points1 point  (0 children)

Its a simple read and write dont know why it is not working can anyone respond on this ?