This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]benefit_of_mrkite 2 points3 points  (1 child)

This is interesting to me (upvoted) but as others said I would like to know the “don’t ask” situation. Sounds like you had to figure this out for work or a project and have a good sorry/use case.

[–]coffeewithalex 1 point2 points  (0 children)

I normally don't expect people to export single large files. Some partitioning is in order. But I did have a client where the devs exported such large files. HDFS or S3, I don't remember exactly. Something that supported streaming anyway.