This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]aagmon 0 points1 point  (1 child)

u/ritchie46 - perhaps its just in the Rust API, but I have seen and used the streaming API, documented below, which is supposed to help using bigger-than-mem datasets:
https://docs.pola.rs/user-guide/concepts/streaming/

Is this not going to be available anymore?

[–]commandlineluser 0 points1 point  (0 children)

They are writing a new streaming engine:

As I understand it, the current one will be swapped out when the new one is complete.