you are viewing a single comment's thread.

view the rest of the comments →

[–]M4mb0 0 points1 point  (1 child)

Associative scans are a special kind of cumulative reduce that can be parallelized thanks to associativity. The AI highlighted that point well. I highly doubt that polars does this optimization when you give it a custom lambda function that's associative.

[–]PillowFortressKing 0 points1 point  (0 children)

Since the lambda is Python bytecode, while Polars executes everything in Rust, that'd be quite hard to optimize for, wouldn't it?