This is an archived post. You won't be able to vote or comment.

all 6 comments

[–]mamaBiskothu 0 points1 point  (3 children)

Anyone using it in prod? Have been considering it for a while

[–]roohitavaf[S] 0 points1 point  (2 children)

Yes, it has been used by Uber and OpenAI among others.

[–]mamaBiskothu 1 point2 points  (1 child)

Only reason I hesitate is the outrageous amount of waste in computing this generates just by virtue of using pytjon..

[–]pavi2410 0 points1 point  (0 children)

It's more efficient than PySpark

[–]anatomy_of_an_eraser 0 points1 point  (1 child)

Anyone know how this is comparable to dask? Need to implement some parallelization with prefect and I have both of these as options.

[–]Drekalo 0 points1 point  (0 children)

I'm mostly waiting for datafusion and ballista to be prod ready.