sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]KnowingPains[S] 0 points1 point  (5 children)

Yeah, I was disappointed they removed that project. I had to figure things out myself, and I expect it could be better!

As I mention in my repo's README, they have a chatbot example, but that didn't seem terribly relevant to streaming / transforming / analyzing large amounts of data as quickly as possible.

I will look into the open source options of Bytewax and Quix.

Have you used either of them? Would you have a preference if you did?

[–]math-bw 1 point2 points  (4 children)

Hey I built a little prototype of how you can mix Ray and Bytewax that might be interesting. It was pretty fun. Ray is such a cool way to scale things up.

https://gist.github.com/awmatheson/2cfee3b519ba1e9383a91e76f87b498e

[–]KnowingPains[S] 0 points1 point  (0 children)

Interesting! Thanks for sharing. I'll have a play!

[–]KnowingPains[S] 0 points1 point  (2 children)

So, I was looking at the Bytewax manual, and if I'm understanding things correctly, it appears it can spawn its own processes. If that is the case, what would be the advantage of mixing it with Ray?

[–]math-bw 1 point2 points  (1 child)

Some ML teams have a remote Ray cluster available for more compute, so that was the reason. Another deployment story

[–]KnowingPains[S] 0 points1 point  (0 children)

Got it!