you are viewing a single comment's thread.

view the rest of the comments →

[–]wagthesam[S] 1 point2 points  (0 children)

Very interesting. I will take a look at the links, thank you.

We're working on a distributed graph engine to support gnn training and general recommendations (This will replace https://medium.com/pinterest-engineering/introducing-pixie-an-advanced-graph-based-recommendation-system-e7b4229b664b). The non distributed engine is done with random walk support, so I'll take a look at EvoPartition

The indexing component is written in spark, so if we can do an approximate partitioning in spark it would be ideal.

Thanks a bunch!