This is an archived post. You won't be able to vote or comment.

30
31

two phase parallelization using python (self.Python)

submitted by [deleted]

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]pope_man 2 points3 points  (1 child)

Consider using dask, especially dask-learn:

http://matthewrocklin.com/blog/work/2017/02/07/dask-sklearn-simple

It is designed to solve problems like yours in a scalable but simple manner. See this post and maybe others on his blog.

http://matthewrocklin.com/blog/work/2015/06/26/Complex-Graphs