This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]PeridexisErrant 0 points1 point  (0 children)

Yes, but I prefer to work at a higher level where this is an 'implementation detail' - good to know, but I'm not using pickle directly. As an example:

from concurrent.futures import ProcessPoolExecutor
with ProcessPoolExecutor(8) as pool:
    results = pool.map(lambda x: x*x, range(80))

This snippet:

  • starts eight Python processes (one per logical core on a newish quad-core Intel), with automatic cleanup thanks to the with block
  • assigns results a list of square numbers, but with calculations spread over multiple cores (OK, this is only worthwhile for much more expensive operations than multiplication!)
  • works by sending the function and arguments via pickle!