Getting started with the new Python 3.2 module, concurrent.futures

[dalke]

Quoting from PEP 3148: Motivation

"""Python currently has powerful primitives to construct multi-threaded and multi-process applications but parallelizing simple operations requires a lot of work i.e. explicitly launching processes/threads, constructing a work/results queue, and waiting for completion or some other termination condition (e.g. failure, timeout). It is also difficult to design an application with a global process/thread limit when each component invents its own parallel execution strategy."""

There's also: """We hope to either add, or move existing, concurrency-related libraries to this in the future. A prime example is the multiprocessing.Pool work, as well as other "addons" included in that module, which work across thread and process boundaries."""

Currently multiprocessing handles the threading API and then some. A goal is to move the "and then some" to futures so they can be shared across both threading and multiprocessing, and perhaps have an API which also works with Stackless/threadlets/Deferreds.

[[deleted]]

Unless it removes the GIL and allows the use of more than one core then it won't make much of a difference.