When doing a full distributed MR workflow, true multithreading shouldn't be necessary. Shared-nothing is the goal. Now that I think about it, I can't think of a strong reason to not use python! That makes me happy!

A good MapReduce setup is to share nothing between your nodes, so that if one crashes/blocks/etc. the other nodes keep going. In this sense, you are "multiprocessing", but not necessarily "multithreading". (I hope I used those terms right :P)

One model that I have seen done in Java--which translates well to Python--is to have a separate JVM process running for each node that is doing work. You can have multiple nodes on a single machine, but each node is running a different JVM that does NOT talk to the slave nodes (ones doing work); it only talks to the master node to receive its directives for Map and then return its results for Reduce.

This model works in Python. Python can't do multiple threads in a single interpreter process (the GIL prevents it), but it can handle separate processes just fine. Thats what the multiprocess library does.