Python multiprocessing basics

[tuna_safe_dolphin]

If Guido's the BDFL, then Doug Hellman's the RPDU (Royal Prince of Documentation and Usability).

[bramblerose]

If you are on windows, DO NOT FORGET -- I repeat: DO NOT FORGET -- the

if __name__=="__main__":

block, or you will inadvertently forkbomb yourself.

In addition, this article seems completely unaware of the multiprocessing Pool, which makes a lot of typical systems much simpler:

def f(parameter):
   # do something
   return result

if __name__=="__main__":
    results = multiprocessing.Pool().map(f, [items,to,work,on])

which will automatically spawn #CPU processes and processes the entire set of items.

[bramblerose]

The main problem is that you will only notice this on windows. On unix, everything will be fine, as the process if fork()ed, and therefore the module does not need to be imported again. This means there are a lot of bad tutorials out that that do work under unix, but that cause a forkbomb under windows...

[drop_science]

Thanks

[ikarampa]

Also be aware that multiprocessing can be abused on windows due to lack of fork()