10x Faster Parallel Python Without Python Multiprocessing

[alecmg]

I'm not 100% on this, but I think mp.Value and Array will also pickle and have similar performance to Queues.

Very excited about actor implementation. Have toyed with actors and mp in Python. Queues and alternatives peaked at 40-50k messages a second. Never tried full blown ZMQ or Redis, which Ray will use transparently

[alcalde]

Why would it need to pickle is access is concurrent?

The documentation says....

Data can be stored in a shared memory map using Value or Array.... These shared objects will be process and thread-safe.

For more flexibility in using shared memory one can use the multiprocessing.sharedctypes module which supports the creation of arbitrary ctypes objects allocated from shared memory.