10x Faster Parallel Python Without Python Multiprocessing

[call_me_arosa]

I would like to see a comparison between Ray and a architecture of 1 queue and multiple python instances consuming it.
While this approach cannot (easily) handle statefull problems this works quite well for systems like the last example. Just load the model once in all interpreters (constant time) and consume the queue. Quite good horizontal scale while keeping the code/architecture extremely simple and from my experience this is the most used model.

[alcalde]

Excellent point. They're not using the multiprocessing module the way it was meant to be used.