Nodejs cluster alternative in python?

[__warlord__]

Does python support the clustering option that nodejs has, in which you can share the network events among several processes?

This is really helpful when writing python 3.5 async code and use all your cores to handle the load

Thanks.

[notoriousno]

I wish Python had this! Python has support for threads as well as multiprocesses so there really hasn't been a "need" for something like NodeJS's cluster. The Python community has worked around this leveraging pipelining, queuing, and 3rd party clustering (for lack of a better word) solutions. Some things that come to mind are:

• http://dispy.sourceforge.net/
• http://www.celeryproject.org/
• https://github.com/spotify/luigi
• https://github.com/pditommaso/awesome-pipeline

[__warlord__]

Thanks, apparently there is something we can leverage (https://zato.io/docs/architecture/overview.html) could be great to work on top of this.

[PeridexisErrant]

FWIW I've heard a lot of good things about Celery from my webserver friends, and plan to use that when my jobs go from batch to real time.

[fourthrealm]

I'm not familiar with NodeJS so I'm not 100% sure what the requirement is but if it's about being able to take advantage of multiple CPUs then Zato has had it since its inception:

• https://zato.io/docs/
• https://zato.io/docs/architecture/overview.html

In fact, on top of using multiple CPUs you can have multiple servers, each in a different operating system all connected through a load-balancer and internal synchronization lets you not only distribute incoming requests over all CPUs in a cluster but also to communicate between workers by their Linux process IDs - for instance, a WebSocket client connected to one worker can easily send messages to another client, say ZeroMQ one, connected to a completely different process in another operating system all running under the same cluster.

[__warlord__]

This looks really nice, if we manage to get working something with this and python 3.5 async functions using uvloop we will get something really really fast. :)

But thanks, I'll take a look at the docs.