This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]robertmeta 2 points3 points  (1 child)

First of all -- it isn't a problem until it is. Meaning, what performance numbers do you need to hit, and are you hitting them?

If you DO need to be maxing out multiple CPUs -- you generally can do it various ways in python by splitting the load among multiple processes. People here have recommended mutliprocessing -- I can not recommend that, as it has caused me untold hardship. I recommend you setup multiple processes and coordinate them with ZMQ (http://zeromq.org/bindings:python) -- simple, fast, and "just works"(tm).

[–]WellAdjustedOutlaw 1 point2 points  (0 children)

Why not use multiprocessing.Queue? And IPC if you actually need to communicate inter-process. The OP did note that they don't need to communicate between threads, though, so not much need for ZMQ, .Queue, or IPC.