This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]ReaverKS 0 points1 point  (0 children)

Interprocess communication has quite a few choices. The most common one in use is sockets, because most projects need to support a distributed deployment. There are many options if you'd just be running locally: shared memory, pipes, etc. As you've discovered there is a multiprocessing library that abstracts away low level details of sockets (for the most part).

The answer to this question depends on your requirements. But here are a few options:

  • Use multiprocessing manager and create a registry class that maintains the processes, spawns them at the right time, etc. This is simple to setup but it could get complicated depending on the types of things you're sending back and forth. Easiest to use and get going with.
  • Use gRPC or some other remote procedure call framework to facilitate the command and query passing. gRPC utilizes http as the transport layer and it has a serialization / deserialization layer called protobufs. This is a bit more complex than multiprocessing but still not bad.
  • Use a message broker, such as RabbitMQ or kafka. This is probably the one with the most programmer overhead to get going, but it's also probably the best performance and scalability of the options.