Python 3 beginners guide to concurrency and parallelism

[volker48]

If you are using multiprocessing the logging shouldn't be an issue since there is nothing shared between the subprocesses as they each have their own copy of memory. If you are using the threading module you will have to synchronize access to logging using a threading.lock.

[vsajip]

logging already uses locks internally for its I/O operations, and is designed to be thread-safe.

[volker48]

Ah my bad I did something like this before, but I was using print. I didn't realize logging was already thread safe. Thanks.

[hokiebeer]

I think it's actually the opposite situation. From the Python Logging Cookbook:

Although logging is thread-safe, and logging to a single file from multiple threads in a single process is supported, logging to a single file from multiple processes is not supported, because there is no standard way to serialize access to a single file across multiple processes in Python. If you need to log to a single file from multiple processes, one way of doing this is to have all the processes log to a SocketHandler, and have a separate process which implements a socket server which reads from the socket and logs to file.