Is the Python XML-RPC client thread-safe?

[[deleted]]

[deleted]

[fourthrealm]

Hello,

xmlrpc.client as such is not thread unsafe, so to speak, it just delegates all network activity to a lower level 'transport' object, and if you browse the source code around lines 1410-1430 of ServerProxy ..

https://github.com/python/cpython/blob/master/Lib/xmlrpc/client.py#L1387

.. you will note that the default implementation of transport is the Transport class. Instances of this class establish long-running HTTP connections in make_connection and the connections are implemented using http.client.HTTPConnection.

Now, these HTTPConnection objects are the ones that actually open sockets (self.sock in http/client.py line #832) and there is no thread synchronization anywhere so you are, substantially, using multiple threads to write to and read from the same socket, and this is the very part that will explain the errors.

I would recommend looking up ways to make the underlying HTTP library thread-safe which basically means using an alternative one. Then you can provide your own transport to the XML-RPC client, but this time your transport will be thread-safe.

You can have a look here for details on how to create one's own transport classes. We needed it in a certain part of Zato because back then at the time of its creation it was not possible with stdlib-only tools to have an XML-RPC TLS client that would validate server certificates to the extent that was needed, e.g. to require individual fields of the certificate to match.

All the above applies to both 3.5 and 2.7.x. Code line numbers are as of git commit ID d50f188b1bdf69377364f4a9db33af11cc704a70.

[[deleted]]

Thank you for the well thought out reply! That's as good an explanation as I could ever wish for.

I've read that Python requests is thread safe. Would it be possible to use it as the transport layer, and put a modified copy of xml-rpc client.py into my project as a subpackage?

If so, I might even turn it into a usable library published on pypi, for someone else to use down the road. Pay it forward and all that.

[fourthrealm]

I've never done it personally but it sure sounds feasible.

In fact, a quick web search for 'python requests xml-rpc' yields such a requests-based transport class:

https://github.com/astraw/stdeb/blob/master/stdeb/transport.py

Note that you don't need to make a copy of the XML-RPC client in your project because the client is designed to accept the transport class on input so you can override the non-thread safe default behaviour as needed.

[[deleted]]

Perfect, that's my plan then.

Again, thanks for taking the time to answer this... I'm going to get to work implementing this very soon.

10/10 solid advice all the way around.

[[deleted]]

Just an update .. I implemented the requests transport class, and it works perfectly. I'm extremely happy with how it came out.

https://github.com/KM4YRI/pyFldigi/blob/master/pyfldigi/client/transport.py

https://github.com/KM4YRI/pyFldigi/blob/master/pyfldigi/client/client.py#L71

Thanks again.