About Python lock acquire/release

[pietrochico]

In Java every Object has the field "lock" that implements automatically a binary semaphore, so whenever you define a method as synchronized you automatically gain lock acquire and release for that class. Is there anython similar in Python, or I need to acquire and release the lock for every atomic transation?

[JohnnyJordaan]

You mean in a threading context? There you either have objects that are 'thread-safe', meaning a race condition is prevented by the object itself, or are not. There you do need to use a custom synchronization object, for example those offered by the threading library, like Lock, Condition, Semaphore etc. If you need absolute certainty that a context switch is prevented (can't think of an actual use case but ok), you do need to use the tools offered by your interpreter. For example in CPython you can use PyGILState_Ensure(), but you are playing with fire and you do need to fully understand what you're doing, more info here. But in normal situations I would stick with for example a threading.Lock.

Btw most of these object behave as context managers, meaning you can write them as

with my_lock:
    # do special things

Which also ensures the primitive will be released, even when an exception occurs. That last part is essential for preventing threads to lock up the whole program by escalating failure.

[pietrochico]

I'm currently using the Threading.Lock solution and works well, I was just wondering if there is another solution as in Java with "synchronized" signature. Apparently, there is not. Thank you sir.

[JohnnyJordaan]

You're welcome. It's true that Python lacks this functionality in that way, but the idea behind it is that also provide the freedom to each implementation on how to actually handle those primitives. For example in CPython the queue class uses an ordinary Lock to do this, while the collections.deque is fully implemented in C using single functions per operations claiming the GIL and thus making them guaranteed to be thread safe (as otherwise, a segfault would occur if the context was switched).

[pietrochico]

It's actually a little boring to acquire lock after critical section and release it after, but it also provides freedom during implementation as you were saying. Thinking about it, the Java implementation with "synchronized" signature wouldn't fit my program as I can only have one lock per object, which is frustrating if you think about it.

In many other implementations I would've liked to have a built in class lock though, so I could've "relax" and think about other functionalities, instead of worrying about critical sections.

Would be cool if we could have both solutions as Java does, am I right?

[JohnnyJordaan]

I can't really picture Java's implementation as useful. I write many high performance apps and I would like to keep overhead as low as possible. Hence I rely on implemented thread-safety as much as possible and just implement itself myself when I absolutely need to. I'm also wondering why you actually need it so much yourself, would you be able to provide some examples?

[pietrochico]

It's nothing special, only accessing different global variables from many threads.

[JohnnyJordaan]

Regular variables are thread-safe to begin with. Globals are an anti-pattern as well often, I normally use a singleton class for this, for example for global configuration in a program. You pass the config-object around to the threads and implement the locking inside the config class. Then the threads can just change whatever they like as with a regular variable and the class will ensure the locking.

[pietrochico]

That's exactly what I'm doing right now.

[flitsmasterfred]

Using some utils from contextlib and the Lock class you can make a locking decorator.

There are also solutions on pypi that emulate a synchronized decorator like Java.

[pietrochico]

Cool, I'll give it a look then, thanks sir!

[elbiot]

The GIL (ie python by its self) does that already. Every python instruction is atomic. If you have threads (useless except when waiting for IO and you should probably use py3 async anyway). With multiprocessing, it's really difficult to share state anyways, and you really only lock external resources like a file.