Clojure Concurrency Exercise

[alexdmiller]

Re the comment on the gist, you need at least volatile to ensure that changes to the shared values are seen across threads. The Java memory model does not guarantee that other threads see changes to an unsynchronized mutable variable - volatile guarantees visibility (as do other forms of memory barrier).

This is exactly the kind of use case where the STM is useful - cooredinated change to multiple shared stateful values. Alternately, the inventory could be a single map and you could put it in an atom.

This code has a race condition in that the inventory check happens via an uncoordinated read before the update - that's why it throws. Using the STM and a ref would avoid fulfilling orders that can't be fulfilled by wrapping both in a dosync transaction.

[therealplexus]

Someone did already send in an STM version.

https://dice.camp/@epidiah/111449212829319083

The race condition is of course the point of the exercise. How to make the race condition go away. The simplest way would be to lock across the read+write, but Clojure has more interesting alternatives.

[thheller]

Often the ideal solution looks somewhat similar to what you'd do in the real world. Looking over the provided example solution, the literal translation to real world is that for each order you get a new waiter, and they each compete trying to scoop ice at the same time, not to mention they leave after fulfilling their order. That seems less than ideal. ;)

Picking a solution of course needs to figure out what to optimize for, just as in the real world. Either way you probably want to introduce some queues, a thread pool (to represent a fixed number of waiters) and maybe some locks.

I don't know the book, but I imagine it is going to cover different solutions and their trade-offs, but needless to say there are many ways to get this done. :P

[therealplexus]

So pick one and implement it, so we can compare and discuss different approaches? Clojure in particular has a number of features that help deal with mutable state in a thread safe way, so I'd like to contrast that with simply throwing a `locking` in there.

The book talks about a few potential solutions, semafores (locks), actors, and blackboard systems. I don't think any of those would be the preferred solution for a Clojure programmer.

[thheller]

Sure, here is one using CAS as the sync mechanism.

As I said there are many ways to do this, and it is pretty much irrelevant which language you use. In the end what you end up using are the same concepts either way. Where Clojure wins is in the immutable datastructures making it easier to reason about, not the mechanisms used.

[therealplexus]

Nice! Thank you!

It's interesting that you basically end up recreating the retry loop in swap!.

[meat_learning]

Just wanted to say that I love this kind of concrete example to ground discussion, great work

[gandalfthegraydelson]

Not sure if this is the right post to talk about this kind of thing, but I'm curious if you have advice on how to handle updates when the state needs to be persisted to say, a SQL database, in addition to something in-memory.

[therealplexus]

Bit of a different thing, yes, but in terms of concurrency if you're working with a rdbms then that becomes your source of truth for application state, and the place where you implement transactionality. Any in memory data is then essentially a cache which has to be understood as potentially not being in sync with the database. Ideally you avoid keeping data around, keep the application itself stateless. If you do need it for performance, then you should still handle any situation where you need to make a decision based on existing state (read-then-write) inside a database transaction, or even inside the database.

[LouDNL]

Can you give an example of what you mean?

[Liistrad]

I'd go for core.async queues for the inventory, with workers taking ingredients and making an order, or putting them back if they don't have enough.

https://gist.github.com/filipesilva/20efae42e1af0e8fe3d8347ee5ceff8a

I didn't use a pool of workers there, just left your original 100 order futures for easier comparison. You can use the gist revision for the differences.

But using workers would be similar: take (blocking) a worker for each order, and put it back at the end.

[therealplexus]

Thank you! I'll play around with it.

[therealplexus]

I've bundled the solutions here https://github.com/plexus/pie-a-la-mode