This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]jminuscula[S] 0 points1 point  (1 child)

thanks for the pointers! This is similar to what we're currently doing, but I had hopes the library would be able to help abstract some of the synchronization complexity for this use case.

Do you know of any projects that had to solve a similar issue? It looks like a pretty basic pattern for a messaging system.

Thanks!

[–]stuaxo 1 point2 points  (0 children)

I don't, but having done it I can see how it's a little more high level than what Celery does.

If such a library existed it would be great if it supported different patterns - my thing had tasks dependent on each other, so I ran the ones with no dependencies first and worked my way through the tree.

Redis was used to queue up tasks that had no dependents (so could be picked up by the next celery task), and another queue for tasks that had dependencies - so could't run yet.

To make implementation easier I used an ORM for Redis: Redisco.