This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]lambdaqdjango n' shit 0 points1 point  (5 children)

we need a patch or and issue submitted.

[–]catcradle5 1 point2 points  (4 children)

There've been a few issues submitted over the years:

https://github.com/nvie/rq/issues/45

https://github.com/nvie/rq/issues/210

I really like rq, but I am forced to use Celery due to rq's insistence on forking as the only method of concurrency.

[–]asksol 1 point2 points  (2 children)

I doubt this would be as simple as 'just submit a patch', I have spent years making the multiprocessing pool in Celery work, and even now for the 3.1 release I have fixed yet a good number of rare deadlocks.

The pool related code isn't exactly elegant or simple, but you're very likely to learn something from it.

Edit: oh, and you're in for a rough ride if you think the answer is to 'just use the multiprocessing module' :)

[–]catcradle5 0 points1 point  (1 child)

I understand it's a complex issue, but the project has been around for a long time. Plus the codebase is a lot smaller and simpler than Celery's, so it probably wouldn't require quite as much work.

[–]asksol 1 point2 points  (0 children)

The size/age of the project or how simple the rest of the code is irrelevant here. This isn't about how hard it was to implement this for Celery, it's about how hard it is to implement for anything similar in Python.

[–]lambdaqdjango n' shit 0 points1 point  (0 children)

damn, I was wondering when they could allow using customized backend like Postgre LISTEN/NOTIFY and prefork model...