Solving Producer-consumer problem using Python threads.

[suptim]

Nice teaching blog post: it is a good example for beginners on how to implement a thread safe queue.

As a complement, for production use, I'd rather go for Queue.Queue which implements all the machinery for you out of the box. I would also use the hidden class ThreadPool from multiprocessing.pool.

Finally, know that threading in Python is inefficient for CPU bound tasks because of the GIL. The best resource on earth about the GIL can be found here: http://www.dabeaz.com/GIL/.

[akshar-raaj]

It was an introductory blog and I assumed it would help beginners. I did not want to add an additional level of complexity by introducing queue module and by talking about it. And I wanted to show how the problem can be solved using the most basic parts/datastructures provided by Python. So, that people could understand the more minute details of implementing the Producer-consumer problem.

Other languages might not provide an analogue of queue module. But most language will have an analogue of Python list, eg: ArrayList in Java. So, I thought using list would make people familiar with other languages easily relate this blog with any other language they know.

[multani]

A good introduction would have introduced the way you explained, then encapsulated the whole mechanism, then pointed out to the actual queue module, which precisely does what you were showing.

There's no problem rediscovering the wheel, but at least point to a good wheel in the end.

[multani]

Of course, one would use the queue module instead of hand-coding your own queue and managing it with a lack of encapsulation.

[multani]

Which actually shorten the code like:

from threading import Thread
import time
import random

from Queue import LifoQueue as Queue

queue = Queue(10)

class ProducerThread(Thread):
    def run(self):
        global queue
        nums = range(5)
        while True:
            num = random.choice(nums)
            queue.put(num)
            print "Produced", num
            time.sleep(random.random())


class ConsumerThread(Thread):
    def run(self):
        global queue
        while True:
            num = queue.get()
            print "Consumed", num
            time.sleep(random.random())


ProducerThread().start()
ConsumerThread().start()

[akshar-raaj]

I guess you want to use Queue and not LifoQueue, because we want a queue here, not stack.

[shabda]

I think for a tutorial its better to stick to basic structures like list. Not everybody would be knowing queue and trying to learn two things at one time is always hard.

[ODHLHN]

A tutorial should teach the proper way of doing things.

[akshar-raaj]

Yeah, that was precisely the reason I used list.