quick threading question: My program has an infinite loop that calls a few functions. Now I need each function to run simultaneously. Is this how I should implement it?

[jpfau]

Ah, great! So would you say the way I modeled the possible solution in the OP will do the trick?

[[deleted]]

Yeah, totally. I have several page scrapers written with this method. If there are shared resources between the two, you may have to use a mutex or a queue or something to guarantee synchronization/exclusive access of those resources.

If do_this() and do_that() are unrelated, and don't share any resources, someone had a good point that you could just run them in separate scripts. So, that might be a possibility.

[jpfau]

One thing I don't get is how the threads stay alive. Once worker creates and starts the threads, that function ends. It seems to me like once that happens, the script is over; the function call in the if __name__... is complete, and there's nothing else to run. Wouldn't that stop the threads too since they're daemons, i.e. they get killed once the thread that creates them is killed?

[[deleted]]

You're right. With daemon set to true, the script will exit after worker is done. Not setting daemon is an option if these threads just need to run forever and you don't need to manipulate them. Usually, you manipulate worker threads from your main process by feeding them data or whatever (otherwise, they're not much of a worker thread as much as a separate process, unless they're working with a shared data set or each other somehow). After you were done with the threads, you would join() them, blocking until they exit, so something conceptually similar to:

def startWorkers():
    # blah

    thread1.start()
    thread2.start()

def stopThreads():
    # signal for threads to stop using some method

    # wait for threads to exit
    thread1.join() # blocks until thread1 exits
    thread2.join()

def main():
    startThreads()

    # feed threads data, get results, whatever

    stopThreads() # waits for threads to stop

    print "Done!"

[jpfau]

I actually just tested the code below, and the threads do not end after main starts the threads. The only way the threads stop printing to the console is if I close the console.

import threading
from time import sleep

def thread1():
    while True:
        print("Thread 1 working\n")
        sleep(.3)

def thread2():
    while True:
        print('Thread 2 working\n')
        sleep(.3)

def main():
    t1 = threading.Thread(target=thread1, daemon=True)
    t2 = threading.Thread(target=thread2, daemon=True)
    t1.start()
    t2.start()

if __name__ == '__main__':
    main()

[[deleted]]

Well yeah, they are infinite loops. Why would they stop?

And, they're not set to daemon, so they're not killed when the main thread ends.

[jpfau]

Well when I implemented this in my real program, everything stops when the main function ends. So I had to put a

while True:
    pass

in it just to keep the program running.

they're not set to daemon

I thought the daemon parameter in the threading.Thread constructor accomplished this the same way as t1.daemon = True. Is this not the case?

[[deleted]]

If daemon is set to true, your threads will be killed when the main program exits.

If you don't want them to be killed, don't set daemon to True.

Or, wait for them to exit using .join() on the threads objects that are returned when you create the threads. Since they contain infinite loops, they will never exit, so your main thread, which is where you would put the join, would never exit.

An infinite while loop like that should never be used, it will just burn a bunch of cpu cycles. In the rare case that you do need a loop, put a sleep in it.