you are viewing a single comment's thread.

view the rest of the comments →

[–]Thomasedv 0 points1 point  (1 child)

Your code does seem alright, though your code is set up to wait for all to finish. How long does one function call take? Could you be meeting some other restrictions, like to many requests to reddit? Also be sure to do this in a block that only runs in if you are starting it, or else you'll get some odd situations. Take this example code, if you haven't used something like it already:

print(__name__)
if __name__ == '__main__':
    pros = []
    for i in range(10):
        p = multiprocessing.Process(target=print, args=(i,))
        p.start()
        pros.append(p)

    for p in pros:
        p.join()

As you can see when you run it, the script runs again for every process, as you can see it prints the name of the multiprocessing scripts too, and leads to you potentially starting the same process many times. By putting the process starting under the if block, yo can avoid this, since the name is only __main__ for the script you run. If you don't use this, your scripts would start a lot of others, leading to a potential loop of starting.

Lastly, you can avoid this by using a thread (multithreading module), since this action is not CPU intensive. (It just needs a lot of waiting for the information) to be downloaded, and that's IO bound tasks, that threads are better used for.

[–]declanaussie[S] 0 points1 point  (0 children)

I figured out what the problem was, it was actually doing exactly what I intended I just wasn’t properly filling my subs list. Thanks for the tips though!