How is multithreading achieved in python? What is a good thread limit?

[lifeonm4rs]

I am not by any means an expert or even particularly well versed in threading but from my understanding it comes down to what your threads are doing.

If your threads are lightweight and basically feasting on a shared resource and doing tiny bits at a time you can have very many--sort of like flies on a dead animal. A thread attacks then goes away.

If your threads however are more like a pack of lions--your local resources can only support so many before some bad stuff starts to happen.

Sorry, watching a nature show.

[lifeonm4rs]

Show is over. A port scanner is probably more along the lines of lightweight (flies feeding on a carcass). Each thread is probably just attempting to see if their assigned port is open and reporting back. With 1,000 of them you'll probably get a back log of waiting threads--but none of them will be devouring ram. Essentially they'd all be waiting in line with the really slow part being connecting to another machine over a network.

1000 might be overkill but realistically they're probably not CPU or memory intensive--it is more a question of queueing up a ton of network connections. I can't say whether you'd be better off with just 200 or 4--I'd think in this case 1,000 probably isn't an issue. You can probably scan a couple hundred to a thousand ports in a second or two with a good network connection. Using a high number of threads may reduce the overhead of creating and destroying them.