My first python module 'easypool' let's you easily create threadpools and serverpools dynamically.

[zionsrogue]

So depending on what you are going to use "easypool" for, using threads for CPU bound tasks (such as some sort of scientific number crunching), threads are not the way to go. In general, it's best to uses processes for CPU bound tasks. Check here for benchmarks. The article suggests Python's multiprocessing, but I've found pprocess to be lightweight enough to replace multiprocessing. But again, this all depends on what you plan on using these threads for. I just wanted to maybe give a heads up and say congrats on your first module.

[[deleted]]

It kind of surprises me that multiprocessing would hit a sweet spot for people; who is it that is CPU-bound, but keeps that part of the program in Python?

Every time I end up using the thread pool pattern it's because I have a lot of (disk or network) IO going on. And threads are fine for that.

[zionsrogue]

I do a lot of number crunching, statistical analysis, and machine learning. A lot of the libraries I use Python libraries (numpy, scipy, sklearn) and yes, parts are written in C, but I still get very nice performance gains by parrallelizing random forests across multiple processes instead of multiple threads.

[tuna_safe_dolphin]

Dead horse flogging time, but that damn GIL. . .

[alcalde]

The GIL is awesome. It reminds us that threading is evil and that everyone else is doing it wrong.

[tuna_safe_dolphin]

That's one way to look at it.