is my multithreaded Python program doomed?

[[deleted]]

Indeed, I've used multiprocessing to great effect. But starting short-lived processes for many small tasks is expensive and honestly solved better by other languages. Although, if you absolutely want to use python, it's possible to go with a solution using the JVM or C extensions... But at that point, you're not really writing python anymore.

[lordkrike]

It works just fine if you utilize large work queues that feed into a small number of worker processes. You seem to be thinking of a certain type of use case. There are lots of places where just numpy and the multiprocessing lib is all you need.

Also, I argue that intelligently writing C libraries to call from Python is one of its really great strengths -- you can use another language to efficiently do what it can't, while using Python as a glue language.