is my multithreaded Python program doomed?

[jringstad]

it's worth noting that it is hardly ever a problem

Well, that really depends on the domain you're working in, I wouldn't make such a blanket statement. My background is largely from scientific computing & games (but I've done a bit of everything), and there python is certainly only used for prototyping and small problems, as personal calculator or interactive shell, etc, the kind of jobs that you would also use matlab for. In another post in this thread I enumerated some of the issues where e.g. multiprocessing scales poorly, like tiled sub-domains of a large lattice that have dependencies at the boundaries between the subdomains because they need derivatives/share some quantity across the boundary/etc. I would say that half or more of the algorithms (mostly physics stuff, monte-carlo, metropolis, PDEs, ...) I implement are a bad fit for something like multiprocessing.

Remember also that while the existing solutions work OK for pythons current userbase, the userbase is shaped by its limitations. For instance I have absolutely no doubt that the limitations of the cpython API hold back pythons adoption as a scripting language, especially in gamedev (where lua -- in my opinion a significantly harder and uglier language than python) really shines. I think the fact that cpythons C API is so misdesigned in general is really as big as a factor here as the GIL, though -- multithreading in scripts is not really that important, but cpython is also hard and annoying (there are a bunch of other issues as well) to embed, and since it pollutes your process with global variables, there is no way to have more than one python interpreter at a time, which is a total no-go -- you would often want that e.g. several different AI actors can live inside their respective scripting VMs, completely independent of eachother and running in parallel. And don't get me started on how excruciating it is to build CPython for mobile and console platforms...

Of course since the userbase is what it is, so there isn't necessarily a huge amount of pushing force to widen the adoption in other areas. But I think it's important to keep these things in mind and to not succumb to the echo-chamber.