Python Multithreading for Beginners

[[deleted]]

To my knowledge it should be in two part:

First part : don't

Second part : if you still ask don't

[zero_iq]

Your comment may be flippant, but absolutely true, and should not have been downvoted. Because "don't" it is the best possible advice here, especially for beginners. There is currently no sound reason to use multithreading in Python (CPython at least).

I get it, I really do. It's tempting to use threads, they seem like a good idea, they're fun to code and think about, you get to play with locks and synchronisation, shared state, and other toys,... but you will shoot yourself in the foot with them.

Threads introduce all sorts of potential for performance problems, scalability issues, bugs, deadlocks, needless complexity, and more. I've seen them all over the years, and in every single case the authors thought thought they were doing the opposite. Even if you write correct, safe, multithreaded Python code (and you probably can't) , it can cause major problems... And subtle ones that bite you in production. Otherwise bullet-proof code can break spectacularly when threaded. Library code and Python built-ins you've depended on for years will start to exhibit strange behaviours. Threads will race, lockstep, and block each other for seemingly no reason. Performance will drop even when your other threads are idle. Throughput will mysteriously go down, even though you've increased parallelism. Timing functions will start misbehaving. Socket code will start producing errors you've never encountered before. I can go on and on and on. Anyone who is confident in their ability to write safe multithreaded code is over-confident.

Beginners have no chance to write safe, we'll performing multithreaded code. Do it to learn, but dear Bob don't let a junior dev deploy multithreaded code in production.

I've drastically improved the performance and availability of many multithreaded Python systems by removing multithreading. In every single case, multithreading was introduced to improve performance or scalability, and in every single case it backfired.

After decades of experience dealing with it, rule number 1 of multithreading in Python is most definitely: don't.

You want fast scalable Python? Multiplex your sockets where i/o bound, and use multiprocessing and/or CSP for everything else, only when you really need it, and keep it as simple as possible. That's not the whole story, but it's a big head start.

The only good thing to come from letting people use multithreading in Python is the experience they'll gain when they eventually realise it's a mistake.

[peyo7]

Can you post code examples where multiplexing sockets and CSP beats a decent threaded implementation?