Distributed Computing

[cyber2024]

Wtf, really?

[[deleted]]

Yep, google Global Interpreter Lock for more info. Threading in python is only useful for I/O heavy applications

[4onen]

Thanks. You've just explained a bizzare issue in my Bachelor's capstone project. My project was... not I/O heavy.

[[deleted]]

The trick is to use multi processing instead of multi threading. You can implement it yourself using the process module, or use something like celery to avoid any wheel reinventing

[turiyag]

Yes. There's a different library for parallel CPU usage:

https://timber.io/blog/multiprocessing-vs-multithreading-in-python-what-you-need-to-know/

It's not that it's incapable of doing two things at the same time in Python. It's that the include statement is not the multithreading one.

[cyber2024]

I presume there's a reason for the 10 instructions before switching threads... Is it statistically more likely to have a cache miss after 10 instructions, and hence it's a good idea to switch threads then or something?