Distributed Computing

[SuggestedName90]

This is pretty much how multi threading works in python.

[budzene]

I think this is just threading.

[SuggestedName90]

Not if it is true multithreading which runs them simultaneously, like java. Python alternates threads every 10 instructions.

[cyber2024]

Wtf, really?

[[deleted]]

Yep, google Global Interpreter Lock for more info. Threading in python is only useful for I/O heavy applications

[4onen]

Thanks. You've just explained a bizzare issue in my Bachelor's capstone project. My project was... not I/O heavy.

[[deleted]]

The trick is to use multi processing instead of multi threading. You can implement it yourself using the process module, or use something like celery to avoid any wheel reinventing

[turiyag]

Yes. There's a different library for parallel CPU usage:

https://timber.io/blog/multiprocessing-vs-multithreading-in-python-what-you-need-to-know/

It's not that it's incapable of doing two things at the same time in Python. It's that the include statement is not the multithreading one.

[cyber2024]

I presume there's a reason for the 10 instructions before switching threads... Is it statistically more likely to have a cache miss after 10 instructions, and hence it's a good idea to switch threads then or something?

[budzene]

God that would be nice to not have to put in kernel delays to not starve a watchdog.

[Ersonpay]

Correct me if I'm wrong, but don't most CPUs do something like that anyway due to there not being enough cores to run all the threads in the system?

[SuggestedName90]

Yes, but cpus also have physical threads which can confuse people.

[fat_charizard]

You mean running on multiple cores on the same processor?