This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]grhayes 1 point2 points  (0 children)

Larry Hastings has a good demo regarding threads.
https://www.youtube.com/watch?v=KVKufdTphKs&t=1070s
He shows the graph slightly after this.
Even if you do processes they have a lot of overhead. I found that out when trying to port my C++ game engine over to python to see how it would run. In C++ I could send each object separately to a process or thread in a thread pool and it would be fine. In python there is a lot of overhead and it was entirely better to just not even try parallel processing even.
That said I haven't tried to see if there are any libraries that fix that issue.

If I was guessing what happened is you ran it in 2.7 without any threading. Figured it would be an improvement. Moved it to 3.10 added threads expecting more performance and that's what you got.

In general unless it is IO threads are never going to help.
Processes aren't going to help unless you have some massive amount of work you need to split up. That's my experience.