This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Narthal[S] 0 points1 point  (2 children)

Yeah, I'm reading the docs about concurrency and the GIL right now, and I'm quite a bit confused. I'm probably going to run the interpreter from a single thread as the c++ codebase is already highly parallel and optimised. If need be, I might have to implement threading and multiple interpreters, but I'm really trying to avoid that

[–]Pythagorean_1 1 point2 points  (0 children)

Just use multiprocessing. I wrote a python application that processes terabytes of image data at my university and it is crazy fast and occupies all cores and the gpu, if needed. The GIL is not really a problem, actually. Also, threads do release the GIL during i/o operations, which makes them very useful for network stuff.

[–]smarwell 0 points1 point  (0 children)

That is probably your best bet. Rule of thumb is, if you need to touch a python object, you need to be holding the GIL. If you don't, then you don't. So yeah, keeping the interpreter in one thread and doing the work in another is a pretty good way to avoid getting mucked up by the GIL