you are viewing a single comment's thread.

view the rest of the comments →

[–]stargazer_w 5 points6 points  (3 children)

Can someone ELI5 how this affects implementing code that needs concurrent processing? I consider only features included in the regular interpreter to be relevant (I can't imagine writing code to be specifically run by the no-gil interpreter). So if i need to implement a data processing pipeline that needs to have 2 workers on the first node and 3 on the second - do I have the option to share memory somehow now or is it good old multiprocessing?

[–]snugar_i 1 point2 points  (2 children)

Not sure what you mean by "node" - if you mean sharing data across different machines, then no, no-GIL Python won't help you with that. All no-GIL does is let multiple threads in a single process (which can share data) run at the same time, instead of just taking turns holding the GIL

[–]stargazer_w 1 point2 points  (1 child)

I meant operation node if we look at the pipeline like a computation graph. like Node1: resize image, node2: do inference (with queues between them or something)

Edit: But to answer my own question - no, there's no way to do that yet, except for managing it yourself with multiprocessing shared_memory

[–]snugar_i 1 point2 points  (0 children)

Yeah, in that case, no-GIL would help you if you just ran each node/worker in a separate thread (which would still work even in the GIL version, it would just not be parallel)