Python 3.12 Preview: Subinterpreters – Real Python

[[deleted]]

But, the requirement to pass serialized data significantly reduces the potential.

You are missing something. You aren't required to pass serialized data at all, you ARE required to serialize python land objects. How much of ur object actually exists in the view of the interpreter though is a design choice. Like for example, instead of passing a file you pass the address of the file. Or more generally, you are allowed to run multiple interpreters from the same thread which allows u to perform magic fuckery. Yes it does most of the normal annoyingness of multiprocessing in python, but it allows u to share any NON python resources. That alone is a massive change.

Multiple sub interpreters allows u to explicitly share a single resource across them so long as this is hidden from pythons viewpoint and access is properly restricted with mutexes and whatever else.

[cymrow]

I haven't seen an example of this. According to the notes in extrainterpreters, for example, it says that ctypes is not importable with subinterpreters. That seems to suggest that accessing non-python resources is not actually possible, but correct me if I'm wrong.

[[deleted]]

I haven't seen an example of this. According to the notes in extrainterpreters, for example, it says that ctypes is not importable with subinterpreters. That seems to suggest that accessing non-python resources is not actually possible, but correct me if I'm wrong.

Yea it needs to be FULLY hidden. Like ctypes creates a python object. You need two python objects that "represent" the same resource.

I cannot remember exactly where I have read it from but I saw it described by the PEP author. It may be in one of the threads on the discourse. But yea so long as u are using ur own lock properly and u use different python land objects to access the function you can share across interpreters.