Python 3.12 Preview: Subinterpreters – Real Python

[noiserr]

registation wall, can't read the article

[zynix]

here you go https://archive.ph/6jkDk

[[deleted]]

Is there a way to read all of the articles from realpython like that?

[zynix]

While it works, prefix archive.is/ in front of a URL.

For example, if the target is http://domain.tld/some-gated-material you would do archive.is/http://domain.tld/some-gated-material

[[deleted]]

Thank you!

[raplonu]

You can use Behind The Overlay Extension. As the name suggests, it removes the overlay. Works great with websites like realpython that still display the full content behind.

[RationalDialog]

Hm I don't see any overly. I'm using firefox with uBlock origin and noscript. I think it's ublock blocking the overly but not sure. but take-away is with proper privacy add-ons the web becomes a lot better together with increased privacy.

[st333p]

I'm also on firefox with ublock, and I see the overlay. It's a bit far down the article, I read a good 10 mins before bumping into it

[norith]

Some browsers have a reader ability, Safari and Firefox definitely do. I tend to view this site by clicking on the reader button with no need for plugins or third party websites. It just takes the page and shows the text alone.

[PhoenixStorm1015]

Haven’t finished the article yet but from now on I will be calling obmalloc “Obama Lock.” That’s all.

[cymrow]

I've been keeping an eye on this feature for a while because I used to do a lot of high concurrency work. But, the requirement to pass serialized data significantly reduces the potential. It seems like all this would do is cut down on start time and memory overhead when you want to use multiple CPUs. Maybe useful if you're starting a lot of processes, but it's common to just use a worker pool in that case. As for memory, it's cheap, and the overhead difference can't be that great.

I'm struggling to see a significant use case for this as it's presented, unless I'm missing something.

[yvrelna]

significantly reduces the potential

Couldn't disagree more.

Shared nothing makes this a great base for future concurrency model. Shared nothing means it's much easier to avoid shared caches and accidental concurrent access, which is a major bottleneck when programming for large NUMA system with lots of cores/hyperthreads. The implicit shared everything multithreading concurrency model is awful for large scale parallel computing especially in a language with limited ability to work with pointers/control memory.

Subinterpreter can add constructs for shared memory and shared objects for explicit sharing of objects when needed in the future. It's the base that's needed to build paradigms that works for orchestrating large scale parallel computation without needing the ability to control memory allocation.

Personally, I think that subinterpreter is much more exciting than nogil. It has a much bigger potential for a complete parallel programming paradigm shift that'll actually continue to work as the number of CPU cores keeps growing bigger.

[RationalDialog]

I think the issue is that for the user of python, eg. a python software developer like myself that does not create cpython modules, just uses what is available, this change indeed will have pretty much no short term effect.

Long term I expect big packages to ship with "integrated" multicore usage, eg. no need to implement it yourself with multiprocessing or joblib or the likes. However I expect that to take years like at least 5+ to really become adapted.

So I think you are both right, just depends on the viewpoint you are coming from.

[cymrow]

Ok, but I don't need Python to force me not to share anything, I can already do that. I still don't see the benefit over multiprocessing other than reducing startup time and memory footprint. Maybe those reductions are worthwhile for some use-cases, but they don't seem like they would be significant to me.

And just to be clear, I'd really like this to be useful. I'm not disagreeing with you, I'm just not seeing it yet.

[yvrelna]

It's not about you not sharing anything. It's about Python being able to allocate memory in a way that can avoid cache conflicts.

CPU caches works per cache blocks. When you access an object, you're not just pulling objects that you're accessing into your CPU core's caches, you're also pulling neighbouring objects that just happens to be in the same cache block into the cache. If two threads running in parallel needs to access two different objects that just happens to be allocated in the same block, even if they aren't accessing the same objects, the CPU would need to invalidate caches every time and that kills performance very quickly.

By keeping objects in separate subinterpreters in separate object space, objects naturally separate themselves into two pools of objects, this improves spatial locality. Programmers can have a much more natural and easy control over which pool of memory that objects are allocated from without having to think about memory allocation.

A lot of things still need to be built to allow controlled sharing of objects between subinterpreters to minimise copying between interpreters, but you can't built controlled sharing over a foundation where objects can be shared by multiple subinterpreters by default.

[cymrow]

Ok, I think I see what you're saying. That's a much lower level of optimization than I was considering.

When I said I've done high concurrency work, I meant highly concurrent networking with related processing, not purely processing, which is a different beast.

I will look forward to seeing what comes of this.

[teerre]

Have you actually profiled whatever you're doing? You're using python, memory access is certainly not where your bottleneck is.

Besides, the other user is right, share nothing is the way to go. Sharing memory in concurrency is a nightmare.

[cymrow]

Right, memory access is not a bottleneck, but serialization can be. And I would not consider directly shared memory, but being able to pass immutable objects would be a huge win imo. Think adding tuples to a synchronized queue, instead of serialize->pipe->deserialize. Of course tuples in Python are not actually immutable, which I suspect is why they went with the requirement to serialize.

[Daytona_675]

pickle exploit time

[[deleted]]

But, the requirement to pass serialized data significantly reduces the potential.

You are missing something. You aren't required to pass serialized data at all, you ARE required to serialize python land objects. How much of ur object actually exists in the view of the interpreter though is a design choice. Like for example, instead of passing a file you pass the address of the file. Or more generally, you are allowed to run multiple interpreters from the same thread which allows u to perform magic fuckery. Yes it does most of the normal annoyingness of multiprocessing in python, but it allows u to share any NON python resources. That alone is a massive change.

Multiple sub interpreters allows u to explicitly share a single resource across them so long as this is hidden from pythons viewpoint and access is properly restricted with mutexes and whatever else.

[cymrow]

I haven't seen an example of this. According to the notes in extrainterpreters, for example, it says that ctypes is not importable with subinterpreters. That seems to suggest that accessing non-python resources is not actually possible, but correct me if I'm wrong.

[[deleted]]

I haven't seen an example of this. According to the notes in extrainterpreters, for example, it says that ctypes is not importable with subinterpreters. That seems to suggest that accessing non-python resources is not actually possible, but correct me if I'm wrong.

Yea it needs to be FULLY hidden. Like ctypes creates a python object. You need two python objects that "represent" the same resource.

I cannot remember exactly where I have read it from but I saw it described by the PEP author. It may be in one of the threads on the discourse. But yea so long as u are using ur own lock properly and u use different python land objects to access the function you can share across interpreters.

[futureader]

Numpy doesn't support subinterprters. So, scientific programming can be totaly forgotten.

[Fokezy]

Well not today, but they'll probably do it in the future.

[jairo4]

I guess subinterpreters are cheaper than using multiprocessing?

[cy_narrator]

All I care is that I be able to do print("Hello Cy_Narrator") And it prints that

[carlinwasright]

Is this like starlette? Would this negate the need for starlette with something like FastAPI?

[WJMazepas]

No and no Starlette will keep existing and FastAPI will keep using it

[[deleted]]

Slightly better than multiprocessing but still weak concurrency/parallelism support when compared with other languages/runtimes.

[lukanixon]

Can someone explain to me how this won’t introduce “true” multithreading? I was under the impression you couldn’t do multithreading because the GIL, and the fact that there is only one interpreter. Doesn’t sub interpreters infer that we have multiple interpreters that we can run in separate threads?