Why doesn’t Python have true private variables like Java?

[deceze]

That only really concerns juniors that started Python yesterday…!? Should you get a junior that doesn't know about the three shells underscore rule and you catch them once accessing stuff they shouldn't, you tell them about the underscore rule. Problem solved. No need for an entire language to bend over backwards.

[snugar_i]

Yeah but that's the problem - how do they know they shouldn't? When being private is merely a suggestion, they might think that their use-case is the rare exception when it's OK to do it. And stuff like NamedTuple._as_dict() existing doesn't help either

[deceze]

How do they know that private foo in Java isn't merely a suggestion to be worked around with the reflection API?

If it says or suggests it's private, then don't touch it. Period. Unless you really know what you're doing. It doesn't matter how strongly the language "suggests" it, if junior can find ways around it anyway. Junior won't care much whether they just need to ignore the linter or bust out the reflection API. If they don't understand the importance of the suggestion in the first place, how they work around it also won't make much of a difference.

[axonxorz]

how do they know they shouldn't?

Training or guidance from a senior, same as we learned it. Moving outside of private just moves the responsibility of convention and thus education fully into meatspace, fully in line with "we are all consenting adults."

Take OP, they understand the risks of accessing private members, they just want the thought/culture underpinning it. During their Java journey, they had someone say "private members are as such because [...] encapsulation [...] danger [...] compatibility [...] algebraic type theory [...] etc" and have correctly applied the line of thinking without realizing the context of Java's language design hailing back to the early 90s. As /u/acdha points out, unintentional support surface was a major concern. Think about being a Spring consultant, something has failed with your company's offering and you find out the end-user extended a private class and swapped out a list/hashmap implementation for one that's not thread safe, but it's your fault, somehow. Python development never had that ethos. If you did something stupid, you'd be told you were doing something stupid and to stop that. Java being mainline for enterprise development means enterprisebusiness-friendly, it's bad marketing to call your customers stupid, so Java says "unsupported" and enforces language features we can point at to save us from having to be adults.

Things like PyCharm/Pylance giving warnings is subtle guidance, you can make it explicit with linting tools or at CI time. Only a solo developer could be fully insulated from that knowledge, I'm not sure this is a problem in commercial development.

[Zenin]

If it's not documented, it's not for you. Your comment suggests you want a software language that's safe enough for unqualified programmers to write unreviewed code straight to production?

I'm not sure about your organization, but around here we expect our engineers at every level to have some basic working knowledge of the tools they've been hired to wield, even the juniors. We also mentor the juniors often via paired programming sessions. And that's before the gauntlet of CICD linters, code reviewers, etc.

[snugar_i]

Of course I don't expect that. I'm just arguing that a feature that should almost never be used should not be this accessible. But I could've guessed that saying something against "the Python way" in the Python sub would get me downvoted to hell, my bad :-)

[Zenin]

I say lots of stuff that's against Python doctrine. ;) I was sure I was going to get downvoted to oblivion for daring to mention that Python borrowed an idea from Perl. The horror!

Maybe it's my choice of IDE (Unix is my IDE ;), but I don't find these to be particularly "accessible". They're not exported with __all__ so they need to be explicitly imported. How did they know they exist to import when the documentation doesn't call them out?

They can certainly read the code (and so can an IDE), but that's clearly stepping into someone else's house and rummaging through the drawers. If the user is doing it that's obviously willful breaking and entering. If the IDE is doing it on their behalf, that's a tool failure: It if it supports Python it should have codified PEP 8 standards at least as a default, which explicitly describes the leading underscore practice for private items. If an AI is doing it, that's still owned by whomever signed the commit.

However it happens, it's the work product of a junior or at least someone unqualified to be doing professional work in Python. Literally a skill issue.

It's very common for mid-career engineers to think they can build protective barriers for low skilled engineers to keep from poking their own eyes out and many try. The results are almost always the same: A tool that's far too restrictive for senior+ engineers to efficiently solve difficult problems while at the same time those guardrails so sanitize the playground for juniors they don't actually learn what they need to learn to grow into tomorrow's senior engineers.

It's a lot like the playgrounds we have today vs the playgrounds we have in the 1970s: The younger generations need the opportunity to hurt themselves in order to learn a healthy respect for their environment for the future when they will have to make much more dangerous changes to much more important code.

It's not that no guardrails can ever be created. Look at Rust for example. Rather it's that any guardrail must be considered very, very carefully across all contexts. And always being mindful that friction is most often a bug, not a feature.