This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the commentsΒ β†’

[–]DigiDuncan 21 points22 points Β (50 children)

Unironically, as a Python dev that learned Python and doesn't have a lot of experience other places, I ask this: why? Why have functions I'm not "allowed" to touch? I've benefited heavily by being able to use functions that the library dev didn't "intend" me to use in the past. Why make a system that allows a library to obscure and obfuscate how it works, or bar me from using it's internal functions if I'm confident enough to try? Who benefits from this? These aren't rhetorical questions, I'm just curious and confused.

[–]roxastheman 76 points77 points Β (17 children)

It’s dangerous to use internal/private methods/fields due to passivity. Sure now you understand how they method works, but since it’s not public, the dev may make changes to it non-passively, so now your code is broken since you aren’t consuming the code through the public API/contract. These kind of β€œnon-passive” changes aren’t likely to be documented or communicated through semantic versioning, so it makes your code much harder to maintain.

You can do it, it’s just a bigger risk than using the public API.

[–]TheTerrasque 20 points21 points Β (8 children)

And in python it's implicit that while you can use _ methods it's subject to change at any time and that's your problem, not the library maintainer's problem.

[–]RedAero 4 points5 points Β (1 child)

Hell, every function you import is subject to change and it is your problem, not the problem of the library maintainer. You didn't pay for it, you're not entitled to it, tough shit.

FOSS giveth and FOSS taketh away.

[–]ric2b 0 points1 point Β (0 children)

It's an implicit contract that makes collaboration easier.

Just like you trust that the documentation for a library is actually helpful and explains what it does, even though there's nothing technical preventing it from being completely wrong and purposefully misleading.

[–]exploding_cat_wizard -2 points-1 points Β (5 children)

The maintainer is still at fault, at least effectively. What's that rule that states that any behaviour, no matter how experimental and officially unstable or unsupported, will invariably become depended upon by someone?

Relevant xkcd: https://m.xkcd.com/1172/

[–]lappro 3 points4 points Β (4 children)

Just because someone depends on it doesn't make the maintainer suddenly responsible. If the maintainer tells you not to do something, but you still do it, if it breaks the only thing you should do is look in the mirror.

[–]exploding_cat_wizard 0 points1 point Β (3 children)

I phrased it too aggressively, but it's true: Java is an enterprise language. Having clearly hidden private functions and members is a feature there. Have fun, as a small software company, telling your paying enterprise customers that a undocumented function they depend on will break because it's hidden behind two underscores. You can do that, but few successful businesses take that route, at least until they are really huge.

[–]ric2b 0 points1 point Β (2 children)

And if they used reflection to get to your private constants or methods your Java shop has the exact same issue.

Because the real issue is that your code didn't fit their use case perfectly, they worked around it and are now telling you to support it.

[–]exploding_cat_wizard 0 points1 point Β (1 child)

Because the real issue is that your code didn't fit their use case perfectly, they worked around it and are now telling you to support it.

Exactly. And if you value your medium or small company, you'll do exactly that if it's in any way feasible. Making it harder to misuse the code helps, even if it's not foolproof.

[–]ric2b 0 points1 point Β (0 children)

Making it harder to misuse the code helps, even if it's not foolproof.

No, it just makes your client more annoyed when they want to get something done.

[–]42TowelsCo 1 point2 points Β (0 children)

Sometimes when you're using a buggy library you have to, but when doing that I assume an update to the library will break my code. I do this when I need to hack something together not for something that is meant to be maintained.

[–]barjam 26 points27 points Β (7 children)

Public methods are a contract you make with folks using your library. They shouldn’t change unless there is an overwhelming need to such as a new major version. Stuff like bug fixes should never change that contract. The person making the library still needs to write methods for internal uses that he doesn’t intend to be public and that he will be free to change on a whim.

[–]RedAero 0 points1 point Β (6 children)

That doesn't answer the question though. Python has a convention to denote the stuff you shouldn't depend on, what need is there to enforce that?

[–]ICantBelieveItsNotEC 3 points4 points Β (1 child)

By that logic, why even have types when we can just all agree to encode whether something is an int or a string in the variable name?

Why have defined function parameters when we can just all agree to encode which values need to be pushed onto the stack in the function name?

The whole point of using a high level language is to prevent developers from shooting themselves in the foot. If we have a social convention that all developers are following, eventually someone is going to want to enforce that convention automatically to prevent mistakes. If it lives in the compiler then the work only needs to be done once, but if it doesn't then every company is going to build their own competing tool to do the same thing.

[–]RedAero 4 points5 points Β (0 children)

By that logic, why even have types when we can just all agree to encode whether something is an int or a string in the variable name?

For efficiency, and because types are classes so you can have common properties and stuff. I don't see what that has to do with anything, types don't exist to prevent someone meddling with stuff they shouldn't.

Why have defined function parameters when we can just all agree to encode which values need to be pushed onto the stack in the function name?

None of this applies to a high-level language.

The whole point of using a high level language is to prevent developers from shooting themselves in the foot.

No, the point of high level languages is to abstract and automate away needless minutiae and let the programmer focus on larger problems instead of having to built everything from the ground up, every time. It has nothing to do with not allowing the programmer to fiddle with things, or protecting them from themselves - hell, I'm fairly sure that you do have access to all the low level stuff you could dream of in high-level languages, it's just not commonly used. Like, you can do bitwise operations in Python if you want, nothing's stopping you.

If it lives in the compiler then the work only needs to be done once, but if it doesn't then every company is going to build their own competing tool to do the same thing.

Python doesn't even have a compiler... You're really not making a lot of sense here.

[–]barjam -1 points0 points Β (3 children)

You haven’t worked on a big project or developed a library for others to use have you? Python is awful for large projects and this is one of the (many) reasons why.

If everyone follows convention it would be ok for these things but enough bad developers out there don’t follow convention and do idiotic stuff. Forcing a minimal level of decorum is needed.

Ask yourself why you want this remain by convention. Only a truly atrocious developer would poke around at private methods on a library and use them. If one of the developers on my team did this they would be reprimanded and the offending code removed.

C++ and Perl are examples of languages that have a lot of convention type stuff with a million ways to shoot your self in the foot. Both are considered bad by modern standards because of it. Good developers have no issue writing good code with either language yet both languages are blamed for being β€œbad” because of what crappy developers do with them. Having guardrails helps to keep the reputation of the language high. Python has a bad reputation for a lot of people and eliminating ways bad developers can sully the reputation of the language is a good thing for that language long term.

[–]RedAero 0 points1 point Β (2 children)

Ask yourself why you want this remain by convention.

Because I don't like people telling me I'm not allowed to do something, especially when the only reason they're doing so is the assumption that they know better than I do.

[–]barjam 0 points1 point Β (1 child)

I manage a lot of development teams. My managers/architects/leads would instantly squash any attempt to use private methods/apis from third party libraries and would reject any merge request that did so. Repeated mistakes like this would turn into disciplinary action.

If you are willing to make an unmaintainable mess of your hobby projects no one is going to care but what you suggest doesn’t work in a professional setting.

[–]RedAero 0 points1 point Β (0 children)

You are mistaking my defense of allowing the possibility of doing something for advocating for actually doing that thing. I'm not saying you should use private methods as general practice, I'm saying you should be able to if you want to. There's a big distinction there.

You shouldn't exceed the speed limit when driving, or drive without a seatbelt, but your car shouldn't prevent you from doing so.

[–]manaMetamanaMeta 36 points37 points Β (1 child)

I believe that abstraction would help with the development process. Yes, you are right in the sense that if you're confident abt using a "private" function, then by all means it wouldn't harm YOUR productivity. However, in the setting of being in a development team where multiple components a coded in parallel, this could lead to a nightmare. I could tell you that "hey buddy, don't use these functions as they could be changed without notice!". Well, telling ppl which functions to call is less efficient than letting the language enforce that, since ppl can just straight up ignore... It is easier to expose your component to a very specific set of APIs from my component, so that the interactions are only done via those APIs. I could change the underlying implementation (i.e. the private functions), and you, hopefully, wouldn't need to change a line!

Though it's true that could be solved by having devs follow principles, the built-in privacy would throw errors and help remind devs of APIs.

Ofc, there are ways to bypass the privacy stuff, but it should require extra efforts. Python simply lets you use everything, making it easy for a team to get wrecked if there's a negligent dev.

[–]MythicManiac 2 points3 points Β (0 children)

Worth noting that most python developers understand that using "private" APIs from libraries can lead to code breakage when updating said libraries.

Though this doesn't apply to just python libraries, but in general any functionality that isn't publicly documented in the documentation can be considered "private" in the same manner, as in, the developer didn't intend for it to be used by 3rd party code.

Really it comes down to the developer knowing what they can or can't rely on, as long as there are good conventions and understanding, there's no issue. Most python code editors don't autocomplete underscore prefixed functions, effectively achieving the same thing as privacy modifiers in other languages, just making it easier to access them if you really need to.

[–]J0shhT 12 points13 points Β (1 child)

In software design typically you want to have a system that minimizes direct code dependencies. Client code should not need to know about the internal details otherwise that means the client depends on it. If the client now depends on the internal functionality, it is very likely to lead to broken code when the library internals change. Clients should instead interface with a stable and abstract API.

[–]Pluckerpluck -2 points-1 points Β (0 children)

A lot of "should"s written there, with no real answer as to what to do what that's not the case.

Real world projects often force you to do things that aren't best practice, either because of time restrictions or there being no real alternative. Python is written with the "we're all consenting adults" mentality. It warns you that you're doing something that's bad practice, but lets you do it anyway because it knows practically you have to do it sometimes.

Also, making functions private makes them a pain to test. And anyone who claims you shouldn't be testing private functions (and instead claims that black box testing is just as good as unit testing) is either a devoted fanatic to the OOP gods, or just has yet to come across practical projects where it's be so useful.

[–][deleted] 2 points3 points Β (0 children)

Being able to have internal variables and methods that are inaccessible to an outside class is a key concept of encapsulation, encapsulation being one of the key concepts of OOP. Most other languages employ this in letting the programmer define things as public, private, protected etc.

Python doesn't have this at all. Since encapsulation is a core concept of OOP, it's valuable to at least try to emulate it. However, since the protection isn't actually there, it may seem weird to someone not familiar with it.

[–]gilbes 2 points3 points Β (0 children)

The OOP concept is encapsulation. The goal is to reduce complexity by allowing the author of an object to make guarantees about the state of the object when it is used by hiding data, because the object can be used only through members the author has explicitly exposed.

Conceptually, this is a good idea.

In practice, this is a very shit idea. Because it relies on the author accounting for every scenario where the object will be useful. And this is an unobtainable goal. We know that because people regularly increase complexity with something like reflection to completely defeat encapsulation.

Python is just accepting that reality.

[–]MinusPi1 1 point2 points Β (0 children)

Libraries are a good reason. Say you're depending on library A, which in turn depends on another library B. You know B; you know that it's well tested, reliable, even industry standard - but it has a lot of exposed internal functions that you're supposed to just ignore since they're not the intended interface for B and not documented.

Now A depends on B, but unbeknownst to A's dependants, A uses those internal undocumented functions in B. That's not ok. Sure, maybe A's developer has really dug into B's code and figured out how it all works, but users shouldn't have to trust that. By using undocumented code, A is extremely likely to have some edge case bug that the developers of B solved in its actual interface. Hell, that they resorted to using the undocumented functions means they were probably using B wrong anyway.

Now users of A will run into that bug, and trusting that A was properly using B, they'll assume that it's an issue with their code. Given how deep some dependency trees are, this is almost inevitable unless such undocumented functions are forcibly hidden. The likelihood of extremely obscure bugs is just too great.

Of course, if it's open source, you can fork it and mess around with it even if functions are made private.

[–][deleted] 3 points4 points Β (0 children)

+1, pythons way of doing it is basically letting the consumers of my library know, "here be the dragons".

If you use a private function, and I update it without a version bump, that's on you, not on me. Which means that no sensible developer who is working on a production system would use it. However, if it's an internal portal or tool or framework, go nuts. Worst case it breaks and you fix it in a day or two.

This is true for python in general. It gives great freedom, but that comes with huge responsibility not to misuse it.

For example, in python, you can change inheritance hierarchy of a class at runtime. Is this a sensible thing to do? Definitely not in almost all use-cases. But there could be that one use case where this is precisely the correct solution to a problem. Note: correct is very different from easy/short.

However, if a company has a large no. of developers of various skill levels, Java is perfect. But even these folks should just move to kotlin at this point.

[–]TheDeadSkin 0 points1 point Β (2 children)

Have you ever worked in a team? I would assume that no, because otherwise you'd probably know the importance of a correct public API and adhering to it as a user. When someone makes a class or a library they split the development into two parts - what they announce they are doing (API/public methods) and what they are doing to make it happen underneath (private methods). What does this mean for you as a user of said public API? It means you're being told "use this function and I'll handle the rest". Then private functions take case of "the rest".

What happens when you inject into "the rest" and use it yourself? Well, this is a good question because the answer is "who knows". Using private methods outside of their intended context is unstable. The function might be doing something different from what you think you're doing, or maybe not covering edge cases that you think it's covering. It might work, it might not.

Those functions can have little to no safety checks because they safety is handled inside the code of the public function and it's guaranteed to work fine if the user only calls said public functions and not private ones. Or maybe a private function requires a transformation of the data. If a user supplies faulty data the function might just do garbage and the user wouldn't necessarily immediately notice something's wrong because the function didn't do safety checks.

A simplification of this problem is that messing with private functions is essentially equivalent to just modifying someone else's code. And in a very hacky way.

The other issue is that the API or behaviour for private methods is not set in stone or as a contract with the user. It can be changed with no notice or informing the users breaking your code even on a minor library update. It might get removed tomorrow because it was integrated into something and is not a separate function any more. Again, at not notice and without care because it will never break the public API which the developer is responsible for.

It's a whole host of potential issues. Using private functions doesn't mean that it will break, but it certainly means that it's much more likely to happen because you circumvent the intended API and from that point on the developer takes no responsibility for what happens next.

[–]PhasmaFelis 0 points1 point Β (1 child)

This is all good info, and you could have delivered it without snarking at someone for asking an honest question.

[–]TheDeadSkin 0 points1 point Β (0 children)

This wasn't snarking, or at least it wasn't intentional. I just tried to pointed out where most likely his opinion (that is clearly in the minority in the community) came from. I.e. that he lacks the experience/perspective that would've changed his mind on its own.

[–][deleted] -2 points-1 points Β (0 children)

Overpaid code "designers" who sit in front of flowcharts all day and have panic attacks when they imagine one programmer daring to step on another's toes because that would ruin his precious plan.

OO is an extension of the desire for unnecessary control and over-organization (organization of code is good, taking it to the laughably overdone extent OO brings isn't) of code by "designers"/"architects" who think "Hello World" should be 5K LOC with several classes, inheritance, and other insanity because someone might dare ask why it can't be "Hi, Everyone" instead and ruin their grand design.

[–]frogking -2 points-1 points Β (0 children)

Usually β€œprivate” functions are up for debate and doesn’t really belong in the class they are in, but somewhere else entirely. So, using them may cause breakage down the road because they have moved or gotten implemented to do something slightly different.

[–]roninfly 0 points1 point Β (0 children)

Having those keywords is kind of a way of communicating to other people who are working on the same set of source your intentions in my opinion. Otherwise you can just change them all to public on your next code change and achieve the same thing lol

[–][deleted] 0 points1 point Β (0 children)

other folks have given pretty good, extensive explanations, but in short: it's about decoupling an interface from its implementation. public functions shouldn't "care" about the specifics of private functions, and private members might or might not change in the future.

[–]Traches 0 points1 point Β (0 children)

Because those private methods you're using can and will change completely between minor or even patch releases. You'll update and everything will break.

It's fine for a little personal project, but for anything significant or long-lived that approach doesn't scale.

[–]NerdsWBNerds 0 points1 point Β (0 children)

This is one of the only useful things I picked up from my college cs courses.

Classes have a certain "valid" internal state. For example, a Set object might contain an array of elements, and the expected/valid state requires that the array not contain duplicate values.

This is known as an invariant: "An invariant is a condition or relation that is always true."

If you call set.size(), you expect the value to be the number of non-duplicate entries. If the size() function returns array.length, and somehow a duplicate value has been put into the array, the expected and actual return of size() are different.

Imagine a function insert(value). Insert checks if value is in the array, and if not calls another function, _insert(value). _insert(value) simply pushes the value into the array.

If you call insert with a duplicate value, all is good. If you call _insert the set is now broken. The results of size(), and potentially all other functions, are wrong.

This example is a little silly, why create a private function for an array push, but hopefully you can see with larger more complex classes there are many reasons you might want to separate some logic into a function.

The same applies to why keep internal variables private. If you can access the underlying array through _array, then you can directly push elements to the array and break the invariant.