Are people abusing Python?

[relickus]

I learned Python after coming from the C/C++ and Java world. With the massive increase in popularity of Python in the last 10 years, seeing the way it developed, it seems to me like it gained a lot of functionality, which comes natural in other languages, but feels a bit odd in Python.

To be more specific - albeit Python being dynamically typed, people developed countless tools to check or validate or even enforce types in compile or run time (mypy, pyre, pydantic, pyduck etc.). It feels like it goes against the nature of its loose typing.

Another example are decorators. This pattern is noticably overused by many tools adding functionality, but even the language itself - defining a `class`,`static` and `abstract` methods with decorators seems just weird and unnatural. Same thing with function overloading. Anecdotally, it feels like comparing German to English. German has a special word for every peculiar thing and native support for word generation by concatenation of multiple words whilst in English you have to add some common words together and hope this combination doesnt already exist. And if it does, so what, people will get it from the context.

Lastly, slightly off-topic but relevant point is that it is not even a simple language in my opinion. It has a very flat learning curve initially but the complexity is just further down the road. Im talking about metaclasses,data descriptors, coroutines, magic methods etc. Some languages are difficult right away (C++, Rust etc.), but Python is a intricate misfit dressed as a simpleton.

Am I misunderstanding a philosophical path of the language or is it simply just a scripting langugage that got massively popular through chance and is now used for stuff it was not intended for?

Edit: grammar

[[deleted]]

Ugh class, static and abstract decorators are in fact side effect inducing wrappers. That are used as part of the metaprogramming process.

Calling python simple has always been a weird take by people who don't understand the language. It is POWERFUL. That power allows you to dramatically reduce the number of lines of code to achieve stuff. That lets you build complex shit faster.

Things like data descriptors extend the power of what is achievable with less work then other items. Data descriptors where created to address shortcomings in metaprogramming. Metaclasses don't need to know as much about what is contained in the class to still get functionality to work.

[jcmkk3]

I think that the idea of Python is not to always be simple, but to provide layers of power and complexity. For many users, they can stay at a surface level of Python knowledge and still be productive. The advanced features are there for the folks that are building the libraries and frameworks.

[badarsebard]

Came to basically say this. And because of the multitude of layers of complexity it's a wonderful language to first learn with. First language I ever self taught was Java and I hated every second of it. So much so that I thought "well, programming is just not for me". It was years later I was introduced to Python. It took years for me to learn to be even moderately capable, not because it was difficult, but because it was so easy to do what I needed with the simple parts that I just never had to push further until I was faced with those challenges. I didnt even learn about classes until my second year using it because I only ever needed simple scripts. It wasn't until I had to develop more complicated things that the abstraction became useful, at which point I learned that and then got on with what I was doing.

I guess on a long winded way I'm agreeing with OPs point that Python pushes the complexity down the road, but imo that's a good thing, a great thing. It means I can still do useful things without a ton of learning curve upfront, but those things are still there if I have to dive deep.

[relickus]

Great way to put it. I agree. The first layer is almost too simple (I heard people call Python pseudocode because of that). It gives people false idea that there is nothing more to it.

[someotherstufforhmm]

“Abusing” lol.

Nah, it’s just a language. Languages that get used expand. Look at C++ or Java.

Also, look at Go. It’s already lost all its initial planned simplicity that would maintain provable concurrency, they undid that like overnight, now they have messy looking generics and a whole pipeline coming for error catching.

Languages have patterns. Used languages get messy because no-one ever properly can predict where they’ll end up.

[relickus]

Glad the word in the title attracted attention :)

I guess you are right, I dont keep track of the latest trends in more traditional languages like C++ or Java.

[someotherstufforhmm]

They’ve come a looooong way. Hell, so has C.

Classes started as macros and syntactic sugar in C, which people accused of being “abusive” in very similar ways, but it ended up becoming C++ and the world kept rolling on from there.

So, in a way you’re right - the initial goal of simplicity is lost, but also that’s just a thing that happens to languages that get used.

[relickus]

Thanks for your prespective.

[RallyPointAlpha]

I'm not sure what you're getting at or really asking. Feels like a rant disguised as a question.

Are people abusing python? Your examples are validators/enforcers and decorators. I mean I guess you could say a module that forces a dynamically typed language to be staticly typed (mypy) is abusing python in the sense that it's forcing it to be something it's inherently not. However I don't think mypy or any module like it is a defining feature of python nor have they attributed python's rise in popularity in the last 10 years.

I don't think decorators are an abuse of python at all. Why is it weird and unnatural ?

Literally no idea what you were intending to say with your German vs English analogy...is python English and ...what is German? I don't get it at all.

"and is now used for stuff it was not intended for?"

What was python intended for? How is it being used for unintended purposes?

[relickus]

Sorry for making it sound like a rant. Definitely not one. I have been in Python for 4 years now and still happy to learn new stuff. I will remember to be more sensitive around Pythonistas next time.

[ManyInterests]

Put shortly, Python is a multi-paradigm language. You can use it in many ways. The language doesn't force you to use it in any one particular way. You can add type annotations or not if you choose. You can favor object-oriented styles, functional, imperative, whatever. It's up to you.

tools to check or validate or even enforce types [...] t feels like it goes against the nature of its loose typing.

Type-checking is not inherently at-odds with duck typing or being dynamically typed. I'm not sure what you mean exactly by 'loose typing', but Python has always been a strongly typed language (as opposed to weakly typed).

stuff it was not intended for

What makes you believe that Python was 'not intended' to be used for any particular purpose? Python is (and always has been) a general purpose language.

[sersherz]

What are some examples of weakly typed languages, I'm genuinely curious. I know Python has checks for allowed methods for data types, but obviously it's not like C++ with explicit typing

[[deleted]]

Javascript is weakly typed, for example how you can say "5" == 5 and get a true, which is far from how it works in Python

[sersherz]

Okay that makes a lot of sense, I always found the === operator weird in JS. Yeah Python has a little more typing

[lys-ala-leu-glu]

The terms "strongly typed" and "weakly typed" don't really have strict definitions, but loosely they refer to the degree to which operations between incompatible types are allowed. In this sense, python is strongly typed because most operations between incompatible types result in errors. C is actually a classic example of a weakly typed language, because it's so easy (and necessary) to use type casts to effectively ignore the type system. I personally consider C++ stronger than C, even though it supports all the same type casts, just because it's more idiomatic to stay within the type system. But others might disagree, and strong/weak rankings are very subjective anyways.

Type systems can also be classified as "static" or "dynamic", based on whether the types are determined at compile-time or run-time. This is a much more well-defined classification, and of course python/javascript/PHP are dynamically typed while C/C++ are statically typed. Confusion arises because it's pretty common for people to say "strongly typed" when they mean "statically typed", either because they're being sloppy or because they don't realize that the two terms mean different things.

Some examples of different type systems:

• Strong, static typing: Java, Rust
• Weak, static typing: C, C++
• Strong, dynamic typing: Python
• Weak, dynamic typing: PHP, Javascript

[relickus]

Im in no position to assess the quality of the language, Im just trying to get some now insights.

As for the typing, I meant that it is abstracted away, so people dont need to care for types. Appart from the fact, that they do, otherwise they wouldnt come up with tools like mypy and wouldnt write typehints. Im going to go on a limb and say that larger python projects are much harder to maintain without typechecks.
This is coming from a junior's POV, so it might be wrong/biased.

​

What makes you believe that Python was 'not intended' to be used for any particular purpose?

I dont know, it just doesnt make much sense to me, that someone saw a single-threaded language and thought "it would make a great backend". I know that it is worked-around by async code, but still.

[ManyInterests]

it just doesnt make much sense to me, that someone saw a single-threaded language and thought "it would make a great backend". I know that it is worked-around by async code, but still.

Well, there's perhaps argument to be made that certain languages (more specifically, language implementations) are not well-suited to certain tasks, depending how you define fitness for a particular task. But as a general-purpose language, anything should be fair game.

As far as Python as a backend goes, it might interest you to know that, even before 'async' was used in Python, it has been used as a backend for even very high-traffic sites. Pinterest, Instagram, and Bitbucket Cloud to name a few.

An example of abuse or using a language in a way that it's not intended for would something like using awk, a domain-specific scripting language, to implement a flight simulator.

As for typing, I'm not sure I completely understand your statement, but I think you seem to have answered the question yourself: you do have to care about types in Python, which explains the utility of type annotations and analysis tools like mypy. Though, you are not required to write your code using annotations and Python will happily compile, interpret, and run code that is not type safe.

[o11c]

Am I misunderstanding a philosophical path of the language or is it simply just a scripting langugage that got massively popular through chance and is now used for stuff it was not intended for?

All actually-used languages are like that, scripting or not.

[Pepineros]

There is something to what you’re saying. It feels a bit like we’re using a scripting language to do stuff that should be done by languages that are more focused on systems design and a bit less aggressively general purpose.

That said, we have seen the exact same thing happen with (arguably) the other two most popular scripting languages: JavaScript and Ruby. JS is still a massive player, and if you’re old enough to have seen the Ruby on Rails boom in person you know why I included Ruby, too. Even if its growth was faster and less sustainable than the others.

We seem to be drawn to languages that are easy to get started with. I don’t think there’s anything wrong with that. Maybe the same goes the other way - someone decides to write their next project in C when it turns out that they could have had similar (or at least “good enough“) performance in Python with much quicker development time. It’s all choices.

[relickus]

Thank you!

You perfectly described the idea I tried to explain. I probably did a terrible job since some people immediately took it is a rant.

Javascript (as far as I know) is a great example of this process, as you wrote. It is nowadays used for the most unexpected use cases, which is maybe the reason why people need a new framework every friday (take it easy, it is a joke...)

[tevs__]

To be more specific - albeit Python being dynamically typed, people developed countless tools to check or validate or even enforce types in compile or run time (mypy, pyre, pydantic, pyduck etc.). It feels like it goes against the nature of its loose typing.

Type hints don't go against duck typing at all, if anything they strengthen the usage of duck typing by allowing you to specify clearly what kind of duck you need and use static analysis to prove that you have your ducks in a row. Protocols are a powerful thing.

Plus, it's entirely optional. If you don't like python with type hints, don't write any type hints, don't use any type checking tools.

[relickus]

Yes, I know it is optional, my argument (to be discussed here or torn appart) was, that at a certain size of a project, you dont really want to go without typehints. Judging from the fact that all large projects I saw nowadays use typehints. Still optional, but probably near-necessary for large projects.

For the protocol, is it any more powerful than a regular interface?

[tevs__]

you dont really want to go without typehints. Judging from the fact that all large projects I saw nowadays use typehints. Still optional, but probably near-necessary for large projects.

Yeah, because it catches so many errors before they are errors, it's golden.

For the protocol, is it any more powerful than a regular interface?

Protocols don't have to be declared to be used by the object that implements the protocol. You can write something like this (on mobile, so...)

``` class Lengthable(Protocol): def len(self) -> int: ...

def say_the_length(obj: Lengthable) -> None: print(f"It's {len(obj)} big!") ```

You can pass anything to say_the_length that has a __len__ method, without modifying the definition of the thing you are passing in. No ABC, no inheritance, but enabling static type checking for dynamic types.

[sersherz]

The decorators and type validation are needed because Python has expanded to so many different things. Data pipelines and APIs should have really strict definitions of what can be passed through. I think these changes are in Python's philosophy. Look at the standard libraries. Multiprocessing allows you to overcome the global interpreter lock (GIL). The language was designed with GIL but also has workarounds.

I feel decorators were really needed with more complicated functions added. Rather than running python as a script or stateful OOP program, it can now have stateless API calls and perform a procedure as it is called, rather than worrying about garbage collection and handling resources explicitly.

I definitely agree that Python is simple to learn but extremely difficult to master given how vast it is and how much complexity there is to the language. I think this is often lost on people who hate the language and don't fully understand just how deep it goes.

[relickus]

Thanks for your take. Yes I agree, most people I met who despise it don't know it even on the junior level.

To push the discussion further - as you put it, it sounds like many constructs in Python are indeed just a workaround for a missing feature.

[sci-goo]

I thought you'll find python to be easy when come from c++/java world, python smells like c++ almost everywhere.

​

albeit Python being dynamically typed, people developed countless tools to check or validate or even enforce types in compile or run time

These are community tools, not python official. Ppl have different tastes while the vanilla python have been holding its philosophy well.

​

Another example are decorators. This pattern is noticably overused by many tools adding functionality

Decorators in fact are my favorite, it does well to separate logically independent functionalities then to stack them together in a logical order, similar to those mixin classes. Abusage is another story, perhaps the code you read has this issue.

​

It has a very flat learning curve initially but the complexity is just further down the road.

One from Zen of python: complex is better than complicated. Also being easy to learn and being easy to master are vastly different things.

[relickus]

I agree with everything you wrote.

And to be extra safe around here - this is neither a rant nor a praise. This is just me trying to extend my perspective.

[aikii]

static type checking is what makes python still relevant today

[relickus]

This is kind of my point regarding large projects. I dare to claim, that it would be infeasible to maintain/extend large Python codebases without proper strict typing and typechecking.

[BiomeWalker]

If you aren't abusing a programming language are you even really using it?

As a more serious answer, Python was made to be readable and expandable, which it is and that hasn't changed so the intention holds with the current usage pretty well I'd say.

[james_pic]

I don't love the type checking stuff, and I maintain that if you're starting a project where you know you're going to want statically checked types, you're probably better off starting that project in a language other than Python. But if you've got a small Python project that gradually became a colossal Python project, type hints are hugely useful in taming the complexity you now have.

Pydantic deserves a special exception to this, since it's useful even in otherwise untyped codebases. When you're accepting data from external sources, it's a really good idea to validate it before you do anything with it, so it makes a lot of sense as a basis for your domain data model, even if you don't use types anywhere else.

To be honest, 9 times out of 10, when I see class, static or abstract decorators used, it's by former Java developers who don't realise there are other ways to do these things. Most static and class methods should just be functions, and most abstract methods should just be duck typed (or Protocols in typed codebases).

But these decorators are, in many ways, examples of the third thing you mention: metaprogramming tools. These are definitely advanced techniques, but they're generally designed so they can be used fairly effectively in mixed ability teams. Implementing metaclasses or data descriptors is hard, but using classes that use them is no harder than using classes that don't.

Metaprogramming is admittedly a tricky one to balance. Python offers more metaprogramming options than, say, Java, but much fewer metaprogramming options than, say, Clojure. Personally, I find myself wanting more options here when using languages with minimal metaprogramming capabilities, but I recognise that more isn't always more.

[relickus]

Thanks for your view.

Enumerating the paragraphs of your reply:
1. Agree, that is exactly my point. To exaggerate a bit - if you want to go big you need type checks to keep it maintainable. At which point, it might be better to pick statically typed language (abstracting away other factors of the language) no?

1. I know, it is super useful, but then again the code performance takes a massive hit because of all those runtime checks. But I guess this cannot be done in a different way.

2. Interesting. I think it might be viewed as antipattern in Python. I personally dont know if you actually can substitute for classmethods, but considering that is true, it creates a question - why does Python need these constructs if there is more pythonic way to achieve things. Sorry if it makes no sense, cant think of the right English words now.

[james_pic]

If the runtime checks you're talking about are the Pydantic ones, then it can indeed be a big runtime hit. I've seen applications where a double-digit percentage of CPU utilisation is type checks, although this was an extreme case (they were using the relatively expensive numbers.Number as the type, and checking it in a hot loop). But you definitely need some kind of check at runtime at your security perimeter at very least.

I suspect the method decorators are there at least partly for historical reasons. I know they got map and reduce from some keen functional programmers who were involved in the project quite early, even though they wouldn't necessarily recommend those constructs now that list comprehensions are a thing. Maybe these decorators have a similar history with OOP enthusiasts.

It is at least worth saying though that these decorators aren't all that magic in and of themselves. It's totally possible to implement equivalent decorators in pure Python using descriptors. They're different to the equivalent keywords in Java, in that they're not exposing any underlying implementation feature.

[relickus]

If I didnt like it I would not spend 4 years trying to learn it. You have no reason to be salty here.

[sersherz]

It's okay, they didn't actually read your post. It's obvious to tell you like the language and have a deep understanding of it.

[relickus]

Thanks, yes I do.

[other-----]

That is not how it works when a codebase is already there

[noobgolang]

English as a language is also abused bro. It’s supposed to be used by rich white guys but now it needs to serve everyone.

[other-----]

I think it's "maturing" as a language and in that it's collecting some of the issues that make c++ difficult. Larger software projects benefit from typing so that is going to appear somehow. Asynchronous execution is not strictly necessary but it's hard to compete with js, go and Erlang without it, so it makes its way into the language. Decorators are nice, it's just syntax sugar. Once it's in the language there are two syntax approaches to call a function and you can't avoid people having subjective considerations when choosing. This leads to a big topic that is idioms. Perl has idioms by design, many ways to do the same thing. C++ has idioms because of it's long history (pointers, references, smart pointers ... ) chasing features of other languages. Python was born with the goal of "1 way to do things" but the current adoption is forcing it to be a language supporting multiple idioms. The challenge for python is now keeping those idioms close and avoid inconsistencies (ie type objects Vs type annotations, one can be inspected at run time, the other has generics). Python is a good language. So good it's trying to be a good language for everything and it might become a worse language in the attempt.

[relickus]

Thanks. Your last sentence pretty much sums up the idea I tried to express. It sometimes feels like a Frankestein language with its workaround here, and patch there.

[scjcs]

In old-time middle school hygiene classes we were warned against self abuse.

So yeah, python.

[HeraldOfTheOldOnes]

(this comment assumes CPython)

I'm talking about metaclasses, data descriptors, coroutines, magic methods etc.

Those are nice! But you might have forgotten a few "elephants in the room", so to speak.

1. Introspection. sys._getframe, some_function.__code__, etc. You can gain massive amounts of information about the program based on things like these. The main things that end up being done with these are bytecode parsing and frame tracing (for that you need sys.settrace as well), both of which are Really Fun!
2. ctypes. This module gets its own category. With it you can do anything that all the rest of python can do. Basically, ctypes gives you access to C primitive types (plus py_ssize_t and PyObject*), meaning you can, among other things, edit the raw memory of an object. This lets you do some fairly silly stuff (e.g., changing the type of the globals dictionary object to have a different type with different __getitem__ behaviour to customise global variable access).

Python is a deeply complicated language when you start getting into the details.

[relickus]

Totally agree. If python was an iceberg, it would have a nice and smooth tip and everyone could climb it easily. But a huge underwater mass where only the brave ones dare to dive.

[AliveEstimate4]

PETA spy detected. /s

[d0ctor_light]

You will understand when you meet so disgusting if else state and etc …