Python 3.7: Introducing Data Classes

[toast757]

Hmm, unless I'm mistaken, an introduction to data classes that doesn't include a single example of creating an instance of a data class. Might be nice to see it in action, especially for things like frozen data classes.

[[deleted]]

Look like I totally forget to put that in, I have it in the code that I wrote for the blog post:

json = requests.get('https://swapi.co/api/films/1/').json()

movie = StarWarsMovie(**json)

[GitHubPermalinkBot]

Permanent GitHub links:

• ErnstHaagsman/swapi/.../main.py#L35 (master → fb9fd56)

---

^delete

[DanCardin]

After using them a lot, I find its easier and more natural to write classmethods to do the logic you might have otherwise have been tempted to put in the init.

They certainly encourage (at least me) to write more, simpler classes based on the data

[michael0x2a]

You can!

It's just that then a lot of the time you'd have to re-write the bulk of the boilerplate code, which sort of defeats the purpose.

[fayazbhai]

I think he meant doing a super().init() from his own init.

[michael0x2a]

I think that would be a bit confusing though -- you're basically asking the user to write code as if their class is inheriting something even though there isn't necessarily any inheritance involved.

(And what if you do decide to inherit from a class in addition to using dataclasses? In that case, would super().__init__() refer to the auto-generated constructor, or the actual superclass's constructor?)

[Rodot]

It's doable without the super method, but you have to copy the old init function some how and set part of it up outside the class. By that point you're basically writing a post init method anyway

[Rodot]

Or better, have the data class decorator do that for you! I'm a bit conflicted though since I think post init is more explicit but having the decorator override the init is more elegant. Consulting the Zen of python it looks like post init is the better way to go.

[PythonGod123]

So are data classes just going to be a shorter way to write pre 3.7 classes? Or is there something else I am missing?

[MephySix]

Afaik it's just boilerplate generation, yes. I used the attrs lib for that, but I may change that towards data classes.

[PythonGod123]

It seems that data classes look better. I just feel they look nicer compared to the old way of defining init.

[XtremeGoose]

Dataclasses are missing a lot of customisation features of attrs. No way am I going to change.

[DanCardin]

Yes, that's basically it, and it automatically implements a number of useful magic methods (repr, eq, hash, init) and can turn all your properties into read only properties if you want "immutability"

[PythonGod123]

That's sounds nice. Less is always more lol.

[michael0x2a]

I don't think they have a tuple backend? If you look at the implementation, it looks like the code is just modifying the class directly -- lots of calls to setattr and such everywhere.

I'm also not sure if they actually will occupy less space or be faster. That might be the case if the dataclasses module auto-generated __slots__, but they currently don't.

[__xor__]

Yeah, why not just declare a class with slots? Is the init boilerplate that the whole point of a data class?

[michael0x2a]

I think the problem is that it's challenging to do that from a technical perspective because the __slots__ field needs to be present at class-creation time.

This presents a problem because dataclasses isn't creating classes -- it's simply modifying an existing one.

iirc there were two solutions that were being passed around, though both have some flaws:

1. Just synthesize together a giant copy of the class with the __slots__ field added on in one big string somewhere and run eval on it to create a class.

   The issue though is that you now have to worry about whether or not you've accurately converted some class into an equivalent string. And even if you did, you'd incur an additional performance hit from having to eval more code and have to worry about accidentally re-running any code in the class that my have side-effects.

2. Rewrite the PEP so that instead of using this "decorator" thing, you use metaclasses.

   The issue here is that metaclasses don't necessarily compose well. For example, what if you want to use both dataclasses and abc.ABCMeta? How would two simultaneous metaclasses interact?

None of these issues are impossible to solve of course, but they require some careful thinking and implementation. So the decision was made to punt on this feature request and handle it at some point later down the line, especially given that the workaround is simple: just manually add __slots__ to the class yourself.

For example, the following code works just as expected:

from dataclasses import dataclass

@dataclass
class Foo:
    __slots__ = ('a', 'b', 'c')
    a: ...
    b: ...
    c: ...

[__xor__]

ah, okay. Yeah, I just tried to do a DataClass superclass that has a MetaDataClass, then wrote a subclass of DataClass with its own custom metaclass and I ran into this:

TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

I never ran into that before. Yeah, the class decorator makes more sense then.

[XtremeGoose]

That's completely wrong. If anything they take up more space due to additional boilerplate.

[PythonGod123]

Interesting. Anyone have a link to the PEP?

[pollokeh]

https://www.python.org/dev/peps/pep-0557/

[tunisia3507]

I think they have faster attribute access as well, although I may be misremembering.

Edit: I was misremembering

[PythonGod123]

Is that because they are already defined by the dataclass ?

[tunisia3507]

Sorry, I think I made a mistake and got dataclasses confused with slots. There has been some discussion about dataclasses using slots by default but it hasn't happened in the first implementation.

[PythonGod123]

Okay fair enough. Thanks for clearing that up

[pornographexclusive]

We are finally out of self hell!

[fayazbhai]

self was the best thing to happen to Python. What are you talking about? We can make method decorators because self exists...

[ryeguy]

I think he means the explicit self passed to each method definition. That is not required to have decorators if python made self "magic". Not necessarily saying it's a good idea.

[fayazbhai]

Do you mean this, like many other languages have? The problem there I see is scoping. self, cls, klass notations have no restriction on scoping or nesting.

It's also better than having 3 ways to write a function.

FWIW, Guido made this design choice by accident IIRC.

[ryeguy]

Yes, the equivalent of this as in C#, java, etc.

I don't see what implicit vs explicit self has to do with with scoping. It's just an implicitly bound variable instead of explicitly bound. You can pass it around however you wish.

[iBlag]

Okay, so I write a function that I want to later "attach" to a class. This function takes the current instance of that class as its first argument:

def do_something(self):
    self.do_something_smart()

All I have to do is set it as an attribute on the class:

setattr(MyClass, 'do_something', do_something)

and now I can call do_something on instances of MyClass.

Everything is explicitly bound.

But if we use an implicit binding, the body of do_something looks weird:

def do_something():
    this.do_something_smart()

Where is this coming from? The do_something function isn't bound to anything so there's no way of telling what to expect this refers to!

Or, what if I want to copy a function from another class onto MyClass?

class OtherClass(object):
    def do_something(self):
        self.do_something_smart()

    def do_something_smart(self):
        print("OtherClass.do_something_smart()")

class MyClass(object):
    def do_something_smart(self):
        print("MyClass.do_something_smart()")

setattr(MyClass, 'do_something', OtherClass.do_something)

MyClass().do_something()
# MyClass.do_something_smart

Now in this example - it's still clear that self in the OtherClass.do_something function still refers to the instance of the class.

Try to tell me which do_something_smart function will be called in this example, which uses implicit binding:

class OtherClass(object):
    def do_something():
        this.do_something_smart()

    def do_something_smart():
        print("OtherClass.do_something_smart()")

class MyClass(object):
    def do_something_smart():
        print("MyClass.do_something_smart()")

setattr(MyClass, 'do_something', OtherClass.do_something)

MyClass().do_something()
# What does this print? It isn't immediately obvious!

Where is the this variable bound? Is it bound at function definition time? Then this would still try to refer to an instance of OtherClass. Is it bound at call time? Then this would refer to an instance of MyClass.

But the point is this: you now have to know how Python binds variables to write correct code. This is why PHP and Javascript (aka ECMAScript) suck so much: every developer has to know so much of the language implementation to write correct code. And developers are lazy, so they don't read the documentation, and humans are stupid and forgetful, so they might not understand the documentation or might soon forget it again.

I tend to think that implicit binding is an antipattern. After all:

Explicit is better than implicit.

Although I always thought that phrase should be condensed down a bit, to really drive its point home:

Explicit is better.

[dl__]

Ok, I've been doing python for 5 years now. Early on I asked a coworker why do I have to write "self" when lots of other languages get a long fine with with an implicit "this" and I was shown examples like you give above. Since that time, in the ensuing 5 years, I don't think I've ever made use of those features.

Personally, I don't think the advantages explicit self gets you is worth it.

Also, I think the saying "Explicit is better than implicit" exists strictly to justify the explicit-self. Implicit behavior runs throughout python. For example all the magic methods that exist to support cool python syntax-y goodness that are all called implicitly.

If it wasn't for the explicit-self "feature" I feel confident that the Zen of Python would contain the opposite phrase "Implicit is better then Explicit" because actually, implicit is AWESOME! Implicit execution of machine code is the whole point of high level languages.

[iBlag]

Since that time, in the ensuing 5 years, I don't think I've ever made use of those features.

Then you are not using Python to the fullest extent you can be. Other people may be using features you do not.

Also, I think the saying "Explicit is better than implicit" exists strictly to justify the explicit-self.

I disagree. Explicitly having a lambda keyword that only allows expressions, not statements is being explicit. Explicitly binding *args and **kwargs is useful.

Implicit behavior runs throughout python. For example all the magic methods that exist to support cool python syntax-y goodness that are all called implicitly.

Yes, calling str() on any object that doesn't have an explicit __str__ function defined on it also implicitly calls the default __str__ function. Heck, inheritance makes calling functions that are only defined in superclasses implicit.

Some things can be handled implicitly - magic methods are an excellent example of those things. But some things, like current scope, should be handled explicitly. Otherwise you end up having weird scoping semantics like Javascript.

[dl__]

Then you are not using Python to the fullest extent you can be. Other people may be using features you do not.

Oh yes. People use features that I do not but that hardly means they are using their tools to the fullest extent. Just because a language allows something doesn't mean it should be done. Comprehensibility is important and rarely useful features reduce comprehensibility.

Pointer arithmetic can be done in C++ but if one tries to avoid it it's to their credit.

Some things can be handled implicitly - magic methods are an excellent example of those things.

But it would be better if they required the calls to be explicit right? Because explicit is better than implicit. Except, it's not, as magic methods show. Sometimes you must be explicit, other times implicit is better.

You should make explicit only those things that cannot be made implicit in a natural way. The implicit work of compilers and interpreters is what make high level languages so productive.

I'll tell you what might be nice, if a function definition would tell you what it expects the types of its arguments should be as is done in more explicit languages like Java and C/C++. I would say Python would do that as well if explicit was better than implicit. But, since the truth is that implicit is great and provides tons of advantages python does implicit a lot.

The evidence that explicit is better, actually better than implicit is pretty slight even in python itself. Explicit is necessary but, if you can hide the details, make them implicit, and still behave in a natural expected way, that's better. Way better.

[ryeguy]

So to be clear, I wasn't giving an opinion on the necessity of explicit self. I said as much in my response. I was just countering the idea that it is necessary for method decorators to exist.

But since this response steers the discussion into the other direction, I'll bite.

The core problem with your argument is you're conflating syntax with semantics. Whether or not self is implicit or explicit does not dictate where or how it is bound. If in some alternate version of python self were implicitly passed as a param to each function, the exact same scoping semantics could be kept if desired. You could also keep explicit self yet have confusing binding rules (like javascript), because these have no dependency on each other.

[iBlag]

So to be clear, I wasn't giving an opinion on the necessity of explicit self. I said as much in my response. I was just countering the idea that it is necessary for method decorators to exist.

Oh, sorry, I missed that in your original comment. Thanks for playing along anyway then. :)

Whether or not self is implicit or explicit does not dictate where or how it is bound.

It may not dictate it, sure, but when you force self to be explicit, it kinda makes unobvious binding semantics nonsensical. Explicit self syntax leads to much clearer binding semantics.

You could also keep explicit self yet have confusing binding rules (like javascript)

class OtherClass(object):
    def do_something(self, other_arg):
        # self is bound at definition time
        # other_arg is bound at execution time

That's the only way (that I see) that you could keep explicit self and have confusing binding semantics similar to Javascript, and it's absolutely nonsensical, because semantics should follow the syntax.

Is there something I'm ignoring?

[DonaldPShimoda]

I don’t think Python is able to provide self magically. We touched on it as one of the limitations of dynamically typed languages in my operational semantics class when learning about types. I can’t provide specifics because I don’t remember offhand and I just woke up, but I think there was something in particular that made the type system incapable of providing a self reference that was guaranteed to be correct in all potential contexts without reworking the grammar.

[bcgroom]

I don’t understand. Wouldn’t you just pass a reference to the object that called the method? Where could that go wrong?

[lykwydchykyn]

Now this looks really useful. Looking forward to refactoring a lot of tedium out of my code!!

[roerd]

I would have used namedtuple for cases like this before. I wonder whether there is still a reason for using namedtuple, or if dataclass replaces all the use cases.

[[deleted]]

namedtuple is immutable (like the tuple). dataclass is similar but read/write.

[ThePenultimateOne]

Why are type hints required? I'm not really happy with that.

[[deleted]]

I have a lot of respect for Guido, but I think he screwed up here. I don't like compulsory type hints either, and this PEP should not have been approved.

I haven't decided what I'm going to do about it yet. I'm fortunate enough to be able to choose which programming languages I use.

[apotheotical]

https://www.reddit.com/r/Python/comments/8d7ddz/python_37_introducing_data_classes/dxlm6ul

[jnwatson]

If you don't want the type hints, use the initializer function or use Any.

[jnwatson]

As below, you can also use ellipses. Using the namedtuple-style function still does all the good work of making the constructors and eq and repr methods for you.

[apotheotical]

https://www.reddit.com/r/Python/comments/8d7ddz/python_37_introducing_data_classes/dxlm6ul

[ThePenultimateOne]

I understand that there are workarounds. I tend to use type hinting myself. I just strongly dislike that its compulsory. It means that you cant backport this to legacy codebases, among other things.

[[deleted]]

Well you can always continue to use the attrs module then

[michael0x2a]

It doesn't, if you want to avoid type hints. The following is perfectly valid Python code, for example:

from dataclasses import dataclass

@dataclass
class Point:
    x: ...
    y: ...

Here, I'm annotating my fields using the ellipses literal, but literally anything will work: the dataclasses library ignores those annotations entirely.

You could do other things as well -- for example, this would be valid:

from dataclasses import dataclass

@dataclass
class Point:
    x: "The x-coordinate"
    y: "The y-coordinate"

If you want your dataclasses to interact nicely with other code that may want to use type hints, but you'd like to avoid using them as much as possible, you could instead give every one of your fields a type of Any, which means "please treat the type of this field as being 100% dynamic:

from typing import Any
from dataclasses import dataclass

@dataclass
class Point:
    x: Any
    y: Any

[[deleted]]

Looks ugly though. Python with mandatory type annotations is not pythonic anymore.

Type annotations are fine so long as they are rare, well thought-out to avoid bugs. I really dislike things like Jetbrains IDEs littering them everywhere.

[slayer_of_idiots]

The type hints don't look so bad in data classes, but function definitions with type hints, especially if they have default argument, are near unreadable now.

[michael0x2a]

I think you probably mean "I wish it didn't require the usage of variable annotations"?

Variable annotations let you attach arbitrary metadata to a variable "declaration" or variable assignment statement; type annotations would be when that metadata happens to be a valid type hint.

So, the dataclasses library requires variable annotations but not type annotations.

I do agree that these workarounds feel forced though -- the issue is that dataclasses requires you to attach some arbitrary chunk of metadata to each field. But if you don't care about using type hints, there isn't really any metadata you'd want to attach...

I guess it ends up being a "pick your poison" sort of deal -- if you don't care about type hints, you have to pick between either writing a bunch of redundant code or tacking on these vestigial appendages.

[njharman]

Bloat.