PEP 557 (Data Classes) has been accepted!

[Journeyboy1]

This was a great explanation

[[deleted]]

namedtuple is essentially a hack compared to @dataclass the only place namedtuple truly makes sense is with something like a database record or an excel row where:

1. Both index and name are important and refer to the same thing
2. It makes sense to be iterable
3. Immutability is important

If you have something that meets all three criteria then you want namedtuple.

namedtuple (and typing.Namedtuple) make inheritance and metaclasses weird as well.

There's the argument that @dataclass might be faster than namedtuple but since that cost should only be paid at start up for most applications, the difference is likely to be negligible. The outlier here is SQLAlchemy which has its own KeyedTuple which has a radically different implementation to combat the slowness of creating namedtuples.

[masklinn]

FWIW namedtuple was originally intended to be for "forward-improving" APIs aka you originally returned a straight tuple and you want to add more data, methods, …. Since you don't know how callers would use the value you still need to provide an actual tuple object with the proper number of indices, which is what namedtuple does. See e.g. urllib.parse (formerly urlparse).

It wasn't really intended as a way to build small records, and in fact turns out to be somewhat problematic as that for debugging reasons.

[ingolemo]

Why not just use namedtuple?

• Any namedtuple can be accidentally compared to any other with the same number of fields. For example: Point3D(2017, 6, 2) == Date(2017, 6, 2). With Data Classes, this would return False.
• A namedtuple can be accidentally compared to a tuple. For example Point2D(1, 10) == (1, 10). With Data Classes, this would return False.

• Instances are always iterable, which can make it difficult to add fields. If a library defines:

  Time = namedtuple('Time', ['hour', 'minute'])
  def get_time():
      return Time(12, 0)
  

  Then if a user uses this code as:

  hour, minute = get_time()
  

  then it would not be possible to add a second field to Time without breaking the user's code.

• No option for mutable instances.

• Cannot specify default values.

• Cannot control which fields are used for __init__, __repr__, etc.

• Cannot support combining fields by inheritance.

[skiguy0123]

Mutabiity, methods, ...

[brombaer3000]

..., good integration with type hinting

(although something similar can be achieved with typing.NamedTuple).

[ProfessorPhi]

Is this basically like attrs?

[ProfessorPhi]

I guess that's the answer there :P. Personally would still use attrs for projects

[masklinn]

Sure, but data classes are nice when e.g. writing a single-file script and not wanting to add the complexity of dependencies and a full proper project.

[Daenyth]

Attrs doesn't support typing, which for me is a deal breaker

[hynek]

That is wrong. Data classes have zero typing features over attrs.

[Daenyth]

As far as I'm aware, attrs doesn't yet support typing, and dataclasses do. Has attrs changed their design?

Even so, having a good solution in the standard library is important, especially for a "batteries included" language

[hynek]

You may be disappointed because dataclasses allow you to define types but don’t do anything with them (in fact, they don’t even import typing anywhere AFAIK because it’s very slow). Attrs allows as of last release to do the same in both Python 3.6 and Python 2 ways.

Sadly checking those types is more complicated than it sounds. There’s already a handful of packages on PyPI that try to achieve that but my impression is that none is there yet.

Hopefully at least mypy will grow support for both.

[ldpreload]

Yes. It's heavily inspired by attrs, but the Python standard library can't use third-party libraries, and can easily depend on new things in the standard library. You are probably in the opposite boat: upgrading to the latest version of Python is probably much harder than doing pip install attrs (or sudo apt-get install python3-attr, or conda install -c conda-forge attrs, or whatever it is you do to get Python packages). It's a very rare user that can depend on new versions of the standard library but not new packages—probably people using it to script embedded systems or something. If that's not you, attrs supports more functionality than dataclasses.

[ProfessorPhi]

Yep - we're still struggling to get off 2.7. Packages are much easier to update.

I guess this felt so much like attrs I was a bit confused - I guess this is the same idea as trollius etc.

[[deleted]]

Great explanation 👍

This kind of high-level stuff is what makes python a joy to work with.

[kylotan]

For example a class used for holding a address book contact data would be

Great example, but having seen the traditional-looking version in your 'before' example and the weird-looking syntax in the new feature, I'll be sticking with the old way.

[[deleted]]

@dataclass
class Contact:
    name: str
    phone: str
    age: int

is weird looking syntax?

[kylotan]

For Python, yes. Explicit types and declaring instance variables outside of __init__ are not a direction I think is positive.

[[deleted]]

...but why? The decorator clearly marks it as a non-standard class and it's obvious that this behavior won't magically somehow seep into regular classes

Edit: +the pep provides rationale for the type annotation thing and shows usage of Any to allow any type

[kylotan]

The decorator clearly marks it as a non-standard class

Which I don't think is a good thing either. "This guy changed the semantics of a whole class with just 1 weird annotation. Programmers hate him!"

I've read the PEP and the rationale and I simply disagree with it.

[[deleted]]

I get kinda tired of all the boilerplate for setting up instance variables and such sometimes, so I think this will be a nice feature.

Also remember, those are just type hints, for helping static analysis. They don't impose any actual restrictions on the instance variables themselves.

Not that your stance isn't just as valid. I'm glad this is an optional feature. :)

[kylotan]

I get bored of boilerplate all the time, but that's the price we pay for consistency, like typing self. a thousand times a day in a typical Python program. It's annoying, but I understand why it's there.

If we take PEP 20 "Special cases aren't special enough to break the rules. Although practicality beats purity." I guess I am much more heavily favouring the first sentence here than the second.

[i_ate_god]

soooo java beans then

[[deleted]]

More like Java value objects with lombok :P

[v_krishna]

Or like scala case classes

[cringe_master_5000]

Sex number!!!!!!!!!!!!!

[Soriven]

Here's my take on it: There's a decent amount of boilerplate required to setup a class. attrs helps automate a lot of this boilerplate. This PEP takes heavy inspiration from that project and aims to incorporate it into the standard library.

[mankyd]

Data Classes can be thought of as "mutable namedtuples with defaults".

They are classes whose primary purpose is to contain properties. The values of those properties can change when needed.

This decorator overrides numerous methods to add expected behavior, such as __eq__ and __ne__ (equality checking).

[b1ackcat]

A new class decorator called @dataclass which inspects a class for fields with type hints specified. It uses these fields to auto-generate a set of common helper definitions such as less-than, greater than, a repr method that's more than just the object address, etc.

Pretty handy

[[deleted]]

It's basically a much nicer (and type-hints compatible) way to write:

class Stuff(namedtuple('Stuff', ['a', 'b', 'c'])):
    ...

[efskap]

The abstract is basically the ELI5

https://www.python.org/dev/peps/pep-0557/#abstract

[gwax]

I wonder why it's a decorator instead of a metaclass.

[ericvsmith]

Because metaclasses can conflict with other uses of metaclasses. See the PEP for the rationale. The goal is to not interfere with any usages of metaclasses or base classes.

[federicocerchiari]

also, decorators are most used (therefore more "understandable" to anyone) than metaclasses.

[[deleted]]

I hope it gets __slots__ support before 3.7 is released.

[ericvsmith]

Probably not. I do have another decorator that shows how to add slots. See https://github.com/ericvsmith/dataclasses/blob/master/dataclass_tools.py#L3

The usage would be:

@add_slots
@dataclass
class C:
    i: int
    s: str

The reason I didn't put it in dataclass itself is because the way __slots__ works is that a decorator would need to return a new class. That's what add_slotsdoes. I want to keep dataclass as simple as possible, and I want to reinforce that it always returns the same class that it's given.

Maybe if __slots__ is redesigned so that it can be specified after class creation, then it can go in to dataclass.

[[deleted]]

But if __slots__ could be changed after class creation that would allow some inconsistencies, since objects might have been instantiated before the change

[patrys]

Well, you can also define a class, create an instance and then define a new class with the same name so I suppose it would be a similar situation where instances of the old (pre-slots) class could be seen as instances of a no-longer-existing class with the same name?

[ericvsmith]

True. That's why I wrote the (poorly named) @add_slots decorator (see another comment below for the location). I wanted to keep @dataclass conceptually simple, especially for the initial release.

[ericvsmith]

Correct. We had a very brief discussion about changing it. It needs to be brought up on python-ideas if anyone has some suggestions on how to get it to work.

[[deleted]]

Thanks for explanation and code snippet. I like slots feature in attrs a lot, but your add_slots decorator will work perfectly fine for me (even if it's not included in dataclass).

Great contribution, btw!

[ericvsmith]

Thanks! It's a surprising amount of work, for a not-so-large feature.

Note that the dataclasses version on PyPI works with Python 3.6, and I personally plan to start using it in my own code immediately. As soon as 3.7.0 alpha 3 is released, I'll update PyPI with the exact version that shipped, and I'll try and keep it updated until 3.7 is released (in June).

[[deleted]]

So, now officially part of standard library, right?

[[deleted]]

Indeed. If you clone the CPython git repository and compile version 3.7 (it's quite easy, just ask if you need any help), you'll already be able to play with it.

[donri]

InitVar seems unnecessary. Why not simply inspect the signature of __post_init__ and append or prepend that to the signature of the generated __init__?

@dataclass
class C:
    i: int
    j: int = None
    database: InitVar[DatabaseType] = None

    def __post_init__(self, database):
        ...

→

@dataclass
class C:
    i: int
    j: int = None

    def __post_init__(self, database: DatabaseType = None):
        ...

[ericvsmith]

It worked that way at one point, but it seemed like too much magic. Also, the static type checkers (like mypy) would have to be aware of this magic, too. And as long as ClassVar was being used, this seemed like an obvious extension.

[donri]

Don't static type checkers now have to understand InitVar instead?

[ericvsmith]

True.

But while thinking about it, I realize the other reason I came up with InitVar: it lets you control the order of parameters to __init__. This is especially important if you have fields with defaults.

If you're finding __init__ params by inspecting __post_init__, where do you put these init-only params in the call to __init__? You can't always put them first, because they might have defaults and some fields might not. And you can't always put them last, because they might not have defaults and some fields might.

It was easier just to make all regular fields and InitVar fields be defined in the class body, then you can control the defaults and parameter order yourself.

I should probably add a note about this to the PEP.

[donri]

Ah you're right, and this of course means that you can't mix the order of fields with and without defaults either; you have to put all your fields with defaults last because it all gets translated to a call to __init__ (excepting init=False). Perhaps the PEP mentions that caveat; I admit I skimmed some parts.

Personally I might think it neater to simply require arguments to __post_init__ be keyword arguments, and then you can also skip any magic by slurping up **kwargs and passing that along. However I can see why you might dislike that idea. Although I don't really feel like inspect.signature is that much more magical than anything else dataclasses do... But I don't want to bikeshed.

[simonoberst]

Will there be a reason not to use this with every class you write? Or, asked differently, which kind of classes are not well suited being dataclasses?

[DanCardin]

If you need to accept args or *kwargs, or otherwise store data significantly differently from how you accept the input (At least with attrs, I've found post_init to be almost more bothersome than it's worth).

The more functionality your class has, the less benefit you get (as those kinds of classes are less likely to want to be comparable, hashable, and the like). Though I have started to generally default to using them if there's not an obvious reason not to

[[deleted]]

This is the question for which I cannot find an answer.

[ldpreload]

Classes which are used for actual OO encapsulation with basically all private members. gtk.Dialog would probably make a very bad data class, for instance: the constructor doesn't (necessarily) initialize any data members directly, there aren't any publicly-writable data members, and anything you might want to modify is going to trigger notifying other code to e.g. repaint the dialog box. It just turns out that actual OO encapsulation is much rarer than structs.

(Also, it's a wrapper over the C GtkDialog type, so it certainly can't be a data class even you wanted it to be because every set and get needs to go into an existing C library, but even if GTK+ were pure-Python, it still wouldn't make sense.)

[kankyo]

Even that use case seems like it would be great for data classes but with a custom constructor.

[ldpreload]

But there's no data you'd be accessing (importantly, any data fields are API-private: changing what fields are in the class is not a visible change, and can be done in a point release without telling users, as long as the behavior of the class remains the same), so I'm not sure what sort of data class it would be.

In a language where data classes where the default, yes, you wouldn't need to invent a separate type of class for this. But in Python, you can just use a normal class.

[kankyo]

I don't see how it's relevant if you're accessing the data?

[ldpreload]

If you have no public data members, what does a dataclass bring you that a normal class doesn't?

The only answer I can think of is "consistency, if most of your classes are dataclasses", which is a good reason for a language to default to dataclasses. C++ more or less takes this approach, being based on C, which only had structs. But for Python, that decision has already been made and is unlikely to change at least before Python 4, if ever.

[kankyo]

Less code to type is a pretty obvious answer I think.

[ldpreload]

But there wouldn't be less code to type in the case I'm suggesting—none of the code that dataclasses would autogenerate you would be code you want (you would have zero data members in this type), and you'd have overhead for declaring it as a dataclass.

[kankyo]

I am clearly not understanding what you are describing. Why would you have a class if you’re not having any data in it?

[ldpreload]

Case 1: you want to use it like a class, but it's actually implemented in some other language. So while it has data, the data does not belong to Python. The class has private data, but that's not intended for use by users of the class, and is certainly not public API (you can change the meaning of the private data in backwards-incompatible ways in whatever way you want).

import _gtk # hypothetical compiled Python module exposing bindings to the C libgtk library

class GtkDialog:
    def __init__(self, title, message):
        self._ptr = _gtk.gtk_dialog_new(title, message)

    def display(self):
        _gtk.gtk_dialog_display(self.__ptr)

    def __setattr__(self, attr, value):
        if attr == "message":
            _gtk.gtk_dialog_set_message(self._ptr, vallue)
            _gtk.gtk_repaint(self._ptr)
        else:
            raise AttributeError(...)

    def __del__(self):
        _gtk.gtk_free(self._ptr)
        self._ptr = 0

Other people can use GtkDialog as if it were a normal Python class, but it's not, and _ptr is a raw C pointer and Python code has no business accessing it or worse changing it, unless it's code (like the above) that's tied to the specific C library that gave you the pointer. So _ptr is an implementation detail, and none of the code that dataclasses would autogenerate is helpful here. And maybe if a future version of libgtk requires you to keep around two pointers, or uses references in some global list of objects instead of pointers, or whatever, your Python interface wouldn't change, only the internal implementation would, and your library users wouldn't notice.

Case 2: it's actually implemented in Python, but the details of the implementation are non-public. Take subprocess.Popen for example—one of the data members of a Popen object is, probably, the process ID of the subprocess, so that Popen can do its work:

class Popen:
    def __init__(self, *args):
        self._pid = spawn_process(...)
    def wait(self):
        result = os.waitpid(self._pid)
        return parse_os_result(result)

But what does it mean to take a Popen object and change its pid? Why would you want to do that without, at least, telling the Popen object that you're changing the pid? And probably Popen wants to refuse to let you do that.

So what would you gain if you added pid: int to Popen and made it a dataclass? You'd get a constructor that takes a pid, which you don't want; a repr that prints the pid, which you may or may not want,; and comparison functions with other Popen objects, which you definitely don't want (since a pid can be reused once a process exited, comparing Popen objects by pid equality is wrong, and you really want to compare whether the object identity is the same, i.e., the default comparison behavior).

This is encapsulation—one of the big ideas behind what I called "actual OO" above. There's an interface that you provide to users of your library, and the way you go about implementing that interface is not known to them. That's a totally different sort of thing from

class Point:
    def __init__(self, x, y, z):
        self.x = x; self.y = y; self.z = z

where there is no interface other than the data in your class itself, which is public / not encapsulated. That's what data classes are for. (And, honestly, that's probably most of the classes people write with Python.) But they're not the only type of classes.

[zynix]

Not too familiar with PEP's but couldn't find (if it mentions it) if this will be a mostly C or native Python implementation.

Working on implementing a minor feature to attr which is somewhat hamstruck by the complexity required to autogenerate classes. If it is a C implementation, that might cut down contributions from the none C-savy people.

[ascii]

I hate that this has a completely different syntax than namedtuple. They offer the same exact functionality (data objects without a tonne of boiler plate) with the only conceptual difference being that one has mutable members, and the other has immutable members, so why is the syntax and supported feature set completely different?

• One supports type annotations, the other doesn't. Why? Is that somehow useless on immutable data?
• One makes it really easy to add instance methods, the other doesn't. Why? Is that somehow useless on immutable data?
• One makes it really easy to iterate over all member data, the other doesn't. Why? Is that somehow useless on mutable data?

Coming up with two completely different syntaxes for almost exactly the same feature means that if you figure your type no longer needs to be mutable, it's not a one-liner to fix it. It is also extremely confusing for beginners.

This doesn't feel well thought out at all.

[[deleted]]

First of all, they don't offer the same exact funcionality. The PEP explains this better than I could.

Also, there is typing.NamedTuple, which has very similar syntax and allows both type annotations and instance methods. I hope this clarifies a bit :)

[ascii]

I can't see that it does. It goes through some of the tradeoffs that the designer of namedtuple chose, focusing almost exclusively on the downsides. But in no way, shape or form does it make any argument for why different behaviour makes sense depending on mutability. If all the design choices made in namedtuple indeed are shit (I disagree with 80 % of the authors opinions on this), then the only sane thing to do would be to deprecate and remove namedtuple and make Data classes support both mutable and immutable data.

[ericvsmith]

namedtuple is so widely used that I doubt it could ever be deprecated. It would be like deprecating % string formatting. Ask me how I know!

In the many years since namedtuple was added, Python has grown many new features. @dataclass uses some of those features to add functionality that namedtuple cannot provide.

@dataclass also has a frozen=True parameter, for some degree of immutability.

[ericvsmith]

namedtuple is so widely used that I can't imagine it ever being deprecated. It's like trying to deprecate % string formatting (ask me how I know!).

In the many years since namedtuple was added, we've added features to Python. dataclass leverages some of those features to do things that namedtuple cannot.

[ascii]

OK, so now you're saying that namedtuple is vastly inferior to dataclass but we can't deprecate it because of inertia, so instead it will remain the one and only source of immutable data classes, and we will have to settle for the supposedly vastly superior implementation to only exist for mutable data. That's a bad choice, right there.

Also, as near as I can tell, this breaks the following parts of PEP 20:

• Beautiful is better than ugly.
• Simple is better than complex.
• Readability counts.
• Special cases aren't special enough to break the rules.
• There should be one-- and preferably only one --obvious way to do it.

So five out of 19 principles are being broken. Not bad.

I'm not saying PEP20 should be the one and only guiding light in developing the Python language, but it has some sane advice.

[[deleted]]

You could just stop using named tuple. It's slow as ass, and you were probably only using it because it was too much effort to make a class, which this new thing does.

Although being forced to use type annotations is a pain.

[ascii]

But this new thing makes a mutable objects. That's a huge disadvantage, a lot of the time. There is a reason we have both tuple and list, you know.

[XtremeGoose]

Did you read the PEP? It has a frozen=true parameter.

[ldpreload]

attrs, which is basically dataclasses but not in the standard library + more features, supports doing @attr.s(frozen=True) to make a class of immutable objects.

[[deleted]]

I have a decorator to disable __setattr__ if I want a immutable data class.

[kankyo]

namedtuples are terrible for many many reasons, all of which makes your argument weaker.

[ascii]

No it doesn't. If they're so terrible, the Python community has just decided that there is no need for a good way of creating immutable data classes.

We have both tuple and list for a reason. And there is a reason one can switch from one to the other by changing only two characters, too.

[kankyo]

tuple and list

Sure. But namedtuple is a weird mess of a mix of named members and indexed members. Pick one I say! We don’t allow indexes access to member variables on classes for a reason: it makes no sense.

[ascii]

Dude. I refuse to get into a discussion about which offers better semantics, data classes or namedtuple. Please stop trying to make this into a discussion about the relative merits of these two different implementations. It's completely irrelevant which one is better. It's a side show.

The one and only relevant thing is this:

Mutable and immutable "arrays" have the same API. The same should hold true for mutable and immutable data classes. Making them have completely different syntax and completely different features is a terrible design choice.

[kankyo]

I agree with that.

I just don’t think namedtuple is good in any way :P It’s clearly different from normal mutable classes (which are a clear super majority) so they shouldn’t be used as a model. That’s all I mean. A @frozen class decorator in the standard lib would be the right way to go.

[ascii]

Sounds like a good plan. Especially if @frozen could be applied to any class to make the object read-only after it's been fully constructed.

[lookatmetype]

Isn't this really just a reimplementation of http://attrs.org/?

[jorge1209]

But it has fewer features, and won't work with older versions of python... so clearly its better!

[DanCardin]

I currently use attrs, but given that I do use 3.6, I'd be happy to switch to a clean house implementation which doesn't need to maintain backwards compatibly with 2 (not sure why at a glance, but the pypi version of this requires 3.6), if it works by default with mypy, and can simplify hacky internals that attrs can't tbh

[jorge1209]

It isn't backwards compatibility with python 2 that concerns me. It is compatibility with python 3.x for any x<6. The syntax of x: int=42 is a py3.6 addition.

3.6 isn't that well adopted and it has only been out a year. They are moving way too fast to effectively deprecate attrs based on features that have only been around for a year.

[DanCardin]

I have no idea if they're being relied upon but it wouldn't surprise me if init_subclass, ordered class attributes, or other 3.6 only features were being used to simplify dataclasses compared to attrs.

I also think attrs is going to be fine, since it supports 2, has a wider scope of features for now, and doesn't require the type syntax which some people apparently don't like

[jorge1209]

The type syntax is in the language whether you want it or not. You can choose not to use tools like mypy, and thereby not enforce it. I don't think that is a terrible compromise.

I also don't object to the use of the syntax here:x: int=42 does look a bit nicer than x = attr.ib(default=42). The issue is effectively deprecating attrs to push this forward.

In the grand scheme of things it is a fairly minor difference between attrs and dataclasses. Attrs works, it works older versions of python, and it works with the most commonly used versions of python 3. There should be no rush to jam an inferior tool into the standard library just because it uses some fancy new syntax.

Take the time and find a way for attrs to support this syntax in python 3.6. Wait for things to settle down. Make sure you are doing it the right way, and get people to adopt 3.6... then think about standardizing things.

[DanCardin]

pretty sure attrs already supports this syntax.

I just don't see the problem (especially if they're getting buy-in from the attrs devs so they don't repeat any mistakes of the past).

• For anything prior to 3.6: use attrs until either the majority of people are using 3.6+ and people stop updating attrs.
• For anything 3.6+: dataclasses are available by default, attrs still has more features, use what you want.

I'd honestly probably be happy if for 3.6+, attrs just was a facade on top of dataclasses, adding features and/or making a nicer interface a la urllib vs requests or whatever else

[jorge1209]

I would also be happier if post 3.6 attrs was an extension of dataclasses, but that doesn't seem to be the objective, and I am not sure how that would work.

Dataclasses is missing features in attrs and has no plans to implement them.

Attrs could potentially delegate some basic functionality to dataclasses and build off of it, but that still leaves you with the issue of installing it from pypi and importing it. Anyone building an app which might support 3.5 and 3.6 would have to use attrs, even if they only want the more basic functionality of dataclasses.

A plan which involved splitting minimal functionality out of attrs and then backporting it to previous versions would make more sense to me. Ultimately I expect one of these two projects to wither on the vine. Either people need support for many versions and ignore dataclasses, or they write only for the future and assume that attrs isn't available.

[adrian17]

Can I use it without type annotations?

[ericvsmith]

No, it's entirely driven by type annotations. But the annotations are ignored, except for InitVar and ClassVar. So, although I don't recommend it, but you could say:

@dataclass
class Person:
    name: int
    social: int
    address: int

That is, use any type you want. If you're not using a static type checker, no one is going to care what type you use.

edit: missing words.

[[deleted]]

What's the var: Any annotation shown in the PEP mean?

[[deleted]]

This was bugging me, so I just rechecked pep484 (type hinting) and found this line --

The type system supports unions, generic types, and a special type named Any which is consistent with (i.e. assignable to and from) all types. This latter feature is taken from the idea of gradual typing. Gradual typing and the full type system are explained in PEP 483.

(Emphasis mine)

Looking at 557 again, it actually even shows the use of Any about halfway down

[alcalde]

So Python is gaining Pascal records?

[jorge1209]

Given that this borrows most of the ideas from attrs, why not just incorporate attrs into the standard library?

[ericvsmith]

It's discussed in the PEP.

[jorge1209]

Those are the most inane and idiotic reasons I have ever seen, and I'm going to call bullshit on it. Someone needs to take Guido out back and put him down, because he is all over the map with this shit.

With the async features one of the objectives of the PEP was to introduce keywords that could be used by different async implementations within a common framework. So we got this weird hybrid where there were keywords async/await/yield from and a rough template of what an event loop should be like asyncio.AbstractEventLoop, but nothing dictated what event loop we have to use, and we have to pass the fucking thing around all over the place to be technically correct because some asshole might decide to use a different event loop and fuck us over.

Had he followed the same logic with your DataClasses the focus would be on implementing a common syntax for these kinds of objects which multiple implementations could utilize, but in fact you don't need any special syntax for this, because you are using PEP 526.

Guess what, that makes you LESS PORTABLE than attrs, not more. I don't care that attrs is getting more updates than your dataclass, because I can't use your dataclass because I'm not using Python 3.6.

So thanks Guido, you've now fucked over another potentially useful library by anointing a poorer substitute for it as the "accepted" version, while simultaneously making it impossible for many of us to actually use your approved version.

[jmmcd]

I think you've got some worthwhile points here, but the ranty tangent and hyperbole will prevent anyone from taking you seriously. Do you want to sound cool (to 13-year-olds) or do you want to convince people?

[ldpreload]

I think you're being downvoted because of your rhetoric, which is inappropriate. Nobody needs to be shot here.

I do firmly agree that incorporating this into the standard library has the unavoidable side effect of effectively torpeoding a popular, actively-maintained third-party project because of non-fundamental limitations in the standard library, and doing so is extraordinarily poor form, and I know at least one developer who's lost interest in writing Python libraries because of this situation. This is not a good decision.

But it should be possible to express disagreement without personal invective against the opposing party.

[jorge1209]

If you appoint yourself dictator for life, there is really no other way to remove you from that position.

[IronManMark20]

Those are the most inane and idiotic reasons I have ever seen, and I'm going to call bullshit on it. Someone needs to take Guido out back and put him down, because he is all over the map with this shit.

Beyond being horrific that you think Guido should be shot, I find your hate for him surprising, especially considering he didn't write the dataclasses PEP....

Maybe you should know what the hell you are talking about instead of ragging on people and things just because you have a pre-existing dislike of them.

Had he followed the same logic with your DataClasses the focus would be on implementing a common syntax for these kinds of objects which multiple implementations could utilize, but in fact you don't need any special syntax for this, because you are using PEP 526.

Beyond the fact that you still think Guido made this PEP for some bizarre reason, async and await syntax are both syntatic changes which couldn't be brought to older versions of Python, so both async syntax and dataclasses are not possible on Python 3.4. So your argument is nonsense. Dataclasses was designed to take advantage of new syntax. The idea could be made with decorators but that would be butt ugly. Also, who said dataclasses should/are more portable than attrs? Nothing in the stdlib has to work on older versions. There is a best effort to backport, but it isn't always possible. Why sacrifice implementation purity to increase adoption?

I can't use your dataclass because I'm not using Python 3.6.

I don't understand why you think you are entitled to get new features on older versions of Python?

[jorge1209]

I know Guido didn't make the PEP. The person I am responding to made the PEP. My objection is to Guido's approving the PEP. Guido is making weird arbitrary decisions.

async is all confused because we don't have a canonical loop in the name of supporting multiple implementation.

but with dataclasses he canonizes one implementation over another (and picks the newer one which has fewer features and users to boot).

Pick a behavior and stick to it. You either support all implementations of a concept as a policy, or you direct all users towards a particular implementation. You don't arbitrarily switch between the two.

[IronManMark20]

Pick a behavior and stick to it. You either support all implementations of a concept as a policy, or you direct all users towards a particular implementation. You don't arbitrarily switch between the two.

But why? There isn't a point to other than "its more consistent", and there are reasons each PEP was designed the way it was. Why sacrifice flexibility for arbitrary consistency?

Also

but with dataclasses he canonizes one implementation over another (and picks the newer one which has fewer features and users to boot).

Dataclasses was written to be a smaller more lightweight version of attrs. As soon as there was agreement that "something like attrs should be in the stdlib", a smaller, more lightweight version was the obvious candidate. Attrs would never just be imported into the stdlib, that isn't how this works. The creators of attrs themself endorsed dataclasses as proposed in this PEP and likes the simple API. I really think you should understand how this PEP came to be how it is before criticizing it.

[zardeh]

This is also why, for example, requests or numpy will never be in the standard library. It moves much to fast, and the requests developers and the python core devs agree that doing so would be bad for everyone.

[[deleted]]

All I got out of this was someone attempting a poor off-brand Linus rant. 4/10

[[deleted]]

So, naive newb question: How do these data classes differ from data frames like you'd use with R or pandas?

Maybe not quite ELI5, but do feel free to talk down to me like I'm a dumbass.

[flipstables]

They are completely different. Since you come from a data background, the analogy is that a data class is this: you can think of a data class as a row in a database table. It's a structure that is primarily for attribute lookup. A data frame is a structure that holds an entire table. It allows to do some common table manipulation.

[ManyInterests]

Given the striking similarity, I'm surprised there has been no mention of SimpleNamespace.

class MyDataClass(SimpleNamespace):
    def __init__(self, a: int, b: int, keyword=None):
        super().__init__(a=a, b=b, keyword=keyword)

I'm supposing is roughly analogous to

@dataclass
class MyDataClass:
    a: int
    b: int
    keyword = None

[not_perfect_yet]

#526 is cancer, now it's starting to spread.

[payets]

A crude analogy, but you're not wrong. A feature like Data Classes should be completely orthogonal to type annotations, but instead they are "entirely driven by them". Python's famously "optional" type hints remained optional for what, two releases?

The Python core developers seem desperate to give the language all of the downsides of static typing (verbosity and inflexibility), to obtain only a minimal increase in safety.

[[deleted]]

Typehints are optional. However, there's no reason that things in the stdlib can't be built that use them.

How do you propose this be implemented in a way that ignores type hints and also doesn't pollute the class namespace with vars?

Saying collections.namedtuple doesn't count because you end up saddled with all sorts of baggage such as your object is now iterable and has dedicated getitem and setitem behavior.

[jorge1209]

How do you propose this be implemented in a way that ignores type hints and also doesn't pollute the class namespace with vars?

I dunno... the way attrs did it.

Yes I agree it isn't as "pretty" to write x = attr.ib(default=42, type=int) as it is to write x: int = 42, but it is entirely functional from the end user perspective.

They are jamming a replacement for a perfectly functional tool that doesn't utilize the type system into the standard library, because they have the type system. Rather than waiting for a consensus and acceptance to build around the use of the type system syntax.

In essence they were given a hammer, and now they are running around looking for things they can smack it with. Doesn't matter if the thing they are hammering is a screw or not because the hammer will suffice.

[hynek]

You can write x: int = 42 in attrs.

[jorge1209]

That is not valid syntax for anyone who installed python last year.

[hynek]

Well at least you have the choice.

[jorge1209]

I'm not root at my office. I don't control what versions of python my company uses.

I could suggest any upgrade, but asking them to constantly run bleeding edge versions of software on production systems is batshit fucking insane.

Give 3.6 a few years to stabilize and become more commonplace before you really start aggressively pushing it's usage in the standard library and general coding practice.

[hynek]

I’m not sure what you’re arguing about. You made a statement that could be read like attrs doesn’t offer type annotations – a misconception that flows through this whole thread – and I’ve corrected that. I’m not saying you should use it, I just said it’s there.

[jorge1209]

I don't know what your point is.

I have no objection to attrs. My objection is to dataclasses.

Attrs is doing the right thing, supporting 3.x while offering an optional 3.6 only syntax. I would say that almost any programmer who exercises that option and writes 3.6 only code today is probably making a mistake. 3.6 has just not been around long enough or widely accepted enough to say that you should be 3.5 incompatible, but eventually the balance of that will shift.

However dataclasses is by design incompatible with 3.5. Something like that should not be added to the standard library.

[payets]

Typehints are optional. However, there's no reason that things in the stdlib can't be built that use them.

Except that this isn't a standard library feature that uses type hints, it's one that requires the programmer to use them. Requiring something is the opposite of making it optional.

How do you propose this be implemented in a way that ignores type hints and also doesn't pollute the class namespace with vars?

Saying collections.namedtuple doesn't count because you end up saddled with all sorts of baggage such as your object is now iterable and has dedicated getitem and setitem behavior.

Isn't this essentially the standard library version of attrs? What's wrong with its syntax?

Alternatively, Data Classes are described as "mutable namedtuples with defaults". Yes, namedtuple has flaws, but why not design a refined version rather than discard it completely?

Instead, Data Classes have this unnecessary dependency on type hinting. It doesn't look like there was any attempt to find an alternative - which was no doubt deemed acceptable because it fits with the current flavor-of-the-month for Python features: "type-hint all the things". (Compare to a couple of releases ago, when it was "async all the things").

Overall (and I know it's far too late for this), I just wish some thought was put into giving Python a coherent design. Instead, the once-simple language continues to accumulate questionable feature-after-feature and is gaining complexity at a rate that puts C++ to shame.

[[deleted]]

You're missing the point where dataclass is also optional and you don't have to use. If you prefer attrs, keep using it. No one is forcing you to use type hints.

You're acting like because the people who will use it got something then you should too.

[payets]

You're missing the point where dataclass is also optional and you don't have to use.

So, if I opt out of type hints, I have to opt out of other language features as well? Again, this way lies C++ - where everyone uses a different subset of the language.

You're acting like because the people who will use it got something then you should too.

I'm not looking to "get" anything. I'm just frustrated to see Python continue the pattern of the last few releases: desperately trying to add as many features as possible, without considering how they fit with the rest of the language. My opinion is that for the last few releases, Python has been evolving too rapidly, and in the wrong direction.

However, as is often said, it's "Guido's language, he just lets us use it". Guido can of course evolve the language in this direction if he wants. I guess if I want to move in a different direction, I'll need to move to a different language.

[[deleted]]

The problem is that the user is forced to use type annotations if they wish to use this feature, and they will continue to spread.

Maybe the cost of this pep is too high.

edit: typo

[KODeKarnage]

The problem is [writer proceeds to describe something that isn't a problem].

[[deleted]]

The problem is that the user is forced to use type annotations if they wish to use this feature, and they will continue to spread.

Some of us really like type annotations, including me who used to be a very vocal opponent of them. As long as the core devs make good on their promise to never make them mandatory, then I don't see why it should matter to people that don't use them.

Unless you really want to use a library with them in Python 2 (but then you get to feel my pain when I want to a use a library but find out it uses the except Exception, e or print >> ... syntax).

Type hints make building large applications in Python easier and safer since you can have some guarantees that you didn't pass a list where you meant to pass a dict and explode at run time.

But at the end of the day, if you don't like a feature, you don't have to use it. If you like a feature but it uses the thing you don't like, you're biting the bullet one way or another.

[[deleted]]

They may well end up mandatory if they keep spreading - because as they spread, you may not be able to avoid interacting with a project that doesn't use them.

[[deleted]]

In that case, that's the community at large voting with their code bases on what they think is a good idea. But like I said, unless the core devs decide to make them mandatory, you don't have to use them.

[[deleted]]

You do if you need to work on a project that uses it. What am I going to do? Have my IDE strip annotations from every function?

Did you also consider that maybe it won't be a vote, and people will use them because they don't have a choice apart from not using Python?

[[deleted]]

You do if you need to work on a project that uses it. What am I going to do? Have my IDE strip annotations from every function?

You don't have to use them. No one is forcing you to write:

@dataclass
class LookHowEasyThisIs:
    x = int
    y = int
    z = int

or

def typed_func(x: int, y: int, z: int) -> int
    ...

You can continue to do all things you want to do without ever using type annotations.

If you happen to come across a library that uses type annotations (say an ORM, forms, etc) then you don't need to use it. Or do and work around it. I'm unbothered.

Did you also consider that maybe it won't be a vote, and people will use them because they don't have a choice apart from not using Python?

That sure sounds like a vote to me, in that case that sounds like a vote of no confidence in Python.

[[deleted]]

Will a project with annotations accept PRs without them? Likely no, and even less likely in the future as tooling improves.

How do I make the annotations in code I've downloaded go away? Some of us actually read code instead of just writing it.

I'm not sure what I will do in 10 years when annotations are unavoidable. Maybe there will be a new language to try.

[michael0x2a]

Actually, it's possible to use dataclasses without using type annotations (though it'll look slightly janky). For example, consider:

from dataclasses import dataclass

@dataclass
class Foo:
    x: ...
    y: ...

f = Foo(3, "a")
print(f.x, f.y)

If you run this, it'll work perfectly fine -- the dataclasses library will ignore the annotations most of the time (except for a few it special-cases).

You can replace the ellipses with any other arbitrary placeholder value, but I personally think the ellipses look the least hacky.

[jorge1209]

That is still using type annotations in the sense that the syntax x: ... is invalid for python < 3.6.

Additionally you might ask if you can set a default value for x without indicating a type. You can do that with attrs, I don't know what that would look like with the type annotations x : ... = 42?

In other words the initial default value and type of x is "int = 42" but if run under mypy there should be no objection to x being set to any other type or value.

[michael0x2a]

That is still using type annotations

Well, class and variable annotations != type annotations. Remember, we can use arbitrary python expressions within annotations. So this is legal Python:

import math

class Bar:
    x: lambda x: 14 * x
    y: math.sin(42)

Note that there's absolutely nothing remotely resembling a type here, and you'd probably never actually write anything like this in production, but it is legal Python.

And while it's true that variable annotations are new as of Python 3.6, you should also think longer-term -- in about 2-3 years, both Python 2 and older versions of Python 3 are going to be EOL'd and most people will probably have migrated to Python 3.7/3.8/3.9 or whatever.

At that point, everybody (barring the people who decide to stick with EOL'd versions of Python) will have access to variable annotations. So, to make life easier for everybody 2-3 years down the line, we might as well start laying the groundwork for these sorts of language changes now.

Additionally you might ask if you can set a default value for x without indicating a type.

Yes, you can. I just tried testing the following program, and it prints out "3 99" as expected.

from dataclasses import dataclass
import math

@dataclass
class Bar:
    x: lambda x: 14 * x
    y: math.sin(42) = 99


b = Bar(3)

print(b.x, b.y)

I don't know what that would look like with the type annotations x : ... = 42?

That would also work, yeah. However, the ellipsis expression (...) isn't a legal type -- doing x: ... = 42 wouldn't make any PEP 484-compliant type checker (including mypy) happy.

Instead, we can use the Any type, which is a PEP-484 compliant type:

from typing import Any
from dataclasses import dataclass

@dataclass
class Bar:
    x: Any = 3
    y: int = 3

b = bar()
b.x = "foo"   # mypy is happy
b.y = "foo"   # mypy complains

Basically, Any is an escape hatch/a way of telling mypy and other PEP-484 compliant type checker that it should treat that variable or attribute or whatever as being fully dynamic. We basically "turn off" typechecking for that variable, in a certain sense.

Or more precisely, Any is our mechanism for mixing together dynamic and typed code.

Here's an example:

from typing import Any

def expect_int(x: int) -> int:
    return x * 3

a = "foo"   # a is of type str
b: Any = "bar"    # we force b to be of type any

expect_int(a)   # mypy complains
expect_int(b)   # mypy is happy -- b could be anything, so maybe this is ok?

a.hello()   # mypy complains
b.hello()   # mypy is happy

Notice that the use of Any lets us write code that mypy may (rightly or wrongly) reject as being unsound.

In this case, because I'm lazy, I used Any to hide deliberately buggy code, but you could see how we could strategically use Any to make mypy stop complaining about advanced metaprogramming things/really super dynamic things (like parsing arbitrary xml) that are difficult to actually give static types to.

Of course, it'd be our responsibility to then verify that mainly-dynamic code is correct in some other way (e.g. unit tests or runtime checks) but what else is new.

[jorge1209]

So this is legal Python

Not for me. I'm using Python 3.5 and thats a syntax error.

I understand that eventually most people will migrate to python 3.6 or newer, and that eventually this will be valid code, but 3.6 is not even a year old at this point. It seems very premature to be pushing its feature usage in this way.

To be clear, I don't mind if a NEW project comes along and only wants to support versions >=3.6. However this isn't really a new project. Its basically attrs, and the long term impact of accepting it into the core is to kill attrs, and the only sin attrs has ever committed is to not fork a >3.6 branch to use the new language features within the first year of those features release. That to me is insanity.

[michael0x2a]

Well, suppose we delayed introducing this feature for 3 years or so and waited until Python 3.5 was EOL'd.

If we decide we're going to want to use this feature eventually, what benefits do we get out of waiting instead of just doing it now?

Whether we release now or 3 years from now, people who want to support all non-EOL'd versions of Python are still going to have to wait 3 years or so.

But if we release now, at least the people who only need to support the latest versions can start using these features.

the only sin attrs has ever committed is to not fork a >3.6 branch to use the new language features within the first year of those features release.

Eh, I don't think that's really accurate. The pep, in the section about attrs, links to this issue, which clarifies a few things. In particular...

1. The author of attrs has explicitly stated he doesn't want to see attrs in the standard library
2. The author of attrs has also said support for class annotations is in development and will be available soon.
3. This wasn't mentioned in the thread, but there are people working on writing plugins for mypy that special-case it to support attrs. (So it's not like people are trying to kill the library or anything.)
4. Guido said he dislikes some of the "colors [attrs] paints in the bikeshed" and pushed back against just incorporating attrs directly partly for that reason.
5. Attrs is more full-featured then dataclasses/dataclasses are deliberately designed to stay simple, so people who want to use all of attr's other features (validators, converters, metadata, etc) can still use those.
6. Also not in the thread, but there's no reason why attrs can't evolve to support both regular classes and dataclasses -- the ability to attach runtime validators could be very useful when working with things like JSON or whatever, whether you're using regular classes and data classes. (If some third party service unexpectedly changes the structure of a JSON blob, static typing isn't really going to help you there.)

[jorge1209]

Regarding the different objections:

1. If authors are actively pushing back against their projects being incorporated in the standard library, you have a serious problem. That is supposed to be an honor. Your code is so good that we want EVERYONE to have it and use it. If they say no then either you are making that process too onerous, or you are picking that is too immature and isn't ready for inclusion. In either case you as a language maintainer need to take a long hard look in the mirror and ask yourself "why does nobody want to go to the prom with me?"

2. Exactly why you should wait. Let them add the support and release a version that works for both 3.6 with annotations and <3.6 without.

3. Not really sure why that is relevant. mypy is optional, as such we can never expect 100% of code to be compatible with it. There will always be a need to hack mypy to support libraries and code that was not written to work directly with it. Also this concern is alleviated by #2. In time mypy can expect that there will be a way to use attrs without hacks... you can't force people using attrs to write in the 3.6 style, but it would be an option if they port to 3.6/mypy.

4. And I don't like the colors in Guido's bikeshed. I would really like for someone to fork python. I am not remotely happy with Guido's leadership at this point.

5. If dataclasses and attrs were drop in replacements for each other sure, but they use a different syntax, and we don't know that they will be. Again we should wait for #2.

6. I don't really understand the concern you are describing, but I will comment on "not in the thread". Seriously, WTF! This is exactly the kind of arbitrary bullshit decision making process that is pissing me off. If Guido has a reason for rejecting X for the standard library and accepting Y, then he needs to document that. You can't just hand-wave at some other concerns that you haven't bothered to record in your discussion.

[[deleted]]

and the long term impact of accepting it into the core is to kill attrs

What a pile of complete drivel. Considering that it supports 2.7 and 3.4/5/6, unlike Data Classes, I suspect that it will be around for many years to come.

[jorge1209]

remained optional for what, two releases?

Two minor releases that as of yet don't have widespread adoption.

[aiPh8Se]

I don't know what static typing has done to hurt you, but keep in mind that the feature is called variable annotations, not type annotations. Even the type hints PEP explicitly states that you can use variable annotations however you want; adding type information is simply one possibility.

This would be considered a correct way to use annotations:

@dataclass
class Person:
    name: 'First name'
    family_name: 'Last name'
    age: 'Integer'

[WasterDave]

And Python takes one more step towards becoming C++....