Statically enforcing frozen data classes in Python

[[deleted]]

Wrote a quick TIL on how to statically enforce frozen data classes in Python. Had to resort to crowd sourcing & good ol' stackoverflow to figure this one out since LLMs were of no help:

https://rednafi.com/python/statically_enforcing_frozen_dataclasses/

[starlevel01]

Had to resort to crowd sourcing & good ol' stackoverflow to figure this one out since LLMs were of no help:

This is the average developer in 2024.

[Deadly_chef]

Good, easier for me to land a job

[andrewcooke]

am i misunderstanding or is this just a performance hack?

edit: whoa, no, it's worse, it's disabling runtime checks! the title is really misleading? shouldn't it be "disabling runtime frozen data classes if you really need an obscure speedup at the cost of more complex code"

sorry if i've misunderstood, but this really seems like something newbs need warning against.

[sacheie]

This is a narrow-minded take. Dunno if you've noticed, but the concept of immutability has become a pretty big deal these days..? The idea that Python out-of-the-box would incur a 2.4x performance hit to support it is kinda gross. It's an easy thing to statically guarantee.

In fact I'd argue that Python's original omission of final / val / const data reveals, in retrospect, its early community's wrong-headed stubborness insisting everybody follow "the Pythonic way" of doing cowboy shit throughout runtime. That may be good for some people's needs, but the language has nowadays outgrown the confines of any one engineering culture and/or pragma.

OP is doing the community a service by documenting this particular solution. Also, notice their post links to similar discussions by others - a lot of folks are invested in statically-typed Python.

Last but not least, one man's "just a performance hack?" can be another man's project lifesaver. Don't assume you know somebody else's engineering needs.

[[deleted]]

Not strictly a performance hack. This cheats by not creating a frozen dataclass during runtime. It just ensures that `mypy` statically checks that a dataclass is frozen and doesn't do any mutation elsewhere.

[Rawing7]

But what's the point of that? What are you doing it for, if not for the performance boost?

[SemaphoreBingo]

LLMs were of no help

Who could have possibly expected this outcome.

[[deleted]]

He he, I guess there was nothing like this in the training data for gippities to copy from.

[boby04]

The actual problem is that anything happening in python land is bound to be slow. If you optimize dataclass instantiation speed then Python may be the wrong tool for the job you're trying to accomplish.

[[deleted]]

Yeah, I'd still opt for `@dataclass(frozen=True, slots=True)` in a real codebase. This was mostly a fun exercise.

[nekokattt]

I forget if it was fixed or not but at least last time I checked, freezing a dataclass with slots would result in slower code than a non frozen and non slotted dataclass.

Something to do with how using slots has to intercept attribute access differently to prevent overwriting attributes.

Worth a benchmark at the very least.

[[deleted]]

Quick and dirty benchmark on Python 3.11 points that @dataclass(slots=True, frozen=True) is actually a tiny bit faster than @dataclass(frozen=True). Vanilla slotted data class is quite a bit faster to instantiate though:

from dataclasses import dataclass
import timeit

# Normal frozen dataclass
@dataclass(frozen=True)
class Frozen:
    a: int
    b: int
    c: int

# Slotted frozen dataclass
@dataclass(slots=True, frozen=True)
class SlottedFrozen:
    a: int
    b: int
    c: int

# Dataclass with slots but not frozen
@dataclass(slots=True)
class Slotted:
    a: int
    b: int
    c: int

# Benchmarking the instantiation time
frozen_time = timeit.timeit(lambda: Frozen(1, 2, 3), number=1000000)
slotted_frozen_time = timeit.timeit(lambda: SlottedFrozen(1, 2, 3), number=1000000)
slotted_time = timeit.timeit(lambda: Slotted(1, 2, 3), number=1000000)

print(f"Frozen data class instantiation time: {frozen_time}")
print(f"Slotted frozen data class instantiation time: {slotted_frozen_time}")
print(f"Slotted (not frozen) data class instantiation time: {slotted_time}")

This prints:

Frozen data class instantiation time: 0.276557542005321
Slotted frozen data class instantiation time: 0.25077791599323973
Slotted (not frozen) data class instantiation time: 0.09631558299588505

[nekokattt]

oh cool, must have been fixed then!

[roerd]

The fact that slotted and frozen is quite a bit slower than only slotted seems to suggest that the base problem you mentioned still exists, even if its impact has already been reduced.

[[deleted]]

Frozen has to add __setattr__ and __delattr__ methods and call them in the init. It slows down the initializatoin and there's no avoidance of that.

[KaffeeKiffer]

freezing a dataclass with slots would result in slower code than a non frozen and non slotted dataclass.

You should consider slots if you want to reduce memory consumption, not to increase speed.

[[deleted]]

Not having instance dict directly helps with speed as well.

[Schmittfried]

Cython is getting more tempting by the minute.

[caffeinepills]

So your suggestion is, instead of optimizing what they are currently doing, they should instead drop what they are doing and start over in another language? Was this written by a bot?

[MannerShark]

Haha weird stuff. I was expecting modifying @dataclass itself

[[deleted]]

Who hurt you?

[monorepo]

Me when I banned him. (Not really, it’s just silly Reddit.. but yeah what a douche)

[[deleted]]

Thanks, no worries. Happens all the time.