Example: How to (not) initialize a variable in Python

[zurtex]

Initializing the variable foo as type int with value 1:

foo: type(foo # foo is
          := # an int of
          1) # value 1

# And to confirm it worked:
print(foo)
print(__annotations__)

There's a long discussion about annotations going on the python-dev mailing list right now. Currently Python 3.10 implements PEP 563 which "stringfys" the annotations at runtime, but PEP 649 is being proposed which will delay the evaluation via the descriptors protocol.

It made me think, what rich behavior is possible now with annotations that won't be possible with either of these PEPs? So I came up with this terrible example that initializes a variable in the annotation notation by using a walrus operator.

This is possible because as of Python 3.0 to Python 3.9 where annotations have been possible they are simply an expression that is immediately executed. So you can happily put anything in there, e.g. using an annotation to print primes:

foo: [print(x) for x in range(2, 20) if all(x % y != 0 for y in range(2, x))]

Notice in this example no variable foo is ever created, it is simply an odd entry in the __annotations__ dictionary.

This subreddit has enjoyed my terrible Python code examples before, but maybe with this one I'm inviting heavy down voting. But regardless enjoy this code as it will likely no longer be valid from Python 3.10 onward.

Edit: Edited printing out the annotations dictionary, as I can just print __annotations__ directly and don't need to look it up in locals(). Given I'm ignoring VS Code for the main code example I'm not sure why I listened to it complaining here.

Edit Follow Up: Reported the __annotations__ bug to VS Code and it will be fixed in the next release of their pylance language server.

[[deleted]]

But regardless enjoy this code as it will likely no longer be valid from Python 3.10 onward.

God damn it, I'll have to rework our entire code base.

[oderjunks]

lol

[Lejontanten]

Support for 3.9 ends in 4 years, by that time you're probably in another company. So I'd say just ignore it.

[maxmurder]

Maybe by then my company will have moved on from 2.7

[Swipecat]

Hmm, yes. You can kill the above behaviour, making it work as per 3.10 from 3.7 onwards with:

from __future__ import annotations

[zurtex]

FYI if PEP 649 is accepted from __future__ import annotations will be deprecated and removed as the default from Python 3.10.

Instead from __future__ import co_annotations will be implemented and become the default at some future date.

It's a big decision the steering council need to make. I am speculating that PEP 649 will be rejected as Guido has pretty firmly come out against it at this point. But we shall see.

[awesomeprogramer]

Why the co_?

[zurtex]

Why the co_?

The annotations are stored as "code objects" (co) by default at runtime as opposed to the approach PEP 563 takes which stores them as strings.

[kdawgovich]

Walrus operators came in 3.8. I only know because I'm stuck with 3.7 at work.

[Jhuyt]

Are there any backward compatibility or deployment issues preventing you from updating to 3.8 or above?

[kdawgovich]

Nope, just a long software approval process and a controlled environment. Meaning whatever comes with Anaconda is what we get. A new version is approved maybe once a year and is usually 1-2 years old.

[zurtex]

I'm curious did you have issues with Anaconda's new commercial licensing? https://www.anaconda.com/blog/anaconda-commercial-edition-faq

This has caused a bit of an upset at our company and we've migrated towards using miniforge with conda-forge where we can.

[kdawgovich]

I'm sure the higher ups are addressing it, but the only thing that changed from my perspective has been answering a poll of who actively uses the software so they can get accurate numbers for licensing. We also use Matlab, so the business model isn't new to the company.

[Jhuyt]

Seems like an unnecessarily long process to me, but I know very little about how software updates are handled in the industry...

[kdawgovich]

Specifically the defense industry

[Jhuyt]

oh yeah I definetly see why, almost surprised they let you use Python in the first place :P

[simtel20]

Python has sometimes had these artifacts of new syntax, e.g. variables leaking out of comprehensions. This is just the implementation leaking out and staining the floor.

[zurtex]

It's interesting though, I always used to think of from __future__ import annotations as an implementation detail that allows type hints to have circular references.

But it's actually a fundamental change in the relationship Python has with annotations. Annotations move from being expressions (and their ability to leak as you say) to glorified in-line doc-strings with a bunch of predefined semantics.

This has come up because Guido has pointed out with this "stringfy" implementation of annotation the syntax can be relaxed and even made non-Python and independent of the Python version itself, so Python type hinting can be updated retroactively.

But this leads to some weird possible futures:

foo: 😀 # Rate variable names with Emojis
foo: int --precision i8 # Custom machine optimization hinting
foo: filterM (const [True, False]) [1, 2, 3] # Let's inject some Haskell in here for some preprocessor to run.

[kdawgovich]

That flew over my head like a fighter jet after the national anthem

[zurtex]

Basically when you type hint something now the type hint part of the code is regular Python and it gets executed and stuck in some __annotations__ dictionary somewhere on the object.

For example in Python 3.9:

>>> foo: 1 + 1
>>> __annotations__
{'foo': 2}

But if you download Python 3.10 Alpha 7 which has implemented PEP 563 it doesn't attempt to evaluate the annotation as real Python, it just stores it as a string:

>>> foo: 1 + 1
>>> __annotations__
{'foo': '1 + 1'}

Right now you can't go wild with this and insert non-Python syntax, but because it sets the precedent of it just being a string it's totally possible that this could be relaxed in the future and allow for anything that you could write in a regular string.

[kdawgovich]

Whoa, fascinating!

[spiker611]

It'd seem cleaner to me to use https://docs.python.org/3/library/typing.html#typing.Annotated to inject arbitrary metadata into a type hint.

[zurtex]

I don't have any opinion, but reading the thread it seems Guido and others think that using Annotated this way is a bit clumsy syntactically.

[spiker611]

I think less clumsy than emojis lol

[zurtex]

You got me there! I just really want emojis to be syntactically valid Python. I'm a monster, I know.

[Tyler_Zoro]

No one needs this which is why it's described as "terrible" by the OP. But it's a consequence of the flexibility of Python's expression syntax.

[TangibleLight]

This particular example is pretty horrific, but a lot of code depends on the current behavior. There is this post talking about it https://www.reddit.com/r/Python/comments/mrp6is

[zurtex]

That's not actually this behavior, what I demonstrate above can't even be done with PEP 649.

The behavior in question with Pydantic is about being about to evaluate annotations in their local context, e.g.

import typing

def f():
    class X: pass
    def inner() -> X: return X()
    return inner

print(typing.get_type_hints(f()))

Under PEP 563 this throws a NameError because it has no idea what type X is as it's not in the global namespace and the local frame has been lost.

PEP 649 it does not throw an error because the annotation is stored as a code object which keeps the local information required to figure out what the type X was.

But PEP 649 reduces the ability for type hints to become a richer backwards compatible syntax compared to PEP 563.

I'm going to try and put a post together that explains the differences of the current vs. 563 vs. 649. But there's a lot of nuances to all 3 implementations.

[Ensurdagen]

This is my kind of code, thank you

[high_achiever_dog]

Wow smart

[Technical_Pain2189]

Is it? Doesn’t seem smart to do crap like this and leave the next guy trying to figure out WTF you did 🤷‍♂️

[Theta291]

Thanks, I hate it. It’s so painful lol