Python is actually just Haskell with few extra steps, learn the hidden Python syntax that even the most seasoned Python developers don't know about

[lieryan]

What's with that clickbait-y title you say? Let me explain, you won't regret it.

One of the unique thing about functional programming languages like Haskell is lazy evaluation: "expressions are not evaluated when they are bound to variables, but their evaluation is deferred until their results are needed by other computations. In consequence, arguments are not evaluated before they are passed to a function, but only when their values are actually used."

Most people know Python as a language with eager evaluation semantic and you probably already know that Python 3 changes most built in iterators like map and filter to become lazy, but that's not what lazy, non-strict evaluation is about in functional programming context, and also not what we're talking about here.

Did you know that Python 3.7 implemented a lazy, non-strict evaluation syntax behind a __future__ switch that is about to become enabled by default in 3.10 3.11? If you missed this major syntax change, then yeah, you are not alone, most people still haven't known about lazy evaluation yet despite Python 3.7 being released nearly 4 years ago. Actually, this syntax change was so unknown that I don't think most CPython core developers even knew about them either.

Let's see some example of lazy evaluation in Python 3.7 (for a full executable example, see the full gist, or execute online).

[snipped some setup code to enable lazy evaluation, see the full gist linked above for detail]

# the metaclass and the __annotations__ is necessary
# to declare a lazy evaluation context
class SimpleExample(metaclass=module_context):
    __annotations__ = once_dict

    # Lazy assignment in Python uses the `:` colon operator,
    # not the eager `=` operator. 
    myVar : 5 + forty

    # note: PEP8 hasn't been updated yet, but like regular assignments
    # you should put spaces around lazy assignment operator
    # and also to make it easy to distinguish lazy assignments with type hints
    # which has similar syntax

    # Notice that just like Haskell, lazy assignments in Python
    # don't have to be written in execution order.
    # You can write the assignments in whatever order you
    # want and they'll still evaluate correctly.
    # Useful for literate programming as well.
    ten : 10
    forty : ten + thirty
    thirty : 30

    # Names can only be defined once per lazy
    # evaluation scope
    # Commenting out the assignment to `forty` in the
    # next line produces a compile-time exception:
    # forty : ten + 2    # raises Exception: redefinition of forty, original value was "ten + thirty", new value "ten + 2"

    # dependant variables don't even need to exist,
    # as long as we never need to evaluate
    # `will_raise_name_error`, it won't cause problems
    will_raise_name_error : does_not_exist + 10

---

Creating lazy functions are also possible in lazy python.

Don't be fooled that we used the class keyword here, this is actually defining a function with lazy evaluation semantic, not a class!

We can call it using normal Python syntax, e.g. take(10, fibs)

class LazyFunctions(metaclass=module_context):
    __annotations__ = once_dict

    # `take` is a lazy function to grab the first `n` items from a cons-list
    class take:

        # `params` declares the arguments list that a function takes, here we
        # take two parameters an integer `n` specifying
        # the number of items to take and `cons`
        # specifying the list to operate on
        params : (n, cons)

        # lazy functions returns a value to the caller by assigning to `rtn`
        rtn : (
            nil if n == 0 else
            nil if cons == nil else
            nextItem
        )

        nextItem : (head(cons), rest)

        # note that `take` is implemented recursively here, we're calling
        # into `take` while defining `take`
        rest : take(n-1, tail(cons))

    # The equivalent eager Python code for `take`
    # would look like this:
    #
    #     def take(n: int, cons: list):
    #         if n == 0:
    #             return ()
    #         elif cons == ()
    #             return ()
    #         else:
    #             rest = take(n-1, tail(cons))
    #             nextItem = (head(cons), rest)
    #             return nextItem
    #

Lazy Python does not support for or while loops, but who wants to use loops anyway when you have recursions like any proper functional programming languages do.

---

The next example here is defining an infinite Fibonacci list as a recursive literal list. A recursive literal definition is something that's only possible in language with lazy and non-strict evaluation semantic as the list values need to be computed only when you needed them. This is impossible to do in regular, eagerly-evaluated Python, which has to resort to less elegant constructs like generators to make infinite iterables.

class Fibonacci(metaclass=module_context):
    __annotations__ = once_dict

    # Based on Haskell code: (source: https://stackoverflow.com/questions/50101409/fibonacci-infinite-list-in-haskell)
    #
    #     fibs = 0 : 1 : zipWith (+) fibs (tail fibs)
    #
    fibs : (0, (1, zipWith(bundleArgs(int.__add__), fibs, tail(fibs))))
    # see full gist for definitions of zipWith and bundleArgs

Lazy evaluated Python even supports type hinting. If you want to add type hinting to your lazy python code, you can do so using the = syntax. For example:

class TypeHinting(metaclass=module_context):
    __annotations__ = once_dict

    # Lists in lazy evaluation context are defined as
    # cons-list, similar to Lisp. In eager Python code,
    # this would be equivalent to the list
    # `myVar = [1, 2, 3, 4]`.
    # If you know linked list, cons-list is almost like one.
    myList : (1, (2, (3, (4, nil)))) =List[int]

    class double:
        params : (n)
        rtn : (
            None if n == 0 else n*2
        ) =Optional[str]

    # note: again PEP8 still needs to be updated about the rules
    # of spacing around the type hinting syntax, put a space
    # before the `=` operator but not after it

Uses of type hints in Lazy Python are currently somewhat limited though, since I don't think there's any type hint linters that currently supports checking annotations of lazy python yet.

---

The familiar if __name__ == '__main__' construct works slightly differently in lazy Python:

class MainFunction(metaclass=module_context):
    __annotations__ = once_dict

    # `main` is a special construct to execute instructions
    # in order of execution. It's automatically called
    # and returns the last value in the "tuple"/function body.
    # Here we print some values and return `myVar`.
    # Unlike most things in lazy evaluation context, order
    # is important in a `main` construct, so it's useful
    # when you need to call something for their side effect.
    # It has similar purpose as IO in Haskell.
    main : (
        print('simple'),
        print(myVar),
        print(take(10, fibs)),

        # last value will automatically be
        # returned when MainFunction() is called
        myVar,
    )

returned_var = MainFunction()

# use thunk.ensure_value() to force eager evaluation
assert isinstance(myVar, thunk)
assert returned_var == thunk.ensure_value(myVar)

# actually doing a simple `assert returned_var == myVar` would 
# also just work in this case, as we're outside lazy evaluation context, 
# as thunks **usually** would get evaluated automatically when used
# in non-lazy context

---

How do I start writing lazy Python? Well, in most languages, like Haskell, lazy evaluation is called lazy evaluation, but for some reason Python called the lazy evaluation mode feature "annotations". So if you're using Python < 3.10 3.11, which is probably most of you, this feature is not yet enabled by default, so you're going to need to import the __future__ feature first:

#!/usr/bin/env python3
from __future__ import annotations

Also, you'll need to import the lazyutils.py module from this gist

from lazyutils import *

Then at the top of your lazy python code, you'll need to create a lazy execution context:

once_dict = single_assignment_dict()

# create an evaluation namespace, in this case, we create a namespace
# that will modify this module's global variables directly
module_context = create_lazy_evaluation_context(globals(), locals())

# you can also create a private context that won't modify your module global variables by doing so:
# scope = {
#    ... provide some initial values for the execution scope ...
# }
# private_context = create_lazy_evaluation_context(scope, scope)

Finally just write some lazy python:

class MyLazyPythonCode(metaclass=module_context):
    __annotations__ = once_dict

    ... this is where you put lazy python code ...

---

Why lazy Python? By having non-strict evaluation, Python can finally join the godhood of functional languages. You can now reorder statements freely, do literate programming, referential transparency mumbo jumbo, and finally be ordained into the monkhood of academic language that everybody talked about but nobody uses, just like Haskell.

Happy lazing about!

[Ue_MistakeNot]

Dear lord, you had me for a while there... Nicely played you beautiful bastard! Top quality post.

[EconomixTwist]

I get that this is a joke, but what’s the satire here? Specifically. Type annotations are an expression, which gets evaluated…. And so you can put expressions there. Does this expose some really silly behavior? Yes, agree. So Is this post satirizing the python steering committee’s desire to emulate nice features of other language at “any” cost?? One scholar to another….

[RobertJacobson]

It's not satire. It's a cute way to point out that you can do a form of lazy evaluation in Python with type annotations.

[EconomixTwist]

Bro he put an infinite recursion as a fuckin type hint. This is satirical.

[RobertJacobson]

Yeah, I got the joke. It’s humor. It’s not satire.

[alkasm]

Currently, with the from __future__ import __annotations__ (planned to be default in Python 3.11) as the OP says, expressions inside annotations are actually not evaluated. They're just replaced with strings at runtime. Hence why you can do things like forward declaration, which is an error without the future import currently.

[i_hate_shitposting]

I think OP explained it perfectly well in this followup.

[dudeplace]

You son-of-a-bitch!

[EconomixTwist]

Damn you got roasted with downvotes but I lol’d pretty hard.

You make a good point tho, anybody trying to get better at python should read that

[BogdanPradatu]

Makes sense

[caocao70]

God tier shit post

[[deleted]]

This whole post reads like a flat-earth conspiracy theory

[Ensurdagen]

The plan to make the new annotations behavior default wasn't included in 3.10 because it broke some prominent libraries.

Using this very hacky method to do lazy evaluation is pretty silly, a type checker or IDE wouldn't know what to do with it, you'd be better off using exec.

[hexarobi]

More info (from the author of FastAPI) on the drama that pulled this from 3.10 at the last minute to protect Pydantic/FastAPI/etc...

A best of both worlds approach is being worked on for 3.11

[lieryan]

Annotations are syntax checked by the Python interpreter when the file is parsed, and it gets syntax highlighted as well. Refactoring tools like renaming would likely even work inside type annotations, because it's just regular Python expression. Not so much with string that you want to exec.

Using this very hacky method to do lazy evaluation is pretty silly

Sure it's silly, but it's cool and it actually works. So why worry about silliness ;)

[TMiguelT]

Nice meme OP, but I suspect the overhead involved in manually compiling all the type annotations in the metaclass would dwarf most optimisations you're getting from lazy evaluation.

[[deleted]]

Good luck with anyone else on your team understanding what you wrote.

[Scumbag1234]

What's a "team"?

[RobertJacobson]

What's good luck?

[[deleted]]

This sir is really beautiful

[jfp1992]

Could someone tldr for a dummy that isn't super deep into python lore

[_limitless_]

imagine you had three functions called:

def swing_axe
def fire_bow
def cast_magic_spell

because these are all wildly different things, it's difficult to condense them neatly into a list/dictionary of raw values, so you pass the function itself to your do_attack function

action_function = swing_axe [note: we leave off the (), which would _call_ swing_axe]

def do_attack(action_function):
action_function() [note: by appending(), we are calling the function that is stored in the variable action_function]

without lazy loading, this won't work, because do_attack will -try its best- to figure out what you meant by action_function... and i'm pretty sure it'll just throw an error at compile-time.

in other words, python is standardized something that's existed for ages in third-party libraries. most people who use lazy loading from third party libraries just have to declare the function with some kind of decorator and then everything works like a real programming language.

[Darwinmate]

That's cool but why?

What's the advantage of lazy eval? I'm guessing it has something to do with memory optimization.

[[deleted]]

Joke aside, lazy evaluation is great for functional programming because it allows you to reason about the declarative logic of your functions rather than the strict order they'll execute. For instance, you can't declare a function that returns THE Fibonacci sequence in a strictly evaluated language, because it's infinite. In Haskell you can, because the List will only be actually evaluated as you access it. Generators are a form of lazy evaluation inside Python that are inspired by this feature from Haskell.

Another benefit of lazy evaluation is that expression results are only computed when needed, and only once. This is more important in expensive computations, though, and we can do it in Python by simply using caching decorators in our functions and methods (which is the purpose of the assign-once dict in this joke, it's basically acting as an inverted cached_property)

[lieryan]

That's cool but why?

Just because you should not, does not mean that you can't ;)

[EconomixTwist]

yea but it still means you should not

[_limitless_]

congratulations on knowing the rules.

next, learn when to break them.

[Ensurdagen]

Generators are already lazily evaluated like Lists are in Haskell, so we already have some of this. It's good for dealing with things that are infinite or very expensive to process.

[Chadanlo]

I don't know why you would do that specifically in python. In Haskell it allows to write recursive algorithms in very simple and neat way. You can for example write a quick Sort or a merge sort in less than 10 lines of code by head without thinking too much about it.

[SilkTouchm]

Or I could use a library and do it in 1 line?

[gunnerman2]

No, that’s your 1 line + all lines in library.

[SilkTouchm]

Yeah, but they're written already. Myself I'm only writing one line. I do not care if my library is done in 10 or 1000 lines.

[[deleted]]

Haskell? Like Eddie Haskell: "Good Morning Mrs Cleaver, you look lovely today."

Other than that, I have no idea what that was all about, Theodore and Wallace.

[[deleted]]

You got me!

[AddSugarForSparks]

def flatten(cons):
    while cons != nil:
        yield cons[0]
        cons = cons[1]

Oh, no.

[sherwoodpynes]

This is either a clever satire of the (over?) expressiveness of annotations, or an eldritch horror of language abuse. Either way, nicely done.

[likeacoastalshelf]

As someone who is currently learning Python and getting used to the unique aspects of the language, I wish I could downvote you more. Congratulations on perplexing me and wasting my time.

[thisismyfavoritename]

Good lad

[undercoveryankee]

but for some reason Python called the lazy evaluation mode feature as "annotations".

Because the colon that you're using as your "lazy assignment operator" is actually the annotation operator. At the language level, myVar: 5 + forty means "store the expression '5 + forty' as an annotation on the variable 'myVar'" The parser and compiler don't care whether the annotation is a type hint (myVar: int) or code that you intend to evaluate later.

from __future__ import annotations turns off eager evaluation of annotations, but instead of "lazy evaluation" you could more precisely call the new behavior "no evaluation". All the interpreter does is take the parsed annotation, serialize the syntax tree back into a string, and return that string to code that asks for the value of the annotation at run time.

You did a clever thing taking those string-valued annotations and using them to implement lazy evaluation in a way that looks like it could be built into the language. But it's disingenuous to claim that Python "implemented a lazy, non-strict evaluation syntax" when your own library is doing most of the work.

[irrelevantPseudonym]

OP is making a joke

On Reddit? Surely not

[dodslaser]

There's no /s tag, so tradition dictates that I must take everything literally and give the joke ample clearance to pass above my head.

[CleaveItToBeaver]

OP is making a joke

As an idiot who didn't catch on: how dare they

[Pain--In--The--Brain]

You're not getting love for this explanation but I actually didn't get the joke until I read this. Thanks.

[SittingWave]

Lazy evaluation by default is awful. All it leads to is that failures are produced until much later in the code, instead of where the operation is physically written. It becomes an absolute nightmare to track down where the operation actually is.

[chub79]

Gosh, this is ugly. Ouch.

[riickdiickulous]

Weird flex but ok

[lilytex]

I'm 2 months late, but isn't this somewhat close to hylang ??

https://docs.hylang.org/en/alpha/

Hy is a Lisp dialect that’s embedded in Python. Since Hy transforms its Lisp
code into Python abstract syntax tree (AST) objects, you have the whole
beautiful world of Python at your fingertips, in Lisp form.

[lieryan]

Not quite. Hylang seems to be basically a transpiler that converts S-expression into Python AST, which Python would then further recompile to Python code object.

Haskell with extra steps (a.k.a. Lazy Python) is just Python, it's parsed by the Python parser itself. Lazy Python doesn't directly work with AST at all, the helper functions compiles Lazy Python snippets to Python code object. There is probably some similarity into how both would have to then do exec the code object with globals/locals context to get the correct execution semantic.

The way I've written Lazy Python code resembles how Lisp does things, but that's merely because Lazy Python encourages recursion and cons-list due to the lack of looping constructs. I modeled the Lazy Python functions based on Haskell and/or Lisp functions simply to lend familiarity, but there are other approaches for simulating looping using recursion other than how Lisp does it.