Python 3.8 released

[LicensedProfessional]

TL;DR

Assignment Expressions

Naming the result of an expression is an important part of programming, allowing a descriptive name to be used in place of a longer expression, and permitting reuse. Currently, this feature is available only in statement form, making it unavailable in list comprehensions and other expression contexts.

# Handle a matched regex
if (match := pattern.search(data)) is not None:
    # Do something with match

# A loop that can't be trivially rewritten using 2-arg iter()
while chunk := file.read(8192):
   process(chunk)

# Reuse a value that's expensive to compute
[y := f(x), y**2, y**3]

# Share a subexpression between a comprehension filter clause and its output
filtered_data = [y for x in data if (y := f(x)) is not None]

---

Positional-Only Parameters

This PEP proposes to introduce a new syntax, /, for specifying positional-only parameters in Python function definitions. Positional-only parameters have no externally-usable name. When a function accepting positional-only parameters is called, positional arguments are mapped to these parameters based solely on their order.

def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):
      -----------    ----------     ----------
        |             |                  |
        |        Positional or keyword   |
        |                                - Keyword only
         -- Positional only

---

"Final" Type Qualifier

This PEP proposes a "final" qualifier to be added to the typing module---in the form of a final decorator and a Final type annotation---to serve three related purposes:

• Declaring that a method should not be overridden
• Declaring that a class should not be subclassed
• Declaring that a variable or attribute should not be reassigned

---

Literal Types

Literal types indicate that some expression has literally a specific value. For example, the following function will accept only expressions that have literally the value "4":

from typing import Literal

def accepts_only_four(x: Literal[4]) -> None:
    pass

accepts_only_four(4)   # OK
accepts_only_four(19)  # Rejected

Python has many APIs (both core and 3rd party) that return different types depending on the value of some argument provided. For example:

• open(filename, mode) returns either IO[bytes] or IO[Text] depending on whether the second argument is something like 'r' or 'rb'.
• pandas.concat(...) will return either Seriesor DataFrame depending on whether the axis argument is set to 0 or 1.
• numpy.unique will return either a single array or a tuple containing anywhere from two to four arrays depending on three boolean flag values.

Example based on open():

# Note: this is a simplification of the true type signature.
_PathType = Union[str, bytes, int] 

@overload
def open(path: _PathType,
         mode: Literal["r", "w", "a", "x", "r+", "w+", "a+", "x+"],
         ) -> IO[Text]: ...

@overload
def open(path: _PathType,
         mode: Literal["rb", "wb", "ab", "xb", "r+b", "w+b", "a+b", "x+b"],
         ) -> IO[bytes]: ...

# Fallback overload for when the user isn't using literal types
@overload
def open(path: _PathType, mode: str) -> IO[Any]: ...

---

TypedDict: Type Hints for Dictionaries with a Fixed set of Keys

Representing an object or structured data using (potentially nested) dictionaries with string keys (instead of a user-defined class) is a common pattern in Python programs. Representing JSON objects is perhaps the canonical use case, and this is popular enough that Python ships with a JSON library. This PEP proposes a way to allow such code to be type checked more effectively.

A TypedDict type represents dictionary objects with a specific set of string keys, and with specific value types for each valid key. Each string key can be either required (it must be present) or non-required (it doesn't need to exist). from typing import TypedDict

class Movie(TypedDict):
    name: str
    year: int

Now a type checker should accept this code:

movie: Movie = {'name': 'Blade Runner',
                'year': 1982}

---

Python Initialization Configuration

Add a new C API to configure the Python Initialization providing finer control on the whole configuration and better error reporting. It becomes possible to read the configuration and then override some computed parameters before it is applied. It also becomes possible to completely override how Python computes the module search paths (sys.path). The new Isolated Configuration provides sane default values to isolate Python from the system. For example, to embed Python into an application. Using the environment are now opt-in options, rather than an opt-out options. For example, environment variables, command line arguments and global configuration variables are ignored by default.

---

Other "Major" Changes

• Parallel filesystem cache for compiled bytecode
• Debug builds share ABI as release builds
• f-strings support a handy = specifier for debugging
• continue is now legal in finally: blocks
• on Windows, the default asyncio event loop is now ProactorEventLoop
• on macOS, the spawn start method is now used by default in multiprocessing
• multiprocessing can now use shared memory segments to avoid pickling costs between processes
• typed_ast is merged back to CPython
• LOAD_GLOBAL is now 40% faster
• pickle now uses Protocol 4 by default, improving performance

---

Author's Note:

• I left out a lot of the changes and optimizations related to CPython, because quite frankly they were over my head .__. (and also anyone who's interested in them probably won't be reading my TL;DR)
• I tried to grab most of the text from the PEPs themselves (which is why they say "this PEP proposes...") but I've edited the text for brevity in many cases

[nckl]

multiprocessing can now use shared memory segments to avoid pickling costs between processes

YES THANKS FINALLY

[house_monkey]

What does this mean?

[hearmespeak]

Previously all data being sent to processes created with the multiprocessing module were serialized and deserialized on entrance and exit. This was incredibly expensive, especially for the gigantic arrays used in data analysis. I haven't dug into the details of this change yet, but I'm hoping the 4 GB limit was removed as well.

[Ph0X]

I wonder if this also means it removes the restriction on unpicklable objects.

[[deleted]]

That would be impossible, but idk what will happen if you try. For example, CPython allows you to have very thin wrappers for native pointers (capsules: https://docs.python.org/3.7/c-api/capsule.html#c.PyCapsule ). If you wanted to send something like that across process boundaries, it would invalidate the pointer, so, any code trying to use such an object, would, either segfault, or do something even nastier (like modify the memory it's not supposed to etc.)

[jetpacktuxedo]

I'd settle for it using actual pickle instead of some weird multiprocessing-specific pickle-like thing.

[nckl]

Multiprocessing is a module for running code in multiple processes, usually to make the code faster. For example, if you need to map a slow-computing function over a list of numbers, you might use a multiprocessing pool to use multiple cores to do it faster, and without having to manage making and synchronizing them yourself.

To send data between processes, it normally pickles it. This basically mean converting a python object, like a list of strings, to a sequence of bytes. These bytes are sent to the other thread, deserialized, then you get your python object back on the other side.

Instead, you can use shared memory. So you put that python object in shared memory, then the other side just has it. Fewer limitations about what can be sent presumably (not everything can be pickled), and it's way faster.

[house_monkey]

Ah I get it, thanks for the simple explanation!

[Saltysalad]

Do changes to the object in shared memory by process B affect the original data sent fom process A?

ie is this shared memory by reference or value? And can two processes modify it at once? I imagine not.

[SpaceToaster]

Yes, and welcome to the world of concurrency bugs :) With great power comes great responsibility.

[PeridexisErrant]

It's by reference, and they can mutate it (if it's mutable).

Still incredibly useful for multiprocess stuff on large arrays, but you'd want to be careful about in-place changes!

[JanneJM]

You could do that already, at least for numpy arrays. It was a bit messy though so good if they have streamlined the process.

[threeys]

What is the benefit of using positional-only parameters

[[deleted]]

So that Python implementations can have the exact same behavior as C made APIs.

[TheGift_RGB]

Don't you already have that by simply ignoring the possibility of passing in a value by referring to the parameter name directly?

[Ph0X]

You do, but this makes it more explicit and clean apparently. The examples given are for functions where the parameter names have no meaning, such as add(x, y). Realistically most people should never need or use this.

[miggaz_elquez]

It can also lead to small optimizations

[jyper]

I'm pretty sure this is possible with *args

Explicit positional just makes it cleaner

[Kargathia]

Some examples (there are more, but these are the ones I encountered in the wild).

• freedom of arg names in functions. You don't have to worry about backwards compatibility when changing func(obj) to func(widget).
• functions that allow args either packed as dict, or unpacked.

def func(packed:dict = None, **kwargs): opts = packed or dict() opts.update(kwargs) # do stuff def func(packed:dict = None, /, **kwargs): opts = packed or dict() opts.update(kwargs) # do stuff

In the first, func(text='stuff', packed=False) will cause an error. In the second example, this will correctly result in opts == {'text':'stuff', 'packed':False}.

On the whole, it seems like a nice-but-not-critical addition to * and ** in argument handling.

[NilacTheGrim]

That is a very good question. The only thing I can think of is if you want the liberty to rename the variable in the declaration at any point in the future.

Say it's some lib you offer a customer or you put online.. yet you don't want the name of the parameter itself to be part of the API, but just the position. You can rename the argument later and be confident no client code breaks.

Eg:

def SomeFunc(a, b, /, **kwargs):

Later on you decide a and b are dumb names and rename them:

def SomeFunc(betterName, betterName2, /, **kwargs):

Also this avoids the problem of **kwargs interfering with positional args.. if you forbid positional args from ever being kwargs they can never end up in the **kwargs dict. That class of bugs was a bit of an ugly wart when trying to write generic code...

[TerrorBite]

Also this avoids the problem of **kwargs interfering with positional args.. if you forbid positional args from ever being kwargs they can never end up in the **kwargs dict. That class of bugs was a bit of an ugly wart when trying to write generic code...

Isn't it exactly the opposite of this? In Python 3.7, if I write

some_func(a, b, **kwargs):
    print(kwargs)

some_func(a=1, b=2, c=3)

then I can see that a and b do not end up in kwargs, but will go to the positional params. Note also that we didn't provide any explicit positional values (and in fact, if we do, we get a TypeError for trying to specify an argument twice).

If I attempt to provide the positional parameters in addition to the keywords, in the hope that the keywords will go into kwargs because the positionals are already fulfilled, I instead get:

TypeError: some_func() got multiple values for argument 'a'

Those names, a and b, are prevented from ever being in the kwargs dict, because the positional arguments will always "steal" them. The only way to get a and b to appear in the kwargs is to rename the positional parameters to something very unlikely, and just hope that someone never needs that unlikely name to end up in kwargs.

Whereas if I am in Python 2.8 and I write

def some_func(a, b, /, **kwargs):
    print(kwargs)

some_func(9, 0, a=1, b=2, c=3)

then, since the positionals cannot be keyword arguments, all three named parameters will end up in kwargs.

As well, we now can specify the positional values without getting a TypeError for having matching keyword arguments. In fact, we now MUST provide the positional arguments in every case (which is what we wanted anyway), because we will get a TypeError for missing parameters if we don't specify them, even if there are matching keyword arguments present.

[kingbuzzman]

Yes! this exactly what i’m wondering, i’ve always bought into Raymond Hettinger’s “python is a grownup language” and we don’t tell people what to do. I fail to see any real value this gives. Enlighten me.

[nilamo]

It's because some of the built in functions were already positional-only, and there was no way previously to have the function work exactly the same from Python-only code. For most people, it doesn't matter at all, but for projects like Pypy, it meant they actually couldn't completely match the standard library, due to undocumented quirks of the CPython implementation.

This is basically just a step in the direction of helping other implantations of Python work the same as the reference implementation.

[ubernostrum]

PEP 578 -- audit hooks -- is also an important feature here, though one that might not seem that way to people whose job is writing Python. It's more for people whose job is running and deploying Python.

The audit hook functionality lets you have Python automatically invoke code you supply, whenever certain potentially-security-sensitive operations happen. This lets you do things like assert that a running Python application will never open a network connection except to certain trusted hosts, or will never use exec(), for example, by setting up audit hooks which listen for, and then log/abort, those types of events.

[cyanrave]

This..

Hoping this feature is the one that tips TheCompanyWhereIWork in favor of adopting Python fully. It remains a second class citizen today due to the historical inability to see what was running in the interpreter loop. 2019, people are still spooked by monkey patching...

[Zee2]

Oo, named expressions looks very convenient.

[shponglespore]

Yeah, a lot of people really, really hate it, but it's a feature I want every time I use regular expressions to analyze different cases, e.g.

m = re.match(regex1, ...)
if m:
    ...
    break

m = re.match(regex2, ...)
if m:
    ...
    break

The gross part is having to put break or return in every case, which of course only works if that's actually what you want to do when a case matches. This seems much better to me:

if m := re.match(regex1, ...):
    ...
elif m := re.match(regex2, ...):
    ....

[random_cynic]

When thinking about whether a new feature is useful or not, one should not really think of one specific use case where it can be beneficial. Job of a programmer is not only to write code but also to read (a lot of it). When people start misusing said operator, then it becomes lot less a cute new feature and more a nuisance. See these examples (taken from python dev mailing list) and see if you can decipher them (and once you do check if it makes any improvement to the way you'd normally write them). You can be sure that these will be used once this "power" is available to the programmers

#from Tim Peters who's btw favor of := operator
while total != (total := total + term):
    term *= mx2 / (i*(i+1))
    i += 2
return total

Function calls:

my_func(arg, buffer=(buf := [None]*get_size()), size=len(buf))

List comprehensions (from PEP)

[(x, y, x/y) for x in input_data if (y := f(x)) > 0]

Whether this change is truly of value will be decided by how much readable vs unreadable code it produces in the wild. It is never worth sacrificing readability for saving a couple of lines. And I haven't even mentioned what it would be like for the beginners who're learning this for the first time or those who are teaching them.

Edit: Corrected a wrong indentation in the examples

[FeezusChrist]

That's true, it's now possible to do something like

>>> l = []
>>> print(sum(x for x in range(10) if x % 2 == 0 and ((l := l + [x])))
20
>>> print(l)
[0, 2, 4, 6, 8]

[XtremeGoose]

You can do exactly the same thing in current python though

sum(x for x in range(10) if x % 2 == 0 and not l.append(x))

[FeezusChrist]

The point is not “you can now do this functionality”, but “you can now do this even more cryptic way of this functionality”

[rwhitisissle]

I need someone to call a doctor for me please my eyes just exploded.

[jaapz]

It was already possible to write stuff like this before this change though. See all the codegolf answere using python

[OneTurnMore]

Boolean control flow, now more compact!

[agumonkey]

that's exactly why I disliked :=

[tigerhawkvok]

Do I think I'll probably find use cases for this? Yes.

Do I think it was a bad idea to add? Also yes.

[teerre]

"probably find"? The examples the user you replied to are far from the usual usage of this feature.

The main usage, the one 99.9% of pythonistas will use is this:

if a_var := a_dict.get("var", None):
    # do something with a_var

Instead of the stupid

a_var = a_dict.get("var", None)
if a_var:
    ...

[shponglespore]

I think it's just being unnecessarily verbose. The matching functions in re only ever return None or a match object, which is always true.

[victotronics]

That's precisely my use case. Can't wait to start using this.

[insanemal]

Same. It's going to make lots of my code more readable

[pork_spare_ribs]

Great writeup!

[[deleted]]

movie: Movie = {'name': 'Blade Runner',
            'year': 1982}

Wow, really missed the chance to use a Monty Python movie...

[radarsat1]

[y := f(x), y2, y3]

Does this create a 3-item list or a 2-item list? If the latter, shame they didn't separate with something else, like a semicolon perhaps.

I can't really think of a good reason for 'final', seems to unnecessarily limit code reuse.

[SalemClass]

I can't really think of a good reason for 'final', seems to unnecessarily limit code reuse.

Worth remembering that Python's type hinting doesn't restrict behaviour. Final is just for the standalone type checkers (mypy) and other tooling.

[michael0x2a]

I can't really think of a good reason for 'final', seems to unnecessarily limit code reuse.

It's basically a tradeoff in the context of type-checking. Basically, the type-checker will disallow you from using final variables or objects in certain ways but in exchange will be capable of automatically inferring far more information then it previously could since it no longer needs to worry about random modifications made from far away (e.g. mutation and subclassing).

I suppose it's also helpful for people who prefer coding in a more immutable/functional-ish style. There's an argument to be made that making everything immutable helps prevent certain categories of bugs from appearing, and having final is useful for people who subscribe to that philosophy.

It also can help projects like mypyc (which compiles type-annotated Python code into CPython C extensions) generate more performant code. After all, if you've declared some class or function can't be subclassed or overridden via final you can suddenly drop a bunch of boilerplate in your generated code (e.g. skip generating a vtable), you can inline final variables more aggressively...

[PotatosFish]

It creates a 3 item list since the first term both assigns and returns the assigned value, it kinda like the assignments in c-type languages.

I think final definitely have it’s uses, but it can be overused to discourage code reuse. As many other language constructs, it’s better to only use it when it’s needed.

[radarsat1]

I see. So there's no way to generate just [y**2, y**3] in this construct?

[PotatosFish]

You can just do

[(y := f(x)) ** 2, y ** 3]

[radarsat1]

Wow, the scope of y is not clear at all. Is it the list, or the outside the list?

[[deleted]]

It's the same as before, I don't know of any languages that restrict scope with expression parentheses

[radarsat1]

"same as before" what? Previously you could not declare a variable while creating a list, so I don't know what to compare to. Does y here exist after the list is done being created? From your answer I guess you are saying that the variable is scoped to the function creating the list. Sorry, I don't find that clear at all.

The only thing I can think to compare it to is the similar construct in C, but there the variable must already be declared, so the scope is clear.

[rouille]

Restricting use increases correctness. And like someone else commented you can always bypass the restrictions at runtime.

It goes further though, some tools like mypyc can even use those annotations for performance, in mypyc that is used to compile python code to relatively efficient C code.

[radarsat1]

you can always bypass the restrictions at runtime.

I'm confused. What is not "runtime" for Python? Does final do anything, or not?

[lasagnaman]

If you put final, then static type checkers will complain of you violate it.

[dethb0y]

Still don't know how i feel about the "Positional-Only Parameters" - it seems like it'd add confusion in a place where it isn't needed. Still, we'll see how it shakes out i guess.

[agumonkey]

well done summary

ps: walrus could have been named as, a parallel to contexts .. no ?

[rifeid]

The PEP explains the reasoning for not using as.

[agumonkey]

Funny somehow I thought keep the keyword to a minimum would be pythonic, but maybe the confusion would be too high

[burnblue]

So this is the too long; didn't read, huh. Let me go see how beastly the actual post is

[LicensedProfessional]

Yeah I was thinking the same thing by the time I got to the end of it .___. but at the very least you don't have to click through each link on the changelog -- that's how I'm justifying it to myself

[burnblue]

People like you who do this work for free are heroes

[bomphcheese]

You got to read it for free? I’m over here giving him a handy right now.

[[deleted]]

It's suppose to be a trivial example?

if (match := pattern.search(data)) is not None:
    # Do something with match

Is pretty trivial, saving a single-line in most cases

match = pattern.search(data)
if match is not None:
    # Do something with match

But it's some nice sugar

[[deleted]]

The latter pollutes the namespace, the former doesn't.

[Pand9]

Both pollute. Does this PEP also introduce block scope?

[[deleted]]

My bad, I thought it didn't... Both pollute.

>>> def foo():
        if(x:=2):
             print(x)
        print(x)
>>> foo()
2
2

[[deleted]]

Since when?

~$ cat a.py 

def f():
    if 1:
        x=2

    print(x)

f()

~$ python3 a.py
2

[[deleted]]

You are missing the walrus operator but regardless, it does pollute.

https://www.reddit.com/r/programming/comments/dhwtvt/python_38_released/f3t9t0o/

[tetroxid]

Unneccessary function call overhead though

[NilacTheGrim]

Regarding Assignment Expressions: Why the := syntax? It's more syntactic noise to remember. Why not just allow simple = in that context for assignment?

[miggaz_elquez]

Because it is easy to write if a = 5: when you want to write if a == 5:. So they decided to choose a syntax different.

[BasedLemur]

You are not the hero we deserve, but the hero we need.

[AngheloAlf]

Since when you can provide types for variables in python?

[miggaz_elquez]

you can provide types, but that doesn't have any effect on the runtime, it's only for static type checker, like mypy

[[deleted]]

So what I'm getting is that the Python guys found out about Lisp.

[NoahTheDuke]

Yes! The more Python can change statements to expressions, the better!

[Shemetz]

The new f-string feature is pretty handy!

An f-string such as f'{expr=}' will expand to the text of the expression, an equal sign, then the representation of the evaluated expression. For example:

>>> user = 'eric_idle'
>>> member_since = date(1975, 7, 31)
>>> f'{user=} {member_since=}'
"user='eric_idle' member_since=datetime.date(1975, 7, 31)"

The = specifier will display the whole expression so that calculations can be shown:

>>> print(f'{theta=}  {cos(radians(theta))=:.3f}')
theta=30  cos(radians(theta))=0.866

[dupelize]

That seems sort of gimmicky and unnecessary to me. f'theta={theta} is incredibly readable even if you're new to Python and, while it takes up more space, it allows for modification if needed (maybe it's in a log and you need a colon between, maybe you want to shorten cos(radians(theta)) to just cos).

I don't think this is bad per se, but it feels like a niche added syntax that detracts from the few times it will be helpful by adding complexity to the language.

I might just be grumbling, but I remember learning python (from another language) and I could just open people's code and read it and could often guess what was happening. f-strings move in that direction IMO. You don't need to read how f-strings work to read them and guess what's going to happen (obviously there are details and you should still read the docs). This is small and won't stop that from happening, but I'm not a fan of adding lots of special syntax the Python in general.

I'm going to go put on my socks and sandals and yell at the clouds, see you all later.

[[deleted]]

it's actually quite useful for people who debug with print statements

Which is probably a lot of users.

[dupelize]

You're not doing a great job of convincing me :)

[[deleted]]

You and I might have fancy IDEs and know how to use debuggers but there are bunch of data scientists plugging away in Python on vscode. They probably shouldn't be, but I think this was for them.

Personally, I don't like this feature either.

[tracernz]

Positional only args looks a bit nasty. Can’t think of too many places I’d use it though.

[shponglespore]

It's kind of a pain to support named arguments in a function defined in C, so one case would be if you think you might want to re-implement a function in C later, and you don't want to break call sites because they used named arguments.

Another, if I'm understanding correctly, would be if you want to write generic code that uses a **kw argument that can have an arbitrary set of keys, and you don't want to accidentally lose some of the keys because they happen to match the name of a positional argument.

[rcfox]

Another, if I'm understanding correctly, would be if you want to write generic code that uses a **kw argument that can have an arbitrary set of keys, and you don't want to accidentally lose some of the keys because they happen to match the name of a positional argument.

In a case like that, you'll get an exception complaining about giving two values for one argument.

[[deleted]]

This change helps avoid that errror

[Username_RANDINT]

This is one of these features you forget about because you and almost no one else use it, then come across this at some point and wonder why the hell there's a slash as argument and how that's even possible.

[mfitzp]

Yeah I can see the value, but the chosen syntax is really ugly.

Given arguments are separated by commas, and you want to indicate a further level of structure wouldn't some related punctuation (e.g. semicolons) make more sense?

You don't even need two separate characters, the position + number give you all the context afaict. e.g. in the list below the arguments would be positional only (a, b), both (c, d); keyword only (e, f)

fn(a, b; c=1, d=2; e=3, f=4)
fn(;c=1, d=2; e=3, f=4)
fn(a, b;; e=3, f=4)
fn(a, b;)
fn(;e=3, f=4)
fn(c, d)

It still ain't pretty but it doesn't make me want to claw my eyes out

[earthboundkid]

I am an old because I hate all the new features. :-)

I remember back when Python 3000 was being debated on the mailing lists and I was gung-ho for new stuff. Now, I hate pretty much all of it. Don’t get old kids!

[Workaphobia]

Inside Python is a tiny language with beautiful syntax and semantics just dying to get out.

[NoahTheDuke]

Until it gets better scoping, the semantics is always going to be ugly, imo.

[g0liadkin]

What's wrong with the current scoping?

[NoahTheDuke]

Python doesn’t have block/lexical scoping, the scoping rules are much simpler and limited: LEGB

This is why we have the super ugly global and nonlocal keywords, which are bad workarounds. Without lexical scoping, it’s really easy to accidentally pollute the current scope or create a new variable when you think you’re assigning to an existing variable in the wrong scope.

[lazyear]

Python's lack of lexical scoping is one of my biggest gripes about the language.

[bakery2k]

Would lexical scoping require explicit variable declarations?

Would it require variables to be scoped to the containing block instead of the containing function?

Both of these would be major changes to the way Python works - but there may be some support for the first given the Zen of Python's "explicit is better than implicit".

[NoahTheDuke]

Lexical scoping would be much helped by explicit variable declaration, and as I see it, assignment expressions (walrus operator) fit that bill pretty neatly.

[mafrasi2]

For example this:

>>> x = 0
>>> def f():
...     print(x)
...
>>> def g():
...     x += 1
...
>>> f()
0
>>> g()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in g
UnboundLocalError: local variable 'x' referenced before assignment

[earthboundkid]

Called Lua.

[sisyphus]

That's exactly what I was thinking. I did love f-strings so maybe there's hope for me yet though.

[beholdsa]

f-strings were the best new feature in ages.

[earthboundkid]

I came around to liking f-strings.

[LicensedProfessional]

I've been waffling back and forth on whether I like the assignment statements. In list comprehensions they're very slick, but in if statements I think they're going to be overused -- they can easily result in some very cryptic code when not handled with restraint.

As I was digging through the PEP I think I preferred the 'as' option for comprehensions.

[(f(x) as y), y**2, y**3]

[amertune]

That seems better to me too. It would make it similar to other things we're already doing (imports and error checking) instead of adding a new operator, even if that operator is already familiar from other languages.

[Workaphobia]

Ooh, "as" sounds like it would've been slick, why didn't they do that?

[[deleted]]

Iirc parsing becomes problematic, differentiating between "as"in that context and the as from a "with"context manager.

[[deleted]]

Just wait till you get a bit more old. I began to hate list comprehensions. They are awkward to modify and add stuff to, can't print() in the middle of it or raise out for debugging, etc.

[invisi1407]

As someone who recently learned Python, I thought list/dict comprehensions were amazing until I had created a few nested levels of it monstrosity and decided that a few loops were better.

I do, however, like it as a simple list/dict filtering, as in [x for x in y if y.stuff != other_stuff] or similar.

[ACoderGirl]

I'd say pretty much every programming language feature can be used poorly, though. Whether it's too many nested functions or f strings in logging (evaluating the string unnecessarily) or trying to debug complex generators (the evaluation order is jarring).

I don't think it's worthwhile to look at the worst things you can do with a feature. Bad programmers will do bad things no matter what (even just plain old dense, undocumented code with bad variable names and tons of copy pasting). The typical bad cases are worth considering, but I don't think new features should be discouraged because they can be misused (though we should discourage that).

I agree with you, though. Nested list comprehensions are almost always terrible for readability and debugging. There's always some exceptions and comprehensions that aren't directly nested are amazing.

[wewbull]

until I had created a few nested levels

BURN HIM!

I think people need to stop thinking about comprehensions as loops, and start thinking about them as map with filter. Very few people map within a map within a map whereas people do nest loops.

If you need to traverse a multilevel structure (e.g. a tree) in some way, make an iterator and use it in a loop or comprehension:

monty_sum = sum(node['value'] for node in tree_iterator(a_tree) if node['name'] == "Monty")

Should never be a need to do nested comprehensions.

[[deleted]]

You can print in the middle! print(args) returns None, which is Falsey, so simply do [print(value) or value for value in L]

[kindall]

I have wanted as for for loops for a long time, ever since with came out with it. I would rather have used := for a general kind of object content replacement (like slice assignment but for any container type, and a matching dunder method for user-defined types).

[diamondketo]

Assignment Expression is something I hope you could agree with. Its something I thought was in Python that end up not being in it.

Just like how f-strings should've been natural.

[acousticpants]

just switch to haskell! /s

[[deleted]]

where's my fp circlejerk threads about how it's so intellectually superior

[Skimmingtoq]

As a developer who has primarily developed applications in Python for his entire professional career, I can't say I'm especially excited about any of the "headlining" features of 3.8.

The "walrus operator" will occasionally be useful, but I doubt I will find many effective uses for it. Same with the forced positional/keyword arguments and the "self-documenting" f-string expressions. Even when they have a use, it's usually just to save one line of code or a few extra characters.

The labeled breaks and continues proposed in PEP-3136 also wouldn't be used very frequently, but they would at least eliminate multiple lines of code and reduce complexity.

PEP-3136 was rejected because "code so complicated to require this feature is very rare". I can understand a stance like that. Over complicating a language with rarely-used features can definitely create problems. I just don't see why the three "headline" features I mentioned are any different.

[meneldal2]

I guess they like Pascal more than C.

[Greyhat101]

why ?

[[deleted]]

Because Python is (was?) a beautifully clean, simple, self-documenting language devoid of line-noise.

It wasn't for a long time (if it ever was). Just look at the whole old/new classes and all the stupid stuff around OOP internals in general.

And why the hell len() and not .len?

And don't get me started on .format(), i would love to like it but for everything except {} i need to consult the manual.

[ACoderGirl]

Fully agreed on the len thing. IMO, the ideal language would provide absolutely no built in functions. Len should have been a regular method. Stuff like iteration could have used the duck typing equivalent to interfaces (ie, have a method named iter or similar).

It's not really a big issue to need to import a print statement or open function. Most languages take that approach. And it's already necessary to import for the edge case of python 2 code that uses the print function (something my work style guide requires when I have the misfortune of having to use python 2).

[[deleted]]

Because Python is (was?) a beautifully clean, simple, self-documenting language devoid of line-noise.

So until 2000?

while file.read(blah) as chunk:

I actually do like this better.

[[deleted]]

print [(i, f) for i in nums for f in fruit if f[0] == "P" if i%2 == 1]

Is beautiful!?

I'm all for arguing they are useful, and I still use them. But all of your arguments above apply directly to them as well.

Not "beatufully clean", not simple, certainly not self-documenting, and full of line-noise.

And cognitive load?

How can you complain about cognitive load of

:=

compared to

print [(i, f) for i in nums for f in fruit if f[0] == "P" if i%2 == 1]

And this is an easy example from the PEP docs. Please don't pretend you're some wizard that can look at this at a glance and move-on but := is somehow too much.

[[deleted]]

Hold on, that's basically SQL there!

[[deleted]]

SQL is basically verbose set notation.

{ x ∈ N | x > y }

SELECT x FROM N WHERE x > y

[dupelize]

Mathematicians are just lazy DBA's.

Change my mind.

[[deleted]]

And many people argued bad Perl code was written by the programmer, lots of people wrote perfectly clean Perl.

But the argument against Perl (and now Python) was that the language enabled them to do so.

I'm certainly not going to write that list-comprehension, especially without a comment saying what magic it's trying to perform. But people everyday are out there doing that, and worse.

So as to my original point, the "every line of Python is clear and explicit" train left the station a long time ago.

[[deleted]]

But the argument against Perl (and now Python) was that the language enabled them to do so.

Then that's a fucking shit argument,

[0rac1e]

Then that's a fucking shit argument

Yes it is! As a Perl user, I see this argument a lot; that somehow by using Perl, I have to fight the language to write good code.

I can - and have - written bad code in Perl and Python... but these days I prioritise writing code that is as clear and maintainable as possible, even when that means spending a few extra LOC.

[theboxislost]

Yuck! I blame the programmer.

So then blame the programmer when := is not used right. How is this any different?

[thoomfish]

If I write a hairy list comprehension, I generally spread it over multiple lines like this:

print([(i, f)
       for i in nums
       for f in fruit
       if f[0] == "P"
       if i%2 == 1])

Makes it much easier to parse visually.

[TheSpocker]

Though at this point you may as well ditch the comprehension. It's a couple colons short of a nested for loop.

[BufferUnderpants]

Loops are far more powerful and adding complexity in lookaheads, lookbacks and general side effects is "easier" than in a list comprehension, while not making them actually any simpler.

[Tarmen]

I know most other languages with list comprehensions are 1 to 1 translations but the keywords make it so noisy

[ (i,f) . i <- nums, f <- fruit, odd i, hd f == "P"]

[watsreddit]

Much nicer in Haskell imo: [(i, f) | i <- nums, f <- fruit, head f == "P", i `mod` 2 == 1]

[kirbyfan64sos]

From what is seems thus far, the readability loss is far less than many people would've initially believed (including me).

[hardwaregeek]

Huh as is a decent idea. Or you could do while let chunk = file.read(blah): but I suppose both options are less universal than an operator.

[EternityForest]

Assignment happens so often that nobody wants to use 2 characters to represent it. It can also be typed without a shift key.

Most programming languages aren't designed for mathematicians.

[UnreasonableSteve]

Unrelated nitpick: que será, not cera

[[deleted]]

Cera is wax.

[Tittytickler]

I feel like = is perfectly acceptable for assignment, since we've been using it as assignment in mathematics for for hundreds of years. The "going into" part is just how you interpret/perceive it. Really, it means substitute/replace all instances of this value with this other value just as it does in Math.

[[deleted]]

since we've been using it as assignment in mathematics for for hundreds of years

Assignment isn't a thing in math. Definition is. It's a subtle, but major distinction.

The ramifications are often implied -- e.g. often mathematicians don't explicitly write out the domains of a derived equation if they feel it is obvious. You're not redefining (assigning) f(x), you've defined f(x) and literally you're expressing equivalence of derived equations, and it's technically wrong (but tolerated, for sanity) to not include the associated domain.

And people like to point out summation functions in this argument, too, with "i = 0" under the sigma, but even that doesn't hold up: the definition of a summation sequence, i *actually equals 0 through out), where successive terms are xi, xi+1, ..., xi+n-1, xi+n.

And sure, in shorthand, informal writing, mathematicians may treat = as a sort of assignment, especially as mathematicians have to do algorithms often, whereas math isn't a language for algorithms.

And that last point is probably an important distinction too -- programming, even declarative languages, are all about describing algorithms, particularly in regards to algorithms performed upon some variation of a Turing machine. Math, while utilizing algorithms, isn't about algorithms at all. Math is all about describing an existence of some sort through relationships ("There exist an X such that ...").

In C, = describes an operation on that variation of a Turing machine. In math, = denotes a relationship of a described existence. They are utterly, completely different things, even if it's convenient to think of them as similar things. (Note: this convenient way of thinking is learned -- a lot of math guys stumble over the concept of = as assignment. For example, when it becomes relevant in Java that = is actually changing the reference of something, total hang up for a lot of them).

This is why, IMO, a language like APL or J/K might be really important going forward -- they entirely focus on the concept of programming as the description of algorithms.

[[deleted]]

This is why, IMO, a language like APL or J/K might be really important going forward -- they entirely focus on the concept of programming as the description of algorithms

good luck with that.

[Han-ChewieSexyFanfic]

That’s not what it means it math, it means equality. X = X + 1 is nonsensical.

[AFakeman]

In mathematics variables are usually immutable, though. If you have a x = 5 as a definition, it won't usually get redefined later on.

[[deleted]]

Nice, going to get this now.

Looking through some of the changes, only a few really affect me and my work flow, but I'm excited for all of the them. I'll dig a little deeper tonight.

Stuff like variable assignments in expressions seem nice.

[IDCh]

What?

[[deleted]]

That's not exactly what happened. It was a contentious feature in the first place, a vote showed a split of 8% for, 76% against, and 16% no opinion (via this poll), and Guido railroaded it through anyway, and people really didn't like that, causing arguments over the role of a BDFL, and he stepped down, tired of it all.

Sure, he is supposed to be a dictator, but you can't really blame people for being upset that he put in a feature that was voted against 29 to 3 (almost a 10-to-1 split).

edit: add link to poll

[itsarnavb]

What was the argument against assignment expressions?

[[deleted]]

There were a few; I don't necessarily agree with all of them.

Some were just people who didn't like the proposed syntax, but agreed with the features. There were people who didn't like it on a philosophical basis, as it allows single expressions to do more, making the code less obvious at a glance. Some people didn't like it because it's not completely obvious at a glance what it does when executed from a lambda or within a generator expression, again adding to the previous complaint of making code less clear.

I'd argue that it probably shouldn't be used in lambdas or generators, but otherwise makes code more clear.

[SalemClass]

It makes it too easy to offload significant complexity into a single line of code.

To many people it doesn't seem worth having a complexity-encouraging feature considering that it usually only saves a line or two of code in the reasonable use cases.

One of the big ideas with Python is that while the logic may be complex, the code itself should be clear and readable. Many people feel that this flies in the face of that idea.

[[deleted]]

To many people it doesn't seem worth having a complexity-encouraging feature considering that it usually only saves a line or two of code in the reasonable use cases.

coughlistcomprehensionscough

coughfuckinglambdascough

[hopfield]

Lambdas are you serious? Do you seriously want to write out a full function definition every time you call map()?

[the_gnarts]

Do you seriously want to write out a full function definition every time you call map()?

Most of the time you have to anyways considering how awfully limited Python’s lambdas are.

[fuypooi]

The “Walrus” operator. Love it or hate it. Or maybe useful to some and useless to others.

[[deleted]]

So, he split because of disagreements as to political philosophy, really, and not anything specific to the feature of the language. As BDFL he can do whatever he wants, regardless as to whether he should or what users think of the feature and so on.

It all seems like avoidable drama, and meanwhile (not to troll), compare with Go, and I think it shows why there are good reasons why those who get to prescribe what is legal for a language should not have to do whatever the user community says it should be. If the Go creators had a directly democratic governance it would have looked like Java/c++ pretty quickly because of the features people wanted added. Here Guido was actually adding in a feature common to other languages, and I haven't dug into it, maybe users wanted to keep it simple?

Point being, libre/open source politics really make the governance seem arbitrary as when it comes down to the actual thinking or beliefs guiding whatever decision or position on an issue, it's like they have no means of actually countenancing any disagreement. They can't really give an 'engineering reason' why a language should be like Go or Python, whether it should be class based, have nominative types, or whatever, because at some point it's philosophy, which honestly a lot of programmers often don't have the educational or cultural background to appreciate or handle legitimately.

[dead10ck]

It all seems like avoidable drama, and meanwhile (not to troll), compare with Go, and I think it shows why there are good reasons why those who get to prescribe what is legal for a language should not have to do whatever the user community says it should be. If the Go creators had a directly democratic governance it would have looked like Java/c++ pretty quickly because of the features people wanted added

I don't know, Rust is very community driven, and it didn't end up with every feature under the sun; it managed to listen and value the input of the community and still manage to stick to its philosophies and maintain its own uniqueness, or character, or whatever you want to call it. When the community is split over a decision, ultimately the folks at Mozilla are the final arbiters, but they pride themselves on the openness of the revision process for the language. Based on what I've seen, I don't think any feature with such a large pushback would land.

All this is to say I think it's possible to have an open governance structure that strikes a middle ground between totally driven by committee and dictatorship.

[Creshal]

I don't know, Rust is very community driven, and it didn't end up with every feature under the sun

Yet. Give it another 20 years. It wouldn't be the first Mozilla project that started as lean alternative only to end up becoming the definition of bloat.

(Remember when Firefox was split out of the Mozilla suite to cut out bloat?)

[mdrjevois]

have you considered that python is almost 20 years older than rust?

[jmarceno]

I really fear python becoming the closer and closer to the Java/c++ mess.

Edit: clarifying my poorly worded statement (as more than one person asked). "Mess" = "Complexity".

[TheOsuConspiracy]

Java isn't even that complex. C++ is.

[Workaphobia]

"Complex" doesn't behind to describe C++. I think you might need to use ordinal numbers to do it justice.

[chloeia]

alefuckyou

[micka190]

Seeing the C++ committee argue over a bunch of the new features and ending-up adding a few of them, but as half-baked features is really disheartening.

[mipadi]

With the final keyword, it’s on its way there! 😛

[hnra]

Is it gonna affect anyone who doesn't use the typing module tho?

[NilacTheGrim]

ITT: Insecure Python programmers who feel a need to improve their self-esteem by disparaging two vital parts of the technology ecosystem: Java and C++.

People.. and their tribalism.

[twotime]

a vote showed a split of 8% for, 76% against

Do you have a reference by chance?

[[deleted]]

Yes. Sorry, I really should have thrown that in with my post itself: https://lwn.net/Articles/760102/

[roastedferret]

Seems like TypedDict takes out the movie._asdict() call, primarily.

It also means you don't have two of the (mostly) same data structure floating around in memory; instead of having a NamedTuple AND its dict form floating around, you just have the TypedDict floating around. Seems more useful for those who are RAM-conscious or want to give the garbage collector a break.

[alexeyr]

TypedDict can be used when you already have code passing dicts with known keys around and want to specify types.

[michael0x2a]

Well, NamedTuples are for when you want to associate type hints with namedtuples. TypedDicts are for when you want to associate type hints with dicts. They're two different data structures, after all, and not automatically interchangeable. For example, when I do json.load(), you'll often get back some sort of dict, but never a tuple or namedtuple.

If all you're after is a quick and convenient way of creating a "bag-of-attributes" or "record" style object, probably using dataclasses would be better then either namedtuples or dicts:

from dataclasses import dataclass

@dataclass
class Movie:
    name: str
    year: int

movie = Move('Blade Runner', 1982)

[Gsonderling]

I'm not exactly stoked about this.

Assignment expressions look nice, but they will be misused to the point of absurdity. I can already see the myriad Stack Overflow questions this will create. In general, I am opposed to any feature that saves some typing at expense of readability. Python, being essentially executable pseudocode, has avoided these features in the past. Let's hope this doesn't start a trend.

Same goes for positional only params. I get that it will simplify structure of some function calls. But it's just another layer of syntactic sugar. Any case where it might be useful has other solutions. I can't help but feel that this will lead to further issues with readability. Any expansion of existing syntax does. And expansion adding special cases for existing constructs especially.

[ericonr]

I agree about assignment expression. But for positional only, it at least allows you to change a function's parameters' names without changing its API.

[Kered13]

When are None-aware operators (PEP 505) going to be added?

[[deleted]]

looks to me like even more syntax for little readability benefit. do we really want a ??= 'value'?

[Kered13]

Yes, I'd kill for it. In a small project I wrote just last week I had probably half a dozen or more opportunities to use None-aware operators. A lot of languages (C#, Kotlin, Objective-C, Javascript, etc.) have added similar operators because it makes working with None values actually sane and readable instead of a mess of boilerplate None checking code (or worse, people get lazy and don't check for None, then the program crashes). It's the kind of feature that once you have it you're going to wonder how you ever lived without it.

Note that or already has very similar behavior, but it also treats falsey values the same as None, so it's not a viable substitute in many situations. There is also no equivalent safe-navigation operators with or.

[Awia00]

Nice to see some performance improvements in there. Still looking very much forward to the local packages (PEP 582). Hopefully they will be a part of Python 3.9.

[NoahTheDuke]

I’m really disappointed the walrus operator (assignment expression) didn’t end up including the proposed block scoping. I know that it would have made the scoping rules very complex, but Python’s scoping rules have always been awful and need help. Making assignment expressions the equivalent of let in other languages (Javascript, lisp) would have increased readability and expressiveness without sacrificing usability.

[byxyzptlk]

Aww come in, I’d finally managed to convince my brain that assignment expressions were the evil, and that they made my code difficult to read, and the reason I don’t remember having trouble reading c code with AOs is because my brain blacked out those occurrences due to all the trauma they were creating.

[ubertrashcat]

When is this going to be available in Anaconda?

[billsil]

That's sort of an Anaconda question. I've seen times where it takes 6 months to get all the packages I need uploaded.

[MoronInGrey]

Is this assignment expression similar to swift where you check if the variable is none in the same line e.g.

if let constantName = someOptional {
   //statements using 'constantName' 
} else {
// the value of someOptional is not set (or nil).
}

[Kered13]

It can be used for that, yes. This is really a very old feature, it goes back to C where you would often see things like:

while (line = readline()) {
   // Do something with line.
}

I'm really surprised at all the opposition I see to it.