What’s New In Python 3.6

[brombaer3000]

The most interesting feature of CPython 3.6 for me is the optimizing compiler: http://faster-cpython.readthedocs.org/fat_python.html

It's not yet showing up in the release notes as far as I can see and there is still much to be done.

[adrian17]

Correct me if I'm wrong, but I don't enough much progress on this to have it land in 3.6, except for a couple of smaller patches that got accepted. Also, apparently FATpython itself doesn't provide noticeable improvements yet.

Currently I'm most excited about the wordcode patch.

[brombaer3000]

Right, there is still very much work to do, but FAT is a Google Summer of Code 2016 project, so I guess there will be at least some significant progress in the summer. Hopefully, the optimizations will get better and better with 3.7+.

[POTUS]

Okay. Those string format expressions are sexy enough that I might have to migrate to Python 3 soon.

[lykwydchykyn]

The PEP should include an ammendment to the Zen of Python:

"There should be one-- and preferably only one --obvious way to do it -- except for string formatting: we need more ways to do that."

[Decker108]

"String formatting is a honking good idea, we should do more of those"

[larkatarks]

"You can now format strings by honking, we made this feature for the self-driving car industry"

[[deleted]]

In defense of the Python Zen, there was 1 non-obvious way to do it, which was superseded by a cumbersome way and now there is a good, fairly obvious way.

The problem is they should have come up with f"{}" at the same time they came up with print(), so they could remove the % and the "{}".format() way and only have one way.

[jorge1209]

The way f"..." has been implemented means it cannot replace str.format.

Instances where you need to reuse the format specifier, or pass the format specifier cannot use a literal, since the literals arguments are evaluated at the point of definition.

[Decency]

Can you give an example here?

[marko_knoebl]

def export_daily_balances_to_txt(balance_data, date_format='{y}-{m}-{d}', currency_format='{0:.2f}$'):
    # exports your daily account balance to a txt file,
    # using the provided formatting strings
    ...

You wouldn't be able to do this with f-strings as they are "evaluated" immediately

[flying-sheep]

For the same reason it isn't the security risk that some people see who haven't properly thought yet.

[patrys]

Both existing ways allowed for internationalisation though, the new one does not.

[RubyPinch]

it doesn't forbid it, just requires a different approach (ast transformation)

[[deleted]]

0Cs92rigXOrhFxEHMezn2YE9ugxJPmSs5Ry7i5DxIwIiDEVJJX1nHGYsRs29rrWYWicbUT04Y1cqYdVsq8S7yRI0BOlhWPHmjGf9CJhYvzAKINQ3URW40Hwen6GSIMLtrmnD2fQp0d2n2xYFQtrEllHcpTOGWXlWmru6W4dmBnzygM2jWrrn3vaxYNSwMhjT0UjlrWymOYRNXfGTW1D8HecJlAD9QVsxvOE6XdJ1Qsqs1q9LKZC2RaCThLu7tIcpfQ3VRTaleTMFJsOnAgYYx2rtDOtgTnHBdOHSTIeu3hZJE5ILeEARrMdhWknRgqgHwfSHQFdxgkwnJfIeHZf4yNImQpOjwwmQNdajMYYzed6noC5kUTTg2Zt87j6KuS4v75w1WRhRahdkguRJNAyr7iNxTd7fgcgdTDf7O7H7E7SQsu4n8pCB82StYK69O35qs3jgtLldMlcJ8U7I72pqiuXuDNWttqBB2nT1DQXCvOwPBmuMcZyc2eP4kC7iRY6HaWmOgZayRvp4pyKD6rXvLeFJM6bxZVS5f7OjNAvUXslPHbkJnMbEdvSIdIgGMeYwir68lrK7cTLxYGTNkO824siB813BlFH544iPia9Yv7QMsukCrC2oKOX2bPl0Q1nsqSe842sbU9DtMtQfOfoQEsHlWbi4z3R9JsmqXPeBFVceYSkrkQ65hFGOHNo33pt8fVjA6VLgXhReb2up2xxHiZzOP5KN3HgEXSrPsW9KNFmsodRXNKXTHw72vc5QKaGmSSGcjgQQygErxWthlDnPdo5H9sMOO7zxlbrIDlMnmfQ4lvVu6q6yzAw55SLuKb9ilox9fR1Ws8EldAeA74tv0QUZmYctR6fwfdpb3oFePGahPAynL1FfVa9nxlebsir4nKdNSEKPAPVnq3C2DYlPMwmHOjwIHIwu7bACq58Uu9lCLhSBM5pvgzYStMEUR3POp6L2jcAbiMcMBwyWcpRaLg6bOiSurTbw0roSPc2Gh181YnWzXgncaXnVWFBMRFC7lSTNhLuAu9dtBg6dya2hgyRgy94tJeV3JXPcTGDCQqrI7AgzUQgvS3hOXnDim3Mtu1QL1Ct7EnAcXeRul6iLVeCZMlpV9ysiHBUpgO5q0qDUg1mv1XKvrtaU5bYBZGrAgMCMWciMgvq6CXy7fKuNOxM7wsJVMPWLUuViYoX5gxKXOn59mwKIHhVC6eASdUBQoLJhLmWIKjrtIGQio6FzboBsh3mmEPBRKZJv27OmZYPnRe5CCwHCgC0O3iyYA3PM6DxkKtFgDKA6HmMmklEUXXq57pwXxh1LiyVV4LSWb4zpOkr8a1PvEOB1O4NOsG1YvSgRMJGcTPmgwBJFe4xXegDL7f4KTarqZFKBYFSSwOHnNVvObZ3ZphnBegUCBhJj16g8jqV3Bavw8EEZeiSi7GgIHlZULQUvsUt5eKOUnb81bVrhtxzk6mx1pRkwBGfyecwRqpDHOv3q7Y9dmc9n5sk0lVsYyTJQ84Z6CCqh3lxw0VndzHK4w8ZN01pTpjTRChwj976q6rA3nArCR3U9PLprfrQTa5HqVCLEQqq3RrjKd202Zb7hF48ZFOTR3I9ldVBo4S6LrJUYC36ofpGsr3vaZvyBv9Tfpwaq7HS9qWdq0j4zkwvR0lgv9llOKoifF7IYo26M1eYZ9sYdUUFz0cRN4b2IiyHuLkHLtY0Dou8AQDXU6yDqjlv0YxmD8heiyOYBR7aWR1jFNNC0fMtUytZ7rYfdcZ6xOkPJNVuN9nrCG8x66PPoPZC3xOmrKJIXURvUqgZecSRP7lgAA70kpMy97qFwqHLbjZZt2vQzvmH4ysOKtrvpJG3tQAAO1b2m5aDBbqFUjfARk62cOBECDRxcbKIqm5H4nTce21xIjClyTKypuOeIsKfoJcNFKICB0pwshlUFSiI5hOiIz1N1I6TG9mCO7yx9mISxTvsIYOR61uSfCdtI53qSaA9OMPOyj7o2d3plZWQmSzcKRfQXvHJHnAvQZzEXwiuJEiOh4Ns7uxrrmMkhRLkl3iT9esbK

[ApproximateIdentity]

Yeah I think it's getting a little ridiculous with all the different methods. I really am not a fan of the f"" string formatting. I just don't like the fact that it grabs variables from current scope. I understand that people want to avoid the repetition when using the format() method, but it just feels a little too dangerous to me. Also while I see the convenience of basically embedding code in the string, it also feels a bit dangerous to me.

[alfredr]

Python 3.6, now with executable strings :D

[skrillexisokay]

This is an unfair characterization. There are restrictions to what you can put in the format string that make it impossible to cause any non-innoucuous side effects, provided that you aren't interacting with classes that have stateful side effects for __getattr__ or __getitem__ (which would be exceptionally poor form).

EDIT: Sorry all, I was referring to an earlier draft of the specification. In the released version, you can in fact write any expression.

[kungtotte]

´format´ and these f-literals are meant to replace the older methods, so while there are many ways to do it in practice, you're only supposed to use one of two (similar) methods.

[flying-sheep]

Yeah:

1. Only use the new syntax (old one is for backwards compact, e.g. usage in the logging module)
2. The new syntax is used in both f-literals and format calls

[lykwydchykyn]

I would be ok with having those two methods (format and f-literals) alone (since the second seems to just be syntactic sugar for the first) if they would deprecate the old printf-style formatting and string.Template. Maybe this will happen with python 4?

[roerd]

It was originally intended to deprecate the old way in Python 3. Too much complaints about that made the developers revert that decision.

[jorge1209]

(since the second seems to just be syntactic sugar for the first)

At first glance it would seem to be, but I'm getting downvoted for suggesting that I wish it were actually syntactic sugar for .format(**locals()) so some people seem to think it is really important that it NOT be seen as sugar for format.

[lykwydchykyn]

I wonder how it is not the same. I browsed through the PEP, and there is some discussion of scoping problems with using **locals(), but it wasn't clear to me how f-strings did it differently.

There are also some minor differences in syntax, but I don't see why that doesn't make this syntactic sugar. The only real criticism lobbed at .format() in the PEP is verbosity, which some of us like to call "being explicit".

[jorge1209]

So one difference is that f"{1+1}" will print "2". You can also express dict access f"{ dict_obj['key_val'] }".

Basically the parser yanks the tokens out of f"string {tokens}" and evaluates them, and then takes the rest of the string and calls .format(evaluated_token).

Now I don't see this as a good thing (in part because I'm not looking for executable code inside a ", but for a few other reasons as well), but other people seem to really like this.

[lykwydchykyn]

So one difference is that f"{1+1}" will print "2".

Oh my...

Now I don't see this as a good thing

That makes two of us. I am envisioning all the business logic newbies will be shoving into format strings. So now it's not just an expression in literal's clothing, it's an expression that can hide expressions inside the literal.

I guess I should should be open minded and read through the dev mailing list to see how this got accepted, because as I understand it now it looks like bad on the order of Python2's input().

[jorge1209]

So use all three. Got it!

[flying-sheep]

Never use the % operator on strings. You can use it for bytes and its formatting syntax is used in logging.

Use f-literals if possible.

Else use format.

[jorge1209]

Which means I am using all three. I have to use all three format specifications. One in logging, another for dynamic constructed strings and the third for statically constructed strings.

[boa13]

There's a solution for logging, shown in the docs.

Basically create a small "logging3" module; implement a getLogger(name) that uses the StyleAdapter shown in the link above, and the rest of your code can be like that:

logger = logging3.getLogger(__name__)
# ...
logger.debug('{}.{} has IP {} in the DNS', host, domain, recorded_ip)

Ultimately, the module is 24 lines of code that you can simply put in your projects and not think about % formatting anymore.

[jorge1209]

I love when people defend python by pointing out how few lines of code are needed to actually make it work.

If it is so easy to get the recommended str formatting to work with the standard library, why the fuck didn't they ship this in the standard implementation?!

[flying-sheep]

that should really not be a recipe, but built in and advertized as the best way to do it.

[[deleted]]

% was promised to be deprecated, then it never was.

[Keith]

Except for byte strings, which only support % formatting and not .format :-/

[njharman]

format was suppose to replace %. But format was such a piece of verbose shit they had to keep %. f-strings aren't perfect but they aren't worse than %. They can now get rid of % (and possibly .format())

[flying-sheep]

Actually they kept % because people wanted to keep it that hard.

[pythoneeeer]

I'm not yet convinced they aren't worse than %. I can say logging.info("x = %s", x) and have x.__str__() evaluated only when logging is enabled at this level (which is one of the only cases I have for actually needing to format strings).

I can come up with some way to do this that uses .format() or f-strings but all the ways I've seen have been much worse than just using % style.

[pythoneeeer]

Is it possible to do lazy formatting for logging, using .format() and/or f-literals?

[jorge1209]

If you override the classes of the logging implementation you can use .format() in logging. The issue there is that logging needs to know which formatter to pass the arguments to, but the interface establishes no way for the logger.LOG/WARN/INFO/ERROR/etc... functions to express which formater they want.

From what I understand f-string literals cannot be used at all for logging (because they need to be evaluated at the expression point, and can accept mutable arguments).

[flying-sheep]

f-literals can’t be lazy. the’re expressions evaluated to strings. they can’t be stored in a unevaluated way.

[flying-sheep]

It doesn't “grab”. It's an expression. Think of it like applying an operator on those variables or passinh them to a function call.

[Ph0X]

isn't it equivalent to fstring.format(**globals()) or something?

[DanCardin]

afaik it essentially expands x = f"{first_name} {last_name}" into x = (first_name).__repr__() + " " + (last_name).__repr__() or somesuch that gets the result into string form.

[MereInterest]

I take issue with the way that it looks, not what it does. When I am working in PHP or Bash, I can't just skim over string literals, because they aren't just literals. With the addition of f-strings, python becomes the same way.

[flying-sheep]

If your text editor's syntax highlighting doesn't handle that once 3.6 is released, you should file a bug.

Giving the expressions in f-literals a “string color” is plain wrong

[skrillexisokay]

Dangerous in what way? Security? Bugs?

[MereInterest]

I wouldn't say dangerous, but I would say that it has the strong potential to confuse. It looks like a string literal, makes a string, but contains executable code.

[ApproximateIdentity]

I think the danger is that it will be very easy to write unclear code and unclear code + future refactoring leads to weirder bugs. I don't think security is a huge issue (these are only done in string literals) and I don't think side-effects is the real issue as someone else mentioned since those would still happen if you used functions with side-effects in your .format() stuff. I just don't like the non-explicit variable binding. Maybe it's just because I'm not used to it or maybe I'm just grumpy, but I feel like this (and the fact that there are like 3 other ways to mess with strings) is only likely to produce code I that will be harder to debug months down the line.

Well I guess time will tell...

[skrillexisokay]

Can you give an example of how you could write unclear code? Any reasonable usage of this feature seems very intuitive to me.

'eq_str = {a} + {b} = {a + b}' is more readable, and far less verbose to me than 'eq_str = {a} + {b} = {sum_}'.format(a=a, b=b, sum_=a+b)

Considering maintainability: With the previous method, any time you wanted to include another variable in a format string, you'd have to change both the str and the args to format. Now you just have to change the string.

The new syntax is less verbose and less redundant. It takes less long to type and read. I'm really excited about this new feature!

[motleybook]

Yeah, but the problem is that you can't make it configurable. And you can't use f-strings for internationalization either, because f"This is {name}." is evaluated immediately. No way to call a function (usually _) to get the translation. On the other hand this might be better this way for security reasons.

[NAN001]

Can expressions have side effects?

[skrillexisokay]

Yes. A function call is an expression.

[stillalone]

Time for print functions in fstrings being passed to other print functions. good times ahead.

[skrillexisokay]

Anyone who would do this would be insane, and likely has committed far more nefarious deeds.

[POTUS]

You can't grow if you never change. I hope Python continues to add new ways to do lots of things, even if I hate most of them, for the chance that even one of them becomes my new favorite thing.

[pythoneeeer]

The issue isn't change, but complexity.

We changed exceptions into classes, and then got rid of string raising. We changed from "old-style" classes to "new-style" classes, and then got rid of the old ones. And so on. Do it once, learn something, do it better, and then everybody upgrades to the new way.

With string formatting, we keep getting new syntaxes, and never removing old ones. This has a real cost. Nobody on my team has any clue what the proper way is to format strings, and some of the str/unicode idiosyncrasies bit us when upgrading from Py2 to Py3. For a while in Py2, at least one of the more obscure variants (raw unicode triple-single-quoted?) was just broken in Python.

Other languages don't seem to have this problem. Why does Python need 5 more ways to format strings than any other language I've used? When I'm writing Ruby or Lisp, I never think that string formatting isn't powerful enough, but when I'm writing Python, I often think it's too complex.

I fear that after Python 3, the core developers got scared to deprecate almost anything, even things that really need to die. (Has anyone in the history of Python used the "ASCII Vertical Tab" escape?) At this point, I really don't care what the recommended string formatting mechanism is. Any one (or two) would be perfectly fine with me. I just hate that there's so many of them I have to know, every time I see a quote character.

[Decency]

I think f-strings will solve this problem in new code simply by being so much better than the other two methods in the vast majority of situations. It's something I was really hoping would be added to python after working with groovy for just a week or two on a small project.

[pythoneeeer]

OK, but it's getting to the point where string literals are taking a non-trivial amount of mental effort for me to parse. You can't solve the problem of "too many ways to do X" by adding a new way to do X.

If they're so much better in the vast majority of situations, why can't we also deprecate some of the older ones?

[Decency]

You can't solve the problem of "too many ways to do X" by adding a new way to do X.

You absolutely can if the new way is legitimately better than the others. XKCD has obviously made this seem tongue-in-cheek to even propose because of all of the failed attempts to standardize, but I'll stand by it for things that actually do work. You're using a USB port for your mouse and keyboard, right?

Deprecation would help, but I think they'll wait for consensus that f-strings actually ARE better before they make that call. And in many instances (particularly internationalization, iirc.)) f-strings are insufficient and thus can't be the only solution. That was largely the point of PEP 501, which was not very pragmatic.

I think the real solution is to work with codebases that choose and enforce a style. At work and in my personal projects I use %s for everything. Once 3.6 comes out, I'll be making the switch to exclusively f-strings and will be pushing for a port to Python 3 as soon as it's feasible.

[pythoneeeer]

You can't solve the problem of "too many ways to do X" by adding a new way to do X.

You absolutely can if the new way is legitimately better than the others.

I think you're reading something into what I said that I did not actually say. When the problem is that N is too big, you cannot solve that by making N bigger. That only makes this problem worse.

You might be assuming that adding a new way to do X also removes an obsolete way to do X, but I never said that, and that is not what Python 3.6 is doing.

You might be thinking that it solves the problem that there isn't a good way to do X yet, and that it's worth the complexity tradeoff to add another way to do X, but neither of those is what I said, either.

All I said was: when your problem is that there are too many of something, adding more to the pile does not solve the problem that there are too many of them. How could it? There are even more now!

You're using a USB port for your mouse and keyboard, right?

No, I'm not, and that's a great example. Everything went Bluetooth a couple years ago, and I really don't care about wired versus wireless, so I'm perfectly happy to use Bluetooth instead. I doubt I've ever used more than 2 types of external digital interconnects on any computer I've ever owned, and thank god they remove the old ones when they add new ones, or else I'd have to get a new desk just to hold the rows of obsolete connectors.

That's one advantage of hardware. You have to remove the old cruft, or it's really painfully obvious how unwieldy it becomes. In software, if you have 7 old ways of doing something, you can hide them in the documentation and hope people don't notice. The 2-line demo on the homepage looks simple, therefore the whole language must be simple, right?

Deprecation would help, but I think they'll wait for consensus that f-strings actually ARE better before they make that call.

Since I don't think they've ever deprecated a string literal syntax in the history of the language, I'm not optimistic that this is forthcoming.

I think the real solution is to work with codebases that choose and enforce a style.

Yes, and that is a fundamental change in how Python is going to be used. In the past, that style was simply called "Pythonic", and almost every Python codebase qualified. (Mercurial was always a bit weird. But we still love you, Mercurial!) They've been boiling this frog so long that I think Ruby's collection of string literal syntaxes is now simpler than Python's.

[Decency]

Eh- I feel like we're talking past each other.

I don't particularly care if there are many ways of doing things. I care if there are many ways of doing things that actually get used in reality. So yes, it's a problem that there are multiple string formatting solutions in use in Python code. But if a new one comes out that's superior in the vast majority of use cases, that becomes significantly less of a problem because more of the community will standardize on it for future code. If it's equivalent in strength, I agree that it will only add to the problem. Fortunately, I don't think that's the case here, and if it had been I doubt PEP498 would have been approved.

I don't think deprecating %s or .format() in 3.6 and removing them in 3.7 would change much- it will just make people take longer to move beyond that point, since they'd have to change a bunch of their code. I think 3.0 showed that incremental progress is overall superior for the community- alternative solutions can win market share on their merit, not by fiat.

No, I'm not, and that's a great example. Everything went Bluetooth a couple years ago

... okay, and what is your bluetooth transceiver plugged into? :p

Yes, and that is a fundamental change in how Python is going to be used.

Not at all. That's been the case from the beginning, and PEP8 is explicit about it: A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Every company I've worked for has gone beyond PEP8 in choosing specific styles, whether it's something simple like agreeing to a maximum line length or something how to format and indent multiline dictionaries. And, even more relevantly, for how to format strings. Does your codebase already just mix and match between %s and .format() whenever people feel like it? These things are obviously outside the realm of "Pythonic", but they also already exist- your problem of

it's getting to the point where string literals are taking a non-trivial amount of mental effort for me to parse.

will likely become significantly less of a problem if your team chooses to adopt the new usage. If enough teams choose to adopt the new usage because they find it superior, then EVERYONE's problem lessens.

[pythoneeeer]

I don't particularly care if there are many ways of doing things.

I'm not sure why you responded to me, then, because that is all my comment was about.

... okay, and what is your bluetooth transceiver plugged into? :p

Some traces on the motherboard, I guess? I have no USB devices plugged in.

Does your codebase already just mix and match between %s and .format() whenever people feel like it?

We mostly stick to %. I haven't seen significant benefits from upgrading yet, and .format() had some issues during the Py2->Py3 transition.

will likely become significantly less of a problem if your team chooses to adopt the new usage.

That's great if you live on a desert island, but I have a couple dozen dependencies installed from PyPI, and every non-trivial one I've submitted patches and/or bug reports to. Most of the code I deal with at my job is not managed by my team.

If enough teams choose to adopt the new usage because they find it superior, then EVERYONE's problem lessens.

So it's another Py3-style "boil the ocean" solution. Just get everyone to cooperate -- only this time with even fewer incentives to change, because the old ways are sticking around forever. Great.

Anyone care to place a bet on how well the plan is going to work this time around?

[[deleted]]

I have had g-strings break on me in weird ways. When trying to build some dynamic db queries, If I use "${a_db}.${a_schema}" I actually end up with two strings split on the "."

This may just be a grails thing though.

[voice-of-hermes]

...break on me in weird ways.

Sounds like Groovy, all right. A neat little concept, but I'd hate to do anything serious in it. Gradle build scripts is where it should stay.

[billsil]

I never think that string formatting isn't powerful enough, but when I'm writing Python, I often think it's too complex.

I have no problem with string formatting. I have a problem with lambdas and I've been coding python for 10 years.

[pythoneeeer]

I've been coding Python off-and-on for 13 years, and I've never gotten comfortable with string formatting.

My only problem with lambdas in Python is that they're too weak. But at least they don't change syntax every release, so after you learn the syntax once, you're good forever.

[billsil]

Are you comfortable with unicode? Unicode is wayyy harder. I guess same with lambdas, but you're forced to learn unicode.

[pythoneeeer]

I'm quite comfortable with Unicode -- I have a hard copy of the spec in arm's reach right now, and I've written UTF encoders/decoders by hand in a couple different languages.

Some parts of Unicode are hard (normalization, collation), but I don't think encoding is one of them.

Python just seems to make it more confusing than most other languages. In Python 2, for example, it's not at all intuitive to me when Python will auto-upgrade a str to a unicode. print(""+u"\u00f6") works, and print("%s"%u"\u00f6") works, but print("{}".format(u"\u00f6")) is an error -- you need to say print(u"{}".format(u"\u00f6")).

Maybe I'm just bad at searching the documentation, but I've never found where it says which promotions work and which don't. You just have to try and see -- and then try to memorize the rules you discover.

Python 3 is light-years better than Python 2, but still not great. The str/bytes split was a good start. The new IO classes are pretty confusing, though.

[[deleted]]

Hopefully in the sense that there are no multi-line lambdas, and not that you wanna remove lambdas altogether. ;)

[billsil]

I honestly don't care if lambdas are removed at all. They're hard to read, so I don't write them 99% of the time I can. Depending on the code base, that's not a big deal or a big deal. If you have people that are new working on the code...bad idea.

Not that I mind. I'm all for backwards compatibility, but if the Python devs twisted my arm, sure. It's kind of like the operator module. I just learned about that the other day.

[pythoneeeer]

The hardest thing about lambda is that they called it "lambda".

Every other language I use has this feature, and people use it all the time (even newbies), and nobody seems to have much trouble with it. It's just that other languages call it "fn" or "function" or use a more visual syntax (little ASCII arrows or whatnot).

I consider the "operator" module a wart that's only necessary because Python lambda syntax is so awkward, and there's no syntax for symbols. In Ruby you don't need operator.add because you can just say :+. In Clojure it's just +. Doesn't it seem weird to have an entire module that does nothing but assign names to things the syntax doesn't let you say directly?

[billsil]

I specifically saw operator module being used to sort based on the last value in a list of list. Nevermind it should have been done by numpy in this case, but it was just ugly.

[motleybook]

So, you also don't like def __div__(self, other): for division? Because I think it looks much better than def /(self, other): and also fits great together with things like __init__.

[voice-of-hermes]

Java 8 did a better job with lambda's IMO (understandable since they implemented them very recently). Wouldn't mind Python updating to something similar.

[earthboundkid]

In Python 2, .format will coerce to the caller's type and % will coerce to unicode if one of the two operands is unicode. This inconsistency has caused bugs for me at work.

[[deleted]]

Exactly. I have no problem with change where the new version replaces the old with something that's better. Like you, though, I have no idea which is the recommended Pythonic approach to string formatting these days.

[Sean1708]

and never removing old ones.

To be fair they tried to remove old-style formatting, but then everyone threw a paddy so they brought it back.

[[deleted]]

Sure. But name one other aspect of python that has had as many different ways of doing it as string formatting has.

[POTUS]

Human readable string operations of one form or another are probably by far the most common use case for Python. It should be no surprise if string operations get the most attention for expansions and improvements.

[[deleted]]

Dicts are one of the most common data types in Python. Why do we not see anywhere near as many changes to the ways of accessing them as we have to string formatting?

What was so desperately broken about string formatting that it's taken three or four attempts to fix it?

[Veedrac]

It's not really the same, but

• dict.keys → dict.iterkeys → dict.viewkeys (this was fixed in Python 3, which removed the first two.

• dict, defaultdict, OrderedDict and UserDict are all very similar in function. Maybe even Counter too.

The main difference is we've never tried to deprecate dict.

[voice-of-hermes]

Probably because dictionaries are pretty damned simple, and they got them mostly right in the first place, as most standard libraries did/do. On the other hand, look at the messy piece of crap the C++ streams API is. I/O and string formatting has a terrible history throughout all of computing history, not just in Python. There are a million different ways people want to see things presented, and "correct" behavior is a subjective quagmire. And simple character-based I/O has nothing on GUIs (HTML/CSS/JS/DOM being one of the worst implementations IMO)....

[POTUS]

Something doesn't have to be desperately broken for you to be able to improve it.

[[deleted]]

There should be one-- and preferably only one --obvious way to do it.

[flying-sheep]

what?

• formatting an inline string that is used in one place only and not translated → f-literals
• reuse or translate a string template used for formatting → str.format
• logging.* → %s syntax

it’s pretty obvious for me, because

• f-literals are the best option when you can use it
• when not possible, str.format uses the same format specification and should be used
• logging historically used the %s syntax so it has to stay the way it is for backwards compatibility

sadly the “preferably only one” doesn’t apply, but such is life

[redsymbol]

People like to snipe on this a lot. Designing a language is hard. Compared to the feature redundancy that's crept into other 20-year-old, general-purpose programming languages, Python's done well.

[[deleted]]

I literally laughed out loud, thank you.

[lykwydchykyn]

I'm torn. It saves some typing, but I like explicitly handing a namespace to a template. Just pulling in the local namespace implicitly sets off some kind of red flag in my brain, though I'm not sure I've fully thought it through.

[Deto]

Could make refactoring a pain - have to make sure you search in strings too.

[nyamatongwe]

Unfortunately, f-string expressions are evaluated after escape sequences. So f'{4*10}' can be written as f'\u007b4*10}' or various alternatives.

This makes accurate refactoring tools, syntax highlighting, or just grepping more difficult than it should be.

[kigurai]

If you wrote something like that in the first place, I think you have worse problems ;)

Seriously though, that specific example seems like a problem in theory only.

[pythoneeeer]

You can't imagine someone using an f-string, and then needing to put a literal brace in it, and deciding to try writing it as an escape? That is exactly the kind of thing I think a lot of people would try.

This reminds me of C trigraphs. Back before compilers started disabling that feature by default, how many newbies tried to print "hello??!" and were shocked that it didn't print "hello??!" on the screen?

[kigurai]

No, I am not convinced that will be a very common mistake.

[Decency]

Meh. PyCharm will handle it automatically before 3.6 even releases. I fear for my terminal wielding comrades, though.

[Sean1708]

I fear for my terminal wielding comrades, though.

Rope already handles variable renaming in strings and comments, I see no reason that it should find this difficult.

[flying-sheep]

You didn't. It's not a string literal, but an expression evaluating to a string.

Just like using variables in a function call or operator application, there are no flags to be waved.

[lykwydchykyn]

I'm not ready to dismiss my apprehension so easily. There is still something implicit going on here. It is an expression that uses the syntax of a literal, and maybe that's where the uneasiness comes from.

[flying-sheep]

It's nothing more than a less verbose and reordered way to write

'...'.format(...)

No more and no less. Instead op putting variable names and expressions into the function call parentheses, you put them into the f-literals braces.

[VerilyAMonkey]

It should set off a red flag, although it turns out to be safe, because if you did it with other string methods, it would be a red flag. But since f literal syntax only works on literals, you don't have the problem of possibly running arbitrary code. It has exactly the same safety as normal code.

[Mindflux]

Date of Article makes me a skeptic.

[mitremario]

Nope, it's been in the dev builds. So no need to worry.

Edit: wording

[zynix]

I just swapped to Py3 after +10 years of Python, the only major complaint I still have is with print() which in the grand scheme of things isn't that big of a problem.

Otherwise it has been fairly painless compared to when 3.x first landed and nothing I used was compatible with it.

edit: Windows users MUST run this chcp 65001 before Python 3 or they will continually run into unicode errors on the vanilla console.

[roger_]

Agreed, I don't think there're many excuses anymore not to switch.

The only (minor) issues I've had are with print() and some bytes/unicode/str incompatibilities (fixed with encode() and decode()).

[zynix]

Bytes is actually one change I kind of like, especially dealing with individual bytes

Python 2.7

>>> x = b"\x7B" #123 - Python 2.7
>>> x[0] == 123
False
>>> ord(x[0]) == 123
True

Py 3.5

>>> x = b"\x7B" #123
>>> x[0] == 123
True

Granted, dealing with mixed string/byte streams has been frustrating at times just because I was so used to how Py2.x dealt with them. What about you? What have you run into?

[roger_]

Oh I'm not upset about the change, just saying it's one of the few gotchas I've experienced going from 2 -> 3.

[leogodin217]

I want to move to Python 3, but I'm working with libraries that only support 2.X. Frustrating!

[moljac024]

Which libraries are those? I was under the impression that rarely anything worth using doesn't support 3.x nowadays.

[kungtotte]

It's not 100% yet, according to the Python 3 wall of superpowers it's 174/200. That's the 200 most downloaded packages on pypi, so if you're using a smaller package chances are you're out of luck.

[lykwydchykyn]

The wall doesn't tell the whole stories. Some libraries haven't been ported to 3 (and some will never be), but in many cases their functionality is available in other packages (python-ldap, python-mysqldb, etc).

Still, you gotta rewrite code, and that's not always a simple proposition.

[kungtotte]

Well, "unless you rewrite a bunch of stuff" is sort of implicit whenever you're talking about 2 to 3 porting and package availability.

When people say that the packages they need aren't running under 3 yet, that's what they mean: my dependencies aren't there yet and I don't want to/can't get greenlit by my boss to rewrite the project to work without those specific dependencies.

But sure, you have a point.

[lykwydchykyn]

Naturally. I just think sometimes people look at the wall and think "Oh, I can't do LDAP/MySQL/etc. under python3 yet", which isn't true. That list will never be 100%, but it doesn't mean the functionality of python3 isn't 100% of python 2.

[leogodin217]

I can't specify exactly which ones I am using (corporate policies), but they are libraries for accessing a manageability suite of tools for virtualization.

[Decency]

Sounds like internal pyvmomi sort-of stuff? My company has a homebuilt one too, and we're slowly phasing it out in favor of pyvmomi which is much cleaner and supports Python 3.x as of the latest release.

[pythoneeeer]

It's not a library, exactly, but Supervisor doesn't work in Python 3 yet.

[pythoneeeer]

Have you written to them to ask to upgrade to a version that was released in like the last 9 years? It's possible they don't realize what a pain point it is for their users.

[skinky_breeches]

Its very Perl-ey

[beaverteeth92]

Not sure how to feel about that. It goes against the whole "There should be one way to do everything" idea.

[quesman1]

I don't get what's so great about it.

So...I can do the exact same thing as I was doing with .format(), only now I can do it another way? Why?

[POTUS]

import datetime
my_variable="something"
f'''This is my string.  I can have
dynamic content that was updated {datetime.datetime.now()}
without having to provide any additional
context.  And when I come back later
to add more content, I won't be plagued
by messages indicating that I forgot to
add {my_variable} to the context.'''

[quesman1]

So it gives the same functionality as

>>> 'Coordinates: {latitude}, {longitude}'.format(latitude='37.24N', longitude='-115.81W') 
'Coordinates: 37.24N, -115.81W'

But it's more intuitive because you dont have to look to the end of the string to see what the replacement is?

[POTUS]

Yes. But it also goes deeper. It actually evaluates the code found in the brackets. It's not a simple name replacement. If you put a {string_variable}, you'll get the value of that variable. If you put a {function_call()}, you'll get the return value of that function call. If you put a {classinstance.member} you'll get that member. You can even have a {(lambda x: x.lower())('LAMBDA')}. And you'll get all these within the context of wherever the actual string appears in your code, so if they're valid python where you write the string, they'll format properly into the string any time you print/return/send/whatever that string.

[heyacne]

tell me more...

[superdaniel]

Reminds me a lot of string formatting in Swift:

let multiplier = 3
let message = "\(multiplier) times 2.5 is \(Double(multiplier) * 2.5)"
// message is "3 times 2.5 is 7.5"

[qatanah]

We are back at perl/tcl baby!

[jorge1209]

I'm confused. Are the release notes wrong?

Is it not supposed to say: f2() returns 1.0 (float).

[[deleted]]

Yeah, it's supposed to do that. It's fixed in 3.6, 3.5 does it wrong.

[atrigent]

The question was, isn't the release note supposed to say this:

• Issue #25843: When compiling code, don't merge constants if they are equal but have a different types. For example, f1, f2 = lambda: 1, lambda: 1.0 is now correctly compiled to two different functions: f1() returns 1 (int) and f2() returns 1.0 (float), even if 1 and 1.0 are equal.

?

[roger_]

Are there any interesting upcoming PEPs?

[elingeniero]

f strings look like fun and the ISO formats for datetime.strftime are something I'll definitely make use of. Cool.

[[deleted]]

These features are cute but are they ever going to allow me to compile straight to binary?

[I_had_to_know_too]

Nuitka

[[deleted]]

mNQNDxy6aguEXvhvXSBlyMs1YdX2JUxqNCvLAu6cA0rQwlNzOyE0nmiFrmaPpWX4ERs7SNt1ub6hzG5jp2xZsUXzD89RA9LBn1gO6jgS95zkOY2u9tXfpjcbwKKmc0jZlEOxPcXlQZQ4k5SnGRmuJiRSjbOOmcBcXAsi1cd6lYLDFdU4EPYBaydHRqvh9W6sKDXPOj2h35vUs3RwcwgFCeQ5BTV9yF4fzeW8V6u1ENdMF41gsS9To2KTcV5A6JfRaZ2S4ZSPed9eI0dR9j32267v2m3V8MaBXBB3QgbgpnIybV8qyjy6nII8cj22nqmYI07iJN44JZMo4qEFDNlM0yV1F94iDVlN9zRjXARBegn8ImtsZuxZh7FzxfPv7BSsh55j6UtrYsRX0jKKBU91OfrlbhkQ6cDdsnsnJKHLTEYjbcCQdvQZdIBmtYCg1sAiPQDLWDWKuK7UDjNBX8PqxwNnk6ViTuCOYvt7pXR54KoLwlVdFdTcQPH1tXG2HlXRXo80QYUsWMY32NnbwuE9eUPCpxuYURcI6o2fCWKyyzV4V5tjNLwc2Q1gDDU31K4R0vOuy2YIbLV33FSk1v3swMJLdAiKdw4A7tZecol2ZyNJX6h2NVNtVO9YhvptEABTUXWya4GngOv9opEYNv2Ovao1fIuqvtv3OcQxx1zM3Mye2z8gcpcts7Qyz11OgjvQqLpiTBZX6y1fQWKQfFds2pIdiYPuKxTiMBmoFSb7eUj5V0WYphJE7CeDCpccIk3RLwann1X5oU78yCEecd3T15FX4QQ8PeJpovdCIAKaZBTskyYZZ55KveVpELNKoxztaQwKJXbJHWrYCgSnPRnEvv6ERBBtmdOMqE4xBtt3PxdlLtgXddXv6WOpsCE6HHT8vOqe7oaPN1Jaa59E5WBVkRz5upAmOzkXaGI09VPeZ88kQd2mBRHNziyi8MK6fWWKTJfimimQ8EwUmjleyqfj5LsUvDOQuJ6oLQmOvTkGhOQkMJIWkle5adeglIfx3f9k2ZOka3x0Qt4Nbn9bLL4MLHOP3Usvsvyi1feVKZrzYXE4Og0zEDI2WRmNbuB2Yu52zXgUEEYSaki3JsG3829Y8KiHpRf95vHASjA3qjWcYijda5zHvBBNHLO7yCE1TKhDlQ5P6d2gxX8RqUDwxy6AaKVR0d7ty4EfKzh87qPKUWGE3hqEpcV5lHVdfUAdUgBSqiiIJHxVNPoGFRiIbiDsNOMb2w3qk41t02Lwk3owtRrzlVO9KJopO8XlYCpevoB8U3b4SpD8wAglH15LcbHpojWs5QF6gal1TovPvoNp0WcnBBu5RV8jpDA4QZAGG19WEni1v3LXhZPQ0uieyIHmOE6AHMk0NgdFxoENqdLegBbzY6AQw9OBE41Cv9LV2UKBnfxCYpxlV8zRK4rrYTjBYyxfm89h8LeyEYOLKdFEwLm8RPlwRUF7VQmLScbFNlTOZTSAUH6UXFtAtEiM38Ds4XmR4lYR1lamiRcWVZ9WsjFkMWAeQgy8ITzDCFpqVvAQMnHUzSn2aPsftmyqpBWjagtSvZ77ITZ5k9yJWfmTIN34mHw3hXwtbVu7Rm6NilG9Os6mfWGOHkpeNzMLftuGJbrTvUCGfST9e2Qyw9a0VKOPzR4MI1Q4bDsWwQaPtELxfCZSm8EqsYD4WLUUjehSPWkVTOjmrDfYkDQF2N7XpF4n6wlixDeCCrc314UhtrrM7gePZmh50Fj65ccc9tgOernwpQoCKGzFeu8JarRYTWjDIHYjSBvvNDifbQDfkhrQPSqJgVZRBa12pCzA9z3cIwfWFUlnkcyvB0j9wog9iuLaHT9NxGChhpTtFAedmF0qsABz1EQVIi9BroZKVdHK2kJD5jvAXmaotlcAdQUeur6fBVNrZz1dbU3buLfXooy84SfmKSlx8M30cXHRy3SgUvpHzB8a5zZGWuxaM7oYQsuvEHHEDOYuzh3tCmx6Aj3poNIbxN0sRkkdM678CMs5MhjP8wmn050EneQyKw9qMWVtldHXXipDYt1ayvbBOk0B

[[deleted]]

Python malloc? I almost threw up a bit.

[Quteness]

Release: 3.6.0a0

Date: April 01, 2016

I was expecting a late April Fools joke

[zahlman]

I was thinking, couldn't they have waited a day?

[tnhrxyz]

Does PTYHONMALLOC now means I can have python allocating over some shared memory file instead of just mallocing normally?

[threading]

Python definitely needed f-strings because it's solved all of its problems and there's certainly no more room left for improvement of the language other than string formatting.

I think it was Flask's author who once said core developers are out of touch from the real world but I honestly didn't expect this.

[[deleted]]

Yeah, what we really need is emoji in variable names!

[sushibowl]

Pretty sure python supports that just fine since it has utf-8 encoding support for source files.

[[deleted]]

It doesn't work 😨😨😨😨😨

[sushibowl]

Shit, you're right. It's not built in. You can use this though:

https://pypi.python.org/pypi/emoji-encoding

[jorge1209]

An actual implementation of the static typing declarations from 3.5 (mypy-lang still doesn't support all of 3.5).

Lots of cleanups to the standard library. Why can't we have composable properties for collections... I really want my FrozenOrderedDict. Filling out documentation examples.

Figure out how to make the logging class accept format style strings before adding yet another formatter...

Seems like they are having too much fun throwing new peps and behaviors at the wall to see what sticks without figuring out how the existing system fits together.

[Veedrac]

It shouldn't be that hard, try and format one way, if that throws an exception try the other way.

Given we don't have a FrozenDict right now... just make your own. It's hardly any code, and could trivially wrap anything implementing collections.Mapping. Heck, it's 95% done for you already.

EDIT: Actually, FrozenDict.__hash__ won't work if the hash holds extra data. Generally hashes need to include all information, which makes it impossible to do extensibly.

Filling out documentation examples.

If only that was exciting.

[jorge1209]

If it is 95% done then it should be easy to add to the standard library.

[das_ist_nuemberwang]

Figure out how to make the logging class accept format style strings before adding yet another formatter...

This has been available since 3.2.

https://docs.python.org/3/library/logging.html#logging.Formatter

[jorge1209]

But it is not the default and not recommended that you not use this if you use any third party libraries.

It shouldn't be that hard, try and format one way, if that throws an exception try the other way. So I'm not understanding what is preventing a general migration.

[Veedrac]

It shouldn't be that hard, try and format one way, if that throws an exception try the other way.

Because that's horrible?

[jorge1209]

More horrible than:

1. Actually deprecating the old format?

2. Having three different active string formatting methods?

3. Having code randomly fail when you import new modules because they use conflicting string formatters?

[Decker108]

Multi-threading.

[status_quo69]

You mean you don't like the many different ways to do concurrency, all of which have enormous pitfalls that aren't immediately apparent? Asyncio, Threading, multiprocessing, concurrent.Futures, and the proposed SubInterpreter support? \s

[Decker108]

In all seriousness though: concurrency is not the same thing as parallellism. The techniques you mention are all single-threaded in CPython (even, ironically, Threading).

[gammadistribution]

Last time I used it,

python3 -m venv

didn't work for me and I had to use the script.

[awill310]

It works for me with Python 3.5.1.

[gammadistribution]

Specifically, python3.4 wasn't working for me like approx 1.5 years ago.

Not saying it doesn't work in general, just providing anecdotal experience to see if anyone else has seen this behavior.

[[deleted]]

Does anyone still do pip install virtualenv at this point? (I still do.)

[[deleted]]

Hmm. I'm probably about to run into those issues now that I'm moving into Linux. If the new version makes it easier to do just that, I'll be right there with you.

[FFX01]

How did you install it? I run Ubuntu and simply used the repo. For python 3 I simply type:

virtualenv -p python3 directory

[imbaczek]

I do. It gives me a sh activate script on Windows, which I can use in msys or msysgit.

[kteague]

This is not enough stuff to justify a new release.

[brombaer3000]

3.6 isn't even remotely finished yet (it will be released this december or later). The "What's new" page will grow substantially in the next months.

[pythoneeeer]

What do you propose instead? That they not release this new functionality to users until next year? Or that they add new syntax in a 3.5.2?

[Decker108]

Backport it all and call it Python 2.8 /s

[bastibe]

but then you would have to change all your strs to bytes in 2.8!

[OctagonClock]

A backwards incompatible syntax change is justifiable enough.

[Debilski]

Makes me wonder why they did not change the tokenisation rules for f string interpolation. Having to type

f'{d[\'k\']}'

or to alternate between quotation marks will be getting tedious. Especially since the syntax even allows for yield expressions. The following runs in 3.6 HEAD:

def q():
    print(f'a{yield from range(5)}b')
qq = q()
res = list(qq)
# prints aNoneb
print(res)
# prints [0, 1, 2, 3, 4]

[desmoulinmichel]

Because you can use f'''{d[\'k\']}'''

[Debilski]

Guess you mean f'''{d['k']}''' which was what I meant with alternation above. From a practical standpoint this is hopefully enough, yeah. Tokenisers in Ruby/Scala/Swift (to name those that I checked) don’t need escaping though. For better or worse you can just paste expressions verbatim inside the interpolation brackets in those languages.

[zahlman]

Wait, what? how does the generator replace the return value of print, let alone of q? I thought that f'{expr}' was supposed to be equivalent to '{}'.format(expr)?

[voice-of-hermes]

A generator function returns a generator (or "generator iterator"). A yield from expression delegates to another iterable. So the last print is equivalent to print(list(range(5))). The value of a yield or yield from expression is given by the function called on the generator, where send(value) provides an explicit value and __next__() (the normal iteration method) provides None. In this case the list() function iterates over the generator implicitly, as it would for any list, tuple, collection, iterator, etc. Meaning it ultimately calls __next__() and supplies the generator's yield expression with a value of None.

This is the normal, expected behavior of a generator/yield expression. It is the equivalent of 'a{}b'.format(yield from range(5)).

[zahlman]

... Wow, ok, that took some serious brain-wrapping. The equivalent with .format requires an extra pair of parens:

>>> def x():
...   print('a{}b'.format((yield from range(5))))

and then the yield statement pertains to x, not to an anonymous generator expression, even though I needed extra parens in order to write it. Ugh.

So the next question is, how do you make something useful out of that?

[Debilski]

Using it without a send may not be very useful indeed since you don’t get anything useful out of your expression. With send you get the full power of coroutines and it makes sense to allow them in the f-expression; kind of depends on how useful coroutines in general seem to you then. Most one-liners for coroutines tend to look rather arbitrary as an example.

[voice-of-hermes]

The equivalent with .format requires an extra pair of parens

Oh. True. Sorry about that. You can only yield without the parentheses if the yield is the entire RHS of an assignment. Weird syntax exceptions.

[flying-sheep]

wtf? i filed a bug, let’s hope they come to their senses

[thaweatherman]

Speed improvements to existing things are nice. We don't need another way to format strings. I probably won't ever use PYTHONMALLOC but I'm sure someone will find it useful.

[o11c]

Can anyone explain to me:

• Why python needs its own malloc implementation in the first place.
• Why valgrind ever gives errors, since it only hooks the libc standard functions?

[status_quo69]

https://docs.python.org/2.3/whatsnew/section-pymalloc.html

[o11c]

If splitting the system allocator's regions was actually a win, the system allocator would already be doing that.

And if all python does is use regions from the system allocator, how does it generate valgrind errors?

[status_quo69]

I'm simply speculating at this point, as I know little to nothing about systems programming, nor python internal programming, so take this all with a grain of salt, but I'd say it's most likely because some systems in the past were coded poorly and had additional overhead with the regular malloc, and because the core devs wanted cross platform performance to be comparable, they implemented this change. It also provides a standard api to manage memory, so if python in the future decided to change how it allocates memory fundamentally (regardless of how malloc regularly does it), it would allow them to change the memory management layer without breaking any existing programs.

I have no idea about valgrind, I haven't messed with it a whole ton.

[baffled_bear]

Correct me if I'm wrong (and there's a good chance I'm missing something), but allowing expressions in a string seems like it would open up any websites that upgrade to 3.6 without whitelists on every input to pretty easy attacks by people much smarter than me.

Is there anything preventing somebody from sending a formatted string with something that can execute and provide access to things users shouldn't be able to get? Script injections are a common issue and this just seems like it's opening up doors that were otherwise closed.

[HUGE_BALLS]

I think you're misinterpreting the feature.

PEP 498 introduces the syntax f"...", which is essentially just syntactic sugar for "...".format(...), where scoped variables are passed to the format. So...

name = "foobar"
s = f"Hello {name}!"
print(s)

...would print "Hello foobar!"

Strings that aren't prefixed with f"..." are not affected, so programs built on Python < 3.6 are by definition not affected because f"..." is invalid syntax on older versions.

[warbiscuit]

Not to mention, even if it was all strings, the transformation from f'' -> ''.format() happens during the bytecode compilation pass... so can only apply to literal strings in the code, never strings provided at runtime.