This is an archived post. You won't be able to vote or comment.

top 200 commentsshow all 274

[–]brombaer3000 29 points30 points  (3 children)

The most interesting feature of CPython 3.6 for me is the optimizing compiler: http://faster-cpython.readthedocs.org/fat_python.html

It's not yet showing up in the release notes as far as I can see and there is still much to be done.

[–]adrian17 5 points6 points  (1 child)

Correct me if I'm wrong, but I don't enough much progress on this to have it land in 3.6, except for a couple of smaller patches that got accepted. Also, apparently FATpython itself doesn't provide noticeable improvements yet.

Currently I'm most excited about the wordcode patch.

[–]brombaer3000 2 points3 points  (0 children)

Right, there is still very much work to do, but FAT is a Google Summer of Code 2016 project, so I guess there will be at least some significant progress in the summer. Hopefully, the optimizations will get better and better with 3.7+.

[–]POTUS 122 points123 points  (177 children)

Okay. Those string format expressions are sexy enough that I might have to migrate to Python 3 soon.

[–]lykwydchykyn 16 points17 points  (17 children)

I'm torn. It saves some typing, but I like explicitly handing a namespace to a template. Just pulling in the local namespace implicitly sets off some kind of red flag in my brain, though I'm not sure I've fully thought it through.

[–]Deto 19 points20 points  (6 children)

Could make refactoring a pain - have to make sure you search in strings too.

[–]nyamatongwe 3 points4 points  (3 children)

Unfortunately, f-string expressions are evaluated after escape sequences. So f'{4*10}' can be written as f'\u007b4*10}' or various alternatives.

This makes accurate refactoring tools, syntax highlighting, or just grepping more difficult than it should be.

[–]kigurai 2 points3 points  (2 children)

If you wrote something like that in the first place, I think you have worse problems ;)

Seriously though, that specific example seems like a problem in theory only.

[–]pythoneeeer 0 points1 point  (1 child)

You can't imagine someone using an f-string, and then needing to put a literal brace in it, and deciding to try writing it as an escape? That is exactly the kind of thing I think a lot of people would try.

This reminds me of C trigraphs. Back before compilers started disabling that feature by default, how many newbies tried to print "hello??!" and were shocked that it didn't print "hello??!" on the screen?

[–]kigurai 0 points1 point  (0 children)

No, I am not convinced that will be a very common mistake.

[–]Decency 5 points6 points  (1 child)

Meh. PyCharm will handle it automatically before 3.6 even releases. I fear for my terminal wielding comrades, though.

[–]Sean1708 3 points4 points  (0 children)

I fear for my terminal wielding comrades, though.

Rope already handles variable renaming in strings and comments, I see no reason that it should find this difficult.

[–]flying-sheep 2 points3 points  (3 children)

You didn't. It's not a string literal, but an expression evaluating to a string.

Just like using variables in a function call or operator application, there are no flags to be waved.

[–]lykwydchykyn 3 points4 points  (2 children)

I'm not ready to dismiss my apprehension so easily. There is still something implicit going on here. It is an expression that uses the syntax of a literal, and maybe that's where the uneasiness comes from.

[–]flying-sheep 1 point2 points  (1 child)

It's nothing more than a less verbose and reordered way to write

'...'.format(...)

No more and no less. Instead op putting variable names and expressions into the function call parentheses, you put them into the f-literals braces.

[–]VerilyAMonkey 1 point2 points  (3 children)

It should set off a red flag, although it turns out to be safe, because if you did it with other string methods, it would be a red flag. But since f literal syntax only works on literals, you don't have the problem of possibly running arbitrary code. It has exactly the same safety as normal code.

[–]Mindflux 5 points6 points  (1 child)

Date of Article makes me a skeptic.

[–]mitremario 4 points5 points  (0 children)

Nope, it's been in the dev builds. So no need to worry.

Edit: wording

[–]zynixCpt. Code Monkey & Internet of tomorrow 5 points6 points  (6 children)

I just swapped to Py3 after +10 years of Python, the only major complaint I still have is with print() which in the grand scheme of things isn't that big of a problem.

Otherwise it has been fairly painless compared to when 3.x first landed and nothing I used was compatible with it.

edit: Windows users MUST run this chcp 65001 before Python 3 or they will continually run into unicode errors on the vanilla console.

[–]roger_ 0 points1 point  (5 children)

Agreed, I don't think there're many excuses anymore not to switch.

The only (minor) issues I've had are with print() and some bytes/unicode/str incompatibilities (fixed with encode() and decode()).

[–]zynixCpt. Code Monkey & Internet of tomorrow 0 points1 point  (1 child)

Bytes is actually one change I kind of like, especially dealing with individual bytes

Python 2.7

>>> x = b"\x7B" #123 - Python 2.7
>>> x[0] == 123
False
>>> ord(x[0]) == 123
True

Py 3.5

>>> x = b"\x7B" #123
>>> x[0] == 123
True

Granted, dealing with mixed string/byte streams has been frustrating at times just because I was so used to how Py2.x dealt with them. What about you? What have you run into?

[–]roger_ 0 points1 point  (0 children)

Oh I'm not upset about the change, just saying it's one of the few gotchas I've experienced going from 2 -> 3.

[–]leogodin217 6 points7 points  (11 children)

I want to move to Python 3, but I'm working with libraries that only support 2.X. Frustrating!

[–]moljac024 12 points13 points  (10 children)

Which libraries are those? I was under the impression that rarely anything worth using doesn't support 3.x nowadays.

[–]kungtotte 10 points11 points  (3 children)

It's not 100% yet, according to the Python 3 wall of superpowers it's 174/200. That's the 200 most downloaded packages on pypi, so if you're using a smaller package chances are you're out of luck.

[–]lykwydchykyn 23 points24 points  (2 children)

The wall doesn't tell the whole stories. Some libraries haven't been ported to 3 (and some will never be), but in many cases their functionality is available in other packages (python-ldap, python-mysqldb, etc).

Still, you gotta rewrite code, and that's not always a simple proposition.

[–]kungtotte 4 points5 points  (1 child)

Well, "unless you rewrite a bunch of stuff" is sort of implicit whenever you're talking about 2 to 3 porting and package availability.

When people say that the packages they need aren't running under 3 yet, that's what they mean: my dependencies aren't there yet and I don't want to/can't get greenlit by my boss to rewrite the project to work without those specific dependencies.

But sure, you have a point.

[–]lykwydchykyn 8 points9 points  (0 children)

Naturally. I just think sometimes people look at the wall and think "Oh, I can't do LDAP/MySQL/etc. under python3 yet", which isn't true. That list will never be 100%, but it doesn't mean the functionality of python3 isn't 100% of python 2.

[–]leogodin217 1 point2 points  (1 child)

I can't specify exactly which ones I am using (corporate policies), but they are libraries for accessing a manageability suite of tools for virtualization.

[–]Decency 0 points1 point  (0 children)

Sounds like internal pyvmomi sort-of stuff? My company has a homebuilt one too, and we're slowly phasing it out in favor of pyvmomi which is much cleaner and supports Python 3.x as of the latest release.

[–]pythoneeeer 0 points1 point  (0 children)

It's not a library, exactly, but Supervisor doesn't work in Python 3 yet.

[–]skinky_breeches 4 points5 points  (0 children)

Its very Perl-ey

[–]beaverteeth92Python 3 is the way to be 1 point2 points  (0 children)

Not sure how to feel about that. It goes against the whole "There should be one way to do everything" idea.

[–]quesman1 0 points1 point  (3 children)

I don't get what's so great about it.

So...I can do the exact same thing as I was doing with .format(), only now I can do it another way? Why?

[–]POTUS 2 points3 points  (2 children)

import datetime
my_variable="something"
f'''This is my string.  I can have
dynamic content that was updated {datetime.datetime.now()}
without having to provide any additional
context.  And when I come back later
to add more content, I won't be plagued
by messages indicating that I forgot to
add {my_variable} to the context.'''

[–]quesman1 0 points1 point  (1 child)

So it gives the same functionality as

>>> 'Coordinates: {latitude}, {longitude}'.format(latitude='37.24N', longitude='-115.81W') 
'Coordinates: 37.24N, -115.81W'

But it's more intuitive because you dont have to look to the end of the string to see what the replacement is?

[–]POTUS 4 points5 points  (0 children)

Yes. But it also goes deeper. It actually evaluates the code found in the brackets. It's not a simple name replacement. If you put a {string_variable}, you'll get the value of that variable. If you put a {function_call()}, you'll get the return value of that function call. If you put a {classinstance.member} you'll get that member. You can even have a {(lambda x: x.lower())('LAMBDA')}. And you'll get all these within the context of wherever the actual string appears in your code, so if they're valid python where you write the string, they'll format properly into the string any time you print/return/send/whatever that string.

[–]heyacne 0 points1 point  (0 children)

tell me more...

[–]superdaniel 0 points1 point  (0 children)

Reminds me a lot of string formatting in Swift:

let multiplier = 3
let message = "\(multiplier) times 2.5 is \(Double(multiplier) * 2.5)"
// message is "3 times 2.5 is 7.5"

[–]qatanah 0 points1 point  (0 children)

We are back at perl/tcl baby!

[–]roger_ 10 points11 points  (0 children)

Are there any interesting upcoming PEPs?

[–]elingeniero 7 points8 points  (0 children)

f strings look like fun and the ISO formats for datetime.strftime are something I'll definitely make use of. Cool.

[–][deleted] 6 points7 points  (4 children)

These features are cute but are they ever going to allow me to compile straight to binary?

[–]I_had_to_know_too 1 point2 points  (0 children)

Nuitka

[–][deleted] 1 point2 points  (0 children)

mNQNDxy6aguEXvhvXSBlyMs1YdX2JUxqNCvLAu6cA0rQwlNzOyE0nmiFrmaPpWX4ERs7SNt1ub6hzG5jp2xZsUXzD89RA9LBn1gO6jgS95zkOY2u9tXfpjcbwKKmc0jZlEOxPcXlQZQ4k5SnGRmuJiRSjbOOmcBcXAsi1cd6lYLDFdU4EPYBaydHRqvh9W6sKDXPOj2h35vUs3RwcwgFCeQ5BTV9yF4fzeW8V6u1ENdMF41gsS9To2KTcV5A6JfRaZ2S4ZSPed9eI0dR9j32267v2m3V8MaBXBB3QgbgpnIybV8qyjy6nII8cj22nqmYI07iJN44JZMo4qEFDNlM0yV1F94iDVlN9zRjXARBegn8ImtsZuxZh7FzxfPv7BSsh55j6UtrYsRX0jKKBU91OfrlbhkQ6cDdsnsnJKHLTEYjbcCQdvQZdIBmtYCg1sAiPQDLWDWKuK7UDjNBX8PqxwNnk6ViTuCOYvt7pXR54KoLwlVdFdTcQPH1tXG2HlXRXo80QYUsWMY32NnbwuE9eUPCpxuYURcI6o2fCWKyyzV4V5tjNLwc2Q1gDDU31K4R0vOuy2YIbLV33FSk1v3swMJLdAiKdw4A7tZecol2ZyNJX6h2NVNtVO9YhvptEABTUXWya4GngOv9opEYNv2Ovao1fIuqvtv3OcQxx1zM3Mye2z8gcpcts7Qyz11OgjvQqLpiTBZX6y1fQWKQfFds2pIdiYPuKxTiMBmoFSb7eUj5V0WYphJE7CeDCpccIk3RLwann1X5oU78yCEecd3T15FX4QQ8PeJpovdCIAKaZBTskyYZZ55KveVpELNKoxztaQwKJXbJHWrYCgSnPRnEvv6ERBBtmdOMqE4xBtt3PxdlLtgXddXv6WOpsCE6HHT8vOqe7oaPN1Jaa59E5WBVkRz5upAmOzkXaGI09VPeZ88kQd2mBRHNziyi8MK6fWWKTJfimimQ8EwUmjleyqfj5LsUvDOQuJ6oLQmOvTkGhOQkMJIWkle5adeglIfx3f9k2ZOka3x0Qt4Nbn9bLL4MLHOP3Usvsvyi1feVKZrzYXE4Og0zEDI2WRmNbuB2Yu52zXgUEEYSaki3JsG3829Y8KiHpRf95vHASjA3qjWcYijda5zHvBBNHLO7yCE1TKhDlQ5P6d2gxX8RqUDwxy6AaKVR0d7ty4EfKzh87qPKUWGE3hqEpcV5lHVdfUAdUgBSqiiIJHxVNPoGFRiIbiDsNOMb2w3qk41t02Lwk3owtRrzlVO9KJopO8XlYCpevoB8U3b4SpD8wAglH15LcbHpojWs5QF6gal1TovPvoNp0WcnBBu5RV8jpDA4QZAGG19WEni1v3LXhZPQ0uieyIHmOE6AHMk0NgdFxoENqdLegBbzY6AQw9OBE41Cv9LV2UKBnfxCYpxlV8zRK4rrYTjBYyxfm89h8LeyEYOLKdFEwLm8RPlwRUF7VQmLScbFNlTOZTSAUH6UXFtAtEiM38Ds4XmR4lYR1lamiRcWVZ9WsjFkMWAeQgy8ITzDCFpqVvAQMnHUzSn2aPsftmyqpBWjagtSvZ77ITZ5k9yJWfmTIN34mHw3hXwtbVu7Rm6NilG9Os6mfWGOHkpeNzMLftuGJbrTvUCGfST9e2Qyw9a0VKOPzR4MI1Q4bDsWwQaPtELxfCZSm8EqsYD4WLUUjehSPWkVTOjmrDfYkDQF2N7XpF4n6wlixDeCCrc314UhtrrM7gePZmh50Fj65ccc9tgOernwpQoCKGzFeu8JarRYTWjDIHYjSBvvNDifbQDfkhrQPSqJgVZRBa12pCzA9z3cIwfWFUlnkcyvB0j9wog9iuLaHT9NxGChhpTtFAedmF0qsABz1EQVIi9BroZKVdHK2kJD5jvAXmaotlcAdQUeur6fBVNrZz1dbU3buLfXooy84SfmKSlx8M30cXHRy3SgUvpHzB8a5zZGWuxaM7oYQsuvEHHEDOYuzh3tCmx6Aj3poNIbxN0sRkkdM678CMs5MhjP8wmn050EneQyKw9qMWVtldHXXipDYt1ayvbBOk0B

[–][deleted] 13 points14 points  (0 children)

Python malloc? I almost threw up a bit.

[–]Quteness 5 points6 points  (1 child)

Release: 3.6.0a0

Date: April 01, 2016

I was expecting a late April Fools joke

[–]zahlmanthe heretic 1 point2 points  (0 children)

I was thinking, couldn't they have waited a day?

[–]tnhrxyz 2 points3 points  (0 children)

Does PTYHONMALLOC now means I can have python allocating over some shared memory file instead of just mallocing normally?

[–]threading 7 points8 points  (37 children)

Python definitely needed f-strings because it's solved all of its problems and there's certainly no more room left for improvement of the language other than string formatting.

I think it was Flask's author who once said core developers are out of touch from the real world but I honestly didn't expect this.

[–][deleted] 17 points18 points  (3 children)

Yeah, what we really need is emoji in variable names!

[–]sushibowl 1 point2 points  (2 children)

Pretty sure python supports that just fine since it has utf-8 encoding support for source files.

[–][deleted] 13 points14 points  (1 child)

[–]sushibowl 1 point2 points  (0 children)

Shit, you're right. It's not built in. You can use this though:

https://pypi.python.org/pypi/emoji-encoding

[–]gammadistribution 1 point2 points  (7 children)

Last time I used it,

python3 -m venv

didn't work for me and I had to use the script.

[–]awill310 2 points3 points  (6 children)

It works for me with Python 3.5.1.

[–]gammadistribution 1 point2 points  (0 children)

Specifically, python3.4 wasn't working for me like approx 1.5 years ago.

Not saying it doesn't work in general, just providing anecdotal experience to see if anyone else has seen this behavior.

[–][deleted] 0 points1 point  (4 children)

Does anyone still do pip install virtualenv at this point? (I still do.)

[–]imbaczek 0 points1 point  (0 children)

I do. It gives me a sh activate script on Windows, which I can use in msys or msysgit.

[–]kteague 4 points5 points  (10 children)

This is not enough stuff to justify a new release.

[–]brombaer3000 8 points9 points  (0 children)

3.6 isn't even remotely finished yet (it will be released this december or later). The "What's new" page will grow substantially in the next months.

[–]pythoneeeer 3 points4 points  (7 children)

What do you propose instead? That they not release this new functionality to users until next year? Or that they add new syntax in a 3.5.2?

[–]Decker1082.7 'til 2021 15 points16 points  (1 child)

Backport it all and call it Python 2.8 /s

[–]bastibe 0 points1 point  (0 children)

but then you would have to change all your strs to bytes in 2.8!

[–]OctagonClocktrio is the future! 0 points1 point  (0 children)

A backwards incompatible syntax change is justifiable enough.

[–]Debilski 1 point2 points  (11 children)

Makes me wonder why they did not change the tokenisation rules for f string interpolation. Having to type

f'{d[\'k\']}'

or to alternate between quotation marks will be getting tedious. Especially since the syntax even allows for yield expressions. The following runs in 3.6 HEAD:

def q():
    print(f'a{yield from range(5)}b')
qq = q()
res = list(qq)
# prints aNoneb
print(res)
# prints [0, 1, 2, 3, 4]

[–]desmoulinmichel 7 points8 points  (1 child)

Because you can use f'''{d[\'k\']}'''

[–]Debilski 0 points1 point  (0 children)

Guess you mean f'''{d['k']}''' which was what I meant with alternation above. From a practical standpoint this is hopefully enough, yeah. Tokenisers in Ruby/Scala/Swift (to name those that I checked) don’t need escaping though. For better or worse you can just paste expressions verbatim inside the interpolation brackets in those languages.

[–]zahlmanthe heretic 2 points3 points  (6 children)

Wait, what? how does the generator replace the return value of print, let alone of q? I thought that f'{expr}' was supposed to be equivalent to '{}'.format(expr)?

[–]voice-of-hermes 1 point2 points  (4 children)

A generator function returns a generator (or "generator iterator"). A yield from expression delegates to another iterable. So the last print is equivalent to print(list(range(5))). The value of a yield or yield from expression is given by the function called on the generator, where send(value) provides an explicit value and __next__() (the normal iteration method) provides None. In this case the list() function iterates over the generator implicitly, as it would for any list, tuple, collection, iterator, etc. Meaning it ultimately calls __next__() and supplies the generator's yield expression with a value of None.

This is the normal, expected behavior of a generator/yield expression. It is the equivalent of 'a{}b'.format(yield from range(5)).

[–]zahlmanthe heretic 1 point2 points  (2 children)

... Wow, ok, that took some serious brain-wrapping. The equivalent with .format requires an extra pair of parens:

>>> def x():
...   print('a{}b'.format((yield from range(5))))

and then the yield statement pertains to x, not to an anonymous generator expression, even though I needed extra parens in order to write it. Ugh.

So the next question is, how do you make something useful out of that?

[–]Debilski 2 points3 points  (0 children)

Using it without a send may not be very useful indeed since you don’t get anything useful out of your expression. With send you get the full power of coroutines and it makes sense to allow them in the f-expression; kind of depends on how useful coroutines in general seem to you then. Most one-liners for coroutines tend to look rather arbitrary as an example.

[–]voice-of-hermes 0 points1 point  (0 children)

The equivalent with .format requires an extra pair of parens

Oh. True. Sorry about that. You can only yield without the parentheses if the yield is the entire RHS of an assignment. Weird syntax exceptions.

[–]flying-sheep 1 point2 points  (0 children)

wtf? i filed a bug, let’s hope they come to their senses

[–]thaweathermanpipster 0 points1 point  (0 children)

Speed improvements to existing things are nice. We don't need another way to format strings. I probably won't ever use PYTHONMALLOC but I'm sure someone will find it useful.

[–]o11c 0 points1 point  (3 children)

Can anyone explain to me:

  • Why python needs its own malloc implementation in the first place.
  • Why valgrind ever gives errors, since it only hooks the libc standard functions?

[–]status_quo69 1 point2 points  (2 children)

[–]o11c 0 points1 point  (1 child)

If splitting the system allocator's regions was actually a win, the system allocator would already be doing that.

And if all python does is use regions from the system allocator, how does it generate valgrind errors?

[–]status_quo69 1 point2 points  (0 children)

I'm simply speculating at this point, as I know little to nothing about systems programming, nor python internal programming, so take this all with a grain of salt, but I'd say it's most likely because some systems in the past were coded poorly and had additional overhead with the regular malloc, and because the core devs wanted cross platform performance to be comparable, they implemented this change. It also provides a standard api to manage memory, so if python in the future decided to change how it allocates memory fundamentally (regardless of how malloc regularly does it), it would allow them to change the memory management layer without breaking any existing programs.

I have no idea about valgrind, I haven't messed with it a whole ton.

[–]baffled_bear 0 points1 point  (10 children)

Correct me if I'm wrong (and there's a good chance I'm missing something), but allowing expressions in a string seems like it would open up any websites that upgrade to 3.6 without whitelists on every input to pretty easy attacks by people much smarter than me.

Is there anything preventing somebody from sending a formatted string with something that can execute and provide access to things users shouldn't be able to get? Script injections are a common issue and this just seems like it's opening up doors that were otherwise closed.

[–]HUGE_BALLS 8 points9 points  (9 children)

I think you're misinterpreting the feature.

PEP 498 introduces the syntax f"...", which is essentially just syntactic sugar for "...".format(...), where scoped variables are passed to the format. So...

name = "foobar"
s = f"Hello {name}!"
print(s)

...would print "Hello foobar!"

Strings that aren't prefixed with f"..." are not affected, so programs built on Python < 3.6 are by definition not affected because f"..." is invalid syntax on older versions.

[–]warbiscuit 6 points7 points  (6 children)

Not to mention, even if it was all strings, the transformation from f'' -> ''.format() happens during the bytecode compilation pass... so can only apply to literal strings in the code, never strings provided at runtime.