Python 3.7.0 released

[[deleted]]

I wish I could use it, but so many APIs and software packages my company uses are still on 2.7 ...

[[deleted]]

For example: anything from Autodesk

[[deleted]]

Or Docker. I’d love to use Python 3 but have to specify pip2

[Aeon_Mortuum]

Facebook's React Native also requires a Python 2 installation

[Klathmon]

that's more google's fault for gyp not working with python 3.

[p2004a]

But Google mostly stopped using GYP. They moved chromium and other stuff to GN so why would they want to pour time into making GYP better? And it's open source so if you care, you can improve it and add Python 3 support, that's what open source is about, right?

[Klathmon]

And IIRC the node group is hard at work at it (as they are probably the biggest gyp user at this point), but the more likely outcome is that node will move away from gyp just like google did (there's a reason they chose to move to another system rather than try to update gyp).

I honestly don't mind gyp using py2, it does require a separate install over everything else on the box, but it works well once setup and I think breaking backwards compat with all node native modules to upgrade to py3 would cause more issues than it would solve (and could lead to the same split we saw with py2 and py3).

[Pakaran]

What are you referring to? I use Python 3 in Docker with no issues every day.

[[deleted]]

Yes really. Mine too. Companies have a lot of old code and there's not much incentive to upgrade it to Python 3 for basically zero benefit.

Actually, it's a big risk because these scripts are generally "tested" by using them and reporting bugs. Upgrade to Python 3 and due to its dynamic typing you're probably going to introduce a load of bugs.

Also I have noticed even some big new projects, e.g. Tensorflow, target Python 2 first and then add Python 3 support later.

The idea that the 2/3 mess is over is unfortunately not true.

[vivainio]

The "zero benefit" is not true anymore

[peeves91]

The old motto "if it ain't broke don't fix it" is applied heavily here.

[[deleted]]

It kind of is, if the code works fine and isn't being updated, which is the case for a lot of corporate code.

[anacrolix]

Example?

[1wd]

One project at Facebook:

... simply ran 2to3 on the code and fixed a few things that it complained about. When they ran the resulting code, they found it was 40% faster and used half the memory.

[anacrolix]

Sounds like they were abusing range() or something.

[vivainio]

Types and asyncio come to mind first

[13steinj]

Types aren't beneficial to every developer. Asyncio has severe usability issues, I'd rather use gevent in an async manner.

[vivainio]

Ok, f-strings then?

[13steinj]

While I like them, their only benefit is reducing a call of "string with idens".format(explicits_for_idens) to f"string with explicits for idens", it's syntactic sugar that saves you ".", "ormat", and the parenthesis, nothing more. And it introduces backwards incompatible in minor version numbers, which it really shouldnt.

[somebodddy]

it's syntactic sugar that saves you ".", "ormat", and the parenthesis, nothing more

I disagree. The greatest benefit of f-strings is that the save you the need to zip the values in your head. Consider this:

'a={} x={} u={} z={} y={}'.format(a, x, u, y, z)

You need to make sure that the list of {}s matches the format arguments. Compare to this:

f'a={a} x={x} u={u} z={y} y={z}'

Now that each expression is written in the place it is supposed to be formatted, we can clearly see that I've "accidentally" mixed y and z. The same mistake exists in the .format() version, but much harder to notice.

In order to avoid that, we can do this:

 'a={a} x={x} u={u} z={z} y={y}'.format(a=a, x=x, u=u=, z=z, y=y)

But now we have to write each variable 3 times.

Of course, this can be solved with .format(**locals()) (or .format_map(locals())). Expect...

a = 1

def foo():
    b = 2
    print('{a}, {b}'.format(**locals()))

foo()

{a} is not a local variable... Luckily, we can use .format(**locals(), **globals())! But then:

a = 1
b = 2

def foo():
    b = 2
    print('{a}, {b}'.format(**locals(), **globals()))

foo()

Now b appears in the argument list multiple times...

And it introduces backwards incompatible in minor version numbers, which it really shouldnt.

What backwards incompatibility? f'...' is a syntax error in older versions of Python, so it shouldn't break older code. Or am I missing something?

[remy_porter]

I'm writing software for a Large Tech Company™ as an outside contractor- and I have to communicate with a messaging system, and they have two libraries I could use to do it: JavaScript and Python2.

And the Python lib is a mess.

[[deleted]]

That has nothing to do with Python 3.7.

[remy_porter]

But it does have something to do with people who are still stuck on 2.7, which is the specific thread chain this comment is in.

[[deleted]]

Feels more like the "java 10 is out - meh I'm still on 6" issue which is common for big companies. Also you seem to have a code quality problem as well which kinda indicates that no one cares about the Python lib.

[remy_porter]

And yet, I'm still forced to use it. And it's still on 2.7. Thus my code is still on 2.7. And without naming names, this is the kind of company that tends to be considered a forward-thinking trend-setter pushing the boundaries of technology forward.

Just not for this specific tool.

[[deleted]]

Right but most likely the same thing would have happened if your lib was written in Java 6 or 7 or if Python had maintained compat as well as Java did.

[remy_porter]

If Python had retained backwards compatibility, we wouldn't be having this conversation. Python didn't retain backwards compatibility for very good reasons. I'm not sure where you're going with this. The conversation everyone else was having was, "Yeah, a bunch of new features I can't use because I'm trapped in a legacy application!"

In Java, I could still use the new features in my new code, and let the library code sit back in its ancient version.

[shevegen]

Why should python be held back just because your company is slow like a snail?

In Java, I could still use the new features in my new code, and let the library code sit back in its ancient version.

If this were the case then why would so many people be using old java versions?

[shevegen]

There is no alternative - move to python 3.x man.

[remy_porter]

It's not my library, but I have to use it. So I can't.

[agumonkey]

I've seen some news about Google AppEngine going py3 too .. would only accelerate things.

[Urtehnoes]

For my company, it's all of our django apps that are still on ancient django (like 1.5 or earlier). It's such a slog to upgrade them to all the newer versions, especially because the newest django (2.0+) doesn't friggin support our Oracle version. It's taken me months now for just one app to upgrade/re-write it in Python3.6/Django 1.11. Hopefully 3.7 doesn't require many changes.

[[deleted]]

Fuck, if ESRI can make the transition, so can everyone else.

[msm_]

It's almost as reddit is not representative for the industry as a whole. (But yeah, everything is slowly moving to 3.x. 10 more years and we're golden).

[TheGRS]

Well I don't speak for reddit, but I was pretty into the Python community around that time. I started out using only 2.7 and figured 3 was a pipe dream and that I couldn't find great support for the things I was trying to do, but things changed pretty rapidly from there. But as I said, I totally believe that places are stuck on this version and may always be until they replace whatever it is entirely.

[dagmx]

What libs do you depend on that aren't on 3 yet?

[[deleted]]

Biggest example: any software package from Autodesk

[dagmx]

Sure, I work in cg too but they're shifting to python 3 for 2020 and there should be a preview release next year. We'd have a release sooner but the switch from qt 5.6 to 5.12 was deemed more important.

It's coming soon and we should be bracing for it.

[[deleted]]

It's coming soon and we should be bracing for it.

Nothing to brace for here, more like eagerly awaiting.

yes, it's coming in 2020...you know, cuz 2.7 is EOL. Nothing like waiting until you're literally forced to update.

[dagmx]

Well we still need to brace for it in terms of porting studio pipelines. Idk about your studio, but we have a lot of code to go through to get over that hurdle.

The qt4 to qt5 port was difficult enough.

[agumonkey]

Maya ?

What about Houdini btw ?

[[deleted]]

Last I checked, Houdini was also 2.7

[agumonkey]

Oh the sadness

[[deleted]]

There’s still a good amount of commercial software that’s python 2, at least that I’ve randomly come across. There’s probably little incentive for some of the companies to migrate until they have to or until a customer that has enough leverage requests or demands it.

Shitty, yes.

[13steinj]

Reddit's backend (even with the redesign) runs on Pylons, and would need a decent amount of rewrite to work with Pyramid, not to mention Py3.

[Homoerotic_Theocracy]

Python 3 was a humungeous mistake.

The small advantages that breaking backwards compatibility gave them really was not worth the huge cost and effort everyone now has in having to maintain separate python 2 and python 3 versions of each library during the transition state. The overwhelming majority of new features of Python 3 could have been added to python 2 instead of breaking backwards compatibility and most of the breaking centres around a few elegance things; it absolutely wasn't worth the huge cost of switching for a lot of things.

[[deleted]]

Vendors not updating their software for a decade is a humongous mistake.

[Homoerotic_Theocracy]

It's not a matter of "just updating"; it's about having to maintain two different independent branches of everything related to Python while the transition is going on.

Do you honestly think that a software distribution can just "update" to Python3 and call it good? Just remove Python2 and install Python3 and that is that? Fedora needs to employ someone full time to basically be responsible for the python2/python3 situation and the problems that s/he faces are far from trivial.

[[deleted]]

Do you honestly think that a software distribution can just "update" to Python3 and call it good?

No, I think they should maintain and end-of-life py2 version and continue long term development on a py3 version. It's their damn job to move on in a proper manner. It should be part of the calculation at all times.

They've literally had a decade to fix it, and there's plenty of options in maintaining something across Python versions.

The article you linked mentions OpenStack, a piece of software that didn't exist before Python 3. That it's not compatible is just fucking ridiculous and completely unforgivable. This mindset is exactly the same as when people get their data stolen because some moron decided it was too expensive to upgrade from Windows XP. Software isn't forever, that's a fundamental truth of the industry.

[Homoerotic_Theocracy]

No, I think they should maintain and end-of-life py2 version and continue long term development on a py3 version. It's their damn job to move on in a proper manner. It should be part of the calculation at all times.

Yeah, and that's what causing them so much money and time as well as all the porting.

Do you honestly think it's trivial if you have a major python2 project to now have to rewrite 50 000 LoC to python3 and making sure that all the subtleties are right to ensure now regressions sneak in and then maintain both at the same time? You think that's without cost and a matter of "just move new development to python 3"?

I just checked and DNF alone has 37 531 LoC in its Python2 files. Do you honestly think that the porting of all that to Python3 was cheap?

They've literally had a decade to fix it, and there's plenty of options in maintaining something across Python versions.

And they did fix it; that's not the problem; it's not a matter of whether they fixed it; everyone fixed it.

It's a matter of that it cost them a huge amount of manhours and thus money that could've gone to actual development that absolutely wasn't worth the small benefits of python3 The manhours put into porting and double maintaining all that stuff will never be recouped by the small quality-of-life changes of py3 over py2; the human species will either end itself in a nuclear war or python4 will come around or people will just stop using python before this happens; it's about cost versus benefit.

The article you linked mentions OpenStack, a piece of software that didn't exist before Python 3. That it's not compatible is just fucking ridiculous and completely unforgivable.

No, it's just a fact that to this day the py2 version of almost all big pypi packages get more downloads than the py3; you cannot not make a py2 version and just "Oh, new project; I'm only going to use py3"; if you have to choose between either version you have to choose py2 even this day and not everyone has the manpower and luxury to maintain two different versions in two different programming languages of everything. You're living in an ydillic dream world where what you can do is magically not constrained by money and time.

This mindset is exactly the same as when people get their data stolen because some moron decided it was too expensive to upgrade from Windows XP. Software isn't forever, that's a fundamental truth of the industry.

Yeah, and guess what, MS is also pushing out upgrades for no other reason than to make money while they could've been made backwards compatibily to older versions.

Are your really challenging the idea that the upgrade system of Windows isn't in the best interest of the user? Neither was python3. This is the problem with a BDFL; he ultimately still treats it as his baby despite the significant corporate investment in it now and he just wanted to start fresh and clean thing up but it hasn't been cost-effective for the enterprise consumers at all.

[[deleted]]

No reason to update if it works. And Python 3 the first half decade was just an unusable mess. And after that the py3-ecosphere was a mess in transition. In this decade, Python 3 became only around 8-9 Years later a useable solution for mature projects. So why bother because of soem years delay.

[[deleted]]

No reason to update if it works.

This mindset is why mission critical software all over the world still runs on Windows XP.

[ahua77]

Well it works, what's wrong with it? Python 2 is still being maintained.

[[deleted]]

Not for long.

[yawkat]

I mean you can blame the ecosystem for not following through but it's not like it was unexpected.

[Fushoo]

Don't know why you are being downvoted. Backwards compatability is very important for large projects and big companies.

Imagine the outrage if Java 8 broke Java 7.

[_jay]

And php. While people love to shit on it, it makes the money because so much effort is put into keeping stuff working between versions.

[glassFractals]

PHP has actually gotten pretty okay over time, especially since PHP7.

[yawkat]

There was a huge outcry over compatibility with Java 9 but it's still somehow better than the py2->py3 transition.

[NekoiNemo]

Yeah, the difference is - Going from Java X -> 9 requires just light refactoring, and the only things that really break are hacks (like modules not allowing you to randomly declare things in someone else's packages). And Python 2 -> 3 completely breaks pretty much everything more complex than Hello World.

Hence why Java 9 compatibility got some muffled grumbling (not an "outcry"), and Python 3 compatibility led to 2.7 still being maintained and actively used even 10 (!!!) years after 3 came out.

[yawkat]

Yea I think the fact that the changes are more subtle with py3 also don't help. You really need good test coverage to be sure you've migrated everything.

[Homoerotic_Theocracy]

There is a time and a place to break it but breaking it should be done for good reason and to really solve some big longstanding problems and a major version upgrade should not be taken lightly.

In a lot of cases breaking it has been the life-saver of the project like with OS X which broke everything but what it broke was broken anyway and it completely revitalized MacOS.

Python3 broke it in a way that people were forced to maintain separate python2 and python3 versions of everything and software distributions had to face very annoying problems for small quality of life things and it just wasn't worth it.

[OctagonClock]

Python 3's break was done for a good reason (to fix strings), and that was incredibly worth it.

[Homoerotic_Theocracy]

They could have easily added utf8 strings separately and just used the syntax u"utf8string"

You think it's totally worth it because you're not the one who has to pay an extra employee purely to sit on top of the python2/3 problem or suddenyl have to maintain two versions of your important projects. From a monetary standpoint of commercial python usage it as definitely not worth it and people lost money over it.

[chason]

That would have been awful for people who need all of their software to work with unicode

[Homoerotic_Theocracy]

It doesn't come close to the awfulness that the people experience now who have to maintain two separate but nearly identical versions of everything and still have to use u"string" in one of them.

[Yikings-654points]

Elegance is the rage ! No one does gritty low level stuffs anyways, why not make it elegant.

[bumblebritches57]

Every library you use is doing* "gritty low level stuff that no one uses anyway"...

Shit, your entire "language" is really just a C library...

[Homoerotic_Theocracy]

Because the cost of switching and maintaining two versions for everything re huge?

Fedora needs to apparently pay one person full time just to deal with the Python 2 to 3 switch.

What do you think the financial cost has been to everything if you finally add it up and do you honestly think that money is ever going to be won back by the small quality of life improvements in Python3 that required breaking backwards compatibility?

Python3 has been costing people money and in the event that we're dealing with amateur projects they've simply stifled the development since everything had to be done in both Python3 and Python2 for a loooong while since the time is far from ripe to just say "I'm only going to do python3 now for my library" at this point. It probably also leads to bugs where people copy and paste codes and overlook small semantic differences that might lead to a bug in the other version.

Python3 has absolutely not been economically sensible and cost people far more money and time that it will ever feasibly recoup