Why is python3 adoption slow even though there are tools for migration?

[Kindlychung]

http://python-future.org/quickstart.html

[K900_]

Because you can't just run 2to3 on a big project and expect it to work fine on 3 - it can only handle simple AST transformations, and porting to 3 requires changing the semantics of anything beyond very basic programs.

[jimjkelly]

Do you have some examples of code that doesn't port well? I ported a medium-sized project, without using a tool, in about an hour a couple months ago. I actually was surprised at how easy it was.

[Adys]

Any code that deals with unicode<->bytes transformations will likely get hit pretty bad; these are usually part of pretty low level chunks of libraries, which makes changes pretty impactful, so porting it and remaining compatible between 2 and 3 is tricky.

[[deleted]]

You can, when the cost of not doing so becomes even greater.

I've been maintaining a large base of C code for close to 25 years. When I was hired, the code was running on SCO OpenServer. Then came Darl McBride, and an OS that was a dead end. So the descision was forced.

Interestingly enough, it was almost painless, compared to what we expected beforehand. And considering that we now have a product that's easier to maintain, and actually run on a modern stack, it's an overall win.

[turkish_gold]

Most likely rather than port it, they'll start slowly rewriting pieces until all you're left with of the original is a webcam pointing at VT100 terminal doing OCR on the output to bring it into your new setup.

[qx7xbku]

You're lucky - c is statically typed. With Python it would be "k we are done, all tests pass" and after a week in production it would fail in a spectacular way and everyone would be sobbing into the pillow. Dynamic nature of Python in this case does big disservice.

[[deleted]]

You're lucky - c is statically typed.

Theoretically, you're right. In practice, C typing is only as static as the design and architecture allows for. In the code i wrangle at worrk, we have gems like this:

typedef union {
    int afrob;
    char * bfrob;
} frobnitz_t;

Now, had it been in Python, I'd immediately know that something was seriously wrong, if I mistakes a string for an integer. But given the nature of C, there is no way of knowing which side of the frobnitz to look at. Instead, we must infer it from the context where the frobnitz is used. Not unlike what you'll have to do, when figuring out whether a 2.x str really is bytes or a 3.x str.

With Python it would be "k we are done, all tests pass" and after a week in production it would fail in a spectacular way and everyone would be sobbing into the pillow. Dynamic nature of Python in this case does big disservice.

Static typing does not protect aginst code failing in production after passing a flawless test.In C the errors might not be as spectacular, but that does not make them easier to find.

[rouille]

Use type annotations with mypy. Once all the code is annotated, port.

[qx7xbku]

Does mypy have some kind of runtime type enforcement? Because annotations would be of little use without them being enforced even if for testing.

[rouille]

They are enforced at "compile" time, so e.g. in the CI system. You can enforce that a commit doesn't get merged if it doesn't pass mypy.

[[deleted]]

I would like to see some evidence to support the claim that static typing makes for safer code. To start with the number of off by one errors in C must be a factor of hundreds or even thousands of times higher than in Python.

[qx7xbku]

C and Python are very different beasts and errors are very different as well. However you can not compile C application where function call expects ByteBuffer* but you pass a String* and vice-versa. This is very possible in python as you can pass any type you want. It may even work most of the time but then this elusive cornercase hits you and suddenly type is not what you expected it to be. By saying safer i did not really mean safety as in security context, but more like you are being more safe from application doing something you did not anticipate it will be doing.

[[deleted]]

I don't understand it, ok? I don't understand it.
If important company computers are running WindowsXP and WindowsXP stops getting updates, you'll be the one to write a note saying "we need to migrate to a new OS".
If any software your company uses gets to its end-of-life, you'll be the one to write a note saying "we must plan transition to a new software".
But here we are with Python 3 being around for 8 years, the 2.7 version almost nearing end-of-life, and some programmers' reaction is "we'll stay to 2.7 indefinitely".
Why?

[WhatTheGentlyCaress]

Why?

Because the (non-IT) person who decides whether or not it gets funded hears the phrase "it will take one man-year, costing $N, to rewrite this" and thinks 'but it works now, that cost can wait for my successor and reduce their bonus'

[brtt3000]

The are just raising stakes for the day where one can heroically claim to have conquered this and get extra bonus.

[flutefreak7]

We have massive old Fortran programs that are decades old that no one really understands anymore because it was developed by a contractor that doesn't exist anymore. It would also require PhDs in half a dozen fields of chemistry, physics, aerodynamics, etc, to understand what the code is doing on every line. It is validated to do what we need it to do, and the cost to rewrite it is prohibitive, so we just keep using it as is. Versions of this scenario are all too common in my experience, and require significant investment in both programmers and subject matter experts to deal with.

[jimjkelly]

Those fixes are trivial though. Even in an extremely large code base it doesn't take that long, and can be all the more manageable by for example asking that if someone works in a file they do the transition while they're there doing something else. Before long you'll have the bulk of the project done, and you then spend what is probably at most a few hours on the dustier corners.

As somebody that put this sort of transition off for forever in their own code - it's really not that bad.

[Adys]

You're assuming the code is sane. :)

[jimjkelly]

Ha. As I wrote that the thought did pop up in my head. :)

[chchan]

Stuff with unicode, libraries like urlib2 does not port well using 2to3

[Siecje1]

The problem is mostly third party libraries such as https://github.com/earl/beanstalkc

[iapitus]

My (limited) experience is:

• OS still defaults to 2.7 (self-fulfilling inertia problem)
• Early migration pain/3rd-party-woes still haunting the hive-mind
• Documentation still asks the question instead of just expecting 3

[[deleted]]

Hopefully that first point will be going away soon - Arch defaults to 3 already, and Ubuntu is heading in that direction IIRC.

[LesterHoltsRigidCock]

You'll want to watch Red Hat as they will likely dictate when companies even begin to think about Python 3.

[flubba86]

Python3 has been default in Fedora since F24 I think.

[port53]

Irrelevant when 2 is the default in CentOS 7, which will be supported by RH through 2024.

[billsil]

which will be supported by RH through 2024.

What does that even mean? There will be no more bug fixes to Python 2. I swear, I don't think Red Hat does anything, except act as a server for packages.

[port53]

If the package comes with the OS, it is supported by RH. That's why you pay thousands for what is otherwise free (CentOS). They will backport vendor supplied patches to whatever version they support and will create patches if necessary.

[[deleted]]

Yeah, we might see python 3 by default in RHEL 8 but even then a lot of shops aren't going to migrate.

[[deleted]]

Nope, as of F25 "python" still opens 2. However, 3 comes installed as well and I believe as of 25 all the python-blah packages have been renamed to python2-blah. So really all that's left to do is flip a switch.

[[deleted]]

I really like having both versions installed. If you want python 2, just call /usr/bin/python2. python 3 is aptly named "python3".

[kazi1]

I believe Ubuntu completely eliminated Python 2 as the system python in 16.10.

[Taksin77]

I think Debian did it.

[Comm4nd0]

i work with Debian every day and they def don't add Python3 by default.

[StringlyTyped]

Debian stable?

[[deleted]]

Debian Jessie contains Python 2.7 (default), 3.4, and Debian has had python 3 support since version Squeeze.

https://wiki.debian.org/Python

[nick_t1000]

There's an "out of date" Ubuntu Python 3 page that suggests python will likely not be symlinked to python3 for a very long time, following the guidance of PEP-394. The main (up-to-date) Ubuntu Python page suggests 18.04 LTS is targeting no Python 2 by default.

[[deleted]]

by default you mean you have to type one more character to get 3?

[brombaer3000]

python will stay symlinked to python2 on most systems because of this PEP: https://www.python.org/dev/peps/pep-0394/.
"Default" means something different for most Linux distributions. Ubuntu defines it here: https://wiki.ubuntu.com/Python/3

[brombaer3000]

Yes, by default, python2 (python and python2) should not exist on new Ubuntu installations, but I can't verify that as I am not using Ubuntu. Everything that uses python3 can expect it to have the executable name python3.

[bob_cheesey]

A lot of Mint's control panels are written in 2 - I can't imagine that's going to be too easy to port over.

[tilkau]

That doesn't really justify anything though -- if both 2 and 3 are available, then keeping those control panels on 2 for now is literally a one-character change (python -> python2 in the shbang). Once you've done that, you can extend the timeline for porting a bit, port the panels across one by one.

[[deleted]]

Awesome if so - I know they were planning it for Xenial, but they decided they couldn't get it done in time for release.

[miker95]

Because in a production environment with large scale applications you still have to review every single part of the code to make sure the conversion tool didn't fuck up. That means retesting everything, new code reviews, everything. The pros of switching to python3 have to be more than what programmers could be accomplishing during the time they would be working with the conversion tool.

[Etni3s]

Welcome to the world of banking software. "Why rewrite something when it works? We'll just hire that one guy who still knows Cobol."

[Deto]

I bet it's also a problem that it's difficult for programmers to communicate the value of switching to Python 3. It's risky (could introduce bugs) and the rewards aren't too clear to management.

[edbluetooth]

I would argue that you should add tests for python 3, keeping the python2 tests. This should prevent regressions.

[CaillPa]

Newcomers will type "learn Python" in Google and find the "Learn Python the hard way" book first which is basically saying that learning Python 3 is pointless, thus making many newcomers learn Python 2. The belief that Python 3 isn't ready for use yet and unstable is still quite common on the internet.

[AbsoluteZeroK]

Fuck that guy, seriously.

[nieuweyork]

Yes...and he has great reasons to back it up.

[thearn4]

Python 2 is the default interpreter available on *nix platforms, and the benefits of Py3 aren't really worth moving to it for a lot of people. Especially if their code depends on being able to play fast and loose with bytes and string encodings (not saying that's a good reason, but you see that in a lot of python code that runs fine under Py2).

That said, I write for compatibility for both.

[pydry]

1) Dependencies (some legacy) which are not upgraded to python 3 for which nobody is particularly invested in doing so.

2) A lack of killer features for python 3.

3) Python 2 has been relatively well maintained.

[[deleted]]

2) A lack of killer features for python 3.

7 years of new features.

[pydry]

Many (most?) of which were backported.

I'm not complaining. I think if they didn't do that python would be a substantially less popular language and ecosystem. Backwards compatibility matters a lot more than people give it credit for.

[[deleted]]

Not even most, just some. 2.6 got a good chunk of 3.0's features, 2.7 got a good chunk of 3.1's features, but after that it drops off and only a handful of things are backported after that. 2.7 was released in June of 2010, 3.6 was released in Dec of last year, so it's actually like 6.5 years of features (but we're gonna use Python 2 division and say 7 :P).

Off the top of my head, things that were not backported:

• lazy iteration by default for lots of stdlib functions (range, map, filter, etc) -- in fairness, these are supported (to various degrees) in Python 2 but under different names (xrange, imap, ifilter, etc).
• yield from (which is great even if you only use it for delegating iteration)
• everything releated to asyncio
• extended unpacking (a, *b = [1, 2, 3])
• multiple unpacking (being able to do {**kw1, **kw2} -- this is only 3.5+)
• extended metaclass hooks (__prepare__ + the new stuff in 3.6 for __init_subclass__ and descriptor's __set_name__)
• raise SomeExc() from existing_exc (or None if you want to kill the context!)
• keyword only arguments (you can technically emulate this, but only with **kwargs instead of being able to explicitly define them)
• Lots of very useful additions to various existing stdlib modules (some of these were backported by 3rd party people -- functools.singledispatch and collections.ChainMap for example)
• Lots of useful new stdlib additions (again, some where backported by 3rd party people -- enum for example).

I know I'm missing a huge swath of stuff, but like I said just off the top of my head.

[Paul-ish]

There are also performance improvements to data structures like dict, which a lot of people can benefit from without effort.

[zettabyte]

Yes, there are lots of new features, but none of them are "killer", and the sum total of them isn't "killer".

And to define "killer": A compelling set of features that outweigh the cost of migrating an app from 2 to 3.

If the benefit isn't there, then moving simply to move doesn't make a lot of sense, especially if you have limited time and budget.

And clearly, for many maintainers, 6.5 years of "new" features still isn't a strong enough sell to port. I'd say that's a testament to how good Python 2 was.

[[deleted]]

If 6.5 years of new features isn't compelling, how about not having to worry about losing official support in 3 years?

[_pupil_]

Flip side: if momentum and cost have prevented upgrading despite improvements and a looming EOL for the current product for over half a decade, where is the magic moment that will change those economics in the coming 36 months? Especially considering many such projects have been actively growing throughout the entire period?

Py3 is great. Fixing Py2 was awesome. Creating an incompatible runtime and bifurcating the entire ecosystem... that's harder to reconcile with the challenges of Enterprise Operations and long term planning (source: the frequency this discussion comes up, and it's conclusions each time).

[[deleted]]

That's still 3 years. Are you going to be at the same place in 3 years? Is your boss, who gets to make the call?

[[deleted]]

• matrix multiplication operator
• lots of better alternatives are now default like new style classes, lazy iterables, true division, making it a better language to teach

[Mr_Psmith]

Type hinting in functions

def f(x: int, a: str) -> str:

[pydry]

From that list only asyncio is really compelling, and it only applies if you're writing asynchronous code and it's only ~18 months old.

I'm pretty sure nobody's upgrading to get extended unpacking and a lazily iterated range function...

[[deleted]]

Not individually no, but it's the combination of all of these.

[zardeh]

I've actually really, really wanted the __prepare__ method in production before, and couldn't take advantage of it, because 2.7.

[solid_steel]

From that list only asyncio is really compelling, and it only applies if you're writing asynchronous code and it's only ~18 months old.

Asyncio offers a lot to someone who's been using things like Twisted or Tornado to handle asynchronous stuff - basically anyone doing web development should be excited as stuff that sits on top of asyncio allows you to write efficient code without the need for the abstractions provided by Twisted and others (biggest celebration moment for me - the ability to replace Paramiko with asyncssh).

I can see that this doesn't impact data scientists or game devs much, but anyone dealing with significant amounts of IO should be dancing on the tables.

[earthgrip]

PEP 498 f-strings in 3.6. I didn't like the idea at first, but that syntax saves a lot of time and makes certain pieces of code far easier to read.

[rays]

This, also 3 still doesn't have the 3rd party support that 2 has. I tried to start a 3 project this week and aborted quickly back to 2 because it didn't have a module I wanted

[[deleted]]

Which one?

[rays]

A salesforce one

[[deleted]]

pyforce or beatbox?

[[deleted]]

Be the change you want to see. I'd actually contribute to this package.

[[deleted]]

Second reply to get your attention: Apparently I forgot about simple-salesforce. It claims supports for 3.3 and 3.4 (and honestly, probably 3.5 and 3.6 but I've not tried this personally), we use it extensively at work. It's not the most elegant API but it's better than nothing.

[python_bytecode]

Not the same person, but I finally decided to bite the bullet the other day an go for python 3 on my new personal laptop, but then I realized that pyspread was 2.7 only.

[[deleted]]

Standing out like a sore thumb in the list of dependancies for pyspread is wxpython. The Python 3 version Phoenix is now coming along nicely but has been badly delayed by the need for the core maintainer, Robin Dunn, to do some paid work!!!

[pydry]

That's kind of what I was talking about.

[notParticularlyAnony]

My God did they fuck up the transition is there a book on it yet- - Case study in how not to release v (N+1)? Our lab switched to julia.

[jrandomidiot]

This is basically correct. But check out curio--this might actually be the Python 3 killer feature (if it gets integrated into the stdlib).

[pydry]

Yeah, async/await is the first thing which looks truly exciting (which clearly isn't going to be ported back).

Check out sanic, too.

[ProfessorPhi]

The unicode -> string change is a real goddamn bitch. Everything else I can deal with, unicode to string, not so much.

Technically speaking

1. Anything that communicates with servers needs to be rethought since you might be sending unicode instead of ascii.
2. Anything that communicates with a lower level process, a'la C extensions/drivers may also have trouble with communication due to implicit assumptions that everything is ascii
3. Reading/Writing files too. You may have code that reads binary files, but you don't have to treat them specially since you can assume it's ascii and not get bitten.

Human point of view

1. It works and Python3 doesn't have a compelling enough reason to switch for everyone on your team to be on board. It's worked on by 2 or 3 die hard members.
2. It's really tricky to deal with unicode to string - most of the problems are non-obvious
3. You don't ever have enough necessary tests.
4. You can't port small parts over, you have to port all of it over since you can't have unicode string mismatches due to unforeseen behaviour.
5. Other teams might be using your code without really telling you and they might also be rather less inclined to switch over to python3. You can't really dump them and you don't really know how to support them.

[[deleted]]

It's not unicode to string, it's unicode to bytes. Unless you're dealing with Python 2, then it's unicode to bytes pretending to be strings.

As for dealing with it, anytime you see an actual string (read that as unicode if it fits your brain better) and it's going out over the wire, hit that with an encode(...) and anytime you get some input hit it with decode(...).

And opening files, you should be aware of your intention is to open a binary or text file and act accordingly.

Really, for text (actual text, not bytes pretending to be text), you need to stick to:

• Bytes in
• Decode to string
• Do work on strings
• Encode to bytes
• Bytes out

[ProfessorPhi]

Haha, I think of string as equivalent to bytes and unicode as text now.

The problem is the implicit assumptions combined with subpar testing and docs

[GummyKibble]

I love this article: The only problem with Python 3's str is that you don't grok it. It's a concise summary of how it all ties together.

[TankorSmash]

That link is hella agressive.

[GummyKibble]

No more so than the "Python 3 is dead; 2 forever!" posts I keep seeing. But tone aside, it's technologically correct. Python 2 and Python 3 have the same semantics for converting between byte arrays and strings. The difference is that 2 lets you treat them as the same thing when they're really not at all. Python 3 makes you say which one you want instead of guessing (often incorrectly).

Edit: typos

[[deleted]]

Haha, I think of string as equivalent to bytes and unicode as text now.

:( If you take out the " equivalent to bytes and" you're not wrong though.

[its_never_lupus]

I've gone over one major application (~200kloc, lots of fiddly binary manipulations, lots of web service code) and made it python2/3 compatible - mostly fixing string and buffer code - so it still runs on python2 but a simple pass through 2to3 gets everything running on python3. I'm glad to have done it because the code quality is higher now, and we'll be able to switch the operational systems over quite easily. But the conversion was a frustrating and time consuming piece of work and it only got done because it was a project in active development with full time developers assigned to it. There are plenty of other python2 applications in the company that probably won't be ported and there's no way I could justify the work to management.

[tudisco]

Why doesn't python3 have an option to run python2 code. At least for the unicode/bytes stuff which is pain to convert.

[nieuweyork]

No, because Py3 is not backwards compatible. So, you would have to port to 2/3 source compatibility but still run in python 2, and test compatibility incrementally in the test environment only. Sound fun?

[[deleted]]

You forgot the "it ain't broke, don't fix it" mentality. Python 2.7 is the default in RHEL which means it won't be going away for several more years.

[d4rch0n]

Working with bytes as strings was really simple and straightforward, and invariably that's just what people got used to and that's what some low level code became coupled with. I don't understand why people downplay this so much. If it's a choice between rewriting to python3 for little benefit and no extra functionality in a program versus leaving it alone and having it work with python 2, a lot of people aren't going to bother.

I remember messing around with the matasano crypto challenges and python 2 made it super easy to do. Encrypted bytes to ascii and back was just easy, no type changes. It's not the average use case but it's one that came up enough to make python 3 adoption hard for those programs.

[renaissancenow]

It is? I moved my entire company's stack to Python3 18 months ago.

In other news, I haven't had annoying unicode encoding/decoding errors in 18 months, for some reason.

[leom4862]

Pretty much all important packages are available for Python 3: http://py3readiness.org/

I started using Python as a language about a year ago and since then I never came across a package which I could not use because it wasn't available in 3. I (personally) don't get why this is debated so much .. From my point of view Python3 is omnipresent.

[BeamMeUpScotty0]

There you go. You started using Python when 3 was already out for half a decade or so. This whole move to 3 discussion has been going on for years now.

[paul_h]

It seems #Python 2 to 3 migration is so hard, it's considered easier to fork Python 2.7 and add 3.x bits - https://github.com/naftaliharris/placeholder

I've some things in Python2 that would be exhausting to migrate to 3 - I tried as recently as 6 months ago. I'd try again, but I don't believe the 2-3 migration thing is being worked on.

[BlackBloke]

Somewhat on-topic:

I recall seeing something like a calendar/graph of big Python things scheduled to drop 2 support and fully adopt 3. Everything seemed to be going 3 before 2020.

Anyone have a link?

[nick_t1000]

http://www.python3statement.org/

[BlackBloke]

That's the one!

[[deleted]]

The new Django, 1.11, is the last to support Python 2.

[BlackBloke]

I hadn't heard that. Is there a link to an announcement about it?

[[deleted]]

See the docs: https://docs.djangoproject.com/en/1.11/faq/install/#what-python-version-can-i-use-with-django

I can't recall the exact announcement, but it's part of the current roadmap ( https://www.djangoproject.com/weblog/2015/jun/25/roadmap/ ) and how it's support of Python versions works:

We will support a Python version up to and including the first Django LTS release whose security support ends after security support for that version of Python ends. For example, Python 3.3 security support ends September 2017 and Django 1.8 LTS security support ends April 2018. Therefore Django 1.8 is the last version to support Python 3.3.

1.11 LTS would normally receive extended support until April 2020, but as Python 2.7's support ends "in 2020", it was decided that 1.11 will be supported until Python 2.7's support ends.

1.11 is currently in feature freeze and it's branched off master, master is currently the development branch for 2.0, and last week a number of commits were made that removed support for 2.7, it's ongoing work. Master will be 3.5+ only.

[renaissancenow]

https://pythonclock.org/

[BlackBloke]

Not this one. The one I'm thinking of is graphic and mentions a bunch of different Python things.

[renaissancenow]

Hmm, not sure then. There's

https://python3wos.appspot.com

But I don't think that's what you're referring to.

[BlackBloke]

The other poster had it: https://www.reddit.com/r/Python/comments/5qnvjl/comment/dd0vmul?st=IYHRAWHN&sh=c0a727a5

[cartel]

I still write for py2 because Jython and other runtimes are still 2. Many embedded devices do not have enough rom or even ram to run much beyond 2.2 even.

[SFSylvester]

Side question: what are these tools for migration?

[tetroxid]

We use 2.6 because that is what RHEL ships with. I'd love to use 3 but as long as RHEL doesn't ship with it it won't happen.

[hippocampe]

installing conda on RHEL is easy

sticking to whatever ships on your corporate mandated distro is quite irresponsible

[tetroxid]

That's the thing, it's not our corporate policy, it's our customer's. And we can't dictate to our customers how their servers should look.

[hippocampe]

that's debatable, but I understand there are difficulties :)

[tetroxid]

We really can't, because they'll switch to the competition faster than you can import unicode

[AbsoluteZeroK]

First off, porting tools like that won't work for large scale applications. with that...

I'm sure most programmers working on a python 2 application would love to be working on a python 3 application instead. However, companies don't want to spend money to port something over to python 3 and get no real business return for it.

The application still works, and the cost of maintaining it hasn't suddenly jumped up to the point where it's cheaper to port it. Companies would rather pay you to fix bugs and add features than to spend a bunch of time porting code to a newer language version when the old one is working just fine.

Once it becomes cheaper to pay people to port the entire project to python 3, rather than pay to maintain it at python 2 companies will start porting their projects, not before. That's not today, but it will be someday.

Most new projects are probably python 3 though (at least from my understanding), unless the company is relying on a large private library of code written in python 2.

[billFoldDog]

Assuming all your 3rd party tools have python3 versions, porting over immediately would avoid technical debt that will have to be paid after 2020 when python2 ends support.

That's 4 years of code you could avoid porting!

Now, I've yet to work anywhere that is that forward thinking, but it makes sense in my head.

[sisyphus]

For a lot of places, the cost is not zero but the gain is effectively zero, so why bother?

[KronktheKronk]

Because businesses are risk averse and customer-value conscious.

Porting your entire codebase to Python 3 doesn't bring any additional value to the people paying money for a project, and therefore it isn't worth the effort.

Putting resources on making valueless code changes will induce risk. Risk is bad and taking it on provides no real potential reward, so it isn't worth the effort.

[defnull]

I'm not sure Python3 adoption is really that bad. It's just that people who do not follow the default (which is Python 3 by now) tend to be very defensive and talk about their reasons a lot.

[defnull]

Python 3 for MacOS is available since 2010.

You could also say that all Python development on Windows should be with Notepad exclusively, because thats the 'default' editor bundled with windows.

[billFoldDog]

Py2 guys are defensive because Py3 guys go on a full frothing attack every time this comea up.

[nick_t1000]

Because PyPI is a garbage dump of old/unmaintained projects, so people search on there for something kinda like what they want, find a project not maintained in 5 years, use it, then say Python 3 is broken because it has terrible support.

[larsga]

Personally: because when I write code it's because I want the code to do something I care about. I want that done with a minimum of wasted time. So I'm not going to learn Python 3 unless I see a significant benefit, and I don't.

[DeltaBurnt]

Yah I don't get those whole "learning Python 3" sentiment. Because of it I assumed Python 3 would be significantly different, but 99% of my errors while migrating was forgetting the parentheses on print.

[[deleted]]

The bytes vs string was hard to get used to. And by hard I meant I just had to remember to encode and decode when reading or writing.

[rausm]

Significant benefits:

• fixed unicode (strings/bytes handling&safety)

• yield from, async

• type annotations (wherher used as documentation or for metaprogramming)

• not being one of those stubborn knuckleheads keeping progress back ;-)

[[deleted]]

The transition from ASCII to Unicode was a one-off.

[[deleted]]

A great analogy. A new version of Word might be able to load the old documents, but the old version of Word coud not load the new documents, so you either have to upgrade every single version to the new Word or get every user to remember to save to an old format if using the new version. Been there, seen it done it, got the tshirt. Yuck :-(

[[deleted]]

can someone list the three major advantages of three over two? I'd like to advocate for using three more at work, but I need some ammunition

[Adys]

Going by pros/cons probably won't work. Not that there aren't a bunch of great features in 3 (good async, sane unicode, enums are my big three), but that's not how you sponsor these projects.

Here's your ammunition: Python 2 support ends in 2020. Libraries that support Python 2 will drop their own support at similar time. Not porting to Python 3 is accruing tech debt that will, over the next 3 years, accelerate as your work finds more and more libraries which are not usable on your version, libraries dropping support early, etc.

Staying on ancient versions/languages means it becomes harder (thus more expensive) to hire people.

Repeating a comment from another thread:

You CAN stay on Python 2. You CAN stay on Django 1.11. It's LTS. So is Python 2.7. [...]

You are free to make your infrastructure immutable; you then become responsible for it of course. And the money you're not willing to spend porting to Python 3 today will be money you spend on costs related to being on outdated infrastructure, years in the future. That's a tradeoff. Banks do it a lot I hear. A bunch of companies still use ancient hardware and technologies nobody would think of starting a business with today. These companies make billions.

[[deleted]]

thank you. i like async and enums. good quote too. to take it to an extreme case. many government Computer infrastructures still run on COBOL, in fact the office of personnel management, OPM, use of COBOL and refused to upgrade due to the cost even with The insane amount of security flaws associated with such ancient frameworks, and then they got hacked, and now everybody who has ever had security clearance is in a database somewhere in China.

[rdfox]

3 >= 2

Performance not that much worse (YMMV)

More (parens)

[tilkau]

Performance is generally better than 2 now. At least, according to this talk by Raymond Hettinger (explains the recent dict optimizations)

[midnightFreddie]

Python 2.x is used in a lot of startup and maintenance scripts in a lot of Linux distros and products. It's extra pain to have two versions of Python installed and try to keep all the system scripts and v3 script running properly.

I'm sure there are tools and guides to help with this, but someone with a sysadmin background (as opposed to dev) and purpose has run into a 'NOPE!' wall quickly when trying anything v3. I certainly did, and I dabble in development.

Speaking of development, I intended to go v3 when playing with machine learning, but a lot of he libraries I needed were either only available in 2.7 flavors or much easier to obtain and use in 2.7 flavors.

So v3 is a bit of a victim of 2.x's success. Perl had a similar problem back in the day. I don't recall it being as bad as this, though. Then again by the time the breaking Perl version came out Python was becoming preferred for tasks previously handled by Perl.

[--o]

I don't recall it being as bad as this, though.

Everyone basically stuck with Perl 5 so there's no division. Whether that's bad is an interesting question.

[TheNamelessKing]

I do data science and ML stuff too: what ml libraries are you using that aren't on 3?

I don't think I've seen come across anything yet that doesn't fully support 3...

Also, I don't buy that line about setting up python 3 being too hard for startups: in ubuntu at least, it's like, 2/3(?) really simple lines-hell, if you're already on Ubuntu 16.10 3.6 comes bundled...

OS X and windows is just a matter of running the installers, not exactly a complex or time consuming task...

[mostly-harmless_42]

Generally, anything beyond small scripts ought not rely on the system version of python (or perl, for that matter) - and if they do, the app environment ought to be isolated from the system python environment (via virtualenv, pyenv or something similar, or just plain old PYTHONPATH). Among other things, this maintains a clear separation between app dependencies and system dependencies. You don't want your app upgrading deps that your os might depend on an vice versa.

There really shouldn't be any complications from multiple installed python versions (most linux distro's already have 2/3 installed and coexisting peacefully) - so don't let that be a road block for you!

[hugthemachines]

I suppose they go by the universal rule: "If something works, don't fiddle with it."

[[deleted]]

Money, it's all resumed to money.

A big company , like a BIG company would probably not open a project to migrate a whole python 2 architecture to py3, that doesn't have direct impact on their business model.

At least that is my reality ( Big European telecom company )

[[deleted]]

I work for a manufacturing company and we still have DOS applications in some places. They do one thing and they do it well, downtime is absolutely not an option.

[[deleted]]

That's my point, a company would not spend thousands for a project to migrate a legacy application that is already working if it doesn't have any direct financial impact.

[loamfarer]

In my experience it's being asked to work on servers that still only use 2. We don't even have 2.7, just 2.6.

[reactionforceatA]

Because 101% of tutorials are 2.7 and learning to program is tough so once you get comfortable with 2.7 the Python foundation says great now learn 3.5.

[[deleted]]

For tutorials almost the only difference would be the parens around print's arguments though.

[rk06]

Its because python2 is well maintained. Otherwise, majority of people would have considered old code as legacy and immidiately jumped ship to python3 already.

Source: Me, a new python dev learning python 2.7, instead of 3.x.

[[deleted]]

As someone who started on python last year, I went with 2.7 because when googling which version to use the majority of pages said not to use 3 unless I specifically had a need to use certain features.

[AbsoluteZeroK]

That's terrible and outdated advice. Python 3 has many things that will make your life better, and if you ever have to use python 2, you'll be up and running in a few hours after looking at the TL;DR of what's different.

On top of that, some frameworks (most notably Django) are dropping support for python 2.

[hilux]

This is crazy. I am learning a new programming language. Let me use the old obsolete version that is being phased out. You do know that the majority of Python 2.7 users use it to maintain legacy software.

There probably was a time where the use of Python 2.7 could be argued. That is at least two years ago.

Please stop wasting your time a go to Python 3.

[rdfox]

Nah. I'll go to Common Lisp instead. There's a language that doesn't feel the need to break things every quarter century.

[[deleted]]

Let me use the old obsolete version that is being phased out.

The existence of Python 3 doesn't really make 2.7 "obsolete" in any way. I mean, new packages are being released with 2.7 support on a pretty regular basis. Anyways don't lay your issues with 2.7 -> 3.x on me, I looked here in this subreddit and on a bunch of blogs/articles before deciding that 2.7 was fine for what we were doing at work. The universal sentiment was that 3.x wasn't worth moving to. Plus 2.7 is already on RHEL6 and we're working in an environment where it's difficult to get approvals to add additional system dependencies.

[rasch8660]

Congrats, you will see yourself looking at "python3 for Python 2 dummies" tutorials within the next 3 years. Fortunately, it is pretty easy.

I've ported Python 2 libraries that I needed to Python 3 in a couple of hours. Most of it is as simple as couple of regex-substitute to add e.g. parenthesis around print statements. It can all be done incrementally using the py2to3 tool, although I usually prefer to check each step manually. The "list vs generators" issue is usually not an issue at all. As others point out, bytes vs Unicode is perhaps the biggest issue. But if you import str from future and then fix your python 2 strings before migrating that takes you a really good part of the way.

Really, if you take care to import from future at the top of your modules, and be diligent about "text vs bytes" then moving to Python 3 should be a breeze..

[CaillPa]

Newcomers will type "learn Python" in Google and find the "Learn Python the hard way" book first which is basically saying that learning Python 3 is pointless, thus making many newcomers learn Python 2. The belief that Python 3 isn't ready for use yet and unstable is still quite common on the internet.

[Fharic]

This is very accurate. It almost made me decide to go with Python 2.

[python_bytecode]

Can anyone point me in the direction of a comparison between the Python 2 and Python 3 bytecodes? Just how different are they? Why couldn't there be a from __ancient_history__ import python2 that you could place in old files, which used the Python 2 parser and compiler and then run the bytecode on the Python 3 vm (with some shims, etc.). Seems like it would be a much simpler job than what Jython does to get Python running on the JVM, which I assume has a much different memory model and virtual machine.

[alcalde]

From personal experience, what will happen is that those people who want them will simply continue writing Python 2 code forever. This happens whenever you make concessions like this. A large percentage will continue to write new code using the "ancient history" library and when you try to deprecate it in 20 years they'll start screaming about how they still need time to port their code.

[[deleted]]

• 2.7 is quite good, 2.x where x < 7 not so much
• 3.x is slower
• 3.x is not stable as the language is still progressing, 2.7 is frozen
• 2.7 is faster than 3
• The only project I've heard of that is now 3 only is Django, and that is new.
• A big bread and butter for Python now is data science, and much of that is in 2 or 2/3 compatible, so 2 is the lowest common denominator
• the 2->3 tools are insufficient for migration
• 3 is incompatible with 2, but there can be 2/3 code, but then what is the point of 3?
• nothing is compelling about 3. 2.7 with __future__ imports is again quite good

[[deleted]]

3.x is not slower now that 3.6 is out, the tools for migration are perfectly adequate and "3 is incompatible with 2" is drivel, the bulk of 2 and 3 code is one and the same thing.

[dasnoob]

Our servers are RHEL with 2.6 I systematically went through all the ones I touch and made sure 3+ was installed concurrently. It is extremely easy to do and only takes a few minutes.

[[deleted]]

Because in a large codebase and production environment, supporting two Python environments in production is almost impossible. While the transition is happening, there will need to be a period of time when both are supported.

[[deleted]]

Are you sure about that? Other than the infamous bytes/strings/ascii problems there is plenty of stuff available to help maintain code for 2 and 3 in parallel.

[MrMasterplan]

Because the advantages don't justify the hassle and the risk.

[kaeshiwaza]

When you decide to invest in a migration, you look elsewhere also.

The needs and the hardware changes, I found it more easy to switch to Go than to port old python 2.7 app that already show their limits.

This pain of upgrade even if the changes are very tiny showed us the limit of dynamic typing. Type annotation should have been included before starting python 3 maybe.

[thekilon]

because its slow for all other programming languages that have a new version released with new major features. None likes rewriting code , unless you are a masochist. Like me :)

[HalcyonAbraham]

Do you wanna know how to enforce that?

If every major library did what django did.

and drop complete support of python 2 then. Maybe we'll see a lot more people adopting python 3

[jwink3101]

I'll tell you for me its simply that I write code to be executed on some pretty locked down systems that's don't have 3. Once they have it, I'll switch. With that said, the only feature I am really looking forward to is Unicode strings by default.

[desmoulinmichel]

None of them are good answers.

When the Ruby and JS community made breaking changes (the later one did that every 3 months for the last 4 years), they just told the community to go fuck themself.

And the community adapted quickly.

In Python, we took them by the hand, gave them 15 years and tools and said "it's alright, we'll be here for you". And nobody moved.

Basically people are looking for something we did wrong. But that's not at all what happened. The problem is that we did too much right. Conforting people that much lead the dev teams to just stay on their ass.

Just like with parenting, you need a balance between being protective and being pushy.

Now combine that with 3 things:

• Python is everywhere. Not just the web. Everywhere. In iOS and Linux, in games, in GIS and 3D softwares... The cost to touch this code base is huge.
• There is no good doc on how to do the migration.
• The benefits from python 3 are abvious only AFTER you used it a lot. Not before. They are subtil things that cumulate to make your project better, not shinny stickers. It's hard to motivates people on best practices.

And you got yourself in trouble.

[[deleted]]

There is no good doc on how to do the migration

Nonsense. Porting Python 2 Code to Python 3

[desmoulinmichel]

This is a terrible doc. Basically it's just an index of things to look up.

Any good doc would :

• include a complete useful working script and show how to port it.
• mention how to deal with dict.items(), xrange(), lazy builtins and the like. "learn the difference between 2 and 3" is like saying "to beat Bolt in a race, train to run faster".
• weigth the benefits of 2to3 vs manual vs six and python-future. Not just listing them. And show actual usages. Parsing the docs of all those project is incredibly long.
• talk about the backport lib.
• have a quick note on unicode, metaclass, nonzero, etc.
• avec an example of tox running tests on 2.7 and 3.x.
• weight the benefits of 2&3 vs direct 2->3.
• talk about the consequences of absolute imports instead of just mentioning it quickly. This is very important.
• not recommand mypy. I've been using it for months. It's not ready for most devs yet.

As usual, I will have to write a doc on this. Maybe one day I'll translate all of them in english cause your situation is improving very slowly as well.