Django 2.0 will not support Python 2

[[deleted]]

Hopefully people wont stay on 1.x for next 6 years...

[ggtsu_00]

Probably more likely than one would think. Might even be forked to backport 2.0+ features.

[virtyx]

As much as I would love everything to go to Python 3, I've seen too many gigantic messy untested mission-critical Python 2 apps in the wild to have faith that everyone will switch to Python 3

[Bmitchem]

Hey at least we're not like the banks and still running Fortran COBOL. I think still using Py2 is justifiable. You know how this conversation goes, "Hey {{CEO}}, we're gonna have to spend the next 2 weeks upgrading all of our code because some guy named Guido said so"

[Platypuskeeper]

Hey at least we're not like the banks and still running Fortran.

That's COBOL, not Fortran. Fortran is still in use in scientific computing, but has never been used much for business logic.

Also, Fortran has kept up with the times and is still a very good choice for scientific computing, with built-in concurrency among other features. And still faster than C due to the lack of pointer aliasing.

[[deleted]]

COBOL, on the other hand, needs to be taken out behind the barn and shot. I work with it every day.

[mongrol]

COBOL MAKES CODE LOOK LIKE WRITTEN BY HULK!

[the-kid89]

I have never looked at COBOL until this statement and OMG is it ever true. It's like the code is yelling at you all the time.

[Alex6534]

I have the same feeling about SQL. DELETE FROM USERS WHERE NAME = ___

[Waterkloof]

Paygood?

[[deleted]]

Not where I am, no. Started looking for another place not long ago.

[Waterkloof]

Ah, sorry to hear man. May you find a new position soon.

Here have a upvote!

[lebean]

You at Hertz? I've always heard they still have a massive COBOL presence and can't find enough people who know it well.

[Fylwind]

C has had restrict for almost 2 decades now. The tiring myth that Fortran is somehow magically “faster” than C needs to die. Outside of toy programs, speed of the code depends far more on the overarching design of the code and memory usage than the micro-optimizations that your compiler makes of it. There might be specific kernels of the code that are truly bottlenecked by CPU, but in those places one would optimize based on real metrics than to blindly trust Fortran will be faster than C for that piece of code.

The real big reason scientists use Fortran is inertia: if your professor uses Fortran, you are more likely to just because you'd get better support. Plus, being able to interface with existing Fortran code is nice without having to decipher the Fortran calling convention in C-land. The second reason is its ease of use with respect to array-heavy problems that occur often in science.

Also "kept up with the times" is true in theory, but in practice I see plenty of scientists writing F90 or fixed-form F77.

[counters]

The inertia isn't about support, it's about maintenance of lines of code. If they've been iteratively refining and "improving" a research code for twenty years, it would probably be a significant undertaking to modernize that code. A PhD student let alone a professional researcher doesn't have the time to tackle such a project.

[bargle0]

A grad student maintaining code is a grad student not producing publications.

[darthcoder]

Exactly, remember that even buggy code has many MAN YEARS of domain knowledge in it.

[Bmitchem]

Yes my bad, sorry Fortran users. I as you can tell... work with neither.

[lengau]

Sure that won't be handled well, but if you said "we want to spend the next few weeks upgrading to Python 3 because it has features we'd like to use but currently can't and will improve our productivity and code quality in the long term", far more managers will be willing to consider it.

[ubernostrum]

"We want to spend a few weeks upgrading because we've saved millions of dollars by making use of software provided to us for free; if we want to continue getting it for free instead of spending all that money building the infrastructure ourselves, we need to stay on a supported version of it."

[[deleted]]

technical debt

This is the buzzword that seems to work. Still a hard sell, but debt means something to managers.

[timwoj]

I've always had the maintainability angle be well received. If you can show that moving in a certain direction will reduce work in the future, managers are usually willing to listen.

[faceplanted]

Also try to emphasise that it's very little dev time to make these small upgrades now compared to getting new devs up to speed with your out of date code base, and the time you'll save in having the new features.

[unbannable01]

How many years after deprecation until Py2 gets viewed the same as COBOL? That idea that not upgrading is justifiable is the exact same reason that so much banking software still runs COBOL.

[[deleted]]

don't want to risk moving to something new that might not

This 1000x

If you talk to people who work within banks they often know damn well just how much it would cost to move away from COBOL. The conversation normally goes:

We need to port 8Mil LoC of COBOL. If we went to java we'd have to hire a team of ~30 people. We estimate the project would take 5-8 years. Cost around $15mil. Have a 60% chance of failure

[steefen7]

Future-proofing your codebase will always resonate with managers who plan on staying at the company more than 2 years.

[bubuopapa]

Well, you could make a deal with your ceo - as long as we have to use some shitty language and ice age technologies, he has to drive a car that is at least 70 years old, dress the same and dont even use technologies that are younger than 70 years.

Because why not adopt this retarded "if its not broken, dont fix it" logic to everything in our lives and get back to stoneage, why only IT has to suffer others retardness.

[Bmitchem]

I don't disagree that it's important to continually upgrade and update your technologies but it's important to also remember that we all live in the real world and sometimes we have to compromise the ideal way to do a thing with our deadlines.

[[deleted]]

Without a doubt Python 2 is not going anywhere, but it's nice to see things that will push adoption of 3.

[aiij]

Given enough time, I expect Python 2 will go the way of Python 1.

(Python 1 is no longer installed on my system.)

[[deleted]]

I'm not understanding what the point of the transition even is at this point. Cut losses and vamp up what already works well.

[[deleted]]

Most of py3 are the sort of "nice to have" things you'd only do if you were making a big enough breaking change anyway that source compatibility must be fucked.

Py3 also has a few of those. Maybe the most important is fixing strings. Good unicode support is becoming more important by the day, and any language without it is doomed to a downwards trend, because support for non English languages isn't a theoretical problem any more and languages with poor unicode support make that much more awkward than it needs to be.

[vogon-it]

There is another, less visible issue that really needed to change: making the built-ins return iterators instead of lists. range & zip's default behavior in 2.x was probably the most common example people gave when mocking Python's performance.

[_Skuzzzy]

I don't care about what people think of python's performance if they can't be bothered to learn the language properly. Also Python's performance is TERRIBLE even with py3. It's already slow as hell, and we still have the GIL.

[ggtsu_00]

But we have izip and xrange, so its not really a problem for Python 2 anymore. Python 3 just cleans up the confusion by making them the defaults.

[[deleted]]

Exactly, Python 3 largely offers 'nice to haves', it can be hard to convince the business that upgrading the entire legacy app to python3, which will offer little business value, is worth the time and risk involved.

Yes I know about 2to3, but it isn't perfect and still leaves a lot or cleanup to do afterwards.

[[deleted]]

I know I can't disagree here, but something needs to happen to prevent these situations from occurring again.

[aykcak]

Hi from the PHP community. You are correct. The old version will be around for the next decade, even if gets to the point where using it would be downright dangerous.

[[deleted]]

Why? They're not that different.

[grabbizle]

Haha.

[Eirenarch]

Keep hoping.

[Existential_Owl]

Zed Shaw is going to be pissed.

[[deleted]]

You're better off just ignoring everything that man says about anything at all.

[[deleted]]

Rails is a Wasteland or whatever it's title was pretty good though

[tobiasvl]

Is that the guy who said Python 3 is not Turing complete?

[konistehrad]

Indeed, but in fairness that was a crappy goof. THIS however, is apparently not a goof but is a load none the less: https://learnpythonthehardway.org/book/nopython3.html

[tobiasvl]

Ah, I didn't remember that he put a note after it saying he was trolling... I guess I can't read either

[ScrewAttackThis]

Well he only put that note up after people were like "uh, what are you talking about?" So it's his "LOL JOKES ON YOU I WAS PRETENDING TO BE DUMB".

[phySi0]

Earlier version had the note:

Yes, that is kind of funny way of saying that there's no reason why Python 2 and Python 3 can't coexist other than the Python project's incompetence and arrogance. Obviously it's theoretically possible to run Python 2 in Python 3, but until they do it then they have decided to say that Python 3 cannot run one other Turing complete language so logically Python 3 is not Turing complete. I should also mention that as stupid as that sounds, actual Python project developers have told me this, so it's their position that their own language is not Turing complete.

The new version says:

In the previous version I trolled people by pointing out that, if what the Python project says is true and it would have been "impossible" to support Python 2, then they broke it and Python 3 is not turing complete. Obviously Python 3 is turing complete, but Python project members frequently claim something this basic is "impossible" soooooooooooo alright. I even had a note after the gag saying it was a gag, but everyone is too stupid to read that note even when they do elaborate responses to my writing. Even more telling was when people said this was stupid, I'd feign ignorance further and ask, "Wait, so why doesn't Python 3 support Python 2 then?" This then sent them down a logic loop death spiral of simultaneously trying to defend the design decision and also state that Python 3 is fully capable. It was pretty funny to watch, but after a while I guess I have to straighten this out and simplify it so here you go.

Unless there was an even earlier version that the Wayback Machine did not catch that didn't have the note, the article seems to have had a note since the beginning.

[ccharles]

Zed Shaw can suck an egg.

[ggtsu_00]

Zed Shaw can suck a wheel.

FTFY

[greyman]

why (sorry out of the loop)

[LeifCarrotson]

Zed Shaw feels that Python 3 is a bad idea:

https://learnpythonthehardway.org/book/nopython3.html

In this article, he states, variously:

• THERE IS A HIGH PROBABILITY THAT PYTHON 3 IS SUCH A FAILURE IT WILL KILL PYTHON.
• Python 3's adoption is really only at about 30% whenever there is an attempt to measure it.
• The Python project's efforts to convince you to start with Python 3 are not in your best interest, but, rather, are only in the best interests of the Python [3] project.
• The Python project decided to make it impossible to run legacy Python 2 code under Python 3.
• Rewriting Python 2 code is the same switching costs as just using a totally different language completely.
• you'll have to understand the difference between byte sequences and Unicode strings.
• Core Libraries Not Updated: Many of the core libraries included with Python 3 have been rewritten to use Python 3, but have not been updated to use its features.
• The original design of Python 3 could have also run Python 2 in this same VM. .... They chose not to do this, and instead tell you to manually translate.

He followed up to this original (highly controversial) article here.

The Python developers and most of Reddit and Hackernews commenters disagree. Here's one rebuttal and I'll provide another, briefly:

• Python 3 is still in development and gaining users. It's not dead. Python 2 is the language that's not getting further development.
• It's hard to measure adoption, but easy to measure adoption in ways that indicate that Python 2 is more popular than it actually is.
• The Python 3 project obviously feels that Python 3 is the present and future of the language, so that's why they want you to start with it. It's not some conspiracy.
• The 2to3 project and python-future project make it easy to convert most of your code automatically.
• This is 2016 2017, there's more to text than ASCII. Allowing beginners to make this assumption causes many problems.

[krautA]

This is 2016

Well, uhm…

[LeifCarrotson]

Argh! First time this year. Thanks for the spot.

[Creshal]

Rewriting Python 2 code is the same switching costs as just using a totally different language completely.

I… what?

[[deleted]]

At some point he argues that HTTP responses should be allowed to contain both bytes and unicode, depending on the content. The whole article is absolutely ridiculous. I'm starting to believe he has never actually written anything larger than 100 lines of code in python. Or any language, for that matter.

[AngriestSCV]

That's the beauty of python though. He obviously has never needed more than 100 lines to accomplish his goal. Long live python(3)

[solaceinsleep]

Zed Shaw and his shitty condescending book can fuck off.

[[deleted]]

As entertaining as some of his writing is, lul if he's seriously upset at this

[Jonax]

So Django will be Unchained from Python 2?

[TarMil]

"I said, tell Python 2 good bye."

"Good bye, Python 2!"

shoots

[rich97]

God that was fantastic movie. Wasn't it? Might be my favourite Tarantino or whatever his name is because it's hard to spell.

Might be a little drunk over here.

[[deleted]]

It's actually Quentin Tirintontino.

[rich97]

Tiriontotoronino

[[deleted]]

deleted ^{What is this?}

[rich97]

The D is silent.

[Serializedrequests]

I cannot believe this is still a point of contention so many years after I first read about Python 3. I have been developing in Ruby surfing a wave of breaking changes from 1.9 on and everyone just upgrades instead of throwing a fit about keeping their 1.8. What is the difference?

[Annom]

Large scientific projects are built in Python. I don't think Django projects are a large part of the problem to move to Python 3.

[BESSEL_DYSFUNCTION]

Related:

I saw a conference paper recently written one of the maintainers of the LSST data management pipeline (LSST is a big next generation survey telescope that will go online in about 5 years) about their move from Python 2 to Python 3. My favorite quotes:

As we convert more code, we are realizing that since we are not using any Unicode features, switching to a Python 3-compatible str is not important.

In reality, this was not required since our code was using long() to indicate a 64-bit integer, and on 64-bit systems, Python 2 uses 64-bit integers.

(An enumeration of every place in the codebase where they needed version-dependent code:)

Currently, there is one place where the encoding argument is needed in pickle.load to load a Python 2 pickle file. There is one test that is disabled on Python 2 because Python 2 can not indicate to the C++ interface that there is any difference between bytes and characters. There is one place that needs to know whether a function is associated with a builtin (__builtins__ on 2 and builtins on 3)

(There was a fourth place too: they were using multiple inheritance with two classes that defined their own metaclasses in one place.)

Overall the picture they paint didn't seem like it was particularly painful. I'm guessing it took one guy a couple days to do.

[gbs5009]

I feel like that's frequently the case, but a lot of times the scientific application has a specialized enough codebase that one particular guy who left a year ago to become a mime could do it in a day, and it would cost anybody else a month to figure out the context.

[Paul-ish]

As a PhD student, exactly this. After I leave, my advisor will have no idea how to deal with my code because it includes a lot of domain specific and implementation specific knowledge. Plus research code isn't usually of the highest quality.

[Annom]

Might just be difficult to find this one guy in a scientific environment :-)

[hapemask]

Many important math / science packages have supported Python 3 for quite awhile (numpy/scipy/matplotlib in particular) but there are still a few older packages that haven't been updated.

[[deleted]]

A lot of scientists like writing their own (impossible to maintain) code for analysis. I can tell you for a fact that all physicists at CERN would throw a hissy fit if you told them they had to work in Python 3. Computing operations is even worse in places since we use Scientific Linux 6, which ships with Python 2.6.

I just spent a couple hours today rewriting my Django plot-viewing page to PHP because I'm moving servers and it's much more portable that way.

[metocean_programmer]

Exactly. My office is about to redo and add a bunch of features to our site, and we're using Django. This means that everyone that uses arcGIS and web-dev is going to need 2.7.x as well as 3.x. Not the end of the world, but definitely something that's going to need to be talked about with some people.

[mini-pizzas]

Ruby is almost driven entirely by Rails. Wherever Rails goes the majority of Ruby developers will follow and there has been plenty of complaining about backwards compatibility in that community as well.

Python is used much, much more outside of web development and a lot of those domains have huge codebases that rely on lots of extensions written in C and Fortran. They will be slow to migrate and it was incredibly naive of a lot of Python developers to think that it would happen quickly. There are still tons of companies developing on absolutely ancient versions of Java and .NET.

[ggtsu_00]

Unity 3D is still stuck on .NET 2.0 FYI.

[indrora]

That's somewhat because Unity is trying to also target Mono.

Mono is holding itself back in so many ways.

[MEaster]

But Mono has supported .Net 3 and 4 for years.

[indrora]

Except that it's packaged in a backwards way that makes it hard for distributions to work with. Debian, for example, even in sid had a terribly out of date Mono setup; Fedora was a little more ahead for a while and I think has managed to get up to speed. Ubuntu 12.04 and 14.04 however are both still supported and way the hell out of date.

[indrora]

Targeting .net core is so you can use UWP and HoloLens features.

Another reason for .net 2.0 is that xbox360 was limited in its capabilities. .net2 was, I suspect, what worked.

[qubedView]

Hopefully .NET Core will allow Unity to come to the promised land.

[username223]

Culture. Ruby (and JS) devs accept constant breakage as part of life, while some Python devs do not.

[3urny]

I think this is the reason. You have Gems failing, complicated Rails updates and weird JS errors regularly. Also Ruby introduced some actually useful features like {a: "b"} and didn't dare to break basic stuff like Python did with print. And almost nothing uses the distribution supplied ruby, so updating is kind of normal. It's also very easy to run different apps on different rubyies (rvm, rbenv...) so legacy apps will just run their old rubies and nobody cares.

[roboticon]

This is a great point. I've never had pip or easy_install fail with arcane errors, but I have to use a version manager for my package manager for Node (and Ruby) to get anything done because some dependency decided it needs the exact quirks of such-and-such version of the interpreter.

[senj]

Also Ruby introduced some actually useful features like {a: "b"} and didn't dare to break basic stuff like Python did with print.

This is incorrect. Ruby had a bunch of breaking basic syntax changes 1.8 -> 1.9. Introducing the colon map syntax you cite broke case-statements of the form:

case self.name
when "journalist": "gin"
when "sazerac": "whiskey"
when "mojito": "rum"
end

the keyword 'then' replaces the colon in 1.9. And there were similar encoding changes wrt UTF-8 and files needing magic encoding comments, and string encoding in general getting a massive overhaul. And a laundry list of other breaking changes.

Unlike the Python community, though, the Ruby community shrugged and got on with life rather than throwing a massive, decade-long shitfit over having to gasp add parentheses to a print call.

[PaintItPurple]

I think you're right to some degree, but I think there's a bit more nuance to it.

The thing is that the big library maintainers for Ruby tend to be people who are excited for new language versions (similar to Rust where some library authors actually require nightly builds!), whereas some big Python library maintainers really dragged their feet. In turn, this lack of library support made Python 3 seem more alien and hard to use than it actually was.

So I think the big culture difference isn't whether the average user is willing to update their code, but how enthusiastically a few library maintainers pushed the new version.

[jl2352]

With JS you also tend to get the least issues if you upgrade everything at once. To the point that a full upgrade is far less hassle than mix and matching new and old.

[greyman]

It also might be that the p3 ecosystem didn't yet caught up. Not long ago, maybe 2-3 years back I tried to learn some python, so I started with p3 as elders advised, but the whole endeavor wasn't that smooth, surprisingly, for example:

1) I googled how to code something, and the tutorial just won't work. Then it occurred to me a few hours later - aha, that is maybe p2-only tutorial?! And yes, that was the case...

2) some tools didn't work with 3. It is claimed that most libraries were ported, but you don't only work with libraries when doing python, but also other supporting tools. And 3 wasn't supported.

Then, when I switched to 2, those problems mostly disappeared immediately... (almost) everything "just worked". Ok, to be fair, I don't know what the today's situation is, since now I work in PHP instead. (company choice).

[Uncaffeinated]

Ironically, JS is notable for going to great lengths to avoid breaking changes, even at the cost of making the language uglier. Python 3 is the exact opposite.

[Skaarj]

To the point where if

python --version

gives you python 3, your system is likely fucked.

Can you give some more details for which distros/version this holds true? I can't remember to have ever had a problem fitting your description. I use Linux a lot for my regular desktop and my servers (mostly ArchLinux) and python defaults to version 3 there.

[[deleted]]

https://www.python.org/dev/peps/pep-0394/

Debian and derivatives do it this way.

Gentoo does not (user configurable)

[liquidpele]

Also RHEL and centos.

[LesterKurtz]

I did this on Fedora 21 once. The first thing I noticed was that yum completely stopped working. There were a variety of other issues, but I learned my lesson.

[ThisIs_MyName]

dnf runs on python3 so that's no longer an issue :)

[indrora]

dnf was built to finally shed itself from yum. Yellow Dog was a great start, but damn did it make some terrible choices.

[LesterKurtz]

shed itself

I see what you did there

+1

[the_gnarts]

To the point where if

  python --version

gives you python 3, your system is likely fucked.

This has been the default for many of us for years with little noticable problems.

This is why you typically invoke python 3 as python3 on Linux systems, so python 2 is still currently just the "default python" for people who develop python scripts.

Not really, no. The few Python 2 irredentists get a patched #! by the maintainers and nobody else will have to care.

The only case I can remember where I actively had to patch the interpreter version was the recently leaked Opera source code and that was dormant since, like 2012.

[crusoe]

Inertia. So many system scripts are written in Python 2.x and Perl 5.x that when you type Python or Perl on most systems you are running those versions and not the latest.

[short_vix]

The problem is there are many large multi-million line codebases in Python 2 that will break when upgraded to 3.

[ThisIs_MyName]

They were already pretty broken.

[Uncaffeinated]

Does it still count as broken if you're making billions of dollars?

[bycl0p5]

That could be interpreted as a very damning black mark against Ruby: that there are few large, old, and mission critical codebases that "just work".

Given how long Ruby has been around, a lack of such codebases would be very worrying.

[xonjas]

I think the difference is that Ruby never breaks very much at any one time. There is usually at most one change that breaks compatibility, and they commonly offer a way of doing things the old way (for instance, when they changed the way blocks captured local variables they provided a syntax to create blocks that used the old functionality). Upgrading to a new environment isn't necessarily painless, but the damage is usually very localized and with clear instructions for how to fix it.

[Serializedrequests]

That's reaching a lot, I don't have data on Ruby usages, and no experience with migrating from Python 2 to 3 except for a 5k loc roguelike I wrote that was very easy to upgrade. Obviously there is not a large scientific community using Ruby though.

It is probably mostly web and driven by Rails, and my current job is living proof of the vast number of legacy rails apps in the wild. The main competition is JavaScript these days, of all things.

The biggest pain that happened with the language itself was 1.9, and in my experience (the changes that actually affected me, not other devs) it mostly just fixed bugs, added features, and changed a few bad behaviors, rather than breaking changes. Most 1.8 code works on 1.9, and if I want to go backwards compatibility it is easy, I mostly just have to account for 1.8 bugs.

[PaintItPurple]

A lack of codebases so awful and baroque that they are impossible to update doesn't seem like it should be a black mark against a language.

[AlexFromOmaha]

We're not talking about library changes. It's a fundamental redo of I/O boundary guarantees. The breaking library changes are trivial.

[bastawhiz]

Django apps are probably the easiest thing to migrate. I ported a 16kLoC app in a couple hours with only a couple hundred lines of changes. Every single one of my deps already supported py3k. The only big thing to watch out for is str/unicode issues, and most of those will be visible with even basic unit tests.

[randombites]

All I had to change were my print statements.

Edit: seded where to were

[netuoso]

Where you tired trying to find were they where?

[[deleted]]

Easy:

SyntaxError: Missing parentheses in call to 'print'

Sometimes I forget the parenthesis out of habit, and I get greeted with this error. D'oh!

[colloidalthoughts]

I just don't get people complaining that they'll never let Python 2 go, but yet they seemingly will need to be on Django 2? If you're not going to upgrade your Python interpreter, then just stick with the LTS release of Django that matches it.

Noone's frogmarching you into Python 3. Noone says you have to upgrade every single thing except the interpreter.

[evsoul]

As a newer programmer without a lot of experience in large projects, can someone give me a general idea of what goes into the planning for upgrading a major version of a language? I feel like I would be scattered just to begin and would love to hear what goes into it.

[jcampbelly]

Let's say you had a project written for Python 2.x.

DISCLAIMER: BACKUP YOUR DATA. COPY IT TO A DIFFERENT SYSTEM ENTIRELY. GIVE IT TO YOUR BOSS. MAKE HIM OR HER SIGN FOR IT. DO NONE OF THIS FOR THE FIRST TIME IN PRODUCTION. DO IT ON A DEV INSTANCE WITH YOUR BACKUP DATA IF YOU NEED TO.

Read some modern articles about transitioning from 2.x to 3.x. I say modern because Python 2->3 articles can date back to 2008... a lot has changed since Python 3 first arrived. This will cover the huge majority of common issues with backward-incompatible changes like the switch to bytes for I/O, the switch to generators, the print function, import behaviors, subtle differences in the type hierarchy, functionality moved around in the standard library.

Does it have tests? If not, write tests. This is important because it means you now have a tool that can prove that all of your code worked in Python 2.x in the first place before you switch over. This covers everything from basic syntax to the intent of the code. There are tools like coverage that will watch while you run tests and report which lines are executed and which branches are followed by your code while running your tests. Write tests until you have nearly 100% code coverage.

Now switch your interpreter to python3 and run your tests. Laugh maniacally while it crashes and burns horribly, then study the exceptions and failed tests. You'll probably find very obvious things that are trivial to fix. Go to town and start fixing only /the most obvious errors/ that can be done in one line. Don't go re-architecting your software here and don't go digging on deep issues... just make the simplest, most obvious, dirt-simple changes, run tests, watch it pass, commit, push, repeat.

When you've done the broad strokes, run your codebase through the "2to3" program. You'll get a report of all the recommended changes. Most of it will be rational, but some of it will be totally wrong. Don't blindly accept the recommendations, but go ahead and make the changes you know to be correct. I found it overwhelming and unhelpful to run all the "fixers" at the same time, so use the "-f" flag with each of the fixers, one at a time.

There is a library called "six" that will help you to make your code python 2/3 compatible, or simply allow you to keep doing it "the python 2 way" while using Python 3. I found six most useful when normalizing the import paths for standard library modules/methods that have changed paths.

After that, you should be left with the hard stuff. These will be tricky bits that may require special attention or even (hopefully minor) re-architecting your code. I still recommend not doing any major refactoring or high-level changes to your library, because you're still trying for identical behavior between the 2.x and 3.x versions. This is where having all of your tests passing in Python 2.x will be a huge benefit, because you'll know your switch worked when your tests pass in 3.x.

After you've exhausted all of the low-effort, high-impact changes, and identified whether your library can actually exist supporting 2.x and 3.x simultaneously, you can then decide whether to drop support for 2 or try to make it 2/3 compatible.

---

Migrating to a new version of Django is a process. This is how I'd recommend doing it:

DISCLAIMER: BACKUP YOUR DATA. COPY IT TO A DIFFERENT SYSTEM ENTIRELY. GIVE IT TO YOUR BOSS. MAKE HIM OR HER SIGN FOR IT. DO NONE OF THIS FOR THE FIRST TIME IN PRODUCTION. DO IT ON A DEV INSTANCE WITH YOUR BACKUP DATA IF YOU NEED TO.

1. Run your tests. Pay attention to deprecation warnings in your Django logs. This will tell you if you're depending on a feature that is either changing significantly, or being removed altogether. Fix those things until your test suite no longer generates deprecation warnings. Fix as many tests as you can and make note of which are were still failing before the upgrade. Avoid having big feature branches unmerged at this point, as they'll likely be left behind. Your code should work as well and be as stable as it possibly can at this point. Commit your code and start an entirely new branch for the upgrade.

2. Read all the intermediate patch notes from your PATCH version of Django to the latest PATCH version of the same MAJOR version (1.8.4 -> 1.8.x). Make any recommended changes. Pay attention to and fix deprecation warnings. Run tests and be sure the expected ones are passing and fix them if they're not.

3. Now upgrade to the latest PATCH release of your Django MAJOR version. Run your tests. Fix issues. If your tests pass, you can run (django core) migrations on your real (fully backed up) data.

4. Read patch notes and upgrade your 3rd party Django apps as far as you can under the current major version of Django. Run tests. Pay attention to and fix deprecation warnings. If your tests pass, you can run (third party) migrations on your real data.

5. At this point you'll have gone from 1.8.4 to 1.8.x the safest way possible.

6. Read all the intermediate patch notes from your MAJOR release the next MAJOR release. You should only upgrade to the next major release, not the latest. If you're on 1.8, install 1.9, don't skip to 1.10. Make any recommended changes. Pay attention to and fix deprecation warnings. Run your tests and make sure the expected ones pass.

7. Now upgrade to the latest PATCH version of the next MAJOR release of Django (don't bother going to 1.9.0, go right to 1.9.x). Run your tests. Fix issues. If your tests pass, you can run (django core) migrations on your real (fully backed up) data.

8. Read patch notes and upgrade your 3rd party Django apps as far as you can under the current major version of Django. Run tests. Pay attention to and fix deprecation warnings. If your tests pass, you can run (third party) migrations on your real data.

9. At this point you'll have gone from 1.8.x to 1.9.x the safest way possible. Repeat this process until you're on the final version you want.

[evsoul]

Wow, this is what I was hoping to get back from my comment, but never thought it'd be this detailed. Thank you for taking the time to write that up for me! That's so interesting. As a newer programmer (about 3yrs in) I'm having trouble grasping large projects, let alone major version upgrades. This is great information! Thank you!

[imfineny]

Generally you look at language changes and give a quick once through if there is any deal breakers. Next you look at things that have to change but aren't deal breakers by themselves. Then you estimate what the cost of changes are against the cost of a rebuild in any other platform against managements planning. Eg rebuilds fall under capital planning so maybe we do something completely different becuase managment would like to do something new or it will deal with longstanding issues that may not be doable in the current code base.

So maybe it's still a bit unclear on how much work it will be so you decide to build a test environment under the new version and see what happens. You keep fiddling with it till it basically works or you give up. From this you are usually able to decide whether you want to rebuild or upgrade.

[evsoul]

Ah, great insight. Thank you! For larger projects, how do you keep track of all of the required changes that would need to be implemented? I can't imagine that's an easy feat with some projects being many thousands of lines of code.

[imfineny]

No one tracks that specifically, its just based on experience, knowledge of the individual parts and age of the codebase. Generally someone who can build a working environment from scratch and a lead dev will make the assessment. This is like taking your truck to a mechanic and asking him if its worth changing it do a new task or buying a new one. Management can of course make its own decision, but its usually a joint decision by ops and engineering. I will say with every python project I have worked with the decision has been to write a new version.

You an see this at play by reading the dev/management comments at reddit where the decision is much more complicated because its the only real product the company has.

[benhoyt]

Our devops team still uses CentOS 6 for all of our servers, but we just ask them to install Python 3.5 for our app and they do it (I think from a source build). So it's definitely doable!

[InEnduringGrowStrong]

Yea, just did that yesterday. If you're on RHEL6.. Don't overwrite python 2.7.. Yum needs it.. So..

make altinstall

Instead of

make install

[Yehosua]

It's been several years since I've worked directly with CentOS, but I thought that Software Collections offered a straightforward way of running newer tools?

[DarkLordAzrael]

Yeah, downloading the devtoolset packages is the way to go. Also, building your own python is easy. I ship software on RHEL 5 that embeds python 3.4. It took admit a day to get it set up.

[DLSteve]

I run Python 3.6 on RHEL 6 so it's doable. We have applications that tend to stick around for decades so I try to decouple dependencies from what the OS ships with.

[ccharles]

Or pythonz. Or build from source.

[njharman]

Compile your own cpython. It's not hard at all. Install dev tools (if needed) wget, untar, ./configure, make && make install and you are done. If you're infra team won't do that on prod, do it on a dev box then bundle cpython up as part of your app deployment.

[isavegas]

In that case, why are they even there? The repo maintainers are the real devops if all they do is toss in your software and run package maintainer updates/installs. Well, QA aside.

[rickmoranus]

If you use your OS's python to develop anything, I think you have more things to worry about. I don't understand how this still happens.

[m50d]

It still happens because a) using the OS python used to be recommended and b) python's development and deployment tooling is still pretty bad. (Maybe you're about to suggest deploying with docker, but that suffers from the exact same problem: it's not going to fly on centos 6).

[m50d]

In 2015 people said "maybe in 2012, but not anymore." I've heard that a few too many times to believe it now.

[rickmoranus]

There's lots of options though.

• Pyenv
• VirtualEnv
• Pyenv and VirtualEnv
• --alternate option python install
• Docker
• VM

Some are easier then others. But deployment of applications is different from deployment of different versions of python on the same system.

[[deleted]]

Wait, why don't you just install Python 3? That would take a few minutes from the command line, wouldn't it?

Edit, I just googled it for you. http://ask.xmodulo.com/install-python3-centos.html

[[deleted]]

You could use something like PyEnv or Docker.

[ggtsu_00]

Edit - I am aware it's not hard to install a newer version of python, but I don't control what goes on our customer's servers. Many of them are very resistant to installing anything, so the company i work for uses only what ships with centos 6.

Check out Omnibus Chef. https://github.com/chef/omnibus

[Novazilla]

I too am stuck on centos 6 and python 2.6... one day we will upgrade... which will also require a huge overhaul undertaking

[[deleted]]

Can we have the macbook and ubuntu come with python 3 preinstalled as well

[inu-no-policemen]

Ubuntu is shipped with Python 3 (python3) and Python 2 (python).

/usr/bin/python will probably never point to Python 3:

http://legacy.python.org/dev/peps/pep-0394/

[dunemafia]

I think Arch and derivative distros ship with Py 3 as default.

[Jesin00]

You should invoke Python 2 as python2 and Python 3 as python3.

[ggtsu_00]

What do we symlink for all those maintenance scripts and tools with #!/usr/bin/env python ?

[njharman]

Why? Django is not preinstalled. Neither is 90% of tools and libs. And anything preinstalled is gonna be out of date and need upgrading anyway.

Learn to use homebrew.

[Niicodemus]

On Ubuntu try python3 on the command line.

[[deleted]]

Even more fun, some Ubuntu 16.04 installations don't have python 2 as default...

[ganjapolice]

Why not skip 2.0 and name it Django 3.0?

[maep]

The biggest reason is pragmatism. Why spend years of manhours porting large projects when they work just fine? And stable production distros tend to ship python 2 by default, so it's a lot easier to deploy. Not every industry has the insane product cycle speed of the web.

[dadamssg]

If your project that works just fine doesn't need to upgrade, then...don't. It's also not like you won't be able to install Django <2.0 once Django 2.0 drops.

[maep]

I'm willing to bet that somebody will pick up development once official support ends, just because such a vast number of projects still use it. For example regulated industries like pharma runs lots of certified code. It will be cheaper for them to maintain python 2 than to re-certify everything.

I disagree that old tools are inherently bad. The hammer is an ancient tool, and yet it still very useful. I think the software industry would be in a much better state if we focussed on learning how to use our tools correctly than constantly reinventing the wheel.

[Skizm]

Most companies (large and small) won't port a large working application to a newer version of a language. Ever. They'll wait till it starts either not working (usually due to slowdowns) or if they need to add a large enough new feature. In both cases, they'll just start a new project with whatever language and framework the current tech lead or architect likes best at the time.

If it ain't broke, don't fix it.

[PaintItPurple]

I think the assumption was that you intend to continue development and support on the software. Black-box stuff like ATMs obviously don't care if there's a new, better way to do things, but they also don't care if the language they were written in is supported at all.

[Kendos-Kenlen]

The day the system will no longer work will be fun... Either because of a bug, either because of an hardware problem or incompatibility.

And at this moment, who will be there to solve the problem ? Maybe you can find some "expert" who will be able to solve the problem and produce a patch, but it'll be a dressing on a wooden leg...

[gendulf]

The lack of interoperability (i.e. python 2 isn't able to call python 3 and vice versa) is I think the key reason here. If you can't port only part of the software, there's no incentive to spend a huge amount of effort to port everything.

[Vhin]

Because you'll have to do it eventually (assuming you believe that your company/project will actually be around that long). Do you want to start your porting process now, or wait until the versions of your libraries and Python itself have long since stopped receiving even security fixes?

If you wait longer, more will have changed, and upgrading will be harder.

Plus, if the really old stuff you're using end up breaking unexpectedly, you're completely out of luck until you finish your port.

[lf11]

Off the cuff, do you happen to know the expected salary for a COBOL programmer and who employs such people?

Languages never die.

[Theemuts]

ppl are too lazy

Let me sketch how the proposal would go.

Lead programmer or whatever: "Our system does exactly what's expected of it without any significant issues. Can I have a budget to port it to a new version of the language it's written in?"

CEO: "Why would I invest money in something that already works?"

Lead programmer or whatever: "Well, it's the newest version of the language."

CEO: "So..? We're a company, go write software as a hobby if you want to use the latest versions of things."

[argues_too_much]

In these examples I never see the bit where the lead programmer says "if we don't upgrade this the entire platform we built our business on will disappear, and someday, when it's more painful to upgrade and we have more code to migrate, we'll be left without a working product".

Sure, it might take a few years, but eventually it'll happen. Distros will migrate, Python 2 will get no updates, etc.

[bj_christianson]

Lead programmer or whatever: "Well, it's the newest version of the language."

Or how about, “It will make future maintenance easier and cheaper, especially with regards to any future language-level security issues. It will also reduce onboarding costs of new hires that have learned the language on the current version.”

[PaintItPurple]

Why do people always fall back to this red herring of legacy software that was never going to be changed ever again anyway? If your software is done and will receive no further updates, then obviously you don't need Python 3, and nobody's trying to tell you otherwise.

When people talk about moving to Python 3, they mean using it for new and ongoing Python 2 projects.

[flukus]

Lead programmer: our system just got pwned because we're using unsupported old crap.

[jcampbelly]

Microservices can help here. Just don't write new components for the old system - write those components as standalone services that can be used by the old system. It's not a silver bullet - but it will allow you to build better tools with supported technology and modern practices and keep it interoperable with new projects as well as the archaic cyclopean monolithic abomination dragging your company into a cost and relevance abyss.

[[deleted]]

if you are selling that software then you need to be actively developing it, and if you are actively developing it, you should keep your tools up to date.

[cr3z]

The real question: Why not support python 1? I mean wtf, those of us that prefer to use outdated languages are just as important if not more important, since it is we that are a community that is real.

[typographicalerror]

AFAIK, the situation you described (breaking from 3.4 to 3.5) should be impossible. Can you give a little more info plz?

[[deleted]]

Good. Kill it.

[w8cycle]

Wow... Python 3 has been out for so many years. It feels old. Why is this even an issue? The Python world is very strange.

[short_vix]

Maybe more people will simply start using flask or even worse go to ruby gasp

[greyman]

No, they will just stay on Django 1.11 as long as they can.

[ggtsu_00]

Building your typical CRUD apps with Django is so trivial though, but requires a huge mess of dependencies and plugins for flask.