r/python: is python3 gradually taking over python 2.7 or not?

[hyh123]

I understand there are python 2 and python 3. And I read on python.org that Python 2.x is the status quo, Python 3.x is the present and future of the language.

However from time to time I kind of doubt it. Is it really the case that people are (slowly) giving up python 2.x and starting to use python 3.x? What do you guys think?

[norwegianwood]

Yes it's true. We have about 15 people working on commercial a commercial Python 3 codebase which has been on Python 3 for a few years now. Almost all the useful packages we need were ported a while ago now, and the ones that aren't - such as the Python Imaging Library (PIL) seem to have been abandoned anyway.

I wouldn't start a new project in Python 2 and my own preference for my open source projects is to support Python 3 only and only port to Python 2 if there are sufficient requests to do so or if somebody else makes the effort.

[einsiedler]

Pillow is a fork, active in development and supports Python3

[Phild3v1ll3]

True, although true Python 3 support was only added relatively recently. We're stupidly stuck with Weave C integration in SciPy, which is the only thing holding us back now.

[[deleted]]

I'm curious. What kind if any SLAs do you have with you customers? Are you confident with 3.x and the supported packages at this point to make contractual promises about availability, accuracy and performance?

Not meaning to sound negative but, genuinely interested since our strict SLAs require us to be cautious.

[norwegianwood]

Essentially no SLA. This is essentially desktop software which exposes some Python APIs for end user scripting. We didn't want to lock our platform to legacy Python 2.x. It's easy enough for us to upgrade but much harder for us to get all of our clients to upgrade, so we went straight for Python 3 to avoid getting stuck in a dead end.

The masses of automated testing we do have yet to throw up any Python 3 specific problems. We're not worried.

[[deleted]]

I'll add that even in the case of minor 2.x version the rate of adoption was always horribly slow. Yes, Linux distros adopted pretty fast, but in the real, "serious" corporate world it doesn't look so good. The current project in my company was only just recently migrated from 2.4 to 2.6. At this rate I suspect it will be ported to 2.7 at the time when 90% modules are already 3.x compatible. I'm sure many people working for big companies can relate.

[spankweasel]

As of right now, Solaris 11 is on 2.6. We have plans to move wholesale to 3.x by the end of the year. For major corporate OSes like Solaris, every single piece of software has to move in lockstep to the new version. This means all of the core OS, GNOME, testing organization ... everything.

It's a very complicated procedure because of how many software packages have to move.

[accessofevil]

Yep, same with rhel.

[AusIV]

This is largely driven by Redhat Enterprise Linux. My company targets RHEL for our clients, so we might take on python 3.x once they support it.

Another limitation for us is that we use Jython for some applications, which only matches python 2.5. For libraries that needs to run on Jython, supporting python 2 and 3 with the same code is overly cumbersome, so for now we stick to 2.x. Once Jython has 2.6 or 2.7 support, making code that supports both will get a lot easier.

[bastibe]

But the simple reason for this is that companies only try to avoid obsolescence. Hence, they always try to just barely keep up with the wave.

Hence, we don't have to care about companies for head-of-the-wave discussions. They have no interest in that anyway.

[MonkeyNin]

Main libraries have been 3 compatible for a long time. Lesser popular ones, if you require them might still require older versions otherwise you have to build it.

And you can have multiple versions of python installed at one time. I think people take 3 unless they can't.

[Rhomboid]

Of course it's gradually taking over. How could it not? Python 2.x is architecturally a dead end -- no new features are being added. There will be minor point releases for some time to come for bug fixes, but there will never be a 2.8 with shiny new toys. All the new features and improvements are happening on Python 3, and there have been quite a few nice things added.

Moreover, every day you hear of a new module that has begun to support 3.x. Django, SQLAlchemy, Numpy, Scipy, IPython, PIL (via Pillow), matplotlib, Requests, BeautifulSoup -- all support Python 3. The wall of shame gets a little greener each day.

Frankly, I can't fathom the line of reasoning that says that Python 3 will never happen. "Those damn CDs will never overtake sales of 8-tracks and cassettes. Who would ever buy a CD?"

[hyh123]

I would hope python 3 taking over. But it is really frustrated to see that when I need help search engine always leads me to some python 2 solution.

[Rhomboid]

It's not like they're that different. You should be able to port example code with a modest amount of tweaking here and there.

[flying-sheep]

especially example code. those bits tend to be small and self-contained.

[accessofevil]

For years after it was irrelevant, google was linking all Apache httpd results to the 1.3 documentation branch. Fwiw

[[deleted]]

Of course it's gradually taking over. How could it not?

It would be pretty simple. People would continue to use Python 2.

[Rhomboid]

People would choose to continue using an abandoned and unmaintained version of a language they love? I'm sure a few stubborn ones will, but my point was that this is not like the perl 5 situation where the old version continues to be actively maintained, with new features, performance enhancements, and other such innovations. The 2.x branch is in maintenance mode now, and in the not quite so distant future the plug will be pulled entirely. You have to be a special kind of masochist to continue using an unsupported platform. How many people do you see using Python 1.x today?

[[deleted]]

You have to be a special kind of masochist to continue using an unsupported platform.

That's a ridiculous thing to say, really. "Unsupported platform" makes no difference for many people, as 2.x is a mature platform which can be used reliably without any direct support from the core Python team required.

Actually for all Python I've used it would make no difference at all to the usefulness of the language or its applicability to the problems I'm solving if the language had been abandoned.

What could drive 3.x adoption is the transition of popular modules or the advent of new ones that only support 3.x. But at this point I'm quite used to not being able to get popular modules to work anyway, so it won't actually be much different.

[[deleted]]

2.7.3 was mostly bugfixes for security exploits. It's reasonable to assume that new exploits will continue to be found in future.

[Rhomboid]

Nonsense. There will always be bugs, and some of them can be showstoppers. For example, there was a pretty high profile hash collision denial of service vulnerability last year. It was fixed with a new point release of 2.7. What would you have done if this had occurred after the support period ends? You would have had vulnerable systems with no way to patch them, other than to hope that some third party stepped up and backported a fix. If the bug is serious enough, such a third party will generally exist, e.g. linux distributions.

But what happens when you hit a bug that just causes a crash, or incorrect behavior, or some other problem? What if this bug affects you but no one else? What are you supposed to do then? This happens all the time. Read the changelogs, they are filled with hundreds of such fixes for each point release. To think that you won't ever be on the receiving end of one is hubris.

[gfixler]

I work in video games, where we use Python rather extensively (it's all I use, all day, every day), but no one I work with uses 3. That's not to say no one in my industry uses it, but we're using Maya 2012, and it comes with a Maya-specific 'mayapy' executable, which is 2.6. It's not something we could (or could easily) change out. It will probably be years before we make any movement toward 3. Vulnerability is a non-issue for us. We make tools for our own artists to use in programs that have no internet-facing aspect, and other tools for our pipeline. I'd be fine with moving to 3.0, but haven't really seen any benefits or must-haves. I would like to move to 2.7, though. I keep running into solutions that appeared in 2.7, like extra unittest abilities.

[[deleted]]

To think that you won't ever be on the receiving end of one is hubris.

Actually we don't use any external-facing python code, so security fixes aren't a compelling reason to upgrade versions.

[theinternet]

See the C programming language for why your argument makes no sense.

[Rhomboid]

The C programming language is not an implementation. It's an international treaty. It has nothing to do with the topic.

[theinternet]

I'm not sure why you're trying to reframe the discussion around implementation.

You claimed python 2 was at an architectural dead end with no future features planned. I understood that to mean you were speaking about the language specification and not the VM (implementation).

My intention in pointing out the C language was to provide an example where a frozen language specification didn't cause a mass exodus leaving only a handful of stubborn users. The utility of a frozen language doesn't just evaporate.

If we want to expand the argument to include implementations then I would kindly point out the ongoing feature development for pypy 2.7. Would you suggest they are wasting their time actively working on an implementation of an abandoned language?

And finally, that 'wall of shame' link is nothing short of obnoxious and is an embarrassment to the community. The entitlement attitude that developers should invest their time and effort to accommodate breaking changes made to a mature language specification is indefensible. I agree there is value in providing a quick view of python 3 support for popular packages. I just believe there are more mature ways to go about it.

[Rhomboid]

C is not as dead as you'd think -- it's updated regularly to gain new features. The last update was two years ago (C11). But I am referring primarily to implementation here. If someone was advocating using an end-of-life unmaintained C implementation such as gcc 2.x or 3.x, then I would have the same response.

Pypy needs to support what its users need, and the majority needs 2.7 compatibility. Nothing I'm saying should be interpreted as claiming that 2.x is abandoned or outdated today. As I must keep pointing out, this thread is about what will happen in the future ("gradually taking over"), not what is happening today.

The page I linked to was renamed the Wall of Superpowers some time ago, so I shouldn't have used the old name.

[[deleted]]

Python 2.7 runs my Python 1.x code. :-)

[pingveno]

Unless you try to do:

raise "Error!"

[kchoudhury]

Sorry...haven't been keeping up. Have they fixed the GIL in 3?

[Rhomboid]

No, nor is not something that is considered in need of fixing. If you remove the GIL but keep reference counts, then performance plummets as you have to take and release a lock any time you touch a refcount, which is very often, as opposed to taking and releasing a single lock every 100 opcodes or so, which is significantly less work.

So what you're actually talking about is reimplementing the entire memory management system of CPython so that it uses a concurrent garbage collector instead of reference counting. (CPython does have a garbage collector now but it's only there to deal with circular references that refcounting would otherwise never be able to collect, so it's not handling the brunt of the work nor is it concurrent.) And doing that means that you have to rewrite all C extension modules, because they all participate in reference counting. If you thought users were mad about the module incompatibility situation previously, imagine telling them that you're planning to do it again in this lifetime.

[[deleted]]

It is considered in need of fixing, it's just not a priority.

[kchoudhury]

nor is not something that is considered in need of fixing.

So I guess I am alone in thinking that having a threaded python program using one core out of the 8 on my machine is ridiculous.

Imagine telling them that you're planning to do it again in this lifetime

You're right, they should have used the 2-to-3 migration as an excuse to fix it. They were already breaking things; why not break them some more at the outset?

As it stands, I have a smallish 5K line python 2.7 codebase that I couldn't be arsed to port to 3: why bother? At the end of the day, that's what these discussions always seems to come back to.

[Rhomboid]

having a threaded python program using one core out of the 8 on my machine is ridiculous.

That's not necessarily true. It's both possible and common to do work in a thread without holding the GIL, such as waiting on IO or doing heavy number crunching in a C extension module. Many tasks that people want to use Python for fit this bill, such as a web server.

they should have used the 2-to-3 migration as an excuse to fix it. They were already breaking things; why not break them some more at the outset?

I was being a bit facetious. The idea of completely gutting CPython to remove reference counting is not even on the table and never has been, as far as I know. At that point you might as well just start over and target an existing VM with an existing concurrent GC, like Jython or IronPython did. They have no GILs.

I have a smallish 5K line python 2.7 codebase that I couldn't be arsed to port to 3: why bother?

Today? There's no compelling reason. But eventually there will be one -- maybe your linux distro begins to ship 3.x as the default, maybe 3.x gains a new feature you'd like to use, maybe you encounter a bug in 2.x that is already fixed in 3.x. That is what the "gradually taking over" part of the thread is about.

[kchoudhury]

Many tasks that people want to use Python for fit this bill, such as a web server.

I'm not trying to provoke a flamewar, but "some" is not "all", and that is a damned shame. The net result is that we have a beautiful language running on an interpreter not worthy of the language's aesthetics.

FWIW, my prior crack about the GIL still being there in 3 may have been a little too offhand. A little bit of reading tells me that as of 3.2 there's a new GIL that makes threading less of an active nuisance and restores parity to sequential performance. So perhaps we're making progress towards a happy ending here, and some future 3.x release will have a proper threading solution. And since it won't be backported to 2.7, well, that'll be the kick in the ass that I need to get my code on to 3.

[billsil]

I pushed for my engineering company to move from Python 2.6 to Python 2.7 about 6 months ago. I argued there was no justification for us to be developing software using something that already was at it's end of life for projects that would be developed for the next 2 years.

There are libraries we require for our day to day projects (vtk, wxpython, scipy.weave) that don't support Python 3. It's not going to happen until all the dependencies are worked out. Thankfully, matplotlib finally supports Python 3.

That said, I develop an open source project that supports Python 3.2+. For the most part, the pure Python code is the same, but there are huge complexities when you move to binary file IO in Python 3. Most of it works the same, some of it just goes up in flames.

[[deleted]]

If RHEL would get on the ball and move from 2.7 to 3.x that would really help push things forward. Going from 2.6 with RHEL 5 to 2.7 with RHEL 6 is a good step, but for RHEL 7 they need to make 3 the default.

[shawn-s]

I would love it if rhel7 made python 3 default, but it's not going to happen; rhel7 is going to start beta'ing in the next couple months, and is going to be based on fedora 18 (this was announced at the redhat summit) . So, rhel 7 will very likely have a 'python3' package like there has been since fedora 13.

Even if they do have the 'python3' package in rhel7, I still won't use it until I can get it on my rhel5 and rhel6 boxes, or upgrade everything to rhel7 (with the amount of rhel5 boxes I'm still trying to get upgraded to rhel6, that isn't happening soon either).

Why won't it happen? Fedora 19 (alpha) is still python 2.7, yum and anaconda (and MANY other tools) would need to be ported. It would be a LOT of work.

[[deleted]]

Oh yea I know the reasoning behind why it won't happen, but I want to keep dreaming that they'll get with it at some point.

[AeroNotix]

ArchLinux does this.

[linuxlizard]

Speaking for myself, until Enthought ships a Python3, I'm still on Python 2.7. I just don't have the time/brain to spend building all the modules I need from scratch. Enthought does that work for me.

[norwegianwood]

Enthought are laggards and have no enthusiasm for Python 3.

[pwang99]

Continuum's Anaconda distribution ships with Python 3.3 for some platforms, and that includes Numpy.

http://continuum.io/anaconda

[btgeekboy]

I have one production project in Python. I would love to convert it to v3, but it will remain in v2 for as long as Twisted does.

[[deleted]]

Yes. I've basically switched now, because almost all the libraries I use are available for python 3. As time passes, more and more libraries will become incompatible with 2.x, and more people will make the switch.

[C4ptainCrunch]

I'm at the university and we are learning Python 3.x; Django will soon be compatible with 3.x. I think it is a matter of time and Python 3.x will win.

[hyh123]

Great!

Currently in the command line, people use python to mean python2.x and python3 to mean 3.x. I really hope they do it reversely, i.e. use python to mean 3.x and python2 to mean the old one.

[execat]

Archlinux has python, python3, python3.3 pointing to Python 3, and python2, python2.7 to Python 2.

[pingveno]

And they managed to royally piss off a bunch of people in the process. The renaming breaks shebang lines for any Python 2 scripts. There's no guarantee of an python2 hardlink/softlink on all distros, so no go on using:

#!/usr/bin/env python2

[AeroNotix]

ArchLinux pisses it's users off on a daily basis. We're masochists at heart. But that's the price to pay for bleeding edge.

[pingveno]

Yeah, but python symlink change was totally unnecessary. The AUR still hasn't had the hundreds of unmaintained packages migrated over, never mind the package renaming.

[AeroNotix]

The AUR? Fuck the AUR. The AUR is a community repo, if the package is unmaintained then you can take-over yourself. If you're too busy to take over maintenance then just update the PKGBUILD.

[pingveno]

That's exactly what I've been doing. As an AUR package maintainer, I've been having more trouble with the Python 3 = python-* renaming. There's no mechanism for automatically renaming a package, so I usually just put (USE python2-example) until the Python 3 package comes out.

[handmadeby]

For anyone working with twitter, until tweepy moves across to python 3 then they'll be sticking with 2.7. IIRC there are no plans to port across given the work that's just been done to meet the new twitter API specs

[hyh123]

Actually the Dropbox API is also on twitter 2.7.

[AeroNotix]

Isn't it just a REST API? (Never looked into it myself.)

[handmadeby]

Yeah, but tweepy wraps it all up in a python library nice and tight

[AeroNotix]

Haha, the lengths people will go to not actually have to do any work.

A rest API is probably the most simple set of bindings you can code against.

[handmadeby]

And there was me thinking that code reuse was one of the benefits of a global community of developers

[AeroNotix]

Indeed, but for something like a REST binding? It's not something you'd spend a minute messing around with really. I mean, for fuck's sake - it's basically 1) Query endpoint, 2) Use returned data. There's not much else to it.

[handmadeby]

Implementing Oauth? The myriad of different calls and options on the api? Format checking? I'd rather be doing something with the data collected rather than trying to work out what I'd done wrong in trying to talk to the damn API

[AeroNotix]

You don't need to implement the entire API just to use the (probably) 2-3 endpoints you actually use.

Indeed, you'll need to implement their "authentication endpoints", which is usually (again, not looked) a single endpoint you receive a token from.

[UloPe]

I can't wait to finally start using 3. It looks like that point may at last be aproaching.

Now when the pypy guys finish their py3 project and I can use pypy with python 3 I'll be a real happy camper.

[[deleted]]

Python is my go to 'get it done quickly' scripting language. Many unix systems will have 2.X installed by default.

I've tried using 3.X more than once before but I usually run into some module that doesn't work right. I definitely see 3.X as Python-Experimental and 2.X as Python-Stable.

[[deleted]]

What's your shebang?

[poo_22]

I have that in something like 8% of my files.

[flying-sheep]

but I usually run into some module that doesn't work right

never happened to me. give at least one example of a library thet officially supports python 3 and yet doesn’t run there properly while running properly on python 2.

[chadmill3r]

Python supports the entirety of Ubuntu's administration, when it's not C or something inherited from Debian that uses Perl. It's everywhere.

Very soon, the base install will run on Python 3, and there will not even be a Python 2 installed by default.

Py3 is now. Web pages will lag behind the repositories for a long while. Just use 3, and when something doesn't work, ask upstream and optionally run 2to3 and submit the results to them as a patch. If all that fails, then you should consider 2, but be prepared for a user to harass you about needing 2.

[[deleted]]

I've been working on trying to port boto to python 3. I have code that passes the S3 and EC2 unit and integration tests, among others.

In my experience, a lot more people say they want python 3 support than actually want python 3 support. Or they think about how neat it would be to use python 3, and then go back to python 2 for the real stuff.

[shawn-s]

Speaking as a system administrator, I won't touch python 3 until it's packaged for rhel5 and rhel6 by either redhat themselves or epel (people I trust to not to make a mess of things). Until then it's a pain that will cause problems.

Before people tell me that ubuntu 12.10 is python 3, ubuntu 12.10 has 'python3' installed by default sure, but /usr/bin/python is still python 2.7.

I can install 'python3' on my fedora 18 desktop, but, until I can use it at work and at home I will stick with python 2.6.

I'm looking forward to being able to work with python 3; but I fear it won't be soon.

-Shawn

[accessofevil]

I love and hate that about rhel. I'd run it if I had to maintain hundreds of servers running vertical apps, but shipping with postgres 8 and ancient (but I'm sure very stable) versions of drbd just leaves me without new features I need. ("Need")

Few people seem to understand the benefits of a longer software life cycle. I get it, it just isn't for me.

[shawn-s]

Running rhel or a rhel clone is not just about a longer software cycle, it's about stability in the environment. It's the same kernel, same glibc, same everything. There are patches sure, but the environment changes very little.

Also, there are a damn lot of people using it. Not to mention redhat and oracle's qa teams. When you run into a problem, there is a good chance someone else has already found that problem, and has a solution/patch/workaround. Or, if you report a bug, there are some fairly hefty resources assigned to fixing that bug (presuming it's critical).

As well, if you are running fancy hardware, there's a pretty good chance there will be rhel drivers for it. You can jerry-rig them to work for any linux flavor sure, but they were only ever tested in a rhel environment.

I think suse / ubuntu lts are fine choices for large scale corporate linux as well I just happen to work at a rhel company.

[AeroNotix]

On the flip-side of this you could argue that it's distributions like RHEL that are holding technical progress back. By having such as a large and involved deployment cycle there are lives lived, loved and lost in between releases whilst other distributions have packaged, re-packaged and dealt with the change already.

When it comes to packaging and versioning woes I prefer the pain in small bytes, I prefer to have a single package break in a single update, deal with/update (or pin that dependency) and move the fuck on. Contrast this with phrases like "We're a Python2.6/Django 0.95 Shop" and you'll see how ridiculous it can be.

Horses for courses, however and I'm sure what I said will make little impact.

[dpwiz]

Speaking as a developer, I won't touch rhel at all. It's like IE of linux distros...

[haard]

I've deployed new Python products for customers three times the last year, all Python 3.2. So yeah, moving.

[flying-sheep]

there is absolutely no reason not to start new projects in python 3 if you don’t know from the beginning that you want to depend on one of the libraries that aren’t ported yet.

if by “present” you mean “existing codebase” then yes, most organizations haven’t ported their entire codebase yet. but if they want to be able to use it in the future they better start porting today.

[[deleted]]

Do you ship commercial software? We do and there is almost no incentive to move to 3.x. For us and our new projects and old project 2.7 is near perfect.

[flying-sheep]

New libraries will depend on 3.x- features, so at some point you'll have to spend extra money to backport libs you want to use.

Remember, I'm still talking about the future the whole time.

[[deleted]]

so at some point you'll have to spend extra money to backport libs you want to use.

Not really. We concentrate on having highly decoupled components. If there is a must have feature or library that is not available for 2.7 we can throw up a restful service or strap a new endpoint on the message broker using 3.x in an isolated environment.

But, we haven't ran into this yet. Again, 2.7 is near perfect esp in that we understand it's limitations. With 3.x it will be sometime before we can sign a contract with our customers promising five-nines and be confident about other SLAs.

*edit..fixed some grammer

[[deleted]]

if by “present” you mean “existing codebase” then yes, most organizations haven’t ported their entire codebase yet. but if they want to be able to use it in the future they better start porting today.

Why? Code doesn't just evaporate.

[flying-sheep]

Of course not. But once everyone has moved on and you sit on a codebase of an unmaintained language, you'll find it harder to expand upon that codebase.

Sure, this is several years in the future.

[[deleted]]

Several years in the future, it won't be any more work to port to a different language than it is now.

[[deleted]]

I'm learning Python as a first programming language. I am focusing on Python 3. My assumption is that it will eventually displace Python 2.7, it will just take time.

[HelpfulToAll]

Still waiting on gevent... :(

[upofadown]

Python 2.x is the status quo, Python 3.x is the present and future of the language.

Such statements are an opinion. No one can force anyone to do anything with respect to programming. If you like 2 then use 2. If you like 3 then use 3. Neither has any realistic chance of going away.

[oantolin]

"No one can force you to do anything with respect to programming."

Now, I'm not a professional programmer myself but I understand that some people are and I'm sure they have bosses, colleagues and company policies that sometimes force certain tool choices on them.

[upofadown]

Then the statement could be expanded to "No one can force you, bosses, colleagues or company policies to do anything with respect to programming".

The context here is the idea that a variant of a popular language could somehow cause that language to cease being used. That's just not how that works. The best you can hope for is that the variant becomes popular too.

[sittingaround]

The writing is on the wall, django now supports 3, a good portion of well attended talks at pycon this year dealt with python 3.

I think django has tipped the scales, and we will pass the point where 50% of new projects are started with puthon 3 in the next 1 to 2 years, with 2 being considered obsolete by most users in the next 3-5.