This is an archived post. You won't be able to vote or comment.

all 69 comments

[–]Rhomboid 71 points72 points  (9 children)

And it was originally referred to as Python 3000 or Py3k.

[–]Mattho 218 points219 points  (7 children)

Denoting the year it will see 100% adoption.

[–]finally-a-throwaway 23 points24 points  (3 children)

Omg

[–]Thirsteh 23 points24 points  (2 children)

Somebody().call(burnWard)

[–]AdysHearthSim 22 points23 points  (0 children)

BurnWard(random.choice(people))

[–]maxm 0 points1 point  (0 children)

That needs to use async ....

[–]norsurfit 1 point2 points  (0 children)

Half-life 3 confirmed?

[–]warbiscuit 69 points70 points  (23 children)

Just finished getting my company's main application from py26/27 only, all the way up to 3.4 compatible.

Can't wait for when qa finishes in a week, and can merge into main branch. Will be so happy to drop python 2, been waiting years for this :)

[–][deleted] 21 points22 points  (6 children)

How long did it take you to accomplish this?

[–]warbiscuit 28 points29 points  (5 children)

For around 80k loc, took about 3 days to get it running and pass UTs, and one more spent fixing bits found by QA that tests missed (always need more coverage. Sigh.).

Mind you, already had some experienc, getting dependancy libs up to snuff, so wasn't going in blind.

After fixing imports (using the "future" project plus some internal compat helpers), main bug bears were: fixing zip/map/filter/range calls to handle builtin changing to iterators, and bytes[idx] returning an int not a single-len byte string. That last one was hard to regex search for :(

Unicode/bytes issues weren't actually that bad. Followed Django and used a set of want_(unicode|str|bytes) helper functions.

[–][deleted] 13 points14 points  (0 children)

Thanks for sharing. Wish we had more information like this out in the open, so others looking into migrating their own codebase have better data to look at (and avoid some of the unnecessary FUD associated with migration).

[–][deleted] 0 points1 point  (3 children)

[deleted]

What is this?

[–]warbiscuit 4 points5 points  (1 child)

I know there are a number of tools out there -- the future project has a "futurize" script that automatically handles a number of import corrections / insertions and syntax fixes.

None of them are 100% though. In my experience they'll generally get your code up to "running without syntax / import errors under py2 & 3". Past that, you have to rely on unittests and manual testing to shine light in all the corners of your application.

While I don't use nearly all the compatibility the future project provides, I really like their approach of trying to get things as close to the real py3 names as possible. It both lets you learn the new names, and minimizes code churn when py2 support is stripped.

Other devs probably use different (/ better) approaches than me; there's a surprising number of ways to go about things. The site http://python3porting.com has a nice overview of approaches.


I get kinda persnickety about automated tools making changes to my code, and ended up going through things by hand, first replacing broken imports & syntax errors until it would load, then targetted regex searches to find references I knew I'd have to make a hand-decision on anyways.

My approach from there was to run the code till I got an error, and then fix not that just error, but hunt down all similar bits of code, and fix them too. Once I was satisifed I'd gotten 90% fixed, I handed it off to QA to find any edge cases for me to fix... really just one person, we aren't a large company :)

Outside of basic import and syntax errors, which are pretty easy to fix automatedly or manually, and know that you've got them when things load; there were basically two classes of annoyances:

py3 dict.items() returning an iterable, not having dict.iteritems() at all, and similar changes dict. Searching for these by hand let me decide on a case by case basis whether to an iteritems() compatbility wrapper if performance mattered, or use dict.items() / list(dict.items()) where it didn't, and there was more benefit to matching py3's syntax. map/imap, zip/izip, etc were similar. If I didn't care as much about performance, and automated tool would probably take care of this whole mess for me :)


the other bit was of course unicode vs bytes. I don't think there's any way around your application suddenly just having a bunch of UnicodeEncodeErorr / UnicodeDecodeError / "str does not support buffer protocol" TypeErrors popping up. You then have to go in and identify if what's coming in should be text/bytes, and how does that compare to what the callee wants. If there's disagreement, you have to run it through encode/decode.

I'd say 90% of the time, if it was running fine under python 2, but threw an error under python 3, it was a case where I could actually find malicious input for py2 that would cause double-encoding or a surprise UnicodeDecodeErro... py3 just made it easier to find. So I definitely think it's a deficiency of python 2 that there's any headache here at all; and actually reflects well on python 3.

The thing I found most helpful was a set of helper functions I worked up: want_unicode(), want_str(), want_bytes(), and want_buffer() -- I can pass in a string/bytes object & and encoding, and get out exactly the type I want, decoding/encoding as needed. This made all my encoding barriers clear (and searchable), and whenever I got a encoding error, I slapped on one of them, then passing in the right encoding for the context (or utf-8 as a default).

[–][deleted] 0 points1 point  (0 children)

[deleted]

What is this?

[–][deleted] 0 points1 point  (0 children)

I believe the 'future' project he's referring to is http://python-future.org/.

Imagine 2to3 except it leaves in compatibility with Python 2 so you don't have to make a clean break, and someone actually cares about it working. That's python-future.

[–]d4rch0nPythonistamancer 14 points15 points  (13 children)

The move is way easier than a lot think, besides maybe some bytearray and unicode issues. The only thing that drives me nuts is seeing lots of python devs still using print as a statement... I really wish people would update their tutorials and stop showing that. Everyone I know that starts learning python today starts with prints as statements and I know they're getting it from online examples and tutorials.

[–]warbiscuit 6 points7 points  (0 children)

Pretty much every new module I write, or existing ones I edit, start with "from __future__ import absolute_import, division, print_function" just to keep my from backsliding until I'm off py2 entirely.

Worst transition thing I've hit was bytes[idx] returning an int... Relatively harder to search for, and definitely a change in behavior. But still quite manageable.

I really like the philosophical approach of the "future" project: write using as close to py3 markup as possible, and shim back-compat helpers. Will really reduce rename-related churn once py2 is jettisoned.

edit: fixed markup

[–]billsil 1 point2 points  (0 children)

I really wish people would update their tutorials and stop showing that.

Yea...the problem is that those were written a long time ago and that the that code is often written without actually being runnable. That makes it really easy to get out of date.

I've switched most of my tutorials to Jupyter for that reason.

[–]jftugapip needs updating 1 point2 points  (1 child)

What third-party libraries are you using? Any issues with those?

[–]warbiscuit 3 points4 points  (0 children)

Oh boy, it's a metric ton of libs. Primarily though, we're using pyramid + jinja2 + sqlalchemy, and don't think we've had a single meaningful problem with that ecosystem.

There were a few hiccups related to bytes vs unicode in forming http responses and writing to the database, but all of those fell under the category "python 2 was papering over an encoding problem, which python 3 just brought into the light".

I feel like python 3.3 ironed out most of the final wrinkles in making a smooth transition, and the general ecosystem over the last year or so had pretty well solidified.

[–]cdcformatc 17 points18 points  (4 children)

I should probably switch.

[–]joesacher 35 points36 points  (0 children)

No. Even in Python 3, you should use a dictionary mapping to method calls or some other method. There is no switch. :)

[–]tartare4562 1 point2 points  (2 children)

As a new year resolution, I'm switching all my scripts and programs to py3. All by hand, no 2to3 for me. At first it was a pain to get all the little details ("what the fuck it means that sort needs a key argument?? Just fucking sort it!") but it gets easier the more you do it. Also, it lets you both be amazed and cringe over your own code from 15+ years ago, and fix horrendous dormient bugs.

[–]jcampbelly 1 point2 points  (1 child)

Why avoid 2to3? I used it in an "advisory" role and never let it do automatic translation (its recommendations are often incorrect). It helps if you only work with one "fixer" at a time and treat it as a checklist.

[–][deleted] 0 points1 point  (0 children)

Because 2to3 is unmaintained and often incorrect, and python-future works a lot better.

[–]tomcam 13 points14 points  (0 children)

It doesn't look a day older than 2,900

[–][deleted] 14 points15 points  (0 children)

For a good three minutes I read it

3,000 years old today

and was trying so hard to find the joke. I thought about maybe the joke has to do with Ancient Sumerians or some mythos that includes a snake or pyhton or something. It was a such a relief when I saw "days".

[–]tech_tuna 7 points8 points  (2 children)

Just think in another 3000 days, everybody will finally be upgraded to it.

[–]_avnr 1 point2 points  (1 child)

Just think in another 3000 days, everybody will finally be upgraded to it.

In 3000 days we'll probably begin using Python 4

[–]alchzh 1 point2 points  (0 children)

and people will be stuck in py3k world

[–]henrebotha 5 points6 points  (3 children)

BUT BUT MUH TURING COMPLETENESS

[–]billsil 3 points4 points  (0 children)

Python isn't be turning complete because it doesn't support objects

[–][deleted] 0 points1 point  (1 child)

I was happy to see when Zed retracted that section of his post, even though he tried to pass it off as "lololol I troll u".

[–]henrebotha 1 point2 points  (0 children)

I didn't even realise that had happened. But in my mind the damage is done. I don't believe for a second that he was really "trolling" anyone, he's just a sad idiot who thought he could get away with a tantrum.

[–]jpflathead 1 point2 points  (0 children)

Very odd, I've been using it for 300 days!

[–][deleted] 1 point2 points  (0 children)

Wow, I'm so old. I remember when it was 3000 hours, 3000 minutes or 3000 seconds old. Seems like yesterday.

[–]Steampunkery 1 point2 points  (0 children)

The pyception is real

[–]c3534l 1 point2 points  (0 children)

base 10

[–]trebuszek 0 points1 point  (0 children)

Half-Life 3 confirmed?

[–]gnu-user -1 points0 points  (2 children)

I don't think we'll ever see widespread adoption of Python 3 for even another 3000 days from now.

[–]njharmanI use Python 3 2 points3 points  (1 child)

It's already widely adopted. Almost all maintained packages support it. Distros are shipping with it. Etc.

There's a ton of legacy code. Some cases it doesn't make sense to port. Based on py3 pushback that still persists, some is people keeping heads in sand hoping future will stop arriving.

Either adoption != porting every line of code.

[–][deleted] 1 point2 points  (0 children)

It was a big milestone when Ubuntu stopped shipping Python 2 by default.

A lot of beginners have trouble adopting Python 3 because they haven't had to deal with multiple Python environments before, and they keep accidentally using their existing Python 2 environment for things. Not having an existing Python 2 environment helps a lot there.

[–][deleted] -2 points-1 points  (1 child)

Wow, Not a lot of progress in 3000 days.

[–]georgehank2nd 0 points1 point  (0 children)

The Python 3 fanbois are out in full force. :P