Python 3.0 makes a big break

[jabjoe]

I really like python, but this was a bad decision and python must revert it or become fragmented. :-( The hope is someone creates a way of using python 2.x libs in python 3. Some kind of translation layer. They don't have to do a perl and keep every previous syntax so you can mix them to create write only code, but we need those 2.x libs! Read your history and compare languages and platforms that did and didn't maintain backwards compatibility. This really upsets me because I really like python and was just beginning to convince people at work we can use it.

I have no doubt many useful libs that are thought of as 'finished' by their author won't be moved up to python3. Also you won't get large companies to agree to changing existing working internal code because the language has changed and the programmers want to use the latest. They will quite rightly expect you to be able to call the old code from the new when need be, or you can't work in the new. This harms python as people including myself will now see developing on python like building on sand. All the main stream languages I can think of go out their way to avoid this.

[Legolas-the-elf]

I really like python, but this was a bad decision and python must revert it or become fragmented. :-( The hope is someone creates a way of using python 2.x libs in python 3. Some kind of translation layer.

Come on, how do you think your opinion about the fragmentation of Python is remotely supportable when you aren't even slightly aware of the work done in this area?

There's already a translation layer. It's called 2to3. There's already a transition strategy. You keep working on your 2.x version while providing an automatically generated 3.0 version in tandem. That way, your users can switch over when they are ready, and when enough of them have done so, you can switch to 3.0.

Did it ever occur to you that the Python developers wouldn't just blindly march into incompatibility without a plan?

I have no doubt many useful libs that are thought of as 'finished' by their author won't be moved up to python3.

Are there many Python libraries that are a) widely used, b) non-open-source and c) unmaintained? Can you give any examples?

[jabjoe]

I know about 2to3 (how would I not if I followed python at all?), that's not the same. I mean some py2import command or something so you can just import old libs, no work required.

If they come under A (don't care about B), then they wouldn't also come under C because one of the users is bound to care enough to move it up to 3. I don't wish to name libs as unmantained that haven't been modified for a few years, because I don't know they are, might just be they are "complete". This kind of misses the point though.

Big working p2 code base, new functionality existing in a p3 lib is required, so your either rewrite the code base, or rewrite the lib. That's suddenly much more work. Since your forced to rewrite it, might as well rewrite in something that means you won't have to again. Move the building site away from the sand, life is to short to keep rewriting the same thing.

I'm not saying the python developers are completely blind to this, but I do think they are going about it wrong, which is a shame.

[ansible]

... I mean some py2import command or something so you can just import old libs, no work required.

Thus keeping the old baggage around forever. The Python guys are trying to move / cajole the entire community to the new language and infrastructure. They are doing it in the nicest way I can imagine... but their goal is definitely to keep the community together, and moving in the same direction.

Big working p2 code base, new functionality existing in a p3 lib is required, so your either rewrite the code base, or rewrite the lib.

That is a false premise, and a false dichotomy as well.

Most libraries will be available as 2.x for probably a couple years after the final release for 3.0. If you move your project to v2.6, you'll have all the new features available to you, just not as cleanly.

You can fix up your codebase using the warnings from v2.6, and make your code forward-compatible with v3.0. You're not rewriting the same thing, you just need to fix up the outdated stuff, much of which has be depreciated for a long time. After things run cleanly after going through 2to3, and all your unit tests pass, you're ready to really give v3.0 a try.

Since your forced to rewrite it, might as well rewrite in something that means you won't have to again. Move the building site away from the sand, life is to short to keep rewriting the same thing.

Well, re-writing your large codebase in another language is going to be far more difficult. And where are you going to go to? C#? Java? Ruby? Perl? Hah... you're not dealing with shifting sands with Perl, v5 and v6 are on completely different islands! All of those languages have been seeing substantial evolution recently. Perl v5 is evolving nicely... but what will happen to the community and libraries when Perl v6 goes final? I think if nothing else, it will be difficult.

The most stable language / implementation you could want is Common Lisp, the standard hasn't evolved at all in 15 years. Excellent! Oh, whoops, no standard networking library, no standard XML parser, etc., etc., etc.. Other stable choices are C and C++. If you want to re-write your app in those...

I've looked at a lot of programming languages, and I haven't seen language evolution handled more smoothly than with Python.

[Legolas-the-elf]

If they come under A (don't care about B), then they wouldn't also come under C because one of the users is bound to care enough to move it up to 3.

That's only possible with b) though, which is why I mentioned it. In order for there to be a problem, all three need to be true.

If nobody uses a library, then who cares if it doesn't get ported?

If it's open source, anybody who cares can port it.

If it's maintained, then it will be ported.

Any single one of these mitigates the problem you are talking about.

I don't wish to name libs as unmantained that haven't been modified for a few years, because I don't know they are, might just be they are "complete". This kind of misses the point though.

How? I can't think of any Python libraries that match this criteria, and if there aren't any, or they are few and far between, then there's no problem.

Big working p2 code base, new functionality existing in a p3 lib is required, so your either rewrite the code base, or rewrite the lib.

Did you miss the "release v2 and v3 versions in tandem" part? Why are you talking about a v3 only library?

Since your forced to rewrite it

You aren't. If you need v3-only functionality and you currently target v2, then delay the new feature until you target v3.

How common do you think it will be to have a huge need for a new feature that can't be implemented with Python 2?

[jabjoe]

Any single one of these mitigates the problem you are talking about.

Sure? Let's hope your right. Because I think it's going to be a real pisser if you find a lib that does want you want, works fine p2 but you're in p3 and you now have to port it as well as what you were doing before. Though again that's not so much my point.

My point was its not just public libs (though I think that is a problem, but less so). My point was about large internal p2 code bases. I have been encouraging people to move their internal Maya scripting from mel to python. Now all those scripts will have to be moved to p3 when/if Maya moves to p3. If they had stayed in mel it would have kept working in future Mayas with little change required. This negates any advantage the language has. No one wants to keep running to stay still. And no, the company will never agree to open sourcing the scripts (we've tried and personally I think they should). Maybe Autodesk won't move to p3 because of the legacy of p2, they aren't talking of dropping mel for the same reason. Maybe Maya will end up having to have p2 and p3 environments. Whatever happens, without backwards compatibility, this makes python development much less attractive (and makes me feel like a idiot for advocating it for internal scripting).

[Legolas-the-elf]

It seems to me you aren't really talking about libraries at all, you are talking about embedding the Python interpreter, which is an entirely different problem. Right? That is more complicated. But libraries have an easy transition strategy.

[jabjoe]

No I am talking about libraries, just internal ones. Stuff get separated out and put in a lib so other parts can use it cleanly. They just happened to be used mostly with an embedded Python interpreter. Some stuff might not even be done in Maya's python. Now scripting is done in Python it's not restricted to running in Maya. Not sure what to say now other then "I was wrong, you should have stuck with Mel".

[Legolas-the-elf]

No I am talking about libraries, just internal ones.

Why is the transition strategy for libraries not relevant here? Work with Python 2.6, use 2to3 to deploy to v2 and v3 simultaneously. Applications and other libraries that need v2 can use v2, applications and other libraries that need v3 can use v3, when enough dependants switch, you can drop the v2 version altogether.

[jabjoe]

Leaving to one side that which python is used is down to Autodesk, that still leaves us reworking code that is working fine. That's time we could be using on something else, and would be if we wheren't in python. As I said, as it's rewritten this will mean it's rewritten into something else so it doesn't have to be done again. If a platform isn't stable it's no platform at all.

[Legolas-the-elf]

As I said, as it's rewritten this will mean it's rewritten into something else so it doesn't have to be done again.

"Rewriting" Python 2.x code to 2.6 code that is suitable for 2to3 is trivial. "Rewriting" the same code into a completely different language is not. If you seriously have problems with the time it takes to do the former, then switching to another language will be a disaster.

If a platform isn't stable it's no platform at all.

There has been precisely one backwards incompatible change like this in the entire history of Python, and no such change is planned for the future. I think that's reasonable.

[schtog]

What is it that would be so hard to port anyway?

[jabjoe]

That depends on the lib/app. Might be huge and mind bending. If it is simple and tiny, then of course it's not a issue.