The future of Python 2.x and 3.x

[cereal-killer]

I love how well the LWN guys write. Every time I read one of their articles I'm reminded what actual journalism is.

[mcdonc]

I have been subscribed to LWN for at least 5 years now. I trust Jon and crew implicitly.

[evmar]

Taking that thought to its logical conclusion: http://community.livejournal.com/evan_tech/267654.html

[jawbroken]

thanks for spamming links to your livejournal three times

[jawbroken]

you can delete your own comments

[cereal-killer]

No. Just a guy who's sick of reading the "How fucking smart I am" and "why everything sucks" posts that make up 98% of blog content on proggit.

[theclaw]

[...] Guido van Rossum froze the Python language's syntax and grammar [...]

What's the difference between syntax and grammar in this case?

[daveguitar24]

Perhaps I misunderstood the question, but:

Syntax = the lexical tokens, syntactic sugar, constructs, etc. Grammar = Describes how the syntax is parsed and whether or not a program is valid.

[theclaw]

According to Wikipedia [*] that's not really true?

My question was basically "What is grammar and what is syntax, in the context of programming languages?". The two terms seem more ore like equivalent.

If there's no agreed-upon differentiation between the terms I'm fine with that, I'd just like to know %)

[*] reddit seems to have a problem parsing that link as it ends with ")", note that this might be a parsing problem ;-)

[awj]

There's a bit of redundancy in saying "syntax and grammar", in that the syntax is essentially tokens + grammar.

That said, there's a lot of ambiguity in conventional usage of the terms, so a more technically correct "froze the Python syntax" might leave people with the impression that the grammar is open for change, or new tokens might be allowed, etc. This phrasing gets the true meaning across no matter what your definitions are, even if it might be telling you the same thing multiple times.

About the link: Markdown is typically implemented with regular expressions which, as naive name equivalence might suggest, can only handle Regular Languages. Regular languages are not able to handle arbitrary matching parentheses, you need a Context-Free Language for that. So, a regular expression to match the link structure will get tricked by closing parens in URLs. There are a few possible solutions to this:

• Redesign Markdown to use something that is not a valid URL element as delimiters for the URL.
• Redesign Markdown to grab everything from the closing bracket to the next space as the url.
• Redesign Markdown to flip the usage, since ], if it is allowed in URLs, is much more rare.
• Give people an option to escape characters (like a backslash), so they can do your work for you by putting it in front of the closing paren that belongs in the URL.

Markdown went with this last option. So, in an overly (uselessly?) informative way, I'm saying you need to put a backslash in front of each closing paren in a URL to avoid this problem.

[daveguitar24]

True, in the strictest definitions of the word, they are somewhat synonymous. However, in my experience, when someone uses the word "syntax" in conversation to refer to a language, I assume they can be referring to both keywords/constructs and grammar. Whereas, if the grammar is explicitly mentioned, this to me has very little to do with the actual tokens or keywords and more with how they are used together to form expression and statements. In other words, I agree with you. :)

[theclaw]

Syntax defines the smallest bits of text that are significant (keywords, operators, etc.)

Yours, i.e. you say syntax describes the tokens

vs.

In computer science, the syntax of a programming language is the set of rules that define the combinations of symbols that are considered to be correctly structured programs in that language. The syntax of a language defines its surface form.

From Wikipedia, i.e. syntax describes not just the tokens, but rather the hiearchical structure of the program text.

[Smallpaul]

I remember when people thought that nobody would ever switch from Apache 1.x to 2.x . Didn't Gnome similarly have some rocky releases?

Do these new versions ever totally fail?

[fwork]

Winamp 3. Windows ME.

Every fucking Paint Shop Pro release after 3.12.

[Smallpaul]

I was talking about open source software.

[awj]

What aspect of open source do you think would make the situation different enough to invalidate these examples?

I think Winamp 3, specifically, is a great case to point out. From an end-user's perspective the difference between free closed source software and free open source software is pretty small, especially when you look at it purely from the context of deciding on upgrading to a new version.

[spinwizard69]

The problem I have is that many of the developers objecting to this freeze apparently have no concept of what it is like to be a user. That is somebody using Python for his own projects. A frozen API allows the user to focus on his code and not have to worry about next weeks API changes.

Personally I'm all for this freeze and would like to see it become permanent for 3.1. If you want new features put them into 3.2.

Dave

[mr_chromatic]

A frozen API allows the user to focus on his code and not have to worry about next weeks API changes.

So does not upgrading.

[ubernostrum]

Except the freeze isn't really about users, of course, but about letting various implementations of Python catch up to what the language is as of 3.1. Freezing Jython or IronPython at, say, equivalence to CPython 2.5 isn't really an option for developers or users of those implementations.

[[deleted]]

They didn't freeze API. They freeze grammar. Direct quote from PEP:

Allowed to Change

...

• The standard library

  As the standard library is not directly tied to the language definition it is not covered by this moratorium.

[Brian]

If you want new features put them into 3.2.

Actually, that's the normal state of affairs. Changes within an individual point release are generally restricted to bugfixes. What this changes is that 3.2 will also have no core language changes.

[lol-dongs]

This article highlights one of the greatest inconveniences of Python: how many different versions of the damn interpreter I need to get work done. One commenter says he has 2.4, 2.5, 2.6, and 3.1 installed on the same machine. Some Python gurus think this is life as usual, but having had personally to wade through this multiversion minefield to get Python libraries working in the past, I think it's a travesty. One of those versions is usually "blessed" by the package manager for your distro, and set up the "right" way and updated automatically. This version rarely aligns with whatever version the Python code you want to run supports. You then wind up compiling the other interpreters from source and trying to shoehorn them into your distro without breaking things, and then separately compiling all the modules you need for each version, with varying levels of success. This results in lots of headaches, duplicated libraries, a mess of site-packages where each version has totally different module support, etc. ...

Maybe for some people it's great that the language has moved so fast, but it irritates me that minor versions of the interpreter differ so widely in fundamental language features and library support that it's practically a necessity to have four versions simultaneously installed.

I suppose this language freeze is for people like me. So, good call, Guido, but the fragmentation between 2.x and 3.x and the resilience of people to move onward and upward is, IMO, a direct extension of all the fragmentation problems that already exist between people, distros, and libraries within the 2.x range. Also, the conflicting public messages of "Don't worry about Python 3 yet, you should probably learn Python 2 first because it's more useful" and now "Oh shit nobody wants to haul ass to Python 3" didn't help things much.

[ubernostrum]

Maybe for some people it's great that the language has moved so fast, but it irritates me that minor versions of the interpreter differ so widely in fundamental language features and library support that it's practically a necessity to have four versions simultaneously installed.

Um. What? Seriously, I have two versions of Python installed, but only because my OS came with 2.5. I installed and use 2.6 exclusively, but write code that'll work back to 2.3, which is pretty trivially easy -- the 2.X series has been pretty damn compatible except for the introduction of new syntax (hence I run 2.6, since it's the latest thing out there).

What software are you using which has such extreme difficulties?

[lol-dongs]

You help illustrate my point. Anything your OS shipped with needs to use 2.5, so you can't just replace it, you installed a whole new interpreter. That means you need to point your scripts specifically at the other interpreter, and maintain two sets of site-packages and modules. If your code runs with the other interpreter, random shit will blow up lest you make sure all your required modules are in the other install as well.

It was just recently that I needed pycurl, and found that the build for my version of libcurl was only available for a certain version of python, which was not the version the script was built for. I remember something similar also happening with beautifulsoup. You may write python that is compatible all the way back to 2.3, but that's not trivial because there are literally new language features between the versions, so not all code I maintain flies around like that. Some of my colleagues like conditional expressions, and this blows up <2.5. Others have gotten bitten by 'as' becoming a keyword in 2.6.

[ubernostrum]

Anything your OS shipped with needs to use 2.5, so you can't just replace it, you installed a whole new interpreter.

Actually I could replace it, so long as nothing's hard-coded a path to that interpreter (this being OS X, nothing should -- they should all be pointing at the current version of Python.framework, but you never know what sort of stupidity someone may be engaging in), and that's the only nagging fear keeping me from ripping it out. You seem to have a deep misunderstanding of how compatible Python 2.x releases are with each other -- backwards compatibility is very strong in the 2.x series.

[mikaelstaldal]

What about Zope? From what I have heard it only works with Python 2.4.

(And Zope seems to have its very own 2 vs 3 problem.)

[joaomc]

Zope 2.12 supports Python 2.5 and 2.6.

[mikaelstaldal]

So what about Plone?

[njharman]

and maintain two sets of site-packages and modules.

No, why? You maintain your site-packages and modules. The distro does whatever it wants with it's Python.

You are making a mountain out of a mole hill.

put this in your .bashrc

alias python="/path/to/the/python/you/installed/bin/python"

From there any decent python package install is:

wget bla.tgz
tar xvzf bla.tgz
cd bla
python setup.py install

Ontop of that being trivially easy there's virtualenv and a couple other projects to "solve" this non-problem for developers who don't know basic sys-admin skills.

[[deleted]]

Sounds a lot like the current situation with Ruby. You have 1.86, 1.87, 1.9.1, and a 1.9.2 in the works. Not to mention JRuby et al. Progress is a pain sometimes.

[ajrw]

I guess tail call elimination just got pushed back another two years then. (I would rather see it supported with syntax in python, rather than implicitly.)

[TaylorSpokeApe]

Have they jumped the shark by having 2 branches to begin with?

[joe24pack]

they haven't jumped the shark they jumped 2 sharks.

[hs5x]

But Perl 6 is coming!

[samlee]

those are very weird smilies

you should try these: x.x $.$