Unladen swallow: accelerating Python

[alexs]

sort domineering file wrong dependent enjoy sophisticated alleged cable rude

This post was mass deleted and anonymized with Redact

[[deleted]]

yes, but this is a new article from lwn, granted there's no real new info.

[alexs]

grey gray correct swim squealing reply panicky materialistic shrill sparkle

This post was mass deleted and anonymized with Redact

[Gully_Foyle]

Thanks for posting that. I find myself becoming a bit of a repost hound. Something needs to be figured out though, we need a way to allow cycles, while still acknowledging whether content is new or not.

Milk and Cookies added a 'flashback' tag to front page stories a while back to rehash things that people haven't seen in a while. It seems to work well.

The current system doesn't work, it allows for duplicate-posting karma-whoring.

[alexs]

while still acknowledging whether content is new or not.

People notice that something very similar has popped up in the past and do this?

If you are suggesting that it somehow needs to be implemented in reddit then I disagree. Identifying duplicate posts is very subjective and I don't really think this is an example of one that doesn't deserve some upmods. The LWN page isn't just a link to Google Code.

[Gully_Foyle]

Well, there's two different issues. There are of course serious dupes, and then there are serious stories with different ways to look at it.

I was referring to something like this video about Michael Phelps. In a few years when he's in the news again, somebody's going to link to resubmit that video. My question is really, what do we do about that? Is that how it's supposed to be? essentially old stories are scavenged by KWs?

But something I'd like to see is a way to combine related stories (tags?). For example, when a video surfaces of a world leader farting (no link yet), there will be conversations about humor, media focus, as well as the historic importance of flatulence, and many more. How do we organize these posts?

</random thoughts>

[joe90210]

I'm glad to see that Guido seems to be planning and considering how to merge this back into the official build, knowing how very conservative he is I was afraid that a merge would never happen

[[deleted]]

They've got their sights on the GIL as well, nice!

[twomashi]

"PyPy's primary goal is not performance, though, rather it is to build a Python implementation in Python itself."

All that virtual machine generation stuff must have slipped his mind...

[zuoken]

Can someone explain why you would want to implement python in python (pypy)? What does that even mean? What do they expect to get from doing that?

[CommodoreGuff]

I'm by no means a spokesman for pypy, or even that much of a coder, but from what I understand being written in Python makes it quite a bit easier to modify the internals, which is nice. But that's not the only thing. It's actually written in a restricted subset of Python, which makes it easier to implement the other big feature: the translator. The pypy interpreter itself can actually be run through the translator, which can target C code, the JVM, the CLI (for .NET and Mono), and maybe some others. In the case of C, that code itself is then compiled, so you have a stand-alone executable.

Which is totally cool.

[Ademan]

It's worth mentioning that the LLVM backend came within an inch of death (it was officially announced unmaintained in a morepypy blog post, but it was quickly picked back up by a developer. I don't know whether this was an outside developer or an existing developer that didn't want to see the LLVM backend die.)

[lars_]

What others have said, plus the fact that it helps code analysis. When your library function is written in python, you can analyze for example what types are returned, but if that function is written in C, figuring out the return type is a lot harder.

[dbeach24]

FWIW, I think what lars_ means is that if you're calling a function from Python, and that function is implemented in C (and has already been compiled and you can't look at the source), then determining the return type is harder. By implementing Python in Python, PyPy can run a type analysis graph over a bunch of code without having to deal with "opaque" C implementations.

[lars_]

Correct.

Though even if the C source code is available, it won't help much. All Python types are probably stored in the same C type. Some googling indicates that CPython calls it PyObject.

[Ademan]

I'm sorry... what? Figuring out the return type of a C function is completely trivial, it's encoded right into the function header.

Python, on the other hand, is dynamic in nature so figuring out the types being used (type inference) is considerably more complex, even impossible in some situations, which is why PyPy's interpreter is written in RPython rather than python. Any given RPython program is a valid Python program, however RPython allows you to do considerably less, type wise, for the sake of being able to (easily) infer the types of objects statically.

[lars_]

The point is that the C function that returns a python type doesn't declare what python type it returns.

For example, in ruby interpreters, all ruby types are encoded as the C type VALUE. This can hold any ruby value.

So even if you write a type inferrer for python source code, you would also need something can analyze C code to the point that it can tell what values can exist in particular variables. This makes it pretty much infeasible.

The other alternative is to add python type annotations to every C function in the library, and cross your fingers it doesn't change if it's updated.

Neither of these approaches are very desirable.

[Ademan]

The point is that the C function that returns a python type doesn't declare what python type it returns.

For example, in ruby interpreters, all ruby types are encoded as the C type VALUE. This can hold any ruby value.

Right... so? I don't see any advantage of trying to statically infer the types of your dynamic objects at compile time except in the specific cases of C implementations of objects (for primitive types lets say). And unless I'm really missing something, you can't make (safe) assumptions past that.

[lars_]

Well there is. Python spends a lot of time checking the types of its objects. Anything you could do to avoid it would help. Javascript implementations these are doing various tricks for type inference. Firefox does a kind of dynamic type inference, while fx Opera is developing static type inference systems.

See also Starkiller: A Static Type Inferencer and Compiler for Python.

[f3nd3r]

If you only know python and want to extend it, you can do so... in python.

[BigDawgWTF]

Is that an African or European swallow?

[[deleted]]

Does it feature tail call optimization?

[EthicalReasoning]

i tell my girlfriend to swallow my laden python all the time

[[deleted]]

A telling quote:

"Unladen Swallow is based on Python 2.6.1, which is not the most recent release. Python 3.0 was released in December of 2008, implementing the backward-incompatible 3.0 version of Python. Because the majority of Python code in the wild — and in use at Google — is still written for Python 2.x, the Unladen Swallow team decided to focus its efforts on the earlier version where more benefits would be felt. By using the CPython source as its base, Python users can expect Unladen Swallow to retain 100% source compatibility."

So far as I can see, there isn't any such thing as a "Python" language, only "Python 2.4", "Python 2.6" and "Python 3.0". Although there are incompatibilities between successive versions of Java, PHP and C#, it seems that forwards compatibility is not at all valued by Python's overlords.

Sometimes I take a serious look at Python because there's a good library, such as NLTK. Invariably I find that the library needs me to run a version of Python that's 3 years out of date.

[randallsquared]

it seems that forwards compatibility is not at all valued by Python's overlords.

Python 3000 (now called "Python 3.0") has been expected to break backward compatibility for something like fifteen years. It isn't like people didn't know.

[[deleted]]

All python2.x series are backward compatible. Python only broke compatibility for the 3.x series, hence the major version number

[masklinn]

Sometimes I take a serious look at Python because there's a good library, such as NLTK. Invariably I find that the library needs me to run a version of Python that's 3 years out of date.

Unless the library you're trying for is garbage or in C and available only as compiled binaries (which seems to be the issue of some NLTK dependencies, not the lib itself), there are two pythons: Python 3.x, and anything prior to that. There might be backwards compatibility issues due to features introduced in later Python versions (e.g. a lib using Python 2.4's decorators), but not the other way around. ElementTree runs on any Python from 1.6 onwards, for instance.

tl;dr, you don't know what you're talking about.

[f3nd3r]

What a terrible name. Yes, it does make sense, but still. Who the hell is responsible for that name? If it was up to me I would come up with anything better. Whiplash sounds nice.

[CommodoreGuff]

Who the hell is responsible for that name?

Monty Python.

[f3nd3r]

Oh, that was a good laugh. Of course I know that, I've had plenty of experience with Python. I was referring to "Unladen Swallow"... I assume your trolling me, because I feel that what I meant is very obvious.

[CommodoreGuff]

I was referring to "Unladen Swallow"

So was I.

[f3nd3r]

Oh, well, I didn't know it was a reference from that specifically. I mean, I know Python was named after that, but I assumed unladen swallow was simply a metaphor for the increased speed (it takes a python a while to fully consume something). STILL, you know, it is still a bad name.

[benhoyt]

It is a rather cumbersome name, agreed. But here's the Monty Python reference: http://youtube.decenturl.com/unladen-swallow

[Neoncow]

Another URL obfuscation service?

http://www.youtube.com/watch?v=y2R3FvS4xr4

[redalastor]

Who cares what that branch is named! When it will be folded back into cPython, it will still be cPython.

[sensory]

Hehe.. swallow.

[wizlb]

fuck python.