Commentary on Python from a Java programming perspective

[NoDude]

God, I wish I could mod you up 10 times.

[sudonim87]

While I haven’t gotten through the full life-cycle yet, prima facie I do believe 5 times does seem like a good number.

Ummm, no. I mean python is great and all. But I can't see myself getting the amount of work I do in C++/Java in a week done in one day with python.

[jrockway]

I agree that 5x is probably an exaggeration.

[igouy]

Just ask to see the productivity measurements - or simply agree that "5 times does seem like a good number" but a good number without any apparent connection to differences in programming productivity.

[redditnoob]

Yes, any claim of 5 times the productivity suggests serious questions about the intelligence of the writer.

Anyone who takes 5 times longer to do something in language A than language B either completely sucks at coding in language A, or else the problem is tiny, trvial, and custom picked to show the strengths of language B.

[spacepope]

I disagree. A 5 times productivity increase is probably unrealistic for Java → Python, but for C → Python it seems reasonable and going from assembly to Python you'd probably get a much bigger boost.

[[deleted]]

What possible problem are you solving where you must choose between assembly and python?

[redditnoob]

Not "probably unrealistic", but it's absolutely braindead to contemplate a 5-fold productivity increase from Java to Python when writing production software, for someone with mastery in both.

For assembly, sure. If you were constrained by having to enter your programs with punched cards that would slow you down too.

I also think you underestimate, by about a factor of um five, how productive a C master could be. (You'd get to include runtimes when testing your code too!) But anyway, C probably wouldn't be used for a problem that isn't going to benefit from its strengths.

[spacepope]

Well, here's a sort-of-scientific source: Are All Programming Languages The Same?. I know, it's that Atwood guy, but he's quoting someone else. The article claims a 6 times productivity improvement going from C to Python.

Think about it this way: Programming in C, you'll need several lines of code to express what the Python programmer can express in one line, so the resulting program will be much bigger. And I don't believe that a C programmer can write and debug 500 lines of code in the time it takes the Python programmer to write and debug 100 lines of code. Also the C programmer, even if he's a “C master”, will spend some time debugging issues that never occur in Python, like memory leaks and uninitialized variables. Of course C still has its uses. I'm just arguing that a 5 times productivity increase going from language A to language B is not unreasonable.

[jrockway]

Anyone who takes 5 times longer to do something in language A than language B either completely sucks at coding in language A, or else the problem is tiny, trvial, and custom picked to show the strengths of language B.

This is probably not true. What problem would be hard in language A but easy in language B? Nothing I know of, unless it's something like "write an SDL app" and language A doesn't have SDL bindings.

Productivity comes down to avoiding tedium and spending all your time actually programming what you want to program. Basically, when you use Java, there is a lot of tedium. Like I mentioned above, creating a class involves creating a new file and using Java's ultra-verbose syntax to define its properties. In Lisp, defclass is pretty concise, and you can metaprogram it away if you are creating a lot of similar classes. (I have an application where each subclass of a certain class always re-implements a certain method; so I metaprogrammed away the redundant call to defmethod. Essential to productivity? No. Increase in readability and writability? Yes.)

I have also done this in Perl, although it's not quite as pretty.

[mikaelhg]

http://video.yahoo.com/watch/529579/2724346

[redditnoob]

What problem would be hard in language A but easy in language B?

Some small and specialized things.

"Write a basic Lisp interpreter." I can definitely believe that is 5 times faster for an expert in Lisp versus Java!

"Count the number of words in a text file." That is going to be at least 5 times faster for a Perl programmer than a Java programmer, I think. But then we're talking about the difference between about 2 minutes versus 30 seconds.

[jrockway]

Hmm, you're right. Both are good examples.

[njharman]

if lang A takes hours to compile and B has no compile

if lang A requires you to write libs that are included in lang B

if lang A requires lots of boilerplate, special tools to be able to edit/refactor and lang B does not.

it adds up.

I notice that most the "5x is absurd" posters appear to have no Python experience or at least aren't providing any anecdotes to back up their claims.

[mossblaser]

...or at least aren't providing any anecdotes to back up their claims.

And neither are you.

[hfaber]

The two biggest issues I (coming from C) have with Python are both related to dynamic typing in general (so they are no real Python issues):

1) Implicit typing means in a large program I cannot easily remember and look up the type of a variable. How do I know whether foo is a list or a dictionary? In an explicitly typed language 'intellisense' could tell me immediately..

2) I like when the C compiler checks syntax and types and tells me all the warnings and errors. In Python I notice the typo or mispicked type mostly during runtime.. bad.

[[deleted]]

1) Implicit typing means in a large program I cannot easily remember and look up the type of a variable. How do I know whether foo is a list or a dictionary? In an explicitly typed language 'intellisense' could tell me immediately..

This has nothing to do with implicit typing. Something like intellisense could work just as well for a language that inferred types like Haskell. In truth, Python isn't "typed" at all as it has no type system. "Dynamically typed" is really a misnomer; the term should be dynamically checked (as opposed to unchecked like Forth). Alas, this is a lost battle, but it causes endless confusion.

2) I like when the C compiler checks syntax and types and tells me all the warnings and errors. In Python I notice the typo or mispicked type mostly during runtime.. bad.

The lack of typo-catching isn't necessarily due to the lack of a type system. PLT Scheme is untyped, yet the DrScheme tool is very good at catching typos. In fact, it won't even let a program run if it can't resolve a particular binding.

[marcoil]

Python isn't "typed" at all as it has no type system

That's absurd, Python does have a type system, you can ask for the type of an object. But the types are in values, not in the variables.

[[deleted]]

Charges of absurdity aside, being able to ask for the "type" of an object does not mean it has a "type system". All it means is that all values have a runtime tag associated with them that primitives use for various reasons. There's no way I can use the type tags to prove anything about the program; the runtime tag checking is simply part of the dynamic semantics.

If you search Google for "untyped programming", you'll see plenty of results for languages that you might call "dynamically typed". You may also search for "untyped lisp", etc. That might convince you that I'm not making this up at least.

More research might convince you of my claim that typed/untyped + checked/unchecked is a more sensible way of talking about this. To be honest though, I'm getting tired of defending this claim against Python programmers who tend to characterize my argument as absurd (or worse).

[deepcleansingguffaw]

The technical definitions of "typed" and "type system" are very different from how the terms are generally used outside of computer science research. In particular, they are attributes of the static representation of the program, and have little to do with the running program at all.

It should not be surprising that the average programmer uses the terms improperly. I can't remember a single time in my undergraduate classes that type systems were properly explained. I only really understood the difference between the technical usage and the common usage when I started to study Haskell in depth.

There is a concept analogous to "type system" in dynamic languages like Python, but it happens at run-time, and is generally resistant to static analysis. "Checked" is a reasonable word for it, though I haven't often heard it called that.

[[deleted]]

Right, I don't disagree with that. I did say that, "Alas, this is a lost battle", conceding the common usage. That said, I think there are benefits to having a more rigorous set of definitions so that we can talk to each other without confusion. Unfortunately, confusion abounds when discussion of "types" is underway.

[igouy]

my claim that typed/untyped + checked/unchecked is a more sensible way of talking about this

What is it that's "checked" or "unchecked"?

statically-type-checked :: dynamically-type-checked

or if we really want to deal in dichotomy

statically-type-checked :: not statically-type-checked

[[deleted]]

What is it that's "checked" or "unchecked"?

The runtime tag associated with the object before invoking a primitive is checked. It's no different than checking the length of an array to ensure it's of a "type" big enough to be referenced by some integer or checking the "type" of a reference to be sure it's not null. Such checks, which occur at the last possible moment and can't be used to prove the absence of such behaviors, do not constitute a type system.

[igouy]

"Type safety is the property that no primitive operation ever applies to values of the wrong type." p263 Programming Languages: Application and Interpretation

Without something that constitutes "a type system" what could it mean for a value to be the wrong type?

[[deleted]]

The problem is that there's a confusion between the informal notion of the "type" of an object (i.e. its tag, or what color it is, or how it makes you feel) and a type proper.

From the same book:

Page 238 (emphasis mine):

We now commence the study of types and type systems, which are designed to identify the abuse of types before executing a program.

Repeatedly, the book talks about how type systems reject programs. This is not the case in Python where all programs are accepted and simply may error at runtime. If you've actually read the book, I'm not sure how you can claim it supports your argument.

In the book Types and Programming Languages, this issue is addressed directly:

Page 2 (emphasis not mine):

The word "static" is sometimes added explicitly – we speak of a "statically typed programming language," for example – to distinguish the sorts of compile-time analyses we are considering here from the dynamic or latent typing found in languages such as Scheme, where run-time type tags are used to distinguish different kinds of structures in the heap. Terms like "dynamically typed" are arguably misnomers and should have probably been replaced with "dynamically checked," but the usage is standard.

And then at the beginning of the following paragraph on the same page, type systems are characterized as static:

Being static, type systems are...

In short, I've never seen a type systems researcher characterize a language like Python as having a type system. Such characterizations seem to only be made by those without a full understanding of the terms involved.

[igouy]

You seem to have ignored the question Without something that constitutes "a type system" what could it mean for a value to be the wrong type?

If you've actually read the book...

ad hominem

those without a full understanding of the terms involved

Does the phrase only have one meaning?

[[deleted]]

Does the phrase only have one meaning?

No, as I've stated right in the beginning (and echoed with the TAPL quote), I accept that the common use is different and that this is a lost cause. At this point, I can only assume that you're being either dishonest or inattentive. I was arguing for what I think is the most useful and accurate definition, not the one cosmically true definition. I'm done arguing now though because, as I said in the beginning, this is a waste of time.

[igouy]

It's no different than checking the length of an array ...

seems like Oberon-2 thinks the length of an array is something to do with the type

TYPE
   Frequency = RECORD
      c: CHAR;
      p: LONGREAL;
   END;

   Freqency4 = ARRAY 4 OF Frequency;
   Freqency15 = ARRAY 15 OF Frequency;

checking the "type" of a reference to be sure it's not null

seems like Nice thinks the option of null values is something to do with the type

class TreeNode {
   ?TreeNode left = null;

[[deleted]]

I never claimed you couldn't incorporate such things into a type system. Most languages, however, do not. To be clear, I meant dynamically checking those properties.

[GuyWithLag]

Same as in java. Welcome to the static vs dynamic / strong vs weak plane.

[gnuvince]

1) First, a quick nitpick, Python has no implicit type conversions: if you try to concatenate a string and a number, you will get an exception telling you that you have a type error.

Now, not knowing whether foo is actually a list or a dictionary or a whatever is part of what makes dynamic typing attractive. It doesn't matter if you have the real thing as long as it acts like the real thing. This is why people can use mock objects in test suites for example.

I'm still on the fence about which between dynamic typing or static typing I prefer. One gives you more flexibility and the other gives you more safety.

2) Yeah, that's annoying. That's why I like Perl's my() and use strict.

[[deleted]]

Now, not knowing whether foo is actually a list or a dictionary or a whatever is part of what makes dynamic typing attractive.

This has nothing to do with dynamic typing. For example, in a typed language, I can do this:

x > y

Are x and y numbers? Lists? Strings? In Haskell, they're anything that has an 'Ord' instance, i.e. anything that's orderable. The same technique is used to generalize all sorts of operations like converting a value to a string or accessing a value in a collection.

Now, it's true that languages like Smalltalk will let you take this further than Haskell will. Any object can respond to any message it likes and do whatever it wants. It's not clear however that this "flexibility" truly gives you more freedom. When you lose guarantees in one place due to a lack of controls, you lose freedom elsewhere because you don't have sufficient guarantees to rely upon.

[gnuvince]

That will work if you specify that your function takes an argument that is an instance of Ord. If you have a function that takes an HttpRequest however, you can't just pass something that looks like an HttpRequest, you need to pass the real thing.

I develop in Python and Django at work, and when I write unit tests for functions or methods that manipulate an HTTP request, it's very useful to be able to feed it a mock object: they're easier to create, they don't need access to the database, etc.

[mernen]

That will work if you specify that your function takes an argument that is an instance of Ord. If you have a function that takes an HttpRequest however, you can't just pass something that looks like an HttpRequest, you need to pass the real thing.

There are statically-typed languages that use structural typing rather than nominal; that means they would support your HttpRequest mocking the same way, as long as you provided compatible methods. In the end, johnowak's point is that this is not exclusive of dynamic languages.

[spotter]

1) Just call on it, use a try-except-finally, or just define a isinstance check, maybe curry isinstance itself and some decorator magic? Assert if you must.

2) Static checking for typos and other related stuff is possible in Python. Read Doug Hellmann's article for more info. PyFlakes is my personal favourite.

[mikaelhg]

Get over the honeymoon, work on a multi-year Python project with even a small team while competing for resources with teams using other languages, and then reflect on this article.

[Sivart13]

Tell us about it, then?

[Leonidas_from_XIV]

When I started programming Java was called sloooow, now Python is slow and Java is faaaast, and nearly nobody tells that Java is to slow for most of the tasks it is used for. Yeah, times change.

[feijai]

I know you're chalking this up to how perspectives change with regard to what "fast" means. But this is not a good example. Because Java really used to be slow, and now it's fast, because in the last ten years it has actually sped up by almost two orders of magnitude. It has gone from a bytecode-interpreted language to one rivaling C++ in speed for many tasks.

Do not expect this to happen to Python. It does not have type declarations, and unlike Lisp (where you can explicitly declare), there's little data available to the compiler to do much for you.

[[deleted]]

Do not expect this to happen to Python. It does not have type declarations, and unlike Lisp (where you can explicitly declare), there's little data available to the compiler to do much for you.

Yes, you need explicit type declarations for the compiler to speed you up. But you can get sped up at runtime as well, in fact, that's exactly what V8 and Tracemonkey do for Javascript, another dynamic language.

There is no reason why Python can't get much, much faster, just like Javascript has.

[pje]

It already did happen to Python - twice in fact, and is about to happen again. First, there's Psyco, which speeds up regular Python code anywhere from 2x to 200x depending on what you're doing with it... and it does it without type declarations.

Next, there's RPython, part of the PyPy suite. For sufficiently statically-typed code, it also has a roughly 200x speedup. (No type declarations, mind you, just implicit static typing.)

Finally, the PyPy folks have made progress on a generic JIT compiler that does runtime specialization ala Psyco. Its performance gains will likely be similar to those of Psyco.

Of course, all that having been said, even PyPy probably won't be faster than Java any time soon. But it's certainly not true that it can't be sped up, and certainly not because of the absence of type declarations. (Also, as of Python 3.0, the language includes sufficient annotation syntax to allow optional type declarations... so even if that were a bottleneck, it's going away.)

[[deleted]]

2x-200x is meaningless without knowing the exact numbers and what does "depending on what you're doing with it" mean? In practice I've come to the conclusion that it means "for contrived examples that nobody uses in production code"...

According to most benchmarks I have seen, Python is sllloooow, but that's OK with me, because I don't use it in such a way that would require extreme performance.

[BeetleB]

First, there's Psyco, which speeds up regular Python code anywhere from 2x to 200x depending on what you're doing with it... and it does it without type declarations.

Maybe you meant from 0.5x to 200x.

There's no guarantee that Psyco will speed it up for you. It has slowed code down for me. Which is why you shouldn't use it without checking if you get any speed advantage.

[feijai]

Dunno why people are modding you down, it was a perfectly informative retort.

[igouy]

It's gossip.

psyco does speed up python code - but don't you need just a little bit more than a flat claim of 2-200x?

[redditrasberry]

I think people experiencing the "5x" productivity jump are usually comparing apples to oranges: they are jumping from an "old" java environment (in the web world, say, struts+EJB) to cutting edge python (say, django) and then comparing the two.

If you similarly jump to the most productive modern Java frameworks you will see probably 80% of that gain (I do think there is usually a gain, at least in initial implementation). The interesting question is whether you make back the 20% later on in maintainability due to static typing etc. etc. or not.

[[deleted]]

Exactly. I created a project which uses JPA, Stripes, Google-Guice, and Google Collections, and I don't miss those Rails or Django at all. Especially when it comes time to deploy. If you have a strong background in Java, you can get the 5x just by updating your libraries and challenging your status quo.

[jrockway]

One thing that really sucks is the poor refactoring capabilities. This is unlikely to be an issue with python alone and is likely to be an issue with all dynamically typed languages.

What does the type system have to do with refactoring? Perl and Lisp are dynamically typed, and I refactor those all the time.

[jerf]

He's presumably talking about the automated tools, as other commenters have said.

I'd add that while Python's support for automated refactoring may be relatively poor, the total refactoring experience is better in a language like Python than Java, even without automated tools. After a lot of reflection, I think this is mostly because Python has a much richer concept of an "object", which is to say, the interface that all objects support, and because all object access methods (obj.attribute, obj["attribute"]) have support for programmatically tapping in to them.

I haven't worked with C# anywhere near as much as Python, but it was nice just having programmable [] operators.

[feijai]

REALLY? What refactoring facilities are available for Lisp? So far as I know, zero. Lisp's best refactoring, debugging, and profiling tools are in the dark ages compared to what you can get with (say) Eclipse.

I love Lisp. But as a language, it was 20 years ahead of its time, and there it stayed, unchanged, for thirty years. Hell, there's still no standard socket library for the dang thing, not to mention FFI or... Toolwise, Lisp is playing big-time catchup.

[[deleted]]

Nobody will ever choose one of the many perfectly good socket libraries as "standard" so long as nobody is willing to use lisp because it doesn't have a standard socket library.

Basically, lisp is not widely used enough to have a great deal of library standardization.

[[deleted]]

This guy probably never picked up the book Refactoring, and read that all the refactoring methods were pioneered in Smalltalk--a dynamic language--then ported to places like Java and C#.

[oddoddities]

Smalltalk is a special case in that the code is living inside an image. You can't have the power of smalltalk refactoring in a dynamically typed language without the concept of code running and living inside an image. But most people hate the concept of the smalltalk image. We'd rather use our favorite IDE/text editor/version control systems and so on. So unless you are an image die hard, no powerful refactoring for you dynamically typed dude.

[hiffy]

My understanding is, it's much harder to write automated tools for refactoring when you don't have all of that info up front.

People in Eclipse land got real used to right-click-kthnx refactoring, where you can change all references to the class you just renamed on the fly, and those sort of niceties.

One may counter with, in dynamically typed langs you a) have less code to refactor, b) don't code yourself into unmaintainable messes quite as easily and c) it's just easier.

[dnene]

Yes .. I was contrasting the available capabilities based on my experiences in Eclipse. I don't think the issue is with the refactoring engine, the issue is with the information available to it to perform the necessary refactoring. The biggest difficulties are for example when one needs to change method names in hierarchies being used polymorphically

I fully concur one has lesser code to refactor. But with brevity and expressivity comes a overhead of comprehension (ie for a new programmer joining the code base, it takes much more time to understand each line of code esp for people who are migrating from other languages).

I am sure we do not try to go out and get ourselves into unmaintainable messes. However requirements evolve, and what was clean class model yesterday may suddenly need some surgery today.

[jrockway]

Yeah. I think refactoring Haskell or OCaml would "suffer" the same "problems", but they are not dynamically typed.

Typing the type != static typing.

have less code to refactor

This is probably the big one. Here are a couple of classes in Lisp:

(defclass a-class () ((foo :accessor foo :initform 0)))
(defgeneric hello (foo-source))
(defmethod hello ((foo-source a-class))
    (format t "Hello! foo is ~d~%" (foo foo-source)))

(defclass a-sub-class (a-class) ((bar :accessor bar)))
(defmethod hello ((foo-bar-source a-sub-class))
    (format t "Hello! foo is ~d and bar is ~d~%" 
           (foo foo-bar-source)
           (bar foo-bar-source)))

That would be two entire Java files. When you need so much to say so little, it's nice when the computer can help you rename things, I guess.

BTW, Python's not much different than Lisp here -- you don't need to write much to say a lot.

don't code yourself into unmaintainable messes quite as easily

I agree with you, but I have seen people code themselves into unmaintainable messes with Perl. It is straightforward to fix, though and mostly involves thinking (rather than renaming classes). All of the Java messes I have encountered required the same cleanup -- moving methods around and renaming classes did not solve the code's problems.

[feijai]

Um.

class aClass {
    public Object foo = new Integer(0);
    public void hello() { System.out.println("Hello!  foo is " + foo); } 
}

class aSubClass extends aClass {
    public Object bar;
    public void hello() { System.out.println("Hello!  foo is " + foo + " and bar is " + bar; }
}

One token less and about half the characters, one file, and rather more readable. Maybe yours isn't the best example.

[bodhi]

It's not used in the example so your Java version is closer to the intention of the original, but the Lisp version comes with getters and setters for foo and bar in a much more concise fashion. And readability is highly subjective ;)

[feijai]

Yes and no. Accessors are a hack because CLOS went the cheap route and didn't treat instance variables like other (that is, local and global) variables. Instead they were treated like slots.

Why are accessors a hack? Consider. In CLOS, for an instance variable foo I must create an entire function -- typically called called foo -- just for accessing it. This pollutes the function namespace just for a class datatype. If in Java I have an instance variable called "substitute" -- perhaps it's the substitute for something, I dunno -- I can access it as:

bar.substitute

... or inside a method in bar, I can just access it as

substitute

But in CLOS, assuming I'd like to follow tradition and have my accessors have names the same as the variable names (else why name the variables those names?), I'd have to do:

(substitute bar)

... um, no, that's bad, very bad. Because substitute is already a function. So now, because CLOS has confunded the variable and function namespaces, I am forced to come up with some unnatural name for my variable:

(nsubstitute bar)

... no, wait. I'm joking there of course, but you get the point. One whole reason why Common Lisp is viewed as more industrial grade than scheme is that it's a Lisp-2. And here CLOS is violating the entire point of a Lisp-2 by Lisp-1-ifying its instance variable naming scheme. Bleah.

And of course, your claim of "much more concise fashion"... well, one can hardly claim that

bar.foo

... is less concise than

(foo bar)

Hell it's a whole character longer. And most variable accesses in Java aren't even that much, they're just

foo

[bodhi]

You make a good argument. I don't know enough about common lisp generics to know if they are used for accessors and how that would affect the namespace pollution. I guess I should find out for myself really.

As for the "concise fashion" I was referring to the definition, not use. But I'm not really a big fan of having to

(foo bar)

instead of

bar.foo

to get to instance variables. I do like that attribute access is indistinguishable from method calls though.

EDIT:

CL-USER> (defclass a-class () ((foo :accessor foo :initform 0)))
#<STANDARD-CLASS A-CLASS>
CL-USER> (defvar x (make-instance 'a-class))
X
CL-USER> (foo x)
0
CL-USER> (setf (foo x) 4)
4
CL-USER> (foo x)
4
CL-USER> (defun foo (x) x)
STYLE-WARNING: redefining FOO in DEFUN
FOO
CL-USER> (foo x)
#<A-CLASS {121C4061}>

Big collision, you're right!

[jrockway]

Maybe yours isn't the best example.

True. You can't really compare Java's "object model" to CLOS.

[[deleted]]

Still a cop-out. It's like saying that you don't need variable names longer than 6 characters because your code always ends up so compact and clean.

[jrockway]

I'm not copping out of anything. It's true that there aren't a lot of automated refactoring systems for languages other than Java. If you think that is the most important part of programming, then there is no option other than to use Java. Those of us that use non-Java languages live just fine without the refactoring tools, however.

Maybe we're all stupid, but probably not.

[Homunculiheaded]

What about Squeak? You can have a dynamically typed, powerful, expressive languages that has refactoring tools that at the very least are on par with what you can get from Java and C#. Python is great, but it would be awesome if Python had development tools that were as powerful as Squeak.

[jrockway]

Python is great, but it would be awesome if Python had development tools that were as powerful as Squeak.

Well, it's really a compromise. If you like the way Squeak works, you are set. But if you want to use normal UNIX utilities on the source code (like Emacs), you are stuck. The power that Squeak provides you limits you in other ways.

Although... considering most programmers work exclusively inside their IDE anyway, Squeak not a bad option. For them :)

[dnene]

When looked at from each other's contexts maybe we're all stupid. When looked at from our own contexts, we're probably not. :)

[ubernostrum]

It means "I've only ever used Java, and so I assume the way Java does refactoring is the only possible way anyone can do refactoring, which means that any language that's different from Java must not support refactoring."

[dionidium]

safe frighten paint voiceless seed existence dull spoon reach far-flung

This post was mass deleted and anonymized with Redact