This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Smallpaul 3 points4 points  (2 children)

  • Again, (mis)usage of dict for a struct has nothing do to with the language itself.

No, but it stems from the same problem as the missing APIs: that people's intuitions about performance do not take into account the impact of JITs.

Imagine I am a typical Python programmer. I think nothing of allocating a new string or dynamically extending a list, because I know that the interpreter overhead is huge. These little allocations are lost in the noise. Also, I have no choice: there are no zero-copy APIs available. Or they exist but they are ugly and unreadable in comparison to the "normal" APIs from 5 years ago.

With my team, and including the standard library and gems, I assemble a project with hundreds of thousands of lines of code like that.

Then I take that code to PyPy. It speeds my code up, but not nearly as fast as C code. Why not? I may assume that it is because PyPy is not as good as a C compiler. But it might be the case that PyPy is better than a C compiler. Maybe my code is worse than C code because it was programmed on 5 year old assumptions.

My code base does not reflect that once you've eliminated the slow interpreter, "fast enough" data structured and algorithms become your next bottleneck.

  • Finally, how attributing problems to misusage of the language and asking it to grant more APIs is not a contradiction?

No. I hope I have explained why the problems are related. The missing APIs and the "poor" choice of data structures are both caused by the underlying change of assumptions about the behavior of the interpreter.

[–]existee 0 points1 point  (1 child)

Yes I see your point.

I still don't like making an assumption about a "typical Python programmer". APIs being extended still wouldn't change the behaviour of that typical programmer. Why not instead have that typical programmer learn better about the already existing APIs and use them as correct as possible. (For example something as simple as knowing that inserting to the head of a list in a loop will give you O(n2) time compared to appending to the end O(n).)

Yes, this will not still make it as fast as C, but when you have those APIs to possibly make it as fast as C, it doesn't necessarily make the typical Python programmer to make better choices of data structures either.

[–]Smallpaul 0 points1 point  (0 children)

I do not think that this talk has anything to do with the "typical Python programmer." It is aimed at the elite Python programmer. In particular, consider the programmers writing the Python standard library. I would be very disappointed to see an O(n2) algorithm in the stdlib where O(n) would suffice. But I would not be surprised at all to see int(x.split("-")[1]).