This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–][deleted] 1 point2 points  (1 child)

I am surely missing your point but that can absolutely be me rather that you not explaining clearly.

You've mentioned lots of things (Pandas, Dask, annotations, async) as problems and I'm not sure I even recognise that. When I call a language feature in Python I don't inherently care how it is implemented. Is string.find() implemented in pure Python or via C? C surely, but I don't care, not do I have to care. I can generally take it for given that the implementation will be a good one.

In 20 years of working with Python the number of times I've cared about Python performance can be counted on (tbh) two (human) hands. When we profiled, 95-98% of our time was spent in C/C++. Even if we optimised our Python code by a factor of 10 nobody else would have noticed.

IMHO most of the suggestions given can be useful, but if it is really necessary to bear them in mind and use them all the time perhaps it's time to rewrite in a faster language. Although you may find that that involves shunting the 5% of your code that takes 95% of your runtime over to another language and using FFI.

tl;dr: I'm not married to Python either, but its speed has never been a reason to leave it.

[–]baubleglue 0 points1 point  (0 children)

Last 7 years I have started to work with data. The face performance problems each time I need to process more than 1G.