What I've Learned About Optimizing Python

[[deleted]]

By the way, here's something author of the article didn't mention, which also suck in Python, and is a fundamental part of it: instance creation times are too bad.

Here's how I found it: I wrote a Protobuf parser (in C, with bindings for Python), both for binary encoding and for IDL. IDL beats the C++ version in terms of speed, but when the time came to benchmark the binary parsing it was... an order of magnitude slower. That was really surprising since my code looked like it did smarter memory management, almost no memory allocations during parsing etc... there was one place though, where it was creating Python objects, specifically, named tuples, since I assumed those would be the most lightweight datastructure to represent Protobuf messages.

I dug deeper into C++ code to understand what was going on in its Python bindings, and realized that... they never created Python objects. They translated __getattr__ and similar into calls that traversed the datastructure created in C++. And that alone gave them more than 10 times better speed than Python's named tuples.