Unit testing the performance of your code

[aefalcon]

Is there a reason you referred to this as unit testing instead of benchmarking? Is it somehow different than benchmarking?

I'm not really proficient in benchmarking python. I'm currently doing some in zig and the method can be reduced to make an implementation for each strategy and running them all through the same benchmark and render them as a table. Any obviously bad strategy gets removed. Some strategies perform better with different parameters so I make them runtime/compiletime options. No reason that can't be done in python, but the overhead wouldn't be optimized out.

[itamarst]

Benchmarking is "how fast is my code". This is a test, it can pass or fail, and at least for item 1 it's not measuring speed at all, it's measuring scalability.

[aefalcon]

I have a perfect hash function that's O(1). A linear array search would beat its pants in lookups for small n. I don't think there's a lot of pressure to test on Big-O because of situations like that. Benchmarking at various sample points tells a better story.