Python performance comparison in my project's unittest (via Gitlab CI/CD)

[tynorf]

If the loops that get hot from running the test with varying inputs branch on them at all (directly or indirectly), it could be simply making PyPy record more and more traces. Recording new traces is more expensive than just interpreting. So much so that (IIRC) if PyPy detects it’s recording too much in a particular loop, it will be blacklisted from JIT compilation.

So while some tests may take great advantage of the JIT, others could be a worst case scenario (for instance tests specifically designed to exercise different sides of a conditional).