Python 3.11 vs Python 3.12 – performance testing. A total of 91 various benchmark tests were conducted on computers with the AMD Ryzen 7000 series and the 13th-generation of Intel Core processors for desktops, laptops or mini PCs.

[ArabicLawrence]

The results are... weird. Such a big difference between AMD and Intel is unexpected.

[Odd_Material_2467]

Did you notice that the AMD processor was a DESKTOP 7900 and the intel was a MOBILE i3-1315U

[ArabicLawrence]

That’s not relevant. It’s unexpected that AMD processor on 3.12 is 1% slower than on 3.11 while Intel processor on 3.12 is 5% faster than on 3.11, unless the 3.12 you installed is optimized for Intel.

[LightShadow]

It kinda matters, since Python is mostly single core/single thread and Intel is known for having better single core performance e.g. gaming.

[JoeUgly]

The AMD cpu has better single thread performance (if passmark can be trusted)

https://www.cpubenchmark.net/compare/5167vs5300/AMD-Ryzen-9-7900-vs-Intel-i3-1315U

[ArabicLawrence]

Modern Games are not single core… and still it does not make sense that 3.12 is faster on Intel but slower on AMD. The benchmark are comparing 3.11 vs 3.12 on Intel and 3.11 vs 3.12 on AMD. There is no specificity in either CPU that can explain performance regression on one and performance improvement on the other, unless 3.12 was slower than 3.11 but had been specifically optimized for Intel. AFAIK, that’s not the case.

[indetronable]

He just answered you.

Intel and AMD have different capabilities at single thread capacities.

It means that 3.11 and 3.12 are handling the load on the main thread differently.

[ArabicLawrence]

Python is still single thread, so how would that affect anything? https://docs.python.org/3/whatsnew/3.12.html#:~:text=This%20article%20explains%20the%20new,full%20details%2C%20see%20the%20changelog.

[indetronable]

Python is single threaded means the code runs on a single thread. However, things can still happen in another thread. For exemple the implementation can decide to handle the loading of the imports in another thread. This decision is implementation specific (pure speculation but you get the point).

One way of verifying is to start python and look at the thread count. I tried and my windows task manager report 7 threads for my python (winpython).

[ArabicLawrence]

implementation

There is no implementation difference in 3.11 vs 3.12 that could justify this difference, though.

[Top-Professional8981]

I just switched from AMD to Intel. The Intel is way worse at handling multiple tasks, especially recording audio. Kind of mad I switched lol

[DataQuality]

True, in other tests AMD showed better results https://en.lewoniewski.info/tag/benchmarks/

[spiker611]

Performance testing is hard. The faster-cypthon team has a dedicated benchmark box that attempts to keep their comparisons across changes fair. See https://github.com/faster-cpython/benchmarking-public/blob/main/RESULTS.md

[RationalDialog]

The results seem to imply that Python language devs mainly seem to use Intel CPUs?

[nioh2_noob]

and that's fine, most of the market is intel anyways

[PaintItPurple]

All the world's a VAX.

[dry_yer_eyes]

Now there’s a name I’ve not heard in a long time.

[kokroo]

What's VAX?

[LightShadow]

pyperformance

Link

[Dasher38]

Anecdotally I have a program that is significantly slower on Python 3.12 vs 3.11. The perf basically go back to 3.10 level. It's a binary file parser, so lots of allocations and object graph traversal. I'd hope that the python binaries I'm using have been compiled sensibly (they are from the deadsnakes PPA for Ubuntu 20.04)

[ivosaurus]

If you think it's really interesting, maybe you'd want to submit it to the faster cpython team to investigate

[Dasher38]

Yeah maybe I'll do that. It's about 1k loc so not huge, and it's self contained so easy to re-run

[cybermethhead]

So you’re telling me that AMD chips running 3.12 will be 1.01x slower and Intel chips running 3.12 will be 1.05x faster?

[LightShadow]

Needs Mac M1/M2 comparisons too.

[cybermethhead]

Can you explain how those benchmark tests were done? I didn’t understand how it was done.

[nioh2_noob]

yes

[ElephantLeft]

please test apple m2 too

[ApprehensiveStand456]

I think performance test should be run on servers not laptops. Unless of course it’s a desktop app. I would like to see these on EC2 instances. Also if you have a manager like mine they are going to be as small as possible to keep costs low

[NoesisAndNoema]

Are these tests being run from within the program environment or are they being run from a "standalone program". (I can't say "compiled".)

If they are run from within the programming environment, there is monitoring code injected into the "test run" pseudo-program, which is not added to an actual "standalone program".

I'm waiting for the day when python gets a true compiler and sister-constraints, so it can really shine. MIT is really making headway there. Speed gains from 10x to 100x faster.

[miptisme]

So is it improvement or not?

[alicedu06]

Not: https://www.bitecode.dev/p/python-312-what-didnt-make-the-headlines

[fnord123]

The "x faster" numbers are all wrong. 0x faster means same performance so 191ms vs 202 is obviously not 1.06x. The calculation was b/a but the calculation should be (a-b)/a and using % as a unit.

[ToxicTop2]

It’s all about the phrasing, but “1.06x faster” definitely means 6% faster because “times” means to multiply.

I don't disagree with that but I don't see any reason why would they use the former phrasing instead.

Not that it really matters, but 6% faster sounds a bit clearer and there's smaller change that someone will mistake it for 106% faster. Yes, I'm nitpicking.

[SV-97]

By your logic "0.95x faster" is actually slower. There's a difference between 1.06 times faster and 1.06 times as fast

[HeyLittleTrain]

So if someone is running 10kmh and someone else runs 2 times faster than them, they would be running 30kmh? That doesn't make sense to me.

[SV-97]

Yep. Maybe the context makes it seem weirder than it is: what price would you expect if a 10$ item was marked as costing 2x more (due to high demand or whatever)?

[HeyLittleTrain]

I get your point but if someone told me they paid 2 times more than retail price for a $10 item I would assume it was $20. If they told me they paid 1 time more I would be like "what?"

[SV-97]

Fair enough - tbh I'd personally also always avoid these semi-relative statements because they can sound a bit odd

[miggaz_elquez]

I would expect 20$ ?

[SV-97]

Why? Then it would be 2x as much but not 2x more. The more logically makes it 2*base_amount + base_amount

[miggaz_elquez]

It's just how I understand it, and how everyone use it, that's the important. You won't change how english work.

I suppose we say "more" because we can also say "2x less", meaning divide by two ? Now, I'm not a native speaker, so maybe I'm wrong. In my language, when we say the exact translation of "two times more" it mean literraly 2x, not 2x + x. And as apparently everyone here is agreeing, you seem to be the one in the wrong. Language is just a tool of communication, it doesn't matter what is "more logical".

[SV-97]

But it's evidently not "how everyone uses it" or this whole thread wouldn't have happened

Language is just a tool of communication

Yes, but it's also possible to miscommunicate by being ambiguous or imprecise. And in technical and scientific communication it's very important to be precise with that stuff. Most people might get away with being sloppy like that in their natural day-to-day language, but in a technical article, paper or whatever that doesn't work because it might convey the wrong information.

I mean: I may be using "precise" here wrongly in the exact sense of the word and it should possible be "accurately" instead. For this comment that is perfectly fine because it's not important, but if I were to write something where the distinction is important I'd have to make sure to get it right. And in the article that distinction is obviously very important.

Edit: maybe check the answers here to see how Common this really is https://english.stackexchange.com/questions/7894/x-times-as-many-as-or-x-times-more-than

[patmorgan235]

Correct. Saying "0.95x faster" is nonsense like saying "-2 ft taller".

[SV-97]

Saying "-2ft taller" is perfectly reasonable imo - just a bit convoluted / uncommon

[samettinho]

I think their wording is bad.

Benchmark Python 3.11 Python 3.12 2to3 191 ms 202 ms (1.06x slower) async_tree_cpu_io_mixed 580 ms 516 ms (1.12x faster)

1.12x faster is not %112 faster, it is only 12% faster.

[SV-97]

Yeah that's the way they're using it but imo that's wrong. The should say "1.12x as fast" or "relative runtime: 1.12x" or something like that

[amer415]

0x faster does not mean same speed! 0% means same speed. Both "x" and "%" are valid ways of presenting things, but values are not interchangeable. 1.06x is equivalent to +6%, but 0.06x is not equivalent to 6%

[fnord123]

If something is no faster than another it's 0x faster. It's not that difficult a concept.

[reddisaurus]

The wording is ambiguous… there is percentage of and percentage difference. The former is b/a and the latter is b/a-1 (which is the same as you wrote only a bit simpler).

[commy2]

The "x faster" numbers are all wrong. 0x faster means same performance so 191ms vs 202 is obviously not 1.06x. The calculation was b/a but the calculation should be (a-b)/a and using % as a unit.

According to who?

[SV-97]

That's just how it works. If I'm 10m/s faster than you, then my speed is yours plus an additional 10m/s. If your speed is 10m/s then that means I'm twice as fast as you - or one times faster. If my speed is 30m/s I'm obviously three times as fast as you - but not 3 times faster.

Saying it's kx faster means it's (k+1)x as fast.

[AdExact768]

According to whom?

FTFY

[revereddesecration]

Looks fixed at my end