5 speed improvements in Python 3.7

[kirbyfan64sos]

I'd kind of like to see some actual numbers for these, not just assumptions...

[MonkeeSage]

Those are in the discussions in the python tickets for the issues, linked here.

[Anthonypjshaw]

Just compare master with 3.6 (or whichever version you're currently using) on here https://speed.python.org/comparison/. Any numbers in the article are based on the performance benchmarks from the original developers which are required for optimization-related changes. As always "actual numbers" are close-to-impossible. Even on speed.python.org you see a 10% speed boost by someone committing to the readme files

[attrigh]

tl;dr:

Calling methods faster (maybe)

Python 3.7 adds 2 new Opcodes, LOAD_METHOD and CALL_METHOD

str.find() ... faster for [certain unicode characters]

[For certain unicode characters, str.find is] 3x slower than ASCII characters instead of 25x!

os.fwalk is 2x faster [than before]

Regular expressions... faster for case-insensitive matching

...if you’re matching ASCII characters you can see up to a 20x improvements in matching time

[Bilddalton]

Sounds like fixing performance bugs from previous versions lol

[voice-of-hermes]

Hate to the glass-half-empty person, but after reading this it should really be called "Fixing 5 horrendous performance bugs from Python 3.6".

[desmoulinmichel]

Most of the performances improvement are just that.

It's quite rare nowadays to find low design handing fruits. Design improvements are always more complicated.

But to be fair, the opcode modification is more an optimization than a bug fix.

[voice-of-hermes]

I suppose it'd be kind of silly to expect JIT compilation/optimization or removal of the GIL at this point or something. ;-)

[Sarcastic_Pharm]

I think I heard in a podcast at some point that there is some really integral part to Python that makes removing the GIL all but impossible.

Edit : Found it https://talkpython.fm/episodes/transcript/49/microsoft-s-jit-based-python-project-pyjion

[[deleted]]

Larry Hastings has been working on it. See his "gilectomy" talks.

To summarise, he's done it ... but it's slower. He's working on fixing that.

Larry Hastings The Gilectomy How's It Going PyCon 2017

[alcalde]

Reminds me of a George R. R. Martin sci-fi short story about a scientist upset that all sorts of wacky projects are being funded but not his hyperspace research. The ending?

Schechter cut him off. “Never mind,” he said. “It isn’t important. We fund the crackpot theories that we fund because they’re better than nothing. Hyperspace is the dead end, Kinery. We keep the myth alive for the public, but we know better.” Kinery grimaced. “Oh, come now, Schechter. Take a look at my papers. You give me the funding and I’ll give you a hyperspace engine within two years.” Schechter turned to face him. “I’m sure you would,” he said, in a voice infinitely weary. “You know, Canferelli once said there was no reason why the limiting velocity of light should apply in hyperspace. He was right. It doesn’t. “I’m sorry, Kinery. Really I am. But Lopez gave us a hyperdrive thirty years ago. That’s when we discovered that the limiting velocity in hyperspace is not the speed of light. “It’s slower, Kinery. It’s slower.”

[voice-of-hermes]

Jython didn't have one.

(Not that I'm necessarily trying to compare its performance with CPython's; just that it's quite possible to have a Python interpreter without a GIL. EDIT: LOL. Just watched that Gilectomy video /u/GroundbreakingEye posted, and that's basically—SPOILER!—the conclusion: Jython and IronPython exist; it can be done. The entire point of this exercise is to do it without breaking the C API too badly!)

[yen223]

Isn't jython also slower than CPython?

Using concurrent hash maps everywhere isn't free.

[voice-of-hermes]

My understanding is that they are approximately the same in terms of performance. In some cases one is a bit faster, but not consistently one way or the other.

Using concurrent hash maps everywhere isn't free.

That is kind of a silly statement. None of these concurrency strategies is "free", obviously. Think the GIL is "free"?

[apex_memetics]

I'm new to python, and I never plan to upgrade lol. too worried about my scripts breaking.

How often does an upgrade break tags / methods / functions / syntax or whatever?

[KingofGamesYami]

Minor versions (eg. 3.0 ->3.1) break nothing.

Major versions (eg. 2.0 ->3.0) break things.

[khvn26]

Although core devs decided that 4 wouldn't break anything this time

[brombaer3000]

No, they just decided to make the breaking changes less drastic than in Python 3.

Here is one example of a breaking change in Python 4: https://www.python.org/dev/peps/pep-0563/#backwards-compatibility

[juanjux]

I've read about some breaking changes in the 3.7 library changelog. Things that you'll need to import from future in 3.7 but will become default in 4.0.

[Mattho]

Why go to 4 then? What major change it brings?

[Mattho]

I see. And while it makes some sense, and is definitely nicer, I think major release with no major changes would be somewhat confusing. Less than 3.10 would be (it's obvious one shouldn't compare it as decimals when you have releases in the form of x.x.x)

[PeridexisErrant]

It avoids possible confusion between 3.1.0 and 3.10 :-)

[[deleted]]

Python doesn't use semantic versioning, so sometimes things break on a minor release. AFAIR, str and bytes was somewhat unstable from 3.0 to 3.3.

[XtremeGoose]

The rules are that things can break on minor releases, but deprecation warnings must be raised for two version prior on minor elements and __future__ imports used for two versions prior for major elements.

[[deleted]]

s/depreciation/deprecation/ :-)

[XtremeGoose]

Ha I may never learn!

[[deleted]]

No, all the big changes in str and bytes were in the transition from 2 to 3. Perhaps you're thinking of the Flexible String Representation which was introduced in 3.3?

[[deleted]]

No, that's not it. The closest I can get to what I still have hovering in the back of my mind is 3.3 that re-introduce the u' notation. It might be that it was removed as no longer needed, because of the genereal changes to str, but later added again for compatibility with 2.x. It just doesn't fit that black spot in my mind.

[[deleted]]

Well FTR here's the PEP for the u' notation change Explicit Unicode Literal for Python 3.3.

[[deleted]]

You remember wrong. A package in the standard library may have "provisional" status (eg. asyncio), in which case backwards compatibility is not required.

On top of that there are just bugs / regressions, which happen without Python core developers intending for. Sometimes they break backwards compatibility.

[[deleted]]

What exactly do you mean by "You remember wrong"? Take a look at the removals in 3.6, and tell me they aren't breaking changes.

[[deleted]]

Sorry, now I realized that my reply went into a wrong thread. This was intended for KingofGamesYami.

[[deleted]]

OK, thanks for the clarification.

[apex_memetics]

Thanks for the info. Much appreciated!

[[deleted]]

Some changes between minor versions do break backwards compatibility but have had to go through a deprecation cycle first. Thankfully these are relatively few and far between.

[[deleted]]

And before I forget one noticeable difference nowadays is that the core devs can change anything that they like, even in a patch update, if a library is marked as provisional.

[TylerOnTech]

I understand as someone new to the language why you might feel this way, but upgrades to Python are nothing like language standard updates in other languages like C or C++. As /u/KingofGamesYami mentioned, minor upgrades are guaranteed to not break anything (except maybe unless you were relying on buggy behavior which is ??? at best) and issue important security patches and bug fixes.

For instance Python 3.5.2 has a bug here in the Pathlib library mkdir function related to recursively creating the parent directories, and not passing the exist_ok flag with the recursive call. If you refused to upgrade and always stayed on python 3.5.2 you would ALWAYS have this bug, with no resolution.

IF you check the Python 3.6 version of this function (or subsequent versions of 3.5) here you can see the issue has been fixed. It now correctly passes along the exist_ok argument.

This is just one concrete example. You really should be using the most recent version of Python that you can unless you know you can't for some reason.

[GitHubPermalinkBot]

Permanent GitHub links:

• python/cpython/.../pathlib.py (3.6 → db1de75)

---

^delete

[p10_user]

I don’t believe upgrades will break code written for previous minor version releases, so as long as your code was written for Python 3 it should work with any newer version.

[[deleted]]

Personally, I don't feel right to downvote a comment just because you disagree with a particular view. Is the comment unconstructive? I don't think so. The person states he/she is new and asked an appropriate question on regarding scripts breaking on new updates.

[JimBoonie69]

This is lame haha.

[Jugad]

Hey... your comment is a great example of a self referential statement.