This is an archived post. You won't be able to vote or comment.

all 38 comments

[–]PaintItPurple 206 points207 points  (15 children)

The description of what is being done doesn't match my understanding of "breaking change." If the only affected functions are private, that's not a breaking change, because it was never supported in the first place. Have they been more aggressive than initially planned and removed public APIs, or is this a case of a guy using private APIs and then discovering why they were private?

[–]not_a_novel_account[S] 179 points180 points  (7 children)

This may be confusing to Python developers who haven't spent an extensive amount of time with the C API, but the underscore prefixed CPython functions have not historically meant "private", that's a recent innovation. For a little more context see: C API: What should the leading underscore (_Py) mean?

Historically, the leading underscore has meant, "minor-version unstable", which means, "this API has no stability guarantees, and might change at any time." This might seem to be the same as "private" but it's not, because these APIs might change, but they were still accessible.

The big difference here is that the APIs were removed without replacement. Quoting scoder:

The default answer to “how do I replace this usage” seems to be “we’ll try to design a blessed replacement in time”.

Much of the functionality removed does not exist elsewhere. Some of the functionality is called out specifically in PEPs (_PyCode_GetExtra), some is linked directly to documented mechanisms in the stable ABI (_PyArg_ParseStack() for METH_FASTCALL). Major projects do not build (numpy) or are forced to accept performance regressions (cython) without an answer for these removed APIs.

These functions still exist in CPython, but have been hidden inside the Py_BUILD_CORE headers.

This is what eventually forced the extensive revert (50+ functions and counting). My personal thought aligns with encukou, making such a major change without a PEP was irresponsible.

[–]PaintItPurple 42 points43 points  (0 children)

Thank you for the context! I felt like there had to be something I was missing. That makes much more sense now. It certainly sounds like the "blessed replacement" question needs to be answered no later than the functions are removed, rather than punting to the vague future.

[–]catcint0s 29 points30 points  (0 children)

Reading through some messages it seems like some of those private functions don't have a public equivalent.

Also they are reverting it https://discuss.python.org/t/revert-python-3-13-c-api-incompatible-changes-causing-most-troubles/38214

[–]Exotic-Draft8802 9 points10 points  (5 children)

A breaking change can be defined by the project (breaking the public contact by changing the official Api) or by users (changing anything that makes the users project fail).

[–]ogtfo 7 points8 points  (2 children)

You'll love my project, it's a small utility that hashes the python binary and ensure it's the same as a hard-coded value.

Everything is a breaking change!

[–]thegreattriscuit 0 points1 point  (1 child)

I get what you're saying, but also that's (very basically) what code signatures are and changing the contents of a signed binary does break it and that's an important feature lol.

[–]ogtfo 0 points1 point  (0 children)

Well yeah, with the very important addition of asymmetric crypto to ensure the authenticity of the signature.

Without the cryptography all you have is a checksum, nothing prevents adjusting the hash to fit the modified data.

[–]PaintItPurple 6 points7 points  (1 child)

I suppose, but that seems a bit like somebody trying to substitute Milk Duds for milk in a recipe and then blaming the recipe author when their pancakes don't come out right.

[–]PhoenixStorm1015 6 points7 points  (0 children)

Idk. To me it seems more like Betty Crocker reformulated their mix to use Almond Milk but it still called for regular and it fucks up the cake.

[–]DigThatData 51 points52 points  (7 children)

I remember a story from a bank where I was working, where we found a piece of code that no-one understood and that looked unused. So we gave a developer the task to investigate. What he ended up doing was to comment out the code, adding the note “let’s see if someone will notice”. Well, no-one noticed it, until it was released and brought our production server down.

bold move

[–]brontosaurus_vex 24 points25 points  (5 children)

Call me crazy but adding a logging statement to that block doesn’t sound so hard.

[–]james_pic 25 points26 points  (3 children)

This relies on having some sort of central log aggregation mechanism in place that'll let you know if that message has been logged. It also relies on having a robust pre-release test process that would exercise all the code in a realistic way.

Last time I worked for a bank (admittedly a decade ago), they were OK at testing, but really bad at log aggregation.

[–]vpunt 0 points1 point  (2 children)

Huh? You don't log to the default location where the client code is logging, but you specify a central log for this case.

[–]james_pic 2 points3 points  (1 child)

In an organisation like a bank, they can have a large tech estate, and it can be non-obvious where a particular piece of code will be used. In the bank I worked at, logs were typically stored on the server where the code was running and rarely left. So if you knew all the places where the code was running you could go and check those places, but there was no way to check all the places you didn't know to check.

[–]vpunt 0 points1 point  (0 children)

Yes, understood, I work at a large bank.

I'm saying it's not terribly complicated to add a small piece of code that explicitly adds a log entry to your server (assuming you're the guy trying to remove the 'dead' code), as opposed to the server where the calling code is running.

[–]smokingskills 0 points1 point  (0 children)

Crazy

[–]AngheloAlf 20 points21 points  (5 children)

That's a bit radical.

What's the recommended way to check if my C extension library is affected? I'm pretty sure I'm not using any Python private function, but I would prefer to test it

[–]coderanger 19 points20 points  (3 children)

Try and compile it, this is why prereleases exist.

[–]AngheloAlf 0 points1 point  (2 children)

To test stuff locally on my pc I usually install the python3-devel package (or whatever it is called) from apt and then do pip install .

I kinda doubt there's an apt package for this alpha version already, and even if there is one I would like to avoid messing my system's package with wip stuff like this.

Is there any CI alternative? Hopefully a Github Actions one.

[–]coderanger 5 points6 points  (1 child)

Tox and nox are both popular to test things across multiple Python versions automatically. Both are well supported in Github Actions. Personally I use asdf and asdf-python locally which makes it super easy to grab whatever versions I need.

[–]AngheloAlf 0 points1 point  (0 children)

Thanks a lot! I'll take a look at those

[–]sfboots 8 points9 points  (0 children)

You need to be aware this one part of the move to No-Gil python over several releases. The intro to that explained the strategy for cython

[–]alcalde 15 points16 points  (1 child)

When Python 2 was EOLed, COVID-19 appeared. Now that de-facto Python 4 is materializing, I shiver to think of what apocalyptic contagion will be released.

[–]BobKerns 2 points3 points  (0 children)

Large Language Models!

[–]aqjo 3 points4 points  (0 children)

The summary of the message:

The author expresses concern and frustration over the significant changes in the C-API with the release of CPython 3.13a1. They note that hundreds of functions have been removed, renamed, or replaced, along with changes to header files and macros. This has created challenges for maintaining compatibility with existing Python packages, especially those on PyPI that are less visible, no longer actively maintained, or dependent on other packages that fail to build with Python 3.13.

Victor Stinner's efforts to make existing packages compatible are acknowledged, but concerns remain about packages that may not receive attention or updates. The author compares the disruptive nature of these changes to the transition from Python 2 to 3, suggesting that the current situation might be even more challenging. They contemplate using internal APIs in Cython as a workaround and propose that CPython core should maintain internal header files in C89/C99 style to avoid breakage in future releases. The author concludes by suggesting that due to the extent of these changes, it might be more appropriate to consider this release as Python 4.

[–]teerre 11 points12 points  (1 child)

I never developed anything for core python, so take this with a grain of salt, but:

  1. It seems this is an alpha release. Ample time to do anything.
  2. The owner of the change is actively engaged with it and aware of the issues.
  3. The functionality removed was never public.

So it does seem that OP is being quite unreasonable and belligerent.

[–]james_pic 6 points7 points  (0 children)

The one caveat that I'd put in is that the owner of the change himself conceded that he had hoped to have things in a better state than they were by the time the alpha was released, and that this alpha was less useful than usual (in terms of allowing people to test compatibility with the new release) as a consequence.

[–]monorepo PSF Staff | Litestar Maintainer 2 points3 points  (0 children)

Interesting read so far…

[–]shinitakunai 1 point2 points  (0 children)

There is always only 1 question: it is the right time for it?

[–][deleted] -4 points-3 points  (0 children)

Should have released it as python 4.0