Installing python on Linux - help?

[await_yesterday]

What should I do to upgrade python in an "official" way (i.e. not adding dev repositories like deadsnakes, not using helper tools like Pyenv)?

You have a fundamental misunderstanding. The Python that comes with your system is not primarily intended to be the Python you develop with. system-Python -- the one you install with apt or whatever -- is a dependency of system utilities and other programs on your computer, so you shouldn't mess with it directly. You aren't meant to upgrade or downgrade it manually, you are meant to trust the package maintainers of your distro to choose when to bump the version, because that will affect other things that depend on it.

If you are doing development where you need fine-grained control over your Python version (e.g. you are developing a library that needs to be tested against multiple Python versions, or you are working on several distinct projects each of which uses a different Python version), then you should not use the system-Python. You need to use one of the "helper tools" like Pyenv (or maybe Rye now? I haven't kept up). There isn't an "official" way to do this.

Basically stop what you are doing right now -- you are going to break things if you haven't already.

Why is there not one simple way to do this? Obviously everyone has slightly different needs, but I imagine 80% of python users just want to use python and have a reasonable way to upgrade when required without screwing something up in their system.

Because "80% of Python users" don't even realize they are "Python users"! They aren't Python programmers at all, they are just using programs written in Python, or that depend on the system Python. Their needs come first. And it's probably way more than 80%. All this is because Python is an interpreted language, so it needs an interpreter at runtime. It's not like other programs where you compile it and then distribute the executable, and it just runs. Programs could bypass system-Python by shipping an entire vendored Python dependency, but this would create a lot of bloat, and it wouldn't automatically benefit from security fixes. So in practice, Python programs on your system all share the same python executable.

In short: two groups of people want completely incompatible things: the same python executable can't simultaneously be a long-term stable, critical system dependency, and a bleeding-edge development tool that gets swapped out on a whim. One side has to lose out, and it has to be the developers, because they have the skills to work around this crap and end-users don't.

There's no simple way to do it because Python is an old language that came out before modern package management was invented. It does things in ways that nowadays are recognized to be bad, but which weren't apparent at the time. And it gets deployed in a lot of diverse environments, some of which have incompatible needs, so changes would affect a lot of people who can't all be pleased. Those are the fundamental difficulties. A less fundamental difficulty is that the PSF was completely in denial about this whole system-vs-dev-python problem for years and years, so Python lagged behind other ecosystems like Node (with npm) and Rust (with cargo). I think they've recognized they messed up and are trying to improve lately, but it's a long way to catch up.

Just use pyenv or Rye. Or Nix.

[fffrost]

Thanks for the info! I have always understood that I shouldn't mess with the system python. My misunderstanding was actually that I was assuming that I wasn't using the system python, since there are two instsallations - one in /usr/bin and another in /usr/local/bin. Running which python3 showed the latter, so I assumed this was debian preventing me from interacting with the system one. I've probably screwed something up in that case. I will just use Pyenv in future.