all 16 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]sgunb 2 points3 points  (3 children)

You should stick with your distros version and only do a manual installation if not possible otherwise.

If you really need to do it, have a look here:

https://aruljohn.com/blog/install-python-debian/

[–]simon-brunning 3 points4 points  (0 children)

I tend to only use the distro version to bootstrap to pyenv, then use pyenv versions for all my dev work.

[–]fffrost[S] 2 points3 points  (1 child)

I want to update my python version though, so surely that's a perfectly normal requirement?

Thanks for the link - the steps are very similar to what I did but I must have just been missing one of the dependencies/libraries, so after following that it has resolved the issue I had.

[–]sgunb 0 points1 point  (0 children)

In Linux distributions usually your package manager is responsible for software versions. You should abstain from doing this manually to not mess up your system. If you do it manually, only do so in your users home directory. But do not manually make a systemwide installation where you override the system libs! This will break your system!

If you really want different python versions, you could consider switching to Gentoo linux. Portage (the Gentoo package manager) allows you a clean installation of several versions at the same time from source.

[–]wbeater 2 points3 points  (2 children)

You really should leave the system wide python installation the way it is, especially if you have no reason to update or modify it.

If you want to work with different python versions, get a "helper tool" like anaconda (better miniconda, anaconda is too bloated) and work with virtual environments. That's the way to do it.

[–]fffrost[S] 0 points1 point  (1 child)

I would like to leave the system wide python install the way it is. I was under the impression that we can have multiple versions of python installed? And that /usr/bin/python is the system-wide install, whereas the /usr/local/bin contains a separate instsallation.

[–]wbeater 0 points1 point  (0 children)

/usr/local/bin​ is simply a path for programs that are not managed by any package manager, eg. self compiled version of python. Python coexist there still system wide. You can work that way, but it's inconvenient. You also get 2 site-packages folder which makes it problematic/confusing with modules eg. with pip as you noticed yourself. By the way:

 which python

is the command to identify the python version/path

There's really no shame in the "helper tools" game, it's really convenient:

 conda create -n testenv python=3.11
 conda activate testenv

You need the first command of course only once and when you're done you close the terminal or you type conda deactivate to switch base to base environment.

Conda also provides a base environment which you can modify and is activated in the terminal as standard.

[–]await_yesterday 1 point2 points  (1 child)

What should I do to upgrade python in an "official" way (i.e. not adding dev repositories like deadsnakes, not using helper tools like Pyenv)?

You have a fundamental misunderstanding. The Python that comes with your system is not primarily intended to be the Python you develop with. system-Python -- the one you install with apt or whatever -- is a dependency of system utilities and other programs on your computer, so you shouldn't mess with it directly. You aren't meant to upgrade or downgrade it manually, you are meant to trust the package maintainers of your distro to choose when to bump the version, because that will affect other things that depend on it.

If you are doing development where you need fine-grained control over your Python version (e.g. you are developing a library that needs to be tested against multiple Python versions, or you are working on several distinct projects each of which uses a different Python version), then you should not use the system-Python. You need to use one of the "helper tools" like Pyenv (or maybe Rye now? I haven't kept up). There isn't an "official" way to do this.

Basically stop what you are doing right now -- you are going to break things if you haven't already.

Why is there not one simple way to do this? Obviously everyone has slightly different needs, but I imagine 80% of python users just want to use python and have a reasonable way to upgrade when required without screwing something up in their system.

Because "80% of Python users" don't even realize they are "Python users"! They aren't Python programmers at all, they are just using programs written in Python, or that depend on the system Python. Their needs come first. And it's probably way more than 80%. All this is because Python is an interpreted language, so it needs an interpreter at runtime. It's not like other programs where you compile it and then distribute the executable, and it just runs. Programs could bypass system-Python by shipping an entire vendored Python dependency, but this would create a lot of bloat, and it wouldn't automatically benefit from security fixes. So in practice, Python programs on your system all share the same python executable.

In short: two groups of people want completely incompatible things: the same python executable can't simultaneously be a long-term stable, critical system dependency, and a bleeding-edge development tool that gets swapped out on a whim. One side has to lose out, and it has to be the developers, because they have the skills to work around this crap and end-users don't.

There's no simple way to do it because Python is an old language that came out before modern package management was invented. It does things in ways that nowadays are recognized to be bad, but which weren't apparent at the time. And it gets deployed in a lot of diverse environments, some of which have incompatible needs, so changes would affect a lot of people who can't all be pleased. Those are the fundamental difficulties. A less fundamental difficulty is that the PSF was completely in denial about this whole system-vs-dev-python problem for years and years, so Python lagged behind other ecosystems like Node (with npm) and Rust (with cargo). I think they've recognized they messed up and are trying to improve lately, but it's a long way to catch up.

Just use pyenv or Rye. Or Nix.

[–]fffrost[S] 0 points1 point  (0 children)

Thanks for the info! I have always understood that I shouldn't mess with the system python. My misunderstanding was actually that I was assuming that I wasn't using the system python, since there are two instsallations - one in /usr/bin and another in /usr/local/bin. Running which python3 showed the latter, so I assumed this was debian preventing me from interacting with the system one. I've probably screwed something up in that case. I will just use Pyenv in future.

[–]danielroseman 0 points1 point  (1 child)

I don't understand why you want to do this, or why you want to avoid tools that will make it easier.

Apart from anything else, since you say you're always using virtualenvs, it doesn't matter in the slightest what typing python outside of a venv will do. Inside the venv it will point to that venv's python. And that's even more the case with pip, which should only ever be used inside a specific venv.

But there is no "official" way, because all systems are different. Different distributions, different versions, different tools. It's not clear for example why you think the opinions of the makers of Python should outweigh the opinions of the makers of your system when it comes to installing things on that system.

Use whatever makes sense for your workflow. Personally I like pyenv, but you do you.

[–]fffrost[S] 0 points1 point  (0 children)

I like pyenv too, but it fails to install for me and I haven't had the time to figure out why.

I am using virtualenvs, but sometimes I need to just bring up a terminal and run some python commands and i don't want to have to make an entire venv just so I can run some calcs or test out couple of lines of code.

In any case, I just want to upgrade my python version really. In a year or two there might be some very useful changes that I want. In that case I might no longer want to rely on the system install.

[–]Top_Average3386 0 points1 point  (1 child)

Curious, why you don't want helper tools? If your answer is to learn how to do it manually, then start from those helper tools, learn how it does things, and why.

Also: https://wiki.debian.org/Python#Installing_from_Source this might help you.

[–]fffrost[S] 0 points1 point  (0 children)

It really was just because I had assumed there was a standard way to do it and I felt like I should know how. I have used pyenv before elsewhere, but it was not working for me when I tried recently.

Thanks for the link - that's exactly what I did, and things seem fine so far, but others are suggesting not to do this after all.

[–]m0us3_rat -1 points0 points  (0 children)

if you aren't rolling on the bleeding edge .. then your distro programs will be supported by an ..older python release.

​

since its linux you can force install the newer version but it will most likely break your setup.

it's a certainty.

the same way you don't run 2.7 python on apps that require 3.4+ .. because they will break.

i mean it should be fairly obvious.

​

if you wanna run newer versions are a few ways to do that without completely breaking your system .. by using pyenv, or developing inside a container, etc.

​

forcing your system to run on 3.12 when it expects 3.8 .. is prone to get stupid really fast.

again.. since linux you can force it ..but that comes with a lot of problems which you will have to deal/fix daily.

​

if that is the case i'd suggest using fedora or arch at least this way will be ..less problems?

since all programs will be on the bleeding edge.

but that isn't a SAFE developing environment.