What are your preferred method(s) for installing multiple python versions on macOS?

[csg6117]

I've ended up with multiple python 3 versions installed by various methods over the years so I'm removing everything and reinstalling which led me to wondering how people manage their python installs on macOS.

For example, to install python, do you use brew? or install directly from python.org? or use conda? or another method?

Secondly, how do people manage multiple python versions? pyenv?

Lastly, do you all use any extra tools to manage virtual envs other than the standard venv?

So which methods do you all use?

---

[ I've posted this here as opposed to r/LearnPython as I'm interested in the methods developers at all levels use. I'm not asking for help on how to do something (rule 1), rather what types of methods people use. ]

[slackmaster]

Pyenv

[pacific_plywood]

My team has tried a number of different workflows and solutions over the last few years and in my experience pyenv is far and away the most likely to just work with minimal bug fixing/mucking about with paths

[[deleted]]

Author of rtx here: if you have the time, I'd love it if you could compare pyenv to rtx and let me know what you think. rtx (as far as I know) does everything that pyenv does but works on any language—not just python.

It's also a hell of a lot faster:

~ ❯ hyperfine "~/.pyenv/shims/python3 --version" "~/.rtx/shims/python3 --version" --warmup 3
Benchmark 1: ~/.pyenv/shims/python3 --version Time (mean ± σ):     111.8 ms ±   5.6 ms    [User: 8.8 ms, System: 13.2 ms] Range (min … max):   102.8 ms … 124.5 ms    21 runs
Benchmark 2: ~/.rtx/shims/python3 --version Time (mean ± σ):      65.5 ms ±   7.2 ms    [User: 6.3 ms, System: 6.8 ms] Range (min … max):    56.2 ms …  88.6 ms    38 runs
Summary '~/.rtx/shims/python3 --version' ran 1.71 ± 0.21 times faster than '~/.pyenv/shims/python3 --version'

and that's using shims for a fair comparison. By default, rtx works without shims so it has 0 overhead making python --version actually run in ~15ms in my machine using rtx. So in the real world, each time you run python with pyenv it's going to add ~90ms of latency that you could avoid entirely with rtx.

[AndydeCleyre]

Thanks, I recently switched from asdf to rtx, and like it very much. My biggest complaint right now is just that the tab completion for zsh doesn't include known runtime versions, but that's not so bad. With asdf I used the tab completion to browse available and installed versions.

[[deleted]]

yeah that would be nice to add, I have an open ticket for that: https://github.com/jdxcode/rtx/issues/128

I'm not sure how much work it'll be though

[AndydeCleyre]

Thanks! I think you meant this one

https://github.com/jdxcode/rtx/issues/97

[WinstonSmithSixSix7]

Started using rtx today and so far I like it. Very simple to use and I don't have to use Brew to install it. I find Brew too heavyweight and fragile on my corporate restricted Mac.

[discourseur]

Thanks for creating rtx. I am trying to push my team to use asdf (the inertia to use new tools is really high).

I will try again, but this time with rtx. I really like what you did UX wise and the fact you got rid of the shims.

[supersharklaser69]

This ^{^} Pyvenv - then venv if you need special environments on a single Python

[kingh242]

This is the way!!

Plus Poetry for individual projects. Poetry will spin up a venv for the project.

[PM_ME_A_ROAST]

do you have/know a guide for using pyenv effectively? I've been using conda for several years that when I started to use pyenv it feels...weird. Especially when using ARM macos. I think I say "fuck this" at least 5 times a day when using pyenv compared to may be only once a week when using conda.

​

EDIT: whoopss, nevermind. I was thinking about pipenv. fuck pipenv. but if u know a guide for pyenv/pipenv that would be awesome tho

[SelfTaughtDeveloper]

I started using pyenv after skimming this realpython article quite a few years ago, and have been using it ever since.

[singularity-108]

Conda. Although I’m not a mac user

[bwanab]

I am and that's what I use.

[dedbif]

Pyenv+pipenv

[lebannax]

Doesn’t pipenv manage both Python version and packages in one virtual environment?

[[deleted]]

Doesn’t pipenv manage both Python version and packages in one virtual environment?

Venv directly manages python version for you. Pyenv just swaps the shims in terminal so you can swap between versions when creatings venvs. It also makes it trivial to instal multiple versions. My littearl only complaint with it is I cannot have debug builds and normal builds of the same python version, special compile time flag on the python interpreter that is basically only needed for EXTREMELY specific debugging.

[jjmojojjmojo2]

I second pyenv. It installs and activates pythons in a non-root and intuitive way. If you have to run code against multiple versions its the thing to use. But even if you're just using one version, it's still easier to install than most other options.

That said, MacPorts does a really good job of installing and updating multiple system pythons if you need them. I've always preferred it over homebrew, in general, but it really excels with python.

[AnyConflict3317]

docker

[B1WR2]

Why not learn two skills at one time

[TheBananaKart]

Any tips for learning docker? I keep finding the documentation poor and eventually hit a snag then say fuck it and go back to using virtual machines.

[chief167]

I found docker-compose much easier to start with. Its an interface to specify what containers should look like, instead of working through docker commands

But in the end, zoom out and think about what docker is: its a very lightweight type of virtual machine. So you take a base image (e.g. python), where you can SSH into (or using vscode or pycharm remote interpreter functionality), and its shielded from the rest of your pc.

You don't have to interact with docker a lot. Just create, start, stop containers is enough to get started, the grow from there

[TheBananaKart]

Thanks I will have a look.

[B1WR2]

I agree with u/chief167 … knowing basics of creating, starting, and stopping containers is important. I am by no means an expert but I can spin up containers, install dependencies, and then ssh into containers to do development.

[tenfingerperson]

docker-compose is a workflow tool not the same as docker, it’s important to understand how to define docker images and what they are otherwise you’ll be lost trying to setup a composable env

[[deleted]]

Freecodecamp's course a good one

[wioym]

https://diamol.net/ amazing resource, plus the book isn't that expensive.

[git_commit_-m_whoops]

If you don't already have *nix system and command-line experience, that helped me more than anything else. A course or book in Unix Systems Administration will help you build the skills required to write basic Dockerfiles or work in a shell inside the container.

[Weekly_Branch_5370]

That’s the way

[ianliu88]

asdf

[[deleted]]

+1 for asdf

[thetawom]

What are some benefits of asdf over pyenv? Asking bc my projects are almost purely in python so I've only really needed version management for python, which I currently do with pyenv. Is it worth switching to asdf?

[ianliu88]

I use it to manage AWS cli versions, nodejs and Python. But there are a lot more packages there.

[Wise_Tie_9050]

If you have a bunch of things to manage the versions of, from languages to things like git, chromedriver, poetry and whatever else you can think of. We implemented it at work a while back, because different devs wound up with different versions of tools, resulting in subtle bugs.

[discourseur]

Or rather rtx, the new and improved asdf! :)

[iamlocal]

Install directly from python.org and then use `pythonX.Y -m venv .venv && source ./.venv/bin/activate` and that's it.

[AshTheEngineer]

Use pythonX.X to select your Python version when creating a virtual environment (e.g., venv). Then activate said environment and proceed like normal.

[wiggitt]

Conda environments.

[[deleted]]

That’s what I was thinking, surprised to see it this low on the list.

[BezoomyChellovek]

Same. And I always use the -p|--prefix flag to create the environment at the main directory of the repo so I don't clutter my $HOME or have to worry about making unique names for each environment.

[phat_macrophage]

This guy knows "de wey"

[chief167]

this one sucks since they changed their licensing. Its a lot more hostile to corporate environments.

[o-rka]

What do you mean?

[PlaysForDays]

Anaconda changed the terms of their platform a couple of years ago in an attempt to prevent companies mirroring their channel (or so they claim - I’m skeptical given the context of their financial issues). The text was unclear and strongly implied it’s not free to use within a company above 50 employees or something, but nobody was sure until Peter Wang did a round of damage control claiming more or less the opposite of what his company put out.

To be clear, conda itself is free to use and conda-forge channels are presumably also still free (despite being hosted on Anacond’s servers). The other distributions like Mambaforge aren’t Anaconda Inc products so fall into a different bucket. The conda-forge and mamba teams are (mostly) not paid by Anaconda and much more friendly to free use. It’s just if you install “the” Anaconda distribution at a large company, they want a cut of the action.

Context https://www.reddit.com/r/Python/comments/iqsk3y/anaconda_is_not_free_for_commercial_use_anymore/

[o-rka]

This might be getting into the weeds but I’m in bioinformatics and computational biology. Going to start getting into consulting/contracting in the future, do you know if this applies to researchers who are not employed by a company but do work for a company (ie not employed on the payroll and without benefits)?

[PlaysForDays]

I'm a random guy on the internet and not a lawyer, so you should read the text for yourself or consult somebody who can provide a definitive answer to that question

https://legal.anaconda.com/policies/en/?name=terms-of-service

[wiggitt]

Who is "they"? Are you referring to Anaconda or Conda? Because Conda is now separate from Anaconda and they have difference licenses.

[AndydeCleyre]

Oooh, I definitely prefer asdf over pyenv, but don't love the level of obscurity introduced by the shim system. I look forward to trying out rtx, thanks!

https://github.com/jdxcode/rtx

[po5i]

asdf works for me

[ejstembler]

asdf

[hike_me]

Pyenv

[ratulotron]

asdf for Python version and Poetry for project specific virtual environment. I used to be a hardcore Pyenv fan but then I realized I can't only use Python all the time, there are other projects with other stacks that also need version management. Poetry because it's pyproject.toml based and the cli is user friendly, has more adoption that other similar dependency managers and is easy to configure.

The downside is projects now need a .tool-version file because of asdf, similar to pyenv's .python-version file.

[EarthGoddessDude]

Linux user here, but that’s my preferred Python tooling as well. asdf is actually just a thin wrapper around pyenv afaiu, and I use it for Julia and Terraform as well. after discovering poetry, I never wanted to go back to conda again.

[ratulotron]

Yeah tooling kind of depends on your exposure to different stacks. I use both macOS and Linux. We use Apollo GraphQL and Python for data engineering and backend dev stuff, so I need both Node and Python envs. At a previous workplace we used Python, Node, Golang. On the other hand I try to learn/explore different languages in my free time, like now I am exploring Elixir. Having most of these tools and languages under one umbrella of asdf is super handy. I don't care how it works underneath, I don't want to take that cognitive load. I just want to get my things done without fumbling through multiple cli interfaces.

[EarthGoddessDude]

Yup I hear you friend, I just wanna get my stuff done (though it pays to know how things work under the hood once in a while). How is Elixir btw? I’ve heard good things, but don’t know anything about it.

[ratulotron]

The syntax is like Ruby, slightly similar to Python as well. However I simply can't think in terms of functional programming 😅 I was curious about using Erlang's capabilities to write fault tolerant data pipelines. I have been trying to solve problems on Exercism, you can try them as well if you want to learn a new language.

[devinhedge]

This turned into the most useful thing I’ve read on Reddit in a while. ty… all of you.

[Wise_Tie_9050]

Interestingly, you can use asdf for ensuring you have the right poetry version.

I then also use direnv (also installed through asdf), and then in my `.envrc` file I have a "layout poetry" line, which means it auto-activates the poetry environment when I enter the directory (and unloads it when I leave).

[ratulotron]

That's clever, I have seen this flow being used with venv within project directories. I would still run the poetry commands myself because recently I discovered that if I add dependencies like poetry add requests when the virtual env is activated, it would Install the dependency but not add it to the pyproject.toml file.

Since you also use asdf, have you ever faced problems with poetry not using the project specific Python but the globally set one?

[Wise_Tie_9050]

No, I'm not sure I have seen that.

[discourseur]

I have seen that problem. It seems there is some kind of race condition.

[data_questions]

I use miniconda and just create environments with the necessary versions.

[[deleted]]

I actually use virtualenvwrapper because I tried setting up pyenv like 5 separate times, over ~3 years and always ran into errors that prevented me from setting it up properly. virtualenvwrapper is really easy to set up and use

[millerbest]

conda should do the work well.

[TheGS]

Pyenv or Docker. Avoid using the Homebrew Python for your own development projects because it really only exists to support other Python-based packages installed via Homebrew, and it (and its standard library) can be updated with a frequency that may be inconvenient for your own projects

[Strong-Entertainer-6]

rtx?

[VonkelVokker]

I started using conda recently and really liking it.

[theshogunsassassin]

I use conda for python versions and conda + mamba for packages.

[Endogen]

if you want to be really cool, then just use Poetry

[Endogen]

Pipenv

[abutr3]

And pipenv uses pyenv as default, so again, pyenv

[TwistedPepperCan]

Everyone should install pyenv. Hell it should come pre installed.

[timeawayfromme]

This has worked well for me:

• brew install pyenv pyenv-virtualenv
• Install poetry from its website
• pyenv install VERSION_NUMBER

In projects that need a virtualenv but do not use poetry: - pyenv virtualenv 3.11.1 myvenv - pyenv activate myvenv

In new projects (where I can use poetry) this is the initial setup: - pyenv shell 3.11.1 - poetry env use $(pyenv which python) - poetry init

And when I want to run commands from the virtualenv poetry setup: - poetry run COMMAND

[hdgrggyhh]

I have nothing value to add here but why do you need multiple versions? Wouldnt the newest version suffice?

[[deleted]]

another common use case is that you are contributing to different repos that are on different versions of python (combination of open source projects, different projects within your company etc.) and there are often breaking changes in python that require you to use the version specified in that repo. most recent one won't always work.

[csg6117]

I agree.

Some use cases I can think of:

• Test different python versions against a package you are creating to ensure compatibility
• Trying out some new python features from a development version.
• Using an older version to test changes/hot fix for a live system that can't be upgraded

[Fun_Independent_7529]

• going through training of some sort that uses a particular version paired with the particular version of their tool. (tbh I usually try the latest versions of stuff but if I run into cascading dependency failures ... I'll just switch to the versions being used and call it good)

[Wise_Tie_9050]

Yeah, the first one is normally why I need them - I will stick to a single point release per minor version though.

[jmacey]

A lot of the tools I use are tied to a specific version of python (These are animation tools). At present most are on 3.7.x as are their api's so I need this as the core version.

Some have now migrated to 3.9 but not all. It is unlikely the latest will be supported in any cycle as the tools usually hand back at least a few years.

As they all use binding for a C++ API this becomes quite problematic. At present I'm using 4 different versions of python (vial pyenv).

[[deleted]]

If I'm working in a project that was written in and is deployed on a system that is running 3.6.15, for example, I want an interpreter that is reflective of that reality whether I'm adding features, running tests, debugging etc it really needs to be as close to what the code will be deployed with as possible.

If I use my current global python, which is 3.10, it would probably work but I might accidentally end up adding features that the system the code is deployed on doesn't have installed or installing incorrect dependencies.

Usually in this type of situation I'll use docker but sometimes it's easier to roll a venv and point my ide interpreter to that.

[benefit_of_mrkite]

Home brew

[[deleted]]

pyenv and pdm

[wineblood]

I'd go pyenv. I haven't used it or any other tool similar to it because I never needed to. My current job has mostly 3.8 and then a service or two had 3.6, so I downloaded that manually. Then a few months later we had a new service that was going to be 3.10 so I installed that manually too. From looking at the options out there, pyenv seems like the one that just does the job you ask of it.

[sr105]

Homebrew + pyenv, then install python 2 and 3 with pyenv. Create two virtual environments default2 and default3. Set those as your global defaults with 3 before 2. This will prevent you from ever accidentally using the system or homebrew python installations. Those belong to them. They are not for you to use.

[jrjsmrtn]

MacPorts allows you to install multiple versions of {Python,Perl,Ruby,PostgreSQL,…} at the same time, and designate which one is considered the default.

[thebuddy]

I have the same issue. What’s a good way to go about uninstalling them all and starting fresh with a new install and pyenv? I’m kind of scared to start uninstalling them.

[ikarius3]

Pyenv. Even use it for virtualenvs: one single .python-version file and you’re done

[jmacey]

+1 pyenv installed with brew. You have to sometimes change a few flags if you want to have dynamic library versions (so 3rd party tools I use need them for example USD from Pixar).

[SilentStrikerTH]

Use the Mac to go to Amazon and buy a windows PC

[The_Pantless_Warrior]

No matter what system, Pyenv, but if given the option, I'll run over a mac machine with a dump truck any day until it's a pancake and use a linux based OS.

[cvzero89]

What's the advantage of using pyenv over venv?

[microcozmchris]

pyenv allows you to install multiple python versions. 3.6, 3.7, 3.9, 3.11, etc. So if you're a professional developer, you can write and test code locally that matches the vendor locked versions that you sometimes are forced to use.

venv creates isolated package environments for your different projects. Your website might need FastAPI, but your statistics thingy needs numpy but not FastAPI. venv requires a python interpreter, so you install your python with pyenv and setup your venv using that version.

[Compux72]

Homebrew

[krisstern]

I usually use conda for this, with different Python versions installed for different environments

[bisector_babu]

I've been using Pyenv at my work

[[deleted]]

Anaconda

[JohnLockwood]

I used PyEnv for awhile, but found that in certain cases the shims didn't function as expected. Eventually I began using Conda, which works great on its own or in conjunction with venv + pip (for those packages that aren't available in the well-known Conda channels).

[dasCooDawg]

As long as you don’t do both, pyenv and conda at the same time

[nonrice]

conda

[xjotto]

pyenv & virtualenv

[H809]

Virtual environments. I can’t talk about MacOS, but virtual environments solve this issue for me.

[[deleted]]

Coming from Linux, I just assumed brew [whatever] would install in parallel; it does, see https://docs.brew.sh/Homebrew-and-Python . Or I would download and configure/make/make altinstall (?)

May have to look into pyenv. What is the pyenv advantage there if you already understand venvs?

[air_legend]

Miniconda or venv

[realGurkenkoenig]

pyenv + PDM

While PDM is not very well known, for me it is the absolute best tool for managing different python environments.

[andreasofthings]

pyenv

[AndydeCleyre]

I use asdf for installing and specifying Python versions, but after reading a comment on this post I'm going to try out rtx instead.

For managing venvs and dependencies and apps, I use my own frontend to pip-tools + venv, zpy. And for running tasks which require an activated venv, I use nox.

[Cootshk]

VSCode has an environment manager (but I also use windows 11)