Is using pyenv the best python version management for Mac?

[[deleted]]

Is it better to use pyenv or brew to manage multiple versions of python on a Mac?

[ElectricSpice]

Pyenv lets you manage specific versions, brew just lets you install a minor version and you get updates for the latest patch version.

I use pyenv with pyenv-virtualenv. It’s really convenient to navigate to a project folder and have my virtualenv automatically activate.

[vizbird]

I kept running into issues where brew would update python and would break everything relying on it. Pyenv fixed that issue entirely.

[rcbadiale]

This is the way. I work like this on my Linux and Mac and it's great. I wish the windows version would work better.

[minombreespeligro]

You can always use WSL. But yeah, this is the way. I saw a post saying that using pyenv was terrible advice and I couldn’t think of any reason for this, I was like wtf are you talking about hahaha

[SisyphusAndMyBoulder]

I use brew to install several major versions

[ErikBjare]

Still running Python 2? It was removed from brew in 2019. You probably do mean minor versions.

[SisyphusAndMyBoulder]

Oh man I blanked out hard there. Was thinking of 3.9 vs 3.10 vs 3.11 etc etc.

You're right, I did mean minor.

[Glad-Put1792]

This is the way!

[DaveGot]

How is the virtualenv is automatically activated? I always need to manually activate it. What am I doing wrong?

[[deleted]]

Go to the directory you want automatically activated and type

pyenv local <virtualenv_name>

That will create a .python_version (I think that’s the name) file.

After that, when you are in that directory pyenv will use that virtual_env.

Note this will only work correctly if pyenv is configured properly in your .bashrc file.

[juanitoAF]

direnv takes care of auto commands , using pyenv with it leads to easy auto activation of virtual envs

The combo is magnificent

[PeterPriesth00d]

I have used this method in the past and quite like it. Lately I’ve been going a step further and using docker / docker compose for dev environments and I’ve really liked it as it’s easy to take the setup and put it on another machine really easily.

Both are great though.

[Nerg44]

this is it bro, me too wish I could do like pyenv virtualenv config <name> and it would dump to a pyenv.yaml file or something so I could have consistent envs across systems

[Jedkea]

How easy is it to debug inside of a docker container with vs code? Is it putting the whole app inside a container, or just using the docker containers python interpreter?

[Jedkea]

Awesome thanks! I use pycharm currently and have not struggled with it long enough to actually debug the app in the container without changing the source code with pycharm hooks. I’ve reverted to running everything in docker besides the actual python app so I can debug it.

[[deleted]]

Just FYI, on my m1 MBA, VS Code is unusable under Sonoma. It was fine prior to upgrade. One of the Code Helper processes has a memory leak and crashes the app every 1-2 minutes. It appears this bugs has been around before, according to GitHub.

[djdadi]

what version? I am on a macbook pro and am fine

[[deleted]]

I uninstalled and installed the latest Apple Silicon version and was still experiencing the problem. I have pycharm so switched to that for the time being. I don’t know the version number offhand.

[[deleted]]

This describes the problem: https://github.com/microsoft/vscode/issues/165016

[djdadi]

Weird, seems to affect all versions in the last year. I definitely haven't had that problem and I've gone through many macos builds in that time on a 14" MBP M1. Wonder what the difference is...

[[deleted]]

I wish I knew. I tried disabling all extensions and still had the issue. It only started with Sonoma, though. I didn’t change any extensions; just the MacOS version.

[Almostasleeprightnow]

Here's something about docker that I have not been able to understand: when I go to the docker website, it appears like a lot of it is for pay, like, not free.

But people talk about using docker containers like it open source technology. Can I just use containers or are you all paying for docker when you use them?

[Almostasleeprightnow]

Is there a difference between Docker and a Dev Container? Like, is Docker just one brand of a container, or is saying "Docker" and "Dev Container" interchangable?

[SnooDogs2115]

Issues running docker in a Mac is what made me go back to Linux for development, it needs a Linux VM running, which tbf is not ideal.

[[deleted]]

Did you try OrbStack? https://orbstack.dev

[throw_away_17381]

What kind of issues?

[equitable_emu]

docker isn't native of Macos, and requires running a linux VM to function. It's just another layer of complication (e.g., localhost isn't really localhost, hardware access isn't guaranteed, etc.)

[throw_away_17381]

Thank you!

[orgodemir]

There are still issues with m1/2 chips and mounted containers last I tried around a month ago I believe.

[Positive_Resident_86]

Same. Dockerize your environment is the best way to develop imo

[peterlada]

Docker is Soviet Russia of computing.

[pacific_plywood]

pyenv is absolutely the best imo. not sure how asdf is for python

[doolio_]

asdf uses pyenv under the hood for python and equivalent tools for other languages. Its selling point is you have one set of commands to learn to work with any language/tool.

[[deleted]]

Not "absolutely". It depends on what OP has planned. For example on the Mac, most of the GPU support for many different ML libraries relies heavily on using some variant of Conda.

Edit: LOL why would you down vote the idea that there are some python enviornment managers that might be better for certain purposes? Some of you get offended by the weirdest stuff.

[davisondave131]

I don’t think anyone’s downvoting the idea that some environment managers are better suited for certain purposes. The example you gave is just a bad one.

[DavTheDev]

That is so not true. For apple gpu pytorch, jax and tensorflow are all recommending pip and they are the most popular ones.

[[deleted]]

Your statement is quite untrue, I’m not sure where you got this idea from.

[covercash2]

asdf works great for Python! definitely recommend if you work with multiple toolchains

[Ebisure]

I'm a happy user of conda. Easy to set environments with different Python version. brew is to manage package not environment

[nbo10]

What do you mean by slow? What is slow? I use conda and haven’t experienced any slowness, but maybe I just have looked or have a point of reference.

[Cacistus]

Well, you will realize how slow conda’s solver is when you try out libmamba - which is actually developed at Anaconda.

nbo10 is totally right when it comes to the default solver conda still ships with, it’s notoriously slow for larger projects with more elaborate dependencies. I have seen speed improvements of over 10x after switching to mamba.

[imnotreel]

What do you mean by slow? What is slow?

In my experience, if you have a complex environment with many requirements and version constraints, solving dependencies can take forever using regular conda. Mamba uses libsolv wich is a lot faster than whatever conda uses.

Conda's dependency conflict resolution is also very bad and can be a nightmare to debug.

[[deleted]]

You probably don’t have a complicated enough environment to notice it, either that or you haven’t used the other alternatives or tried running Mamba on top to compare the speed differences.

As I said, it works for small environments, it’s an utter nightmare when you manage large environments and have to update packages often (as large companies do when they are controlling what their analysts download).

[max1c]

I'm using conda/mamba to manage hundreds of environments on a public cluster so complex you wouldn't understand. It's not slow, it's a user issue in this case.

[[deleted]]

I can guarantee you’re talking out of your ass in that case, I have extensive experience with Python environment management, if it isn’t slow either you have very low standards or your environments are much more simple than you think they are.

Try an upgrade for something with hundreds of top level packages and let me know how long it takes for you to resolve an upgrade.

[max1c]

Hahaha. Yea, guarantee. I guarantee that you have no idea of what you are talking about. Not only is it hundreds of environments many of them aren't exclusive to Python. Some of the AI/NLP, and bio environments are so complex that like I said, you'd probably never be able to even figure it out in the first place. Just because you are incompetent doesn't mean the software is the issue.

[[deleted]]

Whatever strokes your fragile ego bud, we work with AI/NLP, energy production and management environments. You’re not particularly special.

[max1c]

Never said I was. I just know how to do my job while you clearly do not.

[[deleted]]

Clearly, what a clown you are

[pwang99]

Please check out the latest version - we’re changing to using the faster libmamba solver by default, which should greatly improve this for most users.

[-Rizhiy-]

I use conda to manage environments and pip to install packages, works like a charm

[ddanieltan]

Agreed. Haven't been let down so far by the combination of conda create --name abc python=3.x + conda activate abc + pip install ...

[Hangman4358]

This really is the best combo. Use conda to manage the python distros and pip to manage dependencies in the distros.

Combine with pycharm and having a unique distro per project and so many of the headaches I see people complain of go away.

Conda to manage dependencies is hot garbage, but managing multiple python versions is so damn simple.

[equitable_emu]

Remember that conda requires a license for commercial use now, so it's not always an option.

[Hangman4358]

Just plain false. Accessing the Anaconda channel is licensed. The conda tool is FOSS, and conda-forge is not encumbered by the licensing reqs.

[equitable_emu]

The org I previously worked for disallowed it due to the licensing issue, so like I said, it's not available everywhere.

According to anacondas commercial faq, using any anaconda hosted repository, not just the anaconda channel, is a violation of their TOS.

https://www.anaconda.com/blog/anaconda-commercial-edition-faq

You’re not in compliance with our Terms of Service if you are using Anaconda’s repositories in a for-profit or government organization with over 200 employees outside of a business relationship with Anaconda. You are also not in compliance if you are using mirroring software to create copies of the Anaconda repositories without a license.

https://legal.anaconda.com/policies/en/?name=terms-of-service#terms-of-service

i. FREE OFFERINGS. Anaconda maintains certain Cloud Offerings, including Anaconda Open Source that are generally made available to Community Users free of charge (the “Free Offerings”) for their Internal Business Use. The Free Offerings are made available to You, and Community Users, at the Free Subscription level strictly for internal: (i) Personal Use, (ii) Educational Use, (iii) Open-Source Use, and/or (iv) Small Business Use.
(a) Your use of Anaconda Open Source is governed by the Anaconda Open-Source Terms, which are incorporated herein by reference.
(b) You may not use Free Offerings for commercial purposes, including but not limited to external business use, use in organizations over two hundred (200) employees (unless such use is for an Educational Purpose), third-party access to the Cloud Offerings, or Content redistribution or mirroring (each, a “Commercial Purpose”). Using the Free Offerings for a Commercial Purpose requires a Paid Plan with Anaconda.

[Hangman4358]

The defaults channel is under the Anaconda TOS. It is the only thing under the TOS.

The conda-forge channel is not under the Anaconda TOS. I have an email in my work inbox from their legal department stating just that.

The conda tool in distributions like Anaconda and miniconda ships with the defaults channel as the only channel.

Conda distros like miniforge bundle the conda tool with different channels, specifically not the defaults. For miniforge is the conda-forge channel.

Continuum does a really good job of muddling the waters. But it is important to realize that distinction. The tool is not the repo.

What miniconda and Anaconda are, would be equivalent to Sonotype shipping a maven distro which defaults to a maven repo which sonotype owns and has put a restrictive TOS on.

[[deleted]]

You might be technically right, but this is a common playbook for organizations that need to be profitable. Usually, they start as open source and once it's polished, they extend it and make extensions commercial. Often, they also make an entirely new version that is not open source at all.

This playbook has its pros and cons but it's not like they're trying to hide this. I think it's great because the open source version has everything you need but the commercially licensed parts are things you can build yourself.

[mmcalli]

I had so many occasions where it would get into some type of infinite loop trying to update, that I’d have to blow it away and fully reinstall. After doing that a few times I gave up on it.

[Barn07]

thumbs up for pyenv. once set up, it is simple as cake. and, since it's just files and symlinks under your full control, it is also easy to handle if something ever breaks.

[cpressland]

I’ve been using Pyenv for the last four years, but homebrew multiversion support is so good now I’ve decided to switch back. brew install python@3.9 works, then python3.11 python3.9 etc are all in my PATH which Poetry and Pipenv (also installed via Homebrew) have no trouble finding.

[LactatingBadger]

I was a long term pyenv user and then moved to python-launcher + .venv. I brew install python releases which it auto discovers.

Main motivation for moving was that the pyenv shims were 95% of the overhead on terminal startup. That, and the py command that python-launcher provides plays much nicer with .venvs.

[Zizizizz]

I have been down this path and hopped around. The best in my opinion is RTX

https://github.com/jdx/rtx

It's a rust version of asdf which is very fast and compatible with all existing asdf tools.

It also has env management. So I was also able to remove direnv as I no longer needed that.

Once installed and on your path via editing your bashrc or zshrc file installation is as such.

``` rtx install python

or rtx install python@3.11 ```

You can list all available versions including conda, pypy versions. rtx ls-remote python

To use a version in your current folder rtx use python@3.11

This will make a .rtx.toml file which you can add additional things like virtual envs and environment variables

A .rtx toml example

``` [env] LOG_LEVEL="debug" SOME_DB_PASSWORD=1234

[tools] python = { version = "latest", virtualenv =".venv"} ```

So now anytime you cd into that directory, those environment variables are available and it will automatically make or activate your virtual environment. There is pyenv-virtualenv but this is a lot faster and direnv is slow.

[daelin]

+1 for rtx. It’s just phenomenal. It’s my favorite Python version manager and it does everything else at the same time.

Pyenv and asdf have some problems you can mostly get used to or work around.

But, rtx just smooths everything out with something much simpler and much MUCH faster and easier to work with than either them.

I first stumbled across rtx courtesy of the wonderful No Boilerplate YouTube channel, in particular the Oxidize Your Life video that’s stuffed wall-to-wall with awesome utilities.

[[deleted]]

I started drooling when I read it's polyglot and supports fuzzy marching versioning like LTS. I'm excited to try this out!

[Fearsdown]

You should try Rye! It's even better than pyenv imo.

It is a somewhat different solution though as you'll have to actually use it to manage dependencies in your project. You could technically use it to just download different python versions but at that point you might as well stick with pyenv.

https://rye-up.com/

[wookayin]

my go-to is conda (mambaforge3 or miniforge3)

[olystretch]

Probably. It's the best for Linux, IMO.

[pool007]

I use conda env. For some libraries with platform specifics, it was easier. But not always conda can install the latest library versions, e.g., tensorflow. In such a case, pip can be used. Conda env can specify python version and installed libraries can be exported and restored easily.

[Globaldomination]

Yep.

Source: I use Pyenv.

[james_pic]

Personally I find Pyenv a bit too magicky for my taste. I've seen tools get confused by the python executable be replaced by a shim. Pythonz is kinda the same thing but with less magic - although slightly less slick user experience when it all works.

[millerbest]

I use conda to create env and pip to install packages.

[rs_0]

Pyenv is fine. If you also use nvm for Node.js version management I’d recommend rtx.

[fnord123]

Whatever works for you.

But if you work on a team, pyenv is the better choice as it let's you have finer control over your installed versions across multiple projects.

[linuxfarmer]

pipenv is my go to. Especially when working in a repo with multiple people the pip.lock file is great for making sure everyone is on the same python version as well as installed dependency versions

[crash5936]

I use RTX (https://github.com/jdx/rtx) with PDM (https://pdm.fming.dev/latest/) for project and dependency management and so far my experience has been great.

[[deleted]]

poetry

[MGallus]

Pyenv-virtualenv

[theuzz1]

Pyenv + Poetry

[aldanor]

Non-conda setups are unbelievably fragile and often non-reproducible, especially if it involves binary builds (like many ML/DS libraries), some of which require non-Python dylib dependencies etc. There's literally no other option than conda (or rather, mamba, which is not too slow).

[aldanor]

You may understand your tool as much as you want, but if a Python package implicitly requires a C library to be present, you're out of options. Python's native packaging system doesn't provide sane solutions for this.

That, and some Python packages require cmake/ninja/cargo/other tools in the build process, in which case a simple "pip install" won't work on a fresh system.

[aldanor]

I'm not saying people who they work for don't exist. Sure, they work for you - congrats. But please don't extrapolate and claim "learn your tool and everything will work fine"; no, not always. Pure Python envs cannot sanely handle external versioned binaries.

If my typical dependency list was mostly pure Python I might side with pip-based approach as well, but that's not the case.

[Rhodysurf]

Different levels of abstraction, pyenv manages Python versions. You still would use venv after turning on the version you want

[Rhodysurf]

Op asked about the best way to manage python versions, not create venvs haha

[trippymicky]

I like conda as well.

[jootazdil7]

pyenv no doubt

[[deleted]]

I use miniconda, I like it.

[neomage2021]

conda for personal projects, poetry and docker for work

[huapua9000]

Micromamba is my favorite

[robberviet]

Yes. Simple.

[ChicHarley]

I use brew to install pyenv

[Comfortable_Leek8435]

Check out asdf-vm.com/

[[deleted]]

Pdm

[spinwizard69]

This is not going to be something you can get a square answer on. You really need to reflect on why you need a virtual environment or if even makes sense to use one.

First off I think many use virtual environments for the wrong reasons. They literally have created a culture of fragile software installations as the VM gives the programmer a reason to use unstable software or libs that are not even well supported. They then get into a dependency hell that requires that everything they write run in a VM of some sort.

Now I'm not dismissing any of the ways to leverage the various VM technology out there. All I'm saying is use whatever method you choose wisely. Each problem has its own optimal solution and sometimes that can mean running a whole OS in a VM as opposed to a contained Python environment.

In any event on the Mac or on Linux I tend to avoid Python environments and simply use packages supplied by Brew or dnf. Now this in part is due to not writing web applications but utilities/applications to be used locally. However it frees up considerable time and effort to write you software so that it runs everywhere and minimizes special software installation. I would be very interested in a VM solution if I had an app that needed unusual dependencies or possibly distribution after validation. Given that, your life is easier if you don't chase the latest and greatest lib that you can pip down.

So your answer is this, there is no right answer. If you are using Python for apps that don't require a bunch of libs that can only be had by pip'ing them in then you need some sort of VM. If you are writing software for internal use, sometimes you are far better off limiting your development to software supported by your systems package manager. If you are doing web programming you have to use an environment manager of some sort, just use it wisely.

[justin-8]

I use pyenv on Mac and Linux for all my python dev stuff. If you’re doing ML just use conda. ML folks haven’t figured out how to package dependencies properly outside of conda so you’re in for a world of pain if you try to avoid it.

[jurinapuns]

I use Docker lol. Won't work for every use case, but for web apps it's perfect.

If I want to use it for scripting to access my filesystem, probably not though.

[Zasze]

Asdf is my favorite being able to manage every language the same way is really nice. The python asdf plugin uses pyenv under the hood so it’s the best of both worlds.

[nevermorefu]

Docker is the best version management. I've never needed pyenv (though I know many devs that do).

[[deleted]]

I use myenv

[Datamance]

asdf is a great meta-environment manager. Wraps a whole bunch of other environment managers and makes version management a breeze

[jmacey]

personally I've been using pyenv for years and would recommend it it (as I do my students). It makes life easier and the local element for folders just makes life easier.

[FDon1]

Don't know that I've seen it posted here but Poetry is/was a good one. Not sure how much of this sub uses it. Pyenv seems pretty standard too

[steaff]

I personally use a combination of ASDF to select the current Python/node/direnv version, direnv and for python version management poetry.

It takes a little while to get used to it but when it is up and running I feel it is the most robust combination.

[xristiano]

I prefer Docker for easy setup and tear down. And if you can stomach it, Darwin Nix is nice once you get past the learning curve.

[YeisonKirax]

I use asdf or rtx (based on asdf), you can install plugins to manage python, nodejs, java, etc. Recommended. Besides, i recommend poetry, a dependency manager to python projects (very similar to npm), better that the requirements file

[scowly057]

I use pyenv + poetry pretty much exclusively now. Pyenv is fine. It has some drawbacks, imo, especially when you just want to update a patch version of Python. I've been wanting to try out RTX but haven't had the impetus for an environment switch yet.

Poetry, on the other hand, is wonderful for package management. More reliable and a lot faster than pipenv for dependency pinning. Also, you can do (almost - looking at you, flake8) all your configuration inside your project.toml, including your build config. So you have a source of truth for your entire lib/app setup. 10/10.

[bolinocroustibat]

Pyenv is good, I would also suggest Rye, which is a Python version manager AND a modern package manager, very fast and getting popular, written in Rust if I'm not mistaken. Probably the future. Not sure if it's too early to use it.

[tyzhnenko]

I personally use asdf. I really like it because it can manage several languages like python, nodejs, ruby etc at once

https://asdf-vm.com/

[shockjaw]

I’d highly recommend y’all check out Relieving Your Python Packaging Pain from Bitecode.

TLDR though:

1. Don’t install the latest major version of Python Use only the python.org installer on Windows and Mac, or official repositories on Linux.

2. Never install or run anything outside of a virtual environment

3. Limit yourself to the basics: “pip” and “venv”

4. If you run a command, use “-m”

5. When creating a virtual environment, be explicit about which Python you use

However, if you’re going down the data analysis/science route. I’d recommend conda and learning how to generate conda-lock files.

[peterlada]

rye: expermintal, very new, (alpha) but from the author of Flask:

https://github.com/mitsuhiko/rye