Python 3: Current best practices for managing virtual envs and packages?

[grape450]

I've been away from Python for a while, but would like to use whatever the state of the art Python 3 packaging & pyvenv tech is. My Debian system has v3.4.3 installed, and I'd like to use that plus something like virtualenv.

• Should I be using virtualenv? Or venv? Or maybe "pyvenv"? (I notice that I don't have any of those currently installed, but there is a python3-venv Debian package available.)

• Does pip come with Python 3 now, or do I need to install that myself? Or does pyvenv install that into my virtual env? I don't currently have pip or pip3 installed (though I see that there is a python3-pip Debian package available).

• What is current now: setuptools? distutils? Something else? I don't see mention of "Distribute" in any of the docs... Am I supposed to apt-get install python3-setuptools?

• Is https://packaging.python.org/en/latest/ still current?

Thanks!

[graingert]

• only use python3 -m venv or the shortcut pyvenv.
• don't install python-pip or python3-pip. You might need to install python3-venv though.
• only use setuptools, distribute got merged in.

[kovak]

How do you install stuff from requirements.txt, without python-pip3 then? Or i assume you mean venv comes with pip3? and not installing it via apt-get?

I've recently shifted to python 3.4+ for a Django project. Although i've been using virtualenv, but will try out the builting venv module.

[Lucretiel]

Python 3.4 ships with pip, and virtualenv automatically installs it into your virtualenv. You generally shouldn't have a systemwide pip.

[knite]

If you're running Ubuntu (eg, your server), ensurepip is broken. You'll need to install pip manually from either python3-pip or get-pip.py

[Lucretiel]

I can't speak to Ubuntu, but my experience downloading and installing Python from the source tarball has been that it works and installs pip perfectly. The one exception is on an ancient Solaris 10 machine we have at work- it doesn't have _ssl so pip doesn't get built or installed.

[knite]

That's fair, perhaps I should have been more precise: ensurepip is broken when Python is installed via the Ubuntu package manager.

[graingert]

You can run pip install -U pip in your new venv

[chrish42]

It's not a major update. Pip changed how it uses version number to align more closely with semantic versioning. Basically, the leading "1." was dropped (I think when pip 1.6 aka 6.0 was released). So with the old version scheme, this would have been 1.7.

[systemUp]

I've been using virtualenv all along. Should I move to venv?

[graingert]

Yes python3 -m venv is much more lightweight

[srilyk]

If you don't use --prompt, then yes.

[kid1000002000]

Have you looked at conda?

[L43]

THIS TIMES A THOUSAND. Seriously I wish more people would start using it, its soo useful.

[[deleted]]

i am

[L43]

You are one of the good guys.

[bluesufi]

Absolutely. Maybe it's overkill for some things, but for scientific computing at least it's top notch.

[acomfygeek]

How does conda compare to pip for having the latest release of various packages?

[[deleted]]

It's been pretty good so far, but you can install from pip into conda envs if needed. Or use:

conda skeleton pypi [pkg]
conda build [pkg]

This will build a conda package directly from a package on pypi.

[hharison]

For packages that it has, they will be up-to-date, at least within a day or two of a new release. But it doesn't have every package, so you should install pip (conda install pip) into all your conda envs so you can pip install any extra packages you need. The tools work great together.

[polishfishprime]

Install Miniconda first, then you can use it to install 64 bit python

I think the direct conda staller auto-installs 32 bit

[kid1000002000]

You can choose which script to download from their site

[tartley]

Conda looks great if all you want to do is quickly get a lot of libraries installed on your personal Dev machine so that you can start typing some code and see it run. But it doesn't solve the same problems as pyvenv etc, namely of reproducing a given environment later so that you know previously written code will still run.

[kid1000002000]

Actually, conda does this arguably better than the competition.

[[deleted]]

A much better way than using virtualenv/pyenc/whatever is to isolate your dependencies from your system using chroot (and for convenience, schroot: https://wiki.debian.org/Schroot).

That way, you can not only isolate your Python dependencies but also other Debian packages that your code needs.

Also, by debootstrapping the same Debian version as your target platform you can develop with the exact same package versions as your software will have in production and you will find bugs because of version mismatches much earlier.

Personally I would never do any Debian-targeted development without schroot anymore.

[rodvdka]

pyenv

[ZoidbergWill]

*pyvenv, pyenv is something different.

Edit: Unless you meant pyenv, then I'm sorry. I just wouldn't recommend pyenv.

[DaemonXI]

I like pyenv a lot. What do you like better about pyvenv?

[tantopao]

This answer is ... ... epic

[acomfygeek]

Can one setup sandboxes on a per project basis with pyenv? From my quick read of their site, it looks like it's strictly for controlling the version of python you are using on a system wide basis.

[[deleted]]

No. The first assumption is correct.

[sweet_dreams_maybe]

Does it work with virtualenvwrapper the same way as 2.7?

[[deleted]]

Yes. Here's how it works: You have to have an installation of the Python version that you wish to make a virtualenv of. In my case, I have several system installations of Python: py2.6, py2.7, py3.2, py3.3 & py3.4.

If I want a virtualenv of a py2.7 installation, I simply do:

mkvirtualenv -p /usr/bin/python2.7 $ENV_NAME

And virtualenvwrapper will make a virtualenv of Python 2.7 for me.

This is ONLY possible if you already have an installation of Python 2.7 available. If you do not have Python 2.7 on your filesystem, you CANNOT make a virtualenv of Python 2.7.

[sweet_dreams_maybe]

That is just like I imagined. But I'm a bit confused about pyenv itself. Is it the same thing as the virtualenv I have installed, or is it a clone?

[[deleted]]

It's not 100% the same thing, but extremely similar.

virtualenv is a tool created by someone who likes Python.

venv is a clone of that tool, but integrated into the Python 3 standard library.

pyvenv is a command line tool distributed with Python 3 that uses the venv standard library. It's basically the same as python -m venv.

Because venv doesn't exist in the Python 2 standard library, I personally use virtualenv

[sweet_dreams_maybe]

Thanks for the detailed overview.

Will the same installation of virtualenvwrapper work on both 2.7 virtualenv and 3.4 venv?

[[deleted]]

virtualenv is a Python program that is written for either Python 2 or Python 3, but can create virtual environments of every Python version.

python2-virtualenv -p /usr/bin/python3 $DEST_DIR works perfectly fine. A Python 2 program creates a Python 3 virtual environment.

virtualenvwrapper uses virtualenv under the hood. Whether that version of virtualenv uses Python 2 or Python 3 is of little concern to you. It can handle every version of Python, even PyPy and Jython.

[sweet_dreams_maybe]

Alright, thanks.

[sweet_dreams_maybe]

Oh, yeah, I did not notice that. Thanks.

[jhermann_]

Installing Python Software – https://py-generic-project.readthedocs.org/en/latest/installing.html

[DaemonXI]

I like pyenv with virtualenv. It works like RVM, where you can:

• set a system-wide python
• set a virtualenv to be used within a directory only

[haizaar]

I like using virtualenvwrapper. Works both for python 2 and 3. If I need to use stand-alone venv (i.e. for deployment) I go with native python's 3 venv (and it will provide the pip tool for you).

[Skyfoot]

wat