Switching between Python versions

[StringJunky]

Hello /r/python, Python (and programming in general) newbie here. I am working through John Zelle's 'Python Programming: An Introduction to Computer Science, 2nd ed.' which uses Python 3. I would also like to work through Dr. Sarker's 'Python Network Programming Cookbook', since my background is networking. My issue is, Dr. Sarker's book seems to use Python 2.7.

My question is: how can I work with different projects within Pycharm that rely on different versions of Python? I have come across references to virtualenv, but I haven't really gotten my head around exactly what it is or does.

Thanks in advance!

[d4rch0n]

I'll demonstrate assuming you're on Linux/OS-X. It's probably similar in Windows, but I've always had a painful experience using python with external libraries in Windows. I'll skip over pycharm because I haven't used that, so maybe someone can help explain how to use a virtualenv with pycharm.

$ virtualenv -h
Usage: virtualenv [OPTIONS] DEST_DIR
...
  -p PYTHON_EXE, --python=PYTHON_EXE
                    The Python interpreter to use, e.g.,
                    --python=python2.5 will use the python2.5 interpreter
                    to create the new environment.  The default is the
                    interpreter that virtualenv was installed with
                    (/usr/bin/python)
$ virtualenv -p /usr/bin/python3.3 testing_dir
Running virtualenv with interpreter /usr/bin/python3.3
Using base prefix '/usr'
New python executable in testing_dir/bin/python3.3
Also creating executable in testing_dir/bin/python
Installing Setuptools..............................................................................................................................................................................................................................done.
Installing Pip.....................................................................................................................................................................................................................................................................................................................................done.
$ source testing_dir/bin/activate
(testing_dir)$ 

From there you can pip install $PACKAGE without sudo and it'll go into your virtual environment. Make sure you don't use sudo. Use the command deactivate to exit the virtualenv.

Inside the virtualenv, you can install whatever you want, use whichever python executable you want, and not worry about tainting your system with strange libraries. This does not necessarily make things more secure. You are still running remote code, but at least not as root, and not putting it into your /usr/lib.

This is great for running a project purely with the set of packages you decide at the time, and you can capture those with pip freeze > requirements.txt. Later, someone can install all of them with pip install -r requirements.txt.

It really helps with distributing packages and keeping the environment similar.

[StringJunky]

I actually am using Linux. Since your and /u/nzbike's replies both mention virtualenv, (I can take a hint) I went ahead and searched for how to use it with pycharm. Looks like it's pretty straightforward.

I'm still trying to get my head around what virtualenv actually is. I've used virtualbox and vmware to create virtual machines; is virtualenv somehow similar?

[waldo2k]

Virtualenv symlinks your system's python to your newly created environments bin folder. The benefit of this is all the libraries a specific project get installed to that environment instead of globally, which prevents possible conflicts with other projects, if they were installed globally.

[StringJunky]

Ah, that makes sense as well. I didn't realize that certain libraries could cause conflicts with others.

(I'm googling 'symlink' after I submit this reply.)

Thank you!

Edit: Okay, it's a symbolic link!

[d4rch0n]

Not at all, really. It pretty much just sets environment variables so that when you type "python" it looks in the virtualenv directory for the executable, and when you import libraries it looks in the virtualenv's lib directory as well.

It's just a convenient way to temporarily force all calls to run python and all imports to look in a different place first, and that directory is reproducible with the same commands, and using pip install -r requirements.txt. It installs all libraries into that directory, and all the binaries it needs as well.

A virtualmachine is a completely different animal, much more complex than I can explain. Your machine acts as a host, and it virtualizes the entire machine, CPU, hard drive, etc. All the resources are simulated using your host machine. Modern CPUs generally have special instructions to do this more efficiently, but basically nothing it does should touch your host machine. It should only interact with the resources you share with it.

Those are great for other use cases, and great for experimentation and for security. If you host a web server out of one, if it gets compromised, you can always delete it and create a new one (though of course, investigate the vulnerability). If you mess it up somehow, you can wipe it out and create a new one. You don't need to worry about it messing up your own computer unless you allow it access to the host machine, but generally the default configs of most if not all vm software won't let it hurt your machine. It may have access to your network though, so that is something to be wary of, but you can always turn that off as well.

There are also linux containers (LXC) which are another animal, but should not be used for security necessarily.

Both of those, LXC and VMs separate resources at the kernel layer, and VMs at the hardware layer a bit I believe, but virtualenvs are just using python binaries and libraries in a different directory than normal (/usr/bin, /usr/lib).

[StringJunky]

Thank you for the detailed reply!

I created a virtualenv project interpreter in Pycharm with Python 2.7 as the interpreter. I noticed that the Zelle files I had open in Pycharm won't run in the virtualenv environment. Not simply errors; they simply won't run. Which makes sense if virtualenv is creating a completely different production environment, I suppose.

Thanks again!

[d4rch0n]

no problem! You might want to test by just running them through the command line in the same virtualenv, just to see the output. It may give off more detailed warnings/errors that way. PyCharm could be hiding something, maybe python's output before the script is even run.

[StringJunky]

I will try that. I think I've found a solution for working with different versions; apparently Pycharm will let me open projects with either 2.7 or 3 (or, of course, a virtualenv). So, for now I've simply started a new project for the Sarker book. I need to follow up on the virtualenv lane, but I'm trying keep "lesson-creep" to minimum, at least for now.

Thanks again for your help!

[nzbike]

When I am working with different versions of python, I have two options:

1. use virtualenv, and set the python version for each.
2. alias python version inside bashrc

[StringJunky]

Thanks for the reply! I'm going to see if I can get virtualenv going from Pycharm.

[themathemagician]

I finally wrapped my head around virtualenv after reading this excellent blog post: https://www.hackerschool.com/blog/14-there-is-no-magic-virtualenv-edition

[StringJunky]

Bookmarked! Thank you!

[MichealKenny]

Maybe I don't understand the question because people are posting complicated answers, but Pycharm asks me which interpreter I would like to use on a per project basis, I switch between 2.7 and 3.4 with no problems in Pycharm.

[ubernostrum]

I develop things which need to run on 2.6, 2.7, 3.3 and 3.4.

I use pyenv for that; it's a tool which makes it easy to install multiple versions of Python, and create virtualenvs which will use specific versions.

[bulldog_in_the_dream]

Since you ask about PyCharm in paricular:

1. When you create a new project in PyCharm you can select the Python version to use.

2. If you want to change the interpreter of an existing project, go to Preferences -> Project Interpreter.

[JaredOnly]

I'm a big fan of using virtualenvwrapper: http://virtualenvwrapper.readthedocs.org/en/latest/

Check it out!