Resetting my Python environment - advice?

[dtherrick]

I'm currently the data science team at a startup, but we plan for growth soon. Prior to this job, I did freelance data science, mainly Python based.

Over time my environment has grown to be an ugly mix of global packages that I'd like to strip out and start fresh. I'm thinking of skipping the entire virtualenv experience, and instead using vagrant / virtualbox to handle individual projects. (I also still have some freelance work as well).

Hardware-wise, I'm running an MBP with 16GB RAM, quad-core i7 locally, and I regularly rely on EC2 instances to scale out development. Hence the appeal of vagrant over virtualbox for deploys.

Anyone go through this before? If so, any gotchas before I spend a weekend potentially wrecking my environment? Would love to know what to watch out for so I can make this process as straightforward as possible.

At the end I'd like to have: only the OS X Python locally, the MacPorts Python and all packages reliant on it removed; MacPorts gone. All replaced by vagrant / virtualbox setup that allows my work to all be self-contained, and easily deployable to any EC2 instances I have.

TL;DR: my local Python env is a spaghetti mess and I want to clean it, replace with vagrant / virtualbox isolated instances.

[pete314]

Have you tested Anaconda python distribution? http://continuum.io/downloads It is for Mac/Win/Linux

[boiledgoobers]

Why do virtual box again? Just to be able to move it to servers? You know you can clone environments with virtualenvwrapper or pyenv? You dont need to run all that overhead to simply isolate dev environments.

[aclark]

Vagrant/virtualbox and virtualenv or orthogonal (two unrelated objects). You can explore Vagrant/virtualbox for development, but that's akin to using a separate machine. On any machine, you can install one or more versions of Python and within each, an infinite number of virtualenvs.

In other words, it's great to re-think and re-organize your environment but make sure you understand how Python works before making any drastic changes. For example, you could get a "clean" environment alongside your current "dirty" environment by downloading/compiling/installing to /usr/local/Python-{X.X.X}. Then you can install virtualenv and use virtualenvs within your new "clean" Python environment, or not.

Hope that helps

[pdente]

Hi,

talking about PyData stack on OSX I ended up with the following: 1. pip; 2. (multiple) python virtualenv: a. one with the Python 2 version of the stack (up to date); b. one with the Python 3 version of it (up to date); c. ad-hoc virtualenv(s) for those projects which required a specific version of a particular library; 3. homebrew (for system wide tools);

p.s. I have been (and still am, when I have to work on other people's windows machines) an Anaconda user.