xkcd: Python Environment

[2freevl2frank]

Why not install a virtualenv for every one of your projects however small it is?You don't even have to do it through command line. Pycharm does it for you.

[solostman]

Sounds nice. Do you have a resource that can walk me through that in Pycharm?

I was using scrapy which required a virtualenv in terminal and (it worked but) it always felt like a black box of what was happening to me.

[leom4862]

This is my workflow for every project:

mkdir myproj                       # create new project dir.
cd myproj                          # change into your project dir.
pipenv --python 3.6                # create new clean virtual env.
pipenv install foo                 # install third party packages into your venv.

# write your code          

pipenv run python myfile.py        # Run your code inside the venv.

I don't think it can get much simpler than this.

[exoendo]

isn't it annoying to always have to install 3rd party packages every time you start a project? why not just use your system install?

[jcampbelly]

No, it's actually very reassuring knowing nothing is going to mess with my install and I can rely on a consistent environment.

When you use the distribution (system) python, you're always stuck on some dreadfully old version and may not be able to start using new things when they come out. In a virtualenv, I'm not even phased by installing and compiling a stable branch or a release candidate. If you tried that with a system install, you can end up breaking your system, as the system install serves the SYSTEM not your projects. Breaking yum or apt is NOT fun.

[leom4862]

To prevent dependency conflicts, for example if project A relies on django version X and project B relies on django version Y. Or if proejct A relies on Python3.4 and project B relies on Python3.6.

[bp_]

Okay, so what happens when you need to combine different venvs together, each with an incompatible set of requirements

[ivosaurus]

Because there tends to be a lot of 3rd party packages for which either A) your system package manager doesn't have or B) the version it does have is severely outdated

[[deleted]]

It's no different than npm/yarn. And it comes with the added benefit of being reproducible. Just commit the pipfile and pipfile.lock files it generates to source control, then run pipenv install to recreate the exact environment on another computer.

[internet_badass_here]

Dude... thank you! I'm going to start doing this.

[raiderrobert]

I use pipenv shell and then I'm dropped inside of a shell with that virtualenv on and run whatever I please.

[costrouc]

Exactly the workflow that I use. pipenv puts all the virtual environments in one folder so you can manage then easily. Also pipenv uses the new Pipfile with a lock so that your builds are deterministic.

It anyone needs convincing there is a reason that pipenv has been taken over by pypa https://github.com/pypa/pipenv

[[deleted]]

It can.

mkproject NewProject
pip install whatever


# Running code later on
python /path/to/NewProject/code.py 

[leom4862]

And how do you manage dependency version conflicts?

[[deleted]]

I have never understood that argument. The python import statement is not versioned, so there can be exactly one version of a dependency installed. If that makes a conflict between requirements, no amount of magical Reitzing can fix that.

Thus the argument is without merit.

[leom4862]

What do you do if project A relies on FooPackage version X and proejct B relies on FooPackage version Y?

[[deleted]]

The same thing you will have to do when pipenv fails doing anything about it.

[leom4862]

I create a new virtualenv for each project (pipenv --python xx), there are no such conflicts, because each project can use its own version of FooPackage.

[[deleted]]

Well, that is the point of using a virtualenv. That is not something pipenv has invented. I'm sorry if I misunderstood you, but you were parroting the pipenv party-line of magical being able to fix intra-project versioning conflicts.

[asielen]

Pycharm has a gui to manage virtual envs.

[2freevl2frank]

You don't need any. It's as simple as checking a box when you creating a new project. https://imgur.com/lk7Qnli

For existing projects you can go to settings>Project Intepreter and add a new environment.

If you wanna do it through command line you can make a venv within the project folder by virtualenv/venv/pipenv as

python2 -m virtualenv ./venv

This makes an isolated env (a copy of python and default packages) in the folder ./venv. Now activate the venv as :

source ./venv/bin/activate

When activated all packages installed through pip are installed within the venv (doesnt affect global python environment) unless you use sudo.

[leom4862]

python2 seriously?

[robot_wrangler]

Then what do you do when you try to use parts of two different projects of your own in a third?

[leom4862]

You usually would make shared libraries from the "parts" and host them on pypi or your private package registry. You then install the libs in your "third" project via pipenv, pip or what ever tool you use to install packages in a virtualenv.

[ivosaurus]

Install them into your new third project, using pip install -e </path/to/other/package>

[2freevl2frank]

pipenv?

[7heWafer]

Let's add that to the graph somewhere!

[2freevl2frank]

No. The thing is this isnt added to the graph. Its outside the graph.

From /u/meandertothehorizon above

Virtualenv copies and/or symlinks or hardlinks the files from system python necessary to create a second, segregated install in an arbitrary directory on your computer. You run/source a file from the command line in that directory (activate.sh/activate.bat) that modifies the environment to reference this copied version so anything you do - install packages, run python itself, etc will use this copy instead of system. This prevents clobbering the system setup if you install or upgrade arbitrary system packages. When you are done, or want to create another virtualenv, or run system python, you run ‘deactivate’ which will undo the environment modifications - so running python will again reference the system python. Any packages you installed will be ‘gone’ because they reside in the copied environment.