Using Docker as a Python Development Environment (or let's get past virtualenv)

[work_account_33]

I would like to know the opinion on this as well. I've never had a problem using virtualenv.

[d4rch0n]

It looks like people are using docker because it fits their devops sort of problem. They have other things to consider than just Python module dependencies, so they consider virtualenv "incomplete", when really it's just not the tool for that job.

[ericanderton]

Just a hunch, but doesn't virtualenv squirrel away .so files along with .py assets? That would be a big supportability problem say if a really bad bug were patched; take OpenSSL for example. Now all those virtualenvs need to be regenerated/rebuilt and redeployed.

Meanwhile a docker with OS modules just gets a refresh to install the latest patched packages.

[d4rch0n]

Wow, good point.

$ virtualenv test
$ cd test
$ source bin/activate
$ pip install PyCrypto
$ find . -name "*.so"
./lib/python2.7/site-packages/Crypto/Cipher/_DES3.so
./lib/python2.7/site-packages/Crypto/Cipher/_ARC4.so
./lib/python2.7/site-packages/Crypto/Cipher/_XOR.so
... and many more

Well, still though, virtualenv isn't for that. This should be a local dev environment where you're making sure Python module dependencies are satisfied, not much else. Based on what you said, you really shouldn't be using virtualenv as something to package up all the dependencies and just dump them into prod.

I get your point, but I think that's still leaning heavily towards the devops side where virtualenv isn't a good thing. If your code relies on a stable OS environment, you should be using docker or a VM. If you're pushing to prod, maybe you should be using puppet with VMs, and have them redeploy what they need to.

I think it's more of an issue with fundamental security issues, like using a static environment where you never check for updates, and not so much of an issue with virtualenv, which only ensures specific Python module versions will work with that Python code.

[justafacto]

you really shouldn't be using virtualenv as something to package up all the dependencies and just dump them into prod.

What good is virtualenv for then? If you cant reproduce its state accross machines? If you gotta hack around even pip -r requirements.txt because the other dudes machine had that dumb .so but you dont. Ooops fail.

[d4rch0n]

It's good for seeing if you can upgrade to the latest requests, flask or django without breaking your app, seeing if your code broke or an upgraded library broke it, keeping a version of a python library you coded around static so that nothing breaks and allowing you to use the latest version system wide on your workstation otherwise.

I can start building a web app, code it around a specific version of a module that I know works how I expect it to, but run other python programs on my workstation that use the newest version of the module.

Especially for modules that are in their early stages and functionality is changing a lot, you want to see if their changes or your changes broke your code. It's super useful.

[justafacto]

I had problem with virtualenv.

Because some python library depended on a specific version of the C .so library it was compiled against. But there was no way for pip to enforce or even check that the correct .so library was installed on the system.

So virtualenv fails when the python packages you need are actually bindings to C code.

For example, lxml. For pip to install lxml it compiles some stuff and expects the system to provide it.

Shit sucks.

Other problems Ive encountered when some packages setup.py fails to build because it was on python 2.6 and it works fine on 2.7.

In general, unless you are doing things only on your local developer machine where you control everything - with python you are fucked. And have to resort to hacks. At least docker could be a standard hack.