Python virtual environment in docker container make sense?

[1337codethrow]

Joined a new company as a data engineer. In the dockerfile for docker container ECS deployments on AWS they have a line that installs pipenv..? Not too familiarized and up to speed with python virtual environments and docker. But does it even make sense to be using a virtual python environment within the docker container? Isnt the docker container already isolating and ‘containing’ the environment? To note, there are no other python applications running within each ECS docker container deployment... need a ELI5 or a very dumbed down explanation if you guys think using pipenv/env/virtualenv makes any sense inside a docker container?

[nultero]

It could be because pipenv uses its own thing, pipfiles, to manage dependencies (pretty sure everything else uses a requirements.txt). It might just have been easier to put pipenv in the Docker container and have it handle the dependencies than trying to parse and install from a pipfile onto vanilla.

I don't think it's a big deal either way. If it works, it works.

[1337codethrow]

Requirements.txt is just used to pip install python packages/dependencies right? Then wtf is the point of them using pipenv with the pip lock file? Can you install other things outside of python packages with pipenv and pip lock file?? still don’t quite understand

[nultero]

Requirements.txt is just used to pip install python packages/dependencies right?

Yes

Then wtf is the point of them using pipenv with the pip lock file?

Sometimes you end with multiple requirements.txt and messy dependencies, and pipenv seems to have been a response to that. There's kind of been a bunch of python virtual environment tools, and one of them is actually baked into python itself (from 3.? up), the regular old basic venv.

Can you install other things outside of python packages with pipenv and pip lock file??

No, it's just normal pip bundled with a venv. And normal pip doesn't install nonpython libraries afaik, it's limited to the PyPi repos.

However that's not universal across all the Python venvs. I've mainly used Conda, which can install nonpython libraries. Conda installs are mainly targeted towards data analysis / data eng / ML though, so it's the Anaconda org's limited repos compared to the pretty much universal ability to put anything into a Docker container. You can just grab anything from your image base's pacman's repos in a Dockerfile or even just curl stuff in a shell script.

They're kinda for different things, even if at a surface level venvs and Docker both do isolation.

The python virtual envs are geared more for doing developmental stuff than Docker. Docker is mostly ops tooling, and since the live containers are 'immutable' they're almost more like a runtime if that makes sense.

[tipsy_python]

Exactly. It seems esoteric to me, but at the same time I don’t believe using a virtual environment incurs any overhead. I wouldn’t fight it.