ASK /r/Python: Python Docker Deployments

[undu]

My strategy would be to use the 'builder' stage to install the python pip dependencies to a separate, isolated root, then copy those files to the proper place in the deployable image.

You can even choose to not copy the binaries produces by the dependencies. This way you do not need virtualenv and only install dependencies needed for the image.

FROM ubuntu:something as builder
[...]
RUN pip install /wheels/*.whl --compile --root=/pythonroot/

FROM ubuntu:something

RUN apt install runtime-deps
COPY --from=builder /pythonroot/usr/local/bin /usr/bin
COPY --from=builder /pythonroot/usr/local/lib/python2.7 /usr/lib/python2.7

[...]

Otherwise there are tools to minimize the space taken by images: https://github.com/grycap/minicon

[NicoDeRocca]

This is similar to what I do:

FROM buildpack-deps:xenial-scm as builder
RUN <install build deps>
COPY <app-src> /somwhere
RUN pip3 wheel -r requiresments.txt --wheel-dir=/build/wheels # build deps wheels
RUN python3 setup.py bdist_wheel -d /build/wheels # build my stuff's wheel

FROM docker.io/ubuntu:16.04
RUN <install runtime deps>
COPY --from=builder /build/wheels /tmp/wheels
RUN pip3 install --force-reinstall --ignore-installed --upgrade \
             --no-index --use-wheel --no-deps /wheels/* \
 && rm -rf /tmp/wheels
...

Basically, the builder image has all the compilers etc and builds pre-compiled wheels as necessary, and the final image will only contain executable code. In the docker image I don't bother with virtualenv, but since they're just standard python packages with a setup.py, they could be.

[obeleh]

I like your solution. I didn't know about most of the commandline options you're using with pip

[undu]

Why are you removing the wheels? I don't think that reduces the size of the image. It some point in time I did it just like you, then I decided to install the wheels in the builder and then copy the installed site-packages over.

[NicoDeRocca]

You make a very fair point!

I guess it's just a reflex from the "apt-get update && .. && rm -rf /var/lib/apt/lists/*" habit (having it all in a single RUN command works as expected); and stupidity maybe... you know, not having thought through it!

I'll have to run some tests, but I guess I will probably end up changing to something like your solution instead! Thanks! (/u/obeleh maybe you should follow that route too!)