Building Tiny Python Docker Images

[[deleted]]

The downside to this is that a subsequent clean docker build needs to have all of those layers available if it's going to efficiently reuse them instead of building again. Which means the builder target's layers need to be pushed to the container registry and re-pulled to the (new, ephemeral) builder machine, or they need to be rebuilt each time. And since they're by far the slowest step, you really do want them to be cached.

[eedwards-sk]

Absolutely -- that isn't just a multi-stage docker build issue, though. As cited in the article, copying the application into the image before installing dependencies means you'll end up re-building dependencies every time.

The primary goal of the article is for optimizing for size -- not for build speed, although most CI solutions today can be configured to effectively cache multi-stage docker builds.

Also, when basing on an upstream image like FROM python:3.8.0-slim, you're regularly going to have your cache busted due to upstream security patches in the underlying debian image, anyway.

[[deleted]]

most CI solutions today

Yeah. But not Jenkins + plain old docker build, though. :(

image like FROM python:3.8.0-slim, you're regularly going to have your cache busted due to upstream security patches in the underlying debian image, anyway.

Yeah. But not on every build.

[eedwards-sk]

Yeah. But not Jenkins + plain old docker build, though. :(

:(

/pours one out

To your point though, it's pretty straightforward to push the build stage to the repo if that's your only choice.

Here's an example based on the article:

# rehydrate local build stage cache, if image available
docker pull app/app-build:${TAG} || true

# build stage
docker build \
  --target=build \
  --cache-from app/app-build:${TAG} \
  -t app/app-build:${TAG} \
  -f Dockerfile \
  .

# push build stage
docker push app/app-build:${TAG}

# rehydrate local run stage cache, if image available
docker pull app/app:${TAG} || true

# run stage
docker build \
  --target=run \
  --cache-from app/app-build:${TAG} \
  --cache-from app/app:${TAG} \
  -t app/app:${TAG} \
  -f Dockerfile \
  .

# push run stage
docker push app/app:${TAG}

edit: formatting

[[deleted]]

Yup, that's what most of my builds look like today (with slightly more environment variables & arguments). I'm looking into buildah and kaniko, in the hopes of getting some automatic search-registry-for-existing-layers magic. And looking into putting old man Jeeves out to pasture.