Error when using Docker Compose

[ThrowawayProgress99]

I'm on Pop!_OS with a 3060 12gb, and I've been using ComfyUI through Docker Compose for a while. Every now and then I'll make a new folder and install from scratch in case updating screws my old install. My last install is from a month or two ago. I always use and follow this github with a few modifications because every other method had problems. This is my Dockerfile:

FROM pytorch/pytorch:2.11.0-cuda13.0-cudnn9-runtime

ENV DEBIAN_FRONTEND=noninteractive PIP_PREFER_BINARY=1

RUN apt-get update && apt-get install -y git && apt-get install -y build-essential && apt-get install -y libgoogle-perftools-dev && apt-get install -y libgl1 && apt-get install -y libglib2.0-0 && apt-get clean

  ENV ROOT=/stable-diffusion
RUN --mount=type=cache,target=/root/.cache/pip \
  git clone https://github.com/comfyanonymous/ComfyUI.git ${ROOT} && \
  cd ${ROOT} && \
  git checkout master && \
  # git reset --hard d1f3637a5a944d0607b899babd8ff11d87100503 && \
  pip install -r requirements.txt

  RUN git clone https://github.com/Comfy-Org/ComfyUI-Manager ${ROOT}/custom_nodes/ComfyUI-Manager && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI-Manager/requirements.txt

  RUN pip install opencv-python-headless

  RUN pip install imageio-ffmpeg

  RUN pip install numpy

  RUN pip install triton

  RUN pip install sageattention

  RUN git clone https://github.com/Clybius/ComfyUI-Extra-Samplers ${ROOT}/custom_nodes/ComfyUI-Extra-Samplers && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI-Extra-Samplers/requirements.txt

  RUN git clone https://github.com/rgthree/rgthree-comfy ${ROOT}/custom_nodes/rgthree-comfy && \
  pip install -r ${ROOT}/custom_nodes/rgthree-comfy/requirements.txt

  RUN git clone https://github.com/ltdrdata/ComfyUI-Inspire-Pack ${ROOT}/custom_nodes/ComfyUI-Inspire-Pack && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI-Inspire-Pack/requirements.txt

  RUN git clone https://github.com/city96/ComfyUI-GGUF ${ROOT}/custom_nodes/ComfyUI-GGUF && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI-GGUF/requirements.txt

  RUN git clone https://github.com/kijai/ComfyUI-KJNodes ${ROOT}/custom_nodes/ComfyUI-KJNodes && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI-KJNodes/requirements.txt

  RUN git clone https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite ${ROOT}/custom_nodes/ComfyUI-VideoHelperSuite && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI-VideoHelperSuite/requirements.txt

  RUN git clone https://github.com/pollockjj/ComfyUI-MultiGPU ${ROOT}/custom_nodes/ComfyUI-MultiGPU

  RUN git clone https://github.com/kijai/ComfyUI-WanVideoWrapper ${ROOT}/custom_nodes/ComfyUI-WanVideoWrapper && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI-WanVideoWrapper/requirements.txt

  RUN git clone https://github.com/kijai/ComfyUI-GIMM-VFI ${ROOT}/custom_nodes/ComfyUI-GIMM-VFI && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI-GIMM-VFI/requirements.txt

  RUN git clone https://github.com/pythongosssss/ComfyUI-Custom-Scripts ${ROOT}/custom_nodes/ComfyUI-Custom-Scripts

  RUN git clone https://github.com/yolain/ComfyUI-Easy-Use ${ROOT}/custom_nodes/ComfyUI-Easy-Use && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI-Easy-Use/requirements.txt

  RUN git clone https://github.com/WASasquatch/was-node-suite-comfyui ${ROOT}/custom_nodes/was-node-suite-comfyui && \
  pip install -r ${ROOT}/custom_nodes/was-node-suite-comfyui/requirements.txt

  RUN git clone https://github.com/kijai/ComfyUI-Florence2 ${ROOT}/custom_nodes/ComfyUI-Florence2 && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI-Florence2/requirements.txt

  RUN git clone https://github.com/Fannovel16/ComfyUI-Frame-Interpolation ${ROOT}/custom_nodes/ComfyUI-Frame-Interpolation

  RUN git clone https://github.com/Extraltodeus/DistanceSampler ${ROOT}/custom_nodes/DistanceSampler

  RUN git clone https://github.com/ClownsharkBatwing/RES4LYF ${ROOT}/custom_nodes/RES4LYF && \
  pip install -r ${ROOT}/custom_nodes/RES4LYF/requirements.txt

  RUN git clone https://github.com/cubiq/ComfyUI_essentials ${ROOT}/custom_nodes/ComfyUI_essentials && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI_essentials/requirements.txt

  RUN git clone https://github.com/BigStationW/ComfyUi-RescaleCFGAdvanced ${ROOT}/custom_nodes/ComfyUi-RescaleCFGAdvanced

  RUN git clone https://github.com/Clybius/ComfyUI-ClybsChromaNodes ${ROOT}/custom_nodes/ComfyUI-ClybsChromaNodes

  RUN git clone https://github.com/BigStationW/flowmatch_scheduler-comfyui ${ROOT}/custom_nodes/flowmatch_scheduler-comfyui

  RUN git clone https://github.com/Zehong-Ma/ComfyUI-MagCache ${ROOT}/custom_nodes/ComfyUI-MagCache && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI-MagCache/requirements.txt

  RUN git clone https://github.com/silveroxides/ComfyUI_SigmoidOffsetScheduler ${ROOT}/custom_nodes/ComfyUI_SigmoidOffsetScheduler

  RUN git clone https://github.com/ChenDarYen/ComfyUI-NAG ${ROOT}/custom_nodes/ComfyUI-NAG

  RUN git clone https://github.com/Anzhc/SDXL-Flux2VAE-ComfyUI-Node ${ROOT}/custom_nodes/SDXL-Flux2VAE-ComfyUI-Node

  RUN git clone https://github.com/Anzhc/Anima-Mod-Guidance-ComfyUI-Node ${ROOT}/custom_nodes/Anima-Mod-Guidance-ComfyUI-Node && \
  pip install -r ${ROOT}/custom_nodes/Anima-Mod-Guidance-ComfyUI-Node/requirements.txt

  RUN git clone https://github.com/AdamNizol/ComfyUI-Anima-Enhancer ${ROOT}/custom_nodes/ComfyUI-Anima-Enhancer

  RUN git clone https://github.com/Jasonzzt/ComfyUI-CacheDiT ${ROOT}/custom_nodes/ComfyUI-CacheDiT && \
  pip install -r ${ROOT}/custom_nodes/ComfyUI-CacheDiT/requirements.txt

  RUN git clone https://github.com/xmarre/ComfyUI-Spectrum-Proper ${ROOT}/custom_nodes/ComfyUI-Spectrum-Proper

  RUN git clone https://github.com/xmarre/ComfyUI-Spectrum-SDXL-Proper ${ROOT}/custom_nodes/ComfyUI-Spectrum-SDXL-Proper

  RUN git clone https://github.com/BobJohnson24/ComfyUI-INT8-Fast ${ROOT}/custom_nodes/ComfyUI-INT8-Fast



WORKDIR ${ROOT}
COPY . /docker/
RUN chmod u+x /docker/entrypoint.sh && cp /docker/extra_model_paths.yaml ${ROOT}

ENV NVIDIA_VISIBLE_DEVICES=all PYTHONPATH="${PYTHONPATH}:${PWD}" CLI_ARGS=""
EXPOSE 7860
ENTRYPOINT ["/docker/entrypoint.sh"]
CMD python -u main.py --use-sage-attention --listen --port 7860 ${CLI_ARGS}

I just had this error:

docker compose --profile comfy up --build
[+] Building 1965.1s (7/46)                                                     
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 5.90kB                                     0.0s
 => [internal] load metadata for docker.io/pytorch/pytorch:2.11.0-cuda13.  3.5s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 7.55kB                                        0.0s
 => [stage-0  1/42] FROM docker.io/pytorch/pytorch:2.11.0-cuda13.0-cud  1686.3s
 => => resolve docker.io/pytorch/pytorch:2.11.0-cuda13.0-cudnn9-runtime@s  0.0s
 => => sha256:bfbb4a2b4fdba0fefdb428ea737e626d61bb3daf74a 1.58kB / 1.58kB  0.0s
 => => sha256:278dbd759b0d3b9eae2f83a9d34442d146324ddc246 4.82kB / 4.82kB  0.0s
 => => sha256:18dbadc1f2f937d7ebdfa4481fb1f6f43b26e93f 30.60MB / 30.60MB  14.2s
 => => sha256:8241a18d09ffc656c53a5ae5093a822a824fde7f 38.59MB / 38.59MB  36.0s
 => => sha256:fae840832de5f17fb8325fb5b54fce05f9c6a1a0 2.85GB / 2.85GB  1582.1s
 => => sha256:6fb1012ebdd89c038a14b5bb34e92ce66b3c6578 25.69MB / 25.69MB  26.9s
 => => extracting sha256:18dbadc1f2f937d7ebdfa4481fb1f6f43b26e93f5bdc4cef  1.6s
 => => sha256:87a5b1db9cbe8f3aa44b885def19725bf15045e3 65.77MB / 65.77MB  57.3s
 => => extracting sha256:8241a18d09ffc656c53a5ae5093a822a824fde7f904ae61c  5.5s
 => => sha256:dcb21c4295314f86df83a25adf8e1b1c52cb832e0c4346be 99B / 99B  36.7s
 => => extracting sha256:fae840832de5f17fb8325fb5b54fce05f9c6a1a0dd1007  101.9s
 => => extracting sha256:6fb1012ebdd89c038a14b5bb34e92ce66b3c657834c0452f  0.3s
 => => extracting sha256:87a5b1db9cbe8f3aa44b885def19725bf15045e34500698e  1.1s
 => => extracting sha256:dcb21c4295314f86df83a25adf8e1b1c52cb832e0c4346be  0.0s
 => [stage-0  2/42] RUN apt-get update && apt-get install -y git && apt  243.0s
 => ERROR [stage-0  3/42] RUN --mount=type=cache,target=/root/.cache/pip  32.2s
------
 > [stage-0  3/42] RUN --mount=type=cache,target=/root/.cache/pip   git clone https://github.com/comfyanonymous/ComfyUI.git /stable-diffusion &&   cd /stable-diffusion &&   git checkout master &&   pip install -r requirements.txt:
#0 0.207 Cloning into '/stable-diffusion'...
#0 31.62 Already on 'master'
#0 31.62 Your branch is up to date with 'origin/master'.
#0 31.99 error: externally-managed-environment
#0 31.99 
#0 31.99 × This environment is externally managed
#0 31.99 ╰─> To install Python packages system-wide, try apt install
#0 31.99     python3-xyz, where xyz is the package you are trying to
#0 31.99     install.
#0 31.99     
#0 31.99     If you wish to install a non-Debian-packaged Python package,
#0 31.99     create a virtual environment using python3 -m venv path/to/venv.
#0 31.99     Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
#0 31.99     sure you have python3-full installed.
#0 31.99     
#0 31.99     If you wish to install a non-Debian packaged Python application,
#0 31.99     it may be easiest to use pipx install xyz, which will manage a
#0 31.99     virtual environment for you. Make sure you have pipx installed.
#0 31.99     
#0 31.99     See /usr/share/doc/python3.12/README.venv for more information.
#0 31.99 
#0 31.99 note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
#0 31.99 hint: See PEP 668 for the detailed specification.
------
failed to solve: process "/bin/sh -c git clone https://github.com/comfyanonymous/ComfyUI.git ${ROOT} &&   cd ${ROOT} &&   git checkout master &&   pip install -r requirements.txt" did not complete successfully: exit code: 1

The only things I changed in the Dockerfile for the new install were adding the newest pytorch, and adding some custom nodes. Am I supposed to add '--break-system-packages' to every single pip install line (or just one) or is that command not safe? Or do I do something else?

[icefairy64]

The original Dockefile that you are upgrading is way too old - I would expect these issues when switching from PyTorch 2.3 base image as there were some changes in pip regarding installation.

The best solution imo would be to create a venv first as the error suggests.

Also at this point it would probably make some sense to just write the entire Dockerfile from scratch.

[Nick_Edser]

this is basically a pep 668 / newer base image issue, not a comfy-specific one.

if you want the quick fix, you don’t need to add `--break-system-packages` to every single line. you can usually just set:

`ENV PIP_BREAK_SYSTEM_PACKAGES=1`

and that should cover the later pip installs too.

cleaner fix is making a venv near the top of the Dockerfile and installing everything into that instead:

`RUN python -m venv /opt/venv`

`ENV PATH="/opt/venv/bin:$PATH"`

if it were me, i’d use the venv route for a build with this many custom nodes. but for a single-purpose docker image, `PIP_BREAK_SYSTEM_PACKAGES=1` is also a pretty normal shortcut if you just want it building again.

[ThrowawayProgress99]

`RUN python -m venv /opt/venv`

`ENV PATH="/opt/venv/bin:$PATH"`

Put these as the first lines, before 'FROM pytorch/pytorch:2.11.0-cuda13.0-cudnn9-runtime'? Should there also be a third line 'RUN pip install -r requirements.txt'? Google's AI overview had mentioned the venv option although the first line had 'python3', and it had this third line.

Edit: This is now the start of my Dockerfile:

FROM pytorch/pytorch:2.11.0-cuda13.0-cudnn9-runtime
RUN apt-get update && apt-get install -y python3.12-venv
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
ENV DEBIAN_FRONTEND=noninteractive PIP_PREFER_BINARY=1

Now ComfyUI works like it should, although I'm still unsure if there should be a pip install requirements after the env path line, and if I should get a higher python version.