[GUIDE] Stable Diffusion CPU, CUDA, ROCm with Docker-compose

[catzy007]

[UPDATE 28/11/22] I have added support for CPU, CUDA and ROCm. CPU and CUDA is tested and fully working, while ROCm should "work".

Preparing your system

Install docker and docker-compose and make sure docker-compose version 1.72.0 or later is installed.

Then install NVIDIA Container Toolkit or Follow ROCm Docker Quickstart.

For windows, follow CUDA on WSL User Guide then Enabling the Docker Repository and Installing the NVIDIA Container Toolkit

Initial set-up

Create new directory and save the following files.

docker-compose.yml

version: '3'

services:
  stablediff-cpu:
    build: 
      context: .
      dockerfile: Dockerfile.cpu
    container_name: stablediff-cpu-runner
    environment:
      TZ: "Asia/Jakarta"
      COMMANDLINE_ARGS: "--listen --no-half --skip-torch-cuda-test"
    entrypoint: ["/bin/sh", "-c"]
    command: >
      ". /stablediff.env; echo launch.py $$COMMANDLINE_ARGS;
      if [ ! -d /stablediff-web/.git ]; then
        cp -a /sdtemp/. /stablediff-web/
      fi;
      if [ ! -f /stablediff-web/models/Stable-diffusion/*.ckpt ]; then
        echo 'Please copy stable diffusion model to stablediff-models directory'
        echo 'You may need sudo to perform this action'
        exit 1
      fi;
      python launch.py"
    ports:
      - "7860:7860"
    volumes:
      - ./stablediff.env:/stablediff.env
      - ./stablediff-web:/stablediff-web
      - ./stablediff-models:/stablediff-web/models/Stable-diffusion
  stablediff-rocm:
    build: 
      context: .
      dockerfile: Dockerfile.rocm
    container_name: stablediff-rocm-runner
    environment:
      TZ: "Asia/Jakarta"
      ROC_ENABLE_PRE_VEGA: 1
      COMMANDLINE_ARGS: "--listen --precision full --no-half"
    entrypoint: ["/bin/sh", "-c"]
    command: >
      "rocm-smi; . /stablediff.env; echo launch.py $$COMMANDLINE_ARGS;
      if [ ! -d /stablediff-web/.git ]; then
        cp -a /sdtemp/. /stablediff-web/
      fi;
      if [ ! -f /stablediff-web/models/Stable-diffusion/*.ckpt ]; then
        echo 'Please copy stable diffusion model to stablediff-models directory'
        echo 'You may need sudo to perform this action'
        exit 1
      fi;
      python launch.py"
    ports:
      - "7860:7860"
    devices:
      - "/dev/kfd:/dev/kfd"
      - "/dev/dri:/dev/dri"
    group_add:
      - video
    ipc: host
    cap_add:
      - SYS_PTRACE
    security_opt:
      - seccomp:unconfined
    volumes:
      - ./stablediff.env:/stablediff.env
      - ./stablediff-web:/stablediff-web
      - ./stablediff-models:/stablediff-web/models/Stable-diffusion
  stablediff-cuda:
    build: 
      context: .
      dockerfile: Dockerfile.cuda
    container_name: stablediff-runner-cuda
    runtime: nvidia
    environment:
      TZ: "Asia/Jakarta"
      NVIDIA_VISIBLE_DEVICES: all
      COMMANDLINE_ARGS: "--listen"
    entrypoint: ["/bin/sh", "-c"]
    command: >
      "nvidia-smi; . /stablediff.env; echo launch.py $$COMMANDLINE_ARGS;
      if [ ! -d /stablediff-web/.git ]; then
        cp -a /sdtemp/. /stablediff-web/
      fi;
      if [ ! -f /stablediff-web/models/Stable-diffusion/*.ckpt ]; then
        echo 'Please copy stable diffusion model to stablediff-models directory'
        echo 'You may need sudo to perform this action'
        exit 1
      fi;
      python launch.py"
    ports:
      - "7860:7860"
    volumes:
      - ./stablediff.env:/stablediff.env
      - ./stablediff-web:/stablediff-web
      - ./stablediff-models:/stablediff-web/models/Stable-diffusion

Dockerfile.cpu

FROM python:3.10.6-bullseye
ENV DEBIAN_FRONTEND=noninteractive \
    PYTHONUNBUFFERED=1 \
    PYTHONIOENCODING=UTF-8
WORKDIR /sdtemp
RUN python -m pip install --upgrade pip wheel
RUN apt-get update &&\
    apt-get install -y wget git
RUN git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui /sdtemp

#torch and torchvision version number refer to
#https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/launch.py
ENV TORCH_COMMAND="pip install torch==1.12.1+cpu torchvision==0.13.1+cpu --extra-index-url https://download.pytorch.org/whl/cpu"
RUN python -m $TORCH_COMMAND

RUN python launch.py --skip-torch-cuda-test --exit
RUN python -m pip install opencv-python-headless
WORKDIR /stablediff-web

Dockerfile.cuda

FROM nvidia/cuda:11.3.1-base-ubuntu20.04
ENV DEBIAN_FRONTEND=noninteractive \
    PYTHONUNBUFFERED=1 \
    PYTHONIOENCODING=UTF-8
WORKDIR /sdtemp
RUN apt-get update &&\
    apt-get install -y \
    wget \
    git \
    python3 \
    python3-pip \
    python-is-python3
RUN python -m pip install --upgrade pip wheel
RUN git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui /sdtemp

#torch and torchvision version number refer to
#https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/launch.py
ENV TORCH_COMMAND="pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113"
RUN python -m $TORCH_COMMAND

RUN python launch.py --skip-torch-cuda-test --exit
RUN python -m pip install opencv-python-headless
WORKDIR /stablediff-web

Dockerfile.rocm

FROM rocm/dev-ubuntu-20.04
ENV DEBIAN_FRONTEND=noninteractive \
    PYTHONUNBUFFERED=1 \
    PYTHONIOENCODING=UTF-8
WORKDIR /sdtemp
RUN apt-get update &&\
    apt-get install -y \
    wget \
    git \
    python3 \
    python3-pip \
    python-is-python3
RUN python -m pip install --upgrade pip wheel
RUN git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui /sdtemp

#torch and torchvision version number refer to
#https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/launch.py
ENV TORCH_COMMAND="pip install torch==1.12.1+rocm5.1.1 torchvision==0.13.1+rocm5.1.1 --extra-index-url https://download.pytorch.org/whl/rocm5.1.1"
RUN python -m $TORCH_COMMAND

RUN python launch.py --skip-torch-cuda-test --exit
RUN python -m pip install opencv-python-headless
WORKDIR /stablediff-web

stablediff.env

export COMMANDLINE_ARGS="--listen"

.dockerignore

stablediff-web/
stablediff-models/
*.ckpt

Building docker image

This will download and install the required packages.

• Using CPU

  docker-compose build stablediff-cpu

• Using CUDA

  docker-compose build stablediff-cuda

• Using ROCm

  docker-compose build stablediff-rocm

Setting up launch parameter

Edit stablediff.env to match your use cases.
You can also add launch parameter such as --lowvram.

• Using CPU

  export COMMANDLINE_ARGS="--listen --no-half --skip-torch-cuda-test"

• Using CUDA

  export COMMANDLINE_ARGS="--listen"

• Using ROCm

  export COMMANDLINE_ARGS="--listen --precision full --no-half"

Initial run

This will create two directory called stablediff-web and stablediff-models.
After initial run, you will get message about missing Stable Diffusion ckpt model.
Grab one and copy it to stablediff-models. in linux, you might need sudo to
do this.

• Using CPU

  docker-compose up stablediff-cpu

• Using CUDA

  docker-compose up stablediff-cuda

• Using ROCm

  docker-compose up stablediff-rocm

Subsequent run

Use the command below every time you want to run Stable Diffusion.

• Using CPU

  docker start -a stablediff-cpu-runner

• Using CUDA

  docker start -a stablediff-cuda-runner

• Using ROCm

  docker start -a stablediff-rocm-runner

Stopping Stable Diffusion

To stop Stable Diffusion press Ctrl + C and use the command below.

• Using CPU

  docker stop stablediff-cpu-runner

• Using CUDA

  docker stop stablediff-cuda-runner

• Using ROCm

  docker stop stablediff-rocm-runner

Testing

I run this using Ubuntu 22.04 with Dual Xeon X5670, 12 GB RAM and Polaris 11 4 GB GPU.
Generating 512x512 image using CPU i get around 17.44s/it using around 8~9 GB RAM.

I also run this using Ubuntu 20.04 with Maxwell 2 GB GPU. Generating 512x512 image,
i get around 5.12s/it.

If you managed to use this method please comment below.
Also this method should work with windows, personally i have not try it but i would like to hear that.

[D-Impaller]

Sorry for the ignorance, but what it is this? I mean, what is its purpose?

[catzy007]

This is stable diffusion web ui wrapped in docker container which is useful is you want to deploy this on a cloud server (GCP AWS). This is also useful if you are in large organization (corporate/university) with shared GPU resources.

[D-Impaller]

Thanks a lot for the explanation!

[catzy007]

No problem, hope this helps.

[whew-inc]

#5680

I'm experiencing the same issue. 5700xt/2700x.
Edit: No external hard drive is used

[catzy007]

Unfortunately at least at the time of writing, you need specific combination of hardware, software and workaround to use ROCm. Here is some of the links that may help.

https://docs.amd.com/bundle/Hardware_and_Software_Reference_Guide/page/Hardware_and_Software_Support.html

https://github.com/RadeonOpenCompute/ROCm-docker/blob/master/quick-start.md

ROCm/ROCm#1714

ROCm/ROCm#887

https://github.com/RadeonOpenCompute/ROCm/issues

https://stackoverflow.com/questions/73229163/amd-rocm-with-pytorch-on-navi10-rx-5700-rx-5700-xt

[JOHNFFFEE]

Hi @catzy007 , thanks for the images code.
I ran the docker-composer, and it created 3 folders instead of one as it was earlier, so I copied the models and run the docker image without putting any variables into the volumes and path

Now when I am trying to reach the api on "http://127.0.0.1:7860/docs" I can't access it.
Where am I wrong ?
The structure of the folder

I will be glad if you can post a video of the whole process?

[catzy007]

From what i can tell, it seems like you modify the docker-compose or dockerfile. This is completely fine if you know what you're doing. I suggest you follow the guide again using cmd/powershell or adapt your modification to the guide. For reference here is what docker ps -a and docker image ls should look like.

76d332401a8d   mystablediff_stablediff-cpu         "/bin/sh -c '. /stab…"   7 weeks ago   Up About a minute        0.0.0.0:7860->7860/tcp, :::7860->7860/tcp   stablediff-cpu-runner

mystablediff_stablediff-cpu         latest                  3473d1dfde16   7 weeks ago    4.28GB

[JOHNFFFEE]

I didn't touched them, I copied them from the code above.
I only made some changes in the api, which were working after I verified them.

let's start from the beginning, where should I put the docker files. inside the a1111 folder or above?

[catzy007]

Without being disrespectful i suggest you slowly follow the guide again using CMD/Powershell instead of Docker Desktop and make sure everything is working properly. Then apply your modification to the project afterwards.

Good Luck!

[catzy007]

Here is a quick summary if you're using CPU.

$ pwd
/home/catzy/Public/myStablediff
$ tree -FL 1
./
├── docker-compose.yml
├── Dockerfile.cpu
├── Dockerfile.cuda
├── Dockerfile.rocm
├── stablediff.env*
├── stablediff-models/
└── stablediff-web/

2 directories, 5 files
$ docker-compose build stablediff-cpu
$ docker image ls
REPOSITORY                            TAG                     IMAGE ID       CREATED        SIZE
myStablediff_stablediff-cpu         latest                  3473d1dfde16   7 weeks ago    4.28GB
$ docker-compose up stablediff-cpu
Copy stable diffusion model to /home/catzy/Public/myStablediff/stablediff-models
$ docker start -a stablediff-cpu-runner
$ docker ps -a
CONTAINER ID   IMAGE                                 COMMAND                  CREATED       STATUS                     PORTS                                   NAMES
76d332401a8d   myStablediff_stablediff-cpu         "/bin/sh -c '. /stab…"   7 weeks ago   Up About a minute        0.0.0.0:7860->7860/tcp, :::7860->7860/tcp   stablediff-cpu-runner
Open Web browser and go to http://localhost:7860/
$ docker stop stablediff-cpu-runner

[RhanCandia]

Here is a quick summary if you're using CPU.

$ pwd
/home/catzy/Public/myStablediff
$ tree -FL 1
./
├── docker-compose.yml
├── Dockerfile.cpu
├── Dockerfile.cuda
├── Dockerfile.rocm
├── stablediff.env*
├── stablediff-models/
└── stablediff-web/

2 directories, 5 files
$ docker-compose build stablediff-cpu
$ docker image ls
REPOSITORY                            TAG                     IMAGE ID       CREATED        SIZE
myStablediff_stablediff-cpu         latest                  3473d1dfde16   7 weeks ago    4.28GB
$ docker-compose up stablediff-cpu
Copy stable diffusion model to /home/catzy/Public/myStablediff/stablediff-models
$ docker start -a stablediff-cpu-runner
$ docker ps -a
CONTAINER ID   IMAGE                                 COMMAND                  CREATED       STATUS                     PORTS                                   NAMES
76d332401a8d   myStablediff_stablediff-cpu         "/bin/sh -c '. /stab…"   7 weeks ago   Up About a minute        0.0.0.0:7860->7860/tcp, :::7860->7860/tcp   stablediff-cpu-runner
Open Web browser and go to http://localhost:7860/
$ docker stop stablediff-cpu-runner

Would it be possible to run this on arm64?
I'm failing the CPU build on this part.

Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cpu
ERROR: Could not find a version that satisfies the requirement torch==1.12.1+cpu (from versions: 1.10.2, 1.11.0, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 2.0.0)
ERROR: No matching distribution found for torch==1.12.1+cpu
The command '/bin/sh -c python -m $TORCH_COMMAND' returned a non-zero code: 1
ERROR: Service 'stablediff-cpu' failed to build : Build failed

EDIT: I removed the +cpu on the requested version install and it went through. I doubt it will work tho. I'm at the step where I added a model now. But it does not recognize safetrensors. Is that correct?

EDIT 2: I used a ckpt model and now it launches. I can't generate images though because I get stuck here:
0%| | 0/20 [00:00<?, ?it/s]
Maybe it really broke when I installed torch and torchvision without the +cpu tag.

EDIT 3: Oh wait, actually. It does work.

  5%|▌         | 1/20 [02:07<40:28, 127.80s/it]
Total progress:   0%|          | 0/20 [00:00<?, ?it/s]

127.80s per iteration, wow. I mean, of course, because it's cpu only. Haha.

[falliwillfollow]

Minor typo in above.
In the section for "Subsequent run" the container name should be stablediff-runner-, not
stablediff--runner

[ClericerQ]

Thank you so much, for this work, i thought to give up to try a cpu version an then i found this thread. (Thank you very much for this)
Can anyone tell me how to increase the ammount of CPU cores used? it allways uses the half of the available cores, 8cores / 4 are used, 16Cores 8 are used and so on.

[sumersm7]

thanks for the code it works, i tested cpu only.

iam new to docker and dockercompose. is there a way to convert docker-compose.yml to simple dockerfile as i use caprover to manage docker and it cant handel docker-compose.yml.

so when i try to conver docker-compose.yml to docker file using GPT it doesnt run.

help appreciated

[grigio]

Thanks @catzy007 !!! Finally SD + ROCM working!

I tried to update the dependencies from

ENV TORCH_COMMAND="pip install torch==1.12.1+rocm5.1.1 torchvision==0.13.1+rocm5.1.1 --extra-index-url https://download.pytorch.org/whl/rocm5.1.1"

to

ENV TORCH_COMMAND="pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm5.4.2"

but I get the error below, i possible to upgrade torch to the latest stable ?

stablediff-rocm-runner | launch.py --listen --precision full --no-half --skip-torch-cuda-test
stablediff-rocm-runner | Python 3.8.10 (default, May 26 2023, 14:05:08) 
stablediff-rocm-runner | [GCC 9.4.0]
stablediff-rocm-runner | Version: v1.5.1
stablediff-rocm-runner | Commit hash: 68f336bd994bed5442ad95bad6b6ad5564a5409a
stablediff-rocm-runner | Launching Web UI with arguments: --listen --precision full --no-half --skip-torch-cuda-test
stablediff-rocm-runner | no module 'xformers'. Processing without...
stablediff-rocm-runner | no module 'xformers'. Processing without...
stablediff-rocm-runner | No module 'xformers'. Proceeding without it.
stablediff-rocm-runner | Loading weights [6ce0161689] from /stablediff-web/models/Stable-diffusion/Stable-diffusion/v1-5-pruned-emaonly.safetensors
stablediff-rocm-runner | 
stablediff-rocm-runner | rocBLAS error: Cannot read /usr/local/lib/python3.8/dist-packages/torch/lib/rocblas/library/TensileLibrary.dat: Illegal seek
stablediff-rocm-runner | Aborted (core dumped)
stablediff-rocm-runner exited with code 134

[xuanmumy]

I'm runing the build command now, and also meet problem downloading torch.
I font some answer online and change to ENV TORCH_COMMAND="pip install torch==1.12.1+cpu torchvision==0.13.1+cpu -f https://download.pytorch.org/whl/torch_stable.html"
now it is still downloading. dont know whether it will pass or not.

[maxemoose]

Used this method to create a CPU runner and it works very well. Only made 2 minor edits to the process.

1: Changed the TORCH_COMMAND in Dockerfile to:

ENV TORCH_COMMAND="pip install torch==2.1.2+cpu torchvision==0.16.2+cpu --extra-index-url https://download.pytorch.org/whl/cpu"

2: Changed the second if statement in the docker-compose.yml starting command to:

if [ ! -f /stablediff-web/models/Stable-diffusion/*.ckpt ] && [ ! -f /stablediff-web/models/Stable-diffusion/*.safetensors ]; then

[FilkerZero]

I'm trying to get this working on Windows11 Pro with WSL2 and everything seemed to be going okay other than during the Nvidia CUDA toolkit and drivers installation where the sudo systemctl restart ... commands - under WSL2, systemd is not being used.

However I cannot bring the container up:

daniel@Dodecagon:~/docker/stablediff$ docker-compose up stablediff-cuda
[+] Running 1/2
 ✔ Network stablediff_default        Created                                                                      12.5s
 ⠋ Container stablediff-runner-cuda  Creating                                                                      0.1s
Error response from daemon: unknown or invalid runtime name: nvidia
daniel@Dodecagon:~/docker/stablediff$ sudo docker-compose up stablediff-cuda
[sudo] password for daniel:
[+] Running 0/1
 ⠋ Container stablediff-runner-cuda  Creating                                                                      0.0s
Error response from daemon: unknown or invalid runtime name: nvidia
daniel@Dodecagon:~/docker/stablediff$

Maybe I have to reboot - I'm not sure how to get the kernel under WSL2 to restart without rebooting the entire PC and nothing docker, Nvidia, or CUDA related shows up when I type the command sudo service --status-all.

I'll try to post an update if I solve this.

[FilkerZero]

Updated progress: I'm a little further along now, having removed the package docker.io and instead installed docker-ce, see https://docs.docker.com/engine/install/ubuntu/ for information on that. At least one of the problems with the Ubuntu docker-io package is that it doesn't provide any support for /etc/init.d type control, just systemd, which doesn't run under WSL2. With that installed, you then use sudo service docker restart to restart the docker daemon, however several of the other things https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html tells you to restart don't seem to have control files in /etc/init.d/.

I'm now getting:

daniel@Dodecagon:~/docker/stablediff$ docker-compose build stablediff-cuda
[+] Building 448.5s (14/14) FINISHED                                                                     docker:default
 => [stablediff-cuda internal] load .dockerignore                                                                  0.0s
 => => transferring context: 82B                                                                                   0.0s
 => [stablediff-cuda internal] load build definition from Dockerfile.cuda                                          0.0s
 => => transferring dockerfile: 853B                                                                               0.0s
 => [stablediff-cuda internal] load metadata for docker.io/nvidia/cuda:11.3.1-base-ubuntu20.04                     0.8s
 => [stablediff-cuda auth] nvidia/cuda:pull token for registry-1.docker.io                                         0.0s
 => [stablediff-cuda 1/9] FROM docker.io/nvidia/cuda:11.3.1-base-ubuntu20.04@sha256:56c32c52df3572bc7eaeb81b2540b  0.0s
 => CACHED [stablediff-cuda 2/9] WORKDIR /sdtemp                                                                   0.0s
 => CACHED [stablediff-cuda 3/9] RUN apt-get update &&    apt-get install -y     wget     git     python3     pyt  0.0s
 => CACHED [stablediff-cuda 4/9] RUN python -m pip install --upgrade pip wheel                                     0.0s
 => CACHED [stablediff-cuda 5/9] RUN git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui /sdtemp     0.0s
 => [stablediff-cuda 6/9] RUN python -m pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index  209.3s
 => [stablediff-cuda 7/9] RUN python launch.py --skip-torch-cuda-test --exit                                     206.1s
 => [stablediff-cuda 8/9] RUN python -m pip install opencv-python-headless                                         8.1s
 => [stablediff-cuda 9/9] WORKDIR /stablediff-web                                                                  0.4s
 => [stablediff-cuda] exporting to image                                                                          23.4s
 => => exporting layers                                                                                           23.3s
 => => writing image sha256:71912df05c6a536ddb77e70b9ad98aa70b6408dde34914b09c4d67d6d884fc17                       0.0s
 => => naming to docker.io/library/stablediff-stablediff-cuda                                                      0.0s
daniel@Dodecagon:~/docker/stablediff$ docker-compose up stablediff-cuda
[+] Running 2/1
 ✔ Network stablediff_default        Created                                                                       0.1s
 ✔ Container stablediff-runner-cuda  Created                                                                       0.0s
Attaching to stablediff-runner-cuda
stablediff-runner-cuda  | Wed Dec 27 22:16:59 2023
stablediff-runner-cuda  | +---------------------------------------------------------------------------------------+
stablediff-runner-cuda  | | NVIDIA-SMI 545.36                 Driver Version: 546.33       CUDA Version: 12.3     |
stablediff-runner-cuda  | |-----------------------------------------+----------------------+----------------------+
stablediff-runner-cuda  | | GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
stablediff-runner-cuda  | | Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
stablediff-runner-cuda  | |                                         |                      |               MIG M. |
stablediff-runner-cuda  | |=========================================+======================+======================|
stablediff-runner-cuda  | |   0  NVIDIA GeForce RTX 3060        On  | 00000000:01:00.0  On |                  N/A |
stablediff-runner-cuda  | |  0%   41C    P8              20W / 170W |    784MiB / 12288MiB |      6%      Default |
stablediff-runner-cuda  | |                                         |                      |                  N/A |
stablediff-runner-cuda  | +-----------------------------------------+----------------------+----------------------+
stablediff-runner-cuda  |
stablediff-runner-cuda  | +---------------------------------------------------------------------------------------+
stablediff-runner-cuda  | | Processes:                                                                            |
stablediff-runner-cuda  | |  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
stablediff-runner-cuda  | |        ID   ID                                                             Usage      |
stablediff-runner-cuda  | |=======================================================================================|
stablediff-runner-cuda  | |  No running processes found                                                           |
stablediff-runner-cuda  | +---------------------------------------------------------------------------------------+
stablediff-runner-cuda  | launch.py --listen
stablediff-runner-cuda  | Please copy stable diffusion model to stablediff-models directory
stablediff-runner-cuda  | You may need sudo to perform this action
stablediff-runner-cuda exited with code 1

After I copyingsd-v1-5.safetensors into stablediff-models, this is what I get when I try to start the container:

daniel@Dodecagon:~/docker/stablediff$ docker start -a stablediff-cuda-runner
Error response from daemon: No such container: stablediff-cuda-runner
daniel@Dodecagon:~/docker/stablediff$

So I'm not quite there yet.

[FilkerZero]

Further update: The command I have to use is docker start -a stablediff-runner-cuda for some reason; with a model in stablediff-models, it still didn't see it:

daniel@Dodecagon:~/docker/stablediff$ docker start -a stablediff-runner-cuda
Wed Dec 27 22:37:39 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.36                 Driver Version: 546.33       CUDA Version: 12.3     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3060        On  | 00000000:01:00.0  On |                  N/A |
|  0%   43C    P8              21W / 170W |    799MiB / 12288MiB |      7%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+
launch.py --listen
Please copy stable diffusion model to stablediff-models directory
You may need sudo to perform this action
daniel@Dodecagon:~/docker/stablediff$

I copied the model file into stablediff-web/models/Stable-diffusion/ and have issued the docker start -a stablediff-runner-cuda again; I'm waiting for anything on the console; I don't know whether it's hanging or just taking its own sweet time getting going.

I will update as things happen (or don't).

Update: Same result as before; it doesn't see the Stable Diffusion model in either place. Do I need to put it under stablediff-models/Stable-diffusion?

[a-vassily]

nothing is hanging. the application is running on http://127.0.0.1:7860/. otherwise you need to put the ckpt file in the models directory. please dont confuse the models with the ckpt file. https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main and
https://huggingface.co/runwayml/stable-diffusion-v1-5/blob/main/v1-5-pruned-emaonly.safetensors

[miiwo]

Hi!

This discussion has helped me with figuring out how to get this locally on my machine and I wanted to pass it forward! I'm not too sure if you're still running into this problem, however changing the line in the docker-compose file:
if [ ! -f /stablediff-web/models/Stable-diffusion/*.ckpt ]; then
to
if [ ! -f /stablediff-web/models/Stable-diffusion/*.ckpt ] && [ ! -f /stablediff-web/models/Stable-diffusion/*.safetensors ]; then
should fix it. Since the original script was meant as a check for ckpt files, this should allow for safetensor as well.

Hope this helped!

[a-vassily]

excellent article. Thank you !!! This is working really fine. One question only: why does it say that the extensions are disabled . This happens if we fo to the extensions tab and try to install from URL.
*** Arguments: ('', 'https://github.com/Mikubill/sd-webui-controlnet', '') {}
Traceback (most recent call last):
File "/stablediff-web/modules/call_queue.py", line 57, in f
res = list(func(*args, **kwargs))
File "/stablediff-web/modules/ui_extensions.py", line 641, in
fn=modules.ui.wrap_gradio_call(lambda *args: [gr.update(), *install_extension_from_url(*args)], extra_outputs=[gr.update(), gr.update()]),
File "/stablediff-web/modules/ui_extensions.py", line 344, in install_extension_from_url
check_access()
File "/stablediff-web/modules/ui_extensions.py", line 23, in check_access
assert not shared.cmd_opts.disable_extension_access, "extension access disabled because of command line flags"
AssertionError: extension access disabled because of command line flags

[a-vassily]

found it. it needs the command line argument --enable unsaffe extensions

[hqnicolas]

hello! @catzy007
here is your solution!
https://gist.github.com/hqnicolas/6d9bfa997667626742c0918b95e67171

PS:
Stable diffusion Version: v1.7.0
Version: v1.8.0 will broke the python 3.9

[souravsk]

docker-compose up stablediff-cpu
Recreating 8951ca12cd43_stablediff-cpu-runner ...

ERROR: for 8951ca12cd43_stablediff-cpu-runner 'ContainerConfig'

ERROR: for stablediff-cpu 'ContainerConfig'
Traceback (most recent call last):
File "/usr/bin/docker-compose", line 33, in
sys.exit(load_entry_point('docker-compose==1.29.2', 'console_scripts', 'docker-compose')())
File "/usr/lib/python3/dist-packages/compose/cli/main.py", line 81, in main
command_func()
File "/usr/lib/python3/dist-packages/compose/cli/main.py", line 203, in perform_command
handler(command, command_options)
File "/usr/lib/python3/dist-packages/compose/metrics/decorator.py", line 18, in wrapper
result = fn(*args, **kwargs)
File "/usr/lib/python3/dist-packages/compose/cli/main.py", line 1186, in up
to_attach = up(False)
File "/usr/lib/python3/dist-packages/compose/cli/main.py", line 1166, in up
return self.project.up(
File "/usr/lib/python3/dist-packages/compose/project.py", line 697, in up
results, errors = parallel.parallel_execute(
File "/usr/lib/python3/dist-packages/compose/parallel.py", line 108, in parallel_execute
raise error_to_reraise
File "/usr/lib/python3/dist-packages/compose/parallel.py", line 206, in producer
result = func(obj)
File "/usr/lib/python3/dist-packages/compose/project.py", line 679, in do
return service.execute_convergence_plan(
File "/usr/lib/python3/dist-packages/compose/service.py", line 579, in execute_convergence_plan
return self._execute_convergence_recreate(
File "/usr/lib/python3/dist-packages/compose/service.py", line 499, in _execute_convergence_recreate
containers, errors = parallel_execute(
File "/usr/lib/python3/dist-packages/compose/parallel.py", line 108, in parallel_execute
raise error_to_reraise
File "/usr/lib/python3/dist-packages/compose/parallel.py", line 206, in producer
result = func(obj)
File "/usr/lib/python3/dist-packages/compose/service.py", line 494, in recreate
return self.recreate_container(
File "/usr/lib/python3/dist-packages/compose/service.py", line 612, in recreate_container
new_container = self.create_container(
File "/usr/lib/python3/dist-packages/compose/service.py", line 330, in create_container
container_options = self._get_container_create_options(
File "/usr/lib/python3/dist-packages/compose/service.py", line 921, in _get_container_create_options
container_options, override_options = self._build_container_volume_options(
File "/usr/lib/python3/dist-packages/compose/service.py", line 960, in _build_container_volume_options
binds, affinity = merge_volume_bindings(
File "/usr/lib/python3/dist-packages/compose/service.py", line 1548, in merge_volume_bindings
old_volumes, old_mounts = get_container_data_volumes(
File "/usr/lib/python3/dist-packages/compose/service.py", line 1579, in get_container_data_volumes
container.image_config['ContainerConfig'].get('Volumes') or {}
KeyError: 'ContainerConfig'

I have changed the .ckpt to .safetensors after do that run docker compose up command and I'm getting this error

[ruslan-sm]

@souravsk Try to execute docker compose up stablediff-cpu

[souravsk]

thanks worked

[souravsk]

I have question
I'm creating my UI with the A1111 API and then after that, I want to push that to production for public use.
The problem is that right now it can only process one image at a time. if user B requests an image then it gets into queues
I want Parallel processing. so that every user can generate an image

[souravsk]

I got a new error it is killing itself after generating image but not even showing on web

docker-compose up stablediff-cpu
Starting stablediff-cpu-runner ... done
Attaching to stablediff-cpu-runner
stablediff-cpu-runner | launch.py --api --listen --no-half --skip-torch-cuda-test
stablediff-cpu-runner | Python 3.10.6 (main, Aug 23 2022, 08:25:41) [GCC 10.2.1 20210110]
stablediff-cpu-runner | Version: v1.8.0
stablediff-cpu-runner | Commit hash: bef51aed032c0aaa5cfd80445bc4cf0d85b408b5
stablediff-cpu-runner | Launching Web UI with arguments: --api --listen --no-half --skip-torch-cuda-test
stablediff-cpu-runner | no module 'xformers'. Processing without...
stablediff-cpu-runner | No SDP backend available, likely because you are running in pytorch versions < 2.0. In fact, you are using PyTorch 1.12.1+cpu. You might want to consider upgrading.
stablediff-cpu-runner | no module 'xformers'. Processing without...
stablediff-cpu-runner | No module 'xformers'. Proceeding without it.
stablediff-cpu-runner | Warning: caught exception 'Torch not compiled with CUDA enabled', memory monitor disabled
stablediff-cpu-runner | ==============================================================================
stablediff-cpu-runner | You are running torch 1.12.1+cpu.
stablediff-cpu-runner | The program is tested to work with torch 2.1.2.
stablediff-cpu-runner | To reinstall the desired version, run with commandline flag --reinstall-torch.
stablediff-cpu-runner | Beware that this will cause a lot of large files to be downloaded, as well as
stablediff-cpu-runner | there are reports of issues with training tab on the latest version.
stablediff-cpu-runner |
stablediff-cpu-runner | Use --skip-version-check commandline argument to disable this check.
stablediff-cpu-runner | ==============================================================================
stablediff-cpu-runner | Loading weights [6ce0161689] from /stablediff-web/models/Stable-diffusion/v1-5-pruned-emaonly.safetensors
stablediff-cpu-runner | Creating model from config: /stablediff-web/configs/v1-inference.yaml
stablediff-cpu-runner | Running on local URL:  http://0.0.0.0:7860
stablediff-cpu-runner |
stablediff-cpu-runner | To create a public link, set `share=True` in `launch()`.
stablediff-cpu-runner | Startup time: 6.2s (import torch: 2.4s, import gradio: 0.8s, setup paths: 0.9s, other imports: 0.5s, load scripts: 0.4s, create ui: 0.5s, gradio launch: 0.2s, add APIs: 0.3s).
stablediff-cpu-runner | Applying attention optimization: InvokeAI... done.
stablediff-cpu-runner | Model loaded in 4.8s (load weights from disk: 0.7s, create model: 0.6s, apply weights to model: 3.2s, calculate empty prompt: 0.2s).
100%|██████████| 1/1 [00:12<00:00, 13.00s/it]
stablediff-cpu-runner | Killed | 0/1 [00:00<?, ?it/s]
stablediff-cpu-runner exited with code 137

[Adam-J-Roberts]

This is pretty amazing! I used it to create a github repo that only runs cpu, so I can just pull it in portainer and it takes care of itself.

Huge thanks!

[stiw47]

Hi.

First of all BIG THANK for the GREAT GUIDE!!!

Related with GPU use: From docker-compose build stablediff-cuda

[root@archvideo docker-compose-automatic111]# docker-compose build stablediff-cuda
.......
18.12 stderr: ERROR: Ignored the following versions that require a different python version: 1.25.0 Requires-Python >=3.9; 1.25.0rc1 Requires-Python >=3.9; 1.25.1 Requires-Python >=3.9; 1.25.2 Requires-Python >=3.9; 1.26.0 Requires-Python <3.13,>=3.9; 1.26.0b1 Requires-Python <3.13,>=3.9; 1.26.0rc1 Requires-Python <3.13,>=3.9; 1.26.1 Requires-Python <3.13,>=3.9; 1.26.2 Requires-Python >=3.9; 1.26.3 Requires-Python >=3.9; 1.26.4 Requires-Python >=3.9; 2.0.0 Requires-Python >=3.9; 2.0.0b1 Requires-Python >=3.9; 2.0.0rc1 Requires-Python >=3.9; 2.0.0rc2 Requires-Python >=3.9; 2.0.1 Requires-Python >=3.9; 2024.0.1 Requires-Python >=3.9,<4.0; 2024.1 Requires-Python >=3.9,<4.0; 2024.1.1 Requires-Python >=3.9,<4.0
18.12 ERROR: Could not find a version that satisfies the requirement numpy==1.26.2 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 1.13.3, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0, 1.21.1, 1.21.2, 1.21.3, 1.21.4, 1.21.5, 1.21.6, 1.22.0, 1.22.1, 1.22.2, 1.22.3, 1.22.4, 1.23.0rc1, 1.23.0rc2, 1.23.0rc3, 1.23.0, 1.23.1, 1.23.2, 1.23.3, 1.23.4, 1.23.5, 1.24.0rc1, 1.24.0rc2, 1.24.0, 1.24.1, 1.24.2, 1.24.3, 1.24.4)
18.12 ERROR: No matching distribution found for numpy==1.26.2
18.12 
------
failed to solve: process "/bin/sh -c python launch.py --skip-torch-cuda-test --exit" did not complete successfully: exit code: 1

I don't know if something was changed meanwhile in base image nvidia/cuda:11.3.1-base-ubuntu20.04, but I would say it apt-get'ing python 3.8:

 > [stablediff-cuda 7/9] RUN python launch.py --skip-torch-cuda-test --exit:                                                                                                                                                             
0.220 Python 3.8.10 (default, Jul 29 2024, 17:02:10)

And it seems that numpy 1.26.2 requires python 3.9: https://pypi.org/project/numpy/1.26.2/

Anyway, I am pretty sure this could be done in some better and more elegant way, but this was fastest for me and it is working - Dockerfile.cuda:

FROM nvidia/cuda:11.3.1-base-ubuntu20.04
ENV DEBIAN_FRONTEND=noninteractive \
    PYTHONUNBUFFERED=1 \
    PYTHONIOENCODING=UTF-8
WORKDIR /sdtemp
RUN apt-get update && \
    apt-get install -y software-properties-common && \
    add-apt-repository ppa:deadsnakes/ppa && \
    apt-get update &&\
    apt-get install -y \
    wget \
    git \
    python3.9 \
    python3-pip \
    python-is-python3
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 1
RUN update-alternatives --config python3
RUN python -m pip install --upgrade pip wheel
RUN git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui /sdtemp

#torch and torchvision version number refer to
#https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/launch.py
ENV TORCH_COMMAND="pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113"
RUN python -m $TORCH_COMMAND

RUN python launch.py --skip-torch-cuda-test --exit
RUN python -m pip install opencv-python-headless
WORKDIR /stablediff-web

Thx

[stiw47]

Ok, I would like to share once again my improvements. It shouldn't mean this fits to everyone, but this is what I wanted for my use case.
I have home server with i7-4770, 4 cores 8 threads, from Q2'13. Beside that, 16GB RAM, and poor GPU - NVIDIA 1650. It is not even Ti or SUPER, just a 1650. Running Archlinux on hardware, and this is the place where I'm running Docker containers for my home stuff. I am total n00b for AI image generation (not for images in general), but from time to time I like to spent spare time in this field.
It seems this^^ was TL;DR 😁

Ok, wanted to run Automatic1111 Stable Diffusion in Docker, also with additional extensions sd-webui-controlnet and sd-webui-reactor. But struggled to make it work with GPU, because:

1. Either I would get:

ImportError: libGL.so.1: cannot open shared object file: No such file or directory

2. Or insightface missing would be a problem when installing ControlNet and Reactor extensions. Installing it manually from container with pip install insightface would break some other things if I remember correct, and so on.
3. There were some more errors, I forgot, and in general as said, struggled to make it wok with GPU.

So decided to inspect Dockerfile.cuda from start, stage by stage manually, by installing things in container, and updating Dockerfile.cuda accordingly. This is what I got at the end, and it builds without any errors, sd-webui-controlnet and sd-webui-reactor are installing also without any errors (insightface pre-installed), also xformers installed and in use (wasn't case with original Dockerfile.cuda, at least on my end):

What I noticed so far, are just few warnings on the first run, about few deprecation from torch.

Dockerfile.cuda:

FROM nvidia/cuda:12.6.1-base-ubuntu20.04
ARG DEBIAN_FRONTEND=noninteractive
ENV DEBIAN_FRONTEND=noninteractive \
    PYTHONUNBUFFERED=1 \
    PYTHONIOENCODING=UTF-8
WORKDIR /sdtemp
RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y \
    wget \
    build-essential \
    zlib1g-dev \
    libncurses5-dev \
    libgdbm-dev \
    libnss3-dev \
    libssl-dev \
    libreadline-dev \
    libffi-dev curl \
    libsqlite3-dev \
    libbz2-dev \
    liblzma-dev \
    libgl1-mesa-glx \
    libglib2.0-0 \
    git && \
    wget https://www.python.org/ftp/python/3.10.6/Python-3.10.6.tgz && \
    tar -xvzf Python-3.10.6.tgz && \
    cd Python-3.10.6 && \
    ./configure && \
    make -j $(nproc) && \
    make install && \
    cd .. && \
    rm -rf Python-3.10.6*
RUN git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui /sdtemp
RUN python3 -m pip install --upgrade pip wheel
RUN python3 -m pip install insightface
ENV TORCH_COMMAND="pip install torch==2.4.0 torchvision==0.19.0 xformers --extra-index-url https://download.pytorch.org/whl/cu121"
RUN python3 -m $TORCH_COMMAND
RUN python3 launch.py --skip-torch-cuda-test --exit
WORKDIR /stablediff-web

So we can see^^, I decided to:

1. Give a try to CUDA 12
2. Since did not found some elegant way (repos/PPA) to install Python 3.10.6 exactly in Ubuntu 20.04, 22.04, 22.10, 24.04.... and startup message in container log says:

INCOMPATIBLE PYTHON VERSION

This program is tested with 3.10.6 Python, but you have 3.12.3.
If you encounter an error with "RuntimeError: Couldn't install torch." message,
or any other error regarding unsuccessful package (library) installation,
please downgrade (or upgrade) to the latest version of 3.10 Python
and delete current Python and "venv" folder in WebUI's directory.

You can download 3.10 Python from here: https://www.python.org/downloads/release/python-3106/

I decided to compile Python 3.10.6 manually
3. Decided to give it a try with torch 2.4.0 // torchvision 0.19.0
4. Pre-installed insightface to have it for later ControlNet/Reactor, and for my usecase
5. Maybe something else

docker-compose.yaml:

services:
  stablediff-cuda:
    build: 
      context: .
      dockerfile: Dockerfile.cuda
    container_name: stablediff-cuda-runner
    restart: unless-stopped
    runtime: nvidia
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    environment:
      TZ: "Europe/Belgrade"
      NVIDIA_VISIBLE_DEVICES: all
      COMMANDLINE_ARGS: "--listen"
    entrypoint: ["/bin/sh", "-c"]
    command: >
      "nvidia-smi; . /stablediff.env; echo launch.py $$COMMANDLINE_ARGS;
      if [ ! -d /stablediff-web/.git ]; then
        cp -a /sdtemp/. /stablediff-web/
      fi;
      if [ ! -f /stablediff-web/models/Stable-diffusion/*.ckpt ]; then
        echo 'Please copy stable diffusion model to stablediff-models directory'
        echo 'You may need sudo to perform this action'
        exit 1
      fi;
      python3 launch.py"
    ports:
      - "7860:7860"
    volumes:
      - ./stablediff.env:/stablediff.env
      - ./stablediff-web:/stablediff-web
      - ./stablediff-models:/stablediff-web/models/Stable-diffusion
      - ./controlnet-models:/stablediff-web/models/ControlNet
      - ./lora-models:/stablediff-web/models/Lora
      - ./outputs:/stablediff-web/outputs

^^Not changed to much. Just created mounts on host system for some additional folders, because I like it more this way, and TBH I am not sure at all is the part with deploy: necessary at all.

stablediff.env:

export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
export COMMANDLINE_ARGS="--listen --api --no-half --lowvram --xformers --enable-insecure-extension-access"

^^I'm running app with above command line (poor GPU), and still experimenting with PYTORCH_CUDA_ALLOC_CONF variable, because the only bad thing so far - sometimes I have no enough VRAM when using ControlNet (again, poor GPU). E.g.

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 26.00 MiB. GPU 0 has a total capacity of 3.63 GiB of which 34.75 MiB is free. Process 76491 has 3.59 GiB memory in use. Of the allocated memory 2.77 GiB is allocated by PyTorch, and 9.28 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

.dockerignore:

controlnet-models/
lora-models/
outputs/
stablediff-models/
stablediff-web/
*.ckpt
*.safetensors

Once again - I don't want to say this is the best approach. I am n00b for AI image, and I don't know that. Just want to share something what perfectly working for me at the moment, maybe will be useful to someone.

@catzy007 : Thank you once again for great Docker concept/configuration - this is the best I found on internet for Automatic1111 Stable Diffusion.