Stable Diffusion GUI

Created in March 01, 2022

2022   ·   HPC   ·   HPC

Screen

Introduction

  • Stable Diffusion is a text-to-image and image-to-image generative model: you type a prompt (for example, “a watercolor city skyline at sunset”), and the model synthesizes an image that matches that description.
  • It belongs to the family of diffusion models: the system starts from random noise and gradually “denoises” it into a coherent picture, guided by your text prompt (and optional settings such as steps, CFG scale, and seed).
  • Stable Diffusion is popular because it can run locally (or on a shared cluster), supports many community models and extensions, and does not require sending every prompt to a closed cloud API.

Why do we need a GPU?

  • Image generation is heavy matrix math. On a CPU alone, even a single image can take a very long time; on a modern GPU (for example an A100 or similar), the same job is often seconds to a minute.
  • That is why most serious Stable Diffusion work happens on:
    • a personal PC with a strong GPU, or
    • an HPC compute node with allocated GPUs
  • On HPC, you should not run this on the login node. Request a compute / GPU node, then open the web UI through a tunnel — same idea as Port Forwarding and Jump Host.
  • Two widely used web interfaces for Stable Diffusion are:
    • Automatic1111 (A1111 / Stable Diffusion WebUI) — a form-style UI: prompt box, samplers, models, and many extensions. Friendly for beginners who want click-and-generate.
    • ComfyUI — a node graph UI: you wire steps (load model → encode prompt → sample → save image). More flexible for complex workflows, and popular for advanced or reproducible pipelines.
    • Download and run both A1111 and ComfyUI are quite straightforward on PC, however it takes some setup to work on share resources like HPC system.
  • Both serve a browser UI on a port on the GPU node. Your laptop reaches that UI with port forwarding (SOCKS or local -L), not by opening the compute node directly from the public internet.

What this post covers

  • How to install and run Automatic1111 and ComfyUI on an HPC GPU node
  • How to open each web UI from your laptop browser using port forwarding
  • Practical notes so you can generate images on shared cluster GPUs without installing everything only on your personal machine

Install A1111

Step 1: Create conda env

$ conda create -n sd python=3.10 pip
$ conda activate sd

Step 2: Download A1111 repo to your local folder

$ cd /your_local_folder_location
$ git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git

Step 3: Download one sample checkpoint (pre-trained model) for your usage.

I suggest to download one from HuggingFace, runwayml

  • Navigate to https://huggingface.co/runwayml/stable-diffusion-v1-5
  • Select Files and versions
  • Right click and copy link address of the ckpt file
$ cd stable-diffusion-webui/models/Stable-diffusion
$ wget https://huggingface.co/runwayml/stable-diffusion-v1-5/blob/main/v1-5-pruned-emaonly.ckpt

You can download as many checkpoint in the format of ckpt or safetensors to the models folder for your usage.

Step 4: Run A1111 from local PC

To run A1111 from local PC is very simple:

$ ./webui.sh
# or
sh webui.sh

Go to your local browser like Firefox and browse to the address 127.0.0.1:7860

Screen

=> We will talk about running A1111 on HPC in the next few steps

Install ComfyUI

Step 1: Create conda env (You can utilize the existing conda env sd)

$ conda create -n sd python=3.10 pip
$ conda activate sd

Step 2: Download the model (either by gitclone or download zip file from their repo):

$ cd /your_local_folder_location
$ git clone https://github.com/comfyanonymous/ComfyUI.git
$ cd CompfyUI
$ pip install -r requirements.txt
$ cd ComfyUI/custom_nodes
$ git clone https://github.com/ltdrdata/ComfyUI-Manager.git

Step 3: Download one sample checkpoint (pre-trained model) for your usage.

In addition to HuggingFace, you can also download checkpoint in safetensors format from Civitai:

  • Go for Model and select any sample that you want then download the model.

  • For example this model

  • The safetensors should be moved to ComfyUI/models/checkpoints

Step 4: Run ComfyUI from local PC

To run ComfyUI from local PC is very simple:

$ python main.py

Go to your local browser like Firefox and browse to the address 127.0.0.1:8188

Screen

Port-Forwarding to run A1111 and ComfyUI on HPC

Step 1: ssh to login node with dynamic port:

  • Here I use port number 8000 to login to our HPC system
$ ssh -C -D 8000 mp

Step 2: Request a GPU node and Load module and activate conda env

$ srun -N1 -c10 -G1 --mem=64G --pty $SHELL
$ module load conda gcc/11 cuda
$ conda activate sd

Step 3: Do either on of the command to load A1111 or ComfyUI:

Step 3a: For A1111:

$ cd stable-diffusion-webui
$ sh webui.sh

Step 3b: For ComfyUI:

$ cd CompfyUI
$ python main .py

Step 4: Open new terminal and run port-forwarding

  • Assuming I am given a GPU node called bcm-dgxa100-0001:

  • For A1111: make sure the port 7860 is used 
    $ ssh -J mp tuev@bcm-dgxa100-0001 -L 8000:localhost:8000 -L 7860:localhost:7860
  • For ComfyUI: make sure the port 8188 is used 
    $ ssh -J mp tuev@bcm-dgxa100-0001 -L 8000:localhost:8000 -L 8188:localhost:8188

Step 5: Setup configuration in your Firefox with local host and port 8000

  • Follow Port Forwarding

  • Paste into the browser the correct address for each model:

  • A1111: 127.0.0.1:7860
  • ComfyUI: 127.0.0.1:8188

For more information please watch my recording: