palette

Full course · Free access

Stable Diffusion & ComfyUI Pro

schedule15+ hours rocket_launch8 projects downloadWorkflows to download updateLast updated: April 2026

Module one

Work environment

1.1

Installation on Windows (NVIDIA GPU)

To run Stable Diffusion and ComfyUI locally on Windows, you will need the following components:

Hardware requirements

✓An NVIDIA GPU with at least 8GB VRAM (RTX 3060 and up recommended)
✓20GB+ of free disk space (SSD recommended)
✓At least 16GB RAM

Software requirements

✓Python 3.10.x only — not 3.11 and not 3.12
✓Git for Windows
✓NVIDIA CUDA Toolkit 12.1+

warning Very important

Use Python 3.10.x only. Newer versions break compatibility with several core PyTorch and ComfyUI libraries.

Step 1 — installing Python 3.10

Download frompython.org/downloads/release/python-31011/ and check "Add Python to PATH" during installation — this is critical.

Step 2 — installing Git

Download fromgit-scm.com/download/win and leave all settings on Default.

Step 3 — cloning ComfyUI and installing Dependencies

Command Prompt (as Administrator)

# clone ComfyUI
git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI

# install PyTorch with CUDA 12.1
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

# install additional dependencies
pip install -r requirements.txt

Step 4 — downloading the SDXL model

bash / powershell

# download to models/checkpoints/
# from HuggingFace:
# https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0
# file: sd_xl_base_1.0.safetensors (~6.5GB)

Step 5 — first run

Command Prompt

python main.py --listen 0.0.0.0
# open a browser: http://127.0.0.1:8188

Tip: Create a desktop shortcut to the bat file to launch ComfyUI in one click.

1.2

Installation on Mac (Apple Silicon)

macOS with M1/M2/M3 supports Metal Performance Shaders (MPS) — Apple's internal GPU accelerator. Performance is lower than an NVIDIA RTX, but still very usable on an M2 Pro and up.

Terminal (macOS)

# install Homebrew if you do not have it yet
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# install Python and Git
brew install python@3.10 git

# clone ComfyUI
git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI

# install PyTorch for Mac (MPS)
pip3.10 install torch torchvision torchaudio

# install dependencies
pip3.10 install -r requirements.txt

# run with MPS and FP16
python3.10 main.py --force-fp16

M1 / M1 Pro

8–16GB — basic, slow generation

M2 Pro / M3

16GB+ — good for regular use

M3 Max / M4 Pro

36GB+ — excellent performance

A note about ControlNet on Mac: Some ControlNet preprocessors are not supported on MPS. In that case, add --cpu-vae to the launch, or use the CPU fallback in ComfyUI Manager.

1.3

Google Colab Setup (without a local GPU)

Google Colab gives you an NVIDIA T4 for free — enough for SDXL generation at reasonable quality. Colab Pro (about $10/month) gives access to A100 and V100 for professional performance.

Google Colab — cell 1: installation

!git clone https://github.com/comfyanonymous/ComfyUI
%cd ComfyUI
!pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 -q
!pip install -r requirements.txt -q

Cell 2: downloading the SDXL Turbo model

!wget -q -O models/checkpoints/sdxl-turbo.safetensors \
  "https://huggingface.co/stabilityai/sdxl-turbo/resolve/main/sd_xl_turbo_1.0_fp16.safetensors"

Cell 3: launching with a Cloudflare Tunnel

!pip install cloudflared -q

import subprocess, threading, time

def run_comfyui():
    subprocess.run(['python', 'main.py', '--listen', '0.0.0.0', '--port', '8188'])

thread = threading.Thread(target=run_comfyui)
thread.daemon = True
thread.start()
time.sleep(5)

# create a tunnel — the link will appear in the output
!cloudflared tunnel --url http://localhost:8188

Tip: Colab Pro is recommended for continuous use. The free T4 disconnects after 90 minutes of inactivity. Save the models in Google Drive so you do not have to download them again every session.

1.4

ComfyUI Basics: structure and Nodes

ComfyUI runs on a Graph-based architecture — every operation is a Node and Wires connect them to carry the data flow. This is completely different from Automatic1111, but many times more flexible.

Basic Nodes

Node	Role
Load Checkpoint	Loads a model (SDXL, Flux, Realistic Vision)
CLIP Text Encode	Converts a Prompt into a semantic vector
KSampler	Performs the Denoising process
VAE Decode	Converts a Latent into a visible image
Empty Latent Image	Sets the Canvas size
Save Image	Saves the output to disk

A basic Workflow — Text to Image

Flow diagram

[Load Checkpoint] ──model──► [KSampler]
                  ──clip──►  [CLIP Text Encode (+)] ──conditioning──► [KSampler]
                  ──clip──►  [CLIP Text Encode (-)] ──conditioning──► [KSampler]
                  ──vae──►   (stored for decode)

[Empty Latent Image] ──latent──► [KSampler]
[KSampler] ──latent──► [VAE Decode] ──image──► [Save Image]

KSampler settings — a quick guide

Parameter	Recommended value	Explanation
steps	20–30	for quality. 4–8 for Turbo models
cfg	7.0	Guidance. 1.0–2.0 for Flux
sampler	dpm_2_ancestral	for quality. euler for LCM/Turbo
scheduler	karras	for most cases. sgm_uniform for SDXL Turbo
seed	-1 (random)	Set a seed for reproducibility

Tip: Always start from the Default Workflow built into ComfyUI (the first Queue Prompt after installation). Understand this structure before you move on to complex Workflows.

🎯

Module 1 project

A full environment — the first 20 images

Goal: set up your work environment and create 20 varied images to prove everything works.

5 images

Portrait

5 images

Landscape

5 images

Product Photo

5 images

Abstract

Save all the Workflows as JSON files inComfyUI/user/default/workflows/

Module two

SDXL and Flux Mastery

2.1

Differences between the models

Model	Speed	Quality	VRAM	Suits
SDXL Base	Medium	High	8GB+	General use
SDXL Turbo	Very fast	Medium	8GB	Drafts, real-time
Flux Schnell	Fast	High	16GB+	Creative work
Flux Dev	Slow	Excellent	24GB+	Production
Realistic Vision	Medium	Realistic	6GB	Portraits

2.2

Prompt Anatomy for SDXL

An optimal Prompt structure for SDXL that yields consistent results:

Prompt structure

[Subject] [Action/Pose] [Environment] [Lighting] [Style] [Quality Tags]

# a practical example:
"a young woman sitting in a cafe, reading a book,
warm afternoon sunlight through window,
cinematic photography, shallow depth of field,
high detail, 8k, award winning photograph"

A universal Negative Prompt

"ugly, blurry, low quality, distorted, watermark, text,
extra limbs, bad anatomy, bad hands, deformed,
lowres, worst quality, normal quality, jpeg artifacts"

2.3

CFG, Steps and Sampler Guide

Use	Steps	CFG	Sampler	Scheduler
Quick draft	8	7.0	euler	normal
Standard quality	25	7.5	dpm_2_a	karras
High quality	40	8.0	dpm_3m_sde	karras
Flux Schnell	4	1.0	euler	simple
Flux Dev	25	3.5	euler	beta

2.4

Seed Management and ComfyUI API

Saving a Seed enables reproducibility — the exact same image on every run. Here is a function that combines this with the ComfyUI API:

Python — Seed Management

import json
import urllib.request
import urllib.parse

def queue_prompt(prompt_workflow: dict, seed: int = -1) -> dict:
    """send a workflow to ComfyUI with a fixed seed"""
    for node in prompt_workflow.values():
        if node.get('class_type') == 'KSampler':
            if seed == -1:
                import random
                node['inputs']['seed'] = random.randint(1, 2**32)
            else:
                node['inputs']['seed'] = seed

    data = json.dumps({"prompt": prompt_workflow}).encode('utf-8')
    req = urllib.request.Request("http://127.0.0.1:8188/prompt", data=data)
    return json.loads(urllib.request.urlopen(req).read())

# usage — always the same image:
with open('my_workflow.json') as f:
    workflow = json.load(f)

result = queue_prompt(workflow, seed=42)

2.5

Workflow: High-Quality Portrait

A full ComfyUI Workflow for a high-quality portrait with RealVisXL:

1Load Checkpoint: realvisXL_v5.safetensors
2CLIP Text Encode (+): "professional headshot, business portrait, woman 30s, sharp focus, studio lighting, 8k uhd"
3CLIP Text Encode (-): a universal negative prompt
4Empty Latent Image: 1024×1024
5KSampler: steps=35, cfg=7.5, sampler=dpm_2_ancestral, karras
6VAE Decode → Save Image

🎯

Module 2 project

20 images in a consistent style

Choose one subject (product / person / landscape) and create 20 variations in a completely uniform style. Focus on: lighting consistency, handling CLIP Skip, saving the Workflow + Seed for each image.

Save all 20 Seeds — this is your Dataset for the following lessons.

Module three

ControlNet — full control over composition

3.1

OpenPose — pose control

OpenPose lets you define a precise body pose by providing a reference image. ComfyUI extracts a Pose Skeleton from it and applies it to the generation.

bash — installation via ComfyUI-Manager

# open ComfyUI → Manager → Install Custom Nodes
# search: "ControlNet" → install: comfyui_controlnet_aux

# download ControlNet Models:
# https://huggingface.co/lllyasviel/sd_xl_controlnet
# sd_xl_controlnet_openpose.safetensors → models/controlnet/

A Workflow with OpenPose

ComfyUI Graph

[Load Image (pose ref)] → [DWPose Estimator] → [Apply ControlNet]
[Load Checkpoint] ──model──► [Apply ControlNet] ──model──► [KSampler]
[CLIP Text Encode] ──cond──► [Apply ControlNet] ──cond──►  [KSampler]

Parameter	Recommended value	Explanation
strength	0.7–0.85	How strongly ControlNet influences
start_percent	0.0	Start from the first step
end_percent	0.7	Stop at 70% — allows creativity in the details

3.2

Depth Map — three-dimensional composition

A Depth Map defines the space in the image — what is near and what is far. Especially useful for Interior Design, Architecture and Product placement in different environments.

ComfyUI Depth Workflow

[Load Image] → [Depth Anything V2] → [Apply ControlNet (depth, strength=0.6)]

Tip: strength=0.6 compared to OpenPose — a Depth Map "guides" more than it "forces". This gives the model freedom in the details while preserving the three-dimensional structure.

3.3

Canny Edge — composition stabilization

Canny Edge Detection preserves lines and basic structure. Perfect for converting a Sketch into a full image, preserving architectural structure, and a logo onto a person/object.

Python — Canny Preprocessing

import cv2
import numpy as np

def extract_canny(image_path: str, low: int = 100, high: int = 200) -> np.ndarray:
    img = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)
    img = cv2.GaussianBlur(img, (5, 5), 0)
    edges = cv2.Canny(img, low, high)
    return edges

Usage: The low/high threshold determines how many lines are extracted. Lower values = more lines (more control but less creative freedom). Try low=50, high=150 for simple Sketches.

3.4

IP-Adapter — Reference Style

IP-Adapter lets you sample a "visual style" from an existing image and apply it to a new generation — without defining a style in the prompt.

bash — downloading the model

# https://huggingface.co/h94/IP-Adapter
# ip-adapter_xl.bin → models/ipadapter/

1Load IP-Adapter Model
2Load Reference Image (the image you take the style from)
3Apply IP-Adapter (weight=0.6)
4Combine with ControlNet for a definitive result

A powerful combination: IP-Adapter (style) + Depth ControlNet (structure) + Prompt (content) = full control over every dimension of the image.

🎯

Module 3 project

Product Photography Pipeline

Goal: create 10 professional product images from a regular product photo.

1.Generate a Depth Map from the original product
2.Use ControlNet Depth to preserve the product shape
3.Change the background to studio / nature / lifestyle
4.IP-Adapter for style consistency across the images

Module four

LoRA Training — train your own model

4.1

Dataset Preparation

Character LoRA

15–30 images of a face / character

Style LoRA

50–100 images in a style

Object LoRA

20–40 images of the object

Rules for a good Dataset

✓ Varied images — angles, lighting, different backgrounds
✓ Resolution 512×512 to 1024×1024
✓ A Caption for each image (BLIP auto-captioning)

What to avoid

✕ More than 30% images that are too similar
✕ Watermarks, text, logos in the images
✕ Blurry or low-quality images

4.2

Kohya SS on Google Colab

Colab — cell 1: installation

!git clone https://github.com/kohya-ss/sd-scripts
%cd sd-scripts
!pip install -r requirements.txt -q
!pip install xformers -q

Colab — cell 2: generating Captions with BLIP

!pip install transformers -q
from transformers import BlipProcessor, BlipForConditionalGeneration
from PIL import Image
import glob, os

processor = BlipProcessor.from_pretrained("Salesforce/blip-image-captioning-base")
model = BlipForConditionalGeneration.from_pretrained("Salesforce/blip-image-captioning-base")

for img_path in glob.glob("training_data/**/*.jpg", recursive=True):
    image = Image.open(img_path).convert('RGB')
    inputs = processor(image, return_tensors="pt")
    out = model.generate(**inputs, max_new_tokens=50)
    caption = processor.decode(out[0], skip_special_tokens=True)

    # add a trigger word
    caption = f"[trigger_word], {caption}"

    txt_path = img_path.replace('.jpg', '.txt')
    with open(txt_path, 'w') as f:
        f.write(caption)
    print(f"{os.path.basename(img_path)}: {caption}")

4.3

LoRA Training Script

bash — training a LoRA for SDXL

accelerate launch train_network.py \
  --pretrained_model_name_or_path="stabilityai/stable-diffusion-xl-base-1.0" \
  --train_data_dir="./training_data" \
  --output_dir="./output_lora" \
  --output_name="my_lora_v1" \
  --network_module="networks.lora" \
  --network_dim=32 \
  --network_alpha=16 \
  --resolution="1024,1024" \
  --train_batch_size=1 \
  --learning_rate=1e-4 \
  --lr_scheduler="cosine_with_restarts" \
  --lr_warmup_steps=100 \
  --max_train_steps=1000 \
  --save_every_n_steps=200 \
  --mixed_precision="bf16" \
  --xformers

Parameter	Value	Explanation
network_dim	32	Balances flexibility/size. Raise to 64 for complex styles
max_train_steps	800–3000	Character=800–1500, Style=1500–3000
learning_rate	1e-4	Lower to 5e-5 if the results Overfit
save_every_n_steps	200	Check each checkpoint to pick the best one

4.4

Testing and Fine-tuning

ComfyUI — Load LoRA Node

# Load LoRA Node settings:
#   lora_name: my_lora_v1-000800.safetensors
#   strength_model: 0.8
#   strength_clip: 0.8

# a Prompt for testing:
# "[trigger_word], portrait photo, professional lighting"

Results too similar = Overfitting

Reduce max_train_steps by 20%, increase dataset variety

The LoRA does not take = Underfitting

Increase steps by 30% or increase learning_rate to 2e-4

4.5

Character, Style and Face LoRA

Type	Trigger Word	Dataset	Steps	Network Dim
Face	"person_name"	15–25 faces	800–1200	32
Character	"char_name"	20–40 images	1000–1500	64
Style	"art_style"	60–100	2000–3000	128
Object	"obj_name"	20–30	600–1000	32

🎯

Module 4 project

A LoRA for a face + style

1.Collect 20 images (a face / person / character of your choice)
2.Prepare a Dataset with BLIP Captions
3.Train a Face LoRA + Style LoRA separately
4.Create 5 images that combine both in ComfyUI

Module five

ComfyUI Pro — Nodes, API and Batch

5.1

Custom Nodes

Installing ComfyUI-Manager, managing Custom Nodes, searching for a Node by function, updates and maintenance. Getting to know the community's central node library.

ComfyUI-Manager Impact Pack WAS Suite

5.2

Advanced Workflows

Hires Fix, Img2Img, Inpainting/Outpainting, Face Restoration with CodeFormer, Upscaling with Real-ESRGAN. Building a complete Workflow from scratch for maximum variety.

Hires Fix CodeFormer ESRGAN

5.3

ComfyUI API — Batch Generation

Full automation of generation — send a list of Prompts, wait for the results and save to disk. Useful for content production at scale:

Python — ComfyUI API Batch Generation

import json
import urllib.request
import time
import os

COMFYUI_URL = "http://127.0.0.1:8188"

def load_workflow(path: str) -> dict:
    with open(path, 'r', encoding='utf-8') as f:
        return json.load(f)

def queue_prompt(workflow: dict, client_id: str = "batch_job") -> dict:
    p = {"prompt": workflow, "client_id": client_id}
    data = json.dumps(p).encode('utf-8')
    req = urllib.request.Request(f"{COMFYUI_URL}/prompt", data=data)
    return json.loads(urllib.request.urlopen(req).read())

def get_history(prompt_id: str) -> dict:
    url = f"{COMFYUI_URL}/history/{prompt_id}"
    return json.loads(urllib.request.urlopen(url).read())

def batch_generate(workflow_path: str, prompts: list, output_dir: str):
    """generate images for each prompt in the list"""
    os.makedirs(output_dir, exist_ok=True)
    workflow = load_workflow(workflow_path)

    for i, prompt_text in enumerate(prompts):
        # find and update the CLIP Text Encode node
        for node in workflow.values():
            if node.get('class_type') == 'CLIPTextEncode':
                if 'positive' in node.get('_meta', {}).get('title', '').lower():
                    node['inputs']['text'] = prompt_text

        # send to the queue
        result = queue_prompt(workflow)
        prompt_id = result['prompt_id']
        print(f"[{i+1}/{len(prompts)}] Queued: {prompt_id[:8]}...")

        # wait for completion
        while True:
            history = get_history(prompt_id)
            if prompt_id in history:
                print(f"  Done: {prompt_text[:40]}...")
                break
            time.sleep(1)

    print(f"\nDone! {len(prompts)} images generated")

# Usage:
prompts = [
    "a red sports car on mountain road, golden hour",
    "vintage blue bicycle in paris street, rainy day",
    "modern kitchen with marble countertops, natural light",
]

batch_generate("my_workflow.json", prompts, "./output/batch_001")

5.4

Batch Processing — strategies and tips

Producing images at scale requires planning ahead. Here are key principles:

✓Queue Management: ComfyUI supports a multi-job Queue — send everything and let it work in the background
✓WebSocket Monitoring: Connect to ws://localhost:8188/ws for real-time progress updates
✓VRAM Management: Add a delay of 2–3 seconds between jobs to allow VRAM cleanup
✓Saving Metadata: Save prompt_id, seed and prompt_text for each image for future referencing

🎯

Capstone project — Module 5

An automatic AI Art Pipeline

Build a full Pipeline that receives a list of topics, generates images in 3 different styles with your LoRA, and saves everything to an organized folder with metadata.

This is the big project — share it in the Discord community to get Feedback from course members.

workspace_premium

Course completion

Congratulations!

You completed the Stable Diffusion & ComfyUI Pro course. You are now ready to work as a professional AI Artist — with deep knowledge of SDXL, Flux, ControlNet, LoRA Training and the ComfyUI API.

Modules

Lessons

Projects

15+

Hours

forum Share on Discord Browse more courses

Full access to all materials — forever

The private SD Pro Discord community

Share your projects, ask questions about workflows, and get Feedback from experienced AI Artists. The community is active and you are part of it.

Join the community arrow_back

Exclusive content for registered members

Stable Diffusion & ComfyUI Pro

Work environment

Installation on Windows (NVIDIA GPU)

Installation on Mac (Apple Silicon)

Google Colab Setup (without a local GPU)

ComfyUI Basics: structure and Nodes

A full environment — the first 20 images

SDXL and Flux Mastery

Differences between the models

Prompt Anatomy for SDXL

CFG, Steps and Sampler Guide

Seed Management and ComfyUI API

Workflow: High-Quality Portrait

20 images in a consistent style

ControlNet — full control over composition

OpenPose — pose control

Depth Map — three-dimensional composition

Canny Edge — composition stabilization

IP-Adapter — Reference Style

Product Photography Pipeline

LoRA Training — train your own model

Dataset Preparation

Kohya SS on Google Colab

LoRA Training Script

Testing and Fine-tuning

Character, Style and Face LoRA

A LoRA for a face + style

ComfyUI Pro — Nodes, API and Batch

Custom Nodes

Advanced Workflows

ComfyUI API — Batch Generation

Batch Processing — strategies and tips

An automatic AI Art Pipeline

Congratulations!