Stable Diffusion
The complete guide — ComfyUI, SDXL, Flux & LoRA

What is Stable Diffusion and why is it revolutionary?

Stable Diffusion is an Open Source model for generating images from text that you can run locally on your machine — no payment, no subscription, no content restrictions. Released in 2022 by Stability AI, it completely changed the world of AI image generation.

Unlike Midjourney (server, paid) or DALL-E (paid API), SD runs at home on your own machine. No one sees your prompts, there are no limits on the number of images, and you can change every aspect of the generation process.

How does the Diffusion Process work?

The Diffusion process works in two stages:

A quick comparison — SD vs Midjourney

The bottom line: if you want professional control, LoRA for faces, ControlNet for poses, batch generation — SD is the choice. If you want beautiful images instantly in 2 seconds with no settings — Midjourney.

What can you do with Stable Diffusion?

• txt2img — generating an image from text (the basic function)
• img2img — editing an existing image by Prompt
• Inpainting — editing a specific area of the image
• Outpainting — extending the image's boundaries to new areas
• ControlNet — preserving pose/depth/lines while changing style
• LoRA Training — training a model on a specific face/style
• API / Automation — producing hundreds of images in a Python script

Installation — ComfyUI & A1111

There are two main options: ComfyUI (recommended — based on a Node Graph, powerful, flexible) andAUTOMATIC1111 (A1111 — a classic interface, easier for beginners). We'll explain both.

ComfyUI — the recommended installation

# Step 1 — make sure Python 3.10–3.11 is installed
python3 --version

# Step 2 — clone the repo
git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI

# Step 3 — install dependencies
pip install -r requirements.txt

# Step 4 — download an SDXL model (1.8GB)
# place in: models/checkpoints/dreamshaper_xl.safetensors

# Step 5 — run
python main.py --listen
# open in browser: http://localhost:8188

AUTOMATIC1111 — the classic interface

# Mac / Linux:
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui
cd stable-diffusion-webui
./webui.sh

# Windows:
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui
cd stable-diffusion-webui
webui-user.bat

# open at: http://127.0.0.1:7860

Models, Checkpoints and VAE

The heart of Stable Diffusion is theCheckpoint model — a .safetensors file that contains the neural network weights. Each checkpoint gives a different style and quality. Download them fromCivitai.com and HuggingFace.co.

Checkpoint Models — a full comparison

VAE — what it is and why it matters

The VAE (Variational Autoencoder) is the "translator" between Latent Space (the model's internal latent space) and pixel images. When the VAE is poor — images come out with oversaturated colors, blurry faces, and strange eyes.

• SDXL VAE (sdxl_vae.safetensors) — the official VAE for SDXL. Download separately and save inmodels/vae/
• vae-ft-mse-840000 — an excellent VAE for SD 1.5

ControlNet Models

• controlnet-canny-sdxl-1.0 — contour detection (Canny Edge)
• controlnet-depth-sdxl-1.0 — preserving 3D depth
• controlnet-openpose-sdxl-1.0 — preserving body pose
• ip-adapter-plus_sdxl_vit-h — transferring style and face

LoRA Models

LoRA files (.safetensors, 50–200MB) add specific styles on top of the main Checkpoint. Save inmodels/loras/.

Writing Prompts for images

A Prompt for an image works differently from a Prompt for text. It's a descriptive structure, not a question. The basic formula:

[quality] + [main subject] + [style] + [lighting] + [photographer/artist] + [additional details]

A full Prompt — an example with explanation

POSITIVE PROMPT:
(masterpiece, best quality:1.2), ohwx woman,
portrait photography, studio lighting, sharp focus,
bokeh background, wearing elegant suit, confident pose,
Canon EOS R5, f/1.8, 8k, skin texture, photorealistic

NEGATIVE PROMPT:
(worst quality, low quality:1.4), deformed hands,
extra fingers, blurry, watermark, text, jpeg artifacts,
bad anatomy, disfigured, ugly, mutation

Word weighting — Prompt Weighting

(beautiful:1.3)     # stronger emphasis — 30% more influence
(beautiful:0.7)     # weaker emphasis
[fade out]          # a fade-out effect
(red:1.2) dress     # emphasizing the color only
((very important))  # doubling = maximum emphasis

Critical parameters

Quality Tags — a quick list

# high quality:
masterpiece, best quality, ultra detailed, 8k, RAW photo

# realistic style:
photorealistic, hyperrealistic, photography, DSLR, film grain

# lighting:
studio lighting, golden hour, cinematic lighting,
soft diffused light, dramatic shadows, rim lighting

# artistic style:
oil painting, watercolor, digital art, concept art,
anime style, illustration, artstation

ComfyUI Workflows

ComfyUI works with a Node Graph — each step in creating the image is represented by a Node, and the Nodes are connected by Wires. This gives full transparency and control over every aspect of the process.

The basic Nodes

• CheckpointLoaderSimple — loads the Checkpoint model
• CLIPTextEncode — encodes the Prompt into Embeddings (always two: positive and negative)
• EmptyLatentImage — creates an empty image (Latent) at the desired size
• KSampler — the heart — performs the Denoising process
• VAEDecode — decodes the Latent into pixels
• SaveImage — saves the image to disk

Workflow — img2img

# instead of EmptyLatentImage:
Load Image → VAEEncode → KSampler (with denoise: 0.5–0.8)

# denoise 0.3 = a slight change (keeps 70% of the original image)
# denoise 0.8 = a large change (only 20% of the original)

Sharing Workflows — JSON

Every workflow in ComfyUI can be saved/loaded as a JSON file. You'll find thousands of workflows atcomfyworkflows.com and openart.ai/workflows. Just drag it into the interface.

ControlNet — precise control over the image structure

ControlNet is one of the most significant innovations in SD. It lets you control the structure, pose, depth and contour of the image — while keeping the flexibility of the prompt. This is what sets SD apart from Midjourney.

Types of ControlNet and when to use them

• Canny Edge — detects sharp contours. Excellent for preserving shapes: buildings, products, logos
• Depth Map — detects 3D depth. Excellent for preserving scene structure while changing style
• OpenPose — detects 18 body points. Lets you copy a pose to another character
• IP-Adapter — Style Transfer — takes the "essence" from one image and transfers it to another
• Lineart — preserving Lineart lines. Excellent for animation and illustration
• Inpaint — preserving the background while changing specific areas

Python code — ControlNet with Diffusers

from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel
import torch, cv2, numpy as np
from PIL import Image

# loading ControlNet Canny
controlnet = ControlNetModel.from_pretrained(
    "diffusers/controlnet-canny-sdxl-1.0",
    torch_dtype=torch.float16
)

# loading the Pipeline
pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    controlnet=controlnet,
    torch_dtype=torch.float16
).to("cuda")

# preparing the Canny Edge
image = Image.open("input.jpg")
img_array = np.array(image)
canny = cv2.Canny(img_array, 100, 200)
canny_rgb = np.stack([canny] * 3, axis=-1)
canny_pil = Image.fromarray(canny_rgb)

# generating an image with ControlNet
result = pipe(
    prompt="portrait of woman, studio lighting, photorealistic, 8k",
    negative_prompt="blurry, deformed, worst quality",
    image=canny_pil,
    controlnet_conditioning_scale=0.8,   # ControlNet strength
    num_inference_steps=25,
    guidance_scale=7.5
).images[0]

result.save("output_controlnet.png")

IP-Adapter — Style Transfer and Face Transfer

IP-Adapter is a special ControlNet that takes a "reference" image and uses it to guide style or face. Excellent for creating images of the same person in different styles.

from diffusers import StableDiffusionXLPipeline
from ip_adapter import IPAdapterPlusXL

pipe = StableDiffusionXLPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

ip_adapter = IPAdapterPlusXL(
    pipe,
    image_encoder_path="h94/IP-Adapter/models/image_encoder",
    ip_ckpt="ip-adapter-plus_sdxl_vit-h.bin",
    device="cuda"
)

ref_image = Image.open("face_reference.jpg")

images = ip_adapter.generate(
    pil_image=ref_image,
    prompt="portrait, oil painting style, Van Gogh",
    negative_prompt="worst quality, blurry",
    scale=0.6,       # IP-Adapter strength
    num_samples=1,
    num_inference_steps=30,
    seed=42
)

Flux.1 — the best model of 2025

Flux.1 was released in 2024 by Black Forest Labs (the original founders of Stable Diffusion) and represents a significant leap in quality. 12 billion parameters, a Transformer architecture (not U-Net), and realistic images at a level not seen before.

Flux.1 Dev vs Schnell vs Pro

GGUF Quantization — Flux on 8GB VRAM

GGUF quantization lets you run Flux.1 even on cards with 8GB VRAM by compressing the weights from BF16 to Q4_K_S (4-bit). Slightly lower quality, but accessible to most users.

# download Flux.1 Schnell GGUF (8GB):
# https://huggingface.co/city96/FLUX.1-schnell-gguf
# choose: flux1-schnell-Q4_K_S.gguf (~8.3GB)

# ComfyUI — use UNETLoader instead of CheckpointLoader:
# models/unet/flux1-schnell-Q4_K_S.gguf

# recommended settings for Flux Schnell:
# Steps: 4  |  CFG: 1  |  Sampler: euler  |  Scheduler: simple

5 practical projects

The projects are ordered by difficulty — beginner to advanced. Each project includes what you need, the Workflow, and where to start.

import requests, json, random

COMFY_URL = "http://127.0.0.1:8188"

def generate_image(prompt, seed=None):
    seed = seed or random.randint(0, 2**32)
    workflow = {
        "3": {"class_type": "KSampler",
              "inputs": {"seed": seed, "steps": 25, "cfg": 7,
                         "sampler_name": "dpmpp_2m", "scheduler": "karras",
                         "denoise": 1.0, "model": ["4", 0],
                         "positive": ["6", 0], "negative": ["7", 0],
                         "latent_image": ["5", 0]}},
        "4": {"class_type": "CheckpointLoaderSimple",
              "inputs": {"ckpt_name": "dreamshaper_xl.safetensors"}},
        "5": {"class_type": "EmptyLatentImage",
              "inputs": {"width": 1024, "height": 1024, "batch_size": 1}},
        "6": {"class_type": "CLIPTextEncode",
              "inputs": {"text": prompt, "clip": ["4", 1]}},
        "7": {"class_type": "CLIPTextEncode",
              "inputs": {"text": "worst quality, blurry", "clip": ["4", 1]}},
        "8": {"class_type": "VAEDecode",
              "inputs": {"samples": ["3", 0], "vae": ["4", 2]}},
        "9": {"class_type": "SaveImage",
              "inputs": {"images": ["8", 0], "filename_prefix": "batch"}}
    }
    r = requests.post(f"{COMFY_URL}/prompt",
                      json={"prompt": workflow})
    return r.json()

prompts = [f"portrait of person {i}, studio lighting" for i in range(100)]
for i, p in enumerate(prompts):
    generate_image(p, seed=i)
    print(f"Generated {i+1}/100")

Cheat sheet — a full Cheat Sheet

Sampler Comparison — which Sampler to choose?

CFG Scale Guide

Resolution Best Practices

Common Errors — fixing common errors

Quality Tags — a quick list

# Positive — always add at the start of the Prompt:
(masterpiece, best quality:1.2), ultra detailed, 8k

# Photography:
RAW photo, DSLR, sharp focus, f/1.8, Canon EOS R5

# Lighting:
studio lighting, golden hour, cinematic, rim light

# Faces:
detailed eyes, perfect skin, natural makeup

# Negative — always add:
(worst quality, low quality:1.4), deformed,
extra fingers, bad anatomy, watermark, text,
blurry, duplicate, mutation, ugly