Stable Diffusion: AI Image Generation on Your Server Step by Step

What Is Stable Diffusion?

Stable Diffusion is an open-source model for generating images from text. You write a description like "modern office building in Riyadh at sunset" and it produces a realistic image. The difference from DALL-E and Midjourney: you run it on your server, no subscriptions, no content restrictions, and no limit on the number of images.

The model works through a process called "diffusion" — it starts from random noise and gradually removes the noise until an image matching the text description appears.

Core Requirements

Minimum

Component	Specs	Notes
GPU	8 GB VRAM	SDXL needs 8 GB minimum
CPU	4 vCPU	For request processing
RAM	16 GB	System + processing
Storage	50 GB SSD	Models + generated images
OS	Ubuntu 22.04	With NVIDIA Drivers

Recommended

Component	Specs	Notes
GPU	16-24 GB VRAM	Higher quality and faster images
CPU	8 vCPU	Parallel processing
RAM	32 GB	Multiple models
Storage	200 GB SSD	Model library + LoRA

Important note: Stable Diffusion requires a GPU — without one, each image takes 5-10 minutes instead of seconds.

Interface Options

Interface	Features	Best For
Automatic1111 (A1111)	Oldest and most stable, extensive extensions	Advanced users
ComfyUI	Node-based interface, full flexibility, faster	Developers and professionals
Fooocus	Simplest interface, Midjourney-like experience	Beginners
SD.Next	Enhanced fork of A1111	Those wanting extra features

This guide uses Automatic1111 as it is the most common and best documented.

Step-by-Step Installation

Step 1: System Preparation

sudo apt update && sudo apt upgrade -y
sudo apt install -y python3 python3-pip python3-venv git wget
sudo apt install -y libgl1 libglib2.0-0

Step 2: Install NVIDIA Drivers

# Check for GPU
nvidia-smi

# If not working, install drivers
sudo apt install -y nvidia-driver-535
sudo reboot

Step 3: Install CUDA Toolkit

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt install -y cuda-toolkit-12-4

Step 4: Download Stable Diffusion WebUI

cd /opt
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
cd stable-diffusion-webui

Step 5: Download the Model

cd models/Stable-diffusion

# SDXL 1.0 — the enhanced base model
wget -O sdxl_base.safetensors \
 "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors"

# SDXL Refiner — for detail enhancement
wget -O sdxl_refiner.safetensors \
 "https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/resolve/main/sd_xl_refiner_1.0.safetensors"

cd /opt/stable-diffusion-webui

Step 6: First Launch

./webui.sh --listen --port 7860 --xformers --api

The first launch downloads required libraries — this may take 10-15 minutes.

Step 7: Configure as System Service

sudo cat > /etc/systemd/system/sd-webui.service << 'EOF'
[Unit]
Description=Stable Diffusion WebUI
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/opt/stable-diffusion-webui
ExecStart=/opt/stable-diffusion-webui/webui.sh --listen --port 7860 --xformers --api
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable sd-webui
sudo systemctl start sd-webui

Performance Settings

Acceleration with xformers

xformers reduces VRAM usage and speeds up generation by 20-30%:

# Activated automatically with --xformers in the launch command
# To verify activation, look in logs for:
# "xformers version: ..."

Limited VRAM Settings

If VRAM is less than 8 GB:

./webui.sh --listen --medvram --xformers
# Or for 4 GB VRAM:
./webui.sh --listen --lowvram --xformers

VAE Acceleration

# Download optimized VAE for SDXL
cd models/VAE
wget -O sdxl_vae.safetensors \
 "https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/sdxl_vae.safetensors"

Writing Effective Prompts

Ideal Prompt Structure

[main subject], [details], [artistic style], [lighting], [quality]

Examples

Architectural photo:

modern office building in Riyadh, glass facade, desert landscape,
architectural photography, golden hour lighting, 8k, ultra detailed

Product photo:

premium black laptop on minimalist desk, clean white background,
product photography, studio lighting, sharp focus, commercial quality

Technical photo:

server room with blue LED lights, rows of rack servers,
data center interior, cinematic lighting, photorealistic, 8k

Negative Prompts

What you do not want in the image:

blurry, low quality, distorted, watermark, text, logo,
deformed, ugly, bad anatomy, bad proportions

Recommended Generation Settings

Setting	SDXL	SD 1.5
Dimensions	1024x1024	512x512
Steps	25-35	20-30
CFG Scale	7-9	7-8
Sampler	DPM++ 2M Karras	Euler a

Using the API

import requests
import base64

url = "http://localhost:7860/sdapi/v1/txt2img"

payload = {
 "prompt": "modern Saudi office building, architectural photo, 8k",
 "negative_prompt": "blurry, low quality, watermark",
 "steps": 30,
 "width": 1024,
 "height": 1024,
 "cfg_scale": 7,
 "sampler_name": "DPM++ 2M Karras",
}

response = requests.post(url, json=payload)
image_data = base64.b64decode(response.json["images"][0])

with open("output.png", "wb") as f:
 f.write(image_data)

Useful Extensions

Extension	Function
ControlNet	Control image composition based on reference image
ADetailer	Automatic facial detail enhancement
Ultimate Upscale	High-quality image upscaling
Regional Prompter	Different prompts for different regions of the image

Operating Cost

With a GPU-equipped server on CLOUD (cloud.net.sa):

• 1,000 images daily at 1024x1024 — easily achievable
• No per-image cost — resources are yours
• Models are free and open source
• Commercial use allowed with most models

Bottom Line

Stable Diffusion on your own server gives you an unlimited image factory. CLOUD provides ready servers with the Stable Diffusion template — choose your specs, pay, and start generating within minutes. Your data and images stay on your server in Riyadh.

---

Get Started

Ready to run AI on your own server? Explore AI hosting solutions or configure your server with the specs you need.