Open WebUI + Ollama: Advanced Production Setup

What Is Open WebUI with Ollama?

Open WebUI is an open-source web interface that serves as a private alternative to ChatGPT. Ollama is an engine for running AI models locally. Together, they provide a complete AI chat experience running on your own server — with zero data sent to external parties.

This guide goes beyond basic setup to cover advanced configurations: running multiple models, setting up authentication, optimizing performance, and integrating with external systems.

Advanced Requirements

Requirement	Minimum	Recommended
CPU	4 vCPU	8 vCPU
Memory	16 GB RAM	32 GB RAM
Storage	100 GB SSD	200 GB SSD
OS	Ubuntu 22.04+	Ubuntu 24.04 LTS
Docker	24.0+	Latest stable

Why 32 GB? Running two mid-size models concurrently (e.g., Llama 3 8B + Mistral 7B) requires approximately 24 GB, with the remainder for the OS and cache.

Advanced Docker Compose Setup

Instead of running each container manually, use Docker Compose to manage all services as a single unit. The recommended docker-compose.yml includes:

• Ollama container with persistent model storage (volume mapping)
• Open WebUI container connected to Ollama via internal Docker network
• Nginx container as a reverse proxy with SSL certificate
• Automatic restart (restart: unless-stopped) for all services

Critical Configuration Points

• Model directory binding: Map /root/.ollama to a host directory so models persist across container rebuilds
• Memory limits: Set maximum memory per container to prevent one service from consuming all resources
• Internal network: Do not expose Ollama's port (11434) to the internet — Open WebUI connects internally via Docker network

Managing Multiple Models

Ollama supports running multiple models. Each model loads into memory on demand and unloads after a period of inactivity. For effective management:

Model Selection by Use Case

Use Case	Recommended Model	Size	RAM Required
General chat	Llama 3.1 8B	4.7 GB	8 GB
Coding	CodeLlama 13B	7.4 GB	16 GB
Arabic summarization	Jais 13B	7.2 GB	16 GB
Document analysis	Mistral 7B	4.1 GB	8 GB

Model Performance Optimization

• Quantization: Use Q4_K_M format models for the best balance between quality and performance. Accuracy difference is less than 2% compared to the full model
• Context length: Reduce context length to 4096 instead of 8192 if long conversations are not needed — saves 40% memory
• Model retention: Set OLLAMA_KEEP_ALIVE to control how long a model stays in memory after the last request

Authentication and Permissions

Open WebUI includes a built-in authentication system. For enterprise deployment:

• Create an admin account on first login
• Enable closed registration (invite-only) via security settings
• Restrict available models per user group
• Enable conversation logging for audit purposes (if required by regulations)

External System Integration

Connecting to OpenAI-Compatible APIs

Open WebUI supports any OpenAI-compatible API. This allows using external models as a fallback alongside local models. In settings, add an external API endpoint as an additional model source.

Knowledge Base Integration (RAG)

The most powerful feature of Open WebUI: uploading PDF or text documents and linking them to conversations. The system converts documents to embeddings stored locally, then uses them to improve model responses.

• Upload your company manual or internal policies as a knowledge base
• The model cites actual documents in its responses
• All data remains on your server — nothing is sent externally

Monitoring and Maintenance

• Memory monitoring: AI models consume variable memory. Set alerts at 85% utilization
• Model updates: Ollama releases periodic improvements. Update via pull command weekly
• Model cleanup: Remove unused models to reclaim storage space
• Backups: Back up the Open WebUI data directory (conversations and settings) plus the models directory regularly

Expected Performance

On a CLOUD (cloud.net.sa) server with 8 vCPU and 32 GB RAM:

• Llama 3.1 8B: 15-20 tokens per second
• Mistral 7B: 18-22 tokens per second
• Time to first token: 2-4 seconds (initial response is slower due to model loading)
• Concurrent users: 3-5 with acceptable performance

Open WebUI with Ollama on a Saudi server gives you a private ChatGPT for your organization — with full data protection and localization and at a lower cost than monthly subscriptions to external services.

--- ### Get Started Ready to run AI on your own server? Explore AI hosting solutions or configure your server with the specs you need.