LocalAI: How to Replace OpenAI API with a Private Version on Your Server

What Is LocalAI?

LocalAI is an open-source alternative to the OpenAI API that runs entirely on your local server. It provides the same API interface that OpenAI uses — meaning any application that works with OpenAI works with LocalAI by changing just one line: the API address.

The core idea: Instead of sending your data to OpenAI servers in America, you run models locally and use the same API. The application does not know the difference.

Why LocalAI and Not Ollama?

Criteria	LocalAI	Ollama
OpenAI API Compatibility	Full 100%	Partial
Supported Models	LLM + Images + Audio + Embeddings	LLM + Embeddings only
Image Generation	Yes (Stable Diffusion)	No
Text-to-Speech	Yes (TTS)	No
Speech-to-Text	Yes (Whisper)	No
Function Calling	Yes	Partial
Embedding API	Yes	Yes
Installation Ease	Medium	Very Easy
Resource Usage	Higher	Lower

Summary: Ollama is easier if you only need text models. LocalAI is more comprehensive if you need a full OpenAI-compatible API including images and audio.

What Can You Run on LocalAI?

Language Models (Chat/Completion)

Model	Size (Q4)	Use Case
Llama 3.1 8B	4.7 GB	General conversations
Mistral 7B	4.1 GB	Analysis and classification
Qwen 2.5 7B	4.4 GB	Arabic content
Phi-3 Mini	2.2 GB	Lightweight tasks
CodeLlama 7B	4.0 GB	Programming

Other Models

Function	Model	Size
Image Generation	Stable Diffusion	4-7 GB
Speech-to-Text	Whisper	1.5 GB
Text-to-Speech	Piper TTS	100 MB
Embeddings	all-MiniLM-L6	80 MB
Code Generation	StarCoder	3-7 GB

Server Requirements

Without GPU (Text Models Only)

Component	Minimum	Recommended
CPU	4 vCPU	8 vCPU
RAM	8 GB	16 GB
Storage	40 GB SSD	80 GB SSD
OS	Ubuntu 22.04	Ubuntu 24.04

With GPU (All Functions)

Component	Minimum	Recommended
GPU	8 GB VRAM	16-24 GB VRAM
CPU	4 vCPU	8 vCPU
RAM	16 GB	32 GB
Storage	100 GB SSD	200 GB SSD

Step-by-Step Installation

Step 1: Prepare Docker

sudo apt update
sudo apt install -y docker.io docker-compose-v2
sudo usermod -aG docker $USER
newgrp docker

Step 2: Create Docker Compose File

mkdir -p /opt/localai && cd /opt/localai

cat > docker-compose.yml << 'EOF'
version: '3.8'
services:
 localai:
 image: localai/localai:latest-cpu
 # Or for GPU:
 # image: localai/localai:latest-gpu-nvidia-cuda-12
 restart: always
 ports:
 - "8080:8080"
 environment:
 - MODELS_PATH=/models
 - THREADS=4
 - CONTEXT_SIZE=2048
 - DEBUG=false
 volumes:
 - ./models:/models
 - ./config:/config
 # If you have a GPU:
 # deploy:
 # resources:
 # reservations:
 # devices:
 # - driver: nvidia
 # count: all
 # capabilities: [gpu]

volumes:
 localai_models:
EOF

docker compose up -d

Step 3: Download Models

# Download Llama 3.1 8B
curl http://localhost:8080/models/apply -H "Content-Type: application/json" \
 -d '{"id": "huggingface://TheBloke/Llama-3.1-8B-GGUF/llama-3.1-8b.Q4_K_M.gguf"}'

# Or download a ready model from the LocalAI gallery
curl http://localhost:8080/models/apply -H "Content-Type: application/json" \
 -d '{"id": "llama3.1-8b-instruct"}'

Step 4: Verify It Is Running

# List available models
curl http://localhost:8080/v1/models

# Test a chat
curl http://localhost:8080/v1/chat/completions \
 -H "Content-Type: application/json" \
 -d '{
 "model": "llama3.1-8b-instruct",
 "messages": [
 {"role": "user", "content": "Hello, what is LocalAI?"}
 ]
 }'

Configuration Files

Model Configuration

cat > /opt/localai/config/llama3.yaml << 'EOF'
name: llama3.1-8b-instruct
parameters:
 model: llama-3.1-8b.Q4_K_M.gguf
 temperature: 0.7
 top_p: 0.9
 top_k: 40
 max_tokens: 2048
context_size: 4096
threads: 4
template:
 chat_message: |
 <|start_header_id|>{{.RoleName}}<|end_header_id|>

 {{.Content}}<|eot_id|>
 chat: |
 <|begin_of_text|>{{.Input}}<|start_header_id|>assistant<|end_header_id|>
EOF

Embeddings Configuration

cat > /opt/localai/config/embeddings.yaml << 'EOF'
name: text-embedding-ada-002
parameters:
 model: all-MiniLM-L6-v2-q4_0.bin
backend: sentencetransformers
embeddings: true
EOF

Replacing OpenAI in Your Application

Python

from openai import OpenAI

# Instead of:
# client = OpenAI(api_key="sk-xxx")

# Use:
client = OpenAI(
 base_url="http://localhost:8080/v1",
 api_key="not-needed" # LocalAI does not need a key
)

response = client.chat.completions.create(
 model="llama3.1-8b-instruct",
 messages=[
 {"role": "system", "content": "You are a helpful assistant."},
 {"role": "user", "content": "Write a brief project plan."}
 ]
)

print(response.choices[0].message.content)

JavaScript/Node.js

const OpenAI = require('openai');

const client = new OpenAI({
 baseURL: 'http://localhost:8080/v1',
 apiKey: 'not-needed',
});

const response = await client.chat.completions.create({
 model: 'llama3.1-8b-instruct',
 messages: [
 { role: 'user', content: 'What are the best information security practices?' }
 ],
});

console.log(response.choices[0].message.content);

cURL (Any Language)

curl http://localhost:8080/v1/chat/completions \
 -H "Content-Type: application/json" \
 -d '{
 "model": "llama3.1-8b-instruct",
 "messages": [{"role": "user", "content": "Hello"}],
 "temperature": 0.7
 }'

Embeddings API

curl http://localhost:8080/v1/embeddings \
 -H "Content-Type: application/json" \
 -d '{
 "model": "text-embedding-ada-002",
 "input": "Text to convert to a vector"
 }'

The result: a vector in the same format as OpenAI — works directly with pgvector or any vector database.

Text-to-Speech (TTS)

# Download TTS model
curl http://localhost:8080/models/apply -H "Content-Type: application/json" \
 -d '{"id": "voice-en-us-amy-low"}'

# Convert text to speech
curl http://localhost:8080/v1/audio/speech \
 -H "Content-Type: application/json" \
 -d '{
 "model": "voice-en-us-amy-low",
 "input": "Welcome to our service.",
 "voice": "amy"
 }' --output speech.wav

Compatible Applications

Any application using the OpenAI API works with LocalAI directly:

Application	Type	Change Required
Open WebUI	Chat interface	Change Base URL only
n8n	Automation	Change Credentials
Langflow	AI app builder	Change Base URL
Flowise	Chatbot builder	Change Base URL
LangChain	Python library	Change base_url
Any custom app	Various	Change API endpoint

Monitoring and Maintenance

Performance Monitoring

# Service health
curl http://localhost:8080/readyz

# Resource usage
docker stats localai

Backup

# Back up models and config
tar -czf localai-backup-$(date +%Y%m%d).tar.gz /opt/localai/

Cost Comparison: LocalAI vs OpenAI API

Comparison	OpenAI API	LocalAI on CLOUD
GPT-4o Mini (1M tokens)	0.60 USD	0 (fixed)
Embeddings (1M tokens)	0.10 USD	0 (fixed)
TTS (1M characters)	15 USD	0 (fixed)
Whisper (per minute)	0.006 USD	0 (fixed)
Monthly Server Cost	0	~competitive pricing
Break-even Point	~2,000 requests/day	—

For heavy usage (more than 2,000 requests per day), LocalAI saves over 70% in costs.

Bottom Line

LocalAI transforms your server into a complete OpenAI replacement — same API, same format, but data stays with you. With a Cloud Server from CLOUD (cloud.net.sa) in Riyadh, you get sub-very low latency and full compliance with Saudi data protection regulations. The Enterprise plan (8 vCPU, 16 GB) runs LocalAI with 7B models efficiently. The template is ready for one-click deployment — change one line in your code and move from OpenAI to your own server.

---

Get Started

Ready to run AI on your own server? Explore AI hosting solutions or configure your server with the specs you need.