Observability for AI Apps With Self-Hosted Langfuse

Langfuse is an open-source observability platform for LLM applications. It records every request that passes through your app as a complete trace: what the user asked, which chunks your RAG pipeline retrieved, the exact prompt sent to the model, the answer, token counts, latency and cost. You need it because regular application logs tell you the request succeeded with a 200 in 4 seconds, yet they cannot tell you the answer was wrong because search pulled a 2023 leave policy, or that one request cost 0.40 SAR because someone pasted a 30-page document. The short answer for running it: clone the official Langfuse repository, change the secrets marked in docker-compose.yml, run docker compose up -d on a server with 4 cores and 16GB of memory, then add three environment variables and an @observe decorator to your app functions. Within an hour you can see every request step by step, with a cost dashboard by day and by user. Since conversations usually contain customer data and sensitive internal questions, running Langfuse on a Saudi server instead of the external cloud version settles the question of where that data lives from the start.

Why regular logs are not enough

An AI application is not one function but a chain. It embeds the question, searches the vector store, reranks, builds the prompt, calls the model, and sometimes calls a tool and then the model again. When an employee complains about a bad answer, you need to see the whole chain to find where it broke. There are four questions that Nginx or Laravel logs cannot answer:

• What did the model actually see? The full prompt after chunks were inserted, not just the original question.
• Where did the time go? Were the six seconds spent on search or on generating 900 tokens?
• What did this request cost? Input and output tokens multiplied by the model price.
• Was the answer good? A thumbs up or down from the user, or an automated judgment by another model.

Langfuse answers all four on one screen and groups them by user, session and prompt version, so you can compare version 3 and version 4 of your system prompt across 500 real requests.

What it takes to run, in numbers

The current Langfuse release (version 3) is not a single container but six services: the web app, a background worker, PostgreSQL for metadata and users, ClickHouse for traces and analytics, Redis for queues, and MinIO as S3-compatible object storage for raw events. The official Docker Compose file runs them all together, and the official guidance for a single machine is at least 4 cores and 16GB of memory, because ClickHouse alone uses 3 to 6GB under load.

Load	Specs	CLOUD option	Annual incl. VAT
Trial and small team, up to 20K traces a day	4 vCPU / 8GB	Business plan	11,500 SAR
Mid production, up to 100K traces a day	4 vCPU / 16GB	Business + 4 RAM add-ons (8GB)	13,900 SAR
Larger production with long retention	8 vCPU / 16GB + 200GB	Enterprise + 2 SSD add-ons	19,000 SAR

The second row works out as the Business plan at 11,500 SAR plus four memory add-ons of 2GB at 600 SAR each, which is 2,400 SAR, for a total of 13,900 SAR per year. Storage grows with traces. An average RAG trace takes 5 to 20KB after ClickHouse compression, so 100 thousand traces a day means roughly 1GB per day before further compression, which is why you need a clear retention policy.

Step-by-step setup

On a cloud server running Ubuntu 24.04 with Docker installed:

git clone https://github.com/langfuse/langfuse.git
cd langfuse
# change every value marked with the CHANGEME comment
grep -n CHANGEME docker-compose.yml
openssl rand -hex 32   # for ENCRYPTION_KEY
openssl rand -base64 32   # for NEXTAUTH_SECRET and SALT
docker compose up -d
docker compose ps

Do not skip the secrets step. The default passwords in the file are public knowledge, and any of these services exposed to the internet means your company conversations are readable. Just as important, the internal ports (5432 for PostgreSQL, 8123 for ClickHouse, 6379 for Redis and 9000 for MinIO) must never be published on the public interface. Bind the web UI to 127.0.0.1:3000, put it behind Nginx with an SSL certificate, set AUTH_DISABLE_SIGNUP=true after creating the first account, and add teammates by invitation.

server {
    server_name langfuse.example.sa;
    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto https;
    }
}

Connecting it to a RAG app

Create a project in the Langfuse UI and copy its public and secret keys, then in your app:

pip install langfuse

export LANGFUSE_PUBLIC_KEY=pk-lf-...
export LANGFUSE_SECRET_KEY=sk-lf-...
export LANGFUSE_HOST=https://langfuse.example.sa

from langfuse import observe

@observe()
def retrieve(question: str) -> list[str]:
    return vector_search(question, k=5)

@observe(as_type="generation")
def answer(question: str, chunks: list[str]) -> str:
    return llm.chat(build_prompt(question, chunks))

@observe()
def rag(question: str) -> str:
    return answer(question, retrieve(question))

Every call to rag produces one trace with two nested steps, each showing inputs, outputs and duration. If you use the OpenAI SDK, LangChain or LlamaIndex, Langfuse has ready integrations that capture tokens and model names automatically with a single import line. Attach a user ID and session ID to each trace so you can later open everything a specific employee asked when they report a problem.

Masking sensitive data before it is stored

Even on your own server there is no reason to store national ID and mobile numbers as they are. The SDK supports a mask function that runs over every input and output before sending, so replace any 10-digit number starting with 1 or 2 with a fixed token, and treat any mobile starting with 05 the same way.

Cost per request in riyals

Langfuse calculates cost from a model price table in dollars, and you can define your own models with a price per token. For cloud models, multiply by 3.75 to get riyals or enter the converted price directly. For a local model running through Ollama the math is different, because the cost is fixed and equals the server itself. A server at 18,000 SAR a year costs about 49 SAR a day, so if it processes 2 million tokens a day the cost is about 2.5 SAR per 100 thousand tokens, and every extra request lowers the unit cost further. Define that price in Langfuse and a single dashboard compares local and cloud requests on the same scale.

In Dashboards, show cost aggregated by day, by calendar month, by user and by feature, and set an alert when a single user passes 50 SAR in a day. That threshold usually catches a script calling the API in a loop before the bill arrives.

Managing prompt versions

Most RAG improvements come from editing the system prompt rather than changing the model, and the problem is that the prompt usually sits buried in code, so every edit needs a deploy and nobody remembers which version worked last week. Langfuse includes prompt management. You save a prompt under a stable name such as hr-assistant, every edit creates a numbered version, and you put a production label on the approved one. Your app fetches the prompt by name and label with a local cache, so the fetch adds only a few milliseconds, and when the content owner rewords the prompt they move the label to the new version without a developer touching code.

The bigger win is that every trace is linked automatically to the prompt version it used. If you ship version 7 on Sunday and the average score drops from 0.82 to 0.71, you see it on one dashboard and move the label back to version 6 within a minute.

Retention and backups

Traces pile up fast, and a year-old trace rarely has much value. Pick a retention period that fits your operational and regulatory needs, for example 90 days for detailed traces, while keeping scores and aggregated numbers longer. Take a daily PostgreSQL backup with pg_dump and a ClickHouse backup with its own backup tooling, and store both off the server. The daily backup add-on with 7-day retention costs 800 SAR per year and covers the whole server if a disk fails or a directory gets deleted by mistake.

Measuring answer quality

Cost and latency are easy, while quality takes a decision. Start with two methods. The first is a rating button in your UI that sends a score of 1 or 0 tied to the trace ID, and the second is an automated evaluation where a language model reads the question, chunks and answer and judges whether the answer is grounded in the chunks or made up. Langfuse runs these evaluations on a sample, such as 10% of requests, so cost does not double. After two weeks you will see which kinds of questions score lowest, and you will often find the problem is a missing document rather than the model.

To run the local model on the same infrastructure, look at the AI hosting templates, which ship with Ollama and Open WebUI preinstalled.

Getting started

1. Provision a server with 4 cores and 16GB in Riyadh, and pick the size from pricing and plans.
2. Run docker compose after changing every CHANGEME value, and bind every port to 127.0.0.1 except Nginx.
3. Add @observe to your retrieval and generation functions, with user and session IDs and a mask function for sensitive numbers.
4. Define your model prices in riyals and an alert at 50 SAR per user per day.
5. Turn on automated evaluation for 10% of requests and review the lowest-scoring questions every week.