Skip to main content

Langfuse

Langfuse is an open-source LLM engineering platform with a polished tracing UI, session grouping, and cost attribution. It has a generous free cloud tier and a self-host option via docker-compose.

Signals: traces only. Deployment: local (docker compose) or cloud. Cost: OSS (self-host) / free tier + paid (cloud).

Cloud (fastest)​

Sign up at cloud.langfuse.com, create a project, and grab the public + secret keys from Settings → API Keys.

Option A — plugin-specific env vars:

export OTEL_LANGFUSE_PUBLIC_API_KEY="pk-lf-..."
export OTEL_LANGFUSE_SECRET_API_KEY="sk-lf-..."
# Optional — defaults to EU cloud. A root URL, ".../api/public/otel" or the
# full ".../api/public/otel/v1/traces" all work; the plugin completes the path.
export OTEL_LANGFUSE_ENDPOINT="https://cloud.langfuse.com/api/public/otel/v1/traces"
# US region:
# export OTEL_LANGFUSE_ENDPOINT="https://us.cloud.langfuse.com/api/public/otel/v1/traces"

Option B — Langfuse-standard env vars from their docs:

export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_BASE_URL="https://cloud.langfuse.com"

Both forms work for the credentials. In env-var mode (no backends: list), Langfuse is only selected when at least one OTEL_LANGFUSE_* variable is set, so Option B alone does not turn export on; add OTEL_LANGFUSE_ENDPOINT (or use a backends: entry). The plugin automatically constructs the Authorization: Basic ... header from the two keys — you don't need to base64-encode anything yourself.

Self-hosted​

Langfuse self-host is a full stack (Langfuse + Postgres + Redis + ClickHouse + MinIO). The plugin ships with a ready-to-go compose file:

cd ~/.hermes/plugins/hermes_otel
docker compose -f docker-compose/langfuse/docker-compose.yaml up -d
# Wait ~60s for ClickHouse to start

Pre-seeded test keys:

export OTEL_LANGFUSE_PUBLIC_API_KEY="lf_pk_hermes_dev"
export OTEL_LANGFUSE_SECRET_API_KEY="lf_sk_hermes_dev"
export OTEL_LANGFUSE_ENDPOINT="http://localhost:3000"   # completed to /api/public/otel/v1/traces

UI at http://localhost:3000.

The same file runs Langfuse v4 with LANGFUSE_VERSION=4 docker compose -f docker-compose/langfuse/docker-compose.yaml up -d (use down -v when switching major versions). Export works unchanged on v4. Its default events_only mode removes /api/public/traces, /api/public/observations and /api/public/sessions (all 404, #246); the dashboard's Langfuse adapter detects that once per query URL and reads v4 through /api/public/v2/observations and /api/public/v2/metrics instead (see Dashboard). Verified on 4.56.0 and on 3.225.11 on 2026-10-09; query_api: v3 or v4 on the entry pins the API.

Multi-backend config​

# ~/.hermes/hermes_otel.yaml
backends:
  - type: langfuse
    public_key_env: LANGFUSE_PUBLIC_KEY
    secret_key_env: LANGFUSE_SECRET_KEY
    base_url: https://cloud.langfuse.com
    # Or override the full OTLP path:
    # endpoint: https://cloud.langfuse.com/api/public/otel/v1/traces

Secrets should live in env vars (*_env: keys). Plaintext public_key: / secret_key: also work but are discouraged.

What you'll see​

Langfuse groups traces into sessions automatically. hermes-otel's agent / cron root spans show up as top-level traces; nested llm.* / api.* / tool.* appear as observations within.

  • User message lands on gen_ai.content.prompt / input.value on the llm.* span.
  • Assistant response lands on gen_ai.content.completion / output.value.
  • Token counts use gen_ai.usage.input_tokens and gen_ai.usage.output_tokens on api.* spans.
  • Tool calls appear as child spans with inputs/outputs.

Attribute convention​

Langfuse keys off gen_ai.*. The plugin emits that alongside the OpenInference convention so the same span serves both UIs — see Attribute conventions.

Metrics​

Langfuse doesn't accept OTLP metrics — it's trace-only. The plugin auto-skips the metrics exporter when Langfuse is the sole backend. If you want token/tool/cost metrics too, fan out to a metrics-capable backend in parallel; see Multi-backend.

Dashboard​

A trace ingested through OTLP keeps its real root observation (the plugin's agent span); only when Langfuse holds no single parentless observation does the bundled dashboard build one top-level node from the trace record to hold the tree together. That node carries synthetic: true and a synthetic.reason, so it is never mistaken for a span the agent emitted.

Langfuse v4 (events_only, the default): the trace list is built from the AGENT root observations of /api/public/v2/observations (one per Hermes turn; session id, an ERROR level and the native k=v parameters filter server-side, name prefix and minimum duration on the rows), and each page's token totals, cost, observation count and model join from one /api/public/v2/metrics query grouped by trace. The detail is the trace's observations with their parent links and status; the v4 public API carries no input, output or per-observation usage, so the previews and per-span tokens are absent and the trace's totals sit on the root. Langfuse Cloud retires the v3 endpoints on 2026-11-16, so cloud projects read through this path from then on.

Troubleshooting​

"Auth failed / 401 from Langfuse"

  • You need both keys (public + secret). Langfuse won't authenticate with only one.
  • LANGFUSE_BASE_URL / base_url take the site root; OTEL_LANGFUSE_ENDPOINT / endpoint accept the root, …/api/public/otel or the full …/api/public/otel/v1/traces — every form is completed to the traces URL (a root URL used to be posted to as-is and answered 405).

"Nothing arrives and there is no error"

  • The OTLP exporter's failures (a 405 or 401 from Langfuse) are logged by the opentelemetry Python logger, not the plugin's debug log (#167). Run Hermes with PYTHONWARNINGS/logging at WARNING for opentelemetry.exporter to see them, or export one span with the SDK directly to confirm the URL and keys.

"Spans show up but without message content"

  • Check capture_previews — if it's false, the plugin is suppressing input.value / output.value at the source.
  • Remember: by default input.value is just the latest user turn. To capture the full conversation history, enable conversation capture.

"Self-hosted Langfuse won't start"

  • ClickHouse needs ~60 seconds to come up. The plugin will show connection refused errors until it's ready. Wait and retry.