Logs, alerts, and callbacks for every LLM call. See what happened, get notified when things break, pipe data to your existing tools. Cost is the provider’s number — never ours, never reconstructed.
Latest request
Financial records retained indefinitely
Provider header — never reconstructed
Per model, per key, per org
Per-org policy: zero / meta / full / redacted
Every request is visible, every integration is a toggle, and every cost number comes straight from the provider header — no reconciliation drift.
Every LLM call captured with model, status, latency, cost, tokens, and full prompt/response (if your data policy allows). Search by request ID, model, status, or time range.
Pipe logs to Langfuse, Datadog, S3, Slack, or custom HTTP endpoints. One toggle per provider. Each callback receives structured JSON — you build your own dashboards in your existing stack.
Percentile latency calculated per model, per virtual key, per team, per org. See where your tail latency lives, set SLO alerts, identify the model + provider mix that hits your budget.
LLM errors, budget thresholds, provider outages, latency spikes, rate-limit hits, guardrail blocks, key expirations, spend anomalies. Each toggleable per org, multi-channel delivery.
Choose what is logged per org: nothing, metadata only (default), full prompts and responses, or full-with-PII-redaction. Switch at any time — applies immediately to future requests.
Log ingestion happens after the response is returned to the caller. Callbacks to external services are also processed asynchronously — they never block the request path.
Cost is read from the x-nemo-response-cost header on the provider response — the safety invariant for analytics, billing, and the credit-ledger reconcile job. Logs are recorded, callbacks fan out async.
Async log path
Provider
Vertex / Anthropic / OpenAI
Returns x-nemo-response-cost header — the source of truth.
Nemo Intelligent Proxy Router
In-process ASGI
Computes cost, attaches metrics, emits log event.
Nemo Backend
:8090 — FastAPI
Records request_log, fans out to enabled callbacks async.
Callbacks
Langfuse · Datadog · S3 · Slack
Fire-and-forget delivery with retry queue.
The Nemo Intelligent Proxy Router runs in-process inside Nemo Backend — there is no separate gateway service to harden, no extra hop where the cost header can be lost or rewritten.
We never compute cost ourselves. The Nemo Intelligent Proxy Router owns cost; we read x-nemo-response-cost off the response and write it to the request_log + the credit ledger in the same transaction. If the header is missing, the row is flagged for reconcile — never silently zeroed.
Cost integrity
The same x-nemo-response-cost value lands in: (1) the request_log row visible in the dashboard, (2) the credit_transactions ledger entry, (3) any active callbacks (Langfuse, Datadog, S3), and (4) the spend-by-tag analytics aggregation. There is one upstream value and four downstream consumers — no recomputation, no reconciliation drift.
Cost-header integrity
Per-org control
Choose exactly what gets stored per organization. Every level is available on every plan. Change your policy at any time from the dashboard — updates apply to all future requests immediately. PII redaction is powered by Microsoft Presidio and runs pre-storage.
Zero logging
Nothing stored beyond billing-essential token + cost data.
Metadata only
Model, tokens, cost, latency, status, request ID. The default.
Full logging
Complete prompt + response stored. 90-day retention.
PII-redacted
Full logging with Presidio masking pre-storage.
Enable callbacks with a single toggle and start receiving structured log data in seconds. Each callback receives JSON with full request metadata: model, tokens, cost, latency, status.
Open-source LLM observability. Send traces, spans, and generations for prompt engineering, cost tracking, and quality evaluation. Full trace context preserved.
Enterprise APM and monitoring. Correlate LLM request metrics with your existing infrastructure dashboards, set up monitors, track SLOs alongside application metrics.
Archive raw request and response logs to S3 for long-term storage, compliance audits, and batch analytics. Configure retention policies and lifecycle rules.
Real-time alert notifications in your team channels. Errors, budget thresholds, provider outages, and guardrail blocks where your team already communicates.
Per-user email digests and Microsoft Teams channel notifications. Same alert types, same multi-channel delivery semantics.
Send structured JSON to any HTTPS endpoint with signed payloads. Build custom integrations with internal tools, data warehouses, or workflow automation.
90-day retention · 4 callbacks · 8 alerts
Sign up, enable a callback or two, watch the data flow into your existing stack. No agents to install, no schema to invent.