One API. 97+ models.
Call Anthropic (Claude), Google (Gemini), and OpenAI (GPT, o-series) live today — AWS Bedrock shipping next through a single endpoint. Cost, latency, and weighted routing, deterministic A/B splits, configurable fallback chains — config, not redeploy.
Live routing trace
- Routing strategies
- 3
- Retries
- Configurable
- A/B splits
- Deterministic
- Fallback chains
- Opt-in
Cost, latency, weighted — per org
Per-org retry count + per-request override
Model + prompt variants, retry-safe
On error + on rate limit, cross-provider
Three strategies. One config field.
Pick the signal your workload cares about — cost, latency, or your own weighted split. The router does the rest, per org, no redeploy.
costprice per tokenPicks the cheapest capable model in your candidate set — optimize the bill, not the benchmark.
latencyrecent p95Steers each request to the model with the lowest recent p95 latency for your org.
weightedyour splitCall nemo-auto and we split traffic across a basket of models by the weights you set.
A routing decision, end to end
Strategy match, primary provider, fallback chain, settle. A substitution is surfaced on the x-nemo-routed-by response header — routing itself adds < 1 ms.
Routing decision flow
Request
POST /v1/chat/completions
Bearer sk-nemo-..., model="gemini-2.5-flash"
Strategy match
cost · candidate set
Alias resolves to the cheapest capable model in your set.
Primary provider
Vertex AI · gemini-2.5-flash
5xx / timeout triggers the fallback chain.
Fallback chain
Anthropic → Bedrock → OpenAI
Ordered backup list per model; each link retried in turn.
Settled
cost + latency logged
Strategy, primary, fallback, retries — all in the request log.
Two-line migration from OpenAI
OpenAI-compatible
Change the base URL and the key — that’s it
The unmodified OpenAI SDK works against Nemo Router on every supported language. Streaming, tool use, JSON mode, embeddings, image generation — all proxied transparently.
- Python, Node.js, Go, Java, Ruby, PHP, cURL
- Base URL: https://api.nemorouter.ai/v1
- Auth: Bearer sk-nemo-...
- Drop-in for chat.completions, embeddings, and images
Two-line change
Full routing reference — strategies, fallbacks, retries, A/B
- Fallback chains
- Ordered backup list per model — on error and on rate limit. A 5xx or timeout triggers the next link, cross-provider (Vertex → Anthropic → Bedrock). Opt-in per org; the chain is configured, not coded.
- Retries + timeouts
- Per-org retry count (default 3) and request timeout, with a per-request override. Provider retry-after hints are honored on the retry.
- Deterministic A/B splits
- Split a model or prompt into variants and bucket traffic deterministically — the same request_id always lands on the same variant, so retries stay consistent and results are attributable.
- Never depend on one model
- A missing, unknown, or retired model resolves to your default (a weighted basket, your org default, or a platform default) instead of failing — the call still succeeds.
- OpenAI-compatible surface
- Same chat.completions, embeddings, and images endpoints. Base URL https://api.nemorouter.ai/v1, auth Bearer sk-nemo-... Streaming, tool use, and JSON mode proxied transparently.
- Live model catalog
- Capabilities and fallback chains live in the catalog, not your repo. New models are added within hours of provider launch — public read-only catalog at /api/public/models.
Common routing questions
One key. One bill. Every model.
Replace your homegrown failover layer in an afternoon
Sign up, paste your virtual key, change the base URL. Cost / latency / weighted routing, A/B splits, and fallback chains are unlocked on every plan.