Auto-failover for long runs
Agents fan out dozens of calls per task. A provider hiccup at step 11 triggers the next fallback link transparently — step 12 never knows, and the failover is logged.
Use Case · AI Agents
Agents that stay up and stay in budget.
An autonomous agent fans out dozens of LLM calls per task. Nemo Router keeps a long run alive when a provider degrades, caps each agent with a per-key budget, and logs every tool call for audit.
agent-run · key sk-nemo-7f2a
One agent, one key, one budget
Steps this run24 calls
Fallback usedstep 11
Run statuscompleted
Budget$2.40 / $5.00
Tool calls logged24 / 24
Runaway protection402 ceiling
per-agent keyfailover-safeaudited
- Long runs
- Stay up
- Per-agent spend
- Budgeted
- Tool calls
- 100% logged
- Catalog
- 97+
Fallback chains survive a provider blip
A 402 stops a runaway loop
Every step in the request log
models on Anthropic, Google & OpenAI
Why Nemo for agents
Reliability, spend control, and a trail
Agents are unpredictable by design. The gateway makes them dependable: a run that survives a provider blip, a budget it cannot exceed, and a log of everything it did.
Per-agent budgets
One virtual key per agent with its own budget. A runaway reasoning loop hits a hard 402 ceiling instead of burning credits; per-key RPM/TPM throttles a misbehaving agent.
Every tool call audited
Planning, tool selection, reflection — every LLM call lands in the request log with model, latency, cost, and result. Filter by the agent’s key to replay a whole run.
Model choice per step
Strong reasoning model for planning, fast model for tool-argument extraction — pick per call from one catalog, no provider account, no SDK swap.
How it works
An agent run, end to end
Each agent carries its own virtual key. Every planning and tool-selection step is a separate LLM call — routed, failover-protected, budget-checked, and logged.
Agent run flow
Agent key
sk-nemo-... · per agent
One virtual key per agent — its own budget and rate limit.
Plan + tool calls
many /chat/completions
Each reasoning + tool-selection step is a separate call.
Routing + failover
fallback chain
A degraded provider triggers the next link — the run survives.
Budget check
reserve + settle
Out of budget → 402, before the call. No runaway spend.
Logged per step
request log
Model, latency, cost, result — replay the whole run.
The budget check is a reserve-then-settle: credits are reserved before the call and settled at the real cost after. If an agent is out of budget, it gets a 402 — never a surprise overspend.
The code
Point your agent framework at one endpoint
Nemo Router speaks the OpenAI API, so any agent framework that targets the OpenAI SDK works with a base-URL and key change. These snippets are generated from the same SDK examples the playground uses — give each agent its own key and the per-key budget does the rest.
Install
pip install openai| 1 | # Cache: enabled (org default). Pass nemo_cache: false to skip. |
| 2 | from openai import OpenAI |
| 3 | import os |
| 4 | |
| 5 | client = OpenAI( |
| 6 | api_key=os.environ["NEMOROUTER_API_KEY"], |
| 7 | base_url="https://api.nemorouter.ai/v1", |
| 8 | ) |
| 9 | |
| 10 | response = client.chat.completions.create( |
| 11 | model="gemini-2.5-flash-lite", |
| 12 | temperature=1, |
| 13 | max_tokens=1024, |
| 14 | top_p=1, |
| 15 | messages=[ |
| 16 | {"role": "user", "content": "Hello! What models do you support?"}, |
| 17 | ], |
| 18 | extra_body={ |
| 19 | # "nemo_cache": False, # Uncomment to skip cache |
| 20 | }, |
| 21 | ) |
| 22 | |
| 23 | print(response.choices[0].message.content) |
Issue one virtual key per agent — spend, rate limits, and the request log all scope to that key.
FAQ
Common agent questions
Reliable, bounded, auditable
Run agents on a gateway built for long runs
Auto-failover, per-agent budgets, and a full tool-call log — unlocked on every plan, behind one Nemo Router key.