Nemo Routernemorouter.aiWhy We're Managed, Not a Self-Hosted Proxy
The same LLM-gateway features exist as an open-source proxy you can run yourself. We chose to run it for you instead. Here is the reasoning — and when self-hosting is genuinely the right call.
NemoRouter is built on LiteLLM, which is open source — meaning you could run the proxy yourself, for free, today. We chose to build a managed service on top of it anyway. That might look like we're charging for something you can get for nothing, so it's worth being direct about what "managed" actually does, what self-hosting really costs, and when running it yourself is the better choice. We'd rather you make an informed decision than a default one.
What "free to self-host" leaves out
The proxy is free. The operation of it is not:
| Self-hosting hands you | Ongoing cost |
|---|---|
| Scaling + availability | Capacity planning, load balancing, on-call |
| Upgrades | Tracking releases, testing bumps, migrating schemas |
| The database | Postgres ops, backups, the ledger invariants |
| Security | Patching, secret management, tenant isolation |
| The governance layer | Budgets, RBAC, audit — much of which you'd build |
"Free" is the license. The bill is paid in engineer-hours, forever. For most teams the managed fee is far cheaper than the salaried time self-hosting consumes — that's the trade a managed service exists to offer.
What managed actually means here
Managed isn't "we host the same thing and charge you." It's that we own the operational surface so you own none of it:
- We run the proxy — scaling, availability, the in-process mount, upgrades.
- We run the database — one Postgres of record, backups, the credit ledger.
- We provision providers — no BYOK; new models appear because we onboarded them.
- We built the governance — budgets, guardrails, RBAC, observability, all there on day one, on every tier.
You get a key and credits; the gateway is someone else's pager.
The real question isn't cost — it's where your engineers should be
A self-hosted proxy is rarely expensive in dollars; it's expensive in attention. Every hour spent upgrading a gateway, debugging a migration, or hardening a port is an hour not spent on your product. Managed is a bet that your engineers are worth more building your thing than operating ours.
When self-hosting is the right call (honestly)
Managed isn't universally correct, and pretending it is would be the kind of marketing we don't do. Self-host when:
- You have hard data-residency or air-gap requirements that a managed service can't meet.
- You have deep platform expertise and want total control of the routing internals.
- You operate at a scale where running it yourself is genuinely cheaper than any fee, and you already have the ops muscle.
- You have large pre-negotiated provider contracts you must draw down directly (which also points away from our no-BYOK model).
If that's you, the open-source proxy is right there, and we mean that — we build on it and read its source. Managed is for the (much larger) set of teams for whom operating a gateway is a distraction from their actual product.
The takeaway
"You can self-host it for free" is true and incomplete: the license is free, the operation isn't, and the operation is paid in your engineers' attention indefinitely. We chose managed because for most teams, a predictable fee beats an open-ended operational burden — and because it lets us offer one key, one bill, and a full governance layer on day one. When you have hard residency needs or genuine scale-and-expertise, self-host; otherwise, let the gateway be our pager, not yours.