You could run your own LLM gateway — there are capable options out there, and for some teams that's genuinely the right call. We chose to build a managed service instead. So it's worth being direct about what "managed" actually does, what self-hosting really costs, and when running it yourself is the better choice. We'd rather you make an informed decision than a default one.
What "run it yourself" leaves out
Standing up a gateway is the easy part. The operation of it is not:
| Self-hosting hands you | Ongoing cost |
|---|---|
| Scaling + availability | Capacity planning, load balancing, on-call |
| Upgrades | Tracking releases, testing every bump, keeping current |
| The database | Storage ops, backups, keeping the spend ledger correct |
| Security | Patching, secret management, tenant isolation |
| The governance layer | Budgets, RBAC, audit — much of which you'd build |
For most teams the managed fee is far cheaper than the salaried engineer-time self-hosting consumes — that's the trade a managed service exists to offer.
What managed actually means here
Managed isn't "we host the same thing and charge you." It's that we own the operational surface so you own none of it:
- We run the gateway — scaling, availability, upgrades, keeping everything current and reliable.
- We run the data layer — one system of record, backups, the credit ledger.
- We provision providers — no BYOK; new models appear because we onboarded them.
- We built the governance — budgets, guardrails, RBAC, observability, all there on day one, on every tier.
You get a key and credits; the gateway is someone else's pager.
The real question isn't cost — it's where your engineers should be
Running a gateway is rarely expensive in dollars; it's expensive in attention. Every hour spent upgrading it, debugging a migration, or hardening infrastructure is an hour not spent on your product. Managed is a bet that your engineers are worth more building your thing than operating ours.
When self-hosting is the right call (honestly)
Managed isn't universally correct, and pretending it is would be the kind of marketing we don't do. Self-host when:
- You have hard data-residency or air-gap requirements that a managed service can't meet.
- You have deep platform expertise and want total control of the routing internals.
- You operate at a scale where running it yourself is genuinely cheaper than any fee, and you already have the ops muscle.
- You have large pre-negotiated provider contracts you must draw down directly (which also points away from our no-BYOK model).
If that's you, self-hosting is a legitimate choice, and we mean that. Managed is for the (much larger) set of teams for whom operating a gateway is a distraction from their actual product.
The takeaway
"You can run it yourself" is true and incomplete: standing it up is the easy part, the operation isn't, and the operation is paid in your engineers' attention indefinitely. We chose managed because for most teams, a predictable fee beats an open-ended operational burden — and because it lets us offer one key, one bill, and a full governance layer on day one. When you have hard residency needs or genuine scale-and-expertise, self-host; otherwise, let the gateway be our pager, not yours.
