The wedge claim: NemoRouter is the only LLM gateway that gives every customer all enterprise features — guardrails, A/B tests, prompt management, evals, budgets — free for life, with 2,000+ models behind one API key. Tiers vary the platform fee (4% / 2% / 0%); they never lock features.

If you typed "llm routing strategies" (or "how to choose an LLM router", or "llm router comparison") into Google in 2026, you are almost certainly past the "do we need an LLM gateway at all" question and into the harder one: which routing-decision shape fits the way our team actually wants to ship LLM traffic?

The market has answered the first question with eight different gateways and counting — we wrote the broader gateway-buyer's guide for that surface in /blog/product/llm-gateway-buyers-guide-2026. This post is for the buyer one level down: you've shortlisted two or three routing-decision products — likely some combination of Unify AI's benchmark-driven dynamic router, Not Diamond's ML-trained per-query classifier, and an AI-native LLM gateway with operator-controlled routing — and you need a decision tree across the three intelligence shapes before you commit your LLM traffic.

The honest summary in one paragraph: there are exactly three durable shapes the routing intelligence takes in 2026, and the buyer's question is which shape's failure modes you'd rather own. Benchmark-anchored routers (Unify-style) anchor decisions on published benchmark rankings + operator- supplied quality thresholds — failure mode: the benchmark stops reflecting your traffic. ML-classifier routers (NotDiamond-style) train a per-query classifier on operator-supplied response data — failure mode: the classifier is opaque to read and re-trains on a vendor cadence. Operator-controlled routers (NemoRouter-style) put the routing rule, the A/B variants, and the fallback chain in code the operator writes — failure mode: someone on your team has to pick. This guide names the trade-offs explicitly and gives you a buyer-stage decision tree at the end.

This is NOT a competitor disparagement post. Unify AI and Not Diamond are both real products with credible theses — we ship a paired vertical head-to- head against each (/blog/product/unify-ai-alternative and /blog/product/notdiamond-alternative) and both posts have a long "when this competitor is genuinely the right call" section. This guide sits one level up: it's the buyer-stage decision tree across the routing-intelligence-shape axis itself. Pick the shape that fits your team's mental model first, then pick the product on that shape.

The three routing-intelligence shapes (with a decision-shape table)

A routing intelligence is whatever process decides — for each LLM call you make — which upstream model and provider serves the call. The process can take three durable shapes. Each shape buys you different observability, different failure modes, and a different mental model for how the routing decision relates to your team's code.

Routing-intelligence shape	Where the decision rule lives	What changes the decision	What's observable to the operator	Where this shape ships today
Benchmark-anchored	Vendor-published benchmarks + operator-supplied quality thresholds	Benchmark updates + operator threshold changes	Benchmark rankings + threshold config + per-call decision logs (per vendor)	Unify AI
ML-classifier-anchored	A trained classifier the vendor ships (and the operator can fine-tune on their own data)	Classifier re-training cadence + operator-supplied fine-tune data	Classifier inputs + classifier confidence + per-call decision logs (per vendor)	Not Diamond
Operator-controlled	Routing rules + A/B variants + fallback chains the operator writes in code (or in an operator-controlled gateway config)	Operator commits + variant edits	Every input to the decision + every output (rules are code the operator reads)	NemoRouter + every AI-native LLM gateway where the operator pins models — OpenRouter, Portkey, LiteLLM, Helicone, etc.

Two structural things follow from this table that the rest of this guide leans on:

The shapes are not strict substitutes. A benchmark-anchored router does not give you per-query classifier accuracy, and an ML-classifier router does not give you observable-from-code routing rules. The decision is which failure mode you'd rather own — which is a team- mental-model question, not a feature-comparison question.
You CAN stack shapes. Operator-controlled routers can call benchmark-anchored or ML-classifier routers as upstream models if the team explicitly wants that — the operator-controlled layer is the substrate. The reverse is harder: a vendor-owned routing-decision layer cannot easily delegate the rule to operator code without flipping its product scope.

A buyer choosing across the axis is really answering one question: "Do I want my routing rule to live in a vendor's intelligence surface, or in my own code?"

Shape 1: benchmark-anchored routing (Unify-style)

A benchmark-anchored router decides the routing call by combining vendor- curated benchmark rankings (LiveBench, MMLU, public cost-quality tradeoff curves) with operator-supplied quality thresholds. The operator picks a quality floor; the router picks the cheapest model on the vendor's benchmark surface that clears it.

Buyer's mental model: I trust published benchmark rankings to be a reasonable static proxy for my traffic, and I'd rather configure a quality floor than write per-model rules.

What this shape buys you:

Bench-grade decision rule. The decision rule is "use the cheapest model above this benchmark threshold" — well-defined, explainable, and stable across vendor releases.
Low operator setup cost. You set a threshold, you ship — no per-route rule writing, no fallback chain to maintain.
Decision logs you can read. Each routing call shows which benchmark drove the choice and which threshold cleared.

What this shape costs you:

Benchmark drift risk. Public benchmarks lag the model-release cycle and rarely reflect your traffic distribution. A benchmark-anchored router's decision quality is bounded by how well the benchmark surface generalizes to your prompts.
No per-query adaptation. The same threshold + benchmark rule applies to every query; if your traffic has clusters (cheap chat + expensive coding + multilingual), the static rule undershoots adaptation an ML-classifier or operator-written ruleset would catch.
Vendor dependency on the benchmark surface. If the vendor stops curating new benchmarks (or curates them in a direction that doesn't match your needs), your decision quality degrades silently.

Pick this shape if: your traffic is broad-distribution (no obvious clusters that demand per-segment rules), your buying motion explicitly wants a single quality knob, and you trust the vendor's benchmark curation to stay aligned with your traffic over the next 12 months.

Shape 2: ML-classifier-anchored routing (NotDiamond-style)

An ML-classifier router trains a per-query classifier on response data — vendor-supplied at first, optionally fine-tuned on the operator's own traffic — and routes each new query through the classifier to pick the model most likely to give the best response.

Buyer's mental model: My traffic has structure the vendor can learn from. I want the routing decision to adapt per-query based on a model trained on actual response quality, not on a static benchmark.

What this shape buys you:

Per-query adaptation. The classifier sees the query and routes accordingly — different prompts get different upstreams without operator-written rules.
Fine-tunability on your data. If your traffic is large enough and your response-quality signal is clean enough, you can re-train the classifier on your own data and lift decision quality above the off- the-shelf surface.
Vendor roadmap focused on routing intelligence. When the entire vendor product surface is the classifier, every roadmap dollar lands on routing-intelligence improvements (more router-model versions, broader supported-provider lists, more fine-tune capability).

What this shape costs you:

Opaque decision rule. The classifier picks; you tune via training data. You cannot read the routing rule in code — only the inputs and the outputs.
Data + analytics cost to evaluate. Routing-decision quality on your traffic requires the operator to run their own continuous evaluation (which models the classifier picks, vs the model that would have been best per ground truth or downstream signal). Without that loop, you cannot tell whether the classifier is helping or hurting on your distribution.
Vendor cadence dependency. Classifier versions ship on the vendor's release cadence. A new classifier release can shift decisions on your traffic in ways the operator did not initiate.
Governance still has to live somewhere. A specialized routing-decision layer typically delegates the broader LLM governance surface — guardrails, prompt-template versioning, operator-controlled A/B tests, per-team budgets, virtual keys — back to the operator's own stack.

Pick this shape if: your traffic has enough volume + signal density to fine-tune the classifier on your own data, your buying motion is specifically "the routing decision itself is the lever," and you're comfortable delegating the broader LLM governance surface to a separate downstream tool.

Shape 3: operator-controlled routing (NemoRouter-style + every AI-native gateway where the operator pins models)

An operator-controlled router puts the routing rule, the A/B variants, and the fallback chain in code the operator writes (or in an operator- controlled gateway config the operator reads and edits). Every input to the routing decision and every output of it are observable to the operator — no opaque classifier, no static benchmark surface owns the decision.

Buyer's mental model: My team will write the routing rule. I want to read it in code, version it, A/B test variants against each other on my own terms, and own the failure mode.

What this shape buys you:

Observable from code. The routing rule is text the operator wrote (or config the operator reads). You can audit it, diff it, A/B test variants, and trace any routing decision back to a specific commit.
Native multi-tenancy + per-team budget enforcement. Operator- controlled routers in AI-native LLM gateways ship per-team budget enforcement, per-customer virtual keys, and prompt-template versioning as first-class governance surfaces alongside the routing rule.
No vendor cadence dependency on the rule. Your rule changes when you commit. No classifier upgrade and no benchmark refresh changes how your traffic gets routed without your explicit edit.
Stacking. If you ever want to call a benchmark-anchored or ML-classifier router from inside an operator-controlled router (as an upstream "smart routing" provider), you can — the operator-controlled substrate makes the inversion easy.

What this shape costs you:

Someone on your team has to pick. Operator-controlled routing means the operator writes the rule. If your team has no opinion about which model fits which prompt — and no appetite to develop one — the cost of writing the rule lands on you, not the vendor.
Less out-of-the-box "smart routing." Operator-controlled routers do not ship per-query ML adaptation or benchmark-anchored quality thresholds as the default routing decision (you can stack one in front if you want; the substrate doesn't pick the rule for you).
Setup is a few minutes longer than a quality-threshold knob. Pinning models, defining A/B variants, and setting fallback chains is more work than entering a single benchmark threshold — though typically less work than fine-tuning an ML classifier on your traffic.

Pick this shape if: your team wants the routing decision observable in code, you value LLM governance breadth (guardrails, prompt management, evals, per-team budgets, virtual keys) live on day one alongside the routing rule, and you'd rather own the failure mode of "we picked the wrong rule" than the failure mode of "the vendor's intelligence surface drifted off our traffic."

Buyer-stage decision tree (the punchline)

Here's the explicit decision tree. Answer the three questions in order; each answer narrows to one shape. None of the answers is wrong — they just sort buyers into the shape whose failure mode their team is best equipped to own.

Question 1 — Where do you want the routing rule to live?

In a vendor's intelligence surface (benchmark or classifier). → Shapes 1 or 2. Skip to Question 2.
In code my team writes. → Shape 3 — operator-controlled. Done. The rest of the buyer's-guide work is then picking the AI-native LLM gateway whose operator-controlled routing surface fits your stack — see /blog/product/llm-gateway-buyers-guide-2026 for the broader gateway-buyer's surface and /blog/product/feature-gating-audit-portkey-litellm-helicone for the feature-gating axis.

Question 2 — Do you want the routing rule to adapt per query?

No — one quality threshold across all traffic is fine. → Shape 1 — benchmark-anchored. Unify AI is the clean exemplar of this shape; see /blog/product/unify-ai-alternative for the head-to-head against an operator-controlled gateway.
Yes — I want a per-query decision tuned on my traffic. → Shape 2 — ML-classifier-anchored. Continue to Question 3 to confirm.

Question 3 (for Shape 2 candidates only) — Do you have enough traffic volume + response-quality signal to fine-tune the classifier on your own data AND a separate downstream tool stack you're happy delegating governance to?

Yes to both. → Shape 2 — ML-classifier-anchored. Not Diamond is the clean exemplar; see /blog/product/notdiamond-alternative for the head-to-head against an operator-controlled gateway.
Either is no. → reconsider Shape 3 — operator-controlled. Without volume + clean signal, fine-tuning the classifier on your data does not lift quality above the off-the-shelf surface, and a specialized routing- decision layer that delegates governance assumes you already have a working governance stack. If you don't, the operator-controlled shape ships the routing rule AND the governance surface together — typically less integration cost than wiring a classifier router on top of a separately assembled governance stack.

There is no fourth shape coming. The three above carve the routing- intelligence axis cleanly — vendor-owned static (benchmark) vs vendor- owned learned (classifier) vs operator-owned written (rule). Any new routing product in 2026 lands on one of the three.

Where NemoRouter fits (and where it doesn't)

NemoRouter is an operator-controlled router with deep LLM governance baked in. The wedge claim above is verbatim from our public README: every governance feature — guardrails, A/B tests, prompt management, evals, budgets — free for life, on every tier, with 2,000+ models behind one API key. Tiers vary the platform fee (4% on Tier 1 PAYG, 2% on Tier 2 monthly, 0% on Tier 3 annual prepay) — they never lock features.

Where NemoRouter explicitly does not compete:

Benchmark-anchored routing. NemoRouter does not ship a benchmark- curation surface that picks models off LiveBench / MMLU rankings against operator quality thresholds — that is Unify's product, and a team specifically buying Shape 1 should evaluate Unify directly.
ML-classifier-anchored routing. NemoRouter does not ship an ML-trained per-query classifier whose decision rule is opaque to read in code — that is Not Diamond's product, and a team specifically buying Shape 2 should evaluate Not Diamond directly.

Where NemoRouter is structurally distinct:

The full LLM governance surface alongside operator-controlled routing. Guardrails, prompt management, evals, A/B tests, per-team budgets, virtual keys ship on Tier 1 from signup. No plan upgrade unlocks them; tiers vary only the platform fee + RPM/TPM.
2,000+ models behind one API key. OpenAI-compatible endpoint. One base URL, one key, one client library, every model.
The reservation-arbitrage margin engine. Long-term margin comes from buying provider-side reservations (Azure OpenAI PTU, GCP GSU / Committed Use Discounts, AWS Bedrock Provisioned Throughput) at the aggregated customer-volume level and earning the spread vs. retail PAYG — annual reservations save up to 70%, monthly up to 30% per provider published documentation. That spread funds the "all features free" wedge; the platform fee is intentionally not the long-term margin engine.

Pricing — the routing-intelligence-shape framing

We are not publishing a transcribed dollar-for-dollar comparison across Unify AI and Not Diamond here, because both vendors' plan structure prices the routing-decision usage itself (per-call + per-plan-tier rows that ship on each vendor's release cadence). The honest comparison if you're an existing customer of either is: take your current routing-product plan cost + the upstream LLM provider bill + the operational cost of whatever tooling currently covers your guardrails / prompt management / evals / per-team budgets, and compare against the equivalent on NemoRouter's 4% / 2% / 0% platform-fee curve where all of that ships in one surface.

The NemoRouter pricing tiers:

Tier	Price	Platform Fee	RPM	TPM	Best for
Tier 1 — PAYG	$0	4%	500	500K	Trying NemoRouter; under $2.5k/mo of LLM spend
Tier 2	$100/mo min	2%	500	500K	$2.5k–$10k/mo spend, ready to commit monthly
Tier 3	$1,200/yr min	0%	1,000	1M	$10k+/mo spend, annual budget approved
Enterprise	Custom	0%	Custom	Custom	F1000, BAA, SOC2-prep, multi-region

Tier 1 is real. No card required, $5 in API credits auto-granted on signup. Enough to wire a guardrail, define an operator-controlled A/B test variant across two named models, and run a prompt template before you decide anything.
Tier 3 is the acquisition priority by design. Annual prepay funds the next annual reservation cycle; the spread compounds. See /blog/product/payg-to-tier3-annual-prepay for the breakeven math.
The breakeven math is short. At Tier 1's 4%, every $2,500/mo of LLM spend = $100/mo platform fee, which is the Tier 2 minimum. Past $2.5k/mo, Tier 2's 2% saves you money the moment you cross. Tier 3 starts paying back vs. Tier 2 around $10k/mo of annualized spend.

Switch cost across the three shapes

Migration across the three shapes follows the same pattern: the call shape is OpenAI-compatible on the major routers on each shape, so the SDK-level migration is one base URL + one API key. The substantive migration cost is conceptual, not code: when you switch shapes, your team's mental model for "where the routing rule lives" changes.

Shape 1 → Shape 3. You take back the quality threshold and write per-model rules in code. The benchmark threshold becomes your A/B test variant. Setup cost: a few minutes longer than the threshold-knob version.
Shape 2 → Shape 3. You take back the per-query routing decision and pin models / define operator-controlled A/B variants / set fallback chains. The classifier's opaque pick becomes a rule the operator writes. Tradeoff: you lose the per-query adaptation (unless you stack a Shape-2 router as an upstream) and gain a rule that's observable from code. Setup cost: a few minutes for the routing config; the bigger lift is the conceptual shift from "classifier picks" to "operator picks."
Shape 3 → Shape 1 or 2. You delegate the routing rule to a vendor- owned intelligence surface. The substrate is still operator-controlled (the gateway holds the call), but the decision rule is now the vendor's.

NemoRouter targets sub-60-second signup-to-first-call.

Try it (the only CTA)

Tier 1 is free. No card, no commitment, $5 in API credits auto-granted on signup. You can be making real model calls — through a guardrail, against a prompt template, with an operator-defined A/B test variant assigned across two named models — in under 60 seconds.

→ Start free at nemorouter.ai/signup

Past $10k/mo of LLM spend and weighing the routing-intelligence-shape decision against your team's mental model? The 0% Tier 3 walk-through is a 30-minute call — bring your current routing-product plan (if any), your LLM provider bill, and we'll do the breakeven math live + an honest discussion of which routing-intelligence shape fits your team's mental model best.

Sources

NemoRouter README — Business Model section + canonical pricing tiers
- 2,000+ models claim + provisioned-capacity 70%/30% reservation discounts + operator-controlled routing as the substrate.
Unify AI — official Unify product page and docs. https://unify.ai, https://docs.unify.ai/, https://unify.ai/pricing. Defer all dollar numbers and per-plan feature gating language to Unify's published pages on the date of this audit.
Not Diamond — official Not Diamond product page and docs. https://www.notdiamond.ai, https://docs.notdiamond.ai/, https://www.notdiamond.ai/pricing. Defer all dollar numbers and per- plan feature gating language to Not Diamond's published pages on the date of this audit.
Azure OpenAI Provisioned Throughput (PTU) — https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/provisioned-throughput.
Google Cloud Vertex AI generative-AI pricing (GSU / Committed Use Discounts) — https://cloud.google.com/vertex-ai/generative-ai/pricing.
AWS Bedrock Provisioned Throughput — https://aws.amazon.com/bedrock/pricing/.

Last verified: 2026-06-09. Trademark notice: Unify and Unify AI are trademarks of Unify Technologies Ltd. (and/or its affiliates). Not Diamond and NotDiamond are trademarks of Not Diamond, Inc. (and/or its affiliates). NemoRouter is not affiliated with or endorsed by either. All Unify and Not Diamond claims defer to each vendor's own published docs and pricing pages on the dates linked in the footnotes.

LLM routing strategies 2026: benchmark-anchored vs ML-classifier vs operator-controlled — how to pick the routing intelligence that fits your team

The three routing-intelligence shapes (with a decision-shape table)

Shape 1: benchmark-anchored routing (Unify-style)

Shape 2: ML-classifier-anchored routing (NotDiamond-style)

Shape 3: operator-controlled routing (NemoRouter-style + every AI-native gateway where the operator pins models)

Buyer-stage decision tree (the punchline)

Where NemoRouter fits (and where it doesn't)

Pricing — the routing-intelligence-shape framing

Switch cost across the three shapes

See also

Try it (the only CTA)

Sources

More from Buyer's Guide

LLM gateway buyer's guide 2026: routing, guardrails, evals, prompt management