Embeddings and chat, one key
The same endpoint serves the embeddings call that builds your index and the chat call that synthesizes the answer — zero provider config.
Use Case · RAG
Retrieval-augmented generation, one key for embed and chat.
A RAG pipeline calls two model families — embeddings to index and query, a chat model to synthesize the answer. Route both through one Nemo Router endpoint, cache the repeats, and track cost per stage.
rag-pipeline · request trace
One pipeline, two model families
Index embedgemini-embedding
Query embedgemini-embedding
Synthesisgemini-2.5-flash
Cachehit · 0 ms
Stages taggedindex · query
Provider confignone
embed + chatcachedcost-tracked
- Model families
- Embed + chat
- Repeat queries
- Cached
- Cost attribution
- Per stage
- Catalog
- 97+
Both on one OpenAI-compatible endpoint
Exact-match hits skip the provider call
Index vs. query, split by metadata tag
models on Anthropic, Google & OpenAI
Why Nemo for RAG
The four things a RAG pipeline needs
Two model families, repetitive traffic, real cost pressure, and providers that occasionally fail. Nemo Router handles all four behind one key.
Caching for repetitive traffic
FAQs re-asked, identical context windows — exact-match repeats are served from cache and skip the provider call entirely. Override with nemo_cache: false.
Cost per pipeline stage
Tag index vs. query traffic in request metadata and the dashboard attributes real per-call cost to each stage of the pipeline.
Failover keeps retrieval answering
A degraded embedding or chat provider triggers the next fallback link transparently — your index build finishes and the query path keeps answering.
How it works
A RAG request, end to end
Index once, then query: embed the question, retrieve context from your own vector store, and synthesize with a chat model. Nemo sits on the two LLM hops — embeddings and synthesis — and logs the cost of each.
RAG pipeline flow
Index documents
POST /v1/embeddings
Chunk + embed your corpus once; store vectors in your DB.
Query embedding
POST /v1/embeddings
Embed the user question with the same model.
Retrieve context
your vector store
Nearest-neighbour search runs in your own database.
Synthesize answer
POST /v1/chat/completions
Chat model answers from retrieved context — cached if repeated.
Settled + logged
cost per stage
Embed cost, chat cost, cache hit — all in the request log.
Nearest-neighbour search stays in your database. Nemo Router handles the two LLM hops — embeddings and synthesis — with caching, failover, and per-stage cost tracking.
The code
Same client for embeddings and chat
A RAG pipeline is just two endpoint calls against one key. These snippets come straight from the SDK examples the playground and dashboard use — set NEMOROUTER_API_KEY and the chat call runs as-is; the embeddings call uses the same client and base URL.
Install
pip install openai| 1 | # Cache: enabled (org default). Pass nemo_cache: false to skip. |
| 2 | from openai import OpenAI |
| 3 | import os |
| 4 | |
| 5 | client = OpenAI( |
| 6 | api_key=os.environ["NEMOROUTER_API_KEY"], |
| 7 | base_url="https://api.nemorouter.ai/v1", |
| 8 | ) |
| 9 | |
| 10 | response = client.chat.completions.create( |
| 11 | model="gemini-2.5-flash-lite", |
| 12 | temperature=1, |
| 13 | max_tokens=1024, |
| 14 | top_p=1, |
| 15 | messages=[ |
| 16 | {"role": "user", "content": "Hello! What models do you support?"}, |
| 17 | ], |
| 18 | extra_body={ |
| 19 | # "nemo_cache": False, # Uncomment to skip cache |
| 20 | }, |
| 21 | ) |
| 22 | |
| 23 | print(response.choices[0].message.content) |
The same client object also calls client.embeddings.create() — one key covers the whole pipeline.
FAQ
Common RAG questions
One key for the whole pipeline
Ship a RAG pipeline without juggling providers
Embeddings, chat, caching, and per-stage cost tracking — all behind one Nemo Router key. Every feature is unlocked on every plan.
Building autonomous workflows on top of retrieval? See the AI agents use case.