Engineering
Mounting LiteLLM In-Process: One Fewer Network Hop
Running LiteLLM as a separate service adds a network hop, a port to harden, and a process to operate. Here is why NemoRouter mounts it in-process as an ASGI app — and the trade-off that came with it.
Nemo Team
9 min