fastapi

Mounting LiteLLM In-Process: One Fewer Network Hop

Running LiteLLM as a separate service adds a network hop, a port to harden, and a process to operate. Here is why NemoRouter mounts it in-process as an ASGI app — and the trade-off that came with it.