A SOC 2 Checklist for LLM Gateways

The moment your company's data flows through an LLM, your security team has questions — and so will your SOC 2 auditor. An LLM gateway concentrates exactly the things audits care about: access to sensitive data, who did what, how long it's kept, and whether one tenant can reach another's information. This checklist maps the controls a gateway should satisfy to the SOC 2 trust criteria, so you can evaluate one (or prepare your own) without starting from a blank page.

This is a practical engineering checklist, not legal advice — pair it with your auditor's guidance.

Access control (CC6)

Who can do what, and is it enforced?

• Role-based access — owner / admin / member / viewer with distinct permissions; writes require the right role.
• Least privilege for keys — management authority (master key) is separated from usage (virtual keys); customer traffic never uses the master key.
• Tenant isolation enforced at the database — not just the app. Row-level security so one org cannot read another's data even if app code has a bug. (See org/team scoping.)
• MFA available for privileged accounts.
• Key display is safe — full secret shown once, only last-4 thereafter.

Audit trail (CC7)

Can you reconstruct who did what, when?

• Admin actions are logged — key created/rotated/deleted, member added/removed, role changed, settings changed.
• Attribution is complete — each entry records the actor, the action, and the source (including client IP, consistently).
• Tamper-evidence — audit entries aren't editable through the normal app surface.

A gap auditors love to find: an action that's logged most of the time. Consistent attribution across every mutating route is the standard. (More: building an audit trail.)

Data retention & minimization (CC3, P)

Are you keeping only what you need, only as long as you need it?

• Configurable logging level — from metadata-only to full content, set per org. (See what to log.)
• PII redaction at write time — sensitive values never persist when redaction is on.
• Bounded retention on request logs — raw content expires; aggregates persist.
• Financial records retained per obligation, separately from prompt content.
• Deletion path — a documented way to honor data-subject deletion requests.

Encryption & transport (CC6)

• TLS everywhere — no plaintext transport.
• Secrets at rest — provider keys and credentials encrypted, not stored in plaintext config.
• Security headers — CSP, HSTS, X-Frame-Options on customer-facing surfaces.

Confidentiality & tenant isolation (C)

• No cross-tenant leakage — proven by tests, not assumed. A user in org A querying org B's data returns zero rows.
• Same-id discipline — one organization UUID end to end, so there's no mapping layer to get isolation wrong.
• Guardrails for sensitive content — PII detection and redaction inline on the request path.

Availability (A)

• Provider fallback — an upstream outage doesn't take you down (fallback chains).
• Spend safety can't be raced — caps hold under concurrency (reserve-and-settle), so a runaway can't both overspend and degrade service.
• Health checks + monitoring on the request path.

Change management (CC8)

• Tested changes — controls are covered by tests that run before merge.
• Reversible prompts/config — versioned prompts so a bad change rolls back without a deploy.

The takeaway

Routing data through LLMs doesn't invent new compliance categories — it concentrates the familiar ones. Work the checklist: role-based access, complete audit attribution, minimized and bounded logging, encryption in transit and at rest, database-enforced tenant isolation, and tested availability controls. A gateway that satisfies these turns "is our AI usage compliant" from an open question into a documented answer. See the trust and security pages for how NemoRouter maps to each.