A proxy for LLM traffic

Your agent keepsspeaking its ownAPI.

Switchyard is a Rust proxy and library that routes requests across providers and translates between the OpenAI Chat, Anthropic Messages, and OpenAI Responses formats. Point Claude Code or Codex at a local model and neither side has to learn the other's dialect.

Read the proxy github.com/NVIDIA-NeMo/Switchyard

What the translation touches Illustrative

Agent sends	Backend receives
POST /v1/messages Anthropic Messages	POST /v1/chat/completions OpenAI Chat
system: top level string	messages[0] with role system
stop_sequences: [ ]	stop: [ ]
content: block list	content: string or parts

Field shapes are illustrative. The project lists protocol translation across OpenAI Chat, Anthropic Messages, and OpenAI Responses as its first feature.

Routes to vLLM NVIDIA NIM Ollama any OpenAI-compatible endpoint

What the metrics cover

requests

errors

latency

tokens

routing overhead

Bar widths are illustrative. The five named series are the ones the README lists for its Prometheus metrics. A proxy that measures its own overhead is one you can argue with.

Maturity

The README calls this pre-alpha and experimental software, not for production use. The API and algorithms are expected to change before v1.0.

Protocol translation

Convert between OpenAI Chat, Anthropic Messages, and OpenAI Responses so the agent never changes clients.

Multi-backend routing

Random routing, LLM-as-classifier routing, a signal-driven stage router, or an algorithm you write yourself.

Operational metrics

Prometheus metrics cover requests, errors, latency, tokens, and the routing overhead the proxy itself adds.