LiteLLMOpen source AI gateway for 100+ LLMs. Self-hosted. Call any LLM in the OpenAI format.

The route is the prefix.

01openai/gpt-4oStraight to OpenAI. Same call shape as every other provider.

02anthropic/claude-sonnet-4-20250514Straight to Anthropic. Swap providers without rewriting your code.

03litellm_proxy/claude-sonnet-4-5Through your AI Gateway. A model group on your proxy; drop the prefix to call the provider directly.

Model strings from the project’s README examples.

Use it as a Python SDK, or deploy the AI Gateway for your team with virtual keys, spend tracking, guardrails, load balancing and an admin dashboard. Models, MCP tools and agents can all go through the same gateway.

Run the AI Gateway View source

Models

One interface for 100+ LLM providers, in the OpenAI format.

from litellm import completion

completion(model="openai/gpt-4o", messages=[...])
completion(model="anthropic/claude-sonnet-4-20250514", messages=[...])

Tools

Add an MCP server to the gateway, then call its tools through /chat/completions.

"tools": [{
  "type": "mcp",
  "server_url": "litellm_proxy/mcp/github",
  "server_label": "github_mcp"
}]

Agents

Run Claude Code, Codex, OpenCode or Deep Agents on any model.

litellm.agent(
    Harness.CLAUDE_CODE,
    "Find why tests/test_router.py is flaky and fix it.",
    sandbox=sandbox.local("./repo"),
    model="litellm_proxy/claude-sonnet-4-5",
)

With the gateway set, every model call the agent makes goes through it, tagged harness,claude_code. A2A agents get their own route too: /a2a/my-agent.

Endpoints

/chat/completions /responses /embeddings /images /audio /batches /rerank /a2a /messages

SDK

uv add litellm — then call any provider with completion().

Gateway

uv tool install 'litellm[proxy]' and litellm --model gpt-4o; point any OpenAI client at it.