MCP (Model Context Protocol) Support
Relevant source files
The following files were used as context for generating this wiki page:
Model Context Protocol (MCP) support in LiteLLM provides a standardized way to integrate external tools, data sources, and contextual information into agentic workflows. LiteLLM acts as both an MCP client, capable of connecting to external servers for tool execution, and an MCP gateway through its proxy implementation. This architecture allows LLMs to interact with third-party services like Zapier, Jira, and ScrapeGraph through a unified interface.
Sources: AGENTS.md, mcp_servers.json
Architecture Overview
The MCP implementation consists of three primary layers: the Experimental MCP Client, the LiteLLM Proxy MCP Server, and the Semantic Filter hook. The client handles connections to external servers using various transports (such as SSE), while the proxy layer exposes these tools to the LLM dashboard and API consumers.
The diagram shows the flow of a tool-calling request from an LLM through the LiteLLM Proxy to external MCP servers.
Sources: litellm/proxy/hooks/mcp_semantic_filter/ARCHITECTURE.md, litellm/experimental_mcp_client/client.py
Key Components
Experimental MCP Client
The LiteLLM_MCP_Client is responsible for managing sessions with external MCP servers. It supports session initialization, tool listing, and tool calling. It utilizes mcp.ClientSession and handles transport-level communication, specifically supporting Server-Sent Events (SSE).
Key Functions:
connect(): Establishes a session with the remote MCP server and performs initialization.get_tools(): Retrieves the list of tools available on the remote server.call_tool(name, arguments): Executes a specific tool on the remote server and returns the result.
Sources: litellm/experimental_mcp_client/client.py:9-70
MCP Semantic Filter
The MCP Semantic Filter acts as a proxy hook that intercepts requests to determine if they should be enriched with MCP tool definitions. It uses a separate LLM call to perform "semantic filtering," selecting only the most relevant tools for a given user query to avoid context window bloat.
This sequence illustrates how the Semantic Filter reduces the number of tools sent to the primary LLM by selecting relevant ones via a filtering LLM.
Sources: litellm/proxy/hooks/mcp_semantic_filter/ARCHITECTURE.md
MCP Server Management (Proxy)
The proxy implementation includes an experimental MCP server that can expose internal LiteLLM functionality as MCP tools. This server integrates with the LiteLLM dashboard to provide visibility into active MCP servers and their capabilities.
Sources: litellm/proxy/_experimental/mcp_server/server.py:1-25, ui/litellm-dashboard/src/components/public_model_hub.tsx:448-520
External Integrations
LiteLLM comes pre-configured with several well-known MCP servers. These are defined in mcp_servers.json and can be used for agentic workflows.
| Server Name | Protocol/URL | Description |
|---|---|---|
| Zapier | SSE: https://mcp.zapier.com/api/mcp | Integration with Zapier's automation platform. |
| DeepWiki | SSE: https://mcp.deepwiki.com/sse | Access to repository documentation and search. |
| Jira | SSE: https://mcp.atlassian.com/v1/sse | Programmatic access to Jira issue tracking. |
| Linear | SSE: https://mcp.linear.app/sse | Programmatic access to Linear project management. |
| ScrapeGraph | SSE: https://smithery.ai/api/mcp/scrapegraph-mcp/sse | Web scraping, crawling, and agentic workflows. |
Sources: mcp_servers.json:1-20
Data Models and Configuration
MCP Server Configuration (UI)
The LiteLLM dashboard tracks MCP server state, including authentication types and transport methods.
| Field | Type | Description |
|---|---|---|
server_name | String | The unique identifier for the MCP server. |
transport | String | Communication method (e.g., "sse", "http"). |
auth_type | String | Authentication method (e.g., "none", "bearer"). |
url | String | The endpoint for the MCP server. |
allowed_tools | List | A whitelist of tools exposed by this server. |
Sources: ui/litellm-dashboard/src/components/public_model_hub.tsx:75-88, ui/litellm-dashboard/src/components/AIHub/ModelHubTable.tsx:645-680
Tool Transformation Logic
LiteLLM transforms between OpenAI tool formats and MCP tool formats. When a tool is called via the LiteLLM_MCP_Client, the response is parsed from mcp.types.CallToolResult back into a format compatible with OpenAI's tool-call response structure.
# Internal transformation example
# MCP Tool Result -> LiteLLM/OpenAI Result
async def call_tool(self, name: str, arguments: dict):
result = await self.session.call_tool(name, arguments)
# result.content contains the tool output
return result
Sources: litellm/experimental_mcp_client/client.py:65-75, AGENTS.md
Summary
MCP support in LiteLLM enables complex agentic behavior by connecting LLMs to a wide array of external tools through a standardized protocol. By implementing a semantic filtering mechanism and supporting SSE-based transports, LiteLLM allows for efficient and scalable tool usage, even when dealing with hundreds of available functions from providers like Zapier or Jira. The integration into the Proxy and Dashboard ensures that these capabilities are manageable and visible to administrators and developers alike.
Sources: litellm/proxy/hooks/mcp_semantic_filter/ARCHITECTURE.md, AGENTS.md