WebSearch Interception
Relevant source files
The following files were used as context for generating this wiki page:
WebSearch Interception is a server-side architectural feature in LiteLLM designed to provide web search tool execution for Large Language Models (LLMs) that do not natively support search capabilities, such as Amazon Bedrock or specific Anthropic Claude versions. It enables a transparent "agentic loop" on the server, allowing a single client-side API call to return a final answer enriched with real-time search results.
Sources: litellm/integrations/websearch_interception/ARCHITECTURE.md:3-8, CLAUDE.md:33-40
Architecture and Tool Standardization
The system relies on a standardized tool definition, litellm_web_search, which serves as the common denominator for various native provider search formats. When LiteLLM receives a search request from a client (like the Claude Code CLI), it intercepts the provider-specific tool and converts it into this standard format before communicating with the underlying model.
Standard Tool Definition
The standard tool is defined with a specific input schema to ensure consistency across different search providers (e.g., Tavily, Perplexity).
| Field | Value | Description |
|---|---|---|
| Tool Name | litellm_web_search | The unified tool identifier. |
| Constant | LITELLM_WEB_SEARCH_TOOL_NAME | Defined in litellm/constants.py. |
| Input Property | query (string) | The search string generated by the model. |
Sources: litellm/integrations/websearch_interception/ARCHITECTURE.md:12-25
Supported Tool Formats
The interception system detects multiple search tool variants using specific string matching and prefix checks.
| Format | Detection Method | Provider Context |
|---|---|---|
| LiteLLM Standard | Direct name match | Any |
| Anthropic Native | startswith("web_search_") | Bedrock, Claude API |
| Claude Code CLI | Name + Type check | Anthropic Claude Code |
| Legacy | name="WebSearch" | Custom / Backwards Compatibility |
Sources: litellm/integrations/websearch_interception/ARCHITECTURE.md:27-35
Request Flow and Agentic Loop
The core value of the interception feature is moving the complexity of tool execution from the client to the server. Without interception, a client must manually handle tool calls and results across multiple round trips. With interception, LiteLLM manages the loop internally.
Comparison of Request Logic
- Client-Side (Without Interception): Requires at least two API calls. The client receives a
tool_useblock, executes the search locally, and sends the results back to the LLM. - Server-Side (With Interception): Requires only one call from the user. LiteLLM intercepts the
tool_usefrom the model, executeslitellm.asearch(), injects thetool_result, and requests the final response from the model before returning it to the user.
Sequence Diagram: Server-Side Interception
The following diagram illustrates the automated loop handled by the WebSearchInterceptionLogger.
Internal Logic and Conversions
Native Tool Conversion
Native tools are intercepted before the API call to the provider. In the Anthropic message handler, the system detects native types like web_search_20250305 and converts them to litellm_web_search. This prevents the provider from attempting to execute the search natively, which would result in an invalid_tool_input error on models that do not support it.
Sources: litellm/integrations/websearch_interception/ARCHITECTURE.md:52-67
Streaming to Non-Streaming Conversion
If a user initiates a request with stream=True while using a WebSearch tool, LiteLLM automatically converts the request to non-streaming (stream=False). This is necessary because the server-side agentic loop must consume the full model response to detect and execute the search tool before returning the final content to the user.
Sources: litellm/integrations/websearch_interception/ARCHITECTURE.md:121-135
Search Results and Annotations
When search results are processed, LiteLLM supports returning specific metadata and citations. For instance, in OpenAI-compatible search models (like gpt-4o-search-preview), the response includes annotations and url_citation blocks.
Response Data Structure
Responses containing search data are validated for specific fields to ensure the user can trace the information source.
| Field | Type | Description |
|---|---|---|
url_citation | Object | Contains url, start_index, and end_index. |
annotations | List | A collection of citations within the message content. |
Sources: tests/llm_translation/test_openai.py:228-245
Search Provider Selection
The WebSearchInterceptionLogger identifies which search provider to use based on the following hierarchy:
- Specified Name: If a
search_tool_nameis provided in the configuration, it looks it up in thellm_router.search_tools. - Router Tool: If no name is provided, it uses the first available tool in the router's
search_toolsconfiguration. - Default Fallback: If no router or tools are configured, it defaults to
perplexity.
Sources: litellm/integrations/websearch_interception/ARCHITECTURE.md:102-110, litellm/integrations/websearch_interception/ARCHITECTURE.md:141-150
Implementation Components
| Component | Responsibility |
|---|---|
WebSearchInterceptionLogger | Implements the agentic loop hooks (async_run_agentic_loop). |
tools.py | Handles standardization and detection of search tool formats. |
transformation.py | Detects tool_use blocks and builds tool_result messages. |
llm_http_handler.py | Orchestrates the agentic completion hooks after a response is received. |
Sources: litellm/integrations/websearch_interception/ARCHITECTURE.md:102-118
Coding Agent Integration
The system is optimized for coding agents. Documentation suggests that agents like Codex or Claude Code should prioritize high-speed search tools like rg (ripgrep) for local file searches, while using the WebSearch interception for external data beyond their knowledge cutoff.
Sources: litellm/llms/chatgpt/common_utils.py:18-24, litellm/integrations/websearch_interception/ARCHITECTURE.md:38-50
Summary
WebSearch Interception provides a seamless mechanism for enabling search capabilities on non-native models by centralizing tool execution within the LiteLLM proxy. By standardizing tool formats and managing the agentic loop on the server, it reduces client complexity and ensures compatibility with various search providers while supporting detailed citations and annotations in the final response.
Sources: litellm/integrations/websearch_interception/ARCHITECTURE.md:3-10, tests/llm_translation/test_openai.py:247-255