Make `message_history` provider-valid out of the box (repair tool-call/result pairing)

Merged
DO
4mo ago
+1776-35

A run that is cancelled or crashes mid-tool — or a hand-built, truncated, or context-evicted history — can leave the message history with broken tool-call/tool-result pairing that strict providers (Anthropic, Gemini, OpenAI-compatible) reject on the next request: a tool call with no result (OpenAI 400 No tool output found for function call, the AG-UI abort-mid-tool repro in #3461) or a result with no call (OpenAI Responses No tool call found for function call output, #4882). Today every application that persists and resumes histories has to reinvent the same repair walk (see #5637), or worse, monkeypatch _clean_message_history to get such histories through.

This PR makes the history provider-valid out of the box: _clean_message_history in _agent_graph becomes an ordered pipeline of repair passes that run before every model request, so interrupted and hand-built histories can be passed straight back into a run. It resolves the canonical, maintainer-endorsed request in #4728 via the automatic-internal path DouweM called out there. Two earlier attempts implemented the same idea as a history processor (#5021) and then as a capability (#5056) — both closed; this PR lands the repair internally instead, per DouweM's direction. This is the "repair" half of #5637 implemented internally. The public is_provider_valid() validator proposed there is complementary, not obviated: repair fixes a history to send to a model, whereas a validator lets a caller judge a history and branch on it — e.g. snapshot/replay gating that must detect a crashed-mid-tool point and refuse to persist it as a resumable continuation, rather than silently synthesizing a tool return for it. That predicate remains a separate follow-up (#5637).

The guiding principle (DouweM): massage the history however we can to make the model API accept it, and drop only what's fundamentally unsendable — never silently removing content that was semantically meaningful. So the passes only ADD (synthesize) or REMOVE parts no provider could ever accept; nothing meaningful is dropped.

The pipeline

_clean_message_history(messages, *, repair_last_response) runs three pure list[ModelMessage] -> list[ModelMessage] passes in order over regular, locally-executed tool call/result pairing (order matters — matching-graph repairs before the merge that changes message boundaries). Native/builtin parts are left entirely untouched (see Scope):

  1. _drop_orphaned_tool_results (REMOVE) — a ToolReturnPart or tool-bound RetryPromptPart (in a request) whose tool_call_id never appeared as a regular ToolCallPart in any preceding message. No provider accepts a tool_result/function_call_output without its call. A RetryPromptPart with no tool_name is validation feedback (plain user text, not a tool result) and is left alone. Emptied interior request → dropped; emptied last request → kept empty so history still ends on a ModelRequest.
  2. _repair_dangling_tool_calls (ADD synthesized results) — the existing dangling-call pass. Every call with no result gets a synthesized ToolReturnPart ('The tool call was interrupted before a result was produced.', outcome='interrupted', metadata={'pydantic_ai_synthesized_tool_return': True}). This includes a call whose args were cut off mid-stream (unparsable JSON): the call is kept verbatim and closed out like any other dangling call — malformed args are already sendable (serializers degrade them via ToolCallPart.args_as_dict's INVALID_JSON wrapper, which the tool-call retry flow has always relied on), and removing the call would disturb the response's shape (e.g. leave a thinking-only response whose signature was computed over a turn that included the call). Frontier-gated by repair_last_response (below).
  3. _merge_consecutive_messages (normalize) — the existing consecutive-same-role merge, extracted unchanged; runs last, once pairing is valid, so it never separates a result from its call. Changes message boundaries but preserves all content.

Matching in passes 1 and 2 keys on the concrete regular ToolCallPart/ToolReturnPart types; NativeToolCallPart/NativeToolReturnPart are sibling Base*Part subclasses, so native parts are automatically ignored by every pass.

Matching is an ordered walk (per the sketch in #5637): a result only answers a call that is open — produced by an earlier response and not already answered — at that point. A result preceding its call, a duplicate, or a tool_call_id reused across responses therefore can't mask a genuinely dangling call, and an out-of-place result is recognized as an orphan.

Repair is silent. Like the existing consecutive-message merge in _clean_message_history, repair is quiet cleanup — it doesn't warn or raise. The synthesized ToolReturnPart's pydantic_ai_synthesized_tool_return metadata marker makes every repair inspectable in the resulting history, and a Logfire/OTel span attribute is the intended non-noisy observability surface as a follow-up. (A Python UserWarning was tried but, under the test suite's filterwarnings=["error"], it fired inside a legitimate internal flow — an after_model_request hook raising ModelRetry leaves the just-streamed tool call dangling until the wire-level pass synthesizes a return — so it cried wolf on the framework's own retry loops and was dropped.)

The live frontier is protected. The last ModelResponse's tool calls may still legitimately receive results, so repair only touches them when the interruption is evident:

  • History ending on a response with tool calls + no new prompt → run resumption executes them (unchanged). A trailing response ending on an unparsable-args call also stays intact on promptless resume, so local args validation still turns it into a RetryPromptPart the model can react to.
  • Deferred/approval flows: a run ending in DeferredToolRequests persists resp(A executed, B deferred) + req(return A) with a 'complete' trailing request — that shape is never repaired at run start (gated on ModelRequest.state == 'interrupted'), so deferred_tool_results matching keeps working and a forgotten-results resume doesn't close the approval-pending call in the persisted history. The wire-level pass (below) answers it on the copy sent to the model only.
  • History ending on a response with tool calls + a new user prompt → still raises Cannot provide a new user prompt when the message history contains unprocessed tool calls. for state='complete' responses, but for state='interrupted' responses (e.g. a cancelled stream) the calls will never be executed, so they are repaired instead of refused — the "cancel a stream, then continue the conversation" flow now just works.
  • A trailing ModelRequest with state='interrupted' (a run that crashed during tool execution, leaving partial tool returns) means the preceding response's unanswered calls can never be executed, so they are closed out at run start.
  • As a final safety net, the per-step wire-level cleaning (after history processors and Model.prepare_messages) repairs everything including the last response, since at that point the history is definitively being sent. These wire-level repairs are not persisted into run state.

Determinism, idempotency, cache stability

Synthesized parts derive their timestamp from the response they repair and contain no wall-clock or random data: repairing the same history always produces the same bytes, and re-running a repaired history is a no-op (pinned by tests), so repair never churns provider prompt-cache prefixes across turns.

Security interaction (GHSA-jpr8 / #6169)

The AG-UI sanitize_messages tail-strip removes unresolved trailing tool calls from client-supplied history so a forged call can't reach the promptless-resume execute path. Core repair composes with (and never undoes) that: it doesn't touch the trailing response on the execute paths, and for mid-history dangling calls it only answers them with a synthetic return — a repaired call is inert and can never execute, so repair strictly reduces what forged history can do.

Scope — reproduced-and-included vs. deliberately excluded

Every repair was validated against the provider rule and message shape it satisfies. Included (each with a focused test asserting the exact repaired history a capturing FunctionModel receives): dangling-call synthesize (including calls with mid-stream-truncated args) and orphaned-result drop (#4882/#4728).

Excluded, with reasons (left as follow-ups so the surface stays scoped):

  • Dangling builtin/native tool calls, incl. Anthropic code_execution (#6401) — investigated and excluded from the core pipeline. Native/builtin parts (NativeToolCallPart/NativeToolReturnPart) are fundamentally different from locally-executed tool calls: the provider produces the call and its result inline, they're co-located in one ModelResponse (and a tool-search result can even arrive in a later response — see models/anthropic.py _collect_orphan_tool_search_call_ids), and they're contractually preserved in all_messages() (test_xai_builtin_tool_without_tool_call_id pins that an empty-id native call stays in history while xAI's serializer skips it on the wire at models/xai.py:493). A generic per-message "dangling native call" drop in the core graph is both redundant (serializers already repair these — Anthropic repairs tool-search orphans) and harmful (it would surgically remove user-constructed content from persisted history, and would strand a legitimate tool-search call whose result lands in a later response). So #6401's Anthropic code_execution 400 should be fixed in the Anthropic model's serialization as a separate scoped change, not here. The pipeline leaves all native parts exactly where they are (regression test: a history with a dangling native call passes through _clean_message_history byte-identical).
  • Reorder out-of-place tool results (#5964, #4827) — investigated and excluded. Both are downstream serialization/mapping bugs, not history-shape bugs: in #5964 the ModelRequest.parts were already correctly ordered [ToolReturnPart, UserPromptPart] and the AG-UI adapter (_dump_request_parts) moved the user part ahead of the result (fixed by #5972); in #4827 the OpenAI mapping inserts the instructions system message between an assistant tool_calls message and its tool result because it counts all system messages including history-embedded ones. Neither is fixable by reordering parts within the history, and the existing merge already hoists tool results to the front of a merged request — so a history-level reorder pass would be redundant here and risk reordering user/text content. No clean reproduction at the history level → excluded (CLAUDE.md scope discipline).
  • Duplicate tool results for one call — a second tool_result for an already-answered call. Removing "the redundant one" is a which-is-meaningful judgment; left for the broader validity pass.
  • Provider-specific ordering rules beyond call/result pairing — part of #5637's fuller validity definition and the cross-model replay class.

Design decisions for review

  • Keep-and-synthesize (not drop) for unparsable-args calls: originally these were dropped, on #4053's premise that incomplete args must never reach a provider. That premise turned out to be false: ToolCallPart.args_as_dict deliberately degrades malformed JSON to {'INVALID_JSON': '<raw args>'} precisely so it "can still be sent to a model API (e.g. during a retry flow)", and the retry flow has always replayed such calls (a complete-but-invalid-args call answered by a RetryPromptPart round-trips to the provider today). Dropping was also silent surgery on the model's self-model (phantom-action references — deepagents' PatchToolCallsMiddleware synthesizes for this reason) and, worse, could leave a thinking-only response whose signature was computed over a turn that included the call — the exact signed-thinking risk that ruled out drop for regular dangling calls. Now every dangling call gets the same synthesized return, and the REMOVE set shrinks to exactly one thing: orphaned results, the one shape that cannot be truthfully repaired (the missing call's args are unknowable, so any synthesized call would fabricate an assistant action).
  • Marker: metadata={'pydantic_ai_synthesized_tool_return': True} on the synthesized ToolReturnPart, following the DEFERRED_CAPABILITY_TOOL_METADATA_KEY precedent (pydantic_ai_-prefixed key, module-level constant SYNTHESIZED_TOOL_RETURN_METADATA_KEY in _agent_graph). No new public API in this PR, so the key is documented as a literal in the message-history docs.
  • outcome='interrupted': a new neutral BaseToolReturnPart.outcome value — we don't know whether the tool ran, failed, or never started, so the synthesized return must not claim 'failed' (or success). It is never sent to any LLM provider (the content string carries the wording) and UI adapters render it neutrally rather than as an error. It also survives Vercel AI and AG-UI dump/load round-trips: Vercel carries the claim in the pydantic_ai callProviderMetadata channel (failed/denied already ride its typed part states), and AG-UI carries every non-success outcome — regular and native returns, previously all reloading as success — in the encrypted_value carrier alongside tool_kind (>= 0.1.11). Anything that can affect provider serialization must round-trip, or a reloaded history changes the request bytes and breaks prompt-cache prefix stability. Coordinated with #5585, which maps only outcome='failed' to provider native error channels, so 'interrupted' correctly stays out of them.
  • Wording: 'The tool call was interrupted before a result was produced.', styled after 'The tool call was denied.' (ToolDenied.message).

Open questions

  1. Should the UserError for a complete trailing response with unanswered tool calls + new prompt also be replaced by repair? Kept for now as an explicit-policy guard for deferred/approval flows (cf. urbantech's argument in #5836), at the cost of hand-built truncated histories without state='interrupted' still erroring in that one spot.
  2. Should SYNTHESIZED_TOOL_RETURN_METADATA_KEY (and/or an is_provider_valid() companion, #5637) be exposed publicly in a follow-up?

Checklist

  • Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
  • No breaking changes in accordance with the version policy.
  • PR title is fit for the release changelog.