[Bug]: Responses API streaming emits an unopened assistant message item for Anthropic tool calls with no text (breaks Vercel AI SDK)

[berlinaut]

What happened?

When an Anthropic-family model is called through the Responses API (POST /v1/responses) with streaming, and the model returns a tool call with no accompanying text, LiteLLM emits a synthesized assistant message item that is only ever closed, never opened:

response.output_item.done      id=chatcmpl-<uuid>    <-- never had output_item.added
response.content_part.done     id=chatcmpl-<uuid>    <-- never had content_part.added
response.output_text.done      id=chatcmpl-<uuid>    <-- never had any delta

Note the id prefix: chatcmpl-…, not the msg_… id used when the item is opened normally.

Any client that tracks output items by id rejects this. The Vercel AI SDK (ai, used by CopilotKit, OpenCode, and others) throws:

text part chatcmpl-<uuid> not found

which aborts the run before the tool result is delivered, so agentic tool-calling loops break on the first tool call: the tool executes client-side but the model never receives its result.

The same request returns a well-formed stream when the model happens to emit text alongside the tool call — the message item then gets a proper output_item.added + content_part.added with a msg_… id. So the trigger is specifically tool call with empty text content.

Relevant log output

Event sequence for a tool-only response (arguments deltas elided):

response.created
response.in_progress
response.output_item.added             id=tooluse_8G8H6h8nrt79WT4b8sHNd0
response.function_call_arguments.delta id=tooluse_8G8H6h8nrt79WT4b8sHNd0
response.function_call_arguments.done  id=tooluse_8G8H6h8nrt79WT4b8sHNd0
response.output_item.done              id=tooluse_8G8H6h8nrt79WT4b8sHNd0
response.output_text.done              id=chatcmpl-a83572f7-…   <-- ORPHAN
response.content_part.done             id=chatcmpl-a83572f7-…   <-- ORPHAN
response.output_item.done              id=chatcmpl-a83572f7-…   <-- ORPHAN
response.completed

response.completed also lists the phantom item in response.output:

{"type": "message", "id": "chatcmpl-…", "status": "completed", "role": "assistant"}

with content_part.done carrying {"type": "output_text", "text": ""} — i.e. an empty message that never should have been emitted at all.

For contrast, the identical request to deepseek-flash (natively OpenAI-shaped) produces a properly paired stream and works fine.

Steps to reproduce

curl -sN "$LITELLM_BASE_URL/responses" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "claude-sonnet-4-6",
    "stream": true,
    "input": "Search the web for floppy disks. Reply with ONLY the tool call. Do not write any text.",
    "tools": [{
      "type": "function",
      "name": "web_search",
      "description": "Search the web.",
      "parameters": {"type":"object","properties":{"query":{"type":"string"}},"required":["query"]}
    }]
  }' | grep -o '"type":"response\.[a-z_.]*"'

Then check that no response.output_item.added / response.content_part.added precedes the chatcmpl-… .done events.

The instruction to suppress text is only there to make it deterministic — in normal agent use the model reaches this state on its own whenever it decides to call a tool without commentary.

Affected models

Reproduced deterministically (2/2 passes each) on LiteLLM v1.96.2:

Model	custom_llm_provider	Orphan .done events
claude-sonnet-4-6	anthropic	3
anthropic.claude-sonnet-4-6	bedrock	3
anthropic.claude-haiku-4-5	bedrock	3
deepseek-flash	deepseek	0

So it affects both the Anthropic-direct and the Bedrock route.

Expected behaviour

Either emit the matching response.output_item.added and response.content_part.added for the message item, or do not emit the empty message item at all. Per the Responses API contract, every *.done must be preceded by its corresponding *.added for the same item_id.

Related

• [Bug]: LiteLLM Proxy - Responses API streaming omits necessary SSE event types #20975 — same class of bug (Responses API streaming omitting the setup events), reported for Azure and confirmed by others on github_copilot, ollama cloud and vLLM. It was auto-closed as stale rather than fixed; this report adds the Anthropic/Bedrock path and a narrower trigger.
• [Bug]: github_copilot /v1/responses streaming breaks Vercel AI SDK with "reasoning part <id> not found" / "text part <id> not found" #30071 — same symptom in the Vercel AI SDK (text part <id> not found), but a different cause (github_copilot tagging each event with a different item_id). That one was fixed; this is not the same code path.

Are you a ML Ops Team?

No

What LiteLLM version are you on?

v1.96.2

Twitter / LinkedIn details

No response