[Bug]: Gemini thought signature is embedded in the tool call id, so any client that normalizes ids breaks the next turn

[JuakoTm]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

Not a duplicate of #25322

GitHub flagged #25322 as a possible duplicate. It is the opposite failure in the same area, and the two fixes do not overlap:

	#25322	This issue
Endpoint	/v1/messages (Anthropic format, Claude Code)	/v1/responses and /v1/chat/completions
What goes wrong	The signature is never captured, because Gemini put it on a separate thought part	The signature is captured and is sent back, but corrupted in transit
What Google says	Function call is missing a thought_signature in functionCall parts	Base64 decoding failed for "AY89a1/_57b05e78dc"
Symptom	The model degenerates: garbage replies, repetition loops	Hard 400, the session cannot continue

Worth noting the two interact: #25322 treats embedding the signature in tool_call_id as the correct mechanism and asks for it to be fed in more cases. Fixing #25322 alone would make this issue fire more often, not less, because more tool calls would end up carrying a multi-KB id that clients then normalize.

---

For Gemini models the proxy returns Google's thoughtSignature in two places:

• tool_calls[].provider_specific_fields.thought_signature
• appended to tool_calls[].id after a __thought__ separator

Neither is a key a generic OpenAI-compatible client knows about, and the second one is actively harmful: id is by convention a short identifier, so clients normalize it.

Measured against Vertex Gemini 3, the returned tool call id ranges from 843 to 4423 characters depending on the call. On /v1/responses the same blob is placed in both id and call_id.

When a client normalizes that id, the embedded signature is mangled. On the next turn the proxy finds the __thought__ separator, takes everything after it, and forwards it to Vertex as thoughtSignature without validating it. Google rejects the request:

Invalid value at 'contents[1].parts[0].thought_signature' (TYPE_BYTES),
Base64 decoding failed for "AY89a1/_57b05e78dc"

The same corruption produces a second, different error text when the mangled value happens to still be valid base64, which makes this easy to miss when grepping logs:

"message": "Invalid thought signature.", "status": "INVALID_ARGUMENT"

There is no way to opt out: checked v1.83.14-stable, v1.90.0, v1.96.0 and main, the embedding is an unconditional if thought_signature:.

Suggested fix

1. Also emit the signature at tool_calls[].thought_signature (top level). Additive, breaks nothing. It matters because clients already read that key: OpenClaw captures tool_calls[].thought_signature generically for any OpenAI-compatible provider, support that was added for Venice, which returns the signature at exactly that key (OpenClaw #119591 / #119783).

2. Read the same key on the way in. Today the lookup checks provider_specific_fields, the function's provider_specific_fields, and the id, but not the top level. Emitting without reading would only work in one direction.

3. Independently, validate before injecting. If the extracted value does not base64-decode, fall back to the dummy skip-validator signature instead of forwarding it. That alone turns a hard 400 into a graceful degradation, and it is a much smaller change than 1 and 2.

Item 3 is worth doing on its own: any client that normalizes the id today produces a user-visible error that the proxy could absorb.

User Flow

Before a (hypothetical) fix — the agent answers the first question, then every following turn fails and the conversation is dead.

1. An operator asks a Telegram assistant (built on OpenClaw) a question that requires a tool.
2. The client sends POST https://<proxy>/v1/responses with model: gemini-3.1-pro-preview and one function tool.
3. 200 comes back with a function_call item whose call_id is 1223 characters long: call_2156408__thought__AY89a1/Iab3+RVjKCabD+h7qYLWIf2BWZ9CZDSDoMZpQytG...
4. The client stores that call_id. Because it is an identifier, it normalizes it into something id-shaped, ending up with a 41 character value such as AY89a1_S3YvIpUCcBTFSgDfesRLDnA_775ff49bcd.
5. The client runs the tool and sends the next turn, echoing back the normalized call_id together with the function_call_output.
6. The proxy answers 400: Invalid value at 'contents[1].parts[0].thought_signature' (TYPE_BYTES), Base64 decoding failed for "AY89a1/_57b05e78dc".
7. The operator sees LLM request failed in the chat. Retrying does not help: the broken value is now part of the conversation history, so every following turn fails the same way and the session cannot progress.

After a (hypothetical) fix — the same conversation continues normally.

1. An operator asks the same Telegram assistant the same question that requires a tool.
2. The client sends the same POST https://<proxy>/v1/responses with model: gemini-3.1-pro-preview and one function tool.
3. 200 comes back with a function_call item whose call_id is short, such as call_2156408, and the signature arrives in its own field on the tool call instead of inside the id.
4. The client stores both. The call_id is already id-shaped, so normalizing it changes nothing, and the signature is kept separately and untouched.
5. The client runs the tool and sends the next turn, echoing back the short call_id, the function_call_output, and the signature exactly as received.
6. The proxy answers 200 and the model produces the final message.
7. The operator gets the answer and can keep asking follow-up questions in the same conversation.

Proof the bug occurs

Captured against a self-hosted LiteLLM proxy on Kubernetes, hitting real Vertex AI. Host and keys redacted, everything else verbatim.

Config / setup the proxy ran with:

model_list:
  - model_name: gemini-3.1-pro-preview
    litellm_params:
      model: vertex_ai/gemini-3.1-pro-preview
      vertex_project: <redacted>
      vertex_location: global
      vertex_credentials: <service account JSON, redacted>

Version or commit: v1.90.0. The same code path is present in v1.83.14-stable, v1.96.0 and main.

Commands and their full output:

1. /v1/responses, first turn: the returned call_id carries the signature

curl -s -X POST "$PROXY/v1/responses" -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d @turn1.json

turn1.json:

{"model":"gemini-3.1-pro-preview",
 "input":"What is the weather in Santiago? Use the tool.",
 "tools":[{"type":"function","name":"get_weather",
   "parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}],
 "tool_choice":"required"}

Output, trimmed to the relevant item:

{"status":"completed",
 "output":[{"type":"function_call",
   "id":"call_2156408__thought__AY89a1/Iab3+RVjKCabD+h7qYLWIf2BWZ9CZDSDoMZpQytG...",
   "call_id":"call_2156408__thought__AY89a1/Iab3+RVjKCabD+h7qYLWIf2BWZ9CZDSDoMZpQytG...",
   "name":"get_weather","arguments":"{\"city\": \"Santiago\"}"}]}

id and call_id are both 1223 characters.

2. /v1/responses, second turn, call_id echoed verbatim: works

Same request with input containing the function_call and a function_call_output, both using the 1223 character call_id unchanged.

HTTP 200 — "The weather in Santiago is 21 C."

3. /v1/responses, second turn, call_id normalized: fails

Identical to 2 with the call_id shortened the way an identifier normalizer would (sanitized[:30] + "_" + sha256hex[:10]), giving call_2156408__thought__AY89a1/_57b05e78dc:

HTTP 400
litellm.BadRequestError: Vertex_aiException BadRequestError - {
  "error": {
    "code": 400,
    "message": "Invalid value at 'contents[1].parts[0].thought_signature' (TYPE_BYTES), Base64 decoding failed for \"AY89a1/_57b05e78dc\"",
    "status": "INVALID_ARGUMENT",
    "details": [{"@type": "type.googleapis.com/google.rpc.BadRequest",
      "fieldViolations": [{"field": "contents[1].parts[0].thought_signature"}]}]
  }
}

4. /v1/chat/completions: same bug, cause is endpoint independent

First turn returns tool_calls[0].id of 859 characters (call_2254344__thought__AY89a19rfaQstNXhYomMSsTSpNLVRjsNCuBX+b2ZYUcVnGF...) plus provider_specific_fields.thought_signature of 836 characters.

Replaying the history with the id normalized to call_2158562__thought__AY89a1/_ee781c9832:

HTTP 400
litellm.BadRequestError: Vertex_aiException BadRequestError - {
  "error": {"code": 400,
    "message": "Invalid value at 'contents[1].parts[0].thought_signature' (TYPE_BYTES), Base64 decoding failed for \"AY89a1/_ee781c9832\""}
}

Scale, from the same proxy

One client, 19 failed requests over a month on this exact error, up to 22% of that client's calls in the worst hour, because once a corrupted signature enters the history every later turn fails.

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.90.0

Twitter / LinkedIn details

https://www.linkedin.com/in/joaquint/