[Bug]: /v1/responses silently fake-streams Azure deployments with custom names, base_model is ignored by the streaming capability check

[rmabon]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

We route an Azure OpenAI provisioned (PTU) deployment through the LiteLLM proxy. Like most PTU deployments it has a custom deployment name (gpt-5.4-csvo-ptu), so the model entry declares base_model for cost tracking, the setup recommended in https://docs.litellm.ai/docs/proxy/custom_pricing

stream: true requests to /v1/responses for this model return an SSE response, but no event arrives until the entire completion is finished: every event is delivered in a single burst at the end, and the deltas are synthetic 5-character chunks. Azure's diagnostic logs show the underlying call as a non-streaming request, so LiteLLM silently dropped stream=true and synthesized the events afterwards. Time-to-first-token equals total latency, which caused a production latency incident for us on 2026-08-18

Expected: stream=true is forwarded to Azure. LiteLLM cannot know the capabilities of an arbitrary deployment name, but the entry's base_model, which LiteLLM already trusts for cost calculation, identifies the underlying model, and that model streams

Why it happens, verified against v1.92.0 and current litellm_internal_staging: the Responses API decides native vs fake streaming by looking up only the routed model name in the model cost map (supports_native_streaming), and converts "model not found" into "does not support streaming"; model_info.base_model is never consulted. The router registration that could bridge the gap writes deployment metadata under a double-prefixed key (azure/azure/gpt-5.4-csvo-ptu) whenever litellm_params.custom_llm_provider is set explicitly, so the capability lookup can never hit it

Reproduction condition: litellm_params.custom_llm_provider set explicitly, which is the shape DB-managed deployments typically have (our rows are created via /model/new with it set). Omitting it happens to register a usable cost-map key for the deployment name, which masks the bug in minimal configs. api_version does not matter, reproduced identically on 2025-04-01-preview and v1

On duplicates: the same defect was reported as #21090 (identical call chain, vLLM-served custom models, tool events silently dropped) and auto-closed as stale without a fix. This report adds the Azure base_model angle, the router-registration root cause, the exact reproduction condition and measured evidence. The wider pattern of capability gates resolving against the deployment name while only cost tracking consults base_model also produced #31243 (reasoning_effort gate, closed but its fix PR unmerged per the comments), #12744 (o3 vs dumbo_o3, streaming differed with Base Model set, stale-closed while reproducible), #27717 and #28782

User Flow

Before a (hypothetical) fix: a developer streaming from a PTU-backed model gets no first-token latency benefit

1. They send POST https://litellm-domain/v1/responses with {"model": "gpt-5.4-csvo-ptu", "stream": true, "input": "..."}
2. The connection stays silent for the entire generation, then all SSE events arrive at once: for a 60 second answer that is 60 seconds with no output
3. They check the provider side and the Azure diagnostic logs show the request as a non-streaming call even though they asked for a stream

After a (hypothetical) fix: the same request streams for real

1. They send the same POST https://litellm-domain/v1/responses with {"model": "gpt-5.4-csvo-ptu", "stream": true, "input": "..."}
2. Delta events start arriving within the model's normal first-token latency and continue progressively
3. The Azure diagnostic logs show the request as a streaming call

Proof the bug occurs

Config the proxy ran with (the Azure resource holds a deployment named gpt-5.4-csvo-ptu backed by the gpt-5.4 model; env vars are the standard Azure service principal set, values redacted as secrets):

model_list:
  - model_name: gpt-5.4-csvo-ptu
    litellm_params:
      model: azure/gpt-5.4-csvo-ptu
      custom_llm_provider: azure
      api_base: os.environ/AZURE_API_BASE
      api_version: "2025-04-01-preview"
      tenant_id: os.environ/AZURE_TENANT_ID
      client_id: os.environ/AZURE_CLIENT_ID
      client_secret: os.environ/AZURE_CLIENT_SECRET
    model_info:
      base_model: azure/gpt-5.4
general_settings:
  master_key: sk-proof-1234

Version or commit: litellm_internal_staging ff02d5cfc0

Commands and their output:

litellm --config proof.yaml --port 41234

START=$EPOCHREALTIME
curl -sN http://127.0.0.1:41234/v1/responses \
  -H "Authorization: Bearer sk-proof-1234" -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.4-csvo-ptu","input":"Write a numbered list counting from 1 to 150, one number per line. No other text.","stream":true}' |
while IFS= read -r line; do [ -n "$line" ] && echo "$EPOCHREALTIME ${line:0:60}"; done > run.log
awk -v t0="$START" '/output_text.delta/{n++; if(!d){d=1; fd=$1-t0}} {last=$1-t0} END{printf "events %d | text deltas %d | first delta at %.2fs | last event at %.2fs\n", NR, n, fd, last}' run.log

events 108 | text deltas 99 | first delta at 3.47s | last event at 3.48s

run.log, first two lines, first delta and last line: the whole stream lands in one 10ms window after the full generation

1787300206.421098 data: {"type":"response.created","response":{"id":"resp_7cVH
1787300206.421935 data: {"type":"response.in_progress","response":{"id":"resp_
1787300206.422492 data: {"type":"response.output_text.delta","item_id":"msg_08
1787300206.429131 data: [DONE]

For comparison, the same commit plus a patch that lets the capability lookup fall back to base_model gives, for the same request: events 308 | text deltas 299 | first delta at 2.21s | last event at 7.57s, native token deltas arriving progressively

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

reproduced on v1.92.0 and current litellm_internal_staging (ff02d5c, 2026-08-21)