litellm-ai-gateway panics on first outbound TLS connection — "no process-level CryptoProvider available" (rustls 0.23)

[GordonWei]

Summary

The standalone Rust gateway server (litellm-ai-gateway, server feature) panics on the
first outbound TLS connection to a provider (e.g. api.anthropic.com). The panic is caught
by the tokio runtime so the process stays alive, but the request that triggered it fails with
a generic 502.

Steps to Reproduce

1. Build litellm-ai-gateway from litellm-rust/ with the server feature enabled
2. Run it standalone (Mode 2, not embedded in the Python proxy) with any provider key configured
3. Send a request that requires an outbound HTTPS call (e.g. POST /v1/messages with an Anthropic key)

Actual Behavior

thread 'tokio-rt-worker' (7) panicked at rustls-0.23.42/src/crypto/mod.rs:249:14:
Could not automatically determine the process-level CryptoProvider from Rustls crate features.
Call CryptoProvider::install_default() before this point to select a provider manually, or make
sure exactly one of the 'aws-lc-rs' and 'ring' features is enabled.

The pod/process doesn't crash (panic is caught per-task), but the request fails — client sees
502 {"error":{"message":"messages provider request failed"}} instead of a real response
(or a real auth error, if the key is invalid).

Root Cause

litellm-rust/Cargo.lock currently resolves two rustls versions (0.21.12 and 0.23.42),
and both the ring and aws-lc-rs crypto backend crates are present in the dependency tree
(pulled in transitively by different dependencies). Nothing in the workspace calls
rustls::crypto::CryptoProvider::install_default(), so rustls has no way to pick a
process-wide default crypto provider — the first TLS handshake panics.

This is a well-known rustls 0.23+ footgun when a dependency tree ends up with multiple crypto
backends (see e.g. lightningdevkit/ldk-node#991, alloy-rs/alloy#874, openobserve/openobserve#7200
for the same failure mode in other projects).

Suggested Fix

Call CryptoProvider::install_default() once at process startup (before any TLS connection is
attempted), e.g.:

rustls::crypto::ring::default_provider()
    .install_default()
    .expect("failed to install rustls crypto provider");

or align the dependency tree so only one of ring/aws-lc-rs is enabled.

Environment

• Built from source at the workspace's current master as of 2026-08-21 (includes feat(rust): route /chat/completions through the Rust core for anthropic and bedrock #37241,
  routing /chat/completions through the Rust core)
• Deployed standalone (Mode 2) on GKE, tested with a placeholder Anthropic key