Search results

Piecewise Cudagraph support for Gemma4 VSWA cache pools

CUDA Graph

Status: Open.

#14975 In NVIDIA/TensorRT-LLM;

· bmarimuthu-nv opened on Jun 4, 2026

[Feature]: Kernel factory and step3.7 optimizations

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Model optimization<NV>Model-specific performance optimizations and tuning<NV>Model-specific performance optimizations and tuning

Status: Open.

#14967 In NVIDIA/TensorRT-LLM;

· suyoggupta opened on Jun 4, 2026

[AutoDeploy]: Add Ultra model to btk

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14966 In NVIDIA/TensorRT-LLM;

· marinayanov opened on Jun 4, 2026

[AutoDeploy]: Add gpt-oss-120b model to btk

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14965 In NVIDIA/TensorRT-LLM;

· marinayanov opened on Jun 4, 2026

[Feature]: AutoDeploy vLLM prototype

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14963 In NVIDIA/TensorRT-LLM;

· suyoggupta opened on Jun 4, 2026

[auto-dev]: auto-dev wheel discovery improvements

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Infra<NV>automated tests, build checks, github actions, system stability & efficiency.<NV>automated tests, build checks, github actions, system stability & efficiency.

Status: Open.

#14951 In NVIDIA/TensorRT-LLM;

· galagam opened on Jun 4, 2026

[Bug]: Gemma 3 / Gemma 4 fail in TensorRT-LLM 1.3.0rc17 container on NVIDIA L4: FlashInfer unsupported architecture, kv_layout metadata error, AutoDeploy use_cache mismatch

bugSomething isn't workingSomething isn't working

Status: Open.

#14942 In NVIDIA/TensorRT-LLM;

· ylnhari opened on Jun 4, 2026

[RFC]: Semantic KV Cache Reuse

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

RFC

Status: Open.

#14918 In NVIDIA/TensorRT-LLM;

· zbennett10 opened on Jun 3, 2026

[Bug][Disagg] KvCacheAwareRouter silently degrades to load-only routing (per-server block table not populated; matched_tokens≡0) — ~90% vs 96% prefix-cache hit

Disaggregated serving<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

Status: Open.

#14882 In NVIDIA/TensorRT-LLM;

· lishicheng1996-nv opened on Jun 3, 2026

[Feature]: [AutoDeploy] GPT-OSS-120b perf analsys and opt (tp>=2)

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14874 In NVIDIA/TensorRT-LLM;

· taylor-yb-lee opened on Jun 2, 2026

[Feature]: AutoDeploy: DFlash speculative decoding support

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Speculative Decoding<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter

Status: Open.

#14843 In NVIDIA/TensorRT-LLM;

· govind-ramnarayan opened on Jun 2, 2026

[Feature]: Support encoder - prefill disagg in AutoDeploy for multimodal models

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

MultimodalLabel for issues & PRs regarding Multimodal related objectsLabel for issues & PRs regarding Multimodal related objects

Status: Open.

#14840 In NVIDIA/TensorRT-LLM;

· bmarimuthu-nv opened on Jun 2, 2026