[None][test] Add gpt-oss-120b eagle3 accuracy test

[JennyLiu-nv]

Summary

• To speed up testing, a gsm8k with 300 samples was used.
• Accuracy testing for spec dec was missing, and a related bug was encountered: https://nvbugspro.nvidia.com/bug/5875031
• Test results as follows (overall test time < 30m)

[06/05/2026-03:23:54] [TRT-LLM] [I] [executor] get signal from executor worker
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [00:01<00:00, 202.51it/s]
Submitting requests: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [00:02<00:00, 118.89it/s]
Fetching responses: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [16:17<00:00,  3.26s/it]
[06/05/2026-03:40:27] [TRT-LLM] [I] [evaluate] TRTLLM execution time: 980.517 seconds.
[06/05/2026-03:40:28] [TRT-LLM] [I] [evaluate] lm-eval gsm8k results (scores normalized to range 0~100):
|Tasks|Version|     Filter     |n-shot|  Metric   |   | Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|------:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |92.0000|±  |1.5689|
|     |       |strict-match    |     5|exact_match|↑  |61.6667|±  |2.8118|

[06/05/2026-03:40:28] [TRT-LLM] [I] [evaluate] lm-eval gsm8k exact_match,flexible-extract accuracy: 92.00
[06/05/2026-03:40:28] [TRT-LLM] [I] Hypothesis testing report:
===========================================================
= ACCURACY HYPOTHESIS TESTING
===========================================================
Alpha (Type I:  False Positive): 0.050
Beta  (Type II: False Negative): 0.200
Sigma (Standard deviation): 50.000
#Samples: 300
Higher is better: True
Theta (Minimum detectable effect): 10.151
Reference accuracy: 90.300
Threshold: 83.585
===========================================================
Evaluated accuracy: 92.000
===========================================================
[06/05/2026-03:40:28] [TRT-LLM] [I] [llmapi] Thread proxy_dispatch_result_thread stopped.
[06/05/2026-03:40:28] [TRT-LLM] [I] [llmapi] Thread await_response_thread stopped.
PASSED

Summary by CodeRabbit

• Tests
  • Added new accuracy test for single GPU Eagle3 speculative decoding with various backend configurations
  • Extended test suite coverage for Eagle3 performance validation

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 229e98bf-5ab0-41b8-93e6-c2e281f1b5c5

📥 Commits

Reviewing files that changed from the base of the PR and between 81e86a5 and e2877d0.

📒 Files selected for processing (2)

• tests/integration/defs/accuracy/test_llm_api_pytorch.py
• tests/integration/test_lists/qa/llm_spark_func.yml

---

📝 Walkthrough

Walkthrough

This PR adds a single-GPU accuracy test for Eagle3 speculative decoding that disables KV cache block reuse, configures drafting with max_draft_len=5, and evaluates GSM8K performance. The test is parameterized over MOE backend options and registered in the integration test list.

Changes

Eagle3 Single-GPU Accuracy Test

Layer / File(s)	Summary
Eagle3 single-GPU test implementation and registration tests/integration/defs/accuracy/test_llm_api_pytorch.py, tests/integration/test_lists/qa/llm_spark_func.yml	New test_eagle3_1gpu method patches GSM8K constants (MAX_OUTPUT_LEN, NUM_SAMPLES, EVALUATE_KWARGS), disables KV cache block reuse, configures Eagle3 speculative decoding with max_draft_len=5, instantiates LLM with enable_attention_dp=False and all parallel sizes set to 1, and evaluates GSM8K. The test is registered in llm_spark_func for single-GPU cases.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Suggested reviewers

• dongfengy
• litaotju
• StanleySun639

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description lacks required sections (Description and Test Coverage) from the template, though it includes test results and PR checklist.	Add explicit Description and Test Coverage sections explaining the motivation and which test safeguards the changes.
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change: adding an Eagle3 accuracy test for GPT-OSS-120B model on a single GPU.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[JennyLiu-nv]

/bot run

[tensorrt-cicd]

PR_Github #52260 [ run ] triggered by Bot. Commit: e2877d0 Link to invocation

[tensorrt-cicd]

PR_Github #52260 [ run ] completed with state SUCCESS. Commit: e2877d0
/LLM/main/L0_MergeRequest_PR pipeline #41573 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[JennyLiu-nv]

/bot run

[tensorrt-cicd]

PR_Github #52333 [ run ] triggered by Bot. Commit: e2877d0 Link to invocation

[tensorrt-cicd]

PR_Github #52333 [ run ] completed with state SUCCESS. Commit: e2877d0
/LLM/main/L0_MergeRequest_PR pipeline #41638 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation