CoreTests: CLI-based test suite selection with benchmarking#793
Merged
Conversation
Replace commented-out test selection with dictionary-based CLI dispatch. Each test suite is now selectable via command-line args with configurable parameters (NpcNames, iteration count, delay, GPU/CPU, DXGI/WGC). Changes: - Add suite dictionary with Action<string[]> for extensible dispatch - Add global flags: --log-times, --no-gpu, --no-log-update, --dxgi, --delay - Add percentile stats (p50/p95/p99) to --log-times output - Add PPather suite with expansion arg - Parameterize LogEachUpdate in Test_NpcNameFinder - Change screen field to IWowScreen to support DXGI/WGC toggle - Fix NpcNames.ToStringF() to handle combined flags via ToString() - Add build.ps1 and run.ps1 helper scripts - Add CoreTests/CLAUDE.md with usage documentation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Xian55
added
refactor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.\run.ps1 <suite> [args])--log-times,--no-gpu,--no-log-update,--dxgi,--delay <ms>--log-timesoutputbuild.ps1/run.ps1helper scripts andCoreTests/CLAUDE.mddocumentationNpcNames.ToStringF()to handle combined flagsTest position
stable image
Benchmark Commands
All 8 runs used
npc friendly neutral 1000with--log-times --no-log-update:Raw Log Output (
names.log)1/8: DXGI + GPU + delay 10
2/8: WGC + GPU + delay 10
3/8: DXGI + no-GPU + delay 10
4/8: WGC + no-GPU + delay 10
5/8: WGC + GPU + delay 150
6/8: WGC + GPU + delay 100
7/8: WGC + GPU + delay 15
8/8: WGC + GPU + delay 1
Analysis
WGC vs DXGI (GPU enabled, delay 10)
WGC is massively faster — DXGI screen capture is the bottleneck (~5ms per frame vs near-zero for WGC).
GPU vs CPU (WGC, delay 10)
GPU provides consistency — similar median performance but significantly fewer latency spikes at p95/p99.
Delay Impact (WGC, GPU)
Lower delay = more consistent — delay 150/100 have massive p99 spikes (~28ms) likely from frame timing boundaries, while delay 15/1 stay under 1ms at p99. Delay 15 is the sweet spot.
Test plan
dotnet build -c Releasecompiles with 0 errors.\run.ps1shows available suites.\run.ps1 npc friendly neutral 1000runs with correct args.\run.ps1 --log-times --no-log-update npc 1000shows percentile stats.\run.ps1 --dxgi npc 100uses DXGI capture.\run.ps1 --no-gpu npc 100uses CPU pathRound 2: GPU Utilization Monitoring (nvidia-smi)
Same 8 configurations re-run with
nvidia-smi dmon -s u -d 1sampling GPU utilization every second during each benchmark. Hardware: NVIDIA GeForce GTX 1070.Monitoring Script
GPU utilization was sampled using
nvidia-smi dmonrunning in background during each benchmark:Commands (identical to Round 1)
Combined Results: Timing + GPU Utilization
Raw Log Output (
names.log- Round 2)1/8: DXGI + GPU + delay 10
2/8: WGC + GPU + delay 10
3/8: DXGI + no-GPU + delay 10
4/8: WGC + no-GPU + delay 10
5/8: WGC + GPU + delay 150
6/8: WGC + GPU + delay 100
7/8: WGC + GPU + delay 15
8/8: WGC + GPU + delay 1
GPU Utilization Analysis
DXGI vs WGC GPU impact:
GPU compute shader impact:
Delay vs GPU utilization:
🤖 Generated with Claude Code