Skip to content

Add E2E coverage for profile delegate source - #537

Merged
Yuandi (DDKinger) merged 1 commit into
mainfrom
dev/DDKinger/it-delegate-source
Aug 4, 2026
Merged

Add E2E coverage for profile delegate source#537
Yuandi (DDKinger) merged 1 commit into
mainfrom
dev/DDKinger/it-delegate-source

Conversation

@DDKinger

@DDKinger Yuandi (DDKinger) commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds test/e2e/tests/Feature.DelegateSource.Tests.ps1 covering PR #488's strict delegate source routing — the direct counterpart to #486 for #481.

#488 is now merged, so this branch is rebased onto main and its diff is test-only.

Cases (environment-gated: needs a runnable default WSL distro)

  1. An explicit WSL delegate source never falls back to the Windows host — the regression Add profile-scoped command palette agents #488 fixes. A random never-installed agent name deterministically forces the "agent unavailable in WSL" branch; the pane must show bash's own not found.
  2. The default delegate source is never diverted to WSL — guards the removed active-pane WSL inference; the pane must show the real Windows spawn failure.

TerminalPage.cpp's profile resolution (commandPaletteAgent--delegate-source/--delegate-wsl-distro) is only reachable via non-injectable UI (Alt+Shift+B, the Alt+Shift+/ palette, ?<prompt>), so — exactly like Feature.Delegate — these cases drive the delegate engine directly with the flags it now always passes.

Validation

  • Post-merge main build → both cases pass; Invoke-ItE2EReport.ps1 -UpdateReport credits C245 as [x], C244 stays manual.
  • Pre-fix proof: run against a wta.exe built before Add profile-scoped command palette agents #488 (no --delegate-source flag), case 1 fails with AUTOMATION FAILED rather than passing vacuously — confirmed twice, once deliberately on a main-built binary and once accidentally against a stale deployment.

Checklist

New Profile command palette agent section in doc/release-check-list.md, mirroring #481's entries:

  • C244 Profile Command palette agent picker works — manual (picker UI isn't harness-drivable, same rationale as C237).
  • C245 Command palette agent source is strict — mapped to this suite in release-coverage-map.psd1.

The --delegate-source argument-validation branches (parse_delegate_source, require_delegate_agent_for_explicit_source) are already covered by cli/delegate.rs's own unit tests, so they are not duplicated here.

Copilot AI review requested due to automatic review settings August 3, 2026 09:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the profile-scoped delegation work from PR #488 by wiring the new commandPaletteAgent profile setting through the SettingsModel + Settings UI, updating wta delegate to accept explicit --delegate-source/--delegate-wsl-distro, and adding an end-to-end ItE2E suite to validate strict host/WSL routing behavior at the WT↔WTA boundary.

Changes:

  • Add explicit delegate source selection to wta delegate (--delegate-source host|wsl + --delegate-wsl-distro) and make host/WSL launchability checks source-strict (no auto-routing/fallback).
  • Add profile setting plumbing for commandPaletteAgent (settings model, schema, UI pickers, and TerminalPage delegation commandline construction).
  • Add a new ItE2E suite (Feature.DelegateSource.Tests.ps1) and map it into release coverage documentation.

Reviewed changes

Copilot reviewed 34 out of 34 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tools/wta/src/cli/mod.rs Pass new delegate source/distro args through CLI dispatch into delegate runner.
tools/wta/src/cli/delegate.rs Implement parsing/validation for delegate source flags; enforce source-strict launchability; add WSL cwd selection + unit tests.
tools/wta/src/cli/args.rs Add clap flags --delegate-source and --delegate-wsl-distro to the delegate subcommand.
tools/wta/src/cli_tests.rs Update CLI parsing tests and adjust comments to reflect delegate no longer uses active-pane WSL inference.
test/e2e/tests/Feature.DelegateSource.Tests.ps1 New ItE2E coverage for strict host/WSL delegation routing behavior.
test/e2e/release-coverage-map.psd1 Map new checklist item (“Command palette agent source is strict”) to the new E2E suite.
test/e2e/README.md Update suite table and coverage counts to include the new delegate-source suite.
src/cascadia/UnitTests_SettingsModel/ProfileTests.cpp Extend profile inheritance/clear tests to cover commandPaletteAgent.
src/cascadia/TerminalSettingsModel/Profile.idl Add inheritable CommandPaletteAgent profile setting.
src/cascadia/TerminalSettingsModel/MTSMSettings.h Add commandPaletteAgent to the settings X-macro list.
src/cascadia/TerminalSettingsEditor/Resources/en-US/Resources.resw Add localized strings for the new profile “Command palette agent” setting.
src/cascadia/TerminalSettingsEditor/Resources/de-DE/Resources.resw Same: localized strings.
src/cascadia/TerminalSettingsEditor/Resources/es-ES/Resources.resw Same: localized strings.
src/cascadia/TerminalSettingsEditor/Resources/fr-FR/Resources.resw Same: localized strings.
src/cascadia/TerminalSettingsEditor/Resources/it-IT/Resources.resw Same: localized strings.
src/cascadia/TerminalSettingsEditor/Resources/ja-JP/Resources.resw Same: localized strings.
src/cascadia/TerminalSettingsEditor/Resources/ko-KR/Resources.resw Same: localized strings.
src/cascadia/TerminalSettingsEditor/Resources/pt-BR/Resources.resw Same: localized strings.
src/cascadia/TerminalSettingsEditor/Resources/ru-RU/Resources.resw Same: localized strings.
src/cascadia/TerminalSettingsEditor/Resources/sr-Cyrl-RS/Resources.resw Same: localized strings.
src/cascadia/TerminalSettingsEditor/Resources/uk-UA/Resources.resw Same: localized strings.
src/cascadia/TerminalSettingsEditor/Resources/zh-CN/Resources.resw Same: localized strings.
src/cascadia/TerminalSettingsEditor/Resources/zh-TW/Resources.resw Same: localized strings.
src/cascadia/TerminalSettingsEditor/Resources/qps-ploc/Resources.resw Same: pseudo-locale strings.
src/cascadia/TerminalSettingsEditor/Resources/qps-ploca/Resources.resw Same: pseudo-locale strings.
src/cascadia/TerminalSettingsEditor/Resources/qps-plocm/Resources.resw Same: pseudo-locale strings.
src/cascadia/TerminalSettingsEditor/ProfileViewModel.idl Add VM properties for command palette agent list + selection and projected setting.
src/cascadia/TerminalSettingsEditor/ProfileViewModel.h Add VM accessors and list member for command palette agent entries; refactor list rebuild entry point.
src/cascadia/TerminalSettingsEditor/ProfileViewModel.cpp Build/display entries for both agent-pane backend and command palette agents; keep global-vs-profile selection behavior consistent.
src/cascadia/TerminalSettingsEditor/Profiles_Base.xaml Add UI setting container + ComboBox for per-profile Command palette agent selection.
src/cascadia/TerminalApp/TerminalPage.cpp Resolve profile commandPaletteAgent into explicit wta delegate flags and enforce “fail closed” for unknown/blocked profile selections.
README.md Document the new per-profile command palette agent behavior and strictness.
doc/release-check-list.md Add new checklist section/items for profile command palette agent selection and strict routing.
doc/cascadia/profiles.schema.json Add commandPaletteAgent to profile JSON schema with the same host/wsl encoding constraints as agent-pane backend.
Suppressed comments (2)

test/e2e/tests/Feature.DelegateSource.Tests.ps1:79

  • The helper hardcodes a delegate agent that is guaranteed to be missing in WSL. That makes the "default source is never diverted to WSL" case unable to detect a regression where WTA routes to WSL when the delegate agent is available in-distro (the historical inference path). Allow the helper to take a delegate-agent override so the host-default test can pick a command that’s present in WSL but not on Windows (e.g. /bin/sh).
            param([string[]]$ExtraArgs = @())

test/e2e/tests/Feature.DelegateSource.Tests.ps1:122

  • To prove the default source isn't auto-routed into WSL, the delegate agent should be something that would be available in WSL (so a regression could route there) but is not launchable on the Windows host. Using a bogus agent here is absent everywhere, so it can’t distinguish host-vs-WSL routing.
        $d = & $script:RunDelegate

Comment thread test/e2e/tests/Feature.DelegateSource.Tests.ps1 Outdated
Copilot AI review requested due to automatic review settings August 3, 2026 11:43
@DDKinger
Yuandi (DDKinger) force-pushed the dev/DDKinger/it-delegate-source branch from 8c3930c to 2630bdb Compare August 3, 2026 11:43
@DDKinger
Yuandi (DDKinger) changed the base branch from main to dev/DDKinger/profile-command-palette-agent August 3, 2026 11:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (3)

test/e2e/tests/Feature.DelegateSource.Tests.ps1:20

  • This suite invokes wta delegate with --agent 'copilot --acp --stdio' and configures acpAgent/delegateAgent to copilot, but the discovery gate doesn’t check that the copilot CLI is installed. On machines with WSL+winapp but no Copilot CLI, this will run and then fail (unlike Feature.Delegate.Tests.ps1, which gates on Get-Command copilot).
    $script:Ready = [bool](
        (Get-AppxPackage | Where-Object { $_.Name -like '*IntelligentTerminal*' }) -and
        (Get-Command winapp -ErrorAction SilentlyContinue) -and
        (Get-Command wsl.exe -ErrorAction SilentlyContinue)
    )

test/e2e/tests/Feature.DelegateSource.Tests.ps1:36

  • When the default WSL distro probe fails, BeforeAll sets $script:skipReason but still starts the Terminal process. Since both It blocks immediately skip when $script:skipReason is set, returning early here avoids unnecessary app startup work (and reduces noise/flakiness in environment-gated runs).
        if ($distroProbe.ExitCode -ne 0 -or -not $script:distro) {
            $script:skipReason = 'no runnable default WSL distro is available'
        }

test/e2e/tests/Feature.DelegateSource.Tests.ps1:39

  • Similarly, when the default distro name contains characters that can’t be safely represented on the command line, the suite sets $script:skipReason but continues into app startup. Returning early keeps this environment-gate cheap and consistent with Feature.WslAgentBackend.Tests.ps1.
        elseif ($script:distro -match '["\r\n]') {
            $script:skipReason = 'the default WSL distro name cannot be represented safely in a test command line'
        }

@DDKinger Yuandi (DDKinger) changed the title Integration test: cover profile delegate source strictness end to end (#488) Add E2E coverage for profile delegate source Aug 3, 2026
@DDKinger
Yuandi (DDKinger) changed the base branch from dev/DDKinger/profile-command-palette-agent to main August 4, 2026 09:30
Add Feature.DelegateSource.Tests.ps1 (mirrors #481's Feature.WslAgentBackend):
a profile's Command palette agent selects an exact delegate execution source,
so `wta delegate` always receives an explicit --delegate-source and never
re-routes itself.

- An explicit --delegate-source wsl selection never falls back to the Windows
  host when the agent is missing in that distro (regression positive for the
  removed wsl_agent_available==false fallback).
- The default/host source is never diverted to WSL even when a runnable distro
  is present (guards the removed active-pane WSL inference).

TerminalPage.cpp's profile resolution is only reachable via non-injectable UI
(Alt+Shift+B / Alt+Shift+/ / ?<prompt>), so — exactly like Feature.Delegate —
these cases drive the delegate engine directly with the flags it now passes.

Validated live against a freshly built #488 wta.exe; case 1 reproduces the
pre-fix failure when run against a wta.exe built from main.

Track C244 (picker UI, stays manual like C237) and C245 (routing strictness,
mapped to this suite in release-coverage-map.psd1). IDs via Set-ChecklistIds.ps1.

Based on the unmerged #488 head 528251a.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 02129b5b-f6fb-46e0-ab80-c2b9d6798f89
@DDKinger
Yuandi (DDKinger) force-pushed the dev/DDKinger/it-delegate-source branch from 2630bdb to a47ab2f Compare August 4, 2026 09:42
@DDKinger
Yuandi (DDKinger) enabled auto-merge (squash) August 4, 2026 09:49
@DDKinger
Yuandi (DDKinger) merged commit 71271cf into main Aug 4, 2026
10 checks passed
@vanzue
Kai Tao (vanzue) deleted the dev/DDKinger/it-delegate-source branch August 4, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants