Add E2E coverage for profile delegate source - #537
Conversation
There was a problem hiding this comment.
Pull request overview
This PR extends the profile-scoped delegation work from PR #488 by wiring the new commandPaletteAgent profile setting through the SettingsModel + Settings UI, updating wta delegate to accept explicit --delegate-source/--delegate-wsl-distro, and adding an end-to-end ItE2E suite to validate strict host/WSL routing behavior at the WT↔WTA boundary.
Changes:
- Add explicit delegate source selection to
wta delegate(--delegate-source host|wsl+--delegate-wsl-distro) and make host/WSL launchability checks source-strict (no auto-routing/fallback). - Add profile setting plumbing for
commandPaletteAgent(settings model, schema, UI pickers, and TerminalPage delegation commandline construction). - Add a new ItE2E suite (
Feature.DelegateSource.Tests.ps1) and map it into release coverage documentation.
Reviewed changes
Copilot reviewed 34 out of 34 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tools/wta/src/cli/mod.rs | Pass new delegate source/distro args through CLI dispatch into delegate runner. |
| tools/wta/src/cli/delegate.rs | Implement parsing/validation for delegate source flags; enforce source-strict launchability; add WSL cwd selection + unit tests. |
| tools/wta/src/cli/args.rs | Add clap flags --delegate-source and --delegate-wsl-distro to the delegate subcommand. |
| tools/wta/src/cli_tests.rs | Update CLI parsing tests and adjust comments to reflect delegate no longer uses active-pane WSL inference. |
| test/e2e/tests/Feature.DelegateSource.Tests.ps1 | New ItE2E coverage for strict host/WSL delegation routing behavior. |
| test/e2e/release-coverage-map.psd1 | Map new checklist item (“Command palette agent source is strict”) to the new E2E suite. |
| test/e2e/README.md | Update suite table and coverage counts to include the new delegate-source suite. |
| src/cascadia/UnitTests_SettingsModel/ProfileTests.cpp | Extend profile inheritance/clear tests to cover commandPaletteAgent. |
| src/cascadia/TerminalSettingsModel/Profile.idl | Add inheritable CommandPaletteAgent profile setting. |
| src/cascadia/TerminalSettingsModel/MTSMSettings.h | Add commandPaletteAgent to the settings X-macro list. |
| src/cascadia/TerminalSettingsEditor/Resources/en-US/Resources.resw | Add localized strings for the new profile “Command palette agent” setting. |
| src/cascadia/TerminalSettingsEditor/Resources/de-DE/Resources.resw | Same: localized strings. |
| src/cascadia/TerminalSettingsEditor/Resources/es-ES/Resources.resw | Same: localized strings. |
| src/cascadia/TerminalSettingsEditor/Resources/fr-FR/Resources.resw | Same: localized strings. |
| src/cascadia/TerminalSettingsEditor/Resources/it-IT/Resources.resw | Same: localized strings. |
| src/cascadia/TerminalSettingsEditor/Resources/ja-JP/Resources.resw | Same: localized strings. |
| src/cascadia/TerminalSettingsEditor/Resources/ko-KR/Resources.resw | Same: localized strings. |
| src/cascadia/TerminalSettingsEditor/Resources/pt-BR/Resources.resw | Same: localized strings. |
| src/cascadia/TerminalSettingsEditor/Resources/ru-RU/Resources.resw | Same: localized strings. |
| src/cascadia/TerminalSettingsEditor/Resources/sr-Cyrl-RS/Resources.resw | Same: localized strings. |
| src/cascadia/TerminalSettingsEditor/Resources/uk-UA/Resources.resw | Same: localized strings. |
| src/cascadia/TerminalSettingsEditor/Resources/zh-CN/Resources.resw | Same: localized strings. |
| src/cascadia/TerminalSettingsEditor/Resources/zh-TW/Resources.resw | Same: localized strings. |
| src/cascadia/TerminalSettingsEditor/Resources/qps-ploc/Resources.resw | Same: pseudo-locale strings. |
| src/cascadia/TerminalSettingsEditor/Resources/qps-ploca/Resources.resw | Same: pseudo-locale strings. |
| src/cascadia/TerminalSettingsEditor/Resources/qps-plocm/Resources.resw | Same: pseudo-locale strings. |
| src/cascadia/TerminalSettingsEditor/ProfileViewModel.idl | Add VM properties for command palette agent list + selection and projected setting. |
| src/cascadia/TerminalSettingsEditor/ProfileViewModel.h | Add VM accessors and list member for command palette agent entries; refactor list rebuild entry point. |
| src/cascadia/TerminalSettingsEditor/ProfileViewModel.cpp | Build/display entries for both agent-pane backend and command palette agents; keep global-vs-profile selection behavior consistent. |
| src/cascadia/TerminalSettingsEditor/Profiles_Base.xaml | Add UI setting container + ComboBox for per-profile Command palette agent selection. |
| src/cascadia/TerminalApp/TerminalPage.cpp | Resolve profile commandPaletteAgent into explicit wta delegate flags and enforce “fail closed” for unknown/blocked profile selections. |
| README.md | Document the new per-profile command palette agent behavior and strictness. |
| doc/release-check-list.md | Add new checklist section/items for profile command palette agent selection and strict routing. |
| doc/cascadia/profiles.schema.json | Add commandPaletteAgent to profile JSON schema with the same host/wsl encoding constraints as agent-pane backend. |
Suppressed comments (2)
test/e2e/tests/Feature.DelegateSource.Tests.ps1:79
- The helper hardcodes a delegate agent that is guaranteed to be missing in WSL. That makes the "default source is never diverted to WSL" case unable to detect a regression where WTA routes to WSL when the delegate agent is available in-distro (the historical inference path). Allow the helper to take a delegate-agent override so the host-default test can pick a command that’s present in WSL but not on Windows (e.g. /bin/sh).
param([string[]]$ExtraArgs = @())
test/e2e/tests/Feature.DelegateSource.Tests.ps1:122
- To prove the default source isn't auto-routed into WSL, the delegate agent should be something that would be available in WSL (so a regression could route there) but is not launchable on the Windows host. Using a bogus agent here is absent everywhere, so it can’t distinguish host-vs-WSL routing.
$d = & $script:RunDelegate
8c3930c to
2630bdb
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (3)
test/e2e/tests/Feature.DelegateSource.Tests.ps1:20
- This suite invokes
wta delegatewith--agent 'copilot --acp --stdio'and configuresacpAgent/delegateAgenttocopilot, but the discovery gate doesn’t check that thecopilotCLI is installed. On machines with WSL+winapp but no Copilot CLI, this will run and then fail (unlikeFeature.Delegate.Tests.ps1, which gates onGet-Command copilot).
$script:Ready = [bool](
(Get-AppxPackage | Where-Object { $_.Name -like '*IntelligentTerminal*' }) -and
(Get-Command winapp -ErrorAction SilentlyContinue) -and
(Get-Command wsl.exe -ErrorAction SilentlyContinue)
)
test/e2e/tests/Feature.DelegateSource.Tests.ps1:36
- When the default WSL distro probe fails,
BeforeAllsets$script:skipReasonbut still starts the Terminal process. Since bothItblocks immediately skip when$script:skipReasonis set, returning early here avoids unnecessary app startup work (and reduces noise/flakiness in environment-gated runs).
if ($distroProbe.ExitCode -ne 0 -or -not $script:distro) {
$script:skipReason = 'no runnable default WSL distro is available'
}
test/e2e/tests/Feature.DelegateSource.Tests.ps1:39
- Similarly, when the default distro name contains characters that can’t be safely represented on the command line, the suite sets
$script:skipReasonbut continues into app startup. Returning early keeps this environment-gate cheap and consistent withFeature.WslAgentBackend.Tests.ps1.
elseif ($script:distro -match '["\r\n]') {
$script:skipReason = 'the default WSL distro name cannot be represented safely in a test command line'
}
Add Feature.DelegateSource.Tests.ps1 (mirrors #481's Feature.WslAgentBackend): a profile's Command palette agent selects an exact delegate execution source, so `wta delegate` always receives an explicit --delegate-source and never re-routes itself. - An explicit --delegate-source wsl selection never falls back to the Windows host when the agent is missing in that distro (regression positive for the removed wsl_agent_available==false fallback). - The default/host source is never diverted to WSL even when a runnable distro is present (guards the removed active-pane WSL inference). TerminalPage.cpp's profile resolution is only reachable via non-injectable UI (Alt+Shift+B / Alt+Shift+/ / ?<prompt>), so — exactly like Feature.Delegate — these cases drive the delegate engine directly with the flags it now passes. Validated live against a freshly built #488 wta.exe; case 1 reproduces the pre-fix failure when run against a wta.exe built from main. Track C244 (picker UI, stays manual like C237) and C245 (routing strictness, mapped to this suite in release-coverage-map.psd1). IDs via Set-ChecklistIds.ps1. Based on the unmerged #488 head 528251a. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 02129b5b-f6fb-46e0-ab80-c2b9d6798f89
2630bdb to
a47ab2f
Compare
Summary
Adds
test/e2e/tests/Feature.DelegateSource.Tests.ps1covering PR #488's strict delegate source routing — the direct counterpart to #486 for #481.#488 is now merged, so this branch is rebased onto
mainand its diff is test-only.Cases (environment-gated: needs a runnable default WSL distro)
not found.TerminalPage.cpp's profile resolution (commandPaletteAgent→--delegate-source/--delegate-wsl-distro) is only reachable via non-injectable UI (Alt+Shift+B, theAlt+Shift+/palette,?<prompt>), so — exactly likeFeature.Delegate— these cases drive the delegate engine directly with the flags it now always passes.Validation
mainbuild → both cases pass;Invoke-ItE2EReport.ps1 -UpdateReportcreditsC245as[x],C244stays manual.wta.exebuilt before Add profile-scoped command palette agents #488 (no--delegate-sourceflag), case 1 fails withAUTOMATION FAILEDrather than passing vacuously — confirmed twice, once deliberately on amain-built binary and once accidentally against a stale deployment.Checklist
New Profile command palette agent section in
doc/release-check-list.md, mirroring #481's entries:C244Profile Command palette agent picker works — manual (picker UI isn't harness-drivable, same rationale asC237).C245Command palette agent source is strict — mapped to this suite inrelease-coverage-map.psd1.The
--delegate-sourceargument-validation branches (parse_delegate_source,require_delegate_agent_for_explicit_source) are already covered bycli/delegate.rs's own unit tests, so they are not duplicated here.