Skip to content

Commit 2630bdb

Browse files
DDKingerCopilot
andcommitted
test(wta): cover profile delegate source strictness end to end (#488)
Add Feature.DelegateSource.Tests.ps1 (mirrors #481's Feature.WslAgentBackend): a profile's Command palette agent selects an exact delegate execution source, so `wta delegate` always receives an explicit --delegate-source and never re-routes itself. - An explicit --delegate-source wsl selection never falls back to the Windows host when the agent is missing in that distro (regression positive for the removed wsl_agent_available==false fallback). - The default/host source is never diverted to WSL even when a runnable distro is present (guards the removed active-pane WSL inference). TerminalPage.cpp's profile resolution is only reachable via non-injectable UI (Alt+Shift+B / Alt+Shift+/ / ?<prompt>), so — exactly like Feature.Delegate — these cases drive the delegate engine directly with the flags it now passes. Validated live against a freshly built #488 wta.exe; case 1 reproduces the pre-fix failure when run against a wta.exe built from main. Track C244 (picker UI, stays manual like C237) and C245 (routing strictness, mapped to this suite in release-coverage-map.psd1). IDs via Set-ChecklistIds.ps1. Based on the unmerged #488 head 528251a. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 02129b5b-f6fb-46e0-ab80-c2b9d6798f89
1 parent 528251a commit 2630bdb

4 files changed

Lines changed: 112 additions & 7 deletions

File tree

doc/release-check-list.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,11 @@ Net effect: UT shrinks the manual matrix to "did the wiring and UI connect", not
102102
- [ ] `C237` `[new]` `[UT~]` `[E2E]` **Profile Agent pane agent picker works:** Each profile's Agent pane agent picker lists installed, policy-allowed Windows agents plus agents installed natively in that profile's WSL distro; saving persists `agentPaneBackend`, while an unconfigured profile inherits the global Windows agent. _(#481; UT: `ProfileTests::AgentPaneBackendDefaultsAndInherits`.)_
103103
- [ ] `C238` `[new]` `[E2E]` **Profile WSL agent routing is strict:** Changing a profile to a WSL backend rebuilds its helper, routes the exact selected agent through `wsl:<distro>`, and does not fall back to a Windows-hosted agent. _(#481; E2E: `Feature.WslAgentBackend`.)_
104104

105+
### Profile command palette agent
106+
107+
- [ ] `C244` `[new]` `[UT~]` `[E2E]` **Profile Command palette agent picker works:** Each profile's Command palette agent picker offers Host and WSL-distro delegate agents; saving persists `commandPaletteAgent`, while an unconfigured profile follows the global delegate agent. _(#488; UT: `ProfileTests::AgentPaneBackendDefaultsAndInherits`.)_
108+
- [ ] `C245` `[new]` `[E2E]` **Command palette agent source is strict:** A profile's Command palette agent selects the exact delegate execution source — an explicit WSL selection stays in its distro and surfaces the real in-distro error, and a host selection is never diverted to WSL. _(#488; E2E: `Feature.DelegateSource`.)_
109+
105110
## 2. Agent pane chat
106111

107112
**Feature definition:** The agent pane is a per-tab AI chat pane backed by WTA helper/master and an ACP-capable agent. It should be reusable, able to be hidden, and stable across tab/window operations.

test/e2e/README.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -28,16 +28,17 @@ environment. Current status (run on the Store package):
2828
| `Feature.AgentMatrix.Tests.ps1` | §2 non-Copilot built-in agents (Claude/Codex/Gemini) connect+chat through the ACP adapter — ONE consolidated case (Copilot is the in-depth suite); skips when none installed+authed | 1 |
2929
| `Feature.PerTabAgent.Tests.ps1` | C225-C228 + PR #487: `/agent` picker/prefix completion/direct selection, invalid-id safety, per-tab isolation/shared-master reuse, and global-default/override behavior | 8 |
3030
| `Feature.WslAgentBackend.Tests.ps1` | PR #481 profile-scoped WSL agent backend: settings hot reload, helper/master source routing, and authenticated chat | 2 (environment-gated) |
31+
| `Feature.DelegateSource.Tests.ps1` | PR #488 profile-scoped delegate source: strict host/WSL `wta delegate` routing with no fallback in either direction | 2 (environment-gated) |
3132
| `Feature.AgentChat.Tests.ps1` / `Feature.AgentPopup.Tests.ps1` | agent chat + `/` popup/menu interaction | 1 + 3 |
3233

33-
**Coverage: 107 of 108 automatable `[E2E]` checklist items are implemented.**
34+
**Coverage: 108 of 110 automatable `[E2E]` checklist items are implemented.**
3435
**Test status: 102 baseline feature cases pass + 2 documented skips** (`wta sessions list` is
35-
identity-gated — see `Feature.SessionList.Tests.ps1`), plus 2 PR #481 WSL-backend cases that
36-
run only when a dev package, runnable distro, and native supported agent are available; the
37-
chat case also requires authentication. The 107 implemented checklist items map to the
38-
baseline cases plus the deterministic settings/persistence assertions. The remaining new
39-
item is the profile Agent pane agent picker UI; it stays explicit E2E work rather than being
40-
falsely credited by the JSON-level runtime tests. Other
36+
identity-gated — see `Feature.SessionList.Tests.ps1`), plus 2 PR #481 WSL-backend cases and 2
37+
PR #488 delegate-source cases that run only when a runnable distro (and, for the #481 chat
38+
case, an installed+authenticated native agent) is available. The 108 implemented checklist
39+
items map to the baseline cases plus the deterministic settings/persistence assertions. The
40+
remaining new items are the two profile agent picker UIs; they stay explicit E2E work rather
41+
than being falsely credited by the JSON-level runtime tests. Other
4142
environment-dependent items are tracked and auto-skipped when their prerequisite is absent:
4243
**other agent CLIs** (`Feature.AgentMatrix.Tests.ps1` now covers Claude/Codex/Gemini chat,
4344
auth-gated per CLI — each Context runs only when that CLI is installed *and* authenticated,

test/e2e/release-coverage-map.psd1

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,11 @@
4242
'Profile WSL agent routing is strict' = 'Hot reload routes the profile agent through its WSL distro without host fallback'
4343
'Profile WSL agent chat works' = 'profile-selected WSL agent connects and answers a chat round trip'
4444

45+
# PR #488 profile-scoped delegate source routing. The profile picker UI is
46+
# deliberately NOT mapped, same as #481's above: Feature.DelegateSource drives
47+
# `wta delegate` directly and cannot prove the Settings picker renders or saves.
48+
'Command palette agent source is strict' = 'never falls back to the Windows host|never diverted to WSL'
49+
4550
# §0 FRE auto-error (on/off both covered by the single off/on test)
4651
'Automatic error detection on' = 'Automatic error detection off/on'
4752
'Automatic error suggestion on' = 'Automatic error suggestion off/on'
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
#Requires -Modules @{ ModuleName='Pester'; ModuleVersion='5.0.0' }
2+
# PR #488: a profile can pin the command palette (delegate) agent to an exact
3+
# execution source, so `wta delegate` always gets an explicit `--delegate-source
4+
# host|wsl` and never re-routes itself. Both assertions are deterministic once a
5+
# runnable WSL distro is present.
6+
#
7+
# The C++ half (TerminalPage.cpp turning a profile's `commandPaletteAgent` into
8+
# those flags) is only reachable via non-injectable UI — Alt+Shift+B, the
9+
# Alt+Shift+/ palette, `?<prompt>` — so, exactly like Feature.Delegate, these
10+
# cases drive the delegate ENGINE directly with the flags it now always passes.
11+
# The oracle is the rendered pane, not wta-delegate.log: `Invoke-Wta` runs an
12+
# unpackaged copy of wta.exe, which logs to the bare fallback dir rather than the
13+
# packaged one `Assert-Log` reads.
14+
15+
BeforeDiscovery {
16+
$script:Ready = [bool](
17+
(Get-AppxPackage | Where-Object { $_.Name -like '*IntelligentTerminal*' }) -and
18+
(Get-Command winapp -ErrorAction SilentlyContinue) -and
19+
(Get-Command wsl.exe -ErrorAction SilentlyContinue)
20+
)
21+
}
22+
23+
Describe 'Feature profile-scoped delegate source' -Tag 'Feature' -Skip:(-not $script:Ready) {
24+
BeforeAll {
25+
Import-Module (Join-Path $PSScriptRoot '..\ItE2E\ItE2E.psd1') -Force
26+
27+
$script:app = $null
28+
$script:skipReason = $null
29+
30+
$distroProbe = Invoke-Native -FilePath 'wsl.exe' -Arguments @(
31+
'-e', 'sh', '-lc', 'printf "%s" "${WSL_DISTRO_NAME:-}"'
32+
) -TimeoutSec 45
33+
$script:distro = $distroProbe.StdOut.Trim()
34+
if ($distroProbe.ExitCode -ne 0 -or -not $script:distro) {
35+
$script:skipReason = 'no runnable default WSL distro is available'
36+
}
37+
elseif ($script:distro -match '["\r\n]') {
38+
$script:skipReason = 'the default WSL distro name cannot be represented safely in a test command line'
39+
}
40+
41+
$script:app = Start-Terminal -Package (Get-ItTestPackage) -PassFre $true `
42+
-Settings @{ acpAgent = 'copilot'; delegateAgent = 'copilot' }
43+
44+
# A random, never-installed name is absent from BOTH the Windows PATH and every
45+
# distro's PATH, so the "agent unavailable" branch fires deterministically.
46+
$script:bogusAgent = "ite2e-bogus-delegate-$(Get-Random -Maximum 999999)"
47+
48+
# Launch a delegate and return the NEW tab created in this window.
49+
$script:RunDelegate = {
50+
param([string[]]$ExtraArgs = @())
51+
$wid = [string]$script:app.WindowId
52+
$before = @((Get-WtTabs -App $script:app -WindowId $wid).tab_id)
53+
Invoke-Wta -App $script:app -TimeoutSec 40 -Raw -Arguments (@(
54+
'delegate', 'hi', '--agent', 'copilot --acp --stdio',
55+
'--delegate-agent', $script:bogusAgent) + $ExtraArgs) | Out-Null
56+
$newTab = $null
57+
for ($i = 0; $i -lt 30 -and -not $newTab; $i++) {
58+
$newTab = @(Get-WtTabs -App $script:app -WindowId $wid) | Where-Object { $_.tab_id -notin $before } | Select-Object -First 1
59+
if (-not $newTab) { Start-Sleep -Milliseconds 500 }
60+
}
61+
$panes = if ($newTab) { @(Get-WtPanes -App $script:app -WindowId $wid -TabId ([string]$newTab.tab_id)) } else { @() }
62+
@{ Tab = $newTab; Panes = $panes }
63+
}
64+
}
65+
AfterAll { if ($script:app) { Stop-Terminal -App $script:app } }
66+
67+
It 'An explicit WSL delegate source never falls back to the Windows host' {
68+
if ($script:skipReason) { Set-ItResult -Skipped -Because $script:skipReason; return }
69+
70+
$d = & $script:RunDelegate -ExtraArgs @('--delegate-source', 'wsl', '--delegate-wsl-distro', $script:distro)
71+
$d.Tab | Should -Not -BeNullOrEmpty -Because 'even a doomed WSL launch must open a tab so the real error stays visible'
72+
73+
# bash's own "not found" (the launch wraps the command in `exec`) proves it really
74+
# ran in the distro; Windows would say "cannot find the file specified".
75+
$sid = $d.Panes[0].session_id
76+
(Test-Until -TimeoutSec 20 -IntervalSec 1 -Condition {
77+
(Get-WtCapture -App $script:app -SessionId $sid -MaxLines 40) -match '(?i)bash:.*not found'
78+
}) | Should -BeTrue -Because 'a missing WSL delegate agent must surface the real in-distro error, not silently switch to the host'
79+
}
80+
81+
It 'The default delegate source is never diverted to WSL' {
82+
if ($script:skipReason) { Set-ItResult -Skipped -Because $script:skipReason; return }
83+
84+
# No --delegate-source => host. A runnable distro on this machine (proven by the
85+
# probe above) must not tempt the CLI into routing there.
86+
$d = & $script:RunDelegate
87+
$d.Tab | Should -Not -BeNullOrEmpty -Because 'the host delegate launch must still open a tab'
88+
89+
$sid = $d.Panes[0].session_id
90+
(Test-Until -TimeoutSec 20 -IntervalSec 1 -Condition {
91+
(Get-WtCapture -App $script:app -SessionId $sid -MaxLines 40) -match '(?i)cannot find the file specified|is not recognized'
92+
}) | Should -BeTrue -Because 'the default launch must fail with the real Windows error, proving it never ran inside WSL bash'
93+
}
94+
}

0 commit comments

Comments
 (0)