Improve agent pane startup and response latency - #830
Improve agent pane startup and response latency#830Hamza Usmani (hamza-usmani) wants to merge 1 commit into
Conversation
Add tiered provider/helper prewarming, queued input while connecting, consolidated prompt context, non-blocking command recall, control-client isolation, and latency coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
It is a large cross-cutting change spanning WTA orchestration, Terminal Protocol/COM surfaces, UI behavior, and process lifecycle management, so a final human review is needed despite strong test coverage.
Pull request overview
This PR targets end-to-end latency in the agent pane workflow by reducing unnecessary helper/session materialization, allowing one prompt to be accepted while ACP connects, consolidating prompt-context capture into a single Terminal Protocol call, and moving expensive PowerShell recall work off the prompt critical path.
Changes:
- Introduces tiered prewarming (process master lease + default provider prewarm + selected-tab-only speculative helper/session) and adds generation-aware queued prompt dispatch while connecting.
- Adds Terminal Protocol 2.3
GetPromptContext+wtcli prompt-context, including bounded tail capture viaReadBufferTail. - Makes host-agent availability + PowerShell command recall caching asynchronous/single-flight, and adds/updates unit + E2E coverage plus supporting docs.
File summaries
| File | Description |
|---|---|
| tools/wta/src/telemetry.rs | Adds telemetry event for master default-provider prewarm completion. |
| tools/wta/src/slash_command_tests.rs | Updates tests for new binding_generation field in AgentConnected. |
| tools/wta/src/shell/wt_channel/cli_channel.rs | Adds get_prompt_context WTCLI routing + argument builder + tests. |
| tools/wta/src/shell/shell_manager.rs | Adds wt_get_prompt_context wrapper calling get_prompt_context. |
| tools/wta/src/session_registry.rs | Adds _meta.wta.connection_role for master-control clients + tests. |
| tools/wta/src/protocol/acp/turn_metrics.rs | Removes prompt preview tracing from prompt timing metrics. |
| tools/wta/src/protocol/acp/prompt_builder.rs | Logs terminal-context RPC timing; plumbs timing IDs into context request; updates mocks. |
| tools/wta/src/protocol/acp/mock_agent_tests.rs | Updates rename-session dispatch tests to include binding_generation. |
| tools/wta/src/protocol/acp/client.rs | Adds binding_generation wiring + stale binding guards; adjusts provider binding semantics. |
| tools/wta/src/master/session_mcp.rs | Adds test helper for capability registry emptiness checks. |
| tools/wta/src/master/config.rs | Extends master config with model + cloud model fields. |
| tools/wta/src/main.rs | Plumbs new CLI fields into master config. |
| tools/wta/src/helper/runtime.rs | Starts background command-recall prewarm; makes preflight non-blocking; logs first-frame timing. |
| tools/wta/src/command_recall.rs | Reworks PowerShell recall to memory-only lookup with single-flight background refresh + TTL. |
| tools/wta/src/cli/sessions.rs | Adds versioned “master-control” ACP init role stamping to avoid spawning agents. |
| tools/wta/src/app/turn_state.rs | Adds Queued turn state for one visible prompt while connecting + tests. |
| tools/wta/src/app.rs | Adds binding generation + queued prompt plumbing + reconnect/rekey durability + startup preflight obsolescence. |
| tools/wta/src/app_turn.rs | Implements queue/dispatch/cancel/restore paths for queued prompts; adds reconnect interruption handling. |
| tools/wta/src/app_keys.rs | Allows Enter while connecting (single queued prompt); Esc/Ctrl+C cancel queued prompt; improves timing log payload. |
| tools/wta/src/app_events.rs | Adds TabSessionRekeyed event; gates AgentConnected by binding generation; dispatches queued prompt on connect/rekey. |
| tools/wta/src/app_contracts/event.rs | Extends AppEvent with binding_generation and TabSessionRekeyed. |
| test/e2e/tests/Feature.AgentPaneInteraction.Tests.ps1 | Adds E2E coverage for “background tabs do not prewarm helpers”; hardens /model picker assertion; updates cleanup test. |
| test/e2e/README.md | Updates suite description and test count for AgentPaneInteraction. |
| src/tools/wtcli/main.cpp | Adds prompt-context subcommand and protocol version gating helper. |
| src/host/proxy/ITerminalProtocol.idl | Adds COM method GetPromptContext appended for vtable compatibility. |
| src/cascadia/WindowsTerminal/TerminalProtocolComServer.h | Declares GetPromptContext COM method. |
| src/cascadia/WindowsTerminal/TerminalProtocolComServer.cpp | Implements COM GetPromptContext, updates protocol version to 2.3, exposes capability name. |
| src/cascadia/ut_app/SharedWtaTests.cpp | Adds unit tests for master lease/pane ownership and configuration tracking behavior. |
| src/cascadia/ut_app/ProtocolParsingTests.cpp | Adds tests for prompt tail routing and metadata correctness. |
| src/cascadia/UnitTests_Control/ControlCoreTests.cpp | Adds tests for ControlCore::ReadBufferTail. |
| src/cascadia/TerminalSettingsEditor/MainPage.cpp | Forces host-agent availability invalidation + refresh on Settings startup. |
| src/cascadia/TerminalSettingsEditor/AIAgentsViewModel.h | Adds async refresh + list rebuild helpers for agent availability. |
| src/cascadia/TerminalSettingsEditor/AIAgentsViewModel.cpp | Uses cached availability immediately + async refresh; rebuilds ACP agent list with policy-safe selection fallback. |
| src/cascadia/TerminalProtocol/TerminalProtocol.idl | Adds PromptContext struct + GetPromptContext to the WinRT protocol surface. |
| src/cascadia/TerminalProtocol/ProtocolParsing.h | Adds prompt capture routing + bounded tail building helpers. |
| src/cascadia/TerminalControl/TermControl.idl | Adds ReadBufferTail to TermControl WinRT surface. |
| src/cascadia/TerminalControl/TermControl.h | Declares ReadBufferTail. |
| src/cascadia/TerminalControl/TermControl.cpp | Implements TermControl::ReadBufferTail. |
| src/cascadia/TerminalControl/ControlCore.idl | Adds ReadBufferTail to ControlCore WinRT surface. |
| src/cascadia/TerminalControl/ControlCore.h | Declares ControlCore::ReadBufferTail. |
| src/cascadia/TerminalControl/ControlCore.cpp | Implements bounded buffer-tail capture in ControlCore. |
| src/cascadia/TerminalApp/TerminalPage.Protocol.cpp | Adds TerminalPage::GetProtocolPromptContext implementation. |
| src/cascadia/TerminalApp/TerminalPage.idl | Adds GetProtocolPromptContext to TerminalPage projection. |
| src/cascadia/TerminalApp/TerminalPage.h | Adds prewarm/availability scheduling plumbing and protocol prompt-context declaration. |
| src/cascadia/TerminalApp/TabManagement.cpp | Switches to active-tab-only speculative prewarm; fixes SharedWta pane release accounting via transferable one-shot token. |
| src/cascadia/TerminalApp/Tab.h | Adds per-tab agent prewarm state tracker and APIs for speculative materialization/eviction/explicit-close suppression. |
| src/cascadia/TerminalApp/Tab.cpp | Transfers SharedWta pane token across window drag; marks prewarm tracker used/explicitly closed. |
| src/cascadia/TerminalApp/SharedWta.h | Adds process master lease ownership, configuration tracking, and one-shot pane reference token types. |
| src/cascadia/TerminalApp/SharedWta.cpp | Implements lease/pane dual ownership, configuration tracking, and updated crash recovery semantics. |
| src/cascadia/TerminalApp/Pane.h | Stores transferable SharedWta pane reference token on panes. |
| src/cascadia/TerminalApp/FreOverlay.h | Adds agent availability refresh + updates FRE problem kinds. |
| src/cascadia/TerminalApp/FreOverlay.cpp | Uses cached + refreshed agent availability; blocks saving when no valid selection; validates post-install refresh; hooks install gating. |
| src/cascadia/TerminalApp/AgentPaneDragStash.h | Stashes transferable SharedWta token for drag; adds expiry/discard + one-shot release. |
| src/cascadia/LocalTests_TerminalApp/TabTests.cpp | Adds tests for drag-stash discard/release behavior and explicit-close suppression. |
| src/cascadia/inc/FreAgentSetup.h | Introduces helper functions for FRE availability + save/install decision logic. |
| src/cascadia/inc/AgentRegistry.h | Adds policy-snapshot overloads for filtering builtin agent lists. |
| src/cascadia/inc/AgentPolicy.h | Makes Reload() return snapshot; adds snapshot-aware IsAgentAllowed/policy configured helper. |
| src/cascadia/inc/AgentAvailability.h | Adds generation-aware, single-flight cached host agent discovery with test hooks. |
| doc/specs/Multi-window-agent-pane.md | Updates spec to reflect tiered prewarm and queued prompt behavior. |
| doc/specs/agent-pane-latency-results.md | Adds engineering report with methodology, results, and limitations. |
| doc/release-check-list.md | Adds new checklist item C288 with E2E mapping. |
| AGENTS.md | Updates repo invariants to reflect tiered prewarm behavior. |
Review details
- Files reviewed: 68/68 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| fn prompt_context_args(params: &serde_json::Value) -> Vec<String> { | ||
| let source = params | ||
| .get("source_session_id") | ||
| .and_then(json_id_as_str) | ||
| .unwrap_or_default(); | ||
| let fallback_lines = params | ||
| .get("fallback_lines") | ||
| .and_then(|value| value.as_u64()) | ||
| .unwrap_or(24) | ||
| .to_string(); | ||
| let mut args = vec![ | ||
| "prompt-context".to_string(), | ||
| "--fallback-lines".to_string(), | ||
| fallback_lines, | ||
| ]; | ||
| if !source.is_empty() { | ||
| args.extend(["--source".to_string(), source]); | ||
| } | ||
| args | ||
| } |
| GUID source{}; | ||
| const auto hasExplicitSource = !promptContextSource.empty(); | ||
| if (hasExplicitSource) | ||
| { | ||
| source = GuidFromString(promptContextSource); | ||
| if (InlineIsEqualGUID(source, GUID{})) | ||
| { | ||
| exitCode = 1; | ||
| return; | ||
| } | ||
| } |
check-spelling-bot Report
|
| Dictionary | Entries | Covers | Uniquely |
|---|---|---|---|
| cspell:csharp/csharp.txt | 32 | 2 | 2 |
| cspell:aws/aws.txt | 232 | 2 | 2 |
| cspell:fonts/fonts.txt | 536 | 1 | 1 |
Consider adding to the extra_dictionaries array (in the .github/actions/spelling/config.json file):
"cspell:csharp/csharp.txt",
"cspell:aws/aws.txt",
"cspell:fonts/fonts.txt",
To stop checking additional dictionaries, put (in the .github/actions/spelling/config.json file):
"check_extra_dictionaries": []Forbidden patterns 🙅 (1)
In order to address this, you could change the content to not match the forbidden patterns (comments before forbidden patterns may help explain why they're forbidden), add patterns for acceptable instances, or adjust the forbidden patterns themselves.
These forbidden patterns matched content:
Should be preexisting
[Pp]re[- ]existing
Pattern suggestions ✂️ (1)
You could add these patterns to .github/actions/spelling/patterns/b361d91b25dda4b455a7f58ca04909c90f56bae3.txt:
# Automatically suggested patterns
# hit-count: 1 file-count: 1
# container images
image: [-\w./:@]+
Alternatively, if a pattern suggestion doesn't make sense for this project, add a # to the beginning of the line in the candidates file with the pattern to stop suggesting it.
Errors, Warnings, and Notices ❌ (3)
See the 📂 files view, the 📜action log, 👼 SARIF report, or 📝 job summary for details.
| ❌ Errors, Warnings, and Notices | Count |
|---|---|
| ℹ️ candidate-pattern | 1 |
| 54 | |
| ❌ forbidden-pattern | 1 |
See ❌ Event descriptions for more information.
✏️ Contributor please read this
By default the command suggestion will generate a file named based on your commit. That's generally ok as long as you add the file to your commit. Someone can reorganize it later.
If the listed items are:
- ... misspelled, then please correct them instead of using the command.
- ... names, please add them to
.github/actions/spelling/allow/names.txt. - ... APIs, you can add them to a file in
.github/actions/spelling/allow/. - ... just things you're using, please add them to an appropriate file in
.github/actions/spelling/expect/. - ... tokens you only need in one place and shouldn't generally be used, you can add an item in an appropriate file in
.github/actions/spelling/patterns/.
See the README.md in each directory for more information.
🔬 You can test your commits without appending to a PR by creating a new branch with that extra change and pushing it to your fork. The check-spelling action will run in response to your push -- it doesn't require an open pull request. By using such a branch, you can limit the number of typos your peers see you make. 😉
If the flagged items are 🤯 false positives
If items relate to a ...
-
binary file (or some other file you wouldn't want to check at all).
Please add a file path to the
excludes.txtfile matching the containing file.File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.
^refers to the file's path from the root of the repository, so^README\.md$would exclude README.md (on whichever branch you're using). -
well-formed pattern.
If you can write a pattern that would match it,
try adding it to thepatterns.txtfile.Patterns are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your lines.
Note that patterns can't match multiline strings.
Summary
GetPromptContext/wtcli prompt-context, including bounded buffer-tail capture and explicit-pane safety.doc/specs/agent-pane-latency-results.md.Benchmarks
The first Debug launch after deployment remained an 8.46-second outlier because
wta.exeprocess startup itself took 4.3 seconds; subsequent trials stabilized at 3.17-3.37 seconds. Copilot/providerinitializeandsession/newremain the dominant steady startup cost.Validation
Full methodology, implementation contracts, lifecycle decisions, test mapping, and limitations are documented in the linked report.