Contain agent-pane wta in a Job Object + watch its exit

When the agent pane's wta.exe dies (crash / external taskkill / clean
exit), the pane was left bound to a defunct child. Subsequent input or
protocol writes from the UI thread would then block on the dead pipe,
producing the MoAppHang observed in WER. And the wta children (npx →
npm → node → ACP adapter), which inherit the conpty client handles,
were left running indefinitely.

ConptyConnection state can't detect wta exit on its own: those
inherited handles keep the pipe open until the LAST descendant exits,
not when the root wta dies. PowerShell etc. don't show this because
shells don't normally fork long-lived children that inherit stdio.

Fix:

- Wrap the agent pane's wta in a Job Object with
  JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE. Closing our handle to the job
  terminates every member — wta and all its descendants. This handles
  the orphan-process leak directly and (because WT crashing also drops
  the handle) guarantees cleanup even on abnormal WT exit.

- Watch wta's process handle with RegisterWaitForSingleObject. When
  wta exits for any reason, the threadpool callback marshals teardown
  to the UI thread, which closes the pane and the job (reaping any
  surviving children in the process).

- Heap-allocated AgentPaneWtaWaitContext (weak_ref + cancellation
  flag), plus an UnregisterWaitEx(INVALID_HANDLE_VALUE) at teardown,
  make the wait safe to race with deliberate teardown (settings
  rebuild, user pane close, page destruction).

Scope is intentionally limited to the agent pane:

- ConptyConnection is not modified — ordinary shell panes keep their
  existing free-running lifetime (Start-Process notepad etc. still
  survive pane close as users expect).

- Delegation wta (the ?<prompt> path through WtaProcessLauncher) is
  left as a follow-up. Its children leak in the same way and should
  get the same Job-Object treatment, but the launcher is a separate
  code path.

There is a small race window between ConptyConnection's CreateProcessW
and our AssignProcessToJobObject (we hook in via TermControl.Initialized
after the spawn). In practice wta does ACP / npx setup before it spawns
any child, so the window is empty. A future tightening could move the
job assignment into ConptyConnection with CREATE_SUSPENDED + ResumeThread
for strict zero-race semantics.

Author: vanzue

.github/actions/spelling/allow/apis.txt

@@ -37,6 +37,8 @@ ENDSESSION
 enumset
 environstrings
 EXACTSIZEONLY
+EXECUTEDEFAULT
+EXECUTEONLYONCE
 EXPCMDFLAGS
 EXPCMDSTATE
 filetime

.github/actions/spelling/expect/expect.txt

@@ -528,6 +528,7 @@ ETW
 EUDC
 eventing
 evflags
+EOFs
 execd
 executionengine
 exemain
@@ -1641,6 +1642,7 @@ tdbuild
 TDP
 Teb
 Techo
+taskkill
 tellp
 teraflop
 terminalcore
@@ -2016,6 +2018,7 @@ testname
 transitioning
 unk
 unparseable
+unregisters
 Virt
 VMs
 webpage

src/cascadia/TerminalApp/TerminalPage.cpp

@@ -236,6 +236,14 @@ namespace winrt::TerminalApp::implementation
 
     TerminalPage::~TerminalPage()
     {
+        // Critical ordering: disarm the wta process watch BEFORE other
+        // members destruct. The threadpool wait holds a raw pointer into
+        // _agentPaneWtaWaitContext (a unique_ptr member). If we let that
+        // member destruct while the wait is still registered, a wta exit
+        // after this point would fire the callback against freed memory.
+        // _TearDownAgentPaneWtaWatch calls UnregisterWaitEx blocking, so
+        // by the time it returns no callback can fire.
+        _TearDownAgentPaneWtaWatch();
     }
 
     // Method Description:
@@ -1332,9 +1340,191 @@ namespace winrt::TerminalApp::implementation
                a.delegateCustomCommand != b.delegateCustomCommand;
     }
 
+    // Heap-allocated context for the threadpool wait callback. Keeps the
+    // weak_ref + cancellation flag alive independently of TerminalPage so the
+    // callback never dereferences freed memory. Owned by
+    // _agentPaneWtaWaitContext on the page; freed only after the wait has
+    // been unregistered (callback can no longer fire) or after the callback
+    // has run and we marshal back to the UI thread.
+    struct TerminalPage::AgentPaneWtaWaitContext
+    {
+        winrt::weak_ref<TerminalPage> page;
+        std::atomic<bool> cancelled{ false };
+    };
+
+    // Arm process-exit detection + Job-Object containment for the agent
+    // pane's wta. Called from the agent pane's Initialized callback once
+    // ConptyConnection has spawned wta and RootProcessHandle is valid.
+    //
+    // The Job carries JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE: closing our handle
+    // (or losing it via WT crash) terminates wta + every descendant. wta's
+    // future children inherit job membership automatically.
+    //
+    // Race window: wta could (in principle) spawn a child between
+    // CreateProcessW and AssignProcessToJobObject. wta does ACP/npx setup
+    // before spawning anything, so in practice the window is empty. A
+    // future tightening could move this into ConptyConnection with
+    // CREATE_SUSPENDED + AssignProcessToJobObject + ResumeThread.
+    void TerminalPage::_SetupAgentPaneWtaWatch(HANDLE wtaProcessHandle) noexcept
+    try
+    {
+        if (!wtaProcessHandle || wtaProcessHandle == INVALID_HANDLE_VALUE)
+        {
+            return;
+        }
+        if (_agentPaneWtaWait)
+        {
+            // Already armed for some prior wta — disarm first.
+            _TearDownAgentPaneWtaWatch();
+        }
+
+        // ORDER MATTERS. The watch (death detection) must be set up BEFORE
+        // the Job Object (orphan containment). If we created the job first
+        // and a later step failed, the local `unique_handle job` would
+        // destruct, fire KILL_ON_JOB_CLOSE, and turn a "watch arm failed"
+        // soft error into a hard kill of wta + its children. Setting up
+        // the watch first also means death detection still works even if
+        // job containment is unavailable (e.g. nested-job restrictions).
+
+        // 1) Independent SYNCHRONIZE handle so the wait survives across
+        // the ordering of ConptyConnection cleanup.
+        HANDLE dup{ nullptr };
+        if (!DuplicateHandle(GetCurrentProcess(), wtaProcessHandle,
+                             GetCurrentProcess(), &dup,
+                             SYNCHRONIZE, FALSE, 0))
+        {
+            _agentPaneLog("DuplicateHandle(wta) failed");
+            return;
+        }
+        wil::unique_handle wtaDup{ dup };
+
+        // 2) Heap-allocate the callback context.
+        auto ctx = std::make_unique<AgentPaneWtaWaitContext>();
+        ctx->page = get_weak();
+
+        // 3) Register the wait. From here on, if we early-return without
+        // committing to members, UnregisterWaitEx is the cleanup; ctx
+        // auto-frees via unique_ptr.
+        HANDLE waitHandle{ nullptr };
+        if (!RegisterWaitForSingleObject(
+                &waitHandle,
+                wtaDup.get(),
+                &TerminalPage::_OnAgentPaneWtaExit,
+                ctx.get(),
+                INFINITE,
+                WT_EXECUTEONLYONCE | WT_EXECUTEDEFAULT))
+        {
+            _agentPaneLog("RegisterWaitForSingleObject(wta) failed");
+            return;
+        }
+
+        // 4) Commit the watch immediately. Death detection is the core
+        // fix — we keep it even if job setup below fails (we just lose
+        // the orphan-cleanup nice-to-have, not the hang fix).
+        _agentPaneWtaHandle = std::move(wtaDup);
+        _agentPaneWtaWait = waitHandle;
+        _agentPaneWtaWaitContext = std::move(ctx);
+
+        // 5) Now attempt Job Object containment. This is best-effort;
+        // failure just means orphans won't be reaped automatically.
+        wil::unique_handle job{ CreateJobObjectW(nullptr, nullptr) };
+        if (!job)
+        {
+            _agentPaneLog("CreateJobObject failed; orphan cleanup disabled (death-watch still active)");
+            return;
+        }
+        JOBOBJECT_EXTENDED_LIMIT_INFORMATION limits{};
+        limits.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;
+        if (!SetInformationJobObject(job.get(), JobObjectExtendedLimitInformation, &limits, sizeof(limits)))
+        {
+            _agentPaneLog("SetInformationJobObject(KILL_ON_JOB_CLOSE) failed; orphan cleanup disabled (death-watch still active)");
+            return;
+        }
+        if (!AssignProcessToJobObject(job.get(), wtaProcessHandle))
+        {
+            _agentPaneLog("AssignProcessToJobObject failed; orphan cleanup disabled (death-watch still active)");
+            return;
+        }
+        // Commit the job immediately after Assign — no fallible call may
+        // appear between Assign and this assignment, or the local `job`
+        // destructor would fire KILL_ON_JOB_CLOSE on the live wta.
+        _agentPaneJob = std::move(job);
+        _agentPaneLog("agent pane wta watch armed (job + process wait)");
+    }
+    CATCH_LOG()
+
+    // Disarm wait + drop job. Closing the job handle terminates wta and
+    // every surviving descendant (KILL_ON_JOB_CLOSE) — single source of
+    // truth for "stop the agent process group".
+    void TerminalPage::_TearDownAgentPaneWtaWatch() noexcept
+    {
+        if (_agentPaneWtaWaitContext)
+        {
+            _agentPaneWtaWaitContext->cancelled.store(true, std::memory_order_release);
+        }
+        if (_agentPaneWtaWait)
+        {
+            // INVALID_HANDLE_VALUE blocks until any in-flight callback
+            // completes — safe because we never call this from inside the
+            // callback (the callback dispatches to the UI thread and the
+            // UI-thread continuation handles teardown there).
+            UnregisterWaitEx(_agentPaneWtaWait, INVALID_HANDLE_VALUE);
+            _agentPaneWtaWait = nullptr;
+        }
+        _agentPaneWtaWaitContext.reset();
+        _agentPaneWtaHandle.reset();
+        _agentPaneJob.reset(); // KILL_ON_JOB_CLOSE — descendants die here.
+    }
+
+    // Threadpool callback: wta exited. We can't touch most TerminalPage
+    // state from here (wrong thread + no UI marshalling), so just marshal
+    // a teardown request to the UI thread.
+    void NTAPI TerminalPage::_OnAgentPaneWtaExit(PVOID context, BOOLEAN /*timedOut*/) noexcept
+    try
+    {
+        auto* ctx = static_cast<AgentPaneWtaWaitContext*>(context);
+        if (!ctx || ctx->cancelled.load(std::memory_order_acquire))
+        {
+            return;
+        }
+        const auto strong = ctx->page.get();
+        if (!strong)
+        {
+            return;
+        }
+        const auto weak = ctx->page;
+        strong->Dispatcher().RunAsync(
+            winrt::Windows::UI::Core::CoreDispatcherPriority::Normal,
+            [weak]() {
+                const auto p = weak.get();
+                if (!p)
+                {
+                    return;
+                }
+                // If we were disarmed between callback fire and dispatch
+                // resolving, bail — a deliberate teardown is already in
+                // flight (or finished) and we'd otherwise tear down a
+                // freshly-built replacement pane.
+                if (!p->_agentPaneWtaWait)
+                {
+                    return;
+                }
+                _agentPaneLog("agent pane wta process exited — tearing down pane");
+                p->_agentPanePreWarming = false;
+                p->_TeardownAgentPane(); // closes job → reaps surviving children
+                p->_UpdateBottomBarState();
+            });
+    }
+    CATCH_LOG()
+
     // Close the shared agent pane if it is still alive.
     void TerminalPage::_TeardownAgentPane()
     {
+        // Disarm + close job FIRST. Closing the job kills wta and every
+        // descendant (KILL_ON_JOB_CLOSE) so conpty actually sees its
+        // pipe go EOF, ControlCore transitions through Closed, and the
+        // existing teardown path runs cleanly.
+        _TearDownAgentPaneWtaWatch();
         if (auto p = _agentPane.lock())
         {
             _agentPaneLog("_TeardownAgentPane: closing agent pane");
@@ -1849,6 +2039,12 @@ namespace winrt::TerminalApp::implementation
                 _agentPaneLog("agent pane closed — _agentPane cleared");
                 if (auto self = weakSelf.get())
                 {
+                    // Backstop disarm — the pane can be closed through
+                    // generic pane-close paths (Ctrl+W, tab close, etc.)
+                    // that don't go through _TeardownAgentPane. Without
+                    // this, the wta job + wait would stay armed and the
+                    // process tree would leak.
+                    self->_TearDownAgentPaneWtaWatch();
                     self->_agentPane.reset();
                     self->_agentPanePreWarming = false;
                     self->_lastNotifiedAgentTabId.reset();
@@ -1913,6 +2109,21 @@ namespace winrt::TerminalApp::implementation
                 {
                     return;
                 }
+                // wta is now running (connection.Start() ran inside
+                // _InitializeTerminal just before Initialized fired). Arm
+                // the job + process-exit watch before doing anything else
+                // so we can't miss an early-crash scenario.
+                if (const auto tc = termControlWeak.get())
+                {
+                    if (const auto conn = tc.Connection())
+                    {
+                        if (const auto conpty = conn.try_as<winrt::Microsoft::Terminal::TerminalConnection::ConptyConnection>())
+                        {
+                            const auto raw = reinterpret_cast<HANDLE>(conpty.RootProcessHandle());
+                            self->_SetupAgentPaneWtaWatch(raw);
+                        }
+                    }
+                }
                 // Pre-warm has done its job: connection.Start() ran inside
                 // _InitializeTerminal just before this event, so wta.exe is
                 // launching. Drop the reconcile guard before doing anything
@@ -2384,6 +2595,10 @@ namespace winrt::TerminalApp::implementation
                 _agentPaneLog("agent pane closed — _agentPane cleared");
                 if (auto self = weakSelf.get())
                 {
+                    // Backstop disarm — see _AutoCreateHiddenAgentPane's
+                    // Closed handler for rationale. Generic pane-close
+                    // paths don't route through _TeardownAgentPane.
+                    self->_TearDownAgentPaneWtaWatch();
                     self->_agentPane.reset();
                     self->_lastNotifiedAgentTabId.reset();
                     self->_agentSessionsViewActive = false;
@@ -2393,6 +2608,41 @@ namespace winrt::TerminalApp::implementation
             });
         }
 
+        // Arm the wta process watch + Job Object for this newly-created
+        // agent pane too (the auto-create path hooks Initialized to do the
+        // same thing — see _AutoCreateHiddenAgentPane).
+        if (const auto termControl = newPane->GetTerminalControl())
+        {
+            auto weakSelfForWatch = get_weak();
+            auto tokenHolder = std::make_shared<winrt::event_token>();
+            *tokenHolder = termControl.Initialized([
+                weakSelfForWatch,
+                termControlWeak = winrt::make_weak(termControl),
+                tokenHolder
+            ](auto&&, auto&&) {
+                if (const auto tc = termControlWeak.get())
+                {
+                    tc.Initialized(*tokenHolder);
+                }
+                const auto self = weakSelfForWatch.get();
+                if (!self)
+                {
+                    return;
+                }
+                if (const auto tc = termControlWeak.get())
+                {
+                    if (const auto conn = tc.Connection())
+                    {
+                        if (const auto conpty = conn.try_as<winrt::Microsoft::Terminal::TerminalConnection::ConptyConnection>())
+                        {
+                            const auto raw = reinterpret_cast<HANDLE>(conpty.RootProcessHandle());
+                            self->_SetupAgentPaneWtaWatch(raw);
+                        }
+                    }
+                }
+            });
+        }
+
         const auto& activeTab = _GetFocusedTabImpl();
         if (!activeTab)
         {

src/cascadia/TerminalApp/TerminalPage.h

@@ -634,6 +634,40 @@ namespace winrt::TerminalApp::implementation
         std::vector<std::shared_ptr<AgentDelegationEntry>> _agentPipeServers;
         void _RemoveAgentPipeServer(AgentDelegationEntry* entry);
 
+        // Agent-pane wta lifetime management. Solves two problems at once:
+        //
+        // (1) Detect wta death (crash / external taskkill / clean exit) so we
+        //     tear the pane down. Conpty StateChanged can't be used because
+        //     wta spawns long-lived children (npx → npm → node → ACP adapter)
+        //     that inherit the conpty client handles, so the pipe never EOFs
+        //     even when wta itself is gone.
+        // (2) Reclaim those orphaned children. Without explicit job
+        //     containment, killing wta leaves node.exe / npx running.
+        //
+        // _agentPaneJob: KILL_ON_JOB_CLOSE Job Object containing wta and (by
+        // inheritance) all its descendants. Resetting the handle terminates
+        // every member — used during teardown to sweep orphans, and (via OS)
+        // when WT itself exits abnormally. Only the agent pane uses a job;
+        // ordinary shell panes keep their existing free-running lifetime.
+        //
+        // _agentPaneWtaHandle: duplicated SYNCHRONIZE handle to wta. Owned
+        // independently of ConptyConnection's _piClient so the wait stays
+        // valid across teardown ordering.
+        //
+        // _agentPaneWtaWait + _agentPaneWtaWaitContext: RegisterWaitForSingleObject
+        // registration. Context is heap-allocated (so the threadpool callback
+        // has stable storage even if TerminalPage tears down before the wait
+        // unregisters); it carries a weak_ref + cancelled flag so we can race
+        // safely with deliberate teardown.
+        struct AgentPaneWtaWaitContext;
+        wil::unique_handle _agentPaneJob;
+        wil::unique_handle _agentPaneWtaHandle;
+        HANDLE _agentPaneWtaWait{ nullptr };
+        std::unique_ptr<AgentPaneWtaWaitContext> _agentPaneWtaWaitContext;
+        void _SetupAgentPaneWtaWatch(HANDLE wtaProcessHandle) noexcept;
+        void _TearDownAgentPaneWtaWatch() noexcept;
+        static void NTAPI _OnAgentPaneWtaExit(PVOID context, BOOLEAN timedOut) noexcept;
+
         // Pending pipe handles for the next agent-pane wta launch. Set by
         // _OpenOrReuseAgentPane just before _MakeTerminalPane runs, consumed
         // by _CreateConnectionFromSettings when it builds the ConptyConnection