Skip to content

Fix FSEvents delivery liveness - #713

Open
baron wants to merge 1 commit into
mainfrom
wt/fix-fsevents-delivery-liveness
Open

Fix FSEvents delivery liveness#713
baron wants to merge 1 commit into
mainfrom
wt/fix-fsevents-delivery-liveness

Conversation

@baron

@baron baron commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • replace all blocking FSEventStreamFlushSync calls in watcher activation, metadata monitoring, and worktree creation witnessing with asynchronous event-ID delivery barriers
  • scope delivery watermarks to stream generations, reject stale callbacks after restart or journal wrap, and tear streams down on their callback queues
  • remove the global flush circuit breaker/quarantine machinery and replace timing-based tests with injected delivery/deadline oracles

Why

The shard-2 worktree API smoke reached create-start.tab-ready and then timed out at 180 seconds. That leg uniquely enters the worktree-creation witness and watcher activation paths. Those paths used synchronous FSEvents flushes that can block indefinitely when the host does not deliver the cut. This is a production liveness bug exposed by the integration smoke, not a concurrency condition the test should tolerate.

The new contract is deterministic: the callback observes the exact asynchronous flush target for the current stream generation, or the optimization fails closed to its existing fallback. No sleeps, timeout inflation, CI workflow changes, global serialization, or reduced concurrency.

Validation

  • WorktreeAPISmokeHarnessTests/testManageWorktreeAndAgentRunAPISmokeFlow (7.392s)
  • FileSystemAcceptedIngressBarrierTests (21 executed, 1 skipped, 0 failures)
  • WorkspaceRootCreationReceiptCoordinatorTests (11/11)
  • GitLoadedRootAuthorityEvidenceTests (48 executed, 2 skipped, 0 failures)
  • AgentRunWorktreeStartTests (51/51)
  • GitWorkspaceStateAuthorityTests (8/8)
  • make dev-swift-build PRODUCT=RepoPrompt
  • make dev-lint
  • commit and push contribution preflights

A full local root-suite attempt was not counted as green: it stopped making progress in the unrelated CodemapBindingEngineRootLeaseTests/testSequentialRootsRetainGlobalAdoptionLeaseBudgetUntilUnloadThenRecover. Fresh exact-head hosted CI is required before merge.

@baron

baron commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator Author

Triage: current-main textual integration is clean and the historical failure was unrelated, but this is a broad state-sensitive FSEvents generation/liveness change with no human review. As author I cannot self-approve; please obtain non-author lifecycle review and fresh hosted integration evidence.

@baron

baron commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator Author

Blocking current-main recheck on exact head 20f28670b1a8e488104164ffbf2bef9954ac14e2.

App shard 1 fails StoreBackedWorkspaceSearchTests.testMutationAfterCatalogCrawlBeforeWatcherStartIsVisibleToFreshSearch (expected Late.swift, got nil). That test exercises the crawl-to-watcher replay/liveness invariant this PR changes through FSEventStreamFlushAsync and the new callback barrier, so it cannot be classified as unrelated without a root-cause investigation.

Please reproduce/diagnose the missed mutation, fix the owning boundary, and rerun exact-current-base CI. Do not add sleeps, retries, or timeout inflation. I am commenting rather than submitting REQUEST_CHANGES because the authenticated reviewer is also the PR author.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant