feat(runtime): persist a per-subscription log cursor for gap-free resume across restarts#301
Merged
Merged
Conversation
5c65ca3 to
4568808
Compare
A `[[subscription]] resume = true` chain-log subscription now survives an engine restart: the host persists the last dispatched block per subscription and re-opens the poller from there (clamped by `MAX_SYNC_BACK_BLOCKS`) instead of at head, so logs mined while the process was down are re-delivered rather than lost. - Manifest gains `resume: bool` (default false) on the chain-log subscription; whether a subscription needs gap-free resume is module-specific, so it is opt-in and declarative. - The cursor is host-owned, keyed by `keccak(chain_id | address | topic)` in the module's store namespace - not the alloy `Filter`, whose hash uses a process-randomized `HashSet` and is not reproducible across restarts. It is read once at boot and written by the supervisor only after a successful dispatch (never on the task's buffered send), so a block is never recorded as done before the module processed it. - Delivery is at-least-once: the boot resume replays the cursor block in full, deduped by the module's chassis idempotency journal. Exactly-once (a transactional cursor + state commit) is out of scope. Closes #300
4568808 to
2dec80c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Makes chain-log delivery gap-free across engine restarts, not just within a process. A module opts a subscription in with one manifest flag:
The host then persists a durable cursor (the last dispatched block for that subscription) and, on boot, re-opens the
watch_canonical_logs_frompoller from that block instead of at the current head, clamped byMAX_SYNC_BACK_BLOCKS.Why
Closes #300. #299 made log delivery gap-free within a process (the canonical poller + retry layer + sync-back-on-reopen). Across a process restart the poller still opened at head, dropping logs mined during downtime. This closes that gap for subscriptions that want it.
Design (host owns the cursor, module owns idempotency)
keccak(chain_id | address | topic)under the module's own store namespace, alongside the existinglast_dispatched_blockblock marker. It is not derived from the alloyFilter:Filter'sHashuses a process-randomizedHashSet, so it is not reproducible across restarts. The module never manages a cursor or a boot handshake.last_seen_blockadvances when it enqueues a log, but a crash with logs still buffered would record un-dispatched blocks as done. So the supervisor writes the cursor only after a successfuldispatch_chain_log, mirroring the block marker. Single writer (supervisor), one-shot boot read (supervisor); the spawned task never touches the store.resume = falsereproduces today's behaviour exactly (initial_cursor = None-> poller starts at head, no history replay).Testing
cargo clippy --workspace --all-targets --all-features --locked -- -D warningsclean,cargo test -p nexum-runtime(206 passed, includingchainlog_cursor_keystability/uniqueness tests), fullcargo build --workspace, fmt, no em-dashes, all under the pinned 1.94 toolchain.Depends on
#299 (the canonical poller,
poller_resume_block, andMAX_SYNC_BACK_BLOCKS). This PR is stacked on it.AI Assistance
Claude Code used for the seam mapping, the implementation, and this description.