Expose workflow lineage id on WorkflowMetadata#7145
Draft
pditommaso wants to merge 1 commit into
Draft
Conversation
Add a `lineageId` property on `WorkflowMetadata` populated by `LinObserver` from `onFlowCreate` (previously the lid was computed in `onFlowBegin`). Computing in `onFlowCreate` guarantees the value is set on the shared metadata before any other observer's `onFlowBegin` fires (e.g. `TowerObserver`, which can now propagate the lid to Seqera Platform without a secondary `/lineage/resolve` round-trip). The value is the canonical `lid://<hash>` URI; null when the nf-lineage plugin is not active. Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
✅ Deploy Preview for nextflow-docs-staging canceled.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Expose the workflow's lineage identifier as a first-class property on
WorkflowMetadata, populated byLinObserverat flow-create time. Other observers (notably the Tower observer) can now readsession.workflowMetadata.lineageIdand forward it to downstream systems without computing it themselves or making a secondary API round-trip to resolve it.Concretely:
WorkflowMetadatagains a publicString lineageIdfield. ReflectivetoMap()picks it up automatically.LinObserver.onFlowBegin()moves toLinObserver.onFlowCreate(Session). Lifecycle innextflow.Session:init()constructs observers, thenworkflowMetadata;start()then callsnotifyFlowCreate()(firesonFlowCreateon every observer) beforefireDataflowNetwork()callsnotifyFlowBegin(). Computing the hash inonFlowCreatetherefore (a) hassession.workflowMetadataavailable, and (b) completes before any observer'sonFlowBeginruns — regardless of plugin discovery order.WorkflowRunvalue, the history log write, and the in-memoryWorkflowOutputinitialisation move with it.lid://<hash>URI; null when thenf-lineageplugin is not active.This is a building block for a Seqera Platform-side change (PLAT-5235) that will persist
Workflow.lineageIdon the workflow entity at trace-begin time, replacing the current secondaryGET /lineage/resolve?sessionId=…lookup. That PR will land separately.Test plan
./gradlew :nf-lineage:test— full module suite green, including an added assertion inLinObserverTest."should save workflow"that verifiessetLineageId(_)is invoked on the metadata with alid://-prefixed URI../gradlew :plugins:nf-tower:test— green (no source changes there; just verifying the moved lifecycle hook doesn't regress the consumer that readsworkflowMetadata.toMap())../gradlew :nextflow:test --tests "*WorkflowMetadata*"— green.Draft
Marked as draft because the matching Seqera Platform change (PR to follow, tracked under PLAT-5235) hasn't landed yet. The Nextflow side is forward-compatible on its own — older Platform versions just ignore the extra field — so this can merge independently once reviewed.