Skip to content

Expose workflow lineage id on WorkflowMetadata#7145

Draft
pditommaso wants to merge 1 commit into
masterfrom
add-lineage-id-to-workflow-metadata
Draft

Expose workflow lineage id on WorkflowMetadata#7145
pditommaso wants to merge 1 commit into
masterfrom
add-lineage-id-to-workflow-metadata

Conversation

@pditommaso
Copy link
Copy Markdown
Member

Summary

Expose the workflow's lineage identifier as a first-class property on WorkflowMetadata, populated by LinObserver at flow-create time. Other observers (notably the Tower observer) can now read session.workflowMetadata.lineageId and forward it to downstream systems without computing it themselves or making a secondary API round-trip to resolve it.

Concretely:

  • WorkflowMetadata gains a public String lineageId field. Reflective toMap() picks it up automatically.
  • The LID computation that used to live in LinObserver.onFlowBegin() moves to LinObserver.onFlowCreate(Session). Lifecycle in nextflow.Session: init() constructs observers, then workflowMetadata; start() then calls notifyFlowCreate() (fires onFlowCreate on every observer) before fireDataflowNetwork() calls notifyFlowBegin(). Computing the hash in onFlowCreate therefore (a) has session.workflowMetadata available, and (b) completes before any observer's onFlowBegin runs — regardless of plugin discovery order.
  • The store save of the WorkflowRun value, the history log write, and the in-memory WorkflowOutput initialisation move with it.
  • Value is the canonical lid://<hash> URI; null when the nf-lineage plugin is not active.

This is a building block for a Seqera Platform-side change (PLAT-5235) that will persist Workflow.lineageId on the workflow entity at trace-begin time, replacing the current secondary GET /lineage/resolve?sessionId=… lookup. That PR will land separately.

Test plan

  • ./gradlew :nf-lineage:test — full module suite green, including an added assertion in LinObserverTest."should save workflow" that verifies setLineageId(_) is invoked on the metadata with a lid://-prefixed URI.
  • ./gradlew :plugins:nf-tower:test — green (no source changes there; just verifying the moved lifecycle hook doesn't regress the consumer that reads workflowMetadata.toMap()).
  • ./gradlew :nextflow:test --tests "*WorkflowMetadata*" — green.

Draft

Marked as draft because the matching Seqera Platform change (PR to follow, tracked under PLAT-5235) hasn't landed yet. The Nextflow side is forward-compatible on its own — older Platform versions just ignore the extra field — so this can merge independently once reviewed.

Add a `lineageId` property on `WorkflowMetadata` populated by
`LinObserver` from `onFlowCreate` (previously the lid was computed in
`onFlowBegin`). Computing in `onFlowCreate` guarantees the value is set
on the shared metadata before any other observer's `onFlowBegin` fires
(e.g. `TowerObserver`, which can now propagate the lid to Seqera
Platform without a secondary `/lineage/resolve` round-trip).

The value is the canonical `lid://<hash>` URI; null when the nf-lineage
plugin is not active.

Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
@netlify
Copy link
Copy Markdown

netlify Bot commented May 14, 2026

Deploy Preview for nextflow-docs-staging canceled.

Name Link
🔨 Latest commit 84f5755
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/6a05ea9bad43330008db68bf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant