Lineage TaskRun record omits cache-determining inputs (module, eval, stub), so 'lineage diff' can't explain those cache misses

[pinin4fjords]

Bug report

The data-lineage docs present nextflow lineage diff as a way to understand why -resume re-ran a task. However, the lineage TaskRun record stores only a subset of the inputs that go into the task cache hash, so for changes in the omitted inputs the diff shows no meaningful difference even though the task was correctly re-executed.

The cache hash key (TaskHasher.compute()) includes, among others:

• environment module directives (task.config.getModule())
• eval output commands (getOutputEvals(), added in Pipeline uses cached process despite edited eval script #5470)
• the stub-run flag

None of these has a corresponding field in the lineage TaskRun model (modules/nf-lineage/src/main/nextflow/lineage/model/v1beta1/TaskRun.groovy), which stores: sessionId, name, codeChecksum, script, input, container, conda, spack, architecture, globalVars, binEntries, workflowRun.

The module omission is the clearest: the record captures conda, spack, container and architecture but not module, even though all are environment inputs to the same hash.

Steps to reproduce

main.nf:

process GREET {
    module 'samtools/1.9'
    input:  val name
    output: path 'out.txt'
    script: """ echo "Hello ${name}" > out.txt """
}
workflow { GREET(Channel.of('world')) }

nextflow.config:

lineage.enabled = true

1. nextflow run main.nf -dump-hashes
2. Change only samtools/1.9 to samtools/1.17
3. nextflow run main.nf -resume -dump-hashes — the task correctly re-runs

-dump-hashes pinpoints the change:

0a5461eb… [java.lang.String] samtools/1.9     (run 1)
bd743eeb… [java.lang.String] samtools/1.17    (run 2)

nextflow lineage diff <run1-taskrun> <run2-taskrun> shows only the parent run id changing — nothing about the module:

-    "workflowRun": "lid://f54deed4…"
+    "workflowRun": "lid://4c89b0ed…"

The task LID is the task hash, so the two records necessarily have different LIDs, yet the diff surfaces no field that explains it.

Expected behaviour

The TaskRun record should capture the cache-hash inputs it currently omits, so that comparing two task runs reflects what actually invalidated the cache:

• module (sits naturally beside conda / spack / container / architecture)
• eval output commands
• a stub-run indicator (partially covered today, since codeChecksum switches to the stub source under stub runs)

Note this would make lineage diff consistent with the cache hash, but not a full substitute for -dump-hashes: the record stores resolved values rather than the ordered hash-key list, and lineage DataPath checksums are always computed with the default hash mode, so under deep/lenient cacheMode the record can still diverge from the cache decision. -dump-hashes remains the authoritative tool.

Environment

• Nextflow version: confirmed on 26.04.3 (release) and reproduced in master source (TaskHasher keys on module/eval/stub; the TaskRun v1beta1 model has fields for none of them)
• Java version: 21
• Lineage: lineage.enabled = true

Related

• Document the lineage data structure #6250 (documenting the data structure / JSON schema — the model should be complete before a schema is pinned)
• Generate lineage schema from source code #7159 (generate lineage schema from source — same reason)
• Add platform-related metadata to WorkflowRun lineage record #6545 (precedent: enriching lineage records with extra metadata)
• [feature] Resume: print cache invalidation reason to console #4894 (broader "print cache invalidation reason" UX)

A PR adding the module field follows; eval and stub are left for discussion since their representation needs a design decision.

[pinin4fjords]

PR #7203 adds the module field. eval output commands and the stub-run flag remain open here pending a design decision on representation.

[jorgee]

Apart from #7203, I have created #7211 to fix an issue with -stub-run (an error preventing the creation of the TaskRun entry) and add the eval field in the TaskRun entry to store output evals.

Regarding the -stub-run flag, I think we do not need to add it in the lineage entry. The extra stub flag in the hash is only added if the workflow execution is a stub run and the stub block is defined. In this case, the source field of TaskRun will be the stub code. I only see a rare case where it could matter. It is when defining a process where both script and stub blocks contain the same code.