Add nextflow lineage validate command (first pass)#7168
Draft
edmundmiller wants to merge 10 commits into
Draft
Conversation
2 tasks
Normalizes lineage data by stripping ephemeral fields (sessionId, timestamps, paths, LID refs) that vary between runs but don't affect semantic equivalence. Used for comparing workflow runs. Signed-off-by: Edmund Miller <edmund.miller@seqera.io>
Adds 'nextflow lineage validate <lid> --against <baseline>' command to compare two workflow runs semantically. Ignores ephemeral fields and shows diff on mismatch. Exits non-zero if runs differ. Signed-off-by: Edmund Miller <edmund.miller@seqera.io>
Tests for CLI validate: equivalent runs, different checksums, different params, missing --against arg, --ignore-fields option. Signed-off-by: Edmund Miller <edmund.miller@seqera.io>
Provides snapshot testing for pipeline validation in Spock tests: - assertMatchesSnapshot(lid, snapshotId) - compare against baseline - assertEquivalent(lid1, lid2) - compare two runs directly - UPDATE_SNAPSHOTS=true env var to update baselines Signed-off-by: Edmund Miller <edmund.miller@seqera.io>
End-to-end tests exercising validate command with workflow runs: equivalent runs with outputs, different checksums, different params, different output counts, nested outputs, ignore-fields option. Signed-off-by: Edmund Miller <edmund.miller@seqera.io>
Tests for 'nextflow lineage validate' command: - lineage-validate-basic: identical runs should be equivalent - lineage-validate-changes: detect param/output differences - lineage-validate-multi: multi-process pipeline validation - lineage-validate-resume: resume mode produces equivalent lineage Tests use temp lineage stores, are conditional on lineage being available, and follow existing tests/ structure with .checks scripts. Signed-off-by: Edmund Miller <edmund.miller@seqera.io>
Asserts that `nextflow lineage validate <lid> -against <lid>` (and the related -ignore-fields / -output-base) parse through Launcher's JCommander wiring. Marked @PendingFeature because CmdLineage does not yet declare these options, so JCommander rejects them with "Unknown option". The existing CmdLineageTest and LinValidateIntegrationTest both bypass JCommander by constructing commands directly, so they cannot catch this. Signed-off-by: Edmund Miller <edmund.miller@seqera.io>
JCommander 1.35 (used by Nextflow) rejected `-against`, `-ignore-fields` and `-output-base` because the options were not declared on CmdLineage. Declare them with single-dash names (matching the rest of Nextflow's CLI) and forward them into the sub-command args list so LinCommandImpl.validate keeps its existing `--`-prefixed parser unchanged. Removes the @PendingFeature markers added in the previous commit. Signed-off-by: Edmund Miller <edmund.miller@seqera.io>
Add @title, @Narrative, @subject, and @see annotations to LinNormalizerTest, LinValidateIntegrationTest, LineageSnapshotterTest, and the validate-specific specs in LinCommandImplTest. Each method-level @see points at the ADR decision section the spec exercises so future readers can trace any change in behaviour back to the decision that motivates it. Signed-off-by: Edmund Miller <edmund.miller@seqera.io>
Replace six hand-rolled three-pass log-filter chains with the existing test.TestHelper.filterLogNoise helper (already imported in LinCommandImplTest for the legacy tests). Cuts ~30 lines. Leave TODO/FIXME markers on three pre-existing issues surfaced during review but out of scope for this change: unused @tempdir in LinNormalizerTest, @shared fields reassigned per-test in LinValidateIntegrationTest, and the Workflow/WorkflowRun/FileOutput fixture-construction pattern that repeats ~25 times. Signed-off-by: Edmund Miller <edmund.miller@seqera.io>
edmundmiller
force-pushed
the
lineage-validate
branch
from
9cc0b90 to
10f9c23
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First-pass implementation of
nextflow lineage validateper the design in #7167 (stacked on that PR). Ships the v1 surface so we can use it in practice and feed back into the design.What's in:
LinNormalizer— encodes lineage records, strips ephemeral fields, supports--output-baserelativization and--ignore-fields; provides recursive structural compare with diff pathsnextflow lineage validate <lid> --against <lid>— CLI command with-against,-ignore-fields,-output-baseoptions; emits JGit-style unified diff and exits non-zero on divergenceLineageSnapshotter— Spock helper for pipeline tests; supports save-or-compare withUPDATE_SNAPSHOTSenv var, plus directassertEquivalent(lidA, lidB)--ignore-fields, and diff renderingKnown gaps vs the ADR (deferred to follow-ups):
LineageValidatorcore yet; CLI andLineageSnapshotterstill have parallel logic--against <file.json>) or--save-snapshot--json/--summary/ CI auto-detectionLineageResolverSPI; onlylid://works todayTreat this as a working spike to validate the design against real lineage data — feedback expected to inform the v1.1 refactor.
Test plan
./gradlew :nf-lineage:test --tests LinValidateIntegrationTest./gradlew :nextflow:test --tests LauncherTest(regression for JCommander wiring)nextflow lineage validate lid://<runA> -against lid://<runB>, confirm divergence on params/outputs and equivalence on identical runs-ignore-fields commitIdto verify field exclusion🤖 Generated with Claude Code
Stack