Skip to content

feat(experiment): paper-ready reproducibility archive (experiment bundle)#73

Open
NicolasSchuler wants to merge 5 commits into
mainfrom
feat/experiment-bundle
Open

feat(experiment): paper-ready reproducibility archive (experiment bundle)#73
NicolasSchuler wants to merge 5 commits into
mainfrom
feat/experiment-bundle

Conversation

@NicolasSchuler

Copy link
Copy Markdown
Owner

Roadmap item 1.3: one command emits a citeable reproducibility tarball for a tracked run.

hpc-compose experiment bundle [JOB_ID] -f compose.yml [--output PATH] [--dir] [--strict] [--format text|json]

Approved sketch (implemented as designed unless noted)

Bundle layout (staged dir, then deterministic tar.gz; --dir keeps the directory instead):

experiment-bundle-<job_id>/
  README.md                  # generated "methods appendix"
  MANIFEST.json              # schema_version, tool version, per-file sha256, missing[] {item, reason}
  spec/compose.yaml          # copy of record.compose_file
  spec/config.snapshot.yaml  # record.config_snapshot_yaml (already secret-redacted)
  spec/spec-drift.diff       # only when the current spec drifted since submit
  scripts/job.sbatch         # record.script_path from disk; re-rendered + marked when gone
  provenance/record.json     # full SubmissionRecord (git sha/dirty/branch, tags, notes)
  provenance/artifacts-manifest.json  # artifact inventory — never payloads (that's `artifacts`)
  sweep/manifest.json        # sweep trials: joined via sweep_id (seeds live here, not on the record)
  metrics/stats.csv          # stats snapshot as CSV
  metrics/raw/*.jsonl        # gpu/cpu/steps sampler files as-is
  checkpoints/history.json   # checkpoint attempt history

README methods appendix: run identity, submit timestamp, tool version, git SHA + dirty + branch, image references table, resources from the config snapshot, sweep id/trial/variables/seed, attempt history, metrics summary, tags/notes, artifact inventory, and a "Reproducing this run" section.

Degradation policy: every missing ingredient (no snapshot on run-style/legacy records, no metrics, script gone, no provenance, …) warns and lands in MANIFEST.json missing[]; the command still succeeds. --strict fails (after finalizing the archive) when anything is missing.

Design decisions (approved by Nicolas):

  • Image references, not digests: nothing in the codebase resolves container digests, and resolving at bundle time would need network (bad on air-gapped login nodes) and reflect bundle time, not submit time. The bundle ships refs as recorded with an explicit "references, not digests" note in the appendix.
  • Sweep seeds are only in the sweep manifest, so sweep-trial bundles join record → manifest via sweep_id.
  • Drift detection reuses the diff --against-spec machinery (including the sweep-trial interpolation overlay) and is best-effort: an unloadable current spec becomes a missing[] entry, not a failure.

Implementation notes

  • Tar/hash helpers (append_path_to_tar, hash_file) promoted to pub(crate) in src/job/artifacts.rs + new write_tree_tarball; deterministic sorted entries.
  • ExperimentBundleOutput DTO follows the experiment-family precedent (hand-rolled schema_version, not flatten_envelope!); registered as "experiment-bundle"; schema/outputs/experiment-bundle.schema.json blessed and committed.
  • write_stats_snapshot_csv was already generic over impl Write — rendered into a buffer, no refactor needed.
  • ArtifactManifest carries no per-file sha256, so the README artifact inventory lists names/paths only.
  • The JSON files ledger includes MANIFEST.json itself; the on-disk MANIFEST.json files[] necessarily excludes itself (cannot hash its own final bytes).

Tests

Seven e2e tests (happy path with tar extraction, degraded, --strict, sweep seed join, --format json, --dir, drift) plus the schema gates. Full suite green on the branch (1445 tests, 22 targets); fmt/clippy/typos clean; manpages regenerated by the integrator and --check verified.

Per the fast-mode agreement: no pre-PR adversarial review; batch review at the end of the push.

🤖 Generated with Claude Code

NicolasSchuler and others added 5 commits July 5, 2026 21:08
Promote `append_path_to_tar` and `hash_file` to `pub(crate)`, extract the
gzip tar builder open/finish into `create_tar_builder`/`finish_tar_builder`,
and add `write_tree_tarball` for archiving an entire directory tree with
deterministic sorted entries. `write_bundle_tarball` now reuses the shared
helpers instead of inlining the encoder plumbing. These back the upcoming
`experiment bundle` command without duplicating the artifact-export format.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`hpc-compose experiment bundle [JOB_ID]` emits a citeable, paper-ready
archive for one tracked run: the compose spec, the resolved config snapshot
(secret-redacted), the rendered sbatch (re-rendered from the plan and marked
"reconstructed" when the on-disk script is gone), the full submission record,
provenance (git SHA + dirty + per-service image references), the sweep
manifest with seeds for sweep trials, metrics (stats.csv + raw *.jsonl),
checkpoint attempt history, and a generated README.md methods appendix,
alongside a MANIFEST.json with per-file sha256 and a missing[] ledger. A
spec/spec-drift.diff is included only when the current spec differs from the
snapshot.

Image entries are references as recorded at submit time, not content digests,
and are never resolved against a registry (stated plainly in the appendix).
Output defaults to experiment-bundle-<job_id>.tar.gz; --dir writes an unpacked
directory, --strict fails (after reporting) when any ingredient is missing,
and --format json is pinned by the new `experiment-bundle` output schema.
Defaults to the latest tracked run and contacts the scheduler only as much as
`stats` does. Bundle is a subcommand, so MAX_TOP_LEVEL_COMMANDS is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Exercises the real binary end to end (extracting the tarball with `tar`):
happy path with a full record (asserts every staged file, MANIFEST sha256
ledger, empty missing[], README caveat/identity, CSV header); degrade path
with no snapshot or metrics (exit 0, warnings, missing[] entries, README
still generated); --strict failure naming the missing ingredient; a
sweep-trial bundle carrying sweep/manifest.json and the trial seed in the
README; --format json (schema_version present, files array matches disk);
--dir layout (no tarball written); and a drift case (spec-drift.diff present,
drift_detected true, README flags drift).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a CHANGELOG [Unreleased] entry, a "Bundle a run for reproducibility"
section in runtime-observability.md with the full layout table and the
references-not-digests caveat, and a cli-reference.md row (plus an updated
`experiment` parent-command summary).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 5, 2026 19:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new hpc-compose experiment bundle subcommand that emits a “paper-ready” reproducibility bundle (tar.gz or directory) for a tracked run, including spec + snapshot, sbatch script (with reconstruction fallback), provenance, sweep seed join, metrics, checkpoint history, and a generated README/manifest, plus a pinned JSON output schema.

Changes:

  • Implement experiment bundle with best-effort ingredient collection, missing[] degradation, optional --strict, and text/JSON output.
  • Promote/reuse tar + hashing helpers and register a new experiment-bundle output schema contract.
  • Add end-to-end CLI tests plus documentation/manpage/changelog updates covering the new command and bundle layout.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/cli_runtime.rs Adds E2E coverage for bundle creation (tar + dir), strict/degraded modes, sweep join, JSON output, and drift detection.
src/output/contract.rs Registers experiment-bundle in the output schema contract and test allowlist.
src/job/mod.rs Re-exports crate-internal tar/hash helpers for reuse by bundling logic.
src/job/artifacts.rs Promotes hash_file, factors tar builder helpers, and adds write_tree_tarball for bundling.
src/commands/runtime/experiment.rs Implements experiment bundle end-to-end: staging, ingredient capture, README + MANIFEST generation, tar/directory finalize, strict failure behavior.
src/commands/mod.rs Wires the new CLI subcommand to the runtime implementation.
src/cli/help.rs Adds help text for experiment bundle.
src/cli/commands.rs Defines the experiment bundle CLI surface (flags, docs, help wiring).
schema/outputs/experiment-bundle.schema.json Adds the pinned JSON schema for --format json output.
man/man1/hpc-compose-experiment.1 Adds bundle to the experiment subcommand list.
man/man1/hpc-compose-experiment-tag.1 Updates SEE ALSO to include the new bundle command.
man/man1/hpc-compose-experiment-show.1 Updates SEE ALSO to include the new bundle command.
man/man1/hpc-compose-experiment-note.1 Updates SEE ALSO to include the new bundle command.
man/man1/hpc-compose-experiment-bundle.1 New manpage for hpc-compose experiment bundle.
docs/src/runtime-observability.md Documents bundle use cases, layout, degradation policy, and semantics.
docs/src/cli-reference.md Adds experiment bundle to the CLI reference table and description.
CHANGELOG.md Notes the new command, defaults, bundle contents, and strict/degradation behavior.

Comment on lines +850 to +858
fn stage_write(staged_dir: &Path, relative: &str, bytes: &[u8]) -> Result<()> {
let path = staged_dir.join(relative);
if let Some(parent) = path.parent() {
fs::create_dir_all(parent)
.with_context(|| format!("failed to create {}", parent.display()))?;
}
crate::secure_io::write_atomic(&path, bytes, false)
.with_context(|| format!("failed to write {}", path.display()))
}
Comment thread src/job/artifacts.rs
Comment on lines +506 to +513
fn create_tar_builder(tarball_path: &Path) -> Result<Builder<GzEncoder<File>>> {
if let Some(parent) = tarball_path.parent() {
fs::create_dir_all(parent).context(format!("failed to create {}", parent.display()))?;
}
let file = File::create(tarball_path)
.context(format!("failed to create {}", tarball_path.display()))?;
let encoder = GzEncoder::new(file, Compression::default());
let mut builder = Builder::new(encoder);
for relative in copied_relative_paths {
let source = bundle_root.join(relative);
if !source.exists() && fs::symlink_metadata(&source).is_err() {
continue;
}
append_path_to_tar(&mut builder, &source, relative)?;
}
Ok(Builder::new(encoder))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants