refactor(dfir_lang): remove stratum, add push codegen, test fold, fold_keyed, fold_no_replay

[MingweiSamuel]

No description provided.

[cloudflare-workers-and-pages]

Deploying hydro with    Cloudflare Pages

Latest commit:	9868d54
Status:	⚡️  Build in progress...

View logs

[MingweiSamuel]

Diff view: 571efde

[Copilot]

Pull request overview

This PR refactors DFIR language/codegen and pipes to support push-side codegen for blocking/stateful operators (notably fold, fold_keyed, and fold_no_replay), removing the prior stratum-based scheduling requirement and adding new push-side combinators plus tests to validate behavior.

Changes:

• Add unified push-side accumulator infrastructure (Accumulate + AccumState) and new push combinators (Sort, FoldKeyed, ReduceKeyed), then wire DFIR codegen to use them for push-side fold* operators.
• Update/extend tests to cover push-side blocking behavior and adjust existing metrics expectations.
• Minor utility/API improvements (extra derives) and CI/precheck target updates.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
precheck.bash	Adds hydro_test_embedded to the hydro test targets run by precheck.
dfir_rs/tests/surface_push_blocking.rs	New tests ensuring fold, fold_keyed, and fold_no_replay behave correctly when they end up on the push side of a subgraph.
dfir_rs/tests/metrics.rs	Updates the handoff-metrics test pipeline and expected output.
dfir_rs/tests/compile-fail/stable/surface_fold_keyed_generics_bad.stderr	Updates expected compiler stderr output for trybuild tests.
dfir_rs/tests/compile-fail/stable/surface_fold_keyed_badtype_option.stderr	Updates expected compiler stderr output for trybuild tests.
dfir_rs/tests/compile-fail/stable/surface_fold_keyed_badtype_int.stderr	Updates expected compiler stderr output for trybuild tests.
dfir_rs/tests/compile-fail/stable/surface_demuxenum_wrongfields_2.stderr	Updates expected compiler stderr output for trybuild tests.
dfir_rs/tests/compile-fail/stable/surface_demuxenum_wrongfields_1.stderr	Updates expected compiler stderr output for trybuild tests.
dfir_rs/tests/compile-fail/stable/surface_demuxenum_wrongenum.stderr	Updates expected compiler stderr output for trybuild tests.
dfir_rs/src/util/sparse_vec.rs	Adds Clone/Debug derives for SparseVec.
dfir_rs/src/util/mod.rs	Adds common derives for Persistence and PersistenceKeyed.
dfir_pipes/src/push/sort.rs	Introduces a push-side Sort combinator with backpressure-aware finalize draining.
dfir_pipes/src/push/reduce_keyed.rs	Introduces a push-side ReduceKeyed combinator that flushes reduced (K,V) pairs on finalize.
dfir_pipes/src/push/mod.rs	Registers new push modules/re-exports and adds constructors for fold/reduce/accumulate/sort.
dfir_pipes/src/push/fold_keyed.rs	Introduces a push-side FoldKeyed combinator that flushes (K, Acc) pairs on finalize.
dfir_pipes/src/push/accumulate.rs	Adds the unified Accumulate push combinator and AccumState trait.
dfir_pipes/src/push/accum_state.rs	Adds AccumState implementations for fold/reduce/sort patterns.
dfir_lang/src/graph/ops/fold.rs	Removes stratum delay and adds push-side codegen path using the new push accumulator infrastructure.
dfir_lang/src/graph/ops/fold_no_replay.rs	Removes stratum delay and adds push-side codegen path for no-replay semantics.
dfir_lang/src/graph/ops/fold_keyed.rs	Removes stratum delay and adds push-side codegen path using the new FoldKeyed push combinator.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[shadaj]

I reviewed fold first, but this seems almost identical, so LGTM!