SimplicityGuy · SimplicityGuy · Jul 8, 2026 · Jul 8, 2026 · Jul 8, 2026 · Jul 8, 2026
diff --git a/.planning/PROJECT.md b/.planning/PROJECT.md
@@ -106,10 +106,12 @@ Get 200K messy music and concert files properly named, organized into logical fo
 
 **2026.7.2 Multi-Compute Agents — SHIPPED + milestone COMPLETED 2026-07-06.** All 5 phases (72–76) merged (PRs #209/#210/#211/#213/#214). Milestone audit PASSED — 15/15 requirements, 5/5 phases verified & Nyquist-compliant, 6/6 cross-phase integration seams wired, E2E N-compute dispatch→push→reconcile→render flow complete. N cloud-compute agents now dispatch/route/reconcile/fail-isolate simultaneously (compute-side twin of the N-Kueue support) with the `≤1-compute` fail-fasts retired and zero new dependencies. Four low-severity/cosmetic review items closed at close-out by quick `260706-odc` (116 tests green). Deferred to v2: PROV-01 (N-compute-aware orphan recovery), PROV-02/03. Archived to `milestones/2026.7.2-{ROADMAP,REQUIREMENTS,MILESTONE-AUDIT}.md`; `REQUIREMENTS.md` removed (fresh for next milestone). Codebase: ~28.4k Python LOC in `src/`. Remaining ship step: push the `2026.7.1` tag (per STATE.md).
 
-**Active milestone: 2026.7.5 Parallel Enrich DAG (started 2026-07-08).** Retire the linear `FileState` enum; derive per-file, per-stage status from output tables so metadata/fingerprint/analyze are genuinely parallel. Fresh branch `SimplicityGuy/true-parallel` off `main` @ `ce0c6434` (includes PR #221). 14 phases (77–90) mapped; requirements + roadmap defined. **Phase 77 (additive schema `032` + rescan-wipe fix) shipped — PR #223. Phase 78 (Derivation Layer, Eligibility & Anti-Drift Test Harness) COMPLETE + verified 2026-07-08.** Next: Phase 79 (Shadow-Compare Gate, live corpus).
+**Active milestone: 2026.7.5 Parallel Enrich DAG (started 2026-07-08).** Retire the linear `FileState` enum; derive per-file, per-stage status from output tables so metadata/fingerprint/analyze are genuinely parallel. Fresh branch `SimplicityGuy/true-parallel` off `main` @ `ce0c6434` (includes PR #221). 14 phases (77–90) mapped; requirements + roadmap defined. **Phase 77 (additive schema `032` + rescan-wipe fix) shipped — PR #223. Phase 78 (Derivation Layer, Eligibility & Anti-Drift Test Harness) COMPLETE + verified 2026-07-08. Phase 79 (Shadow-Compare Gate, live corpus) COMPLETE + verified 2026-07-08 — the standing implication gate now exists (MIG-02).** Next: Phase 80 (Recovery / Re-enqueue Cutover).
 
 **Phase-by-phase execution history (2026.7.5):**
 
+**2026.7.5 Parallel Enrich DAG — Phase 79 (Shadow-Compare Gate, live corpus) COMPLETE + verified 2026-07-08 (11/11 code-verifiable truths; 1 human item deferred per D-02).** The standing state↔derived implication gate that phases 80–90 keep green and the hard precondition for the destructive `033` (Phase 90). Shipped across 2 plans / 2 waves over the ONE shared assertion core (D-01): (1) `src/phaze/services/shadow_compare.py` — an `INVARIANTS` registry with one `state ⇒ derived-condition` implication per `FileState` value (16 entries; `DISCOVERED` documented-vacuous), `Invariant`/`InvariantResult`/`Report` dataclasses, and async `run_shadow_compare` running a per-invariant corpus-wide anti-join (`state=X AND NOT <derived>`) returning counts + capped `file_id` samples + `hard_fail_total`; reuses Phase-78 `done_clause`/`failed_clause` (D-03, never `stage_status_case`), asserts implication-not-equality, PUSHING/PUSHED loosen to `cloud_job` row-existence (RESEARCH A3/OQ1) while AWAITING_CLOUD keeps `status='awaiting'`, apply-outcome states assert `proposals.status` (not `execution_log`), and the `{fingerprinted, local_analyzing}` soft allowlist is counted-but-never-gated (D-06). (2) `src/phaze/cli/shadow_compare.py` — a thin `python -m phaze.cli.shadow_compare` argparse runner + `[group('db')] shadow-compare *ARGS` justfile recipe driving the SAME core (no duplicated logic), exit 1 iff any HARD invariant diverges (D-05), `--sample-cap`/`--verbose`/`--database-url` flags, DSN never leaked. Hermetic fixture-corpus gate `tests/integration/test_shadow_compare.py` — non-vacuous divergent RED cell + consistent GREEN cell per HARD invariant, implication cell, soft-allowlist cell, DB-free registry cell locking D-04 coverage + D-06 allowlist, and CLI-exit cells; `just test-bucket integration` green in isolation (130 passed, 100% core coverage). Verifier 11/11 (independent codebase read + its own 130/130 suite re-run). Code-review gate: 1 blocker + 2 warnings + 1 info — ALL fixed inline (`ba489b62`): CR-01 (a data-loss footgun — the CLI cells' committed `TRUNCATE agents CASCADE` would wipe the DEV DB on a bare `pytest` with `TEST_DATABASE_URL` unset; now a module-level `_test`-DB guard refuses any non-`_test` target), WR-01 (DSN parse-error redaction + password-masking `URL` threaded to the engine), WR-02 (`--sample-cap` rejects negatives), IN-01 (`ORDER BY id` for deterministic samples). One human-verification item deferred by design: the live ~200K-corpus run (SC-3) is explicitly held to the next homelab rollout per D-02, tracked as the sole Manual-Only check in `79-VALIDATION.md`/`79-HUMAN-UAT.md` and must be recorded before Phase 90's `033`. Ships as its own PR on a worktree branch. **Next: Phase 80 (Recovery / Re-enqueue Cutover) — the first reader cut over off `FileRecord.state`, deliberately before the pending-set/counts readers (READ-03).**
+
 **2026.7.5 Parallel Enrich DAG — Phase 78 (Derivation Layer, Eligibility & Anti-Drift Test Harness) COMPLETE + verified 2026-07-08 (12/12 must-haves).** The single-source-of-truth predicate layer shipped as two new modules, purely additive (no reader/writer cut over — that is Phase 79+). Shipped across 2 plans / 2 waves: (1) `src/phaze/enums/stage.py` — the DB-free, agent-safe half: `Stage`/`Status` StrEnums, `ELIGIBILITY_DAG` topology, the pure-Python `resolve_status()` precedence ladder (`in_flight ≻ done ≻ failed ≻ not_started`, encoding DERIV-02/03/05 + D-03 metadata-failure-only-is-FAILED), and the pure `eligible()` predicate (ELIG-01 enrich independence, ELIG-02 apply-gated-on-approved-proposal, ELIG-03 terminal-failed-analyze guarding the 44.5K over-enqueue class, ELIG-04 failed-fingerprint-stays-eligible); a subprocess banned-import test enforces the T-78-01 agent boundary (no `phaze.models`/`phaze.database`/`sqlalchemy`), 100% module coverage. (2) `src/phaze/services/stage_status.py` — the SQLAlchemy `ColumnElement[bool]` twin: per-stage `done_clause`/`failed_clause`, `stage_status_case` CASE ladder, `inflight_clause` deriving in-flight from `scheduling_ledger` (D-01 authoritative, never `saq_jobs`), and `saq_detail` reading `saq_jobs` only as a `begin_nested()` SAVEPOINT-isolated corroborating detail that degrades safe; the D-01 written decision record lives in the module docstring (INFLIGHT-03/SC#5). The DERIV-04 anti-drift lock is a parametrized SQL⇔Python equivalence matrix (`tests/integration/test_stage_status_equivalence.py`, 25 cases, real PG green via `just test-bucket integration`) asserting `sql_status == py_status == expected` for every (stage × status) cell; 44 DB-free unit cells in `tests/shared/`. Verifier 12/12 must-haves (independent codebase read). Code-review gate: 0 blockers, 3 warnings / 2 info — WR-01 (`done(review)`==`done(propose)` makes `eligible(review)` unreachable under real derivation) + WR-02 (downstream `eligible()` doesn't gate on `IN_FLIGHT`) are the RESEARCH-flagged deferred assumptions (A3/OQ1, backstopped by deterministic-key dedup); WR-03 (`_DONE_FP` fingerprint allowlist duplicated across the two halves) is a real drift nit the equivalence test would catch at runtime — candidate for `/gsd:code-review 78 --fix` before cutover. Ships as its own PR on a worktree branch. **Next: Phase 79 (Shadow-Compare Gate) — the re-runnable live-corpus implication check that must pass before any reader cutover and before `033`.**
 
 **Phase-by-phase execution history (2026.7.2):**

diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md
@@ -93,7 +93,7 @@
 ### Migration & Verification (MIG)
 
 - [x] **MIG-01**: Migration `032` is additive-only — it creates the failure markers, the dedup marker, and the cloud sidecar representation, adds the partial indexes, and backfills them from `FileRecord.state`, **without touching `files.state`**.
-- [ ] **MIG-02**: A committed, re-runnable shadow-compare check asserts per-file *implication* invariants (e.g. `state=ANALYZED ⇒ analysis_completed_at IS NOT NULL`; `state=DUPLICATE_RESOLVED ⇒ dedup marker`) across the live corpus, with `FINGERPRINTED` documented as the one expected divergence; it must pass before any reader cutover and before the destructive migration.
+- [x] **MIG-02**: A committed, re-runnable shadow-compare check asserts per-file *implication* invariants (e.g. `state=ANALYZED ⇒ analysis_completed_at IS NOT NULL`; `state=DUPLICATE_RESOLVED ⇒ dedup marker`) across the live corpus, with `FINGERPRINTED` documented as the one expected divergence; it must pass before any reader cutover and before the destructive migration.
 - [x] **MIG-03**: Rescanning a file no longer resets pipeline progress — with `FileRecord.state` gone, the `ON CONFLICT DO UPDATE SET state = excluded.state` progress-wipe is structurally impossible (fixes the rescan-wipe bug).
 - [ ] **MIG-04**: Migration `033` is destructive and lands last — after the shadow-compare passes on the live corpus and cloud-push lanes are drained/quiesced, it drops `ix_files_state`, drops `files.state`, and deletes the `FileState` enum; its `downgrade()` documents the enum reconstruction from derived sources (and its lossiness).
 
@@ -157,7 +157,7 @@
 | PERF-01 | Phase 77 | Complete |
 | PERF-02 | Phase 82 | Pending |
 | MIG-01 | Phase 77 | Complete |
-| MIG-02 | Phase 79 | Pending |
+| MIG-02 | Phase 79 | Complete |
 | MIG-03 | Phase 77 | Complete |
 | MIG-04 | Phase 90 | Pending |
 | LEGACY-01 | Phase 89 | Pending |

diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md
@@ -22,7 +22,7 @@ Retire the linear `FileState` enum and derive per-file, per-stage status (`not_s
 
 - [x] **Phase 77: Additive Schema & Rescan-Wipe Fix (migration `032`)** — additive-only `032` creates the failure markers, dedup marker, cloud-routing sidecar rows, and partial indexes (mirrored into the ORM), backfilled from `files.state` **without touching `files.state`**; plus the independently-shippable rescan progress-wipe fix (MIG-01, MIG-03, PERF-01) (completed 2026-07-08)
 - [x] **Phase 78: Derivation Layer, Eligibility & Anti-Drift Test Harness** — the single-source predicate module (`enums/stage.py` DB-free + `services/stage_status.py`), `stage_status()` / `eligible()`, SAVEPOINT-wrapped in-flight detection, and the SQL⇔Python equivalence test; carries the **D-01 open decision** (written decision record required at plan-time) (DERIV-01..05, ELIG-01..04, INFLIGHT-01..03) (completed 2026-07-08)
-- [ ] **Phase 79: Shadow-Compare Gate (live corpus)** — a committed, re-runnable implication check between legacy `files.state` and the derived representation; must pass before any reader cutover and before `033` (MIG-02)
+- [x] **Phase 79: Shadow-Compare Gate (live corpus)** — a committed, re-runnable implication check between legacy `files.state` and the derived representation; must pass before any reader cutover and before `033` (MIG-02) (completed 2026-07-08)
 - [ ] **Phase 80: Recovery / Re-enqueue Cutover** — `reenqueue.py` + `reconcile_cloud_jobs.py` derive done/in-flight from `stage_status`/sidecars with no `FileRecord.state` read; deliberately **before** the pending-set/counts readers (double-negation dependency) (READ-03)
 - [ ] **Phase 81: Per-Stage Failure Persistence & Retry Paths** — durable failure markers for analyze + metadata (`report_metadata_failed` records instead of nothing) + reused fingerprint failure; a metadata retry path so a failure is never a permanent dead-end (FAIL-01..04)
 - [ ] **Phase 82: Counts & Pending-Set Cutover** — the three enrich pending sets + `get_pipeline_stats` derived from `stage_status`; the cross-stage deadlock dissolves; four-bucket per-stage counts; the 200K-scale poll latency measured (READ-01, READ-02, PERF-02)
@@ -267,7 +267,7 @@ Deployment-gated verification deferred to the live OCI A1 rollout (see STATE.md
 | 76. Compute/Push Hardening | 2026.7.2 | 3/3 | Complete    | 2026-07-06 |
 | 77. Additive Schema & Rescan-Wipe Fix (migration 032) | 2026.7.5 | 3/3 | Complete    | 2026-07-08 |
 | 78. Derivation Layer, Eligibility & Anti-Drift Test Harness | 2026.7.5 | 2/2 | Complete    | 2026-07-08 |
-| 79. Shadow-Compare Gate (live corpus) | 2026.7.5 | 0/0 | Not started | - |
+| 79. Shadow-Compare Gate (live corpus) | 2026.7.5 | 2/2 | Complete    | 2026-07-08 |
 | 80. Recovery / Re-enqueue Cutover | 2026.7.5 | 0/0 | Not started | - |
 | 81. Per-Stage Failure Persistence & Retry Paths | 2026.7.5 | 0/0 | Not started | - |
 | 82. Counts & Pending-Set Cutover | 2026.7.5 | 0/0 | Not started | - |
@@ -317,8 +317,10 @@ Plans:
   5. Every `saq_jobs` read for `in_flight` is static SQL wrapped in a `begin_nested()` SAVEPOINT that degrades to a safe default; a **written D-01 decision record** fixes the authoritative `in_flight` source so a crashed-mid-run / callback-lost file is never falsely re-enqueued as `not_started`.
 
 **Plans**: 2 plans
+
 - [x] 78-01-PLAN.md — DB-free resolver + eligibility DAG (`enums/stage.py`): Stage/Status enums, ELIGIBILITY_DAG, resolve_status() precedence ladder, eligible() incl. ELIG-03 terminal-failed regression (Wave 1)
 - [x] 78-02-PLAN.md — SQL `ColumnElement[bool]` builders (`services/stage_status.py`) + ledger in_flight + saq_detail SAVEPOINT + D-01 decision record, locked by the DERIV-04 equivalence test (Wave 2)
+
 **Note**: INFLIGHT-03 / D-01 RESOLVED (78-CONTEXT.md): `in_flight` = `scheduling_ledger` AUTHORITATIVE, `saq_jobs` corroborating-only (rejected the naked union). Written decision record persisted in `services/stage_status.py` module docstring. Original open-decision framing — a written decision record is REQUIRED at plan-time (Architecture rejects the naked union; design/Stack lean union). The roadmap deliberately does not resolve it.
 
 ### Phase 79: Shadow-Compare Gate (live corpus)
@@ -332,7 +334,15 @@ Plans:
   2. The check asserts **implication, not equality**, with `FINGERPRINTED` documented as the one expected divergence; any other divergence is a hard fail.
   3. The gate passes on a restore of the live corpus after the `032` backfill, and its output is recorded in the phase VERIFICATION.
 
-**Plans**: TBD
+**Plans**: 2 plans
+Plans:
+**Wave 1**
+
+- [x] 79-01-PLAN.md — Shared assertion core (`services/shadow_compare.py`): INVARIANTS registry (one implication per FileState value, §6.1/D-04; {FINGERPRINTED, LOCAL_ANALYZING} soft allowlist D-06) + `run_shadow_compare` count+capped-sample Report (D-05) reusing done/failed_clause (D-03), + hermetic fixture-corpus CI gate in the `integration` bucket (Wave 1)
+
+**Wave 2** *(blocked on Wave 1 completion)*
+
+- [x] 79-02-PLAN.md — Thin `python -m phaze.cli.shadow_compare` runner over the SAME core (D-01) + `[group('db')] shadow-compare` justfile recipe, nonzero-exit-on-hard-divergence contract (D-05); live 200K restore run DEFERRED to homelab (D-02) (Wave 2, depends 79-01)
 
 ### Phase 80: Recovery / Re-enqueue Cutover
 

diff --git a/.planning/STATE.md b/.planning/STATE.md
@@ -2,16 +2,15 @@
 gsd_state_version: 1.0
 milestone: 2026.7.5
 milestone_name: Parallel Enrich DAG
-status: ready_to_plan
-last_updated: 2026-07-08T17:29:40.711Z
-last_activity: 2026-07-08 -- Phase 78 execution started
+status: "Phase 79 shipped — PR #225"
+last_updated: "2026-07-08T21:04:08.382Z"
+last_activity: 2026-07-08
 progress:
   total_phases: 52
-  completed_phases: 16
-  total_plans: 61
-  completed_plans: 5
-  percent: 31
-stopped_at: Phase 78 complete (2/2) — ready to discuss Phase 79
+  completed_phases: 17
+  total_plans: 63
+  completed_plans: 7
+  percent: 33
 ---
 
 # Project State
@@ -21,20 +20,20 @@ stopped_at: Phase 78 complete (2/2) — ready to discuss Phase 79
 See: .planning/PROJECT.md (updated 2026-07-06 — 2026.7.2 Multi-Compute Agents shipped)
 
 **Core value:** Get 200K messy music and concert files properly named, organized, deduplicated, with rich metadata in Postgres -- human-in-the-loop approval so nothing moves without review. Files stay on file-server agents; decisions stay on the application server.
-**Current focus:** Phase 79 — shadow compare gate (live corpus)
+**Current focus:** Phase 80 — recovery / re enqueue cutover
 
 ## Current Position
 
-Phase: 79
+Phase: 80
 Plan: Not started
-Status: Ready to plan
+Status: Phase 79 shipped — PR #225
 Last activity: 2026-07-08
 
 ## Performance Metrics
 
 **v1.0 Velocity:**
 
-- Total plans completed: 179
+- Total plans completed: 181
 - Total phases: 11
 - Timeline: 4 days (2026-03-27 -> 2026-03-30)
 - Tests: 282 passing
@@ -285,9 +284,9 @@ These are tracked follow-ups; none blocks the 2026.7.1 milestone record. The PRO
 
 ## Session Continuity
 
-Last session: 2026-07-08T16:00:03.447Z
-Stopped at: Phase 78 context gathered
-Resume file: .planning/phases/78-derivation-layer-eligibility-anti-drift-test-harness/78-CONTEXT.md
+Last session: 2026-07-08T19:09:02.828Z
+Stopped at: Phase 79 context gathered
+Resume file: .planning/phases/79-shadow-compare-gate-live-corpus/79-CONTEXT.md
 
 ## Operator Next Steps