Skip to content

Latest commit

 

History

History
66 lines (42 loc) · 10.6 KB

File metadata and controls

66 lines (42 loc) · 10.6 KB

harness-writer — CHANGELOG

v2.3 — 2026-06-14 — patch (distilled from dog-fooding a code-fix loop)

  • Three new gate-wiring-law instances added to design-principles.md, all caught by a lane's own step-10 adversarial review (the meta-loop catching real deadlocks before handover): (1) a STATE phase the engine prompts don't name silently deadlocks the phase-branching gates — reuse a known phase rather than patching three prompts; (2) a stored acceptance command must survive the agent's actual shell + interpreter — use absolute /usr/bin/grep, verify the test env runs before seeding a loop, defer real tests to CI; (3) tier findings + Opus-demote the over-called "clean" set before seeding a fix-loop (a naive "fix everything" loop thrashes on gated/stale/owner-only items). No new figures; red-line #6 intact.

v2.2.1 — 2026-06-14 — patch (two build-run lessons)

  • Doctrine (design-principles.md §Gate-wiring law) +1 bullet — the worktree-gate deadlock. A single-phase lane whose tick.sh prints WAIT: worktree absent for every task deadlocks the ops task that creates the worktree. Gate the worktree-existence check on ready[0].kind in ('code','repair'), not a blanket phase check. Corollary: resolve every ${…}/placeholder in CAP/RULES/KNOWLEDGE to an ls-confirmed path before handover. No new figures; red-line #6 intact.

v2.2 — 2026-06-14 — minor: "The Composed Harness" (wire in Addy Osmani's agent-skills pack)

  • Wiring A — composition. Generated harnesses now bind each loop ring to its matching agent-skills:* skill (the deep, version-pinned playbook) instead of re-deriving a generic prompt. New references/skill-rings.md is the ring→skill map; design-principles.md §"The six loop components" gained the bind-don't-re-derive rule; doc-templates.md HARNESS skeleton gained a skill column; SKILL.md step 6 + reading list updated. Two pack skills are near-twins of house doctrine and are bound, not duplicated: agent-skills:doubt-driven-development ≈ blind Verify (failure mode #2) + Fable reasoning-eye; agent-skills:context-engineering = the authored form of the Memory ladder + Token discipline. Binding is an upgrade, never a hard dependency — absent pack → house-generic fallback.
  • Doctrine grounding. Addy's pack is now on disk (24 lifecycle SKILL.md files). sources.md gained a LOCAL VERIFIED CANON row for it (authored doctrine, not chatter).
  • No new figures entered doctrine; red-line #6 intact. Pack resolved by glob (hash dir changes per update), so version bumps don't break the wiring.

v2.0.4 — 2026-06-14 — patch (distilled from the companion-bridge lane)

  • New doctrine: "A stored guard command must survive a HOSTILE re-run shell — quote globs, guard the scan root, pin the token." The companion-bridge lane's own step-10 adversarial review caught three dead-guard variants in seeded regressionChecks[] — all "looks live, never fires": (a) an unquoted --include=*.swift glob that zsh's nomatch aborts (single-quote it); (b) a ! grep -rEl … <dir> PHI-check that fails OPEN when the scan dir drifts/renames (grep finds nothing → ! → exit 0 = healthy) — fix with a test -d <dir> && existence gate so it fails CLOSED; (c) a bare-word grep -qi 'consent' consent tripwire that greens on any stray comment — pin the load-bearing token (consentRequired). Added to design-principles.md (gate-wiring-law family). The unifying rule: prove a ratchet goes RED when the property is actually violated, under the shell + paths a fresh agent will use, not just GREEN today. The review also caught a HIGH-severity tick.sh bug shared with the template pattern: unblocked() filtered status=="pending" but never excluded ids already in done[], so the documented done[]-retire convention infinite-re-picked the same task — fixed in the lane with and t["id"] not in done. Step-12 meta-loop working as intended. No new figures (sources.md unchanged; 89/62 remains absent).

v2.0.3 — 2026-06-14 — patch (distilled from the security-remediation lane's CodeRabbit round)

  • New doctrine: "Review-feedback loop — harvest the machine-actionable reviewer, but verify it, never blind-apply." Two lessons from running CodeRabbit on the lane's pushed PRs: (1) auto-reviewers emit machine-actionable payloads (committable ```suggestion blocks, literal "🤖 Prompt for AI Agents" fix-instructions, stable fingerprint ids, an "Actionable: N" count) a harness can pipeline as a kind:'review-fix' tick — gh api …/pulls/<n>/comments, one comment per item, fed to an implementer; (2) the reviewer is just another model and must pass the same blind Verify+Guard as the maker — CodeRabbit raised a 🔴 Critical (web-query-sourced) that an inspect.signature+runtime check on the installed gradio 6.18 disproved; blind-applying its committable "fix" would have reintroduced the very regression the change fixed. Added to design-principles.md (extends failure-mode #2, independent-verification, to the external reviewer). Distilled while the user asked the skill to "take advantage of" CodeRabbit's baked-in prompts — step-12 meta-loop. No new figures (sources.md unchanged; 89/62 remains absent).

v2.0.2 — 2026-06-14 — patch (distilled from a packaging lane)

  • New doctrine: "A stored guard command must be self-contained." A seeded regressionChecks[] command (grep … "$DIR/packet") depended on a shell var defined only in the .mjs; re-run standalone by the preflight/verifier agents, $DIR was unset → it scanned a nonexistent path → false-green every tick, silently voiding the lane's only no-secret-in-packet guarantee. Hardcoded to an absolute path; a planted-leak test proved it now fires. Added to design-principles.md (gate-wiring-law family). Caught by the lane's own step-10 adversarial review — the meta-loop (step 12) working as intended. No new figures (sources.md unchanged; 89/62 remains absent).

v2.0.1 — 2026-06-14 — patch (distilled from the security-remediation lane)

  • New doctrine: "Route on an explicit structured field, never inferred free-text." A worktree/repo router built on /regex/.test(name+spec) over-matched — tasks that merely cite another repo in their evidence got routed there to edit files that don't exist (or silently dropped half a cross-repo fix). Fix = explicit enum in the plan schema, set from task.surface; regex only as an absent-field fallback. Added to design-principles.md (gate-wiring-law family). Caught by the lane's own step-10 adversarial review — the meta-loop (step 12) working as intended.

v2.0 — 2026-06-13 — "The Self-Repairing Harness"

Theme: Akshay, "Your Agent Harness Should Repair Itself" (Opik), 2026-06-08 — ideas only, no tooling.

Self-repair of our own doctrine (the headline)

  • Removed a fabricated statistic. The 89/62 accuracy figure ("independent verification beats self-critique") was attributed to Lance Martin but is not in his article (6 of 7 secondary sources confirm; only explainx.ai produced it). Removed from design-principles.md and mjs-template.mjs. Replaced with the qualitative claim + the verified self-preference-bias figure (~10–25%, FutureAGI). See sources.md.
  • Re-attributed "blind verifier" → "disjoint evidence / independent context window" (Thariq's actual terms).
  • Labelled "DriftGuard" a house extension (not Thariq); named the real primitives /goal + /loop.
  • Re-attributed the "doing the right work" quote to Thariq's Fable 5 commentary (06-09), not the harness post (06-02).
  • Restored the "honest feedback" clause to the rubric quote; flagged the "Steve" attribution as unverified.
  • Relabelled "cost concentrates in synthesizers/judges" as a house observation.
  • Added references/sources.md as the anti-fabrication backstop (no figure enters doctrine unverified).
  • Fixed the step-10 .mjs syntax-check recipe: it must strip the leading export (sed 's/^export const/const/') before the async-wrapper node --check — the export const meta line false-failed the documented command. Surfaced while running v2.0's own verification (the meta-loop, step 12, working as intended).
  • The build's own step-10 adversarial review (Opus, blind) caught that newRegressionCheck was wired through the schema + return but never populated by the verifier prompt — the ratchet would never grow (the v2.0 thesis was a no-op). Fixed: the verifier now re-runs the standing ratchet every tick and emits newRegressionCheck on a repair PASS; kind:'repair' added to the PLAN_SCHEMA enum; tick.sh given runnable Python for the breaker/deps/cycle guard; livelock on a permanently-red ratchet now escalates to a human. (The dog-food review doing its job.)

New: every generated harness now self-repairs

  • M1 Regression Ratchet · M2 Circuit Breaker · M3 Pre-flight Health Check · M4 Failure-Driven Repair Entry · M5 Cross-Task DAG · M6 Sharpened Independent Verify · M7 Fail-Closed Default. See self-repair.md.

New: the skill learns too

  • SKILL.md step 12 "Distill back" — verified lessons flow into design-principles.md + sources.md after each authoring run.

De-boilerplate

  • Canonical engine / tick / driver blocks centralized in references for copy-verbatim authoring.

v2.0.1 — 2026-06-13 — dog-food hardening (ios-accessibility lane)

The ios-accessibility lane's own step-10 adversarial review (6-agent, blind) caught two latent holes in the shared template/doctrine (not just that lane), fixed here so future harnesses inherit the fix:

  • Guard fail-branch ignored its own violations[]. mjs-template.mjs only ANDed pass/securityIssues/reasoningSound — a Guard recording a drift/scope/ship-gate/ratchet-tampering finding in the free-text violations[] while leaving pass=true would slip through. Added || (guard.violations && guard.violations.length). New "Gate-wiring law" in design-principles.md.
  • M1's "non-decreasing ratchet" was prose-only. The doctrine claimed "the .mjs asserts length ≥ prior" but the .mjs can't fs-read state mid-tick. Replaced with a ratchetHigh watermark (driver-persisted) + a tick.sh ERROR ratchet shrank tripwire + a ratchetCount baseline into the Guard. Updated self-repair.md M1.
  • Also distilled (design-principles): the driver must branch on every return shape + catch-all (agent-died = transient resume; build failed = halt), and define $WT/$REPO once in the driver doc.

No new figures entered doctrine (sources.md unchanged); the 89/62 figure remains absent (verified by the lane's own fresh-session reviewer).

v1 — 2026-06-10 — initial (built from the first production lane)