Skip to content

Commit a8c4d7c

Browse files
site-shellclaude
andcommitted
ci: a release gate that catches what shipped wrong by hand
Publishing these packages was manual and ungated. The 2026-08-25 release -- the first in two days -- tripped three separate defects, all found by reading rather than by any check: * kirby-combat published with NO runtime dependency declared while its code imported kirby_cost * kirby-combat and kirby-sheet both declared `kirby-cost>=0.3.0` while using modules and attributes absent from 0.3.0; pip resolves that constraint happily and installs a package that breaks on use * kirby-sheet 0.2.0 shipped with __version__ reporting "0.1.0" scripts/check_release.py is those checks, runnable locally and from CI. Each one is written against a failure that actually happened, and the script was itself found unable to fail twice before it worked: first it only imported the top-level package (so a missing submodule slipped through), then it turned out Python puts cwd on sys.path, so running it from the repo validated the working tree instead of the built wheel and reported the local version. It now probes with -P from a temp directory. Stated limit, in the script: attribute-level drift is NOT caught. A floor that is too low because the code needs a FIELD added in a later version still imports clean. A green run means "the modules resolve", not "the floor is right". release.yml uses PyPI Trusted Publishing, so no token lives in the repo -- the reason the previous publish job was deleted from these workflows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xof4ZUS5n6A5ibX3PXNYbs
1 parent bdb356b commit a8c4d7c

3 files changed

Lines changed: 327 additions & 0 deletions

File tree

.github/workflows/release.yml

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
name: Release
2+
3+
# Publishes to PyPI when a version tag is pushed, and NOT otherwise.
4+
#
5+
# No credentials live here. The publish job uses PyPI Trusted Publishing
6+
# (OIDC), so there is no API token in a secret to leak or rotate -- which is
7+
# why the old publish job was deleted from this repo in the first place. See
8+
# the note at the top of tests.yml.
9+
#
10+
# The guards in `scripts/check_release.py` run BEFORE the upload, and each one
11+
# exists because the failure it catches actually shipped:
12+
# * licensed Hero Games content in the artifact
13+
# * a dependency floor lower than the version the code needs
14+
# * the module's __version__ disagreeing with its own metadata
15+
# * a tag that does not match the built version
16+
# On 2026-08-25 the first release in two days tripped three of the four, all
17+
# found by hand. This is that check, automated.
18+
19+
on:
20+
push:
21+
tags: ["v*"]
22+
workflow_dispatch:
23+
24+
jobs:
25+
build:
26+
runs-on: ubuntu-latest
27+
steps:
28+
- uses: actions/checkout@v4
29+
- uses: actions/setup-python@v5
30+
with:
31+
python-version: "3.12"
32+
33+
- name: Install the package and its test deps
34+
run: pip install -e .[dev]
35+
36+
- name: Run the test suite
37+
# Real-character tests skip here by design: they read a corpus path
38+
# that CI does not have. See tests/corpus.py.
39+
run: pytest -q --tb=short
40+
41+
- name: Build the distribution
42+
run: |
43+
python -m pip install --upgrade build
44+
rm -rf dist build *.egg-info
45+
python -m build
46+
47+
- name: Release guards
48+
# Runs from the repo root, which is exactly the case the guard had to
49+
# be hardened against -- cwd on sys.path made an earlier version
50+
# validate the working tree instead of the built wheel.
51+
run: python scripts/check_release.py dist/
52+
53+
- uses: actions/upload-artifact@v4
54+
with:
55+
name: dist
56+
path: dist/
57+
58+
publish:
59+
needs: build
60+
runs-on: ubuntu-latest
61+
environment: pypi
62+
permissions:
63+
id-token: write # Trusted Publishing; no token secret required
64+
steps:
65+
- uses: actions/download-artifact@v4
66+
with:
67+
name: dist
68+
path: dist/
69+
- uses: pypa/gh-action-pypi-publish@release/v1

.github/workflows/tests.yml

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
name: Tests
2+
3+
# kirby-sheet had no CI at all until 2026-08-25. The gap mattered: its
4+
# byte-fidelity tests -- the ones that export a character through real Hero
5+
# Designer and compare the output byte for byte -- SKIP unless
6+
# KIRBY_SHEET_ORACLE, KIRBY_SHEET_HDC and KIRBY_SHEET_HDE are set. A local run
7+
# without them reports "196 passed, 33 skipped" and looks green while the
8+
# package's central claim goes unverified.
9+
#
10+
# Those 33 cannot run here and never will: they need a licensed Hero Designer
11+
# installation, which this repository deliberately does not ship. So this
12+
# workflow gates what a reader can actually run, and the release workflow says
13+
# plainly that the oracle gate is a local, pre-release step.
14+
15+
on:
16+
push:
17+
branches: [main]
18+
pull_request:
19+
branches: [main]
20+
workflow_dispatch:
21+
22+
jobs:
23+
test:
24+
runs-on: ubuntu-latest
25+
steps:
26+
- uses: actions/checkout@v4
27+
- uses: actions/setup-python@v5
28+
with:
29+
python-version: "3.12"
30+
- name: Install package + dev deps
31+
run: pip install -e .[dev]
32+
- name: Run tests
33+
run: pytest -q --tb=short
34+
- name: Report what skipped, so a silent skip is visible in the log
35+
if: always()
36+
run: pytest -q -rs --tb=no | tail -40

scripts/check_release.py

Lines changed: 222 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,222 @@
1+
#!/usr/bin/env python3
2+
"""Guards a release artifact against the four ways a kirby package has shipped wrong.
3+
4+
Run it on `dist/` before uploading, locally or in CI:
5+
6+
python scripts/check_release.py dist/
7+
8+
Each check exists because the failure it catches actually happened, and each
9+
one shipped or nearly shipped a package that pip installs happily and that
10+
breaks, or lies, on use. Exit code 1 on any failure; nothing here is advisory.
11+
12+
1. LICENSED CONTENT. These packages ship no Hero Games data -- no `.hdt`, no
13+
`.hdc`, no `.hde`. That is a licensing boundary, not a preference, and the
14+
only place it can be enforced is the built artifact: a stray fixture path
15+
in MANIFEST.in or package_data would carry one in silently.
16+
17+
2. DEPENDENCY FLOORS THAT DO NOT EXIST. On 2026-08-25 both kirby-combat and
18+
kirby-sheet declared `kirby-cost>=0.3.0` while importing modules that 0.3.0
19+
does not contain. pip resolves that constraint to 0.3.0 without complaint
20+
and hands the user a package that dies at import. So: install the built
21+
wheel into a clean venv with no local packages visible, and import EVERY
22+
submodule.
23+
24+
WHAT THIS DOES NOT CATCH, stated plainly: drift at the ATTRIBUTE level. If
25+
a floor is too low because the code needs `TemplateData.defense` — a field
26+
added in a later version — every module still imports and this check passes.
27+
Only running the package's own suite against the installed distribution
28+
would catch that, and the suite needs a Hero Designer template that CI does
29+
not have. Treat a green run here as "the modules resolve", not "the floor
30+
is right".
31+
32+
3. VERSION DISAGREEMENT. kirby-sheet 0.2.0 shipped with
33+
`kirby_sheet.__version__ == "0.1.0"` while its metadata said 0.2.0 --
34+
pip reporting one number and the code another. kirby-combat had the same
35+
drift at 0.3.28. Three instances make it a pattern.
36+
37+
4. TAG/VERSION MISMATCH (CI only). A tag that disagrees with pyproject means
38+
the published version and the git history point at different code.
39+
"""
40+
from __future__ import annotations
41+
42+
import json
43+
import os
44+
import subprocess
45+
import sys
46+
import tarfile
47+
import tempfile
48+
import venv
49+
import zipfile
50+
from pathlib import Path
51+
52+
LICENSED_SUFFIXES = (".hdt", ".hdc", ".hde")
53+
54+
55+
def fail(msg: str) -> None:
56+
print(f"FAIL {msg}")
57+
58+
59+
def ok(msg: str) -> None:
60+
print(f"ok {msg}")
61+
62+
63+
def artifact_names(path: Path) -> list[str]:
64+
if path.suffix == ".whl":
65+
return zipfile.ZipFile(path).namelist()
66+
return tarfile.open(path).getnames()
67+
68+
69+
def check_licensed(dist: Path) -> bool:
70+
clean = True
71+
for art in sorted(dist.iterdir()):
72+
if art.suffix not in (".whl", ".gz"):
73+
continue
74+
bad = [n for n in artifact_names(art) if n.lower().endswith(LICENSED_SUFFIXES)]
75+
if bad:
76+
fail(f"{art.name} carries licensed content: {bad[:5]}")
77+
clean = False
78+
else:
79+
ok(f"{art.name}: no .hdt/.hdc/.hde")
80+
return clean
81+
82+
83+
def wheel_metadata(wheel: Path) -> dict[str, list[str]]:
84+
z = zipfile.ZipFile(wheel)
85+
name = next(n for n in z.namelist() if n.endswith(".dist-info/METADATA"))
86+
meta: dict[str, list[str]] = {}
87+
for line in z.read(name).decode().splitlines():
88+
if not line or line[0].isspace():
89+
continue
90+
if ": " in line:
91+
k, v = line.split(": ", 1)
92+
meta.setdefault(k, []).append(v)
93+
return meta
94+
95+
96+
def check_install_and_version(wheel: Path) -> bool:
97+
"""Install into a clean venv and confirm it imports and agrees about its version.
98+
99+
The venv is built WITHOUT system site packages and installed with
100+
--no-cache-dir, so a dependency that only resolves because it happens to
101+
be on this machine cannot mask a floor that is too low.
102+
"""
103+
meta = wheel_metadata(wheel)
104+
dist_name = meta["Name"][0]
105+
version = meta["Version"][0]
106+
module = dist_name.replace("-", "_")
107+
108+
with tempfile.TemporaryDirectory() as tmp:
109+
env_dir = Path(tmp) / "v"
110+
venv.create(env_dir, with_pip=True, system_site_packages=False)
111+
py = env_dir / "bin" / "python"
112+
proc = subprocess.run(
113+
[str(py), "-m", "pip", "install", "-q", "--no-cache-dir", str(wheel)],
114+
capture_output=True, text=True, cwd=tmp,
115+
)
116+
if proc.returncode != 0:
117+
fail(f"{dist_name} {version} does not install cleanly:\n{proc.stderr[-600:]}")
118+
return False
119+
120+
# EVERY submodule, not just the top-level package.
121+
#
122+
# A bare `import kirby_sheet` was the first version of this check and
123+
# it was worthless: it passed a wheel pinned to a kirby-cost that did
124+
# not contain the modules the package imports, because nothing at top
125+
# level reached them. Walking the package forces every module-level
126+
# `from kirby_cost.engine.damage import ...` to actually resolve,
127+
# which is how a floor that is too low announces itself.
128+
probe = (
129+
"import json,importlib,pkgutil;"
130+
f"m=importlib.import_module({module!r});"
131+
"bad=[];"
132+
"\n"
133+
"for _mi in pkgutil.walk_packages(m.__path__, m.__name__ + '.'):\n"
134+
" try:\n"
135+
" importlib.import_module(_mi.name)\n"
136+
" except Exception as e:\n"
137+
" bad.append(f'{_mi.name}: {type(e).__name__}: {e}')\n"
138+
"from importlib.metadata import version as v\n"
139+
f"print(json.dumps({{'module':getattr(m,'__version__',None),'meta':v({dist_name!r}),'bad':bad}}))"
140+
)
141+
# Deliberately clear the template variable: a package must import with
142+
# no Hero Designer installation configured.
143+
env = {k: val for k, val in os.environ.items() if k != "KIRBY_COST_HDT"}
144+
# cwd=tmp and -P are both load-bearing, and this check was WORTHLESS
145+
# without them. Python puts the current directory on sys.path, so
146+
# running this from a package's own repo -- the normal way -- imported
147+
# the local source tree instead of the installed wheel. Measured
148+
# 2026-08-25: a wheel pinned to kirby-cost==0.3.0 passed, because the
149+
# probe was resolving kirby_cost from
150+
# /home/.../kirby-cost/kirby_cost and reporting version 0.4.0. The
151+
# guard was validating the working tree and calling it an artifact.
152+
# -P (3.11+) drops cwd from sys.path; cwd=tmp puts it somewhere with
153+
# nothing importable in it. Belt and braces, deliberately.
154+
proc = subprocess.run(
155+
[str(py), "-P", "-c", probe],
156+
capture_output=True, text=True, env=env, cwd=tmp,
157+
)
158+
if proc.returncode != 0:
159+
fail(f"{dist_name} {version} installs but does not import:\n{proc.stderr[-800:]}")
160+
return False
161+
162+
got = json.loads(proc.stdout.strip().splitlines()[-1])
163+
if got["bad"]:
164+
fail(
165+
f"{dist_name} {version} installs but {len(got['bad'])} submodule(s) "
166+
f"do not import -- usually a dependency floor lower than what the "
167+
f"code actually needs:"
168+
)
169+
for line in got["bad"][:6]:
170+
print(f" {line}")
171+
return False
172+
ok(f"{dist_name} {version}: every submodule imports, with no template configured")
173+
if got["module"] is None:
174+
fail(f"{module}.__version__ is not defined")
175+
return False
176+
if got["module"] != got["meta"]:
177+
fail(
178+
f"{module}.__version__ is {got['module']!r} but the distribution "
179+
f"says {got['meta']!r} -- pip and the code would report different numbers"
180+
)
181+
return False
182+
ok(f"{module}.__version__ agrees with the distribution metadata ({got['meta']})")
183+
return True
184+
185+
186+
def check_tag(wheel: Path) -> bool:
187+
"""Only meaningful in CI, where the tag is what triggered the release."""
188+
ref = os.environ.get("GITHUB_REF_NAME")
189+
if not ref:
190+
return True
191+
version = wheel_metadata(wheel)["Version"][0]
192+
if ref.lstrip("v") != version:
193+
fail(f"tag {ref!r} does not match the built version {version!r}")
194+
return False
195+
ok(f"tag {ref} matches the built version")
196+
return True
197+
198+
199+
def main() -> int:
200+
dist = Path(sys.argv[1] if len(sys.argv) > 1 else "dist")
201+
if not dist.is_dir():
202+
fail(f"{dist} is not a directory")
203+
return 1
204+
wheels = sorted(dist.glob("*.whl"))
205+
if len(wheels) != 1:
206+
fail(f"expected exactly one wheel in {dist}, found {len(wheels)}")
207+
return 1
208+
209+
results = [
210+
check_licensed(dist),
211+
check_install_and_version(wheels[0]),
212+
check_tag(wheels[0]),
213+
]
214+
if all(results):
215+
print("\nall release guards passed")
216+
return 0
217+
print("\nrelease guards FAILED -- do not upload")
218+
return 1
219+
220+
221+
if __name__ == "__main__":
222+
raise SystemExit(main())

0 commit comments

Comments
 (0)