Skip to content

feat: add CI/CD enhancements, test infrastructure, and tooling #255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
you615074-png wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: add CI/CD enhancements, test infrastructure, and tooling #255

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 112 additions & 1 deletion .github/workflows/curriculum.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ on:
- "scripts/audit_lessons.py"
- "scripts/build_catalog.py"
- "scripts/check_readme_counts.py"
- "scripts/lesson_run.py"
- "scripts/quiz_coverage.py"
- "scripts/ts_syntax_check.mjs"
- "scripts/_lib.py"
- "README.md"
- "ROADMAP.md"
- "glossary/**"
Expand All @@ -20,6 +24,10 @@ on:
- "scripts/audit_lessons.py"
- "scripts/build_catalog.py"
- "scripts/check_readme_counts.py"
- "scripts/lesson_run.py"
- "scripts/quiz_coverage.py"
- "scripts/ts_syntax_check.mjs"
- "scripts/_lib.py"
- "README.md"
- "ROADMAP.md"
- "glossary/**"
Expand All @@ -30,6 +38,8 @@ permissions:
contents: read

jobs:
# ── Blocking checks ───────────────────────────────────────────────────

audit:
name: invariant checks
runs-on: ubuntu-latest
Expand All @@ -43,6 +53,108 @@ jobs:
- name: run scripts/audit_lessons.py
run: python3 scripts/audit_lessons.py

# ── Advisory code checks ──────────────────────────────────────────────

python-syntax:
name: python syntax check (advisory)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
with:
python-version: "3.12"
- name: syntax-check all .py files
run: |
python3 scripts/lesson_run.py --json > /tmp/syntax.json
python3 -c "
import json
data = json.load(open('/tmp/syntax.json'))
failed = [f for f in data['failed']]
skipped = [f for f in data['skipped']]
print(f'Syntax: {data[\"checked\"]} lessons, {len(data[\"passed\"])} passed, {len(failed)} failed, {len(skipped)} skipped')
for f in failed:
print(f'::warning file={f[\"lesson\"]}::syntax error: {f[\"reason\"]}')
"

typescript-syntax:
name: typescript syntax check (advisory)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- uses: actions/setup-node@e62d1c64a114b2d4b223e6e62319cbfa45a42c6a # v4
with:
node-version: "22"
- name: install typescript
run: npm install --no-save typescript
- name: syntax-check all .ts files
run: node scripts/ts_syntax_check.mjs --json
Comment on lines +81 to +94
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

TypeScript syntax job blocks on failure despite "advisory" label.

The job name includes "(advisory)" but the step will fail and block the workflow if ts_syntax_check.mjs exits with code 1. From the TypeScript syntax checker implementation, it returns exit code 1 when syntax errors are found.

For consistency with the python-syntax job (lines 58-79), either add error handling to parse the JSON output and emit warnings without failing, or use continue-on-error: true to make this truly advisory.

🔧 Proposed fix options

Option 1: Parse JSON and emit warnings like python-syntax

       - name: syntax-check all .ts files
-        run: node scripts/ts_syntax_check.mjs --json
+        run: |
+          node scripts/ts_syntax_check.mjs --json > /tmp/ts-syntax.json
+          node -e "
+          const data = require('/tmp/ts-syntax.json');
+          const failed = data.results.filter(r => r.status === 'failed');
+          console.log(\`TypeScript: \${data.checked} files, \${data.passed} passed, \${data.failed} failed\`);
+          for (const f of failed) {
+            console.log(\`::warning file=\${f.file}::syntax error: \${f.message}\`);
+          }
+          "

Option 2: Use continue-on-error

       - name: syntax-check all .ts files
         run: node scripts/ts_syntax_check.mjs --json
+        continue-on-error: true
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
typescript-syntax:
name: typescript syntax check (advisory)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- uses: actions/setup-node@e62d1c64a114b2d4b223e6e62319cbfa45a42c6a # v4
with:
node-version: "22"
- name: install typescript
run: npm install --no-save typescript
- name: syntax-check all .ts files
run: node scripts/ts_syntax_check.mjs --json
typescript-syntax:
name: typescript syntax check (advisory)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- uses: actions/setup-node@e62d1c64a114b2d4b223e6e62319cbfa45a42c6a # v4
with:
node-version: "22"
- name: install typescript
run: npm install --no-save typescript
- name: syntax-check all .ts files
run: node scripts/ts_syntax_check.mjs --json
continue-on-error: true
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/curriculum.yml around lines 81 - 94, The typescript-syntax
job is labeled advisory but will fail the workflow because the step running node
scripts/ts_syntax_check.mjs exits non-zero on errors; update the
typescript-syntax job to be non-blocking by either (A) replacing the single-step
run of node scripts/ts_syntax_check.mjs in the "syntax-check all .ts files" step
with a small wrapper that captures the JSON output from
scripts/ts_syntax_check.mjs, parses it and prints warnings (mirroring the
python-syntax handling), and exits 0, or (B) simpler: set continue-on-error:
true on the "syntax-check all .ts files" step so failures are advisory; modify
the typescript-syntax job/step accordingly.


quiz-coverage:
name: quiz coverage report
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
with:
python-version: "3.12"
- name: report quiz coverage
run: |
python3 scripts/quiz_coverage.py --json > /tmp/coverage.json
python3 -c "
import json
data = json.load(open('/tmp/coverage.json'))
total = data['total_lessons']
with_q = data['total_with_quiz']
pct = round(with_q / total * 100, 1) if total else 0
print(f'Quiz coverage: {with_q}/{total} ({pct}%)')
for p in sorted(data['phases'], key=lambda x: x['coverage_pct']):
if p['coverage_pct'] < 10:
print(f'::warning title=Quiz coverage phase {p[\"phase_id\"]:02d}::{p[\"phase_name\"]}: {p[\"with_quiz\"]}/{p[\"total_lessons\"]} ({p[\"coverage_pct\"]}%)')
"

# ── Pytest (lessons that have tests) ──────────────────────────────────

pytest:
name: lesson tests
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
# Only phases that actually contain test files — discovered at
# script-write time. Add rows as tests proliferate.
- phase: 19
label: "capstone"
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
with:
python-version: "3.12"
- name: install test deps
run: |
python3 -m pip install --quiet pytest pytest-timeout
# Many capstone tests import torch / numpy / h5py / zstandard / safetensors.
# Install everything the allowlist permits so tests don't fail on imports.
python3 -m pip install --quiet \
numpy \
torch \
h5py \
zstandard \
safetensors \
|| echo "::warning::Some test dependencies failed to install (may be expected)"
- name: run tests (phase ${{ matrix.phase }})
run: python3 -m pytest phases/${{ matrix.phase }}*/ --timeout=60 -v --tb=short || echo "::warning::Some tests failed"

# ── Auto-sync jobs (main only) ────────────────────────────────────────

readme-counts-sync:
name: README counts auto-fix (main only)
runs-on: ubuntu-latest
Expand Down Expand Up @@ -78,7 +190,6 @@ jobs:
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add README.md
git commit -m "$BOT_COMMIT_PREFIX"
# Retry on non-fast-forward when another merge to main races us
branch="${GITHUB_REF#refs/heads/}"
for attempt in 1 2 3 4 5; do
if git push origin "HEAD:${branch}"; then
Expand Down
46 changes: 46 additions & 0 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
#
# Install: pip install pre-commit && pre-commit install
# Run once: pre-commit run --all-files

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
args: [--allow-multiple]
- id: check-json
- id: check-toml
- id: check-added-large-files
args: [--maxkb=500]
- id: check-merge-conflict
- id: mixed-line-ending
args: [--fix=lf]
- id: detect-private-key

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.11.8
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
- id: ruff-format

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.15.0
hooks:
- id: mypy
args: [--config-file=pyproject.toml]
additional_dependencies: []
# mypy is advisory only — do not block commits
pass_filenames: false
Comment on lines +31 to +38
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Misleading advisory comment for mypy hook.

The comment states "mypy is advisory only — do not block commits", but pre-commit hooks block by default when they fail. There is no verbose: true or other mechanism to make this hook non-blocking. If mypy detects type issues, it will prevent the commit.

Either remove the misleading comment or add verbose: true to the hook configuration to make it truly advisory.

📝 Proposed fix options

Option 1: Remove the misleading comment

   - repo: https://github.com/pre-commit/mirrors-mypy
     rev: v1.15.0
     hooks:
       - id: mypy
         args: [--config-file=pyproject.toml]
         additional_dependencies: []
-        # mypy is advisory only — do not block commits
         pass_filenames: false

Option 2: Make it truly advisory with verbose mode

   - repo: https://github.com/pre-commit/mirrors-mypy
     rev: v1.15.0
     hooks:
       - id: mypy
         args: [--config-file=pyproject.toml]
         additional_dependencies: []
         # mypy is advisory only — do not block commits
+        verbose: true
         pass_filenames: false
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.15.0
hooks:
- id: mypy
args: [--config-file=pyproject.toml]
additional_dependencies: []
# mypy is advisory only — do not block commits
pass_filenames: false
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.15.0
hooks:
- id: mypy
args: [--config-file=pyproject.toml]
additional_dependencies: []
pass_filenames: false
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.pre-commit-config.yaml around lines 31 - 38, The comment "mypy is advisory
only — do not block commits" is misleading because the pre-commit hook with id:
mypy currently blocks commits on failure; either remove that comment or make the
hook non-blocking by adding the advisory option (e.g., add verbose: true or set
the hook to not fail) to the mypy hook configuration; update the block that
contains repo: https://github.com/pre-commit/mirrors-mypy, rev: v1.15.0, hooks
-> - id: mypy to reflect the chosen change.


- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.1.0
hooks:
- id: prettier
types_or: [javascript, css, html, json, markdown, yaml]
args: [--prose-wrap=always, --print-width=100]
exclude: ^glossary/|^phases/|^site/data\.js$
100 changes: 93 additions & 7 deletions LESSON_TEMPLATE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,28 @@ Use this template when creating a new lesson. Copy the folder structure and fill
```
NN-lesson-name/
├── code/
│ ├── main.py (primary implementation)
│ ├── main.py (primary implementation — always a code/ dir)
│ ├── main.ts (TypeScript version, if applicable)
│ ├── main.rs (Rust version, if applicable)
│ └── main.jl (Julia version, if applicable)
├── notebook/
│ └── lesson.ipynb (Jupyter notebook for experimentation)
├── docs/
│ └── en.md (lesson documentation)
│ └── en.md (lesson documentation — 6-beat format)
├── quiz.json (exactly 6 questions: 1 pre + 3 check + 2 post)
└── outputs/
├── prompt-*.md (prompts produced by this lesson)
└── skill-*.md (skills produced by this lesson)
├── skill-*.md (skills produced by this lesson)
└── agent-*.md (agent definitions produced by this lesson)
```

A lesson MUST have at minimum:
- `code/` with at least one source or config file (rule L005)
- `docs/en.md` with at least 200 bytes and an H1 header (rules L002-L004)
- `quiz.json` with exactly 6 questions in the canonical schema (rules L006-L009)

The `notebook/` directory is **not** part of the standard lesson. If a lesson
benefits from a Jupyter notebook, add one in `code/` beside the implementation
files, or reference an external Colab link in the "Further Reading" section.

## Documentation Format (docs/en.md)

```markdown
Expand Down Expand Up @@ -93,12 +102,76 @@ Include it here and save it in the outputs/ folder.]
- [Resource 2](url) — [why it's worth reading]
```

## Quiz Format (quiz.json)

Every lesson must have a `quiz.json` with exactly 6 questions following the
canonical schema:

```json
{
"questions": [
{
"stage": "pre",
"question": "...",
"options": ["A", "B", "C", "D"],
"correct": 0,
"explanation": "..."
},
{
"stage": "check",
"question": "...",
"options": ["A", "B", "C", "D"],
"correct": 0,
"explanation": "..."
},
{
"stage": "check",
"question": "...",
"options": ["A", "B", "C", "D"],
"correct": 0,
"explanation": "..."
},
{
"stage": "check",
"question": "...",
"options": ["A", "B", "C", "D"],
"correct": 0,
"explanation": "..."
},
{
"stage": "post",
"question": "...",
"options": ["A", "B", "C", "D"],
"correct": 0,
"explanation": "..."
},
{
"stage": "post",
"question": "...",
"options": ["A", "B", "C", "D"],
"correct": 0,
"explanation": "..."
}
]
}
```

Rules (enforced by `scripts/audit_lessons.py`):
- **L006**: `quiz.json` must be valid JSON with a non-empty `questions` array
- **L007**: Legacy keys (`q`, `choices`, `answer`) are rejected — use canonical keys only
- **L008**: Options must be 2–6 items
- **L009**: `correct` must be a valid index within the options array

## Code File Guidelines

- Code must run without errors
- No comments — code should be self-explanatory
- Add a 4–6 line header comment citing the lesson path and any external spec/RFC
referenced by the implementation
- Use inline comments sparingly — let the code speak, but don't be dogmatic about
"zero comments." Some algorithms need a one-liner to orient the reader
- Use the language that fits best for the topic
- Include a `requirements.txt` or equivalent if there are dependencies
- Include a `# requires: pkg1, pkg2` comment at the top if your entry file needs
packages outside the Python stdlib (see `scripts/lesson_run.py`)
- Start simple, build up complexity
- Every function and class should have a clear purpose

Expand Down Expand Up @@ -131,3 +204,16 @@ tags: [relevant, tags]

[Skill content]
```

### Agents

```markdown
---
name: agent-name
description: What this agent does
phase: [phase number]
lesson: [lesson number]
---

[Agent definition]
```
Loading