Skip to content

Parallelize build-time meta-image rendering with worker threads#20105

Merged
jeffmerrick merged 2 commits into
masterfrom
jeffmerrick/meta-image-build-time
Jul 8, 2026
Merged

Parallelize build-time meta-image rendering with worker threads#20105
jeffmerrick merged 2 commits into
masterfrom
jeffmerrick/meta-image-build-time

Conversation

@jeffmerrick

@jeffmerrick jeffmerrick commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Proposed changes

Extracts the social-card templates and Satori/resvg rendering from scripts/generate-meta-images.mjs into scripts/meta-images/render.mjs, shared by the generator and a new worker_threads entry (scripts/meta-images/render-worker.mjs). Cold runs — a fresh checkout or an OG_TEMPLATE_VERSION bump, where every card re-renders — now fan out across a worker pool sized to the machine's cores (override with OG_WORKERS). Small incremental batches (fewer than 8 cards, the common case) and OG_WORKERS=1 still render inline, skipping worker startup. The moved template code is byte-identical, so cache keys and rendered output are unchanged; verified both paths locally (0 failures).

Unreleased product version (optional)

n/a

Related issues (optional)

n/a

@jeffmerrick @claude
Parallelize build-time meta-image rendering with worker threads
1ed631a 
Extract the card templates and Satori/resvg rendering from
generate-meta-images.mjs into scripts/meta-images/render.mjs so the
generator and a new worker_threads entry (render-worker.mjs) share one
definition. Cold runs (fresh checkout or OG_TEMPLATE_VERSION bump) fan
out across a worker pool sized to the machine; small incremental
batches (< 8 cards) or OG_WORKERS=1 render inline to skip worker
startup. No visual change — the moved template code is byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pulumi-bot

pulumi-bot commented Jul 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Your site preview for commit 121fe14 is ready! 🎉

http://www-testing-pulumi-docs-origin-pr-20105-121fe142.s3-website.us-west-2.amazonaws.com

@jeffmerrick jeffmerrick marked this pull request as ready for review July 7, 2026 14:54
@github-actions github-actions Bot added the review:triaging Claude Triage is currently classifying the PR label Jul 7, 2026
@jeffmerrick jeffmerrick requested a review from cnunciato July 7, 2026 14:54
@github-actions github-actions Bot added domain:infra PR touches workflows, scripts, infra, Makefile, or build config review:in-progress Claude review is currently running and removed review:triaging Claude Triage is currently classifying the PR labels Jul 7, 2026
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Pre-merge Review — Last updated 2026-07-07T14:56:25Z

Tip

Summary: Build-tooling change (no reader-facing content): the meta-image generator is refactored to render social cards in parallel. The pure card-rendering templates move into a new shared scripts/meta-images/render.mjs, and a worker_threads pool (render-worker.mjs) fans cold runs out across CPUs, while small incremental batches still render inline to skip worker startup. The kind of wrongness that would matter here is a build regression — a worker-pool bug that hangs the build or silently drops cards, or the template extraction changing the rendered output. I read both new modules and the rewritten runGenerate/renderPool: the extracted templates are byte-for-byte the originals with the same renderPng signature, the pool settles every job exactly once and drains stragglers if all workers die (so it can't hang), and the OG_WORKERS/PARALLEL_THRESHOLD gating matches its documented behavior. The infra review and claim-extraction passes ran; no content, frontmatter, or Hugo-build surface is touched.

Review confidence:

Dimension Level Notes
mechanics HIGH
facts HIGH No external factual claims; the one extracted item is a design-choice code comment.
Investigation log
  • Cross-sibling reads: not run (not in a templated section)
  • External claim verification: 0 of 1 claims verified (0 unverifiable, 0 contradicted) · 4 specialists (numerical, cross-reference, capability, framing); 0 cross-specialist corroborations · routed: 0 inline, 0 Pass 1, 0 Pass 2, 1 Pass 3 (verified 0, contradicted 0, unverifiable 1).
  • Cited-claim spot-checks: not run (no cited claims)
  • Frontmatter sweep: not run (no frontmatter in diff)
  • Temporal-trigger sweep: ran (recency words present in diff; spot-check in-review)
  • Code execution: not run (no static/programs/ change)
  • Code-examples checks: not run (no fenced code blocks in content files)
  • Editorial-balance pass: not run (not under content/blog/)
🚨 Outstanding ⚠️ Low-confidence 💡 Pre-existing ✅ Resolved
0 0 0 0

🔍 Verification trail

1 claims extracted · 0 verified · 0 unverifiable · 0 contradicted
  • L59 in scripts/meta-images/render.mjs "// Largest font (96..40) whose wrapped title fits 90% of the box height;" → ➖ not-a-claim (evidence: This is a code comment describing the PR author's own font-sizing logic/design choice within their own script, not a falsifiable third-party claim.; source: scripts/meta-images/render.mjs L59)

🚨 Outstanding in this PR

No outstanding findings in this PR.

⚠️ Low-confidence

No low-confidence findings.

💡 Pre-existing issues in touched files (optional)

No pre-existing issues in touched files.

✅ Resolved since last review

No items resolved since the last review.

📜 Review history

  • 2026-07-07T14:56:25Z — Infra-only refactor parallelizing meta-image rendering; reviewed the extracted render.mjs templates and the worker pool, no blocking or low-confidence findings (1ed631a)

Need a re-review? Want to dispute a finding? Mention @claude and include #update-review.
(For ad-hoc questions or fixes, just @claude — no hashtag.)

@github-actions github-actions Bot added review:no-blockers Claude review completed cleanly; outstanding is empty and removed review:in-progress Claude review is currently running labels Jul 7, 2026
CamSoper
CamSoper approved these changes Jul 7, 2026
View reviewed changes

@CamSoper CamSoper left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cnunciato @claude
Harden meta-image render pool against hangs and silent total failures
121fe14 
Two robustness fixes to the build-time card generator:

- renderPool: add a worker `exit` handler so a worker that dies without
  emitting `error` (non-zero exit — OOM, native resvg crash) still settles
  its in-flight job instead of dangling it and hanging the build. Shared
  failure logic is factored into failWorker (idempotent per worker via the
  `live` guard), and a `done` flag stops the pool's own terminate()-driven
  exits from being counted as spurious failures.

- Fail the build on a total render wipeout by comparing failures against
  todo (what was attempted this run), not pages (all cards). With a warm
  cache most pages are cached, so the old pages.length check could never
  fire on an incremental run — a systemic render break would ship green
  with missing cards.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions github-actions Bot added review:stale New commits since last Claude review; refresh on next ready-transition or @claude mention and removed review:no-blockers Claude review completed cleanly; outstanding is empty labels Jul 7, 2026
cnunciato
cnunciato approved these changes Jul 7, 2026
View reviewed changes

@cnunciato cnunciato left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude had a few suggestions that seem reasonable, so pushed a commit to cover those. Nice work!

@jeffmerrick jeffmerrick merged commit e7a40c3 into master Jul 8, 2026
12 checks passed
@jeffmerrick jeffmerrick deleted the jeffmerrick/meta-image-build-time branch July 8, 2026 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CamSoper CamSoper CamSoper approved these changes
@cnunciato cnunciato cnunciato approved these changes

Assignees

No one assigned

Labels

domain:infra PR touches workflows, scripts, infra, Makefile, or build config review:stale New commits since last Claude review; refresh on next ready-transition or @claude mention

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@jeffmerrick @pulumi-bot @CamSoper @cnunciato