synthetic-peer-review

A reusable Claude Science skill that runs a synthetic (mock) peer-review panel on one or more manuscripts before submission.

It crafts several independent expert reviewers with distinct, complementary personas, briefs each on the correct framing (e.g. AI-generated, in-silico, wet-lab out of scope), then synthesizes their reviews into a handling-editor decision letter with a per-manuscript decision and a prioritized, convergent Essential-Revisions list.

Contents

• SKILL.md — the skill card (metadata + guidance).
• kernel.py — Python helpers auto-loaded into the kernel when the skill is loaded:
  • build_shared_frame(work_description, in_scope, out_of_scope, manuscript_blurbs)
  • run_peer_review_panel(host, manuscripts, shared_frame, personas=None, max_tokens=8000, ...)
  • write_peer_review_report(...), DEFAULT_PERSONAS, REVIEW_STRUCTURE, persona_system

Quick start

frame = build_shared_frame(
    work_description="A human-supervised AI agent produced these manuscripts on public data (disclosed, part of the contribution).",
    in_scope="additional in-silico analyses, statistical robustness, clarifications, tempered claims, better caveats, restructuring",
    out_of_scope="wet-bench / clinical experimentation",
    manuscript_blurbs=["PAPER A (research article): title — one line", "PAPER B (Perspective): title — one line"],
)
result = run_peer_review_panel(host, [("PAPER A", md_a), ("PAPER B", md_b)], frame)
# -> {"reviews": [...], "editor": <decision letter>, "recommendations": [...]}
# also writes peer_review_report.md (+ .docx)

Design notes

• Reviewers run on a Sonnet-class reasoning model, fanned out in parallel; each returns per-manuscript summary / strengths / major & minor concerns / feasible revisions / recommendation, plus a cross-cutting comment.
• The editor step deduplicates and prioritizes concerns raised by ≥2 reviewers.
• Advisory only. Reviewers are LLM instances — verify any specific factual claim (a p-value, "paper X shows Y") against the source before acting on it.

Generated as part of an AI-assisted eosinophilic-esophagitis research campaign.