chore(agents): Stop reviewer agents from pre-filtering findings

[yamadashy]

Adjusts the six reviewer-* agents so they report everything they find instead of pre-filtering, and makes the orchestrator the single filter.

Why

The reviewer agents each opened with "report only noteworthy findings", while pr-review and review-loop already re-filter their output ("keep only what you also deem noteworthy"). That is two filters in series, and the first one runs with the least context — a finding an agent suppresses can never be recovered, while one the orchestrator rejects costs a single line.

Anthropic's Opus 5 prompting guide names this pattern directly: review prompts that say "only report high-severity issues" or "be conservative" are followed literally and lower recall, and filtering should happen in a separate pass.

What changed

• All 6 reviewers — the "only noteworthy" opener becomes "report every finding you have concrete evidence for, with severity and confidence; the orchestrator triages".
• Output formats — added a Confidence field (and Severity for conventions) so the triage pass has something to filter on.
• Suppression rules removed — security's "skip theoretical concerns" / "rate conservatively", test-coverage's "prefer fewer high-confidence findings" and "don't flag low-priority gaps when coverage is good", performance's confidence-based dropping.
• Scope rules kept — each reviewer stays in its own angle, linter territory stays out of conventions/code-quality, "don't invent issues" stays everywhere. Performance keeps its Flagging Threshold, since that defines the domain rather than suppressing uncertainty.
• pr-review / review-loop — state the contract on the orchestrator side too: the agents don't pre-filter, you are the filter, drop low-confidence/low-severity unless you can confirm them yourself.

Prompt-only change; no source or test files are touched.

Checklist

• Run npm run test — not run; no source or test files changed
• Run npm run lint — run; the only failures come from the untracked browser/web-ext-artifacts/ build output, unrelated to this diff

🤖 Generated with Claude Code

[github-actions]

⚡ Performance Benchmark

Latest commit:	7fee47c chore(agents): Address PR review feedback on reviewer prompts
Status:	✅ Benchmark complete!
Ubuntu:	0.64s (±0.01s) → 0.65s (±0.01s) · +0.00s (+0.5%)
macOS:	0.84s (±0.15s) → 0.84s (±0.11s) · -0.00s (-0.1%)
Windows:	1.13s (±0.03s) → 1.12s (±0.02s) · -0.01s (-0.9%)

Details

• Packing the repomix repository with node bin/repomix.cjs
• Warmup: 2 runs (discarded), interleaved execution
• Measurement: 20 runs / 30 on macOS (median ± IQR)
• Workflow run

History

1b3b372 chore(agents): Stop reviewer agents from pre-filtering findings

Ubuntu:	0.87s (±0.04s) → 0.88s (±0.02s) · +0.01s (+1.4%)
macOS:	0.51s (±0.02s) → 0.53s (±0.06s) · +0.02s (+3.7%)
Windows:	1.11s (±0.02s) → 1.10s (±0.04s) · -0.01s (-0.5%)

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 804b6220-b63b-484c-96c6-f63a3ffc17f5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Reviewer prompts now require broader evidence-based finding reports with severity and confidence. Finding formats include uncertainty details. Review commands now triage unfiltered findings and retain only confirmed actionable results.

Changes

Reviewer reporting and triage

Layer / File(s)	Summary
Reviewer reporting scope .agents/agents/reviewer-*.md	The reviewer agents now report all evidence-based findings within scope, including borderline and low-priority findings where defined.
Finding fields and uncertainty guidance .agents/agents/reviewer-*.md	Finding formats now include severity and confidence. Reviewers must provide uncertainty details and type-specific evidence, scenarios, impact, or test cases.
Review workflow triage .agents/commands/code/review-loop.md, .agents/commands/git/pr-review.md	The workflow now collects severity and confidence, filters low-confidence or low-severity findings, and independently confirms retained findings when required.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• yamadashy/repomix#1286: Updates the review workflow in .agents/commands/git/pr-review.md.
• yamadashy/repomix#1347: Updates the same reviewer agent and review command files.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: reviewer agents no longer pre-filter findings.
Description check	✅ Passed	The description explains the changes, rationale, scope, and checklist status, and it matches the required template.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch chore/review-agents-report-all

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.agents/agents/reviewer-performance.md:
- Line 75: Update the fallback guidance in the performance review instructions
so suspected costs are reported only when supported by concrete evidence that
they exceed the established realistic-scale threshold. Keep the threshold and
scope boundaries from the surrounding guidance, and do not require reporting
cold-path or micro-optimization concerns without threshold-relevant evidence.

In @.agents/agents/reviewer-test-coverage.md:
- Around line 95-97: Update the finding requirements in the
reviewer-test-coverage guidance so Confidence is mandatory for every finding,
including High-confidence findings. Align the wording with the required output
contract in review-loop and pr-review by removing the exception that permits
High confidence to be omitted.

In @.agents/commands/code/review-loop.md:
- Line 14: Update the triage wording in `.agents/commands/code/review-loop.md`
at lines 14-14 and `.agents/commands/git/pr-review.md` at lines 18-18, replacing
“low-confidence and low-severity” with “low-confidence or low-severity” so
either condition requires confirmation.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8026f948-e827-491b-9578-6602036a0040

📥 Commits

Reviewing files that changed from the base of the PR and between ebed8e7 and 1b3b372.

📒 Files selected for processing (8)

• .agents/agents/reviewer-code-quality.md
• .agents/agents/reviewer-conventions.md
• .agents/agents/reviewer-holistic.md
• .agents/agents/reviewer-performance.md
• .agents/agents/reviewer-security.md
• .agents/agents/reviewer-test-coverage.md
• .agents/commands/code/review-loop.md
• .agents/commands/git/pr-review.md