Skip to content

feat: add multi-persona review panel for document critique - #25

Merged
rjwalters merged 1 commit into
mainfrom
feature/issue-4
Apr 14, 2026
Merged

feat: add multi-persona review panel for document critique#25
rjwalters merged 1 commit into
mainfrom
feature/issue-4

Conversation

@rjwalters

Copy link
Copy Markdown
Owner

Closes #4

Note: Builder completed changes but exited before creating a PR. PR created via direct completion.

Changes

packages/review-panel/package.json                |  27 +++
 packages/review-panel/src/aggregator.ts           | 199 +++++++++++++++++++
 packages/review-panel/src/index.ts                |  56 ++++++
 packages/review-panel/src/personas.ts             | 135 +++++++++++++
 packages/review-panel/src/runner.ts               | 117 +++++++++++
 packages/review-panel/src/test/aggregator.test.ts | 227 ++++++++++++++++++++++
 packages/review-panel/src/test/runner.test.ts     | 163 ++++++++++++++++
 packages/review-panel/src/types.ts                | 121 ++++++++++++
 packages/review-panel/tsconfig.json               |  15 ++
 9 files changed, 1060 insertions(+)

Commits

  • 12c2df9 feat: add multi-persona review panel package

Test plan

  • Verify changes match issue requirements
  • Confirm tests pass

@rjwalters rjwalters added loom:review-requested PR ready for Judge to review loom:reviewing Judge is actively reviewing this PR labels Apr 14, 2026
@rjwalters

Copy link
Copy Markdown
Owner Author

Approved!

Thorough evaluation of the multi-persona review panel implementation. Excellent work.

Evaluation

Acceptance Criteria (Issue #4):

  • ✅ Persona format defined (Persona interface with id, name, perspective, dimensions, promptTemplate)
  • ✅ Four default personas: Critical Editor, Domain Expert, General Reader, Style Reviewer
  • ✅ Independent execution via Promise.all in runAllPersonas()
  • ✅ Aggregation with consensus clustering and ranking by strength (4/4 > 3/4 > 2/4 > 1/4)
  • ✅ Disagreements explicitly separated (consensusItems vs disagreements)
  • ✅ User-configurable personas via ReviewPanelOptions.personas

Code Quality:

  • Clean TypeScript with strict: true
  • Well-structured types with thorough JSDoc documentation
  • Robust JSON parsing with markdown fence stripping and graceful fallback
  • Input validation for categories and severities
  • Clean public API via reviewDocument() entry point

Testing:

  • 20 tests across 9 suites — all pass
  • Good coverage of edge cases (invalid JSON, missing descriptions, same-persona clustering prevention)
  • Tests for custom/user-defined personas

Design Notes:

  • Text similarity using Jaccard index is a pragmatic choice with appropriate comment noting embeddings would improve accuracy
  • Single-linkage greedy clustering is reasonable for the expected scale
  • callModel callback pattern keeps the package backend-agnostic — good design

Test Execution

Test plan from PR description:

  1. Verify changes match issue requirements — ✅ All 6 acceptance criteria met
  2. Confirm tests pass — ✅ 20/20 tests pass, type checking clean

Test Scoping

Strategy: Full build + test (new package, no pre-existing tests)
Result: tsc --noEmit clean, 20 tests pass in 322ms

@rjwalters rjwalters added loom:pr PR approved by Judge, ready for human to merge and removed loom:reviewing Judge is actively reviewing this PR loom:review-requested PR ready for Judge to review labels Apr 14, 2026
Implement a review panel that runs multiple AI personas independently
against a document, then aggregates findings by consensus strength.

Includes:
- Four default personas (Critical Editor, Domain Expert, General Reader,
  Style Reviewer) with configurable prompt templates
- Parallel persona execution via pluggable callModel interface
- Consensus clustering using word-overlap similarity
- Aggregated results ranked by agreement strength (4/4 > 3/4 > etc.)
- Explicit separation of consensus items vs. disagreements
- Support for user-configurable custom personas
- 20 passing unit tests covering parsing, clustering, and aggregation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@rjwalters
rjwalters merged commit 3e70745 into main Apr 14, 2026
@rjwalters
rjwalters deleted the feature/issue-4 branch April 14, 2026 04:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

loom:pr PR approved by Judge, ready for human to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add multi-persona review panel for document critique

1 participant