Skip to content

Add unresolved PR review skill - #47

Merged
kin0992 merged 3 commits into
mainfrom
feats/review-pr-comments
Jul 16, 2026
Merged

Add unresolved PR review skill#47
kin0992 merged 3 commits into
mainfrom
feats/review-pr-comments

Conversation

@kin0992

@kin0992 kin0992 commented Jul 16, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Why

Unresolved pull-request feedback needs a consistent workflow that includes agent-authored comments, records explicit decisions, and prevents accidental replies.

What

  • add the engineering-skills marketplace plugin and review-pr-comments skill
  • collect every actionable comment in unresolved review threads through paginated gh api graphql calls
  • map accept, defer, and ignore decisions to 👍, 👀, and 👎 reactions with gh api
  • suggest replies while requiring exact-text approval before posting
  • add Waza evaluations, package exports, documentation, and a changeset

How to verify

  • run the documented gh api graphql collection query against a pull request
  • run waza check and the review-pr-comments evaluation suite
  • run the skills lint, formatting, and marketplace checks

Risks / follow-ups

  • reactions and replies require an authenticated gh session with access to the target repository
  • accepted suggestions are not implemented automatically

kin0992 and others added 2 commits July 16, 2026 15:39
@kin0992
Add unresolved PR review skill
264b27a 
Introduce an engineering workflow for evaluating every actionable unresolved review comment, recording explicit decisions with reactions, and posting suggested replies only after exact user approval.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@kin0992
Ignore Python bytecode
70f7a2f 
Keep generated Python cache files out of the engineering skill package.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@changeset-bot

changeset-bot Bot commented Jul 16, 2026 •
edited
Loading

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: d0a075f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@kin0992/skills Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

=== review-pr-comments ===

Running benchmark: review-pr-comments-eval
Skill: engineering/review-pr-comments
Engine: mock
Model: claude-sonnet-4-20250514

✓ [1/3] Review Agent Feedback
✓ [2/3] Review Unresolved Comments
✓ [3/3] Should Not Trigger

🧪 Waza Eval Results

Status: ✅ Passed | Score: 1.00 | Duration: 229ms

  • Tests: 3 total, 3 passed, 0 failed, 0 errors
  • Success Rate: 100.0%
  • Score Range: 1.00 - 1.00 (σ=0.0000)

Task Results

Task Score Status Graders
Review Agent Feedback 1.00 ✅ has_output
Review Unresolved Comments 1.00 ✅ has_output
Should Not Trigger 1.00 ✅ has_output

Benchmark: review-pr-comments-eval | Skill: engineering/review-pr-comments | Model: claude-sonnet-4-20250514

=== branch-name ===

Running benchmark: branch-name-eval
Skill: git/branch-name
Engine: mock
Model: claude-sonnet-4-20250514

✓ [1/3] Feature Branch Naming
✓ [2/3] Fix Branch Naming
✓ [3/3] Should Not Trigger

🧪 Waza Eval Results

Status: ✅ Passed | Score: 1.00 | Duration: 229ms

  • Tests: 3 total, 3 passed, 0 failed, 0 errors
  • Success Rate: 100.0%
  • Score Range: 1.00 - 1.00 (σ=0.0000)

Task Results

Task Score Status Graders
Feature Branch Naming 1.00 ✅ has_output, no_ids
Fix Branch Naming 1.00 ✅ has_output, no_ids
Should Not Trigger 1.00 ✅ has_output, no_ids

Benchmark: branch-name-eval | Skill: git/branch-name | Model: claude-sonnet-4-20250514

=== commit-message ===

Running benchmark: commit-message-eval
Skill: git/commit-message
Engine: mock
Model: claude-sonnet-4-20250514

✓ [1/3] Basic Bug Fix Commit
✓ [2/3] Multi-Area Diff
✓ [3/3] Should Not Trigger - PR Description Request

🧪 Waza Eval Results

Status: ✅ Passed | Score: 1.00 | Duration: 230ms

  • Tests: 3 total, 3 passed, 0 failed, 0 errors
  • Success Rate: 100.0%
  • Score Range: 1.00 - 1.00 (σ=0.0000)

Task Results

Task Score Status Graders
Basic Bug Fix Commit 1.00 ✅ has_output, no_conventional_commits, no_period_ending
Multi-Area Diff 1.00 ✅ has_output, no_conventional_commits, no_period_ending
Should Not Trigger - PR Description Request 1.00 ✅ has_output, no_conventional_commits, no_period_ending

Benchmark: commit-message-eval | Skill: git/commit-message | Model: claude-sonnet-4-20250514

=== pr-title-description ===

Running benchmark: pr-title-description-eval
Skill: git/pr-title-description
Engine: mock
Model: claude-sonnet-4-20250514

✓ [1/3] Basic Feature PR
✓ [2/3] PR With Template
✓ [3/3] Should Not Trigger - Commit Message Request

🧪 Waza Eval Results

Status: ✅ Passed | Score: 1.00 | Duration: 232ms

  • Tests: 3 total, 3 passed, 0 failed, 0 errors
  • Success Rate: 100.0%
  • Score Range: 1.00 - 1.00 (σ=0.0000)

Task Results

Task Score Status Graders
Basic Feature PR 1.00 ✅ has_output
PR With Template 1.00 ✅ has_output
Should Not Trigger - Commit Message Request 1.00 ✅ has_output

Benchmark: pr-title-description-eval | Skill: git/pr-title-description | Model: claude-sonnet-4-20250514

═══════════════════════════════════════════════
MULTI-SKILL RUN SUMMARY
═══════════════════════════════════════════════

Skill Status Pass Rate Avg Score
──────────────────────────────────────────────────────────────────────
review-pr-comments ✅ Passed 100.0% 1.00
branch-name ✅ Passed 100.0% 1.00
commit-message ✅ Passed 100.0% 1.00
pr-title-description ✅ Passed 100.0% 1.00

@kin0992
Use gh for PR review workflow
d0a075f 
Remove the Python helper and document direct gh GraphQL and REST commands for paginated collection, decision reactions, and approval-gated replies.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

=== review-pr-comments ===

Running benchmark: review-pr-comments-eval
Skill: engineering/review-pr-comments
Engine: mock
Model: claude-sonnet-4-20250514

✓ [1/3] Review Agent Feedback
✓ [2/3] Review Unresolved Comments
✓ [3/3] Should Not Trigger

🧪 Waza Eval Results

Status: ✅ Passed | Score: 1.00 | Duration: 206ms

  • Tests: 3 total, 3 passed, 0 failed, 0 errors
  • Success Rate: 100.0%
  • Score Range: 1.00 - 1.00 (σ=0.0000)

Task Results

Task Score Status Graders
Review Agent Feedback 1.00 ✅ has_output
Review Unresolved Comments 1.00 ✅ has_output
Should Not Trigger 1.00 ✅ has_output

Benchmark: review-pr-comments-eval | Skill: engineering/review-pr-comments | Model: claude-sonnet-4-20250514

=== branch-name ===

Running benchmark: branch-name-eval
Skill: git/branch-name
Engine: mock
Model: claude-sonnet-4-20250514

✓ [1/3] Feature Branch Naming
✓ [2/3] Fix Branch Naming
✓ [3/3] Should Not Trigger

🧪 Waza Eval Results

Status: ✅ Passed | Score: 1.00 | Duration: 206ms

  • Tests: 3 total, 3 passed, 0 failed, 0 errors
  • Success Rate: 100.0%
  • Score Range: 1.00 - 1.00 (σ=0.0000)

Task Results

Task Score Status Graders
Feature Branch Naming 1.00 ✅ has_output, no_ids
Fix Branch Naming 1.00 ✅ has_output, no_ids
Should Not Trigger 1.00 ✅ has_output, no_ids

Benchmark: branch-name-eval | Skill: git/branch-name | Model: claude-sonnet-4-20250514

=== commit-message ===

Running benchmark: commit-message-eval
Skill: git/commit-message
Engine: mock
Model: claude-sonnet-4-20250514

✓ [1/3] Basic Bug Fix Commit
✓ [2/3] Multi-Area Diff
✓ [3/3] Should Not Trigger - PR Description Request

🧪 Waza Eval Results

Status: ✅ Passed | Score: 1.00 | Duration: 207ms

  • Tests: 3 total, 3 passed, 0 failed, 0 errors
  • Success Rate: 100.0%
  • Score Range: 1.00 - 1.00 (σ=0.0000)

Task Results

Task Score Status Graders
Basic Bug Fix Commit 1.00 ✅ has_output, no_conventional_commits, no_period_ending
Multi-Area Diff 1.00 ✅ has_output, no_conventional_commits, no_period_ending
Should Not Trigger - PR Description Request 1.00 ✅ has_output, no_conventional_commits, no_period_ending

Benchmark: commit-message-eval | Skill: git/commit-message | Model: claude-sonnet-4-20250514

=== pr-title-description ===

Running benchmark: pr-title-description-eval
Skill: git/pr-title-description
Engine: mock
Model: claude-sonnet-4-20250514

✓ [1/3] Basic Feature PR
✓ [2/3] PR With Template
✓ [3/3] Should Not Trigger - Commit Message Request

🧪 Waza Eval Results

Status: ✅ Passed | Score: 1.00 | Duration: 206ms

  • Tests: 3 total, 3 passed, 0 failed, 0 errors
  • Success Rate: 100.0%
  • Score Range: 1.00 - 1.00 (σ=0.0000)

Task Results

Task Score Status Graders
Basic Feature PR 1.00 ✅ has_output
PR With Template 1.00 ✅ has_output
Should Not Trigger - Commit Message Request 1.00 ✅ has_output

Benchmark: pr-title-description-eval | Skill: git/pr-title-description | Model: claude-sonnet-4-20250514

═══════════════════════════════════════════════
MULTI-SKILL RUN SUMMARY
═══════════════════════════════════════════════

Skill Status Pass Rate Avg Score
──────────────────────────────────────────────────────────────────────
review-pr-comments ✅ Passed 100.0% 1.00
branch-name ✅ Passed 100.0% 1.00
commit-message ✅ Passed 100.0% 1.00
pr-title-description ✅ Passed 100.0% 1.00

@kin0992
kin0992 merged commit 225728f into main Jul 16, 2026
6 of 7 checks passed
@kin0992
kin0992 deleted the feats/review-pr-comments branch July 16, 2026 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@kin0992