Skip to content

auto-improvement: daily paper-review loop from Awesome-LLM4Cybersecurity#66

Merged
killertcell428 merged 1 commit into
masterfrom
claude/happy-davinci-c231a6
May 18, 2026
Merged

auto-improvement: daily paper-review loop from Awesome-LLM4Cybersecurity#66
killertcell428 merged 1 commit into
masterfrom
claude/happy-davinci-c231a6

Conversation

@killertcell428
Copy link
Copy Markdown
Owner

Summary

  • Adds a half-automated daily loop that reads ~10 unseen papers/day from Awesome-LLM4Cybersecurity (639 entries today) and proposes Aigis detector candidates via Claude Haiku 4.5.
  • Each candidate becomes a stub in auto-improvement/pending/; a summary issue + bot PR is opened for human review every morning.
  • No detector code is auto-modified. Promotion to actual rules under aigis/ stays a human-driven PR — same convention as the existing 6-hour rotation loop.

Files

How it picks 10 papers

Newest-first by upstream date string. At 10/day, the current 639-entry list takes ~64 days to clear, after which the loop catches each new commit to upstream as the maintainers add it.

Required setup (before the first cron fires)

  • Add repo secret `ANTHROPIC_API_KEY` (Settings → Secrets → Actions).
  • Without it, the scheduled run will fail loudly; `workflow_dispatch` with `dry_run=true` works regardless and exercises the parser.

Cost guard

10 papers × Haiku 4.5 (≈500 output tokens) ≈ a few cents/day → <$1/month at the chosen budget.

Test plan

  • Local dry-run: `python auto-improvement/scripts/paper_review.py --dry-run --max-papers 5` → parses 639 entries, picks the 5 newest correctly.
  • After merge: trigger `workflow_dispatch` with `dry_run=true` to confirm the workflow runs end-to-end on CI without burning credits.
  • Add `ANTHROPIC_API_KEY` secret, then trigger one real run manually to validate the issue + bot PR are filed cleanly.
  • Verify the first scheduled cron at the next 00:15 UTC produces a sane batch.

🤖 Generated with Claude Code

Comment thread auto-improvement/scripts/paper_review.py Fixed
Comment thread auto-improvement/scripts/paper_review.py Fixed
Adds a half-automated loop that, every day at 00:15 UTC:

1. Fetches the upstream LITERATURES.md (639 entries today, growing).
2. Picks ~10 unseen newest entries and asks Claude Haiku 4.5 whether
   each one yields a concrete regex/substring-scoped Aigis detector.
3. Drafts a pending/ stub per relevant candidate (humans promote to
   real rule PRs; ROTATION-loop conventions preserved).
4. Files one summary issue + opens a bot PR with the new pending files
   and updated state.json.

No detector code is touched by the workflow itself — this is the
semi-automatic mode the user picked: candidates + draft patches in,
human review out.

Cost target: <$1/day (10 papers x Haiku ~500 out tokens).

Requires repo secret ANTHROPIC_API_KEY. Dry-run via workflow_dispatch
input dry_run=true exercises the parser without burning API credits.

Parser verified locally against the live source: 639 entries parsed,
newest-first ordering produces the expected top picks (SecLens 2026.04,
CyberThreat-Eval 2026.03, Minerva 2026.01).

Signed-off-by: killertcell428 <killertcell428@gmail.com>
@killertcell428 killertcell428 force-pushed the claude/happy-davinci-c231a6 branch from a0d5e81 to 5d5d962 Compare May 18, 2026 15:40
@killertcell428 killertcell428 merged commit 9bac8c2 into master May 18, 2026
11 checks passed
@killertcell428 killertcell428 deleted the claude/happy-davinci-c231a6 branch May 18, 2026 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants