Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions docs/todos/2026-06-17-issue-919-remember-redact-secrets/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Issue 919 Remember Secret Redaction Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Make the packaged `remember` skill avoid persisting raw secrets while preserving the useful memory fact.

**Architecture:** Keep the fix in the plugin skill documentation and contract tests. Add a focused test that pins the security wording and worked example, then update the skill files to satisfy it without changing runtime APIs.

**Tech Stack:** Markdown plugin skills, Vitest, TypeScript, pnpm.

---

## Files

- Modify: `test/plugin-surface-contract.test.ts`
- Modify: `plugin/skills/remember/SKILL.md`
- Modify: `plugin/skills/remember/EXAMPLES.md`
- Modify: `docs/todos/2026-06-17-issue-919-remember-redact-secrets/todo.md`

## Task 1: Add Remember Skill Redaction Contract

- [ ] **Step 1: Write the failing test**

Add a test in `test/plugin-surface-contract.test.ts` that reads `plugin/skills/remember/SKILL.md` and `plugin/skills/remember/EXAMPLES.md`, then asserts:
- the workflow tells agents to sanitize/redact sensitive values before `memory_save`;
- the skill names common secret classes such as API keys, tokens, passwords, private keys, session cookies, and connection strings;
- the checklist preserves meaning, not raw secrets;
- examples include a security-sensitive note and do not save the raw example key in `memory_save` content.

- [ ] **Step 2: Verify red**

Run:

```bash
corepack pnpm test -- test/plugin-surface-contract.test.ts -t "remember skill redacts secrets"
```

Expected: the new test fails because the current skill has no redaction workflow.

- [ ] **Step 3: Update skill docs**

Update `plugin/skills/remember/SKILL.md` so the workflow sanitizes credentials and secrets before content construction, anti-patterns reject raw secret persistence, and the checklist says to preserve meaning while redacting secret values.

Update `plugin/skills/remember/EXAMPLES.md` with a worked security-sensitive example that redacts a sample API key and points users to the secret manager instead of storing the raw value.

- [ ] **Step 4: Verify green**

Run:

```bash
corepack pnpm test -- test/plugin-surface-contract.test.ts -t "remember skill redacts secrets"
corepack pnpm test -- test/plugin-surface-contract.test.ts
corepack pnpm exec tsx scripts/skills/check.ts
git diff --check
```

Expected: all commands exit 0.

- [ ] **Step 5: Review and prep**

Run the focused cleanup/review chain for the task-owned docs and test diff, then run required security checks for the agent skill/prompt documentation surface before staging or committing.

## Self-Review

- Spec coverage: the plan covers upstream PR 941 summary items: workflow sanitization, secret-redaction wording, and worked example.
- Placeholder scan: no placeholders remain.
- Type consistency: no new runtime types or APIs are introduced.
68 changes: 68 additions & 0 deletions docs/todos/2026-06-17-issue-919-remember-redact-secrets/todo.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Issue 919 Remember Secret Redaction

Scope: `wbugitlab1/agentmemory` issue #919, tracking upstream PR 941.

## Sprint Contract

Goal: Make the packaged `remember` skill instruct agents to preserve useful meaning while redacting credentials and secrets before saving long-term memory.

Scope:
- `plugin/skills/remember/SKILL.md`
- `plugin/skills/remember/EXAMPLES.md`
- focused plugin-surface regression coverage

Non-goals:
- No runtime memory redaction changes.
- No MCP/REST API changes.
- No dependency, schema, auth, or persistence changes.
- No fetch, pull, push, PR creation, or upstream-targeted work without separate approval.

Acceptance criteria:
- `remember` workflow sanitizes secrets before `memory_save`.
- Checklist and anti-patterns reject raw API keys, tokens, passwords, private keys, cookies, and connection strings.
- Worked examples show security-sensitive notes without persisting raw secret values.
- A focused test fails before the docs change and passes after it.

Intended verification:
- `corepack pnpm test -- test/plugin-surface-contract.test.ts`
- `corepack pnpm exec tsx scripts/skills/check.ts`
- `git diff --check`
- Security gates as required for the prompt/tooling docs surface.

Known boundaries:
- This touches agent skill/prompt documentation, so Semgrep and staged Gitleaks are required before commit readiness.
- `origin/main` freshness is unverified unless the user separately approves fetch.

Stop conditions:
- Required verification cannot run after deterministic setup attempts.
- A fix would require runtime behavior, API, auth, persistence, dependency, or remote-state changes.
- Remote writes are needed.

## Feature / Verification Matrix

| Change | Verification method | Status | Evidence |
| --- | --- | --- | --- |
| Remember skill redacts secrets before save | Targeted plugin-surface test + skill lint | Complete | Red/green `corepack pnpm test -- test/plugin-surface-contract.test.ts -t "remember skill redacts secrets"`; focused full shard `corepack pnpm test -- test/plugin-surface-contract.test.ts` passed 171 files / 2238 tests; `corepack pnpm exec tsx scripts/skills/check.ts` passed 15 skills |
| Worked security-sensitive example | Targeted plugin-surface test + read-through | Complete | `plugin/skills/remember/EXAMPLES.md` uses `[RAW_TOKEN_REDACTED_IN_EXAMPLE]` and `[REDACTED_GITHUB_TOKEN]`; security reviewer found no realistic raw-secret examples |
| Branch prep/security gate | Git diff checks, Semgrep, staged Gitleaks before commit | Complete | `git diff --check` and `git diff --cached --check` passed; `semgrep scan --config p/default --error --metrics=off .` passed with 0 findings; `gitleaks protect --staged --redact` found no leaks |

## Subagent Ledger

| Workstream | Scope | Edits allowed | Expected output | Result | Residual risk |
| --- | --- | --- | --- | --- | --- |
| Validity explorer | `plugin/skills/remember`, related docs/tests | No | Validity decision with file evidence | Valid: current checkout lacked upstream PR 941 remember-skill redaction; Explorer inspected skill/docs/runtime paths and local refs | Live remote was separately checked read-only by main agent |
| Security reviewer | `remember` skill docs/test diff | No | Prompt/security risk review | ACCEPT: redaction guidance is clear and examples use placeholders only | None noted |
| Test/scope reviewer | `remember` skill docs/test/task record diff | No | Coverage and scope-fit review | Minor stale matrix finding fixed in this update; otherwise scope/test coverage accepted | Main agent reruns final checks |

## Progress

- 2026-06-17: Preflight: worktree `/Users/A1538552/.codex/worktrees/4780/agentmemory`, original state detached at `3cee91d1`, clean status.
- 2026-06-17: Public read-only GitHub API confirmed `wbugitlab1#919` is open and tracks upstream PR 941; PR 941 changes `plugin/skills/remember/SKILL.md` and `plugin/skills/remember/EXAMPLES.md`.
- 2026-06-17: Local evidence confirmed `plugin/skills/remember/SKILL.md` still said to preserve user phrasing without a secret-redaction step.
- 2026-06-17: Created local branch `github-pr/issue-919-remember-redact-secrets-3cee91d1` because this worktree started detached and another worktree already owns a different issue-919 branch.
- 2026-06-17: Red test verified after deterministic dependency setup: `corepack pnpm test -- test/plugin-surface-contract.test.ts -t "remember skill redacts secrets"` failed at the new assertion because `SKILL.md` lacked sanitize/redact wording.
- 2026-06-17: Green checks after implementation: targeted redaction test passed; full `test/plugin-surface-contract.test.ts` shard passed 171 files / 2238 tests; `corepack pnpm exec tsx scripts/skills/check.ts` passed 15 skills; `git diff --check` passed.
- 2026-06-17: Semgrep security gate passed: `semgrep scan --config p/default --error --metrics=off .` scanned 711 tracked files and reported 0 findings.
- 2026-06-17: Reviewers accepted security and scope/test coverage after the stale matrix finding was addressed.
- 2026-06-17: Final pre-stage checks passed after task-record update: `corepack pnpm test -- test/plugin-surface-contract.test.ts` passed 171 files / 2238 tests; `corepack pnpm exec tsx scripts/skills/check.ts` passed 15 skills; `git diff --check` passed.
- 2026-06-17: Staged checks passed: `git diff --cached --check` and `gitleaks protect --staged --redact` reported no issues.
19 changes: 19 additions & 0 deletions plugin/skills/remember/EXAMPLES.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,22 @@ memory_save {
```

Present the confirmation with the concepts echoed back.

## 4. Security-sensitive operational note

User: "Remember that the GitHub automation token [RAW_TOKEN_REDACTED_IN_EXAMPLE] is only for the release bot, lives in the Actions secret RELEASE_BOT_TOKEN, and must be rotated before the next release."

Invocation:

```json
memory_save {
"content": "The GitHub automation token [REDACTED_GITHUB_TOKEN] is only for the release bot, lives in the Actions secret RELEASE_BOT_TOKEN, and must be rotated before the next release.",
"concepts": "release-bot-token, actions-secret, credential-rotation",
"files": ""
}
```

Present:

> Saved the security-sensitive operational note without storing the raw token.
> Concepts: `release-bot-token`, `actions-secret`, `credential-rotation`.
23 changes: 17 additions & 6 deletions plugin/skills/remember/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,30 +26,41 @@ Saved memory abc12345 with 3 concepts: jwt-refresh-rotation, token-revocation, a
## Why

A memory is only as useful as the terms that retrieve it. Tag with specific
concepts so a future `recall` finds it, and preserve the user's own phrasing.
concepts so a future `recall` finds it, and preserve the user's meaning without
persisting raw credentials or secrets.

## Workflow

1. Pull the core insight, decision, or fact out of `$ARGUMENTS`.
2. Extract 2-5 lowercased concept phrases. Prefer specific over generic
2. Sanitize sensitive values before constructing `content`. Redact API keys,
tokens, passwords, private keys, session cookies, connection strings, and
other secrets. Preserve the useful meaning, not the secret itself.
3. Extract 2-5 lowercased concept phrases. Prefer specific over generic
(`jwt-refresh-rotation` beats `auth`).
3. Extract referenced file paths (absolute or repo-relative). Empty if none.
4. Call `memory_save` with `content`, `concepts` (comma-separated string), and
4. Extract referenced file paths (absolute or repo-relative). Empty if none.
5. Call `memory_save` with sanitized `content`, `concepts` (comma-separated string), and
`files` (comma-separated string).
5. Confirm the save and echo the concepts so the user knows the retrieval terms.
6. Confirm the save and echo the concepts so the user knows the retrieval terms.
Do not echo secret values.

## Anti-patterns

WRONG: `concepts: "stuff, code, notes"` (generic tags nothing can find later).

RIGHT: `concepts: "jwt-refresh-rotation, token-revocation"` (specific, retrievable).

WRONG: `content: "Production API key is [RAW_API_KEY]"` (persists a secret).

RIGHT: `content: "Production API uses a bearer token; the token value was redacted and must be retrieved from the secret manager."`

## Checklist

- Content preserves the user's phrasing, not a paraphrase.
- Content preserves the user's meaning, but redacts API keys, tokens, passwords,
private keys, session cookies, connection strings, and other secrets.
- Concepts are specific, lowercased, 2-5 items.
- File paths are real references, not guesses.
- Confirmation echoes the exact concepts tagged.
- Confirmation does not echo secret values.

## See also

Expand Down
34 changes: 34 additions & 0 deletions test/plugin-surface-contract.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -200,3 +200,37 @@ describe("Generated skill references", () => {
}
});
});

describe("Packaged skill safety contracts", () => {
it("remember skill redacts secrets before saving memory", () => {
const skill = readFileSync(
join(pluginRoot, "skills/remember/SKILL.md"),
"utf8",
);
const examples = readFileSync(
join(pluginRoot, "skills/remember/EXAMPLES.md"),
"utf8",
);

expect(skill).toMatch(/sanitize|redact|replace raw/i);
expect(skill).toContain("memory_save");
for (const term of [
"API keys",
"tokens",
"passwords",
"private keys",
"session cookies",
"connection strings",
]) {
expect(skill).toContain(term);
}
expect(skill).toMatch(/preserves? the user's meaning/i);
expect(skill).not.toContain("preserve the user's own phrasing");
expect(skill).toMatch(/Do not echo secret values/i);

expect(examples).toMatch(/security-sensitive/i);
expect(examples).toContain("[REDACTED_GITHUB_TOKEN]");
expect(examples).not.toContain("sk-live-example");
expect(examples).toMatch(/without storing the raw (key|token|secret)/i);
});
});