diff --git a/docs/sleep/README.md b/docs/sleep/README.md index 2e36eb16..71df638e 100644 --- a/docs/sleep/README.md +++ b/docs/sleep/README.md @@ -54,7 +54,8 @@ skillopt-sleep schedule # install a nightly cron entry for this project The per-agent integrations below still come from the repo; the CLI above is the standalone, pip-only way to run a cycle. Claude Code, Codex, Copilot, and Devin wrap -the shared engine. OpenClaw is a separate reference adaptation and has its own setup. +the shared engine; Hermes Agent is wired into the core engine directly (`--source hermes` +/ `--backend hermes`). OpenClaw is a separate reference adaptation and has its own setup. One engine, thin per-agent shells (see [`plugins/`](https://github.com/microsoft/SkillOpt/tree/main/plugins)): @@ -64,6 +65,7 @@ One engine, thin per-agent shells (see [`plugins/`](https://github.com/microsoft | **Codex** | [`plugins/codex`](https://github.com/microsoft/SkillOpt/tree/main/plugins/codex) | `bash plugins/codex/install.sh` → `skillopt-sleep` skill | | **Copilot** | [`plugins/copilot`](https://github.com/microsoft/SkillOpt/tree/main/plugins/copilot) | register `plugins/copilot/mcp_server.py` as an MCP server | | **Devin** | [`plugins/devin`](https://github.com/microsoft/SkillOpt/tree/main/plugins/devin) | register `plugins/devin/mcp_server.py` as an MCP server | +| **Hermes Agent** | [`plugins/hermes`](https://github.com/microsoft/SkillOpt/tree/main/plugins/hermes) | `--source hermes` / `--backend hermes` (core), or register `plugins/hermes/mcp_server.py` | | **OpenClaw** | [`plugins/openclaw`](https://github.com/microsoft/SkillOpt/tree/main/plugins/openclaw) | adapt the reference wrapper and paths for your installation | To use DeepSeek, vLLM, Ollama, or another Chat Completions server, see diff --git a/plugins/README.md b/plugins/README.md index 51230220..734ccfec 100644 --- a/plugins/README.md +++ b/plugins/README.md @@ -10,8 +10,10 @@ runtime dependency on the paper's `skillopt/` experiment package. ## Available integrations -Four integrations wrap the shared `skillopt_sleep` CLI. OpenClaw is a separate -reference adaptation with its own backend and setup assumptions. +Five integrations wrap the shared `skillopt_sleep` CLI. Hermes is additionally +wired into the core engine (its harvester and backend ship in `skillopt_sleep/`, +so `--source hermes` / `--backend hermes` work from the shared CLI). OpenClaw is +a separate reference adaptation with its own backend and setup assumptions. | Platform | Folder | Mechanism | Status | |---|---|---|---| @@ -19,6 +21,7 @@ reference adaptation with its own backend and setup assumptions. | **Codex** | [`codex/`](codex) | user-level skill and shared runner | installable shared-engine integration | | **GitHub Copilot** | [`copilot/`](copilot) | MCP server exposing seven `sleep_*` tools | shared-engine MCP integration | | **Devin** | [`devin/`](devin) | MCP server plus Devin transcript conversion | shared-engine MCP integration | +| **Hermes Agent** | [`hermes/`](hermes) | core `--source`/`--backend hermes` + MCP server | first-class core integration | | **OpenClaw** | [`openclaw/`](openclaw) | custom DeepSeek/Ollama wrapper | independent reference adaptation; review and adapt before use | ## Install @@ -32,6 +35,7 @@ for your workflow. | **Codex** | `bash plugins/codex/install.sh` | ask Codex to use the `skillopt-sleep` skill | | **Copilot** | register `plugins/copilot/mcp_server.py` using its example MCP config | ask Copilot to run `sleep_status` | | **Devin** | register `plugins/devin/mcp_server.py` using its example MCP config | ask Devin to run `sleep_status` | +| **Hermes Agent** | register `plugins/hermes/mcp_server.py`, or run `python -m skillopt_sleep run --backend hermes --source hermes` | ask Hermes to run `sleep_status` | | **OpenClaw** | follow and adapt [`openclaw/README.md`](openclaw/README.md) | validate paths, credentials, and tasks locally | Python 3.10 or newer is required. Real CLI backends also require the selected @@ -101,9 +105,9 @@ Common implemented flags include: | Flag | Default | Purpose | |---|---|---| -| `--backend mock\|claude\|codex\|copilot\|handoff\|azure_openai` | `mock` | select who performs model calls | +| `--backend mock\|claude\|codex\|copilot\|handoff\|azure_openai\|hermes` | `mock` | select who performs model calls | | `--model NAME` | backend default | select a backend-specific model | -| `--source claude\|codex\|auto` | `claude` | select the transcript source | +| `--source claude\|codex\|hermes\|auto` | `claude` | select the transcript source | | `--project PATH` | current directory | select the project and invoked harvest scope | | `--scope invoked\|all` | `invoked` | limit transcript harvesting | | `--target-skill-path PATH` | managed skill | select a specific `SKILL.md` to stage/adopt | diff --git a/plugins/hermes/README.md b/plugins/hermes/README.md new file mode 100644 index 00000000..8c4ee169 --- /dev/null +++ b/plugins/hermes/README.md @@ -0,0 +1,85 @@ +# SkillOpt-Sleep — Hermes Agent integration + +First-class integration of the [Hermes Agent](https://github.com/NousResearch/Hermes) +CLI with the shared `skillopt_sleep` engine. Unlike the reference OpenClaw adapter, +Hermes is wired directly into core: `--source hermes` and `--backend hermes` work from +the shared CLI, alongside `claude` and `codex`. + +## What you get + +- **Harvester** (`skillopt_sleep/harvest_hermes.py`) — shells out to the stable + `hermes sessions export --format jsonl --redact` interface (schema-drift-proof, + WAL-safe, self-redacting), normalizes sessions into `SessionDigest` records, + layers its own sanitization, and skips the engine's own throwaway sessions. + Warns loudly (never silently) when nothing is harvestable. +- **Backend** (`HermesBackend` in `skillopt_sleep/backend.py`) — drives + `hermes --profile chat -Q -q ""` for the replay/reflect phases. +- **MCP server** (`mcp_server.py`) — exposes the cycle as MCP tools with Hermes + defaults, so any MCP client can run it. + +## Requirements + +- Python 3.10+ +- The `hermes` CLI installed and authenticated, with at least one profile. +- Past Hermes **CLI** sessions with message content (`hermes sessions list` should + show non-empty sessions; ACP/editor-bridge sessions carry no harvestable turns). + Override the home with `HERMES_HOME`. + +## Quick start + +```bash +# 1. Preview — no API spend, no writes: +python -m skillopt_sleep dry-run --backend mock --source hermes --scope all + +# 2. Real cycle through the Hermes CLI, stages a proposal (nothing live changes): +SKILLOPT_SLEEP_HERMES_PROFILE=default \ + python -m skillopt_sleep run --backend hermes --source hermes + +# 3. Review, then adopt: +python -m skillopt_sleep status +python -m skillopt_sleep adopt +``` + +## MCP setup + +Copy `mcp-config.example.json` into your client's MCP config (adjust the path), or run +the server directly: `python plugins/hermes/mcp_server.py`. Cycle actions default to +`backend=hermes` / `source=hermes`; pass `backend=mock` for a dry, free run. + +## Environment + +| Variable | Purpose | Default | +|---|---|---| +| `HERMES_HOME` | Directory holding `state.db` | `~/.hermes` | +| `HERMES_BIN` | Path to the hermes binary | `hermes` | +| `SKILLOPT_SLEEP_HERMES_PROFILE` | Hermes profile for cycle calls | `default` | +| `SKILLOPT_SLEEP_HERMES_MODEL` | Optional model hint | (unset) | + +## What gets evolved (and why) + +The Hermes integration evolves **the skill only**, written to where Hermes actually +discovers global skills: + +``` +~/.hermes/skills/skillopt-sleep-learned/SKILL.md +``` + +The MCP server sets `--target-skill-path` there by default. Run it manually with: + +```bash +python -m skillopt_sleep run --backend hermes --source hermes --no-evolve-memory \ + --target-skill-path ~/.hermes/skills/skillopt-sleep-learned/SKILL.md +``` + +**Memory is left to Hermes.** The shared engine's memory evolution targets a project +`CLAUDE.md`, which Hermes does not read — Hermes manages its own memory via `SOUL.md` +/ `MEMORY.md` and its `curator`/`learning` subsystems. So the Hermes path defaults to +`--no-evolve-memory` (skill-only) rather than staging a doc Hermes would ignore. Proper +`AGENTS.md` memory support depends on a future engine `memory_filename` option and is +intentionally out of scope here. + +## Safety + +Sessions are harvested read-only and secrets are redacted before anything is persisted. +The cycle stages proposals; `adopt` is required to change any live file and always backs +it up first. Harvested text is untrusted — keep tools/plugins disabled when replaying it. diff --git a/plugins/hermes/SKILL.md b/plugins/hermes/SKILL.md new file mode 100644 index 00000000..b93eb081 --- /dev/null +++ b/plugins/hermes/SKILL.md @@ -0,0 +1,64 @@ +--- +name: skillopt-sleep-hermes +description: "Use when the user wants their Hermes Agent to self-improve from past usage, asks about a nightly/offline 'sleep' cycle for Hermes, wants Hermes to review past sessions, learn preferences, or consolidate memory/skills, or to run dry-run/run/adopt/status for SkillOpt-Sleep against Hermes. Drives the skillopt_sleep engine with --source hermes and --backend hermes: harvest past Hermes sessions via `hermes sessions export` -> mine recurring tasks -> replay through the Hermes CLI -> stage validated skill/memory edits behind a held-out gate." +--- + +# SkillOpt-Sleep: usage-driven self-evolution for a Hermes Agent + +SkillOpt-Sleep gives a Hermes Agent a sleep cycle. On demand or on a nightly +schedule it reviews past Hermes sessions, re-runs recurring tasks through the +Hermes CLI, and proposes bounded edits to a configured skill and memory doc. +With the validation gate enabled it keeps only changes that improve a held-out +score. Live files change only through an explicit `adopt`, which backs up first. + +Hermes is first-class here: the harvester and backend live in the shared engine +(`skillopt_sleep/harvest_hermes.py`, `HermesBackend`), so the standard CLI works. +The harvester uses `hermes sessions export` rather than reading the state DB, so +it survives Hermes schema changes and reuses Hermes's own secret redaction. + +## Actions + +- `status` — nights run so far + the latest staged proposal. +- `dry-run` — harvest + mine + replay, report only (no staging). Safe, no writes. +- `run` — full cycle; stages a reviewed proposal. +- `adopt` — apply the latest staged proposal (backup taken first). +- `harvest` — debug: list recurring tasks mined from recent sessions. +- `schedule` / `unschedule` — manage a nightly cron entry. + +## Usage + +```bash +# free preview +python -m skillopt_sleep dry-run --backend mock --source hermes --scope all + +# real cycle through the Hermes CLI (stages only) +SKILLOPT_SLEEP_HERMES_PROFILE=default \ + python -m skillopt_sleep run --backend hermes --source hermes + +python -m skillopt_sleep status +python -m skillopt_sleep adopt +``` + +Or wire the MCP server (`mcp_server.py`) into your client; cycle tools default to +`backend=hermes` / `source=hermes`. + +## What it evolves + +Skill-only, written where Hermes discovers global skills: +`~/.hermes/skills/skillopt-sleep-learned/SKILL.md` (via `--target-skill-path`). +The Hermes path defaults to `--no-evolve-memory`: the engine's memory doc is a +project `CLAUDE.md`, which Hermes does not read (it manages memory itself via +`SOUL.md`/`MEMORY.md` and its curator). Proper `AGENTS.md` support awaits a future +engine `memory_filename` option. + +## Configuration + +`HERMES_HOME` (state DB dir, default `~/.hermes`), `HERMES_BIN` (default `hermes`), +`SKILLOPT_SLEEP_HERMES_PROFILE` (profile for cycle calls, default `default`), +`SKILLOPT_SLEEP_HERMES_MODEL` (optional model hint). + +## Safety + +Sessions are read-only; secrets are redacted before persistence; the engine's own +throwaway sessions are skipped. Harvested text is untrusted — keep tools/plugins +disabled while replaying it. Nothing live changes without `adopt`. diff --git a/plugins/hermes/hermes-rules.snippet.md b/plugins/hermes/hermes-rules.snippet.md new file mode 100644 index 00000000..2e3ec7b3 --- /dev/null +++ b/plugins/hermes/hermes-rules.snippet.md @@ -0,0 +1,17 @@ + + +## SkillOpt-Sleep (self-evolution) + +This project is wired for SkillOpt-Sleep via the `skillopt-sleep-hermes` MCP server. +Overnight, Sleep reads recent Hermes sessions, mines recurring tasks, replays them, +and proposes bounded edits to the target skill/memory doc. Edits are validated behind +a held-out gate and **staged** — nothing live changes until explicitly adopted. + +- Ask for `sleep_status` to see nights run and any staged proposal. +- Ask for `sleep_dry_run` to preview a cycle without staging (safe, no writes). +- Ask for `sleep_run` to run a full cycle and stage a proposal. +- Ask for `sleep_adopt` to apply the latest staged proposal (a backup is made first). + +Backend and transcript source default to `hermes`. Set `SKILLOPT_SLEEP_HERMES_PROFILE` +to pick the Hermes profile used for optimizer/target calls, and `HERMES_HOME` if your +state DB lives outside `~/.hermes`. diff --git a/plugins/hermes/mcp-config.example.json b/plugins/hermes/mcp-config.example.json new file mode 100644 index 00000000..e896444e --- /dev/null +++ b/plugins/hermes/mcp-config.example.json @@ -0,0 +1,13 @@ +{ + "mcpServers": { + "skillopt-sleep-hermes": { + "command": "python3", + "args": ["plugins/hermes/mcp_server.py"], + "env": { + "SKILLOPT_SLEEP_REPO": "${workspaceFolder}", + "HERMES_HOME": "~/.hermes", + "SKILLOPT_SLEEP_HERMES_PROFILE": "default" + } + } + } +} diff --git a/plugins/hermes/mcp_server.py b/plugins/hermes/mcp_server.py new file mode 100644 index 00000000..3c9e3b89 --- /dev/null +++ b/plugins/hermes/mcp_server.py @@ -0,0 +1,195 @@ +#!/usr/bin/env python3 +"""SkillOpt-Sleep for Hermes Agent — minimal MCP server (stdio, stdlib-only). + +Exposes the sleep engine as MCP tools, wired to harvest Hermes Agent sessions +(``~/.hermes/state.db``) and to run cycles through the Hermes CLI backend. No +third-party deps: speaks JSON-RPC 2.0 over stdio. + +Tools mirror the shared engine (status / dry-run / run / adopt / harvest / +schedule / unschedule). Each shells out to `python -m skillopt_sleep ` +with backend and source defaulting to `hermes`. Configure your client to launch: + python plugins/hermes/mcp_server.py +""" +from __future__ import annotations + +import json +import os +import subprocess +import sys + +REPO_ROOT = os.environ.get("SKILLOPT_SLEEP_REPO") or os.path.abspath( + os.path.join(os.path.dirname(__file__), "..", "..") +) +PROTOCOL_VERSION = "2024-11-05" + +TOOLS = [ + {"name": "sleep_status", "action": "status", + "description": "Show how many SkillOpt-Sleep nights have run and the latest staged proposal."}, + {"name": "sleep_dry_run", "action": "dry-run", + "description": "Preview a sleep cycle (harvest+mine+replay) without staging anything."}, + {"name": "sleep_run", "action": "run", + "description": "Run a full sleep cycle; stages a reviewed proposal. Nothing live changes until adopt."}, + {"name": "sleep_adopt", "action": "adopt", + "description": "Apply the latest staged proposal to the target skill/memory doc (backs up first)."}, + {"name": "sleep_harvest", "action": "harvest", + "description": "Debug: list the recurring tasks mined from recent Hermes sessions."}, + {"name": "sleep_schedule", "action": "schedule", + "description": "Install a nightly cron entry to run the sleep cycle automatically."}, + {"name": "sleep_unschedule", "action": "unschedule", + "description": "Remove the nightly cron entry for a project."}, +] +_BY_NAME = {t["name"]: t for t in TOOLS} + +_TOOL_SCHEMA = { + "type": "object", + "properties": { + "project": {"type": "string", + "description": "Project dir to evolve (default: cwd)."}, + "backend": {"type": "string", "enum": ["mock", "claude", "codex", "copilot", "hermes"], + "description": "mock = no API spend; hermes = drive the Hermes CLI (default)."}, + "scope": {"type": "string", "enum": ["invoked", "all"], + "description": "Harvest scope (default: invoked project only)."}, + "source": {"type": "string", "enum": ["claude", "codex", "hermes", "auto"], + "description": "Transcript source (default: hermes)."}, + "model": {"type": "string", + "description": "Backend-specific model override."}, + "tasks_file": {"type": "string", + "description": "Path to reviewed TaskRecord JSON (skips harvest)."}, + "target_skill_path": {"type": "string", + "description": "Explicit SKILL.md path to evolve/stage/adopt."}, + "progress": {"type": "boolean", + "description": "Print phase progress to stderr."}, + "max_sessions": {"type": "integer", + "description": "Cap harvested sessions per run."}, + "max_tasks": {"type": "integer", + "description": "Cap mined tasks per run."}, + "lookback_hours": {"type": "integer", + "description": "Harvest window in hours (default: 72)."}, + "auto_adopt": {"type": "boolean", + "description": "Auto-adopt if gate passes (default: false)."}, + "json": {"type": "boolean", + "description": "Return machine-readable JSON output."}, + "edit_budget": {"type": "integer", + "description": "Max bounded edits per night (default: 4)."}, + "hour": {"type": "integer", + "description": "Hour for schedule (0-23, default: 3)."}, + "minute": {"type": "integer", + "description": "Minute for schedule (0-59, default: 17)."}, + }, + "additionalProperties": False, +} + +# Actions that harvest/replay and therefore want the Hermes backend + source by default. +_HERMES_ACTIONS = {"run", "dry-run", "harvest"} + + +def _default_skill_path() -> str: + """Where Hermes discovers global skills — evolve the learned skill there.""" + home = os.environ.get("HERMES_HOME") or os.path.expanduser("~/.hermes") + return os.path.join(home, "skills", "skillopt-sleep-learned", "SKILL.md") + + +def _run_engine(action: str, args: dict) -> str: + py = sys.executable or "python3" + cmd = [py, "-m", "skillopt_sleep", action] + # Default to the Hermes backend/source for cycle actions unless overridden, + # target the Hermes global skills dir, and evolve skill-only (Hermes manages + # its own memory via SOUL.md/MEMORY.md; the engine's CLAUDE.md is ignored). + if action in _HERMES_ACTIONS: + args = { + "backend": "hermes", + "source": "hermes", + "target_skill_path": _default_skill_path(), + "no_evolve_memory": True, + **args, + } + # String-valued flags + for flag, key in [ + ("--project", "project"), ("--backend", "backend"), + ("--scope", "scope"), ("--source", "source"), + ("--model", "model"), ("--tasks-file", "tasks_file"), + ("--target-skill-path", "target_skill_path"), + ]: + val = args.get(key) + if val: + cmd += [flag, str(val)] + # Integer-valued flags + for flag, key in [ + ("--max-sessions", "max_sessions"), ("--max-tasks", "max_tasks"), + ("--lookback-hours", "lookback_hours"), ("--edit-budget", "edit_budget"), + ("--hour", "hour"), ("--minute", "minute"), + ]: + val = args.get(key) + if val is not None: + cmd += [flag, str(int(val))] + # Boolean flags + for flag, key in [ + ("--progress", "progress"), ("--auto-adopt", "auto_adopt"), + ("--json", "json"), ("--no-evolve-memory", "no_evolve_memory"), + ]: + if args.get(key): + cmd.append(flag) + try: + proc = subprocess.run(cmd, cwd=REPO_ROOT, capture_output=True, text=True, timeout=3600) + except Exception as e: + return f"[error] failed to run engine: {e}" + out = (proc.stdout or "").strip() + err = (proc.stderr or "").strip() + return out + (("\n[stderr]\n" + err) if err else "") + + +def _result(id_, result): + return {"jsonrpc": "2.0", "id": id_, "result": result} + + +def _error(id_, code, message): + return {"jsonrpc": "2.0", "id": id_, "error": {"code": code, "message": message}} + + +def handle(req: dict): + method = req.get("method") + id_ = req.get("id") + if method == "initialize": + return _result(id_, { + "protocolVersion": PROTOCOL_VERSION, + "capabilities": {"tools": {}}, + "serverInfo": {"name": "skillopt-sleep-hermes", "version": "0.1.0"}, + }) + if method in ("notifications/initialized", "initialized"): + return None # notification, no response + if method == "tools/list": + return _result(id_, {"tools": [ + {"name": t["name"], "description": t["description"], "inputSchema": _TOOL_SCHEMA} + for t in TOOLS + ]}) + if method == "tools/call": + params = req.get("params") or {} + name = params.get("name") + tool = _BY_NAME.get(name) + if not tool: + return _error(id_, -32602, f"unknown tool: {name}") + text = _run_engine(tool["action"], params.get("arguments") or {}) + return _result(id_, {"content": [{"type": "text", "text": text}]}) + if method == "ping": + return _result(id_, {}) + return _error(id_, -32601, f"method not found: {method}") + + +def main() -> int: + for line in sys.stdin: + line = line.strip() + if not line: + continue + try: + req = json.loads(line) + except Exception: + continue + resp = handle(req) + if resp is not None: + sys.stdout.write(json.dumps(resp) + "\n") + sys.stdout.flush() + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/skillopt_sleep/__main__.py b/skillopt_sleep/__main__.py index 5a761b66..f4cc78c3 100644 --- a/skillopt_sleep/__main__.py +++ b/skillopt_sleep/__main__.py @@ -13,8 +13,8 @@ --max-tasks N cap mined tasks per run --target-skill-path PATH explicit live SKILL.md to stage/adopt --tasks-file PATH reviewed TaskRecord JSON file to replay instead of harvesting - --backend mock|claude|codex|copilot|handoff - --source claude|codex|auto + --backend mock|claude|codex|copilot|handoff|hermes + --source claude|codex|hermes|auto --model NAME --lookback-hours N --auto-adopt @@ -71,12 +71,12 @@ def _add_common(p: argparse.ArgumentParser) -> None: p.add_argument("--scope", default="", choices=["", "all", "invoked"]) p.add_argument("--backend", default="", choices=["", "mock", "claude", "codex", "copilot", "handoff", - "azure_openai"]) + "azure_openai", "hermes"]) p.add_argument("--model", default="") p.add_argument("--codex-path", default="", help="path to the real @openai/codex binary") p.add_argument("--claude-home", default="", help="override ~/.claude (also isolates state)") p.add_argument("--codex-home", default="", help="override ~/.codex for archived session harvest") - p.add_argument("--source", default="", choices=["", "claude", "codex", "auto"], + p.add_argument("--source", default="", choices=["", "claude", "codex", "hermes", "auto"], help="session transcript source") p.add_argument("--lookback-hours", type=int, default=None, help="harvest window in hours; 0 = scan full history") @@ -87,6 +87,8 @@ def _add_common(p: argparse.ArgumentParser) -> None: help="cap mined tasks for this run") p.add_argument("--target-skill-path", default="", help="explicit live SKILL.md path to evolve/stage/adopt") + p.add_argument("--no-evolve-memory", action="store_true", + help="evolve the skill only; skip the memory doc (CLAUDE.md)") p.add_argument("--tasks-file", default="", help="reviewed TaskRecord JSON file to replay instead of harvesting") p.add_argument("--progress", action="store_true", @@ -135,6 +137,8 @@ def _cfg_from_args(args, task_meta: Dict[str, Any] | None = None) -> Any: if args.project and not os.path.isabs(path): path = os.path.join(os.path.abspath(args.project), path) overrides["target_skill_path"] = os.path.abspath(path) + if getattr(args, "no_evolve_memory", False): + overrides["evolve_memory"] = False if getattr(args, "progress", False): overrides["progress"] = True if getattr(args, "auto_adopt", False): diff --git a/skillopt_sleep/backend.py b/skillopt_sleep/backend.py index cd130a48..2c1ba509 100644 --- a/skillopt_sleep/backend.py +++ b/skillopt_sleep/backend.py @@ -1558,6 +1558,88 @@ def _call(self, prompt: str, *, max_tokens: int = 1024, retries: int = 5) -> str return "" +class HermesBackend(CliBackend): + """Drives the Hermes Agent CLI: ``hermes --profile chat -Q -q ""``. + + Env overrides: + HERMES_BIN path to the hermes binary (default "hermes") + SKILLOPT_SLEEP_HERMES_PROFILE profile name (falls back to HERMES_TARGET_PROFILE) + SKILLOPT_SLEEP_HERMES_MODEL model hint passed through to CliBackend + """ + + name = "hermes" + + def __init__(self, model: str = "", timeout: int = 180) -> None: + super().__init__( + model=model or os.environ.get("SKILLOPT_SLEEP_HERMES_MODEL", ""), + timeout=timeout, + ) + self.hermes_bin = os.environ.get("HERMES_BIN", "hermes") + self.hermes_profile = os.environ.get( + "SKILLOPT_SLEEP_HERMES_PROFILE", + os.environ.get("HERMES_TARGET_PROFILE", "default"), + ) + + def _call(self, prompt: str, *, max_tokens: int = 1024) -> str: + cmd = [self.hermes_bin, "--profile", self.hermes_profile, "chat", "-Q", "-q", prompt] + # Run in a throwaway cwd so the agent can't pick up a project's files; + # the harvester filters these sessions out via the tempdir prefix. + with tempfile.TemporaryDirectory(prefix="skillopt_sleep_hermes_") as clean_cwd: + try: + proc = subprocess.run( + cmd, + capture_output=True, + text=True, + timeout=self.timeout, + cwd=clean_cwd, + env={**os.environ, "HERMES_NO_COLOR": "1"}, + ) + except Exception as exc: # noqa: BLE001 + self.last_call_error = f"Hermes CLI call failed: {exc}" + return "" + if proc.returncode != 0: + stderr = (proc.stderr or "").strip() + self.last_call_error = stderr[:500] if stderr else f"Hermes CLI exited with code {proc.returncode}" + return "" + result = _strip_hermes_boilerplate((proc.stdout or "").strip()) + self._tokens += len(prompt) // 4 + len(result) // 4 + return result + + +def _strip_hermes_boilerplate(raw: str) -> str: + """Drop CLI notices/warnings/tracebacks the Hermes binary prints around a reply. + + Conservative: only a bare ``Traceback (most recent call last):`` header opens + traceback-skip mode, so a legitimate answer that merely starts with the word + "Exception" is preserved. + """ + skip_prefixes = ( + "Bitwarden Secrets Manager:", + "Warning: Unknown", + "session_id:", + "Exception ignored in:", + ) + body: list[str] = [] + in_traceback = False + for line in raw.split("\n"): + stripped = line.strip() + if not stripped: + continue + if any(stripped.startswith(p) for p in skip_prefixes): + continue + if stripped == "Traceback (most recent call last):": + in_traceback = True + continue + if in_traceback: + if stripped.startswith('File "') and ", line " in stripped: + continue + if re.match(r"^\w+(Error|Exception|Warning):", stripped): + continue + in_traceback = False + body.append(line) + return "\n".join(body).strip() + + def get_backend( name: str, *, @@ -1586,6 +1668,8 @@ def get_backend( project_dir or os.getcwd(), ".skillopt-sleep-handoff" ) return HandoffBackend(model=model, handoff_dir=hdir) + if n in {"hermes", "hermes_chat", "hermes_cli"}: + return HermesBackend(model=model) return MockBackend() diff --git a/skillopt_sleep/config.py b/skillopt_sleep/config.py index 4a1c992f..682a40e9 100644 --- a/skillopt_sleep/config.py +++ b/skillopt_sleep/config.py @@ -19,13 +19,15 @@ HOME_STATE_DIR = os.path.expanduser("~/.skillopt-sleep") CLAUDE_HOME = os.path.expanduser("~/.claude") CODEX_HOME = os.path.expanduser("~/.codex") +HERMES_HOME = os.environ.get("HERMES_HOME") or os.path.expanduser("~/.hermes") DEFAULTS: Dict[str, Any] = { # ── scope ────────────────────────────────────────────────────────────── "claude_home": CLAUDE_HOME, "codex_home": CODEX_HOME, - "transcript_source": "claude", # "claude" | "codex" | "auto" + "hermes_home": HERMES_HOME, + "transcript_source": "claude", # "claude" | "codex" | "hermes" | "auto" "projects": "invoked", # "invoked" | "all" | [list of abs paths] "invoked_project": "", # filled at runtime (cwd) when projects == "invoked" "lookback_hours": 72, # harvest window when no prior sleep recorded diff --git a/skillopt_sleep/harvest_hermes.py b/skillopt_sleep/harvest_hermes.py new file mode 100644 index 00000000..e7dd7322 --- /dev/null +++ b/skillopt_sleep/harvest_hermes.py @@ -0,0 +1,252 @@ +"""SkillOpt-Sleep Hermes Agent session harvesting. + +Harvests Hermes Agent sessions into ``SessionDigest`` records via the stable, +purpose-built ``hermes sessions export --format jsonl`` interface rather than +reading ``~/.hermes/state.db`` directly. That command is schema-drift-proof, +handles WAL concurrency safely, and applies Hermes's own secret redaction +(``--redact``); we layer our own sanitization on top as defense in depth. + +Each exported JSONL line is one session dict (id, source, cwd, title, +started_at, ended_at, ... plus a ``messages`` array). Tool arguments and raw +tool outputs are not copied; only tool *names* are kept. + +If the ``hermes`` binary is missing, the export fails, or nothing matches, this +warns to stderr and returns ``[]`` — a no-op, never a silent success. +""" + +from __future__ import annotations + +import json +import os +import subprocess +import sys +import tempfile +from datetime import datetime, timezone +from typing import Any, Dict, List, Optional + +from skillopt_sleep.harvest import _detect_feedback, _is_meta_prompt, _project_matches +from skillopt_sleep.staging import _SECRET_PATTERNS +from skillopt_sleep.types import SessionDigest + +HERMES_HOME = os.environ.get("HERMES_HOME") or os.path.expanduser("~/.hermes") + + +def _warn(msg: str) -> None: + print(f"[sleep] hermes harvest: {msg}", file=sys.stderr) + + +def _sanitize_text(text: str) -> str: + """Redact secrets; return "" for meta prompts (slash commands, pastes, etc.).""" + sanitized = (text or "").strip() + if not sanitized or _is_meta_prompt(sanitized): + return "" + for pattern, replacement in _SECRET_PATTERNS: + sanitized = pattern.sub(replacement, sanitized) + return sanitized + + +def _dedup(xs: List[str]) -> List[str]: + seen: set[str] = set() + out: List[str] = [] + for x in xs: + if x not in seen: + seen.add(x) + out.append(x) + return out + + +def _message_text(content: Any) -> str: + """Normalize a message ``content`` field (string | parts list | dict).""" + if content is None: + return "" + if isinstance(content, str): + return content + if isinstance(content, list): + parts = [] + for part in content: + if isinstance(part, str): + parts.append(part) + elif isinstance(part, dict): + val = part.get("text") or part.get("content") + if isinstance(val, str): + parts.append(val) + return "\n".join(p for p in parts if p) + if isinstance(content, dict): + val = content.get("text") or content.get("content") + return val if isinstance(val, str) else "" + return str(content) + + +def _run_export( + *, + hermes_home: str, + since_iso: Optional[str], + cwd_filter: str, + min_messages: int, +) -> Optional[str]: + """Run ``hermes sessions export`` and return raw JSONL, or None on failure. + + Isolated so tests can patch it without a real ``hermes`` binary. + """ + hermes_bin = os.environ.get("HERMES_BIN", "hermes") + with tempfile.NamedTemporaryFile("r", suffix=".jsonl", delete=True) as tf: + cmd = [hermes_bin, "sessions", "export", tf.name, + "--format", "jsonl", "--redact", + "--min-messages", str(max(1, min_messages))] + if since_iso: + cmd += ["--after", since_iso] + if cwd_filter: + cmd += ["--cwd", cwd_filter] + try: + proc = subprocess.run( + cmd, capture_output=True, text=True, timeout=120, + env={**os.environ, "HERMES_HOME": hermes_home, "HERMES_NO_COLOR": "1"}, + ) + except FileNotFoundError: + _warn(f"'{hermes_bin}' not found on PATH; is Hermes Agent installed?") + return None + except Exception as exc: # noqa: BLE001 + _warn(f"export failed: {exc}") + return None + if proc.returncode != 0: + _warn((proc.stderr or "").strip()[:300] or f"export exited {proc.returncode}") + return None + try: + with open(tf.name, "r", encoding="utf-8") as fh: + return fh.read() + except OSError as exc: + _warn(f"could not read export output: {exc}") + return None + + +def _ts(value: Any) -> str: + if value is None: + return "" + if isinstance(value, str): + return value + try: + return datetime.fromtimestamp(float(value), tz=timezone.utc).isoformat() + except (TypeError, ValueError, OSError): + return "" + + +def _build_digest( + session: Dict[str, Any], + *, + scope: str, + invoked_project: str, +) -> Optional[SessionDigest]: + """Build a ``SessionDigest`` from one exported session dict, or None.""" + session_id = str(session.get("id") or session.get("session_id") or "") + project = (session.get("cwd") or "").strip() + # Never harvest the optimizer's own calls (HermesBackend runs in these tempdirs). + if "skillopt_sleep_hermes_" in project: + return None + + user_prompts: List[str] = [] + assistant_finals: List[str] = [] + tools: List[str] = [] + feedback_signals: List[str] = [] + n_user = 0 + n_asst = 0 + last_assistant = "" + + for msg in session.get("messages") or []: + if not isinstance(msg, dict): + continue + role = (msg.get("role") or "").strip() + content = _message_text(msg.get("content")).strip() + tool = (msg.get("tool_name") or "").strip() + + if role == "user" and content: + sanitized = _sanitize_text(content) + if sanitized: + n_user += 1 + user_prompts.append(sanitized) + feedback_signals.extend(_detect_feedback(sanitized)) + if last_assistant: + assistant_finals.append(last_assistant) + last_assistant = "" + elif role == "assistant" and content: + n_asst += 1 + last_assistant = _sanitize_text(content) or "" + # tool rows contribute only their name; args/outputs are never copied. + + if tool: + tools.append(tool) + + if last_assistant: + assistant_finals.append(last_assistant) + + if n_user == 0 and n_asst == 0: + return None + if not _project_matches(project, scope, invoked_project): + return None + + return SessionDigest( + session_id=session_id, + project=project, + started_at=_ts(session.get("started_at")), + ended_at=_ts(session.get("ended_at")), + user_prompts=user_prompts, + assistant_finals=assistant_finals[-5:], + tools_used=_dedup(tools), + files_touched=[], + feedback_signals=feedback_signals, + n_user_turns=n_user, + n_assistant_turns=n_asst, + raw_path=f"hermes:{session_id}", + ) + + +def harvest_hermes( + *, + scope: str = "invoked", + invoked_project: str = "", + since_iso: Optional[str] = None, + limit: int = 0, + hermes_home: str = "", +) -> List[SessionDigest]: + """Harvest Hermes sessions via ``hermes sessions export``. ``limit=0`` = no limit. + + ``scope="invoked"`` restricts to sessions whose cwd is under ``invoked_project`` + (Hermes CLI/coding sessions); ``scope="all"`` harvests every session with content. + """ + home = hermes_home or HERMES_HOME + cwd_filter = invoked_project if (scope == "invoked" and invoked_project) else "" + + raw = _run_export( + hermes_home=home, + since_iso=since_iso, + cwd_filter=cwd_filter, + min_messages=1, + ) + if raw is None: + return [] + + digests: List[SessionDigest] = [] + for line in raw.splitlines(): + line = line.strip() + if not line: + continue + try: + session = json.loads(line) + except json.JSONDecodeError: + continue + if not isinstance(session, dict): + continue + digest = _build_digest(session, scope=scope, invoked_project=invoked_project) + if digest is not None: + digests.append(digest) + + if not digests: + _warn( + f"no harvestable sessions (scope={scope}" + + (f", cwd={cwd_filter}" if cwd_filter else "") + + "). Run some `hermes chat` sessions, or try --scope all." + ) + return [] + + # Most recent first, then apply the cap. + digests.sort(key=lambda d: d.ended_at or "", reverse=True) + return digests[:limit] if limit and limit > 0 else digests diff --git a/skillopt_sleep/harvest_sources.py b/skillopt_sleep/harvest_sources.py index 501aa285..95380495 100644 --- a/skillopt_sleep/harvest_sources.py +++ b/skillopt_sleep/harvest_sources.py @@ -3,8 +3,11 @@ from typing import Optional +import os + from skillopt_sleep.harvest import harvest from skillopt_sleep.harvest_codex import harvest_codex +from skillopt_sleep.harvest_hermes import harvest_hermes from skillopt_sleep.types import SessionDigest @@ -13,6 +16,14 @@ def harvest_for_config(cfg, *, since_iso: Optional[str] = None, limit: int = 0) scope = cfg.get("projects", "invoked") invoked_project = cfg.get("invoked_project", "") + if source == "hermes": + return harvest_hermes( + scope=scope, + invoked_project=invoked_project, + since_iso=since_iso, + limit=limit, + hermes_home=cfg.get("hermes_home", os.path.expanduser("~/.hermes")), + ) if source == "codex": return harvest_codex( cfg.codex_archived_sessions_dir, diff --git a/tests/test_sleep_engine.py b/tests/test_sleep_engine.py index 36cb88b9..a9209acf 100644 --- a/tests/test_sleep_engine.py +++ b/tests/test_sleep_engine.py @@ -8,6 +8,7 @@ import json import os +import sqlite3 import tempfile import unittest from unittest import mock @@ -1485,5 +1486,197 @@ def _fake_once(prompt, *, max_tokens=1024): self.assertIn("REDACTED", joined) +class TestHermesHarvest(unittest.TestCase): + """Hermes harvesting via the `hermes sessions export` shim (mocked).""" + + def _sessions_jsonl(self, sessions) -> str: + return "\n".join(json.dumps(s) for s in sessions) + "\n" + + def _patch_export(self, jsonl): + return mock.patch("skillopt_sleep.harvest_hermes._run_export", return_value=jsonl) + + def test_returns_empty_when_export_fails(self): + from skillopt_sleep.harvest_hermes import harvest_hermes + + with self._patch_export(None): + self.assertEqual(harvest_hermes(scope="all"), []) + + def test_returns_empty_when_no_sessions(self): + from skillopt_sleep.harvest_hermes import harvest_hermes + + with self._patch_export(""): + self.assertEqual(harvest_hermes(scope="all"), []) + + def test_builds_digest_from_export(self): + from skillopt_sleep.harvest_hermes import harvest_hermes + + jsonl = self._sessions_jsonl([{ + "id": "s1", "source": "cli", "cwd": "/proj", + "started_at": 1000.0, "ended_at": 2000.0, + "messages": [ + {"role": "user", "content": "summarize the changelog"}, + {"role": "assistant", "content": "done"}, + ], + }]) + with self._patch_export(jsonl): + digests = harvest_hermes(scope="all") + self.assertEqual([d.session_id for d in digests], ["s1"]) + self.assertEqual(digests[0].n_user_turns, 1) + self.assertEqual(digests[0].n_assistant_turns, 1) + + def test_normalizes_structured_content(self): + from skillopt_sleep.harvest_hermes import harvest_hermes + + jsonl = self._sessions_jsonl([{ + "id": "s1", "cwd": "/p", "ended_at": 2000.0, + "messages": [ + {"role": "user", "content": [{"type": "text", "text": "hello there"}]}, + {"role": "assistant", "content": "hi"}, + ], + }]) + with self._patch_export(jsonl): + digests = harvest_hermes(scope="all") + self.assertIn("hello there", " ".join(digests[0].user_prompts)) + + def test_includes_tool_names_only(self): + from skillopt_sleep.harvest_hermes import harvest_hermes + + jsonl = self._sessions_jsonl([{ + "id": "s1", "cwd": "/p", "ended_at": 2000.0, + "messages": [ + {"role": "user", "content": "search the web"}, + {"role": "assistant", "content": "using search"}, + {"role": "tool", "content": '{"result": "found 42"}', "tool_name": "search"}, + {"role": "assistant", "content": "the answer is 42"}, + ], + }]) + with self._patch_export(jsonl): + digests = harvest_hermes(scope="all") + self.assertIn("search", digests[0].tools_used) + self.assertEqual(digests[0].n_user_turns, 1) + self.assertEqual(digests[0].n_assistant_turns, 2) + + def test_detects_feedback(self): + from skillopt_sleep.harvest_hermes import harvest_hermes + + jsonl = self._sessions_jsonl([{ + "id": "s1", "cwd": "/p", "ended_at": 2000.0, + "messages": [ + {"role": "user", "content": "fix the parser"}, + {"role": "assistant", "content": "fixed it"}, + {"role": "user", "content": "still broken, please fix properly"}, + {"role": "assistant", "content": "ok now really fixed"}, + {"role": "user", "content": "perfect, thanks"}, + {"role": "assistant", "content": "welcome"}, + ], + }]) + with self._patch_export(jsonl): + sigs = harvest_hermes(scope="all")[0].feedback_signals + self.assertTrue(any(s.startswith("neg:") for s in sigs), sigs) + self.assertTrue(any(s.startswith("pos:") for s in sigs), sigs) + + def test_redacts_secrets_defense_in_depth(self): + from skillopt_sleep.harvest_hermes import harvest_hermes + + jsonl = self._sessions_jsonl([{ + "id": "s1", "cwd": "/p", "ended_at": 2000.0, + "messages": [ + {"role": "user", "content": "use key sk-abc123def456ghi789jkl"}, + {"role": "assistant", "content": "ok"}, + ], + }]) + with self._patch_export(jsonl): + joined = " ".join(harvest_hermes(scope="all")[0].user_prompts) + self.assertIn("[REDACTED_OPENAI_KEY]", joined) + self.assertNotIn("sk-abc123def456ghi789jkl", joined) + + def test_skips_engine_sessions(self): + from skillopt_sleep.harvest_hermes import harvest_hermes + + jsonl = self._sessions_jsonl([{ + "id": "e1", "cwd": "/tmp/skillopt_sleep_hermes_x", "ended_at": 2000.0, + "messages": [{"role": "user", "content": "internal"}, {"role": "assistant", "content": "x"}], + }]) + with self._patch_export(jsonl): + self.assertEqual(harvest_hermes(scope="all"), []) + + def test_scope_invoked_filters_by_project(self): + from skillopt_sleep.harvest_hermes import harvest_hermes + + jsonl = self._sessions_jsonl([ + {"id": "in", "cwd": "/proj", "ended_at": 2001.0, + "messages": [{"role": "user", "content": "a"}, {"role": "assistant", "content": "b"}]}, + {"id": "out", "cwd": "/elsewhere", "ended_at": 2002.0, + "messages": [{"role": "user", "content": "c"}, {"role": "assistant", "content": "d"}]}, + ]) + with self._patch_export(jsonl): + digests = harvest_hermes(scope="invoked", invoked_project="/proj") + self.assertEqual([d.session_id for d in digests], ["in"]) + + def test_limit_and_ordering(self): + from skillopt_sleep.harvest_hermes import harvest_hermes + + jsonl = self._sessions_jsonl([ + {"id": f"s{i}", "cwd": f"/p/{i}", "ended_at": 2000.0 + i, + "messages": [{"role": "user", "content": f"t{i}"}, {"role": "assistant", "content": "d"}]} + for i in range(5) + ]) + with self._patch_export(jsonl): + digests = harvest_hermes(scope="all", limit=2) + # newest ended_at first + self.assertEqual([d.session_id for d in digests], ["s4", "s3"]) + + def test_harvest_sources_forwards_hermes(self): + from skillopt_sleep.config import load_config + from skillopt_sleep.harvest_sources import harvest_for_config + + jsonl = self._sessions_jsonl([{ + "id": "s1", "cwd": "/project", "ended_at": 2000.0, + "messages": [{"role": "user", "content": "hello"}, {"role": "assistant", "content": "world"}], + }]) + cfg = load_config(transcript_source="hermes", projects="all") + with self._patch_export(jsonl): + digests = harvest_for_config(cfg, limit=10) + self.assertEqual([d.session_id for d in digests], ["s1"]) + + +class TestHermesBackend(unittest.TestCase): + """HermesBackend dispatch and CLI output cleanup.""" + + def test_get_backend_dispatch(self): + from skillopt_sleep.backend import HermesBackend, get_backend + + for alias in ("hermes", "hermes_chat", "hermes_cli"): + self.assertIsInstance(get_backend(alias), HermesBackend) + + def test_strip_boilerplate_keeps_answer(self): + from skillopt_sleep.backend import _strip_hermes_boilerplate as strip + + self.assertEqual(strip("Warning: Unknown flag\nHello world"), "Hello world") + self.assertEqual( + strip('Traceback (most recent call last):\n File "x.py", line 1\nValueError: boom\nreal answer'), + "real answer", + ) + # A bare "Exception" in prose must survive. + self.assertEqual(strip("Exception handling is the topic"), "Exception handling is the topic") + + def test_no_evolve_memory_flag_sets_config(self): + import argparse + from skillopt_sleep.__main__ import _add_common + + p = argparse.ArgumentParser() + _add_common(p) + self.assertTrue(p.parse_args(["--no-evolve-memory"]).no_evolve_memory) + self.assertFalse(p.parse_args([]).no_evolve_memory) + + def test_call_captures_error_on_missing_binary(self): + from skillopt_sleep.backend import HermesBackend + + b = HermesBackend() + b.hermes_bin = "hermes-does-not-exist-xyz" + self.assertEqual(b._call("hi"), "") + self.assertTrue(b.last_call_error) + + if __name__ == "__main__": unittest.main(verbosity=2)