feat(cli): materialize user iii config

README.md

@@ -1510,6 +1510,8 @@ npx @agentmemory/agentmemory --data-dir ~/.agentmemory-work/main
 AGENTMEMORY_DATA_DIR=~/.agentmemory-work/main npx @agentmemory/agentmemory
 ```
 
+On first init or startup, agentmemory writes the native iii-engine config to `~/.agentmemory/config/iii-config.yaml` with absolute paths to the resolved data directory. `AGENTMEMORY_III_CONFIG`, a project-local `iii-config.yaml`, and an existing legacy `~/.agentmemory/iii-config.yaml` remain supported for custom setups.
+
 agentmemory does not currently encrypt these iii-engine state files itself. For enterprise deployments that require
 encryption at rest, place `AGENTMEMORY_DATA_DIR` on an encrypted volume or platform-managed encrypted storage. Adding
 application-level encryption, tenant key isolation, or a new encrypted backend would change the storage/data-format

docs/todos/2026-06-18-issue-503-iii-config-materialize/plan.md

@@ -0,0 +1,59 @@
+# iii-config Materialization Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Materialize `~/.agentmemory/config/iii-config.yaml` with absolute data paths so native iii-engine runs independent of caller cwd and avoids watching noisy state-file siblings.
+
+**Architecture:** Add a pure helper module for user config materialization and keep `src/cli.ts` as the production wiring layer. Discovery remains env > cwd > new user config > legacy user config > bundled; materialization only happens after env/cwd/legacy checks cannot satisfy the request, so existing user overrides are preserved.
+
+**Tech Stack:** TypeScript ESM, Node fs/path/os APIs, Vitest, existing CLI runtime config helpers.
+
+---
+
+## Files
+
+- Create: `src/cli/iii-config.ts`
+- Create: `test/cli-iii-config.test.ts`
+- Modify: `src/cli.ts`
+- Modify: `src/cli/build-runtime.ts`
+- Modify: `test/build-runtime.test.ts`
+- Modify: `test/cli-server-log.test.ts`
+- Modify: `README.md` if user-facing config docs need one compact note
+- Modify: `docs/todos/2026-06-18-issue-503-iii-config-materialize/todo.md`
+
+## Task 1: Discovery Precedence
+
+- [ ] Update `findIiiConfigPath` in `src/cli/build-runtime.ts` to consider `join(homeDir, ".agentmemory", "config", "iii-config.yaml")` before `join(homeDir, ".agentmemory", "iii-config.yaml")`.
+- [ ] Extend `test/build-runtime.test.ts` so the documented precedence is env, cwd, new user config, legacy user config, package root, dist.
+- [ ] Run `corepack pnpm exec vitest run test/build-runtime.test.ts --exclude test/integration.test.ts`.
+
+## Task 2: Materialization Helper
+
+- [ ] Create `src/cli/iii-config.ts` with `materializeUserIiiConfig(options)` accepting explicit `targetDir`, `dataDir`, `findBundled`, `onWarn`, and fs hooks for tests.
+- [ ] Rewrite only the two known bundled markers when both are present; otherwise copy verbatim and warn.
+- [ ] Write atomically with tmp file, fsync best-effort, and rename; preserve existing target content.
+- [ ] Add `test/cli-iii-config.test.ts` cases for first write, idempotency, unrecognized shape, missing bundled, write failure, and parallel callers.
+- [ ] Run `corepack pnpm exec vitest run test/cli-iii-config.test.ts --exclude test/integration.test.ts`.
+
+## Task 3: CLI Wiring
+
+- [ ] Import the helper in `src/cli.ts`.
+- [ ] Add a production wrapper that targets `join(homedir(), ".agentmemory", "config")` and uses `dataDirResolution.dataDir`.
+- [ ] Update `findIiiConfig()` so env/cwd wins, existing new user config wins, existing legacy user config wins, and only then bundled config is materialized to the new user path when possible.
+- [ ] Call the wrapper from `runInit()` after `.env` handling so explicit init also prepares the config.
+- [ ] Keep `prepareRuntimeIiiConfig` as the runtime rendering guard for custom config and port/env rewrites.
+- [ ] Update `test/cli-server-log.test.ts` source assertions to cover the materialization call.
+
+## Task 4: Docs And Review
+
+- [ ] Add a compact README note near configuration explaining that native iii config is generated under `~/.agentmemory/config/` with absolute data paths.
+- [ ] Run targeted tests for build-runtime, cli-iii-config, runtime-config, engine-launch, and cli-server-log.
+- [ ] Review diff for unnecessary complexity, stale references, and real-home writes.
+- [ ] Update the Feature / Verification Matrix with evidence.
+
+## Task 5: PR Prep
+
+- [ ] Run broader repo-native tests if feasible.
+- [ ] Run required security/secret gates or record blockers.
+- [ ] Commit only task-owned files.
+- [ ] Push branch to `origin`, create PR against `origin/main`, monitor CI, merge after green or explicitly accepted blockers.

docs/todos/2026-06-18-issue-503-iii-config-materialize/todo.md

@@ -0,0 +1,92 @@
+# Issue 503 iii-config Materialization
+
+## Scope
+
+- Repo: `/Users/A1538552/.codex/worktrees/9677/agentmemory`
+- Branch: `issue/503-iii-config-materialize`
+- Issue: GitHub issue #503, tracking upstream PR 699
+- Base: `origin/main` at `a029b7e117db99c65201ee3d9abb6bb93ff2e173`
+
+## Sprint Contract
+
+Goal: materialize a user iii-engine config at `~/.agentmemory/config/iii-config.yaml` with absolute data paths, without mutating the real user home during tests.
+
+Scope:
+- Add a small CLI helper for first-start user config materialization.
+- Wire it into `agentmemory init` and engine startup/config discovery.
+- Preserve existing precedence for env and cwd config, and preserve legacy `~/.agentmemory/iii-config.yaml` when present.
+- Add isolated temp-home tests for path generation, idempotency, fallback, and concurrency.
+
+Non-goals:
+- Do not change MCP/REST tool surfaces.
+- Do not change schema, auth, remote services, or dependency versions.
+- Do not remove the existing runtime config rendering path unless the new helper makes it redundant for bundled/user configs.
+
+Acceptance criteria:
+- `findIiiConfigPath` can prefer `~/.agentmemory/config/iii-config.yaml` before legacy top-level user config.
+- First startup from bundled config can materialize the new user config with absolute `state_store.db` and `stream_store` paths.
+- Existing env, cwd, and legacy user configs remain respected.
+- Tests use temp directories only and do not write outside the test fixture.
+- Targeted tests and required gates are run or blockers recorded.
+
+Intended verification:
+- `corepack pnpm exec vitest run test/build-runtime.test.ts test/cli-iii-config.test.ts test/runtime-config.test.ts test/engine-launch.test.ts test/cli-server-log.test.ts --exclude test/integration.test.ts`
+- `corepack pnpm test` if practical after targeted checks.
+- Security gates required by repo policy before commit/PR: gitleaks staged scan, plus Semgrep because CLI config/persistence behavior changes.
+
+Known boundaries:
+- This changes local filesystem config behavior but not external APIs.
+- Remote writes are authorized by the delegated request for this issue workflow only against `origin`, never upstream.
+- No production/user-home mutation outside tests.
+
+Stop conditions:
+- A required behavior would shadow a user-supplied legacy config.
+- Required checks fail twice without a new failure-mode hypothesis.
+- Dependency installation or credentialed remote operations require approval not covered by the delegated request.
+
+## Feature / Verification Matrix
+
+| Change | Verification | Status | Evidence |
+| --- | --- | --- | --- |
+| Materialize new user config with absolute paths | New focused tests | Passed | `corepack pnpm exec vitest run test/build-runtime.test.ts test/cli-iii-config.test.ts test/cli-server-log.test.ts test/runtime-config.test.ts test/engine-launch.test.ts --exclude test/integration.test.ts`: 47 tests passed |
+| Preserve config precedence and legacy fallback | Updated build-runtime tests | Passed | `test/build-runtime.test.ts` covers new config dir before legacy and legacy before bundled fallback |
+| Wire init/start behavior | Source contract test plus helper tests | Passed | `test/cli-server-log.test.ts` covers `findIiiConfig()`, `runInit()`, and legacy short-circuit |
+| No real home mutation | Temp-home tests only | Passed | `test/cli-iii-config.test.ts` uses `mkdtempSync(tmpdir())`; no production home writes |
+
+## Subagent Ledger
+
+| Workstream | Scope | Edits | Expected output | Result | Residual risk |
+| --- | --- | --- | --- | --- | --- |
+| Explorer/evaluator | Read-only inspection of CLI/config/tests | No | Validate whether issue is still real and recommend minimal surface | Completed: issue valid but narrow; absolute data-path rendering existed, config-dir materialization and init wiring were missing | Main agent verified source and tests directly |
+
+## Progress
+
+- 2026-06-18: Read AGENTS instructions and `github-feature-loop` routing skill.
+- 2026-06-18: Confirmed repo is `agentmemory`, origin is `wbugitlab1/agentmemory`, upstream is `rohitg00/agentmemory`.
+- 2026-06-18: Fetched `origin/main`; current base is `a029b7e117db99c65201ee3d9abb6bb93ff2e173`.
+- 2026-06-18: Read issue #503 and upstream PR 699 metadata; upstream PR remains open and scoped.
+- 2026-06-18: Created branch `issue/503-iii-config-materialize`.
+- 2026-06-18: Added `src/cli/iii-config.ts`, CLI wiring, discovery precedence update, README note, and focused tests.
+- 2026-06-18: Verification passed:
+  - `corepack pnpm exec vitest run test/build-runtime.test.ts test/cli-iii-config.test.ts test/cli-server-log.test.ts test/runtime-config.test.ts test/engine-launch.test.ts --exclude test/integration.test.ts` — 47 tests passed.
+  - `corepack pnpm run lint` — passed.
+  - `corepack pnpm test` — 199 files, 2777 tests passed.
+  - `corepack pnpm run build` — passed.
+  - `semgrep scan --config p/default --error --metrics=off .` — 0 findings.
+  - `gitleaks protect --staged --redact` — no leaks found.
+- 2026-06-18: Created commit `0f5419cb feat(cli): materialize user iii config`, pushed branch, and opened PR #1005.
+- 2026-06-18: PR CI passed on `test (ubuntu-latest, 22)` and `test (macos-latest, 22)`.
+- 2026-06-18: GitHub required branch update before merge; merged `origin/main` into the task branch.
+- 2026-06-18: Post-base-merge verification passed:
+  - `corepack pnpm run lint` — passed.
+  - `corepack pnpm test` — 200 files, 2778 tests passed.
+  - `corepack pnpm run build` — passed.
+  - `semgrep scan --config p/default --error --metrics=off .` — 0 findings.
+  - `gitleaks detect --source . --redact --log-opts=origin/main..HEAD` — no leaks found for the PR range.
+  - `gitleaks detect --source . --redact` reported 14 full-history findings in `.pnpm-store/...` from old local snapshot commit `6849579677ce25544b480f1bd4fd9fd3b4df6032`; that commit is not an ancestor of `origin/main` and is outside the PR range.
+
+## Review Notes
+
+- The implementation preserves `AGENTMEMORY_III_CONFIG`, project-local configs, and existing legacy `~/.agentmemory/iii-config.yaml`.
+- `agentmemory init` short-circuits materialization when a legacy config exists, avoiding accidental shadowing.
+- `pnpm exec` initially attempted dependency setup and hit ignored-build hardening; per repo instructions, `corepack pnpm install --frozen-lockfile --ignore-scripts` was run, then tests proceeded.

src/cli.ts

@@ -84,6 +84,7 @@ import {
   readyTimeoutMsFromEnv,
 } from "./cli/readiness-timeout.js";
 import { renderPreparedRuntimeIiiConfig } from "./cli/runtime-config.js";
+import { materializeUserIiiConfig } from "./cli/iii-config.js";
 
 const ALL_TOOLS_COUNT = getAllTools().length;
 const CORE_TOOLS_COUNT = getAllTools().filter((t) => ESSENTIAL_TOOLS.has(t.name)).length;
@@ -335,15 +336,41 @@ async function isAgentmemoryReady(): Promise<boolean> {
 
 function findIiiConfig(): string {
   // Precedence (user-overridable wins): explicit env > project cwd >
-  // ~/.agentmemory/ > bundled. The bundled config used to win
-  // unconditionally, so users hitting the observability log-feedback
-  // loop (#519) had no way to drop a tamer config in place without
-  // editing node_modules. Prefer the package-root bundle before the dist
-  // copy because dist is cleaned during local builds; iii treats deleting
+  // ~/.agentmemory/config/ > legacy ~/.agentmemory/ > bundled. The bundled
+  // config used to win unconditionally, so users hitting the observability
+  // log-feedback loop (#519) had no way to drop a tamer config in place
+  // without editing node_modules. Prefer the package-root bundle before the
+  // dist copy because dist is cleaned during local builds; iii treats deleting
   // the active config file as a fatal reload error.
+  const configured = findIiiConfigPath({ moduleDir: __dirname, includeBundled: false });
+  if (configured) return configured;
+
+  const materialized = materializeDefaultUserIiiConfig();
+  if (materialized) return materialized.path;
+
   return findIiiConfigPath({ moduleDir: __dirname });
 }
 
+function findBundledIiiConfig(): string | null {
+  const candidates = [
+    join(__dirname, "..", "iii-config.yaml"),
+    join(__dirname, "iii-config.yaml"),
+  ];
+  return candidates.find((candidate) => existsSync(candidate)) ?? null;
+}
+
+function materializeDefaultUserIiiConfig() {
+  const legacyConfig = join(homedir(), ".agentmemory", "iii-config.yaml");
+  if (existsSync(legacyConfig)) return { path: legacyConfig, created: false };
+
+  return materializeUserIiiConfig({
+    targetDir: join(homedir(), ".agentmemory", "config"),
+    dataDir: dataDirResolution.dataDir,
+    findBundled: findBundledIiiConfig,
+    onWarn: vlog,
+  });
+}
+
 function prepareRuntimeIiiConfig(configPath: string): string {
   if (!configPath) return configPath;
   assertRuntimeHostAllowed();
@@ -2294,19 +2321,27 @@ async function runInit() {
     await copyFile(template, target, fsConstants.COPYFILE_EXCL);
   } catch (err) {
     if ((err as NodeJS.ErrnoException)?.code === "EEXIST") {
+      const configResult = materializeDefaultUserIiiConfig();
       p.log.warn(`${target} already exists — leaving it untouched.`);
       p.log.info(
         `Compare against the latest template: diff ${target} ${template}`,
       );
-      p.outro("Nothing changed.");
+      if (configResult?.created) {
+        p.log.success(`Wrote ${configResult.path}`);
+        p.outro("Initialized iii config.");
+      } else {
+        p.outro("Nothing changed.");
+      }
       return;
     }
     p.log.error(
       `Failed to copy template: ${err instanceof Error ? err.message : String(err)}`,
     );
     process.exit(1);
   }
+  const configResult = materializeDefaultUserIiiConfig();
   p.log.success(`Wrote ${target}`);
+  if (configResult?.created) p.log.success(`Wrote ${configResult.path}`);
   p.note(
     [
       "All keys are commented out by default. Uncomment the ones you want.",

src/cli/build-runtime.ts

@@ -46,21 +46,25 @@ export function findIiiConfigPath({
   homeDir = homedir(),
   moduleDir,
   packageRootDir = join(moduleDir, ".."),
+  includeBundled = true,
   exists = existsSync,
 }: {
   envPath?: string;
   cwd?: string;
   homeDir?: string;
   moduleDir: string;
   packageRootDir?: string;
+  includeBundled?: boolean;
   exists?: (path: string) => boolean;
 }): string {
   const candidates = [
     ...(envPath ? [envPath] : []),
     join(cwd, "iii-config.yaml"),
+    join(homeDir, ".agentmemory", "config", "iii-config.yaml"),
     join(homeDir, ".agentmemory", "iii-config.yaml"),
-    join(packageRootDir, "iii-config.yaml"),
-    join(moduleDir, "iii-config.yaml"),
+    ...(includeBundled
+      ? [join(packageRootDir, "iii-config.yaml"), join(moduleDir, "iii-config.yaml")]
+      : []),
   ];
   for (const candidate of candidates) {
     if (exists(candidate)) return candidate;

src/cli/iii-config.ts

@@ -0,0 +1,119 @@
+import {
+  closeSync,
+  existsSync,
+  fsyncSync,
+  linkSync,
+  mkdirSync,
+  openSync,
+  readFileSync,
+  unlinkSync,
+  writeSync,
+} from "node:fs";
+import { join } from "node:path";
+
+export type MaterializedUserIiiConfig = {
+  path: string;
+  created: boolean;
+};
+
+type MaterializeUserIiiConfigOptions = {
+  targetDir: string;
+  dataDir: string;
+  findBundled: () => string | null;
+  onWarn?: (message: string) => void;
+  exists?: (path: string) => boolean;
+  readFile?: (path: string) => string;
+  mkdir?: (path: string) => void;
+  writeTempFile?: (path: string, content: string) => void;
+  publishTempFile?: (tmpPath: string, targetPath: string) => boolean;
+  cleanupTempFile?: (path: string) => void;
+};
+
+const STATE_MARKER = "file_path: ./data/state_store.db";
+const STREAM_MARKER = "file_path: ./data/stream_store";
+
+function yamlSingleQuote(value: string): string {
+  return `'${value.replace(/'/g, "''")}'`;
+}
+
+function renderUserIiiConfig(raw: string, dataDir: string, onWarn?: (message: string) => void): string {
+  if (!raw.includes(STATE_MARKER) || !raw.includes(STREAM_MARKER)) {
+    onWarn?.("Bundled iii-config.yaml did not contain expected bundled data paths; copying verbatim.");
+    return raw;
+  }
+
+  return raw
+    .replace(STATE_MARKER, `file_path: ${yamlSingleQuote(join(dataDir, "state_store.db"))}`)
+    .replace(STREAM_MARKER, `file_path: ${yamlSingleQuote(join(dataDir, "stream_store"))}`);
+}
+
+function defaultWriteTempFile(path: string, content: string): void {
+  const fd = openSync(path, "wx", 0o600);
+  try {
+    writeSync(fd, content);
+    try {
+      fsyncSync(fd);
+    } catch {
+      // Atomic visibility matters more than durability on filesystems
+      // that do not support fsync for this temp file.
+    }
+  } finally {
+    closeSync(fd);
+  }
+}
+
+function defaultPublishTempFile(tmpPath: string, targetPath: string): boolean {
+  try {
+    linkSync(tmpPath, targetPath);
+    return true;
+  } catch (err) {
+    if ((err as NodeJS.ErrnoException)?.code === "EEXIST") return false;
+    throw err;
+  }
+}
+
+export function materializeUserIiiConfig({
+  targetDir,
+  dataDir,
+  findBundled,
+  onWarn,
+  exists = existsSync,
+  readFile = (path) => readFileSync(path, "utf-8"),
+  mkdir = (path) => mkdirSync(path, { recursive: true, mode: 0o700 }),
+  writeTempFile = defaultWriteTempFile,
+  publishTempFile = defaultPublishTempFile,
+  cleanupTempFile = (path) => unlinkSync(path),
+}: MaterializeUserIiiConfigOptions): MaterializedUserIiiConfig | null {
+  const targetPath = join(targetDir, "iii-config.yaml");
+  if (exists(targetPath)) return { path: targetPath, created: false };
+
+  const bundled = findBundled();
+  if (!bundled) {
+    onWarn?.("Could not locate bundled iii-config.yaml; falling back to legacy config discovery.");
+    return null;
+  }
+
+  const tmpPath = join(
+    targetDir,
+    `.iii-config.yaml.${process.pid}.${Date.now()}.${Math.random().toString(16).slice(2)}.tmp`,
+  );
+
+  try {
+    const rendered = renderUserIiiConfig(readFile(bundled), dataDir, onWarn);
+    mkdir(targetDir);
+    writeTempFile(tmpPath, rendered);
+    const created = publishTempFile(tmpPath, targetPath);
+    return { path: targetPath, created };
+  } catch (err) {
+    onWarn?.(
+      `Failed to materialize iii config: ${err instanceof Error ? err.message : String(err)}`,
+    );
+    return null;
+  } finally {
+    try {
+      cleanupTempFile(tmpPath);
+    } catch {
+      // Best-effort cleanup for races where another caller already won.
+    }
+  }
+}