Skip to content

test(e2e): full command-surface coverage + live AI-tool hook integration#87

Closed
jeff-r2026 wants to merge 2 commits into
mainfrom
feat/e2e-cli-coverage
Closed

test(e2e): full command-surface coverage + live AI-tool hook integration#87
jeff-r2026 wants to merge 2 commits into
mainfrom
feat/e2e-cli-coverage

Conversation

@jeff-r2026

Copy link
Copy Markdown
Collaborator

Summary

补全项目端到端测试的两个真实缺口:(1) 没有覆盖全部 CLI 命令面,(2) 从未真正运行 claude / codebuddy / cursor-agent 来验证 teamai 注入的 hook 是否真的触发。

新增两个 e2e 套件:

  • command-coverage-e2e.test.ts(无需凭证 / 网络,CI 可跑)

    • src/index.ts每一个用户可见命令 + 子命令 spawn 真实 CLI,断言 --help 退出码 0 且打印 usage(可捕获 dynamic import 损坏、option 规格错误、action handler 缺失等接线问题)。
    • 在隔离的空 HOME(无 teamai 配置)下运行只读 / 本地命令,断言它们优雅降级、不出现 broken-import / undefined-reference 这类接线 bug(普通的 "not initialized" 守卫抛错属预期行为,不算崩溃)。
    • 直接喂真实 hook payload 给 hook-dispatch,断言 dashboard 事件(session_start / prompt_submit)被正确写入——无需 LLM 即可端到端验证 hook 调度骨架。
  • tool-integration-e2e.test.ts(gated:TEAMAI_E2E_LIVE_TOOLS=1

    • 便宜模型真实运行 claude / codebuddy / cursor-agent,断言 teamai 注入的 SessionStart hook 端到端真正触发
    • 复用 teamai 真实的 injectHooks() 产物(与生产逐字节一致的 on-disk 形态),仅把 dispatch 命令重定向到隔离 sandbox HOME + 已构建的 dist/index.js
      • 不污染开发者真实的 ~/.teamai
      • 无需全局安装 teamai
      • CLI 仍使用真实 HOME,因此各工具自身的认证照常工作(只有被 spawn 的 hook 子进程看到 sandbox HOME)。
    • 验证信号:hook 触发后 dashboardReportHandler 向 sandbox 的 dashboard/events.jsonl 追加 session_start 事件。

新增脚本:npm run test:e2e:live

本地验证结果

  • 完整 e2e 套件:116 passed / 24 skipped(远程凭证用例按既有方式跳过)。
  • Live 工具集成(haiku 类便宜模型):
    • claudeclaude-haiku-4-5-20251001)—— SessionStart hook 真实触发
    • codebuddyclaude-haiku-4.5)—— SessionStart hook 真实触发
    • ⏭️ cursor-agent —— 已接入(项目级 .cursor/hooks.json + --workspace),运行环境未登录时自动跳过;登录后即可 npm run test:e2e:live 验证。
  • npx tsc --noEmit 通过。

Test plan

  • CI 跑 npm run test:e2e(command-coverage + hook-dispatch,无凭证)
  • 维护者本地在已认证的 claude / codebuddy / cursor-agent 上跑 npm run test:e2e:live
  • 确认 cursor-agent 登录后 live 用例通过(cursor 路径仍待一次真实登录运行确认)

备注

  • live 套件按 env flag gate,CI 无凭证环境会整体跳过,不影响现有流水线。
  • 便宜模型可通过 TEAMAI_E2E_CLAUDE_MODEL / TEAMAI_E2E_CODEBUDDY_MODEL / TEAMAI_E2E_CURSOR_MODEL 覆盖。

Made with Cursor

Adds two end-to-end suites that close real gaps in the e2e coverage:

- command-coverage-e2e.test.ts: spawns the built CLI for every
  user-facing command + subcommand. Asserts `--help` exits 0 with usage
  (catches broken dynamic imports / bad option specs), that read-only
  commands degrade gracefully on an uninitialized HOME without a wiring
  bug, and that the hook-dispatch backbone appends the expected dashboard
  events for real hook payloads (no LLM / credentials needed → CI-safe).

- tool-integration-e2e.test.ts: runs the real `claude`, `codebuddy`, and
  `cursor-agent` CLIs against a cheap model and asserts the hooks teamai
  injects actually fire end-to-end. Reuses teamai's real injectHooks()
  output, redirecting the dispatch command to an isolated sandbox HOME +
  the built dist so it never pollutes the developer's ~/.teamai and needs
  no global install. The CLI keeps its real HOME so auth still works.
  Gated behind TEAMAI_E2E_LIVE_TOOLS=1 (needs authed CLIs + cheap model
  calls). Added `npm run test:e2e:live`.

Verified locally: claude + codebuddy fire the SessionStart hook with
claude-haiku; cursor-agent auto-skips until logged in.

Co-authored-by: Cursor <cursoragent@cursor.com>
…wire into CI (#95)

Decouple the live claude/codebuddy hook-integration e2e from ambient
interactive login by making the Anthropic API key and base URL
configurable via TEAMAI_E2E_ANTHROPIC_API_KEY / TEAMAI_E2E_ANTHROPIC_BASE_URL,
broaden the assertion to cover the full hook lifecycle (session_start,
prompt_submit, tool_use, stop), and add an e2e-live-tools CI job that
runs it on GitHub Actions when the corresponding secret is configured.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
@jeff-r2026 jeff-r2026 closed this Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant