Skip to content

Add GitHub Copilot CLI agent integration#570

Open
khaong wants to merge 8 commits intomainfrom
alex/add-copilot-agent
Open

Add GitHub Copilot CLI agent integration#570
khaong wants to merge 8 commits intomainfrom
alex/add-copilot-agent

Conversation

@khaong
Copy link
Contributor

@khaong khaong commented Mar 2, 2026

Summary

  • Integrate GitHub Copilot CLI as a new agent with full hook lifecycle support (session-start, user-prompt-submit, stop, post-commit, pre-tool-use, post-tool-use) via .github/hooks/entire.json config files
  • Implement JSONL transcript parsing, session directory resolution, and prompt extraction with comprehensive unit tests using real Copilot CLI output as golden fixtures
  • Handle agents without SubagentStart hooks by falling back to pre-prompt state for task file detection, preventing spurious task checkpoints (extracted resolveTaskPreUntrackedFiles helper with unit tests)
  • Fix tmux-based E2E test infrastructure: forward PATH into tmux sessions so the freshly-built binary is used (not stale system-wide installs), and handle Copilot CLI's "Confirm folder trust" startup dialog

Test plan

  • mise run fmt && mise run lint && mise run test:ci — all unit and integration tests pass
  • mise run test:e2e --agent copilot-cli TestSinglePrompt — basic single-prompt E2E
  • mise run test:e2e --agent copilot-cli TestInteractiveMultiStep — interactive multi-step with tmux
  • mise run test:e2e --agent copilot-cli — full copilot-cli E2E suite (5 scenarios)
  • Verify no regressions on other agents: mise run test:e2e --agent claude-code TestSinglePrompt

🤖 Generated with Claude Code

khaong and others added 3 commits March 2, 2026 12:36
Phase 1 research for Copilot CLI integration: implementation one-pager
with captured hook payloads, transcript format, and event mapping.
Probe script for manual/automated hook payload capture.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: cd72011e9e2a
Implements the E2E Agent interface for copilot-cli: non-interactive
prompt execution via `copilot -p`, interactive tmux sessions, and
transient error detection. Uses --allow-all-tools for auto-approval.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 7a17709559b6
Add the copilotcli Go package implementing the Agent and HookSupport
interfaces for GitHub Copilot CLI integration. Includes hook install/
uninstall in .github/hooks/entire.json, lifecycle event parsing for
8 hook types, JSONL transcript handling, and session management.
Registry constants and blank import added for auto-registration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: e7adf08aff2c
Copilot AI review requested due to automatic review settings March 2, 2026 03:25
@cursor
Copy link

cursor bot commented Mar 2, 2026

PR Summary

Medium Risk
Adds a new agent integration that installs/removes repo hook configs and reads/writes Copilot session transcripts, which could affect lifecycle detection and hook configuration in repos. The change is mostly additive but touches agent registry/hook command wiring and filesystem I/O paths.

Overview
Adds a new preview copilot-cli agent integration, including session transcript read/write + JSONL chunking and lifecycle mapping from Copilot hook payloads to Entire events.

Implements hook management for Copilot’s .github/hooks/*.json model by creating/updating .github/hooks/entire.json, supporting idempotent install/uninstall, --force reinstalls, and round-trip preservation of unknown JSON fields/hook types.

Wires the agent into the registry/hidden hooks command, adds an E2E runner for Copilot CLI, and includes a probe script + docs for validating Copilot hook payloads; .gitignore now ignores tmp/.

Written by Cursor Bugbot for commit 3e28095. Configure here.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new GitHub Copilot CLI integration to Entire CLI’s agent framework so Copilot sessions can be detected, hooked into lifecycle events, and exercised by existing E2E scenarios.

Changes:

  • Introduces cmd/entire/cli/agent/copilotcli/ implementing the Agent + HookSupport surface (hook install/uninstall, lifecycle parsing, transcript handling).
  • Adds an E2E agent runner for Copilot CLI (e2e/agents/copilot-cli.go) and registers the agent in the hooks command/registry.
  • Adds a probe script for validating Copilot CLI hook payloads and updates .gitignore to ignore tmp/.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
scripts/test-copilot-cli-agent-integration.sh Adds a local probe script to wire Copilot hooks and capture stdin payloads for validation.
e2e/agents/copilot-cli.go Adds Copilot CLI to the E2E agent runner set (non-interactive prompt + tmux session support).
cmd/entire/cli/hooks_cmd.go Ensures the Copilot CLI agent package is imported/registered for hooks subcommands.
cmd/entire/cli/agent/registry.go Adds Copilot CLI agent name/type constants to the registry.
cmd/entire/cli/agent/copilotcli/types.go Defines Copilot hooks file and hook payload structs used for parsing/writing.
cmd/entire/cli/agent/copilotcli/lifecycle.go Maps Copilot hook invocations into normalized Entire lifecycle events.
cmd/entire/cli/agent/copilotcli/lifecycle_test.go Unit tests for hook-to-event parsing behavior across all hook types.
cmd/entire/cli/agent/copilotcli/hooks.go Implements hook install/uninstall/detection logic targeting .github/hooks/entire.json.
cmd/entire/cli/agent/copilotcli/hooks_test.go Unit tests for hook install/uninstall, idempotency, and unknown-field preservation behavior.
cmd/entire/cli/agent/copilotcli/copilotcli.go Implements the core Agent behavior: detect presence, session read/write, transcript chunking.
cmd/entire/cli/agent/copilotcli/copilotcli_test.go Unit tests for identity, session/transcript operations, chunking, and presence detection.
cmd/entire/cli/agent/copilotcli/AGENT.md Documents Copilot hook types, payload examples, transcript format, and integration limitations.
.gitignore Adds tmp/ to ignored paths.

Comment on lines +32 to +36
type CopilotHookEntry struct {
Type string `json:"type"`
Bash string `json:"bash"`
Comment string `json:"comment,omitempty"`
}
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot hook entries support additional fields (e.g. cwd, timeoutSec, env, and potentially alternative shell keys like powershell). Because CopilotHookEntry only models Type/Bash/Comment, any existing entry fields beyond these will be silently dropped when Entire installs/uninstalls hooks (read → unmarshal into struct → marshal back). Consider either modeling the optional fields explicitly and/or preserving unknown per-entry fields (e.g., with a RawMessage map) so user hook configs don’t get corrupted on round-trip; add a unit test that demonstrates preservation of these entry-level fields.

Copilot uses AI. Check for mistakes.
}

// ProtectedDirs returns directories that Copilot CLI uses for config/state.
func (c *CopilotCLIAgent) ProtectedDirs() []string { return []string{".github"} }
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ProtectedDirs() returning ".github" is much broader than other agent integrations (which protect only their agent-specific config dirs like .claude/.cursor/.gemini/.opencode). Protecting the whole .github tree can prevent cleanup of untracked files under .github during destructive operations (rewind/reset), leaving unexpected residue. Consider narrowing this to just the Copilot hooks path (e.g. ".github/hooks") so protection is scoped to what this integration actually owns.

Suggested change
func (c *CopilotCLIAgent) ProtectedDirs() []string { return []string{".github"} }
func (c *CopilotCLIAgent) ProtectedDirs() []string {
return []string{filepath.Join(".github", "hooks")}
}

Copilot uses AI. Check for mistakes.
Comment on lines +164 to +213
"bash": "cat > '${CAPTURE_DIR}/sessionStart-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"sessionEnd": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/sessionEnd-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"userPromptSubmitted": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/userPromptSubmitted-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"agentStop": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/agentStop-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"subagentStop": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/subagentStop-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"preToolUse": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/preToolUse-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"postToolUse": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/postToolUse-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"errorOccurred": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/errorOccurred-\$(date +%s%N).json'",
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script uses date +%s%N to generate unique filenames inside the hook commands. %N (nanoseconds) isn’t supported by BSD/macOS date, which can lead to non-unique filenames or unexpected output and overwrite captures. Consider switching to a more portable unique suffix (e.g. seconds + PID + RANDOM, or a tiny python/perl helper) and apply it consistently to all hook commands in the here-doc.

Suggested change
"bash": "cat > '${CAPTURE_DIR}/sessionStart-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"sessionEnd": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/sessionEnd-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"userPromptSubmitted": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/userPromptSubmitted-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"agentStop": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/agentStop-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"subagentStop": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/subagentStop-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"preToolUse": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/preToolUse-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"postToolUse": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/postToolUse-\$(date +%s%N).json'",
"comment": "Entire CLI integration probe"
}
],
"errorOccurred": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/errorOccurred-\$(date +%s%N).json'",
"bash": "cat > '${CAPTURE_DIR}/sessionStart-\$(date +%s)-\$\$-\$RANDOM.json'",
"comment": "Entire CLI integration probe"
}
],
"sessionEnd": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/sessionEnd-\$(date +%s)-\$\$-\$RANDOM.json'",
"comment": "Entire CLI integration probe"
}
],
"userPromptSubmitted": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/userPromptSubmitted-\$(date +%s)-\$\$-\$RANDOM.json'",
"comment": "Entire CLI integration probe"
}
],
"agentStop": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/agentStop-\$(date +%s)-\$\$-\$RANDOM.json'",
"comment": "Entire CLI integration probe"
}
],
"subagentStop": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/subagentStop-\$(date +%s)-\$\$-\$RANDOM.json'",
"comment": "Entire CLI integration probe"
}
],
"preToolUse": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/preToolUse-\$(date +%s)-\$\$-\$RANDOM.json'",
"comment": "Entire CLI integration probe"
}
],
"postToolUse": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/postToolUse-\$(date +%s)-\$\$-\$RANDOM.json'",
"comment": "Entire CLI integration probe"
}
],
"errorOccurred": [
{
"type": "command",
"bash": "cat > '${CAPTURE_DIR}/errorOccurred-\$(date +%s)-\$\$-\$RANDOM.json'",

Copilot uses AI. Check for mistakes.
khaong and others added 3 commits March 2, 2026 14:32
Use --allow-all (includes --allow-all-paths) to bypass the "Confirm
folder trust" dialog that blocks interactive sessions in temp dirs.
Default to claude-haiku-4.5 model for cost-efficient E2E testing,
matching the pattern used by the Claude Code E2E runner.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 291f49fbcf7b
Agents like Copilot CLI that lack a SubagentStart hook had no pre-task
state captured, causing DetectFileChanges to treat all untracked files
as new. This created spurious task checkpoints for pre-existing files
(e.g., .github/hooks/entire.json) and left orphaned shadow branches.

Extract resolveTaskPreUntrackedFiles() helper that falls back to the
session's pre-prompt state when pre-task state is nil, and add unit
tests covering all branches.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 550dfcfe8318
Two issues prevented interactive (tmux-based) E2E tests from working:

1. PATH not forwarded to tmux sessions: tmux inherits the server's
   environment, not the client's. The test process prepends the
   freshly-built binary to PATH, but tmux sessions used the stale
   server PATH with an old binary lacking copilot-cli support.
   Fix: forward PATH via env in NewTmuxSession.

2. Wrong prompt pattern: Copilot CLI's interactive prompt uses ❯
   (Unicode U+276F), not > (ASCII). Also added a dialog dismissal
   loop for the "Confirm folder trust" dialog that appears in
   interactive mode for new directories.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: cec165d01d06
@khaong khaong changed the title Add Copilot CLI agent integration Add GitHub Copilot CLI agent integration Mar 2, 2026
@khaong khaong marked this pull request as ready for review March 2, 2026 06:13
@khaong khaong requested a review from a team as a code owner March 2, 2026 06:13
@khaong khaong marked this pull request as draft March 2, 2026 06:14
khaong and others added 2 commits March 2, 2026 17:27
BSD env on macOS requires options (-u) before variable assignments.
The previous commit placed PATH= before -u flags, causing env to
interpret -u as the command to run (exit 127) for agents that unset
environment variables (claude-code, gemini-cli).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 720e4558e8e2
GPT-4.1 has a 0x premium request multiplier on paid Copilot plans,
so E2E test runs won't consume the monthly quota. Claude Haiku 4.5
costs 0.33x per request which adds up across 37 tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Entire-Checkpoint: 62401b0458b4
@khaong khaong marked this pull request as ready for review March 2, 2026 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants