Skip to content

feat: support Devin CLI harness - #545

Open
codeatspeed wants to merge 23 commits into
kunchenguid:mainfrom
codeatspeed:feat_add_devin
Open

feat: support Devin CLI harness#545
codeatspeed wants to merge 23 commits into
kunchenguid:mainfrom
codeatspeed:feat_add_devin

Conversation

@codeatspeed

@codeatspeed codeatspeed commented Jul 14, 2026

Copy link
Copy Markdown

What Changed

  • Adds Devin CLI as a verified Firstmate harness, captain, with spawning, model selection, liveness detection, and foreground supervision support.
  • Configures workspace and task-specific Devin hooks for command guards, turn-end signaling, and preservation of existing user and repository configuration.
  • Centralizes harness process detection and expands documentation and automated coverage for Devin integration and supervision behavior.

Risk Assessment

✅ Low: Captain, the Devin integration is well-bounded and the only remaining issue found is a non-functional duplicate control-flow arm.

Testing

Startup baselines and all focused automated suites passed; manual verification against installed Devin CLI 3000.1.27 demonstrated harness detection, rendered foreground supervision, and safety-hook denial, with the transcript captured as evidence and no testing artifacts left in the worktree.

Evidence: Devin end-user flow transcript
=== Installed Devin CLI ===
devin 3000.1.27 (0d4bf12e)

=== Firstmate detects a Devin session ===
devin

=== Firstmate renders Devin supervision ===
================================================================================
SUPERVISION OPERATING INSTRUCTIONS - primary harness: devin
================================================================================
Current state:
- Lock: held by this session; this session owns normal supervision unless away mode says otherwise.
- Away mode: inactive.
- X mode: inactive; use the default watcher cadence.
- Ordinary wake: follow the continuation in the harness protocol below; do not use shell &.

Mode: Devin foreground checkpoint.

When this session owns supervision and away mode is not active:
1. Drain first with `bin/fm-wake-drain.sh`.
2. Source `/Users/muralinunna/.no-mistakes/worktrees/7a17be65de34/01KYKC0642FYA87SS0P6VXH2NK/config/x-mode.env` first when X mode is active.
3. Run `bin/fm-watch-checkpoint.sh --seconds 180` as one foreground tool call.
4. Never bundle the checkpoint with unrelated commands.
5. Never use shell `&` for watcher supervision.
   A shell `&`, a truncating pipe, or bundling is denied automatically by the PreToolUse seatbelt registered in `.devin/config.json`.
6. When the checkpoint returns with `signal:`, `stale:`, `check:`, or `heartbeat`, drain queued wakes, handle them, then run one fresh checkpoint.
7. A timeout is only a bounded control return.
   If work remains in flight, run another checkpoint after handling user input and queued wakes.
8. Treat any failed checkpoint as an alarm and repair it before ending the turn.
9. Do not send idle progress while a checkpoint is running.

Devin has no verified background-task completion wake contract in Firstmate.
The bounded foreground checkpoint is the conservative supported mechanism, matching Codex's supervision shape while retaining Devin's native blocking Stop-hook backstop.


=== Devin safety hook rejects unsafe background watcher command ===
exit=2
{"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"deny"},"systemMessage":"[watcher-background] a protected watcher command cannot run in an asynchronous shell list or through nohup/disown"}

Pipeline

Updates from git push no-mistakes

⏭️ **intent** - skipped

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 info
  • 🚨 bin/fm-supervision-instructions.sh:147 - The new devin) arm has no preceding ;; after the pi|pi-signed) arm, making the entire script syntactically invalid. This already causes fm-session-start.sh to fail while rendering supervision instructions for every harness.
  • 🚨 bin/fm-bootstrap.sh:719 - The second verified definition overrides the first and drops pi-signed and kimi. Existing dispatch profiles using either supported harness are therefore rejected as invalid; merge the lists and retain all previously verified adapters.
  • 🚨 bin/fm-bootstrap.sh:440 - The first wildcard arm consumes every harness not in the old list, so devin never reaches the later Devin-inclusive arm and dead/missing Devin secondmates are reclassified as unverified-harness instead of recovered. Consolidate this into one verified-harness arm containing the existing adapters plus Devin.
  • 🚨 .agents/skills/harness-adapters/SKILL.md:4 - The frontmatter now contains two description keys. Strict YAML/skill loaders can reject duplicate keys (and the session did not expose harness-adapters as an available skill), which disables the mandatory harness workflow. Replace both entries with one combined description.
  • ⚠️ bin/fm-tmux-lib.sh:96 - The change documents Devin's verified esc to interrupt busy signature but does not add a devin) branch here. Because supplied harnesses intentionally do not use the shared fallback, active Devin turns are classified as idle, allowing premature wakes and stale-task handling. Map Devin to the Codex busy regex.

🔧 Fix: Captain, fix Devin harness integration regressions
2 issues (1 error, 1 warning) still open:

  • 🚨 AGENTS.md:222 - A 153-line stale copy of sections 3–7 was inserted mid-document. It contradicts the authoritative rules above—for example, it omits pi-signed and kimi from verified adapters and says invalid dispatch profiles should be ignored even though the current contract requires failing closed. Remove the duplicated block so agents cannot follow conflicting operational instructions.
  • ⚠️ tests/fm-session-start.test.sh:511 - This helper now runs session startup twice. The second invocation also swaps root and home, drops the Herdr fixture variables, and constructs PATH from the bare fakebin argument, so the Herdr recovery test no longer exercises its intended environment reliably. Remove the appended invocation and instead unset DEVIN_CLI in the shared run_session_start wrapper.

🔧 Fix: Captain, remove duplicate contract and fix session helper
4 issues (2 errors, 2 infos) still open:

  • 🚨 docs/turnend-guard.md:91 - A stale 158-line copy of the turn-end contract was appended after the current contract. It contradicts current behavior—for example, lines 139–141 say Claude allows stop_hook_active=true, while lines 52–55 document that Claude now ignores it and uses bounded re-blocking. Integrate the Devin-specific facts into the current sections, then remove the duplicated legacy contract.
  • 🚨 .agents/skills/harness-adapters/SKILL.md:60 - The mandatory adapter skill contains overlapping, contradictory integration summaries: the newly added “every verified harness” blocks omit pi-signed and Kimi qualifications present immediately above, and the Devin model row is stranded below the model-discovery table. Consolidate each contract into one accurate block and place Devin in the launch-profile table.
  • ℹ️ bin/fm-spawn.sh:461 - This second grok) case is unreachable because line 455 already matches grok; it also preserves a stale launch path that bypasses the operational-input encoder. Remove the dead arm to keep one authoritative Grok launch template.
  • ℹ️ bin/fm-harness.sh:31 - The new shared process helper is sourced twice consecutively. Remove the duplicate source operation so initialization has one owner and future helper side effects cannot run twice.

🔧 Fix: Captain, consolidate Devin contracts and remove duplicate paths
1 info still open:

  • ℹ️ bin/fm-supervision-instructions.sh:86 - This harness-selection case duplicates the preceding arm; the first arm already matches every listed harness except Devin, making the repeated alternatives unreachable. Collapse both arms into one claude|codex|opencode|pi|grok|devin) branch so the supervision protocol mapping has one authoritative list.
✅ **Test** - passed

✅ No issues found.

  • bin/fm-session-start.sh
  • bin/fm-test-run.sh tests/fm-devin-harness.test.sh
  • bin/fm-test-run.sh tests/fm-harness-process.test.sh
  • bin/fm-test-run.sh tests/fm-supervision-instructions.test.sh
  • bin/fm-test-run.sh tests/fm-bootstrap.test.sh
  • bin/fm-test-run.sh tests/fm-session-start.test.sh
  • devin --version and DEVIN_CLI=1 bin/fm-harness.sh
  • Rendered bin/fm-supervision-instructions.sh --harness devin --read-only 0 --afk 0 --x-mode 0
  • Invoked the tracked Devin PreToolUse hook with unsafe bin/fm-watch-arm.sh & input and confirmed denial with exit 2
  • git status --short cleanup audit
✅ **Document** - passed

✅ No issues found.

⚠️ **Lint** - 1 warning
  • ⚠️ linter found issues (exit code 127)

🔧 Fix: Captain, remove duplicate supervision harness lint case
1 warning still open:

  • ⚠️ linter found issues (exit code 127)

🔧 Fix: Captain, lint passes with pinned ShellCheck
1 warning still open:

  • ⚠️ linter found issues (exit code 127)

🔧 Fix: Captain, verify clean lint with pinned ShellCheck
1 warning still open:

  • ⚠️ linter found issues (exit code 127)
✅ **Push** - passed

✅ No issues found.

@kunchenguid

kunchenguid commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Thanks for the PR! This branch currently has a merge conflict with the base branch.

When you get a chance, please rebase onto (or merge) the latest base branch, resolve the conflict, and push. After that, checks will re-run and the PR will get looked at again.

Noted for firstmate#545 at 38a9ee5f.

@codeatspeed codeatspeed changed the title feat: add Devin CLI harness support feat: add first-class Devin CLI harness support Jul 15, 2026
@codeatspeed

Copy link
Copy Markdown
Author

Thanks for the PR! This branch currently has a merge conflict with the base branch.

When you get a chance, please rebase onto (or merge) the latest base branch, resolve the conflict, and push. After that, checks will re-run and the PR will get looked at again.

Noted for firstmate#545 at 38a9ee5f.

Done.

@kunchenguid

kunchenguid commented Jul 16, 2026

Copy link
Copy Markdown
Owner

Automated reminder: thanks for the PR! This branch currently has a merge conflict with the base branch.

When you get a chance, please rebase onto (or merge) the latest base branch, resolve the conflict, and push. After that, checks will re-run and the PR will get looked at again.

Noted for firstmate#545 at c10fd2f2.

@kunchenguid

Copy link
Copy Markdown
Owner

Automated reminder: this PR still looks blocked on a rebase or merge conflict fix.

If you are still interested, please rebase onto the current base branch, resolve the conflict, and push.

If I do not hear back, I may close this as inactive.

@codeatspeed codeatspeed changed the title feat: add first-class Devin CLI harness support feat: support Devin CLI harness Jul 28, 2026
@codeatspeed

Copy link
Copy Markdown
Author

Automated reminder: this PR still looks blocked on a rebase or merge conflict fix.

If you are still interested, please rebase onto the current base branch, resolve the conflict, and push.

If I do not hear back, I may close this as inactive.

Rebased.

@kunchenguid kunchenguid removed the wheelhouse:pending-contributor-action Managed by Wheelhouse label Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants