Skip to content

Route desktop agents through Browser Harness JS#417

Merged
Cheggin merged 2 commits intomainfrom
codex/browser-harness-js-runtime
May 7, 2026
Merged

Route desktop agents through Browser Harness JS#417
Cheggin merged 2 commits intomainfrom
codex/browser-harness-js-runtime

Conversation

@Cheggin
Copy link
Copy Markdown
Collaborator

@Cheggin Cheggin commented May 7, 2026

Summary

  • vendor browser-use/browser-harness-js runtime and interaction skill markdown into the desktop harness
  • replace the legacy helpers.js/TOOLS.json browser-control path with browser-harness-js plus connectToAssignedTarget()
  • restore domain-skills routing in AGENTS.md and frame harness edits as a rare escape hatch
  • wire Codex, Claude Code, and BrowserCode env/prompt setup to the vendored CLI

Verification

  • cd app && npm run typecheck
  • cd app && npm run test -- --run tests/unit/hl/stock-helpers.test.ts tests/unit/hl/harnessBootstrap.test.ts tests/unit/hl/codexAdapter.test.ts tests/unit/hl/browsercodeAdapter.test.ts
  • task agent:run PROMPT="open example.com and report the title" ENGINE=codex JSON=1 returned title Example Domain

Stacked on #415 (fix/onboarding-cli-launch-router) to avoid mixing with that existing branch.


Summary by cubic

Routes desktop agents through a vendored browser-harness-js CLI to drive the assigned Chromium target over CDP, replacing TOOLS.json and most of helpers.js with a thin bridge. Bundles the runtime and recipes in the harness and updates agent guidance to use CDP domains directly.

  • New Features

    • Bundles browser-harness-js runtime (./browser-harness-js/sdk/) and SKILL.md, plus new interaction skill docs (cookies, iframes, dialogs, uploads, etc.) in the harness.
    • Routes codex, claude-code, and browsercode engines through the CLI; injects PATH, CDP_REPL_PORT, and CDP_REPL_LOG via applyBrowserHarnessEnv.
    • Rewrites AGENTS.md for Browser Harness JS; helpers.js is now a minimal CLI bridge; UI/logs show AGENTS.md when edited.
    • Adds targeted tests for harness bootstrap and engine adapters.
  • Migration

    • TOOLS.json is removed; use browser-harness-js snippets instead of adding browser control tools.
    • Existing scripts that touch helpers.js still run, but prefer connectToAssignedTarget() and session.* CDP calls.
    • No external install needed; the vendored SDK is excluded from app lint/TS (eslint.config.js, tsconfig.json).

Written for commit 343ae93. Summary will update on new commits.

@Cheggin Cheggin force-pushed the fix/onboarding-cli-launch-router branch from 18c21ba to 664562d Compare May 7, 2026 20:07
@Cheggin Cheggin force-pushed the codex/browser-harness-js-runtime branch from a5c568f to ce6b623 Compare May 7, 2026 20:10
@Cheggin
Copy link
Copy Markdown
Collaborator Author

Cheggin commented May 7, 2026

@cubic review

@cubic-dev-ai
Copy link
Copy Markdown

cubic-dev-ai Bot commented May 7, 2026

@cubic review

@Cheggin I have started the AI code review. It will take a few minutes to complete.

Cheggin added 2 commits May 7, 2026 13:12
Agents need a local CDP runtime that does not depend on legacy helper tools or global skill installation. This vendors the upstream browser-harness-js runtime and interaction recipes, while excluding the bundled SDK from app TypeScript and ESLint passes because it is a runtime asset rather than app source.

Constraint: App-spawned agents run from userData/harness and need the CLI available relative to that harness.

Rejected: Install the upstream skill globally at task time | mutates the user's toolchain and makes launches depend on external setup.

Confidence: high

Scope-risk: moderate

Directive: Treat browser-harness-js/sdk as vendored runtime; update from upstream intentionally, not via incidental formatting or lint fixes.

Tested: cd app && npm run typecheck; targeted Vitest harness and adapter tests.

Not-tested: Fresh packaged production app launch.
The desktop app now gives agents a vendored browser-harness-js CLI, an assigned-target connection helper, and reference folders for browser mechanics and domain playbooks. This replaces the old helpers.js/TOOLS.json tool-extension path while preserving a narrow manual-edit escape hatch for real runtime blockers.

Constraint: Agents must drive only the BrowserPool-assigned CDP target identified by BU_TARGET_ID and BU_CDP_PORT.

Rejected: Keep regular helper APIs as the primary surface | it keeps encouraging agents to patch tools that the upstream CDP runtime already covers.

Confidence: high

Scope-risk: moderate

Directive: Do not reintroduce TOOLS.json as an agent-facing browser control surface; add browser mechanics as upstream browser-harness-js recipes or tightly scoped app runtime fixes.

Tested: cd app && npm run typecheck; cd app && npm run test -- --run tests/unit/hl/stock-helpers.test.ts tests/unit/hl/harnessBootstrap.test.ts tests/unit/hl/codexAdapter.test.ts tests/unit/hl/browsercodeAdapter.test.ts; task agent:run smoke opened example.com and returned Example Domain.

Not-tested: Fresh packaged production app launch.
@Cheggin Cheggin changed the base branch from fix/onboarding-cli-launch-router to main May 7, 2026 20:12
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 39 files

@Cheggin Cheggin force-pushed the codex/browser-harness-js-runtime branch from ce6b623 to 343ae93 Compare May 7, 2026 20:12
@Cheggin Cheggin merged commit ff44ae6 into main May 7, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant