Skip to content

feat(mcp): collect custom request headers for remote MCP servers#1503

Draft
neubig wants to merge 5 commits into
mainfrom
feat/mcp-header-fields
Draft

feat(mcp): collect custom request headers for remote MCP servers#1503
neubig wants to merge 5 commits into
mainfrom
feat/mcp-header-fields

Conversation

@neubig

@neubig neubig commented Jun 26, 2026

Copy link
Copy Markdown
Member

HUMAN:

Human-tested checkbox left unchecked; this is an AI-authored PR pending human QA.

  • A human has tested these changes.

AGENT:

This PR was created by an AI agent (OpenHands) on behalf of @neubig.

End-to-end verification performed locally:

  • npx tsc --noEmit — clean (0 errors).
  • npx eslint + prettier --check on all changed files — clean.
  • npx vitest run for the mcp / i18n / hooks / mcp-service suites — 204 passed, including 6 new mcp-config cases and 3 new install-server-modal cases exercising headerFields.
  • The new install-server-modal test "installs with the headers map and no api_key in the persisted config" confirms a Datadog-style entry produces a persisted mcp_config with headers: { "DD-API-KEY": …, "DD-APPLICATION-KEY": … } and no api_key/Authorization, and the pre-flight test payload carries the same headers map.
  • The new mcp-config round-trip test confirms a persisted header-based config survives parseMcpConfigtoSdkMcpConfig without loss (previously the custom headers were dropped on reload).

Backend already supports arbitrary headers (mcp_router._RemoteMCPServerSpec.headersfastmcp transport, encrypted at rest by settings/model.py), so no backend change was needed.


Why

Some hosted MCP servers authenticate via custom request headers rather than a single Bearer token or OAuth (e.g. Datadog's DD-API-KEY / DD-APPLICATION-KEY, https://docs.datadoghq.com/mcp_server/setup/?tab=other#authentication). The agent-canvas plumbing already supported arbitrary headers end-to-end, but the UI could not produce a header-based config, and a header-based config was silently corrupted on reload. This closes those gaps in a generalized, provider-agnostic way.

Summary

  • utils/mcp-config.ts: parseMcpConfig now preserves non-Authorization headers (previously dropped → data loss on next save); getRemoteCredentialFields merges api_key + explicit headers into one headers map instead of emitting one or the other, matching the backend's headers.setdefault semantics.
  • hooks/mutation/use-add-mcp-server.ts: forward server.headers on add for sse/shttp (update already did this on edit).
  • install-server-modal.tsx: for shttp/sse options with headerFields, render one input per header (password → save-as-secret toggle, same pattern as stdio envFields), validate required fields, build a headers map in the submit payload, and save marked fields as secrets. The Bearer-style api_key input is now shown only for api_key/bearer/basic strategies.
  • mcp-server-form.tsx: add a KEY=VALUE-per-line Headers textarea for sse/shttp in the manual custom editor (manual escape hatch for header-based servers without a catalog entry); Authorization is rejected there to avoid a double header.
  • i18n: add SETTINGS$MCP_HEADERS_LABEL, SETTINGS$MCP_HEADERS_HELP, SETTINGS$MCP_ERROR_HEADERS_AUTHORIZATION_RESERVED with translations for all 15 supported languages.

Issue Number

Closes #1501. Depends on the extensions schema change in OpenHands/extensions#364.

How to Test

  1. npm install
  2. npm run test (or npx vitest run __tests__/utils/mcp-config.test.ts __tests__/components/features/mcp-page/install-server-modal.test.tsx) — all tests pass.
  3. npm run typecheck — clean.
  4. Manual UI smoke (requires the extensions headerFields schema from PR fix(frontend): translate MCP marketplace keys into all supported languages #364 to be published, or a local link): open the MCP marketplace, install a Datadog-style entry with headerFields, confirm two password inputs render, fill them, and confirm the saved server carries DD-API-KEY/DD-APPLICATION-KEY headers. Re-open the server in the custom editor and confirm the Headers textarea is populated and re-saving preserves them.

Video/Screenshots

Screenshots captured by an AI agent (OpenHands) on behalf of @neubig using placeholder credential values.

Marketplace install modal collecting custom request headers

Datadog install modal with Extra Headers textarea

Custom MCP server editor extra headers field

Custom MCP server editor with Extra Headers textarea

Type

  • Bug fix
  • Feature
  • Refactor
  • Breaking change
  • Docs / chore

Notes

  • The headerFields field on the shttp/sse transport is added in the companion extensions PR (feat(integrations): support custom request headers on MCP shttp/sse transports extensions#364). This PR's frontend reads headerFields defensively through a local McpRemoteTransport view so it compiles against the currently published @openhands/extensions 0.6.0 types; once extensions ships the update, the local view becomes a no-op.
  • No backend change required — the backend already supports arbitrary headers end-to-end.

🐳 Docker images for this PR

GHCR package: https://github.com/OpenHands/agent-canvas/pkgs/container/agent-canvas

Component Value
Image ghcr.io/openhands/agent-canvas
Architectures amd64, arm64
Agent Server ghcr.io/openhands/agent-server:1.29.0-python
Automation openhands-automation==1.0.0a12
Commit d410eec2e1f4a960633621ec6c88427e6d0435fb

Pull (multi-arch manifest)

# Multi-arch manifest — Docker automatically pulls the correct architecture
docker pull ghcr.io/openhands/agent-canvas:sha-d410eec

Run

docker run -it --rm \
  -p 8000:8000 \
  ghcr.io/openhands/agent-canvas:sha-d410eec

All tags pushed for this build

ghcr.io/openhands/agent-canvas:sha-d410eec-amd64
ghcr.io/openhands/agent-canvas:feat-mcp-header-fields-amd64
ghcr.io/openhands/agent-canvas:pr-1503-amd64
ghcr.io/openhands/agent-canvas:sha-d410eec-arm64
ghcr.io/openhands/agent-canvas:feat-mcp-header-fields-arm64
ghcr.io/openhands/agent-canvas:pr-1503-arm64
ghcr.io/openhands/agent-canvas:sha-d410eec
ghcr.io/openhands/agent-canvas:feat-mcp-header-fields
ghcr.io/openhands/agent-canvas:pr-1503

About Multi-Architecture Support

  • Each tag (e.g., sha-d410eec) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., sha-d410eec-amd64) are also available if needed

Some hosted MCP servers authenticate via custom request headers rather than a
single Bearer token or OAuth (e.g. Datadog's DD-API-KEY / DD-APPLICATION-KEY,
https://docs.datadoghq.com/mcp_server/setup/?tab=other#authentication). The
agent-canvas plumbing already supported arbitrary headers end-to-end
(MCPServerConfig.headers, MCPSSEServer/MCPSHTTPServer.headers,
mcp-service.api.ts, use-update-mcp-server.ts, and the backend mcp_router +
fastmcp transport + encrypted settings), but the UI could not produce a
header-based config, and a header-based config was silently corrupted on reload.

Changes:

- utils/mcp-config.ts:
  - parseMcpConfig now preserves non-Authorization headers onto the parsed
    server object (previously they were dropped, causing data loss on the
    next save).
  - getRemoteCredentialFields merges api_key (-> Authorization via ??=) and
    explicit headers into one headers map instead of emitting one or the
    other, matching the backend's headers.setdefault semantics.

- hooks/mutation/use-add-mcp-server.ts: forward server.headers on add for
  sse/shttp (use-update-mcp-server already did this on edit).

- utils/mcp-marketplace-utils.ts: add a local McpRemoteTransport view that
  includes the optional headerFields field, so the install modal is typed
  against the not-yet-published extensions schema (extensions PR #364).

- components/features/mcp-page/install-server-modal.tsx: for shttp/sse
  options with headerFields, seed field state, render one input per header
  (password -> save-as-secret toggle, same pattern as stdio envFields),
  validate required fields, build a headers map in the submit payload, and
  save marked fields as secrets. The single Bearer-style api_key input is
  now shown only for api_key/bearer/basic strategies.

- components/features/settings/mcp-settings/mcp-server-form.tsx: add a
  KEY=VALUE-per-line Headers textarea for sse/shttp in the manual custom
  editor, so users can hand-add header-based servers (e.g. Datadog) without
  a catalog entry. Authorization is rejected there to avoid a confusing
  double header (the API Key field is the canonical Bearer source).

- i18n: add SETTINGS$MCP_HEADERS_LABEL, SETTINGS$MCP_HEADERS_HELP, and
  SETTINGS$MCP_ERROR_HEADERS_AUTHORIZATION_RESERVED with translations for
  all 15 supported languages.

Tests:
- mcp-config.test.ts: 6 new cases covering header preservation, round-trip,
  api_key+headers merge, hand-authored Authorization not clobbered, and
  parsing Authorization+extra headers back into api_key+headers.
- install-server-modal.test.tsx: 3 new cases for headerFields-based installs
  (renders one input per header + no api_key field, blocks on empty required
  header, installs with headers map and no api_key/Authorization).

Closes #1501.

Co-authored-by: openhands <openhands@all-hands.dev>
@vercel

vercel Bot commented Jun 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agent-canvas Ready Ready Preview, Comment Jul 1, 2026 12:20am

Request Review

all-hands-bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Review complete.

This review was performed through OpenHands Cloud Automation. You can log in and view the conversation here.

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Acceptable — Correct, well-motivated feature with solid data-layer test coverage. Two issues worth addressing before merge.


What this PR does

Fixes a silent data-loss bug (custom headers dropped on reload) and enables headerFields-based authentication in the install modal, allowing providers like Datadog that authenticate via named request headers (DD-API-KEY / DD-APPLICATION-KEY) to be installed from the marketplace catalog.

The data-flow is clean end-to-end:

  • parseMcpConfig now preserves non-Authorization headers as extraHeaders
  • getRemoteCredentialFields merges api_key (→ Authorization: Bearer …) and explicit headers into a single map using ??= semantics that match the backend's headers.setdefault("Authorization", …) behaviour in mcp_router.py
  • The install modal renders per-field inputs for headerFields, validates required fields, and builds the correct payload
  • mcp-server-form.tsx adds a manual KEY=VALUE textarea for header-based servers without a catalog entry

Issues

[IMPROVEMENT OPPORTUNITIES]

  • [mcp-server-form.tsx, Line 253] Redundant alias: formatHeaders is a one-liner that delegates to formatEnvironmentVariables with no added behavior. Remove the alias and call formatEnvironmentVariables(server?.headers) directly at the call site (line 421). The current indirection implies a behavioral difference that doesn't exist.

[TESTING GAPS]

  • [__tests__/.../mcp-server-form.validation.test.tsx] validateHeadersFormat is untested: The new validateHeadersFormat in mcp-server-form.tsx adds the Authorization-rejection rule and the KEY=VALUE format check, but mcp-server-form.validation.test.tsx was not updated to cover it. The Authorization rejection in particular is worth a regression test — it's the primary guardrail against a confusing double-header scenario. Add cases for: valid headers pass, invalid format rejected, authorization (case-insensitive) rejected, and that the built MCPServerConfig carries the parsed headers map.

[RISK ASSESSMENT]

  • [Overall PR] ⚠️ Risk Assessment: 🟢 LOW
    All changes are frontend-only. The bug fix is safe (preserves headers rather than dropping them). The ??= merge semantics are correct and match the backend. The defensive delete headers.Authorization in buildConfig limits the blast radius of any validation bypass. No backend changes required.

VERDICT:
Worth merging — Core logic is sound and data-layer coverage is thorough. The two items above (alias removal and form-validation tests) are polish; they don't block the feature.

KEY INSIGHT:
The cleanest decision in this PR is the ??= merge strategy in getRemoteCredentialFields — hand-authored Authorization wins over api_key-derived, which mirrors backend semantics and makes the round-trip deterministic.


Improve this review? If any feedback above seems incorrect or irrelevant to this repository, you can teach the reviewer to do better:

  1. Add a .agents/skills/custom-codereview-guide.md file to your branch (or edit it if one already exists) with the /codereview trigger and the context the reviewer is missing. See the customization docs for the required frontmatter format.
  2. Re-request a review - the reviewer reads guidelines from the PR branch, so your changes take effect immediately.
  3. When your PR is merged, the guideline file goes through normal code review by repository maintainers.

Resolve with AI? Install the iterate skill in your agent and run /iterate to automatically drive this PR through CI, review, and QA until it's merge-ready.

Was this review helpful? React with 👍 or 👎 to give feedback.

This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation. View conversation

return null;
};

const formatHeaders = (headers?: Record<string, string>): string =>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: formatHeaders is a transparent alias — it adds an extra named abstraction without any distinct behavior. The call site on line 421 could just read formatEnvironmentVariables(server?.headers) directly, the same way line 515 already calls formatEnvironmentVariables(server?.env) without a wrapper. Remove the alias to avoid implying a behavioral difference that doesn't exist.

The Headers textarea placeholder and the SETTINGS$MCP_HEADERS_HELP
translation both used DD-API-KEY as the example, which is Datadog-specific.
Replace with the generic HEADER_NAME=value so the manual editor stays
provider-neutral.

Co-authored-by: openhands <openhands@all-hands.dev>
…ll modal

Some hosted MCP servers use a region/account-specific URL (e.g. Datadog's
https://mcp.<site>.datadoghq.com/v1/mcp), so a catalog-pinned URL can return
403 for users on a different site. When a transport declares `urlEditable:
true` (extensions PR #364), render the URL as an editable input pre-filled
with the catalog URL instead of read-only, track it in field state, validate
it on submit, and use the user-edited value in the install payload.

Read-only URLs (the default) are unchanged, so existing integrations keep
their pinned-host behavior.

Adds a test exercising urlEditable: the URL input is enabled and pre-filled,
and the user-edited URL is what gets persisted.

Co-authored-by: openhands <openhands@all-hands.dev>
getMcpMarketplaceCatalog filtered by getDefaultMcpConnectionOption (the first
MCP option), which can be OAuth. An OAuth-only entry (e.g. the published
Datadog catalog before its `api` option lands) has no option the local install
modal can render, so clicking its card opened an empty modal. Filter by
getInstallableMcpConnectionOption instead so entries with no non-OAuth MCP
option don't get a card at all.

Adds tests: OAuth-only entries are excluded; entries with an OAuth default but
a non-OAuth fallback (Slack) are kept.

Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

PR Artifacts Notice

This PR contains a .pr/ directory with PR-specific artifacts. This directory will be automatically removed when the PR is approved.

Fork PRs require manual cleanup before merging.

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

✅ Mock-LLM E2E Tests

60/60 passed

Commit: d410eec2 · Workflow run · Test artifacts

Details
Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 27.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.4s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.7s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 14.5s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.5s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.7s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.7s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 795ms
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 16.8s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 21.7s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 15.2s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 106ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.4s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 7.4s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 14.6s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.5s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 12.6s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 5.4s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 8.3s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 8.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.5s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.8s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.0s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 6.0s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 6.0s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.8s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.6s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.6s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.9s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.4s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 15.8s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.9s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.8s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 15.3s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 7.8s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 9.3s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 16.6s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.8s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 16.5s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 14.7s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 14.5s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

✅ Mock-LLM Docker E2E Test Results

60/60 passed

Commit: d410eec2 · Workflow run · Test artifacts

Details
Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.2s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 32.4s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.3s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 15.9s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.5s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.6s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.8s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 729ms
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 22.2s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 21.8s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 13.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 100ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.2s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.5s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.1s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 12.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 11.2s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.6s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.4s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.5s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 12.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.7s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.7s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.3s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.3s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.7s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.5s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.8s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.2s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.6s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.6s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 14.9s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.6s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.5s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.3s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.3s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MCP: collect custom request headers in install modal + custom editor (e.g. Datadog)

4 participants