From b387747bf78cd1b6d4370db189f5596340694667 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 09:14:30 +0200 Subject: [PATCH 01/90] ci: apply middleware/migrations in the schema job MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MIGRATION_DOMAINS listed five domains and omitted middleware/migrations, the core runtime domain holding 0001-0030. Every migration there had shipped without ever being applied — or re-applied for the idempotency check — against a real Postgres in CI: the entire MCP schema (0003, 0008, 0009, 0010/0013, 0012/0014, 0015/0016, 0017-0020) and every dev-platform migration (0022-0030). Suspected during #330, now confirmed and closed. No latent schema defect was exposed. All 30 files apply and re-apply cleanly against pgvector/pgvector:pg16, in both possible domain orderings and additionally with rows present. The domain is self-contained: no cross-domain foreign keys, no object names shared with the other five domains, and no extension dependency (gen_random_uuid is core since pg13). The comment now records the three domains that remain uncovered, each of which needs its own audit before being enabled. --- .github/workflows/ci.yml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 30dfb27c..4743a556 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -136,7 +136,18 @@ jobs: # in lexical (file-name) order — which matches the numbered # migration convention. Newline-separated so the shell loop can # iterate cleanly. + # + # `middleware/migrations` is the core runtime domain (multi- + # orchestrator, agent-builder graph, MCP, dev-platform). It is + # self-contained: no cross-domain FKs and no extension + # dependencies, so its position in this list is not load-bearing. + # + # Still uncovered (each needs its own audit before enabling): + # middleware/src/conductor/migrations, + # middleware/src/services/graph/migrations, + # middleware/packages/harness-memory-postgres/src/migrations. MIGRATION_DOMAINS: | + middleware/migrations middleware/packages/harness-knowledge-graph-neon/src/migrations middleware/src/auth/migrations middleware/src/plugins/routines/migrations From 9ac4c372acd09426ec5c235ab55e260683d1c2a0 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 09:14:30 +0200 Subject: [PATCH 02/90] test(mcp): first pg coverage for the MCP registry and OAuth schema MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit No pg test touched MCP before this — only memoryStoreConformance, pluginVerdictStore and skillLifecycleStore existed. Covers the registry seed and catalog-kind backfill (0010 + 0013, including that 0013's UPDATE actually lifts the official registry off the 'generic' column default), the kind/auth_kind/source/registered_via CHECK sets, marketplace provenance with ON DELETE SET NULL detaching an imported server from a deleted catalog, the 0014 partial unique index on top-level MCP grants (and that it leaves native grants alone), and the 0015/0016 OAuth surface — authorize-time endpoint pinning plus token/flow cascade on server delete. Each assertion was mutation-checked against a deliberately broken schema. A second suite covers what the CI gate structurally cannot: the CI idempotency check re-applies against an EMPTY database, so it can never catch a migration that only breaks once rows exist. It re-applies all 30 files with MCP rows in place, in its own throwaway database — re-running 0001/0003 drops and recreates the NOTIFY triggers, which must not happen underneath a concurrently running suite. Both suites skip when no test Postgres is reachable and scope every row to a w04-mcp- tenant prefix. Pools are capped: the runner executes files concurrently and ~16 other pg suites each hold a default-sized (max 10) pool, so an uncapped extra pool here exhausts max_connections and cancels an unrelated suite mid-run (observed on ConductorWebhookSubscriptionStore). --- docs/CHANGELOG.md | 51 +++ middleware/test/mcpRegistrySchema.pg.test.ts | 387 +++++++++++++++++++ 2 files changed, 438 insertions(+) create mode 100644 middleware/test/mcpRegistrySchema.pg.test.ts diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 0791c3ea..d4977da4 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -18,6 +18,57 @@ entry. See `CONTRIBUTING.md` § Releases & changelog. ## [Unreleased] +### Fixed — the CI schema job never applied `middleware/migrations` + +- `MIGRATION_DOMAINS` in `.github/workflows/ci.yml` listed five domains and + omitted `middleware/migrations` — the core runtime domain holding `0001` + through `0030`. Every migration there had therefore shipped without ever + being applied, or re-applied for the idempotency check, against a real + Postgres in CI: the whole MCP schema (`0003` agent-builder graph, `0008` + tool verdicts, `0009` call log, `0010`/`0013` registries, `0012`/`0014` + grants, `0015`/`0016` OAuth 2.1 + PKCE, `0017`–`0020`) and every + dev-platform migration (`0022`–`0030`). The gap was suspected during #330 + and is now closed; the domain is applied first, ahead of the knowledge-graph + domain. +- **No latent schema defect was exposed.** All 30 files apply and re-apply + cleanly against `pgvector/pgvector:pg16`, in both possible domain + orderings, and additionally with rows present. The domain is fully + self-contained: no cross-domain foreign keys, no shared object names with + the other five domains, and no extension dependency at all + (`gen_random_uuid()` is core since pg13). Verified locally with a + reproduction of the CI job before the workflow change was pushed. +- The workflow comment now records the three domains that remain uncovered + (`middleware/src/conductor/migrations`, + `middleware/src/services/graph/migrations`, + `middleware/packages/harness-memory-postgres/src/migrations`), each of which + needs its own audit before being enabled. + +### Added — first pg coverage for the MCP schema + +- `middleware/test/mcpRegistrySchema.pg.test.ts` — no pg test touched MCP + before this (only `memoryStoreConformance`, `pluginVerdictStore` and + `skillLifecycleStore` existed). Asserts the registry seed and catalog-kind + backfill (`0010` + `0013`, including that `0013`'s `UPDATE` actually lifts + the official registry off the `generic` column default), the `kind` / + `auth_kind` / `source` / `registered_via` CHECK sets, marketplace + provenance defaults with `ON DELETE SET NULL` detaching an imported server + from a deleted catalog, the `0014` partial unique index on top-level MCP + grants (and that it leaves native grants alone), and the `0015`/`0016` + OAuth surface — authorize-time endpoint pinning plus token/flow cascade on + server delete. +- A second suite covers what the CI gate structurally cannot: the CI + idempotency check re-applies against an **empty** database, so it can never + catch a migration that only breaks once rows exist. That suite re-applies + all 30 files with MCP rows in place, in its own throwaway database — + re-running `0001`/`0003` drops and recreates the NOTIFY triggers, which + must not happen underneath a concurrently running suite. +- Both suites skip when no test Postgres is reachable, and scope every row + they write to a `w04-mcp-` tenant prefix, matching the existing pg-suite + convention. Their pools are capped: the runner executes test files + concurrently and ~16 other pg suites each hold a default-sized (max 10) + pool, so an uncapped extra pool in one file exhausts `max_connections` and + cancels an unrelated suite mid-run. + ### Added — plugin-contributed navigation (#470, phase 1 of the Dev Platform extraction) - New plugin capability `ctx.uiRoutes.registerNav({ navId, href, cluster?, diff --git a/middleware/test/mcpRegistrySchema.pg.test.ts b/middleware/test/mcpRegistrySchema.pg.test.ts new file mode 100644 index 00000000..f5725c20 --- /dev/null +++ b/middleware/test/mcpRegistrySchema.pg.test.ts @@ -0,0 +1,387 @@ +import { strict as assert } from 'node:assert'; +import { readFile, readdir } from 'node:fs/promises'; +import { dirname, join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { after, before, describe, it } from 'node:test'; + +import { Pool } from 'pg'; + +import { runMultiOrchestratorMigrations } from '@omadia/orchestrator'; + +/** + * PG-gated coverage for the MCP registry / OAuth schema in + * `middleware/migrations` (0010 registries, 0013 registry kinds, 0014 the + * top-level grant unique index, 0015/0016 OAuth 2.1 + PKCE). + * + * Context (W0-4): `middleware/migrations` was absent from the CI `schema` + * job's MIGRATION_DOMAINS, so every MCP migration shipped without ever being + * applied or idempotency-checked in CI, and no pg test covered MCP at all. + * Adding the domain closes the apply/re-apply gap; this file closes the + * behavioural gap and adds the one check CI structurally cannot make — the + * CI gate re-applies against an EMPTY database, which cannot catch a + * migration that is only non-idempotent once rows exist. + * + * Isolation: every row this suite writes carries the `w04-mcp-` tenant + * prefix so it cannot collide with the other pg suites sharing the database. + * The destructive re-apply check runs in its own scratch database for the + * same reason — re-running 0001/0003 drops and recreates NOTIFY triggers, + * which must never happen underneath a concurrently running suite. + * Skips when no test Postgres is reachable, mirroring the other pg tests. + */ +const PG_URL = + process.env['GRAPH_PG_TEST_URL'] ?? + process.env['MEMORY_PG_TEST_URL'] ?? + process.env['WS5_PG_TEST_URL'] ?? + 'postgres://test:test@127.0.0.1:55438/test'; + +/** Tenant prefix — unique to this suite, see the isolation note above. */ +const TENANT = 'w04-mcp-'; +const SCRATCH_DB = 'w04_mcp_schema_scratch'; + +const migrationsDir = resolve(dirname(fileURLToPath(import.meta.url)), '..', 'migrations'); + +/** + * Pools here are deliberately capped. The suite is fully sequential, but the + * test runner executes files concurrently and ~16 other pg suites each hold a + * default-sized (max 10) pool — an uncapped third pool in this file is enough + * to exhaust `max_connections` and cancel an unrelated suite mid-flight. + */ +const POOL = { connectionTimeoutMillis: 2000, max: 2, idleTimeoutMillis: 1000 } as const; + +const probePool = new Pool({ connectionString: PG_URL, ...POOL }); +let pgAvailable = true; +try { + await probePool.query('SELECT 1'); +} catch { + pgAvailable = false; + await probePool.end().catch(() => undefined); +} + +async function migrationFiles(): Promise { + return (await readdir(migrationsDir)).filter((f) => f.endsWith('.sql')).sort(); +} + +/** Postgres error code for a statement that violated a constraint we assert on. */ +async function expectRejected(pool: Pool, sql: string, params: readonly unknown[] = []) { + try { + await pool.query(sql, [...params]); + } catch (err: unknown) { + return err as { code?: string }; + } + assert.fail(`expected the statement to be rejected: ${sql}`); +} + +describe('MCP registry + OAuth schema (pg)', { skip: !pgAvailable }, () => { + const pool = probePool; + + async function cleanup(): Promise { + // mcp_servers cascades to oauth tokens/flows and to agent_tool_grants; + // agents cascades to its grants. Registries are referenced by + // mcp_servers.registry_id with ON DELETE SET NULL, so order matters only + // for readability here. + await pool.query('DELETE FROM mcp_servers WHERE name LIKE $1', [`${TENANT}%`]); + await pool.query('DELETE FROM agents WHERE slug LIKE $1', [`${TENANT}%`]); + await pool.query('DELETE FROM mcp_registries WHERE name LIKE $1', [`${TENANT}%`]); + await pool.query('DELETE FROM mcp_oauth_clients WHERE issuer LIKE $1', [`${TENANT}%`]); + } + + before(async () => { + await runMultiOrchestratorMigrations(pool, undefined, migrationsDir); + await cleanup(); + }); + + after(async () => { + await cleanup(); + await pool.end(); + }); + + it('seeds the official and smithery registries with their catalog kinds (0010 + 0013)', async () => { + const { rows } = await pool.query<{ name: string; kind: string; auth_kind: string }>( + `SELECT name, kind, auth_kind FROM mcp_registries + WHERE name IN ('official', 'smithery') ORDER BY name`, + ); + + // 0010 seeds `official` before 0013 adds `kind`; 0013's UPDATE is what + // lifts it off the 'generic' column default. A silent no-op there would + // leave the official registry using the wrong catalog normalizer. + assert.deepEqual( + rows.map((r) => [r.name, r.kind, r.auth_kind]), + [ + ['official', 'official', 'none'], + ['smithery', 'smithery', 'none'], + ], + ); + }); + + it('constrains registry kind and auth_kind to the known sets (0010 + 0013)', async () => { + const badKind = await expectRejected( + pool, + `INSERT INTO mcp_registries (name, url, auth_kind, kind) + VALUES ($1, 'https://reg.invalid', 'none', 'not-a-kind')`, + [`${TENANT}bad-kind`], + ); + assert.equal(badKind.code, '23514', 'unknown registry kind must fail the CHECK'); + + const badAuth = await expectRejected( + pool, + `INSERT INTO mcp_registries (name, url, auth_kind) + VALUES ($1, 'https://reg.invalid', 'basic')`, + [`${TENANT}bad-auth`], + ); + assert.equal(badAuth.code, '23514', 'unknown registry auth_kind must fail the CHECK'); + }); + + it('defaults marketplace provenance to manual and detaches on registry delete (0010)', async () => { + const registry = await pool.query<{ id: string }>( + `INSERT INTO mcp_registries (name, url, auth_kind, kind) + VALUES ($1, 'https://reg.invalid', 'none', 'generic') RETURNING id`, + [`${TENANT}registry`], + ); + const registryId = registry.rows[0]!.id; + + const server = await pool.query<{ id: string; source: string; registry_id: string | null }>( + `INSERT INTO mcp_servers (name, transport, endpoint) + VALUES ($1, 'http', 'https://srv.invalid/mcp') + RETURNING id, source, registry_id`, + [`${TENANT}server`], + ); + assert.equal(server.rows[0]!.source, 'manual', 'pre-marketplace rows are implicitly manual'); + assert.equal(server.rows[0]!.registry_id, null); + + const badSource = await expectRejected( + pool, + `UPDATE mcp_servers SET source = 'somewhere-else' WHERE id = $1`, + [server.rows[0]!.id], + ); + assert.equal(badSource.code, '23514', 'source is restricted to manual|marketplace'); + + await pool.query(`UPDATE mcp_servers SET source = 'marketplace', registry_id = $1 WHERE id = $2`, [ + registryId, + server.rows[0]!.id, + ]); + + // Deleting a catalog source must orphan the imported server, not delete it. + await pool.query('DELETE FROM mcp_registries WHERE id = $1', [registryId]); + const after = await pool.query<{ source: string; registry_id: string | null }>( + 'SELECT source, registry_id FROM mcp_servers WHERE id = $1', + [server.rows[0]!.id], + ); + assert.equal(after.rows[0]!.registry_id, null, 'ON DELETE SET NULL must keep the server row'); + assert.equal(after.rows[0]!.source, 'marketplace'); + }); + + it('rejects a duplicate top-level MCP grant but not the sub-agent equivalent (0014)', async () => { + const agent = await pool.query<{ id: string }>( + `INSERT INTO agents (slug, name) VALUES ($1, 'W0-4 MCP Agent') RETURNING id`, + [`${TENANT}agent`], + ); + const server = await pool.query<{ id: string }>( + `INSERT INTO mcp_servers (name, transport, endpoint) + VALUES ($1, 'http', 'https://srv.invalid/mcp') RETURNING id`, + [`${TENANT}grant-server`], + ); + const agentId = agent.rows[0]!.id; + const serverId = server.rows[0]!.id; + const toolRef = `${TENANT}grant-server:ping`; + + const insertTopLevel = `INSERT INTO agent_tool_grants (agent_id, tool_kind, tool_ref, mcp_server_id) + VALUES ($1, 'mcp', $2, $3)`; + await pool.query(insertTopLevel, [agentId, toolRef, serverId]); + + const dup = await expectRejected(pool, insertTopLevel, [agentId, toolRef, serverId]); + assert.equal(dup.code, '23505', 'the partial unique index must block a repeat grant'); + + // The index is scoped to `agent_id IS NOT NULL AND tool_kind = 'mcp'`: + // a native grant on the same agent must stay unaffected. + await pool.query( + `INSERT INTO agent_tool_grants (agent_id, tool_kind, tool_ref) VALUES ($1, 'native', $2)`, + [agentId, toolRef], + ); + await pool.query( + `INSERT INTO agent_tool_grants (agent_id, tool_kind, tool_ref) VALUES ($1, 'native', $2)`, + [agentId, toolRef], + ); + + const { rows } = await pool.query<{ count: string }>( + 'SELECT count(*)::text AS count FROM agent_tool_grants WHERE agent_id = $1', + [agentId], + ); + assert.equal(rows[0]!.count, '3', 'one mcp grant plus two unconstrained native grants'); + }); + + it('binds an OAuth flow to its authorize-time endpoints and cascades on server delete (0015 + 0016)', async () => { + const server = await pool.query<{ id: string }>( + `INSERT INTO mcp_servers (name, transport, endpoint) + VALUES ($1, 'http', 'https://oauth.invalid/mcp') RETURNING id`, + [`${TENANT}oauth-server`], + ); + const serverId = server.rows[0]!.id; + const issuer = `${TENANT}https://issuer.invalid`; + + await pool.query( + `INSERT INTO mcp_oauth_clients (issuer, client_id, registered_via) VALUES ($1, 'cid', 'dcr')`, + [issuer], + ); + const badRegistration = await expectRejected( + pool, + `INSERT INTO mcp_oauth_clients (issuer, client_id, registered_via) VALUES ($1, 'cid', 'guessed')`, + [`${TENANT}https://issuer2.invalid`], + ); + assert.equal(badRegistration.code, '23514', 'registered_via is restricted to dcr|manual'); + + await pool.query( + `INSERT INTO mcp_oauth_tokens (server_id, user_key, access_token_ref) + VALUES ($1, $2, 'vault://access')`, + [serverId, `${TENANT}user`], + ); + // 0016: the token endpoint is persisted at authorize time so the callback + // cannot be redirected to a re-discovered (attacker-swapped) endpoint. + await pool.query( + `INSERT INTO mcp_oauth_flows + (state, server_id, user_key, issuer, code_verifier, redirect_uri, + token_endpoint, authorization_endpoint) + VALUES ($1, $2, $3, $4, 'verifier', 'https://cb.invalid', + 'https://issuer.invalid/token', 'https://issuer.invalid/authorize')`, + [`${TENANT}state`, serverId, `${TENANT}user`, issuer], + ); + + const flow = await pool.query<{ token_endpoint: string; authorization_endpoint: string }>( + 'SELECT token_endpoint, authorization_endpoint FROM mcp_oauth_flows WHERE state = $1', + [`${TENANT}state`], + ); + assert.equal(flow.rows[0]!.token_endpoint, 'https://issuer.invalid/token'); + assert.equal(flow.rows[0]!.authorization_endpoint, 'https://issuer.invalid/authorize'); + + // Deleting the server must not leave live credentials or pending flows behind. + await pool.query('DELETE FROM mcp_servers WHERE id = $1', [serverId]); + const tokens = await pool.query('SELECT 1 FROM mcp_oauth_tokens WHERE server_id = $1', [serverId]); + const flows = await pool.query('SELECT 1 FROM mcp_oauth_flows WHERE server_id = $1', [serverId]); + assert.equal(tokens.rowCount, 0, 'tokens must cascade with the server'); + assert.equal(flows.rowCount, 0, 'pending flows must cascade with the server'); + }); +}); + +describe('middleware/migrations idempotency under data (pg)', { skip: !pgAvailable }, () => { + // Runs in a throwaway database so the destructive re-apply (0001/0003 drop + // and recreate the NOTIFY triggers) cannot disturb a concurrently running + // pg suite on the shared test database. + const adminUrl = new URL(PG_URL); + adminUrl.pathname = '/postgres'; + const scratchUrl = new URL(PG_URL); + scratchUrl.pathname = `/${SCRATCH_DB}`; + + let scratchPool: Pool | undefined; + let scratchReady = false; + + /** + * Admin connections are opened per operation and closed immediately — + * holding one open for the suite's lifetime is exactly the connection + * pressure the POOL cap above exists to avoid. + */ + async function withAdmin(fn: (pool: Pool) => Promise): Promise { + const admin = new Pool({ connectionString: adminUrl.toString(), ...POOL, max: 1 }); + try { + return await fn(admin); + } finally { + await admin.end().catch(() => undefined); + } + } + + before(async () => { + try { + await withAdmin(async (admin) => { + // DROP/CREATE DATABASE cannot run inside a transaction block, so these + // must stay separate statements. + await admin.query(`DROP DATABASE IF EXISTS ${SCRATCH_DB} WITH (FORCE)`); + await admin.query(`CREATE DATABASE ${SCRATCH_DB}`); + }); + scratchReady = true; + } catch { + // No CREATEDB privilege (or no `postgres` database) — skip rather than + // fail, matching how the pg suites degrade when Postgres is absent. + scratchReady = false; + return; + } + scratchPool = new Pool({ connectionString: scratchUrl.toString(), ...POOL, max: 1 }); + }); + + after(async () => { + await scratchPool?.end().catch(() => undefined); + if (scratchReady) { + await withAdmin((admin) => + admin.query(`DROP DATABASE IF EXISTS ${SCRATCH_DB} WITH (FORCE)`), + ).catch(() => undefined); + } + }); + + it('re-applies every migration cleanly with rows present', async (t) => { + if (!scratchReady || !scratchPool) { + t.skip('no CREATEDB privilege on the test Postgres'); + return; + } + const pool = scratchPool; + const files = await migrationFiles(); + assert.ok(files.length > 0, 'expected migrations to be discovered'); + + // Pass 1 — virgin database. `middleware/migrations` needs no extensions + // (gen_random_uuid is core since pg13) and has no cross-domain FKs, which + // is why this domain can be applied standalone. + for (const file of files) { + await pool.query(await readFile(join(migrationsDir, file), 'utf8')); + } + + // Seed the MCP + dev-platform surfaces so the re-apply runs against real + // rows — the case the CI gate cannot reach, since it re-applies empty. + const server = await pool.query<{ id: string }>( + `INSERT INTO mcp_servers (name, transport, endpoint) + VALUES ($1, 'http', 'https://srv.invalid/mcp') RETURNING id`, + [`${TENANT}scratch-server`], + ); + const agent = await pool.query<{ id: string }>( + `INSERT INTO agents (slug, name) VALUES ($1, 'W0-4 Scratch') RETURNING id`, + [`${TENANT}scratch-agent`], + ); + await pool.query( + `INSERT INTO mcp_registries (name, url, auth_kind, kind) + VALUES ($1, 'https://reg.invalid', 'none', 'generic')`, + [`${TENANT}scratch-registry`], + ); + await pool.query( + `INSERT INTO agent_tool_grants (agent_id, tool_kind, tool_ref, mcp_server_id) + VALUES ($1, 'mcp', $2, $3)`, + [agent.rows[0]!.id, `${TENANT}scratch-server:ping`, server.rows[0]!.id], + ); + await pool.query( + `INSERT INTO mcp_oauth_clients (issuer, client_id, registered_via) + VALUES ($1, 'cid', 'manual')`, + [`${TENANT}https://scratch-issuer.invalid`], + ); + await pool.query( + `INSERT INTO mcp_oauth_tokens (server_id, user_key, access_token_ref) + VALUES ($1, $2, 'vault://access')`, + [server.rows[0]!.id, `${TENANT}user`], + ); + + // Pass 2 — the CI idempotency gate, but with the rows above in place. + for (const file of files) { + await pool.query(await readFile(join(migrationsDir, file), 'utf8')); + } + + // The seeded rows must survive, and the seed INSERTs in 0010/0013 must not + // have duplicated their registries. + const registries = await pool.query<{ name: string }>( + `SELECT name FROM mcp_registries WHERE name IN ('official', 'smithery')`, + ); + assert.equal(registries.rowCount, 2, 'ON CONFLICT DO NOTHING keeps the seed rows unique'); + + const grants = await pool.query('SELECT 1 FROM agent_tool_grants WHERE agent_id = $1', [ + agent.rows[0]!.id, + ]); + assert.equal(grants.rowCount, 1, 're-applying must not drop or duplicate existing grants'); + + const tokens = await pool.query('SELECT 1 FROM mcp_oauth_tokens WHERE server_id = $1', [ + server.rows[0]!.id, + ]); + assert.equal(tokens.rowCount, 1, 're-applying must not disturb stored OAuth token refs'); + }); +}); From 1704d4fe4afeede69feca04816f877d9a5247bd3 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 09:15:37 +0200 Subject: [PATCH 03/90] feat(mcp): preserve structuredContent via an out-of-band sidecar and capture outputSchema MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Issue #547 (W1-3) — plumbing only, no canvas synthesis. Discovery now keeps a tool's declared outputSchema: McpToolDescriptor and McpDiscoveredTool gained an optional outputSchema, and listTools copies it from tools/list (object-valued only; anything else is dropped). It rides along in the existing mcp_servers.discovered_tools jsonb column, so it survives a restart without re-discovery and needs no migration. subAgentToolHydration rehydrates it on the way back out and seeds the manager's cache, since mcpNativeHandler only closes over a tool name. structuredContent is no longer discarded. A new extractStructured() reads it and McpManager hands it to an optional McpManagerOptions.structuredSink as { kind: 'structured_output', serverId, toolName, turnId, structured, outputSchema? }. Error results and absent/null payloads emit nothing. This is deliberately out-of-band rather than a widened return type. callTool() still returns Promise and NativeToolHandler is untouched, which keeps the published plugin contract stable and keeps every MCP result on the 'typeof result === string' path that gates Privacy Shield masking in the orchestrator — a non-string result would bypass the shield. The payload union is a discriminated 'kind' so #544 (MRTR) can add 'input_required' without another refactor. renderToolResult is byte-for-byte unchanged and is now pinned by a golden suite (text-only, mixed blocks, structuredContent-only, empty content, array-valued structuredContent, isError, whitespace fallback, nullish). A mutation check installs a hostile sink that rewrites and deep-mutates its payload and returns a different object, then asserts the LLM-bound string is unchanged; verified to fail against a deliberately in-band mutant. Operator surface: a read-only 'returns structured output' badge in the MCP Control Center, with en + de strings. --- docs/CHANGELOG.md | 29 ++ .../harness-orchestrator/src/index.ts | 7 + .../harness-orchestrator/src/mcp/mcpClient.ts | 175 ++++++- .../src/registry/agentGraphStore.ts | 7 + .../packages/plugin-api/src/agentGraph.ts | 4 + .../src/agents/subAgentToolHydration.ts | 22 +- middleware/test/mcpStructuredContent.test.ts | 451 ++++++++++++++++++ middleware/test/skillToolBindings.test.ts | 2 + .../subAgentToolHydrationTopLevel.test.ts | 8 +- web-ui/app/_lib/agentBuilder.ts | 4 + web-ui/app/admin/mcp/page.tsx | 11 + web-ui/messages/de.json | 2 + web-ui/messages/en.json | 2 + 13 files changed, 717 insertions(+), 7 deletions(-) create mode 100644 middleware/test/mcpStructuredContent.test.ts diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 0791c3ea..6803fb71 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -18,6 +18,35 @@ entry. See `CONTRIBUTING.md` § Releases & changelog. ## [Unreleased] +### Added — MCP structured-content sidecar and `outputSchema` capture (#547, W1-3) + +- Discovery now keeps a tool's declared `outputSchema`. `McpToolDescriptor` + and `McpDiscoveredTool` gained an optional `outputSchema` field, and + `McpManager.listTools()` copies it from `tools/list` (object-valued only; + anything else is dropped rather than propagated). It is persisted with the + rest of the descriptor in the existing `mcp_servers.discovered_tools` + `jsonb` column, so it survives a restart without re-discovery — **no + migration required**. `subAgentToolHydration` rehydrates it on the way back + out. +- `structuredContent` returned by an MCP tool is no longer discarded. A new + `extractStructured(res)` reads it, and `McpManager` hands it to an optional + `McpManagerOptions.structuredSink` as `{ kind: 'structured_output', + serverId, toolName, turnId, structured, outputSchema? }`, keyed so a + consumer can correlate it with the turn that produced it. Error results and + absent/null payloads emit nothing. +- This is deliberately an **out-of-band** channel, not a widened return type. + `McpManager.callTool()` still returns `Promise` and + `NativeToolHandler` is untouched, which keeps the published plugin contract + stable and — more importantly — keeps every MCP result on the + `typeof result === 'string'` path that gates Privacy Shield masking in the + orchestrator. A non-string result would silently bypass the shield. +- Operator surface: the MCP Control Center's tool list shows a read-only + "returns structured output" badge for any tool that declares an output + schema. +- No canvas/synthesis behaviour is attached yet — this change is plumbing + only. The sink's payload union is a discriminated `kind` so the MRTR work + (#544) can add `input_required` without another refactor. + ### Added — plugin-contributed navigation (#470, phase 1 of the Dev Platform extraction) - New plugin capability `ctx.uiRoutes.registerNav({ navId, href, cluster?, diff --git a/middleware/packages/harness-orchestrator/src/index.ts b/middleware/packages/harness-orchestrator/src/index.ts index 75acc579..a635aa9c 100644 --- a/middleware/packages/harness-orchestrator/src/index.ts +++ b/middleware/packages/harness-orchestrator/src/index.ts @@ -126,10 +126,12 @@ export type { } from './registry/agentGraphStore.js'; export { McpManager, + extractStructured, mcpNativeHandler, mcpNativeToolName, mcpToolToLocalSubAgentTool, mcpToolToNativeSpec, + renderToolResult, } from './mcp/mcpClient.js'; export type { McpAuthProvider, @@ -139,6 +141,11 @@ export type { McpCallObserver, McpManagerOptions, McpServerConfig, + McpSidecarIdentity, + McpSidecarKind, + McpSidecarPayload, + McpStructuredOutputSidecar, + McpStructuredSink, McpToolDescriptor, McpTransportKind, } from './mcp/mcpClient.js'; diff --git a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts index d3ed9d0c..2c05a117 100644 --- a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts +++ b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts @@ -43,13 +43,16 @@ import { turnContext } from '../turnContext.js'; * an array. The SDK's strict schema rejects the entire result, and callTool then * throws — which we surface as "-32000 Connection closed", making every call on * that server look like a dead connection. Accepting any `structuredContent` - * keeps well-formed servers unchanged while tolerating this one deviation; we - * only read `content`/`isError` downstream anyway. + * keeps well-formed servers unchanged while tolerating this one deviation. + * + * Issue #547 (W1-3): `structuredContent` is now also read out-of-band via + * `extractStructured` and handed to `McpManagerOptions.structuredSink`. The + * lenient schema is what makes that possible for off-spec (array-valued) + * payloads too — the sink carries whatever the server sent, unnormalised. */ // Cast back to the base schema type: the SDK's callTool overload is typed to the // strict CallToolResultSchema, but our runtime schema only *widens* what parses -// (any structuredContent), so it is a safe superset. We never read -// structuredContent downstream — only `content`/`isError`. +// (any structuredContent), so it is a safe superset. const LENIENT_CALL_TOOL_RESULT_SCHEMA = CallToolResultSchema.extend({ structuredContent: z.unknown().optional(), }) as unknown as typeof CallToolResultSchema; @@ -76,6 +79,14 @@ export interface McpToolDescriptor { readonly name: string; readonly description?: string; readonly inputSchema?: Record; + /** + * Issue #547 (W1-3) — the tool's declared `outputSchema` from `tools/list`. + * Never sent to the model (it would only inflate the prompt); it travels with + * the structured-result sidecar so a downstream consumer can render the + * payload against its declared shape. Persisted with the discovered-tool row + * so it survives a restart without re-discovery. + */ + readonly outputSchema?: Record; } /** Caller taxonomy for the MCP call audit log (epic #459 W2, issue #462). @@ -143,10 +154,61 @@ export interface McpAuthProvider { getConfigEnv?(cfg: McpServerConfig): Promise>; } +// ── out-of-band sidecar (issue #547 W1-3) ─────────────────────────────────── +// +// Everything the model sees still travels as the plain string `callTool` +// returns. Anything richer — an MCP `structuredContent` payload today, an +// `input_required` result type tomorrow (#544 MRTR / W2-1) — leaves the manager +// through this second, out-of-band channel instead of widening the return type. +// +// Widening was ruled out deliberately, for two reasons that are not stylistic: +// 1. `NativeToolHandler = (input: unknown) => Promise` is a published +// plugin contract; every in-tree and out-of-tree plugin implements it. +// 2. The orchestrator gates Privacy Shield masking on +// `typeof result === 'string'`. A non-string result would silently skip +// masking — i.e. bypass the shield entirely. +// The sidecar keeps both invariants intact: no downstream hop changes. + +/** Discriminator for a sidecar payload. W2-1 adds `'input_required'` here. */ +export type McpSidecarKind = 'structured_output'; + +/** Identity carried by every sidecar payload: which turn, which server, which + * tool. `turnId` is null outside a turn (e.g. an operator test-call). */ +export interface McpSidecarIdentity { + readonly serverId: string; + readonly toolName: string; + readonly turnId: string | null; +} + +/** An MCP tool returned a `structuredContent` payload alongside its text. */ +export interface McpStructuredOutputSidecar extends McpSidecarIdentity { + readonly kind: 'structured_output'; + /** The parsed payload exactly as the server sent it — object, or an array for + * off-spec hosted servers. Never a re-parse of the rendered string. */ + readonly structured: unknown; + /** The tool's declared `outputSchema`, when discovery captured one. */ + readonly outputSchema?: Record; +} + +/** Union of everything the sidecar channel can carry. Add new members here; + * consumers switch on `kind`. */ +export type McpSidecarPayload = McpStructuredOutputSidecar; + +/** + * Out-of-band sink for payloads that must NOT reach the model as text. + * Implementations must be fast and MUST NOT throw; the manager additionally + * guards with try/catch so the sidecar can never break a tool call. Mirrors the + * `onToolCall` audit-observer contract. + */ +export type McpStructuredSink = (payload: McpSidecarPayload) => void; + export interface McpManagerOptions { readonly onToolCall?: McpCallObserver; readonly guard?: McpCallGuard; readonly auth?: McpAuthProvider; + /** Issue #547 (W1-3) — see `McpStructuredSink`. Optional: omitting it leaves + * behaviour byte-identical to before. */ + readonly structuredSink?: McpStructuredSink; } /** True when an error/result string looks like an authorization failure. */ @@ -184,6 +246,11 @@ const CLIENT_INFO = { name: 'omadia-agent-builder', version: '0.1.0' } as const; export class McpManager { private readonly pool = new Map(); private readonly connecting = new Map>(); + /** Issue #547 (W1-3) — declared `outputSchema` per `${serverId} ${tool}`. + * `callTool` only receives a name, so the schema learned at discovery (or + * rehydrated from the persisted descriptor by the adapters below) is cached + * here and attached to the sidecar. A miss just omits the schema. */ + private readonly outputSchemas = new Map>(); /** Optional audit observer + dispatch guard (issues #462/#454). Existing * `new McpManager()` call sites keep working unchanged. */ @@ -230,6 +297,42 @@ export class McpManager { } } + /** + * Issue #547 (W1-3) — remember a tool's declared `outputSchema` so a later + * `callTool` (which only gets a name) can attach it to the sidecar. Called + * automatically by `listTools`, and by the adapter factories below so a + * descriptor rehydrated from the DB after a restart is just as good as a + * freshly discovered one. Idempotent; a schema-less descriptor is a no-op. + */ + rememberToolSchema(serverId: string, tool: McpToolDescriptor): void { + if (!tool.outputSchema) return; + this.outputSchemas.set(schemaKey(serverId, tool.name), tool.outputSchema); + } + + /** Emit one structured-result sidecar. Out-of-band by construction: the + * caller has already produced the model-facing string and ignores this. */ + private emitStructured( + cfg: McpServerConfig, + toolName: string, + structured: unknown, + ): void { + if (!this.options?.structuredSink) return; + try { + const ctx = turnContext.current(); + const outputSchema = this.outputSchemas.get(schemaKey(cfg.id, toolName)); + this.options.structuredSink({ + kind: 'structured_output', + serverId: cfg.id, + toolName, + turnId: ctx !== undefined && ctx.turnId !== '' ? ctx.turnId : null, + structured, + ...(outputSchema ? { outputSchema } : {}), + }); + } catch { + /* the sidecar must never break a tool call */ + } + } + /** Discover the tool list a server exposes. Throws on connection failure so * the operator-facing `/discover` endpoint can report it. */ async listTools(cfg: McpServerConfig): Promise { @@ -248,13 +351,22 @@ export class McpManager { const { client } = await this.getOrConnect(await this.withResolvedConfig(cfg), token); const res = await client.listTools(); const tools = Array.isArray(res?.tools) ? res.tools : []; - return tools.map((t) => ({ + const descriptors = tools.map((t) => ({ name: String(t.name), ...(t.description ? { description: String(t.description) } : {}), ...(t.inputSchema ? { inputSchema: t.inputSchema as Record } : {}), + // Issue #547 (W1-3): carry the declared output schema through discovery + // so it can be persisted and later attached to the sidecar. Object-only — + // a server that sends a non-object here gets it dropped rather than + // poisoning the descriptor (arrays are objects in JS, so exclude them). + ...(isPlainObject(t.outputSchema) + ? { outputSchema: t.outputSchema as Record } + : {}), })); + for (const d of descriptors) this.rememberToolSchema(cfg.id, d); + return descriptors; } /** Invoke a tool. Never throws — returns an `Error: …` string on failure so @@ -334,6 +446,14 @@ export class McpManager { return this.handleFailure(cfg, toolName, token, rendered, startedAt); } this.emitCall(cfg, toolName, true, null, startedAt); + // Issue #547 (W1-3) — hand any `structuredContent` to the out-of-band + // sink. `rendered` above is already final and is NOT re-derived from + // this: the model-facing string is byte-identical with or without a + // sink installed. Error results are skipped by `extractStructured`. + const structured = extractStructured(res); + if (structured !== undefined) { + this.emitStructured(cfg, toolName, structured); + } return rendered; } catch (err) { // Drop the connection so the next call reconnects (server may have died). @@ -565,6 +685,10 @@ export function mcpToolToLocalSubAgentTool( cfg: McpServerConfig, tool: McpToolDescriptor, ): LocalSubAgentTool { + // Issue #547 (W1-3): seed the schema cache from the (possibly DB-rehydrated) + // descriptor so the sidecar carries an outputSchema even when this process + // never ran discovery for this server. + manager.rememberToolSchema(cfg.id, tool); return { spec: { name: mcpNativeToolName(cfg.name, tool.name), @@ -609,6 +733,47 @@ export function renderToolResult(res: any): string { return JSON.stringify(res ?? {}); } +/** + * Issue #547 (W1-3) — pull an MCP result's `structuredContent` out for the + * out-of-band sidecar. Deliberately a SEPARATE function from + * `renderToolResult`, which stays byte-for-byte unchanged: the model-facing + * string must not shift because a sink is installed. + * + * Returns the payload exactly as the server sent it (object, or array for + * off-spec hosted servers — see `LENIENT_CALL_TOOL_RESULT_SCHEMA`), never a + * re-parse of the rendered string. + * + * Returns `undefined` for: + * - a non-object / protocol-error result (nothing trustworthy to read), + * - `isError: true` (a failed call has no result to render structurally), + * - an absent `structuredContent`, + * - an explicit `null` — off-spec (the spec requires an object) and carries + * nothing to render, so it is folded into "absent" rather than emitting an + * empty sidecar. Keeps the sink contract simple: a payload arrives only + * when there is genuinely something in it. + */ +// eslint-disable-next-line @typescript-eslint/no-explicit-any +export function extractStructured(res: any): unknown | undefined { + if (res === null || typeof res !== 'object') return undefined; + if (res.isError === true) return undefined; + const structured = res.structuredContent; + if (structured === undefined || structured === null) return undefined; + return structured; +} + +/** Cache key for a per-server tool schema — same shape as the verdict maps in + * `agentBuilder.ts`. `serverId` is a UUID and so contains no space, which makes + * the FIRST space the unambiguous separator no matter what the tool name + * contains; no collision is possible. */ +function schemaKey(serverId: string, toolName: string): string { + return `${serverId} ${toolName}`; +} + +/** True for a non-null, non-array object. */ +function isPlainObject(v: unknown): v is Record { + return typeof v === 'object' && v !== null && !Array.isArray(v); +} + /** Split a shell command line into argv. Honours simple double/single quotes; * not a full shell parser, but enough for `npx -y @scope/pkg --flag "v"`. */ export function splitCommand(line: string): string[] { diff --git a/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts b/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts index 4ea2f5bd..dae4f2d2 100644 --- a/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts +++ b/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts @@ -2148,6 +2148,13 @@ export class AgentGraphStore { } } + /** + * Persist the discovered-tool descriptors verbatim. `discovered_tools` is a + * `jsonb` column and the descriptor is stored whole, so every field a + * `McpToolDescriptor` carries round-trips — including the `outputSchema` + * added in issue #547 (W1-3). No migration is needed to add descriptor + * fields; only the TypeScript shape changes. + */ async setMcpDiscoveredTools( id: string, tools: readonly unknown[], diff --git a/middleware/packages/plugin-api/src/agentGraph.ts b/middleware/packages/plugin-api/src/agentGraph.ts index 08c1f4d9..8d0a52fe 100644 --- a/middleware/packages/plugin-api/src/agentGraph.ts +++ b/middleware/packages/plugin-api/src/agentGraph.ts @@ -106,6 +106,10 @@ export interface McpDiscoveredTool { readonly name: string; readonly description?: string; readonly inputSchema?: Record; + /** Issue #547 (W1-3) — the tool's declared `outputSchema`, captured at + * discovery. Persisted alongside the rest of the descriptor so it survives + * a restart without re-discovery. Never shown to the model. */ + readonly outputSchema?: Record; } export interface McpServerNode { diff --git a/middleware/src/agents/subAgentToolHydration.ts b/middleware/src/agents/subAgentToolHydration.ts index 3befd998..b1cdad61 100644 --- a/middleware/src/agents/subAgentToolHydration.ts +++ b/middleware/src/agents/subAgentToolHydration.ts @@ -184,7 +184,12 @@ export function adaptNativeToolForSubAgent( /** Resolve the discovered descriptor for a granted tool from the server row, * so the DomainTool spec carries the real description + inputSchema. Falls * back to a name-only descriptor (schema-less, still callable) when the - * server has not been re-discovered since the grant. */ + * server has not been re-discovered since the grant. + * + * Issue #547 (W1-3): also rehydrates the persisted `outputSchema`. That is + * what makes the schema survive a restart — discovery may not run again for + * the lifetime of the process, and the structured-result sidecar reads the + * schema from the descriptor the adapters seed it with. */ function discoveredDescriptor( row: McpServerRow | undefined, toolRef: string, @@ -198,11 +203,21 @@ function discoveredDescriptor( ...(hit['inputSchema'] && typeof hit['inputSchema'] === 'object' ? { inputSchema: hit['inputSchema'] as Record } : {}), + ...(isPlainObject(hit['outputSchema']) + ? { outputSchema: hit['outputSchema'] } + : {}), }; } return { name: toolRef }; } +/** True for a non-null, non-array object. Arrays are rejected: an + * `outputSchema` is a JSON-Schema object, and a persisted array would be + * corrupt data rather than a lenient variant. */ +function isPlainObject(v: unknown): v is Record { + return typeof v === 'object' && v !== null && !Array.isArray(v); +} + /** * Adapt one top-level MCP tool grant into a per-agent DomainTool (epic #459 * W0, issue #457). Composes the previously-unwired adapters: spec via @@ -217,6 +232,11 @@ export function mcpGrantToDomainTool( cfg: McpServerConfig, descriptor: McpToolDescriptor, ): DomainTool { + // Issue #547 (W1-3): seed the manager's output-schema cache from this + // descriptor. `mcpNativeHandler` only closes over a tool NAME, so without + // this the sidecar would lose the schema on any process that never ran + // discovery itself (i.e. every restart). + manager.rememberToolSchema(cfg.id, descriptor); const spec = mcpToolToNativeSpec(cfg.name, descriptor); const handler = mcpNativeHandler(manager, cfg, descriptor.name); return { diff --git a/middleware/test/mcpStructuredContent.test.ts b/middleware/test/mcpStructuredContent.test.ts new file mode 100644 index 00000000..4b797f87 --- /dev/null +++ b/middleware/test/mcpStructuredContent.test.ts @@ -0,0 +1,451 @@ +/** + * Issue #547 (W1-3) — MCP structured-content sidecar + outputSchema capture. + * + * Three things are locked down here: + * + * 1. A GOLDEN/characterization suite for `renderToolResult`. The model-facing + * string is the contract every downstream hop (Privacy Shield, KG ingest, + * session log, the LLM itself) depends on. These goldens were captured from + * the pre-change implementation; any diff in them is a behaviour change, not + * a test that needs updating. + * 2. `extractStructured` — the new, SEPARATE reader for `structuredContent`. + * 3. The out-of-band sink: installing one must not move a single byte of the + * model-facing string (the mutation check below), and the payload must be + * the parsed object the server sent, not a re-parse of the rendered text. + * + * Lives in its own file (not `mcpClient.test.ts`) to stay merge-conflict-free + * with the parallel unit creating that file. + */ + +import { after, describe, it } from 'node:test'; +import { strict as assert } from 'node:assert'; +import { randomUUID } from 'node:crypto'; +import { createServer, type IncomingMessage, type Server as HttpServer, type ServerResponse } from 'node:http'; +import type { AddressInfo } from 'node:net'; + +import { Server as McpSdkServer } from '@modelcontextprotocol/sdk/server/index.js'; +import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js'; +import { + CallToolRequestSchema, + ListToolsRequestSchema, +} from '@modelcontextprotocol/sdk/types.js'; + +import { + McpManager, + extractStructured, + mcpNativeHandler, + renderToolResult, + turnContext, + type McpServerConfig, + type McpSidecarPayload, + type McpToolDescriptor, +} from '@omadia/orchestrator'; + +// ── 1. renderToolResult golden suite ──────────────────────────────────────── + +/** Captured from the implementation BEFORE the sidecar work. Each entry is + * [label, result, exact expected string]. */ +const RENDER_GOLDENS: ReadonlyArray = [ + ['text-only', { content: [{ type: 'text', text: 'hello world' }] }, 'hello world'], + [ + 'mixed blocks (text + resource + unknown block)', + { + content: [ + { type: 'text', text: 'line one' }, + { type: 'resource', resource: { uri: 'file:///a.txt', text: 'from resource' } }, + { type: 'image', data: 'AAAA', mimeType: 'image/png' }, + ], + }, + 'line one\nfrom resource\n{"type":"image","data":"AAAA","mimeType":"image/png"}', + ], + [ + 'structuredContent-only (no content array)', + { structuredContent: { city: 'Berlin', tempC: 21 } }, + '{"city":"Berlin","tempC":21}', + ], + ['empty content array', { content: [] }, '[]'], + [ + // The content-array branch wins even when structuredContent is present — + // an empty array renders as "[]", NOT as the structured payload. + 'empty content array + structuredContent', + { content: [], structuredContent: { a: 1 } }, + '[]', + ], + [ + // Off-spec: the MCP spec requires an object here, but hosted proxies (e.g. + // the Strava proxy) send arrays. This is exactly why the lenient schema + // exists — the string form must stay stable for those servers too. + 'array-valued structuredContent', + { structuredContent: [{ id: 1 }, { id: 2 }] }, + '[{"id":1},{"id":2}]', + ], + [ + 'content AND structuredContent — content wins', + { content: [{ type: 'text', text: 'Weather: 21C' }], structuredContent: { tempC: 21 } }, + 'Weather: 21C', + ], + ['isError result', { content: [{ type: 'text', text: 'boom' }], isError: true }, 'Error: boom'], + [ + 'whitespace-only text falls back to the raw content JSON', + { content: [{ type: 'text', text: ' ' }] }, + '[{"type":"text","text":" "}]', + ], + ['null result', null, '{}'], + ['undefined result', undefined, '{}'], + ['empty object result', {}, '{}'], +]; + +describe('renderToolResult golden suite (#547 W1-3 characterization)', () => { + for (const [label, res, expected] of RENDER_GOLDENS) { + it(`renders ${label} byte-identically`, () => { + assert.equal(renderToolResult(res), expected); + }); + } + + it('never returns a non-string', () => { + for (const [, res] of RENDER_GOLDENS) { + assert.equal(typeof renderToolResult(res), 'string'); + } + }); +}); + +// ── 2. extractStructured ──────────────────────────────────────────────────── + +describe('extractStructured (#547 W1-3)', () => { + it('returns an object payload as-is', () => { + const payload = { city: 'Berlin', tempC: 21 }; + const out = extractStructured({ content: [], structuredContent: payload }); + assert.deepEqual(out, payload); + }); + + it('preserves object identity — it is the parsed payload, not a re-parse', () => { + const payload = { nested: { deep: true } }; + const out = extractStructured({ structuredContent: payload }); + assert.equal(out, payload); + }); + + it('returns an off-spec array payload unnormalised', () => { + const out = extractStructured({ structuredContent: [{ id: 1 }, { id: 2 }] }); + assert.deepEqual(out, [{ id: 1 }, { id: 2 }]); + assert.ok(Array.isArray(out)); + }); + + it('returns a scalar payload as-is', () => { + assert.equal(extractStructured({ structuredContent: 42 }), 42); + assert.equal(extractStructured({ structuredContent: 'plain' }), 'plain'); + assert.equal(extractStructured({ structuredContent: false }), false); + }); + + it('folds an explicit null payload into "absent"', () => { + assert.equal(extractStructured({ structuredContent: null }), undefined); + }); + + it('returns undefined when structuredContent is absent', () => { + assert.equal(extractStructured({ content: [{ type: 'text', text: 'hi' }] }), undefined); + assert.equal(extractStructured({}), undefined); + }); + + it('returns undefined on an isError result even when a payload is present', () => { + assert.equal( + extractStructured({ content: [], isError: true, structuredContent: { e: 1 } }), + undefined, + ); + }); + + it('returns undefined for non-object results', () => { + assert.equal(extractStructured(null), undefined); + assert.equal(extractStructured(undefined), undefined); + assert.equal(extractStructured('a string'), undefined); + assert.equal(extractStructured(7), undefined); + }); +}); + +// ── 3. listTools outputSchema capture ─────────────────────────────────────── + +const OBJECT_SCHEMA = { + type: 'object', + properties: { tempC: { type: 'number' } }, + required: ['tempC'], +} as const; + +const STUB_CFG: McpServerConfig = { + id: '00000000-0000-4000-8000-00000000f001', + name: 'stub-server', + transport: 'http', + endpoint: 'http://127.0.0.1:9/mcp', +}; + +/** Swap the manager's private connect step for a canned `tools/list` reply. + * Necessary because the SDK's client-side result schema rejects a malformed + * `outputSchema` outright, so the "non-object" case can't be exercised over a + * real wire — but the mapping still has to drop it rather than propagate it. */ +function managerWithToolList(tools: readonly unknown[]): McpManager { + const manager = new McpManager(); + ( + manager as unknown as { + getOrConnect: () => Promise<{ client: { listTools: () => Promise } }>; + } + ).getOrConnect = async () => ({ client: { listTools: async () => ({ tools }) } }); + return manager; +} + +describe('McpManager.listTools outputSchema capture (#547 W1-3)', () => { + it('copies outputSchema when the server declares one', async () => { + const manager = managerWithToolList([ + { name: 'get_weather', description: 'w', inputSchema: { type: 'object' }, outputSchema: OBJECT_SCHEMA }, + ]); + const [tool] = await manager.listTools(STUB_CFG); + assert.deepEqual(tool?.outputSchema, OBJECT_SCHEMA); + // The pre-existing fields must be untouched. + assert.equal(tool?.name, 'get_weather'); + assert.equal(tool?.description, 'w'); + assert.deepEqual(tool?.inputSchema, { type: 'object' }); + }); + + it('omits the key entirely when the server declares no outputSchema', async () => { + const manager = managerWithToolList([{ name: 'ping', inputSchema: { type: 'object' } }]); + const [tool] = await manager.listTools(STUB_CFG); + assert.equal(tool?.outputSchema, undefined); + assert.equal('outputSchema' in (tool as McpToolDescriptor), false); + }); + + it('drops a non-object outputSchema instead of propagating it', async () => { + for (const bad of ['a string', 42, true, null, [{ type: 'object' }]]) { + const manager = managerWithToolList([{ name: 'weird', outputSchema: bad }]); + const [tool] = await manager.listTools(STUB_CFG); + assert.equal(tool?.outputSchema, undefined, `expected ${JSON.stringify(bad)} to be dropped`); + } + }); +}); + +// ── 4. Integration: a real fake MCP server over streamable HTTP ───────────── + +interface FakeServerHandle { + readonly url: string; + close(): Promise; +} + +/** Build one MCP server instance. Stateless mode (`sessionIdGenerator: + * undefined`) + a fresh instance per request, so each test can connect its own + * `McpManager` without tripping "Server already initialized". */ +function buildMcpServerInstance(): McpSdkServer { + const mcp = new McpSdkServer( + { name: 'fake-weather', version: '0.0.0' }, + { capabilities: { tools: {} } }, + ); + mcp.setRequestHandler(ListToolsRequestSchema, async () => ({ + tools: [ + { + name: 'get_weather', + description: 'Current weather.', + inputSchema: { type: 'object' as const, properties: {}, required: [] }, + outputSchema: OBJECT_SCHEMA, + }, + { name: 'plain_note', description: 'Text only.', inputSchema: { type: 'object' as const } }, + ], + })); + mcp.setRequestHandler(CallToolRequestSchema, async (request) => { + if (request.params.name === 'plain_note') { + return { content: [{ type: 'text' as const, text: 'just text' }] }; + } + // BOTH a text block and a structuredContent payload, with different + // contents: a sink payload derived from the rendered string would be + // detectably wrong (the text is prose that is not even valid JSON). + return { + content: [{ type: 'text' as const, text: 'Weather: 21C in Berlin' }], + structuredContent: { tempC: 21, city: 'Berlin' }, + }; + }); + return mcp; +} + +/** Minimal in-process MCP server over streamable HTTP on an ephemeral port. */ +async function startFakeMcpServer(): Promise { + const handle = async (req: IncomingMessage, res: ServerResponse): Promise => { + const mcp = buildMcpServerInstance(); + const transport = new StreamableHTTPServerTransport({ + sessionIdGenerator: undefined, + enableJsonResponse: true, + }); + res.on('close', () => { + void transport.close().catch(() => {}); + void mcp.close().catch(() => {}); + }); + await mcp.connect(transport); + if (req.method === 'POST') { + const chunks: Buffer[] = []; + for await (const c of req) chunks.push(c as Buffer); + const raw = Buffer.concat(chunks).toString('utf8'); + await transport.handleRequest(req, res, raw.length > 0 ? JSON.parse(raw) : undefined); + return; + } + await transport.handleRequest(req, res); + }; + const http: HttpServer = createServer((req, res) => { + void handle(req, res).catch(() => { + if (!res.headersSent) res.writeHead(500); + res.end(); + }); + }); + // The MCP client pools its connections and never closes them, so keep-alive + // sockets would keep `http.close()` (and the test runner's event loop) + // waiting forever. Track and destroy them explicitly on teardown. + const sockets = new Set(); + http.on('connection', (socket) => { + sockets.add(socket); + socket.on('close', () => sockets.delete(socket)); + }); + await new Promise((resolve) => http.listen(0, '127.0.0.1', () => resolve())); + const { port } = http.address() as AddressInfo; + return { + url: `http://127.0.0.1:${port}/mcp`, + async close() { + for (const socket of sockets) socket.destroy(); + sockets.clear(); + await new Promise((resolve) => http.close(() => resolve())); + }, + }; +} + +const fake = await startFakeMcpServer(); +after(() => fake.close()); + +const FAKE_CFG: McpServerConfig = { + id: '00000000-0000-4000-8000-00000000f002', + name: 'fake-weather', + transport: 'http', + endpoint: fake.url, +}; + +describe('structured-content sidecar over a real MCP connection (#547 W1-3)', () => { + it('discovers the declared outputSchema', async () => { + const manager = new McpManager(); + const tools = await manager.listTools(FAKE_CFG); + const weather = tools.find((t) => t.name === 'get_weather'); + const note = tools.find((t) => t.name === 'plain_note'); + assert.deepEqual(weather?.outputSchema, OBJECT_SCHEMA); + assert.equal(note?.outputSchema, undefined); + }); + + it('leaves tool_result content unchanged and hands the parsed object to the sink', async () => { + const seen: McpSidecarPayload[] = []; + const manager = new McpManager({ structuredSink: (p) => seen.push(p) }); + // Discovery first so the sidecar can attach the declared schema. + await manager.listTools(FAKE_CFG); + + const result = await manager.callTool(FAKE_CFG, 'get_weather', {}); + + // The model-facing string is the TEXT block, untouched by the payload. + assert.equal(result, 'Weather: 21C in Berlin'); + assert.equal(typeof result, 'string'); + + assert.equal(seen.length, 1); + const payload = seen[0]!; + assert.equal(payload.kind, 'structured_output'); + assert.equal(payload.serverId, FAKE_CFG.id); + assert.equal(payload.toolName, 'get_weather'); + // Identity, not a re-parse of the rendered string: the rendered string is + // prose that would not parse as JSON at all. + assert.deepEqual(payload.structured, { tempC: 21, city: 'Berlin' }); + assert.throws(() => JSON.parse(result)); + assert.deepEqual(payload.outputSchema, OBJECT_SCHEMA); + }); + + it('carries the turn id when called inside a turn', async () => { + const seen: McpSidecarPayload[] = []; + const manager = new McpManager({ structuredSink: (p) => seen.push(p) }); + await turnContext.run( + { turnId: 'turn-547', turnDate: '2026-07-30', agentSlug: 'main' }, + () => manager.callTool(FAKE_CFG, 'get_weather', {}), + ); + assert.equal(seen[0]?.turnId, 'turn-547'); + }); + + it('emits nothing for a tool that returns no structuredContent', async () => { + const seen: McpSidecarPayload[] = []; + const manager = new McpManager({ structuredSink: (p) => seen.push(p) }); + const result = await manager.callTool(FAKE_CFG, 'plain_note', {}); + assert.equal(result, 'just text'); + assert.equal(seen.length, 0); + }); + + it('emits nothing when the call fails', async () => { + const seen: McpSidecarPayload[] = []; + const manager = new McpManager({ structuredSink: (p) => seen.push(p) }); + const dead: McpServerConfig = { + id: '00000000-0000-4000-8000-00000000f003', + name: 'dead-server', + transport: 'http', + endpoint: 'http://127.0.0.1:9/mcp', + }; + const result = await manager.callTool(dead, 'get_weather', {}); + assert.ok(result.startsWith('Error:')); + assert.equal(seen.length, 0); + }); + + it('survives a throwing sink without affecting the tool call', async () => { + const manager = new McpManager({ + structuredSink: () => { + throw new Error('sink exploded'); + }, + }); + const result = await manager.callTool(FAKE_CFG, 'get_weather', {}); + assert.equal(result, 'Weather: 21C in Berlin'); + }); + + // ── MUTATION CHECK ──────────────────────────────────────────────────────── + // Counting sink invocations proves nothing about isolation. This proves it: + // a hostile sink that rewrites its payload — including the nested object it + // was handed — must not move a byte of the LLM-bound string. + it('MUTATION CHECK: a sink that returns and writes a DIFFERENT object cannot change the LLM-bound message', async () => { + const baseline = await new McpManager().callTool(FAKE_CFG, 'get_weather', {}); + + const hostile = new McpManager({ + structuredSink: ((payload: McpSidecarPayload) => { + // Rewrite every field of the payload we were handed... + const mutable = payload as unknown as Record; + mutable['toolName'] = 'TAMPERED'; + mutable['serverId'] = 'TAMPERED'; + mutable['structured'] = { hijacked: true, tempC: -999 }; + mutable['outputSchema'] = { type: 'object', properties: { hijacked: {} } }; + // ...deep-mutate the nested payload object too, in case anything + // downstream still holds the original reference... + const structured = payload.structured as Record | undefined; + if (structured && typeof structured === 'object') { + structured['city'] = 'TAMPERED'; + structured['tempC'] = -999; + } + // ...and hand back a completely different object as the return value. + return { totally: 'different' }; + }) as unknown as (payload: McpSidecarPayload) => void, + }); + const withHostileSink = await hostile.callTool(FAKE_CFG, 'get_weather', {}); + + assert.equal( + withHostileSink, + baseline, + 'installing a sink changed the model-facing string — the channel is NOT out-of-band', + ); + assert.equal(withHostileSink, 'Weather: 21C in Berlin'); + // A second call through the same (already-tampered-with) manager must be + // just as clean — no state leaked from the sink back into the manager. + assert.equal(await hostile.callTool(FAKE_CFG, 'get_weather', {}), baseline); + }); + + // ── PRIVACY SHIELD ──────────────────────────────────────────────────────── + // orchestrator.dispatchTool gates BOTH `captureRawToolResult` and Privacy + // Shield masking on `typeof result === 'string'`. A non-string result would + // silently skip masking, i.e. bypass the shield. Assert the value that + // actually reaches that branch — the NativeToolHandler's return — is still a + // string with a sink installed. + it('PRIVACY SHIELD: the value reaching the orchestrator masking branch is still a string', async () => { + const seen: McpSidecarPayload[] = []; + const manager = new McpManager({ structuredSink: (p) => seen.push(p) }); + const handler = mcpNativeHandler(manager, FAKE_CFG, 'get_weather'); + const result: unknown = await handler({}); + assert.equal(typeof result, 'string', 'a non-string here bypasses Privacy Shield entirely'); + assert.equal(result, 'Weather: 21C in Berlin'); + // Sanity: the sidecar did fire, so this is not a vacuous pass. + assert.equal(seen.length, 1); + }); +}); diff --git a/middleware/test/skillToolBindings.test.ts b/middleware/test/skillToolBindings.test.ts index d7b79d88..9e9b455d 100644 --- a/middleware/test/skillToolBindings.test.ts +++ b/middleware/test/skillToolBindings.test.ts @@ -88,6 +88,8 @@ function makeDeps( }); return 'ok'; }, + // Issue #547 (W1-3): adapters seed the output-schema cache; no-op here. + rememberToolSchema: () => {}, } as unknown as McpManager, mcpServers: [server()], defaultModel: 'claude-sonnet-4-6', diff --git a/middleware/test/subAgentToolHydrationTopLevel.test.ts b/middleware/test/subAgentToolHydrationTopLevel.test.ts index f1a51bb0..b9c17b28 100644 --- a/middleware/test/subAgentToolHydrationTopLevel.test.ts +++ b/middleware/test/subAgentToolHydrationTopLevel.test.ts @@ -90,6 +90,9 @@ function fakeDeps(calls: Array<{ server: string; tool: string; args: Record {}, } as unknown as McpManager; return { client: {} as unknown as AnthropicClient, @@ -187,7 +190,10 @@ describe('registerDbSubAgentTools: top-level MCP grants (#457)', () => { describe('mcpGrantToDomainTool', () => { it('carries description and input schema from the discovered descriptor', () => { - const manager = { callTool: async () => 'x' } as unknown as McpManager; + const manager = { + callTool: async () => 'x', + rememberToolSchema: () => {}, + } as unknown as McpManager; const tool = mcpGrantToDomainTool( manager, { id: SERVER_ID, name: 'billing', transport: 'http', endpoint: 'http://x' }, diff --git a/web-ui/app/_lib/agentBuilder.ts b/web-ui/app/_lib/agentBuilder.ts index 433a3769..93223975 100644 --- a/web-ui/app/_lib/agentBuilder.ts +++ b/web-ui/app/_lib/agentBuilder.ts @@ -202,6 +202,10 @@ export interface McpDiscoveredTool { name: string; description?: string; inputSchema?: Record; + /** Issue #547 (W1-3) — the tool's declared JSON-Schema for its + * `structuredContent` payload. Present ⇒ the tool returns structured output + * in addition to text. Read-only signal for the operator. */ + outputSchema?: Record; verdict?: McpToolVerdictField; } diff --git a/web-ui/app/admin/mcp/page.tsx b/web-ui/app/admin/mcp/page.tsx index dc1906a9..5c3cecda 100644 --- a/web-ui/app/admin/mcp/page.tsx +++ b/web-ui/app/admin/mcp/page.tsx @@ -768,6 +768,17 @@ function ServerDetail({
{tool.name}
+ {/* Issue #547 (W1-3) — read-only signal: this tool declares an + outputSchema, so it returns a structured payload alongside + its text. No behaviour attached; purely informational. */} + {tool.outputSchema ? ( + + {t('servers.structuredOutput')} + + ) : null} {v?.acked && !v.ackStale ? ( {t('servers.acked')} diff --git a/web-ui/messages/de.json b/web-ui/messages/de.json index 70eaf4af..330348bb 100644 --- a/web-ui/messages/de.json +++ b/web-ui/messages/de.json @@ -1171,6 +1171,8 @@ "filterAll": "Alle Verdicts", "acknowledge": "Bestätigen", "acked": "Bestätigt von {by}", + "structuredOutput": "liefert strukturierte Ausgabe", + "structuredOutputHint": "Dieses Tool deklariert ein Output-Schema und liefert zusätzlich zum Text ein strukturiertes Ergebnis.", "runDeepScan": "Tiefen-Scan ausführen", "why": "Heuristischer Scan hat markiert: {codes}. Das ist ein Signal, kein Beweis, dass der Inhalt sicher ist.", "llmRationale": "Tiefen-Scan-Hinweis: {rationale}", diff --git a/web-ui/messages/en.json b/web-ui/messages/en.json index e6399178..4af76974 100644 --- a/web-ui/messages/en.json +++ b/web-ui/messages/en.json @@ -1171,6 +1171,8 @@ "filterAll": "All verdicts", "acknowledge": "Acknowledge", "acked": "Acknowledged by {by}", + "structuredOutput": "returns structured output", + "structuredOutputHint": "This tool declares an output schema and returns a structured payload alongside its text result.", "runDeepScan": "Run deep scan", "why": "Heuristic scan flagged: {codes}. This is a signal, not proof the content is safe.", "llmRationale": "Deep scan note: {rationale}", From 4ece874b4f9c215e38eeae2dde7d1336f3b585ee Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 09:25:10 +0200 Subject: [PATCH 04/90] fix(mcp-oauth): validate RFC 9207 iss, make delegation explicit, single-flight refresh MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three live security/correctness defects in the MCP OAuth path. D1 — no RFC 9207 `iss` validation. The OAuth callback trusted the `state` parameter alone. `state` proves a response belongs to a flow we started; it does NOT prove which authorization server issued the code, so a malicious or compromised MCP server could steer the callback and have a code minted by one AS redeemed at another. `iss` is now validated against the issuer bound to the flow BEFORE the code is exchanged, so a rejected callback persists nothing — no token row, no vault write. A mismatched `iss`, or an absent one from an AS that advertised `authorization_response_iss_parameter_supported`, is rejected. That advertisement is captured at authorize time in the new `mcp_oauth_flows.iss_required` column rather than re-discovered at the callback, for the same reason migration 0016 pinned the token endpoint: a server that can flip the flag in between would simply opt itself out of the check. D2 — silent 'operator' fallback (confused deputy). Both the operator router and the runtime McpManager resolved the OAuth user key as `… ?? 'operator'`, so a Teams or Telegram turn whose user had no mapped identity reached the customer's MCP server holding the OPERATOR's token. Resolution now goes through the new `services/mcpDelegation.ts` and the new `mcp_servers.delegation` column: `per_user` yields no token when no identity resolves and the turn fails closed through the existing `onAuthFailure` path with an explanation; `service` is the explicit opt-in to one shared identity. The fallback literal is gone from every call site. D3 — refresh race. `getValidAccessToken` permitted N concurrent refreshes per (server, user). Against an AS with rotating refresh tokens the losers get `invalid_grant` and the last writer can persist an already-retired token, silently disconnecting the user. Concurrent callers now share one in-flight promise keyed by (serverId, userKey), cleared in a `finally` so a failed refresh never poisons later attempts. Also: - `mcp_oauth_tokens.issuer` records which AS minted a token; a rotated issuer drops the stored token instead of replaying it against a different server. - `mcp_call_log.acting_identity` records WHOSE authority each call used (`caller_agent` is the orchestrator slug, not the identity). Resolved via a new optional `McpAuthProvider.resolveIdentity`, threaded through `callTool` before the dispatch guard so denied calls are attributed too. An unattributable call is recorded as `unresolved`, never left blank. - OAuth failure logging goes through `services/secretRedaction.ts`: tokens, `code`, and `code_verifier` can no longer reach a log line, including values a provider echoed back that we never minted. The callback's error page is redacted too. - New `PUT /mcp-servers/:id/delegation` plus a delegation control in McpAuthSection, with `adminMcp.auth.delegation*` keys in en.json and de.json. Tests: 40 in test/mcpOAuth.test.ts covering iss present/absent/mismatched/blank and trailing-slash equivalence, fail-closed resolution, issuer rotation, and redaction. The D3 test is mutation-checked — it asserts exactly ONE token-endpoint HTTP request under 8 concurrent callers (verified to report 8 and fail when the in-flight map is removed), not a count of mock invocations. BEHAVIOUR CHANGE (operator-visible): a fail-closed `per_user` default for every row would break installed deployments whose channel users reach MCP servers today BECAUSE of the 'operator' fallback. Migration 0031 is therefore deliberately asymmetric — every EXISTING `mcp_servers` row that already holds a stored operator token is set to `delegation = 'service'`, preserving today's behaviour, and only NEWLY created servers get the safe `per_user` default. Operators must review grandfathered servers and switch the ones that should be per-user. --- docs/CHANGELOG.md | 51 ++ .../0031_mcp_oauth_iss_delegation.sql | 81 +++ .../harness-orchestrator/src/index.ts | 1 + .../harness-orchestrator/src/mcp/mcpClient.ts | 47 +- .../src/registry/agentGraphStore.ts | 82 ++- middleware/src/index.ts | 51 +- middleware/src/routes/agentBuilder.ts | 173 +++++- middleware/src/services/mcpAuthDiscovery.ts | 6 + middleware/src/services/mcpDelegation.ts | 94 +++ middleware/src/services/mcpOAuthService.ts | 157 ++++- middleware/src/services/secretRedaction.ts | 77 +++ middleware/test/mcpOAuth.test.ts | 573 ++++++++++++++++++ web-ui/app/_components/mcp/McpAuthSection.tsx | 48 ++ web-ui/app/_lib/agentBuilder.ts | 22 + web-ui/messages/de.json | 10 +- web-ui/messages/en.json | 10 +- 16 files changed, 1421 insertions(+), 62 deletions(-) create mode 100644 middleware/migrations/0031_mcp_oauth_iss_delegation.sql create mode 100644 middleware/src/services/mcpDelegation.ts create mode 100644 middleware/src/services/secretRedaction.ts diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 0791c3ea..c5e3dc61 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -18,6 +18,57 @@ entry. See `CONTRIBUTING.md` § Releases & changelog. ## [Unreleased] +### Security — MCP OAuth: issuer binding, explicit delegation, refresh race (W0-1) + +Three live defects in the MCP OAuth path, one migration +(`middleware/migrations/0031_mcp_oauth_iss_delegation.sql`). + +- **RFC 9207 `iss` validation at the OAuth callback.** The callback trusted the + `state` parameter alone. `state` proves a response belongs to a flow we + started; it does **not** prove which authorization server issued the code, so + a malicious or compromised MCP server could steer the callback and have a code + minted by one AS redeemed at another. `iss` is now validated against the + issuer bound to the flow **before** the code is exchanged — a mismatch, or an + absent `iss` from an AS that advertised + `authorization_response_iss_parameter_supported`, is rejected and persists + nothing. Whether the AS advertised `iss` is captured at authorize time + (`mcp_oauth_flows.iss_required`), never re-discovered at the callback, for the + same reason migration 0016 pinned the token endpoint. +- **Confused deputy removed.** Both the operator router and the runtime + `McpManager` resolved the OAuth user key as `… ?? 'operator'`. A Teams or + Telegram turn whose user had no mapped identity therefore reached the + customer's MCP server holding the **operator's** token. Resolution is now + explicit per server via the new `mcp_servers.delegation` column: `per_user` + fails closed through the existing `onAuthFailure` path when no identity + resolves, and `service` is the explicit opt-in to one shared identity. The + fallback literal is gone from every call site. +- **Refresh race.** `getValidAccessToken` allowed N concurrent refreshes per + (server, user). Against an AS with rotating refresh tokens the losers get + `invalid_grant` and the last writer can persist an already-retired token, + silently disconnecting the user. Concurrent callers now share one in-flight + refresh, verified by a test that asserts exactly one token-endpoint **HTTP + request** under 8 concurrent callers. +- `mcp_oauth_tokens.issuer` records which AS minted a token, so a rotated issuer + invalidates it instead of replaying it against a different server. +- `mcp_call_log.acting_identity` records **whose** authority each call used + (`caller_agent` is the orchestrator slug, not the identity); an unattributable + call is recorded as `unresolved` rather than left blank. +- OAuth failure logging now goes through a redactor + (`middleware/src/services/secretRedaction.ts`) — tokens, `code`, and + `code_verifier` can no longer reach a log line, including values echoed back + by a provider that we never minted. + +> ⚠️ **Operator-visible behaviour change.** A fail-closed `per_user` default for +> every row would break installed deployments whose channel users reach MCP +> servers today *because of* the `'operator'` fallback. The migration is +> therefore deliberately asymmetric: every **existing** `mcp_servers` row that +> already holds a stored operator token is set to `delegation = 'service'`, +> preserving today's behaviour, and only **newly created** servers get the safe +> `per_user` default. Review each grandfathered server in the MCP Control Center +> and switch the ones that should be per-user — while a server stays on +> `service`, anyone who can reach an orchestrator it is granted to acts with the +> operator's authority at that server. + ### Added — plugin-contributed navigation (#470, phase 1 of the Dev Platform extraction) - New plugin capability `ctx.uiRoutes.registerNav({ navId, href, cluster?, diff --git a/middleware/migrations/0031_mcp_oauth_iss_delegation.sql b/middleware/migrations/0031_mcp_oauth_iss_delegation.sql new file mode 100644 index 00000000..1096a266 --- /dev/null +++ b/middleware/migrations/0031_mcp_oauth_iss_delegation.sql @@ -0,0 +1,81 @@ +-- ── MCP OAuth: RFC 9207 `iss` binding + explicit delegation mode (W0-1) ───── +-- Three live defects in the MCP OAuth path are closed here: +-- +-- D1 The callback trusted `state` alone and never validated the RFC 9207 +-- `iss` authorization-response parameter against the issuer recorded for +-- the flow. `mcp_oauth_flows.issuer` already exists; what was missing is +-- knowing whether the authorization server ADVERTISED iss support, so an +-- absent `iss` from an AS that promised one can be rejected. That flag is +-- captured at authorize time (never re-discovered at callback — same +-- reasoning as migration 0016). +-- +-- D2 `oauthUserKey()` silently fell back to the shared literal 'operator', +-- so a channel turn (Teams/Telegram) with no mapped identity inherited +-- the operator's authority at the customer's MCP server — a confused +-- deputy. `mcp_servers.delegation` makes the choice explicit per server: +-- per_user → the acting identity must resolve, or the call fails closed +-- service → one shared identity is the deliberate, opted-in design +-- +-- D3 Concurrent refreshes for the same (server, user) raced each other. Not +-- a schema concern, but `mcp_oauth_tokens.issuer` lands here so a stored +-- token can be invalidated when its issuer rotates. +-- +-- ⚠️ OPERATOR-VISIBLE BEHAVIOUR CHANGE — read before deploying. +-- A fail-closed `per_user` default for EVERY row would break installed systems +-- whose channel users reach MCP servers today precisely BECAUSE of the +-- 'operator' fallback. So this migration is deliberately asymmetric: +-- • existing rows that already hold an operator token keep today's shared +-- behaviour (delegation = 'service'), and +-- • only NEWLY created servers get the safe 'per_user' default. +-- Operators who want per-user delegation on an existing server must opt in via +-- the MCP Control Center (or UPDATE the column directly). + +-- ── D2: explicit delegation mode per MCP server ───────────────────────────── +ALTER TABLE mcp_servers + ADD COLUMN IF NOT EXISTS delegation TEXT NOT NULL DEFAULT 'per_user'; + +DO $$ +BEGIN + IF NOT EXISTS ( + SELECT 1 FROM pg_constraint WHERE conname = 'mcp_servers_delegation_chk' + ) THEN + ALTER TABLE mcp_servers + ADD CONSTRAINT mcp_servers_delegation_chk + CHECK (delegation IN ('per_user', 'service')); + END IF; +END $$; + +-- Backward compatibility (see the warning above): every EXISTING server that +-- already has a stored token keeps the shared identity it is working with +-- today. Guarded by to_regclass so the migration is safe on a database where +-- mcp_oauth_tokens has not been created yet. +DO $$ +BEGIN + IF to_regclass('public.mcp_oauth_tokens') IS NOT NULL THEN + UPDATE mcp_servers s + SET delegation = 'service' + WHERE EXISTS (SELECT 1 FROM mcp_oauth_tokens t WHERE t.server_id = s.id); + END IF; +END $$; + +-- ── D1: remember whether the AS advertised RFC 9207 at authorize time ─────── +-- NULL on pre-0031 in-flight flows → treated as "not advertised", so a flow +-- started before this migration is not retroactively rejected for a missing +-- `iss`. A mismatched `iss` is rejected regardless of this flag. +ALTER TABLE mcp_oauth_flows + ADD COLUMN IF NOT EXISTS iss_required BOOLEAN NOT NULL DEFAULT false; + +-- ── D3 companion: bind a stored token to the issuer that minted it ────────── +-- Lets a token be invalidated when the server's issuer rotates instead of +-- being replayed against a different authorization server. +ALTER TABLE mcp_oauth_tokens + ADD COLUMN IF NOT EXISTS issuer TEXT; + +-- ── Audit: record the acting identity on every MCP call ───────────────────── +-- `caller_agent` is the orchestrator/sub-agent slug, not WHO the call acted +-- as. Without this an operator cannot answer "whose credentials touched that +-- server?" — the exact question the confused-deputy bug raises. +ALTER TABLE mcp_call_log + ADD COLUMN IF NOT EXISTS acting_identity TEXT; + +-- rollback: ALTER TABLE mcp_call_log DROP COLUMN acting_identity; ALTER TABLE mcp_oauth_tokens DROP COLUMN issuer; ALTER TABLE mcp_oauth_flows DROP COLUMN iss_required; ALTER TABLE mcp_servers DROP CONSTRAINT mcp_servers_delegation_chk, DROP COLUMN delegation; diff --git a/middleware/packages/harness-orchestrator/src/index.ts b/middleware/packages/harness-orchestrator/src/index.ts index 75acc579..34566321 100644 --- a/middleware/packages/harness-orchestrator/src/index.ts +++ b/middleware/packages/harness-orchestrator/src/index.ts @@ -103,6 +103,7 @@ export type { CanvasPos, McpCallLogRow, McpConfigField, + McpDelegation, McpRegistryRow, McpServerInput, McpServerRow, diff --git a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts index d3ed9d0c..3502eb51 100644 --- a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts +++ b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts @@ -96,6 +96,12 @@ export interface McpCallLogEntry { readonly error: string | null; readonly durationMs: number; readonly calledAt: Date; + /** W0-1 — WHOSE authority this call acted under. `callerAgent` names the + * orchestrator; this names the identity its credentials belonged to. The + * literal `unresolved` marks a `per_user` server that had no identity to + * act as (the call fails closed), which is exactly the case an operator + * needs to be able to find in the audit trail. */ + readonly actingIdentity: string | null; } /** Observer invoked after every tool call. Implementations must be fast and @@ -128,6 +134,15 @@ export interface McpAuthProvider { * decision, so the manager needs no OAuth knowledge. */ onAuthFailure(cfg: McpServerConfig): Promise; + /** + * The identity this server's calls act as, for the audit trail (W0-1). Same + * resolution `getToken` uses, exposed separately so EVERY audited call — + * including denied ones and calls to servers with no OAuth at all — records + * who acted. Returns null when the provider cannot attribute the call. + * Optional: providers that predate W0-1 keep working (identity falls back to + * the turn context). + */ + resolveIdentity?(cfg: McpServerConfig): Promise; /** * Secret config values to inject as request headers for this server (epic * #459). Resolved from the Vault per call so secrets never live on the pooled @@ -199,6 +214,7 @@ export class McpManager { ok: boolean, error: string | null, startedAt: number, + actingIdentity: string | null, ): void { if (!this.options?.onToolCall) return; try { @@ -224,6 +240,9 @@ export class McpManager { error: error === null ? null : error.length > 300 ? `${error.slice(0, 300)}…` : error, durationMs: Date.now() - startedAt, calledAt: new Date(), + // W0-1: never left blank. An unattributable call is recorded AS + // unattributable rather than silently omitted. + actingIdentity: actingIdentity ?? ctx?.mcpUserKey ?? null, }); } catch { /* the audit trail must never break a tool call */ @@ -265,13 +284,24 @@ export class McpManager { args: Record, ): Promise { const startedAt = Date.now(); + // Resolve the acting identity FIRST (W0-1), before the guard can short- + // circuit: a denied call still has to say whose authority it would have + // used. Only paid for when auditing is actually on. + let actingIdentity: string | null = null; + if (this.options?.onToolCall && this.options.auth?.resolveIdentity) { + try { + actingIdentity = await this.options.auth.resolveIdentity(cfg); + } catch { + /* identity resolution must not break the call path */ + } + } // Dispatch-time policy gate (issue #454): checked on EVERY call, so a // verdict that turned risky on re-discover blocks immediately and an // operator ack unblocks immediately — independent of registry rebuilds. try { const denial = this.options?.guard?.(cfg.id, toolName); if (denial) { - this.emitCall(cfg, toolName, false, denial, startedAt); + this.emitCall(cfg, toolName, false, denial, startedAt, actingIdentity); return denial; } } catch { @@ -310,7 +340,7 @@ export class McpManager { lastFailure = failure; continue; } - return this.handleFailure(cfg, toolName, token, failure, startedAt); + return this.handleFailure(cfg, toolName, token, failure, startedAt, actingIdentity); } try { const res = await pooled.client.callTool( @@ -331,9 +361,9 @@ export class McpManager { const protocolError = res !== null && typeof res === 'object' && (res as { isError?: unknown }).isError === true; if (protocolError) { - return this.handleFailure(cfg, toolName, token, rendered, startedAt); + return this.handleFailure(cfg, toolName, token, rendered, startedAt, actingIdentity); } - this.emitCall(cfg, toolName, true, null, startedAt); + this.emitCall(cfg, toolName, true, null, startedAt, actingIdentity); return rendered; } catch (err) { // Drop the connection so the next call reconnects (server may have died). @@ -343,11 +373,11 @@ export class McpManager { lastFailure = failure; continue; } - return this.handleFailure(cfg, toolName, token, failure, startedAt); + return this.handleFailure(cfg, toolName, token, failure, startedAt, actingIdentity); } } // Both attempts hit a transient failure. - return this.handleFailure(cfg, toolName, token, lastFailure, startedAt); + return this.handleFailure(cfg, toolName, token, lastFailure, startedAt, actingIdentity); } /** @@ -363,6 +393,7 @@ export class McpManager { token: string | null, rawFailure: string, startedAt: number, + actingIdentity: string | null, ): Promise { const maybeAuth = token === null || looksUnauthorized(rawFailure); if (maybeAuth && this.options?.auth) { @@ -376,11 +407,11 @@ export class McpManager { /* fall back to the raw failure */ } if (authMessage) { - this.emitCall(cfg, toolName, false, 'auth_required', startedAt); + this.emitCall(cfg, toolName, false, 'auth_required', startedAt, actingIdentity); return authMessage; } } - this.emitCall(cfg, toolName, false, rawFailure, startedAt); + this.emitCall(cfg, toolName, false, rawFailure, startedAt, actingIdentity); return rawFailure; } diff --git a/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts b/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts index 4ea2f5bd..a7ae91c0 100644 --- a/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts +++ b/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts @@ -212,8 +212,18 @@ export interface McpServerRow { /** Epic #459 — NON-SECRET config values `{ key: value }`. Secrets are in the * Vault, not here. */ readonly config: Record; + /** W0-1 — whose authority MCP calls to this server act under. + * `per_user`: the acting identity must resolve or the call fails closed — + * no silent inheritance of the operator's authority (confused deputy). + * `service`: one shared identity, the explicit opt-in. Migration 0031 sets + * `service` on pre-existing servers that already hold a token so installed + * deployments keep working; only new rows default to `per_user`. */ + readonly delegation: McpDelegation; } +/** How an MCP server resolves the identity a call acts as (W0-1, D2). */ +export type McpDelegation = 'per_user' | 'service'; + export interface ToolGrantRow { readonly id: string; readonly agentId: string | null; @@ -435,6 +445,8 @@ interface McpServerDbRow { kg_ingest?: boolean; config_schema?: McpConfigField[]; config?: Record; + // W0-1 delegation mode; absent on pre-0031 rows in tests. + delegation?: McpDelegation; } interface McpRegistryDbRow { @@ -514,6 +526,10 @@ export interface McpCallLogRow { readonly error: string | null; readonly durationMs: number; readonly calledAt: Date; + /** W0-1 — WHOSE authority the call acted under (the resolved MCP user key, + * or `unresolved` when a `per_user` server had no identity to act as). + * `callerAgent` is the orchestrator slug; this is the identity. */ + readonly actingIdentity: string | null; } interface McpCallLogDbRow { @@ -528,6 +544,7 @@ interface McpCallLogDbRow { error: string | null; duration_ms: number; called_at: Date; + acting_identity?: string | null; } interface ToolGrantDbRow { @@ -758,6 +775,9 @@ function mapMcpServer(r: McpServerDbRow): McpServerRow { kgIngest: r.kg_ingest ?? false, configSchema: Array.isArray(r.config_schema) ? r.config_schema : [], config: r.config ?? {}, + // Pre-0031 rows (and hand-built test fixtures) read as the safe mode; the + // migration is what grandfathers real installed servers into 'service'. + delegation: r.delegation === 'service' ? 'service' : 'per_user', }; } @@ -1403,6 +1423,8 @@ export class AgentGraphStore { refreshTokenRef: string | null; expiresAt: Date | null; scopes: string | null; + /** Issuer that minted this token (W0-1) — null on pre-0031 rows. */ + issuer: string | null; } | undefined > { @@ -1413,6 +1435,7 @@ export class AgentGraphStore { refresh_token_ref: string | null; expires_at: Date | null; scopes: string | null; + issuer?: string | null; }>('SELECT * FROM mcp_oauth_tokens WHERE server_id = $1 AND user_key = $2', [serverId, userKey]); const r = rows[0]; return r @@ -1423,6 +1446,7 @@ export class AgentGraphStore { refreshTokenRef: r.refresh_token_ref, expiresAt: r.expires_at, scopes: r.scopes, + issuer: r.issuer ?? null, } : undefined; } @@ -1434,19 +1458,45 @@ export class AgentGraphStore { refreshTokenRef: string | null; expiresAt: Date | null; scopes: string | null; + /** Issuer that minted the token (W0-1) — lets a rotated issuer invalidate + * the stored token instead of replaying it against a different AS. */ + issuer?: string | null; }): Promise { await this.pool.query( `INSERT INTO mcp_oauth_tokens - (server_id, user_key, access_token_ref, refresh_token_ref, expires_at, scopes, updated_at) - VALUES ($1,$2,$3,$4,$5,$6, now()) + (server_id, user_key, access_token_ref, refresh_token_ref, expires_at, scopes, issuer, updated_at) + VALUES ($1,$2,$3,$4,$5,$6,$7, now()) ON CONFLICT (server_id, user_key) DO UPDATE SET access_token_ref = EXCLUDED.access_token_ref, refresh_token_ref = EXCLUDED.refresh_token_ref, expires_at = EXCLUDED.expires_at, scopes = EXCLUDED.scopes, + issuer = EXCLUDED.issuer, updated_at = now()`, - [input.serverId, input.userKey, input.accessTokenRef, input.refreshTokenRef, input.expiresAt, input.scopes], + [ + input.serverId, + input.userKey, + input.accessTokenRef, + input.refreshTokenRef, + input.expiresAt, + input.scopes, + input.issuer ?? null, + ], + ); + } + + /** Set the delegation mode for a server (W0-1, D2). Returns the updated row, + * or undefined when the server does not exist. */ + async setMcpServerDelegation( + serverId: string, + delegation: McpDelegation, + ): Promise { + const { rows } = await this.pool.query( + 'UPDATE mcp_servers SET delegation = $2, updated_at = now() WHERE id = $1 RETURNING *', + [serverId, delegation], ); + const r = rows[0]; + return r ? mapMcpServer(r) : undefined; } async deleteMcpOAuthToken(serverId: string, userKey: string): Promise { @@ -1475,13 +1525,17 @@ export class AgentGraphStore { scopes: string | null; tokenEndpoint: string; authorizationEndpoint: string; + /** Whether the AS advertised RFC 9207 `authorization_response_iss_parameter_supported` + * at authorize time (W0-1, D1). Captured HERE, never re-discovered at the + * callback — same reasoning as the endpoint binding in migration 0016. */ + issRequired?: boolean; }): Promise { // Opportunistic prune of stale flows (older than 15 min) on each create. await this.pool.query("DELETE FROM mcp_oauth_flows WHERE created_at < now() - interval '15 minutes'"); await this.pool.query( `INSERT INTO mcp_oauth_flows - (state, server_id, user_key, issuer, code_verifier, redirect_uri, scopes, token_endpoint, authorization_endpoint) - VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9)`, + (state, server_id, user_key, issuer, code_verifier, redirect_uri, scopes, token_endpoint, authorization_endpoint, iss_required) + VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10)`, [ input.state, input.serverId, @@ -1492,6 +1546,7 @@ export class AgentGraphStore { input.scopes, input.tokenEndpoint, input.authorizationEndpoint, + input.issRequired ?? false, ], ); } @@ -1510,6 +1565,9 @@ export class AgentGraphStore { scopes: string | null; tokenEndpoint: string | null; authorizationEndpoint: string | null; + /** The AS advertised RFC 9207 when this flow started (W0-1, D1), so an + * authorization response WITHOUT `iss` must be rejected. */ + issRequired: boolean; } | undefined > { @@ -1523,6 +1581,7 @@ export class AgentGraphStore { scopes: string | null; token_endpoint: string | null; authorization_endpoint: string | null; + iss_required?: boolean | null; }>( "DELETE FROM mcp_oauth_flows WHERE state = $1 AND created_at > now() - interval '15 minutes' RETURNING *", [state], @@ -1539,6 +1598,9 @@ export class AgentGraphStore { scopes: r.scopes, tokenEndpoint: r.token_endpoint, authorizationEndpoint: r.authorization_endpoint, + // Pre-0031 in-flight flows read false — they are not retroactively + // rejected for a missing `iss` (a MISMATCHED one still is). + issRequired: r.iss_required === true, } : undefined; } @@ -1714,11 +1776,15 @@ export class AgentGraphStore { readonly error: string | null; readonly durationMs: number; readonly calledAt: Date; + /** W0-1 — the resolved acting identity. Always written (never omitted): + * an audit row with no identity cannot answer "whose credentials was + * this?", which is the whole point of the confused-deputy fix. */ + readonly actingIdentity: string | null; }): Promise { await this.pool.query( `INSERT INTO mcp_call_log - (server_id, server_name, tool_name, caller_kind, caller_agent, turn_id, ok, error, duration_ms, called_at) - VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10)`, + (server_id, server_name, tool_name, caller_kind, caller_agent, turn_id, ok, error, duration_ms, called_at, acting_identity) + VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11)`, [ entry.serverId, entry.serverName, @@ -1730,6 +1796,7 @@ export class AgentGraphStore { entry.error, entry.durationMs, entry.calledAt, + entry.actingIdentity, ], ); } @@ -1770,6 +1837,7 @@ export class AgentGraphStore { error: r.error, durationMs: r.duration_ms, calledAt: r.called_at, + actingIdentity: r.acting_identity ?? null, })); } diff --git a/middleware/src/index.ts b/middleware/src/index.ts index bf629e57..4ebef1e5 100644 --- a/middleware/src/index.ts +++ b/middleware/src/index.ts @@ -271,6 +271,12 @@ import { ServiceRegistry } from './platform/serviceRegistry.js'; import { TurnHookRegistry } from './platform/turnHookRegistry.js'; import { NativeToolRegistry } from '@omadia/orchestrator'; import { McpManager, type McpCallLogEntry, type McpServerConfig } from '@omadia/orchestrator'; +import { + SERVICE_USER_KEY, + auditIdentity, + delegationBlockedMessage, + resolveMcpUserKey, +} from './services/mcpDelegation.js'; import { McpOAuthService } from './services/mcpOAuthService.js'; import { McpConfigService } from './services/mcpConfigService.js'; import { @@ -1486,8 +1492,12 @@ async function main(): Promise { // Generic MCP OAuth service (epic #459 W9) — outer scope so both the // McpManager (auth provider) and the operator router (begin/callback routes) - // reference the same instance. userKey='operator' for the operator chat. - const mcpOAuthUserKey = 'operator'; + // reference the same instance. + // + // W0-1: this is now ONLY the shared key for servers whose `delegation` is + // `service`. It is no longer a fallback for unresolved identities — see + // services/mcpDelegation.ts. + const mcpOAuthUserKey = SERVICE_USER_KEY; // Redirect URI the OAuth callback lands on: explicit override, else derived // from the public base. The service activates when either is configured. const mcpOAuthRedirectUri = @@ -1791,18 +1801,49 @@ async function main(): Promise { const server = (await mcpAuditStore.listMcpServers()).find((s) => s.id === cfg.id); if (!server) return null; // Per-user token (codex W9 fold): the turn's authenticated - // user when the entry point set it, else the operator scope. - const userKey = turnContext.current()?.mcpUserKey ?? mcpOAuthUserKey; + // user when the entry point set it. + // + // W0-1 (D2) — THE confused-deputy fix. This used to end in + // `?? mcpOAuthUserKey`, i.e. `'operator'`. A Teams/Telegram + // turn whose user has no mapped identity therefore reached + // the customer's MCP server holding the OPERATOR's token. + // Now a `per_user` server with no identity gets no token and + // the call fails closed through onAuthFailure below; + // `service` delegation is the explicit shared-identity + // opt-in. + const userKey = resolveMcpUserKey( + server, + turnContext.current()?.mcpUserKey, + mcpOAuthUserKey, + ); + if (userKey === null) return null; return mcpOAuthService.getValidAccessToken(server, userKey); }, + resolveIdentity: async (cfg: McpServerConfig) => { + const server = (await mcpAuditStore.listMcpServers()).find((s) => s.id === cfg.id); + if (!server) return null; + // W0-1: every audit row names the identity it acted as — + // `unresolved` when there was none. + return auditIdentity(server, turnContext.current()?.mcpUserKey, mcpOAuthUserKey); + }, onAuthFailure: async (cfg: McpServerConfig) => { const server = (await mcpAuditStore.listMcpServers()).find((s) => s.id === cfg.id); if (!server) return null; + // W0-1 (D2): fail closed FIRST. A `per_user` server with no + // caller identity must never be "fixed" by starting an OAuth + // flow — that flow would bind a token to whoever happens to + // click through, which is the same confused deputy one step + // removed. Explain instead, and send nothing upstream. + const userKey = resolveMcpUserKey( + server, + turnContext.current()?.mcpUserKey, + mcpOAuthUserKey, + ); + if (userKey === null) return delegationBlockedMessage(server.name); // Only OAuth-protected servers get an auth prompt (cached // discovery keeps this cheap per call). const desc = await mcpOAuthService.describeAuth(server); if (!desc.protected) return null; - const userKey = turnContext.current()?.mcpUserKey ?? mcpOAuthUserKey; // Machine block the chat parses into an in-line "Connect" card // + modal (web-ui McpAuthRequiredCard). Mirrors the // block contract: human text stays readable for the model and diff --git a/middleware/src/routes/agentBuilder.ts b/middleware/src/routes/agentBuilder.ts index bec4a697..38cf86de 100644 --- a/middleware/src/routes/agentBuilder.ts +++ b/middleware/src/routes/agentBuilder.ts @@ -44,7 +44,13 @@ import { substituteMcpConfig, deriveMcpConfigSchema, } from '../agents/subAgentToolHydration.js'; +import { + auditIdentity, + parseDelegation, + resolveMcpUserKey, +} from '../services/mcpDelegation.js'; import { rescanAllMcpServers } from '../services/mcpRescan.js'; +import { redactSecrets } from '../services/secretRedaction.js'; import { MCP_SEVERITIES_NEEDING_ACK, refreshMcpGrantPolicy, @@ -115,7 +121,9 @@ export interface AgentBuilderRouterOptions { brokered: boolean; }>; beginAuthorization(server: McpServerRow, userKey: string): Promise<{ authorizeUrl: string }>; - completeAuthorization(state: string, code: string): Promise<{ serverId: string }>; + /** `iss` is the RFC 9207 authorization-response parameter (W0-1, D1), + * validated against the flow-bound issuer before any exchange. */ + completeAuthorization(state: string, code: string, iss?: string | null): Promise<{ serverId: string }>; setManualClient(issuer: string, clientId: string, clientSecret: string | null): Promise; getValidAccessToken(server: McpServerRow, userKey: string): Promise; }; @@ -221,13 +229,28 @@ export function createAgentBuilderRouter( : undefined; if (!server) return null; // Per-user token (bugfix, mirrors the runtime McpManager in index.ts): - // tokens are STORED under the request's session-derived key - // (oauthUserKey below) at connect time, so lookup must use the same - // key. The static `options.mcpOAuthUserKey` fallback only applies - // outside any turn context (no session identity available). - const userKey = turnContext.current()?.mcpUserKey ?? options.mcpOAuthUserKey ?? 'operator'; + // tokens are STORED under the request's session-derived key at connect + // time, so lookup must use the same key. + // + // W0-1 (D2): the previous `?? 'operator'` tail is gone. A `per_user` + // server with no resolvable identity now yields no token and the call + // fails closed, instead of quietly using the operator's authority. + const userKey = resolveMcpUserKey( + server, + turnContext.current()?.mcpUserKey, + options.mcpOAuthUserKey, + ); + if (userKey === null) return null; return options.mcpOAuth.getValidAccessToken(server, userKey); }, + resolveIdentity: async (cfg: McpServerConfig): Promise => { + const graph = options.getGraphStore(); + const server = graph + ? (await graph.listMcpServers()).find((s) => s.id === cfg.id) + : undefined; + if (!server) return null; + return auditIdentity(server, turnContext.current()?.mcpUserKey, options.mcpOAuthUserKey); + }, // Discover/test-call surface needs-auth via the route's describeAuth path, // so the manager itself doesn't need to synthesize a prompt here. onAuthFailure: async (): Promise => null, @@ -1065,14 +1088,18 @@ export function createAgentBuilderRouter( if (!l) return; // Establish the per-request MCP OAuth identity (bugfix): the shared // McpManager's getToken reads turnContext.mcpUserKey to look up the token - // under the SAME key it was stored under (oauthUserKey(req) — see - // auth-status/authorize below), instead of silently falling back to the - // static 'operator' default and missing it. `enter` (not `run`) because - // this scope is naturally bounded by the request's own async chain. + // under the SAME key it was stored under (see auth-status/authorize + // below), instead of silently missing it. `enter` (not `run`) because this + // scope is naturally bounded by the request's own async chain. + // + // W0-1: this carries the session's CANDIDATE identity. Whether it may be + // replaced by a shared one is decided per server by `resolveMcpUserKey` in + // the auth provider — not by a default here. + const discoverIdentity = sessionIdentity(req); turnContext.enter({ turnId: `mcp-discover-${str(req.params.id)}`, turnDate: today(), - mcpUserKey: oauthUserKey(req), + ...(discoverIdentity ? { mcpUserKey: discoverIdentity } : {}), }); try { const servers = await l.graph.listMcpServers(); @@ -1407,10 +1434,18 @@ export function createAgentBuilderRouter( // ── generic MCP OAuth (epic #459 W9) ────────────────────────────────────── // Tokens are keyed to the authenticated operator's identity (codex W9 fold): - // one operator's token is never reused for another. Falls back to a shared - // key only when no session identity is available (single-admin/dev). - const oauthUserKey = (req: Request): string => - req.session?.sub || req.session?.email || options.mcpOAuthUserKey || 'operator'; + // one operator's token is never reused for another. + // + // W0-1 (D2): the identity the SESSION offers, with no fallback baked in. The + // old `|| 'operator'` tail is gone — whether an unresolved identity may + // borrow a shared one is now the server's `delegation` decision, applied by + // `resolveMcpUserKey`, never an implicit default here. + const sessionIdentity = (req: Request): string | null => + req.session?.sub || req.session?.email || null; + /** The key to act as for THIS server, or null when a `per_user` server has + * no resolvable identity (fail closed — never silently the operator). */ + const oauthUserKeyFor = (req: Request, server: McpServerRow): string | null => + resolveMcpUserKey(server, sessionIdentity(req)); const escapeHtml = (s: string): string => s.replace(/[&<>"']/g, (c) => `&#${c.charCodeAt(0)};`); @@ -1426,21 +1461,45 @@ export function createAgentBuilderRouter( return; } if (!options.mcpOAuth) { - res.json({ protected: false, connected: false, issuer: null, needsClient: false, brokered: false }); + res.json({ + protected: false, + connected: false, + issuer: null, + needsClient: false, + brokered: false, + delegation: server.delegation, + identityResolved: sessionIdentity(req) !== null, + }); return; } const desc = await options.mcpOAuth.describeAuth(server); if (!desc.protected) { - res.json({ protected: false, connected: false, issuer: null, needsClient: false, brokered: false }); + res.json({ + protected: false, + connected: false, + issuer: null, + needsClient: false, + brokered: false, + delegation: server.delegation, + identityResolved: sessionIdentity(req) !== null, + }); return; } - const token = await l.graph.getMcpOAuthToken(server.id, oauthUserKey(req)); + // W0-1: null under `per_user` with no session identity — report it as + // not-connected rather than probing the shared operator token. + const userKey = oauthUserKeyFor(req, server); + const token = + userKey === null ? undefined : await l.graph.getMcpOAuthToken(server.id, userKey); const client = desc.issuer ? await l.graph.getMcpOAuthClient(desc.issuer) : undefined; res.json({ protected: true, connected: token !== undefined, issuer: desc.issuer, issuerHost: desc.issuerHost, + // W0-1 (D2): whose authority calls to this server act under, and + // whether this session actually has an identity to act as. + delegation: server.delegation, + identityResolved: userKey !== null, // A brokered server (offers DCR) needs no manual client even without one // stored — DCR self-registers at connect. Only a delegating server does. brokered: desc.brokered, @@ -1466,8 +1525,16 @@ export function createAgentBuilderRouter( res.status(404).json({ error: 'mcp_server_not_found' }); return; } + // W0-1 (D2): authorizing under a borrowed identity is exactly the + // confused deputy — a `per_user` server with no session identity has + // nobody to store the token for, so refuse before starting the flow. + const userKey = oauthUserKeyFor(req, server); + if (userKey === null) { + res.status(403).json({ error: 'delegation_identity_unresolved' }); + return; + } try { - const { authorizeUrl } = await options.mcpOAuth.beginAuthorization(server, oauthUserKey(req)); + const { authorizeUrl } = await options.mcpOAuth.beginAuthorization(server, userKey); res.json({ authorizeUrl }); } catch (err) { // Issuer without DCR needs a one-time manual client first. @@ -1514,13 +1581,50 @@ export function createAgentBuilderRouter( const l = live(res); if (!l) return; try { - await l.graph.deleteMcpOAuthToken(str(req.params.id), oauthUserKey(req)); + const server = (await l.graph.listMcpServers()).find((s) => s.id === str(req.params.id)); + if (!server) { + res.status(404).json({ error: 'mcp_server_not_found' }); + return; + } + // W0-1: only ever delete the token this caller actually owns. With the + // old shared fallback an identity-less session could disconnect the + // operator's token. + const userKey = oauthUserKeyFor(req, server); + if (userKey === null) { + res.status(403).json({ error: 'delegation_identity_unresolved' }); + return; + } + await l.graph.deleteMcpOAuthToken(server.id, userKey); res.status(204).end(); } catch (err) { fail(res, err); } }); + /** Set a server's delegation mode (W0-1, D2). `per_user` requires each + * caller to have its own identity; `service` is the explicit opt-in to one + * shared identity. Migration 0031 grandfathers already-connected servers + * into `service`, so this is how an operator moves one to `per_user`. */ + router.put('/mcp-servers/:id/delegation', async (req: Request, res: Response) => { + const l = live(res); + if (!l) return; + try { + const delegation = parseDelegation(req.body?.delegation); + if (delegation === null) { + res.status(400).json({ error: 'invalid_delegation' }); + return; + } + const updated = await l.graph.setMcpServerDelegation(str(req.params.id), delegation); + if (!updated) { + res.status(404).json({ error: 'mcp_server_not_found' }); + return; + } + res.json({ id: updated.id, delegation: updated.delegation }); + } catch (err) { + fail(res, err); + } + }); + /** OAuth callback: exchange the code, store the token, show a done page. * Hit by the operator's own browser redirect (session cookie present); the * `state` param is the CSRF guard. */ @@ -1541,6 +1645,16 @@ export function createAgentBuilderRouter( const code = typeof req.query['code'] === 'string' ? req.query['code'] : ''; const state = typeof req.query['state'] === 'string' ? req.query['state'] : ''; const providerError = typeof req.query['error'] === 'string' ? req.query['error'] : ''; + // RFC 9207 issuer identifier (W0-1, D1). `state` proves the response + // belongs to a flow we started; it does NOT prove which authorization + // server minted the code. A repeated `iss` (Express gives an array) is + // itself a tampering signal — treat it as a mismatch, not a "pick one". + const rawIss = req.query['iss']; + if (rawIss !== undefined && typeof rawIss !== 'string') { + res.status(400).send(donePage(false, 'The authorization response carried a malformed issuer.')); + return; + } + const iss = typeof rawIss === 'string' ? rawIss : null; if (providerError) { res.status(400).send(donePage(false, `The provider returned: ${providerError}`)); return; @@ -1549,10 +1663,25 @@ export function createAgentBuilderRouter( res.status(400).send(donePage(false, 'Missing code or state.')); return; } - await options.mcpOAuth.completeAuthorization(state, code); + // The service validates `iss` against the flow-bound issuer BEFORE + // exchanging the code, so a rejected callback stores nothing. + await options.mcpOAuth.completeAuthorization(state, code, iss); res.status(200).send(donePage(true, 'The server is now authorized for you.')); } catch (err) { - res.status(400).send(donePage(false, msg(err))); + // Never echo the raw error here: it can carry the code or the PKCE + // verifier (D5). The issuer-mismatch case gets an explicit message. + if (err instanceof Error && err.name === 'McpOAuthIssuerMismatchError') { + res + .status(400) + .send( + donePage( + false, + 'The authorization response came from an unexpected issuer and was rejected. Nothing was stored. Please start the connection again.', + ), + ); + return; + } + res.status(400).send(donePage(false, redactSecrets(msg(err)))); } }); diff --git a/middleware/src/services/mcpAuthDiscovery.ts b/middleware/src/services/mcpAuthDiscovery.ts index cda324f1..e01ad86a 100644 --- a/middleware/src/services/mcpAuthDiscovery.ts +++ b/middleware/src/services/mcpAuthDiscovery.ts @@ -34,6 +34,11 @@ export interface AuthServerMetadata { readonly codeChallengeMethods: readonly string[]; readonly grantTypes: readonly string[]; readonly scopesSupported: readonly string[]; + /** RFC 9207 `authorization_response_iss_parameter_supported` (W0-1, D1). When + * the AS advertises this, an authorization response WITHOUT `iss` is a + * protocol violation and must be rejected — that is what makes mix-up + * detection enforceable rather than best-effort. */ + readonly issParameterSupported: boolean; } export interface DiscoveredAuth { @@ -217,6 +222,7 @@ export class McpAuthDiscovery { codeChallengeMethods: strArr(doc['code_challenge_methods_supported']), grantTypes: strArr(doc['grant_types_supported']), scopesSupported: strArr(doc['scopes_supported']), + issParameterSupported: doc['authorization_response_iss_parameter_supported'] === true, }; } diff --git a/middleware/src/services/mcpDelegation.ts b/middleware/src/services/mcpDelegation.ts new file mode 100644 index 00000000..ceff17e6 --- /dev/null +++ b/middleware/src/services/mcpDelegation.ts @@ -0,0 +1,94 @@ +/** + * Which identity an MCP call acts as (W0-1, D2) — the confused-deputy fix. + * + * Before this, both the operator router and the runtime McpManager resolved the + * OAuth user key as ` ?? 'operator'`. That fallback is the bug: a + * Teams or Telegram turn whose user has no mapped identity would silently reach + * the customer's MCP server holding the OPERATOR's token — full operator + * authority, granted to whoever happened to be typing in a channel. + * + * Resolution is now explicit and per server: + * + * delegation = 'service' one shared identity, deliberately opted into. The + * key stays `operator`, so servers grandfathered by + * migration 0031 keep the token they already have. + * + * delegation = 'per_user' the caller's own identity or nothing. An + * unresolvable identity yields `null`, the caller + * gets no token, and the call fails closed through + * the existing `onAuthFailure` path. + * + * There is deliberately no third branch. Every path that needs a user key goes + * through `resolveMcpUserKey`, so the fallback cannot reappear by accident. + */ + +import type { McpDelegation, McpServerRow } from '@omadia/orchestrator'; + +/** The shared key used when a server opts into `service` delegation. Matches + * the historical literal so pre-0031 stored tokens keep resolving. */ +export const SERVICE_USER_KEY = 'operator'; + +/** Recorded in the audit trail when a `per_user` server had no identity to act + * as. A row that simply said nothing would hide exactly the case operators + * need to find. */ +export const UNRESOLVED_IDENTITY = 'unresolved'; + +/** Just the delegation-relevant slice of a server row, so callers (and tests) + * need not build a whole `McpServerRow`. */ +export interface DelegationTarget { + readonly delegation: McpDelegation; +} + +/** + * The identity this call acts as, or `null` when a `per_user` server has no + * resolvable caller. + * + * @param server the target server (its `delegation` mode decides). + * @param candidate the caller's own identity — a session `sub`/`email`, or the + * turn context's `mcpUserKey`. Blank/whitespace counts as + * absent. + * @param serviceKey the shared key for `service` delegation. Defaults to the + * historical `operator` literal so grandfathered tokens keep + * resolving; only override it in tests. + */ +export function resolveMcpUserKey( + server: DelegationTarget, + candidate: string | null | undefined, + serviceKey: string = SERVICE_USER_KEY, +): string | null { + if (server.delegation === 'service') return serviceKey; + const trimmed = typeof candidate === 'string' ? candidate.trim() : ''; + return trimmed === '' ? null : trimmed; +} + +/** The identity to write to `mcp_call_log`. Never null-by-omission: an + * unattributable call is recorded AS unattributable. */ +export function auditIdentity( + server: DelegationTarget, + candidate: string | null | undefined, + serviceKey: string = SERVICE_USER_KEY, +): string { + return resolveMcpUserKey(server, candidate, serviceKey) ?? UNRESOLVED_IDENTITY; +} + +/** Operator-facing explanation when a `per_user` server has no caller identity. + * Returned through `onAuthFailure`, so the turn fails closed with a reason + * instead of silently borrowing the operator's authority. */ +export function delegationBlockedMessage(serverName: string): string { + return ( + `🔒 The MCP server "${serverName}" is set to per-user delegation, but this conversation has no ` + + `mapped user identity, so there is no one to act as. Nothing was sent to the server. ` + + `Either sign in through a channel that maps your identity, or have an operator switch this ` + + `server to a shared service identity in the MCP Control Center.` + ); +} + +/** Narrow an untrusted string to a delegation mode. */ +export function parseDelegation(value: unknown): McpDelegation | null { + return value === 'per_user' || value === 'service' ? value : null; +} + +/** Convenience for callers that hold a full row. */ +export function serverDelegation(server: Pick): McpDelegation { + return server.delegation; +} diff --git a/middleware/src/services/mcpOAuthService.ts b/middleware/src/services/mcpOAuthService.ts index 52db4689..3f43bca4 100644 --- a/middleware/src/services/mcpOAuthService.ts +++ b/middleware/src/services/mcpOAuthService.ts @@ -12,6 +12,7 @@ */ import { McpAuthDiscovery, serverOrigin, type DiscoveredAuth } from './mcpAuthDiscovery.js'; import { McpOAuthClient, type OAuthClientCredentials } from './mcpOAuthClient.js'; +import { redactedErrorText } from './secretRedaction.js'; import { substituteMcpConfig } from '../agents/subAgentToolHydration.js'; import type { AgentGraphStore, McpServerRow } from '@omadia/orchestrator'; @@ -50,6 +51,40 @@ export class McpOAuthNeedsClientError extends Error { } } +/** + * RFC 9207 issuer validation failed at the callback (W0-1, D1). + * + * The authorization response either carried an `iss` naming a DIFFERENT + * authorization server than the one this flow was started against, or omitted + * `iss` entirely although that AS advertised support for it. Both are the + * mix-up signature: a malicious or compromised MCP server steering the + * callback so a code minted by one AS is redeemed at another. + * + * This is thrown BEFORE the code is exchanged, so nothing is ever persisted. + */ +export class McpOAuthIssuerMismatchError extends Error { + constructor( + readonly expected: string, + readonly received: string | null, + ) { + super( + received === null + ? `authorization response omitted the "iss" parameter although issuer "${expected}" advertises RFC 9207 support` + : `authorization response issuer "${received}" does not match the issuer this flow was started against ("${expected}")`, + ); + this.name = 'McpOAuthIssuerMismatchError'; + } +} + +/** RFC 9207 §2.4: compare issuer identifiers exactly, modulo one trailing + * slash (`https://as.example` and `https://as.example/` are the same AS). + * Deliberately NOT a loose/normalizing comparison — that would reintroduce + * the mix-up the check exists to prevent. */ +function sameIssuer(a: string, b: string): boolean { + const norm = (s: string): string => s.trim().replace(/\/+$/, ''); + return norm(a) !== '' && norm(a) === norm(b); +} + const DCR_PROBE_TTL_MS = 10 * 60 * 1000; export class McpOAuthService { @@ -61,6 +96,20 @@ export class McpOAuthService { * the advertised flag. Cached to avoid re-probing on every status check. */ private readonly dcrProbeCache = new Map(); + /** In-flight refreshes, keyed by (serverId, userKey) — W0-1, D3. + * + * Without this, N concurrent callers whose token just expired each POST the + * SAME refresh token to the token endpoint. Against an AS with rotating + * refresh tokens (the OAuth 2.1 default) the first response invalidates the + * token the others are still using, so the losers get `invalid_grant` and, + * worse, the last writer can persist a refresh token the AS has already + * retired — the user silently ends up disconnected. + * + * Sharing one promise makes exactly one HTTP request per (server, user) + * regardless of caller count. The entry is removed in a `finally` so a + * failed refresh never poisons later attempts. */ + private readonly refreshInFlight = new Map>(); + /** The redirect URI the operator must register with the OAuth provider. */ readonly redirectUri: string; @@ -96,7 +145,10 @@ export class McpOAuthService { } /** A live access token for (server, user), refreshing if near expiry, or null - * when the user has not authorized. */ + * when the user has not authorized. + * + * Concurrent callers that all need a refresh share ONE refresh (D3) — see + * `refreshInFlight`. */ async getValidAccessToken(server: McpServerRow, userKey: string): Promise { const row = await this.deps.graph.getMcpOAuthToken(server.id, userKey); if (!row) return null; @@ -109,20 +161,53 @@ export class McpOAuthService { if (!row.refreshTokenRef || !server.endpoint) { return (await this.deps.vault.get(VAULT_NS, row.accessTokenRef)) ?? null; } - const refreshToken = await this.deps.vault.get(VAULT_NS, row.refreshTokenRef); - if (!refreshToken) return (await this.deps.vault.get(VAULT_NS, row.accessTokenRef)) ?? null; - try { - const discovered = await this.discovery.discover(this.resolveEndpoint(server)); - if (!discovered) return null; - const client = await this.loadClient(discovered.server.issuer); - if (!client) return null; - const tok = await this.client.refresh({ server: discovered.server, client, refreshToken }); - await this.persistToken(server.id, userKey, tok); - return tok.accessToken; - } catch (err) { - this.deps.log?.(`[mcpOAuth] refresh failed for ${server.name}: ${String(err)}`); - return (await this.deps.vault.get(VAULT_NS, row.accessTokenRef)) ?? null; - } + + // ── single-flight (W0-1, D3) ──────────────────────────────────────────── + // Everything below runs at most once per (server, user) at a time. The + // map is checked and populated synchronously — no `await` between the get + // and the set — so two callers in the same tick cannot both miss. + const key = `${server.id}${userKey}`; + const existing = this.refreshInFlight.get(key); + if (existing) return existing; + + const refreshRefKey = row.refreshTokenRef; + const accessRefKey = row.accessTokenRef; + const attempt = (async (): Promise => { + const refreshToken = await this.deps.vault.get(VAULT_NS, refreshRefKey); + if (!refreshToken) return (await this.deps.vault.get(VAULT_NS, accessRefKey)) ?? null; + try { + const discovered = await this.discovery.discover(this.resolveEndpoint(server)); + if (!discovered) return null; + // The issuer rotated since this token was minted (W0-1): the stored + // token belongs to a different authorization server, so replaying it + // here would send one AS's credential to another. Drop it and make the + // user re-authorize against the new issuer. + if (row.issuer !== null && !sameIssuer(row.issuer, discovered.server.issuer)) { + this.deps.log?.( + `[mcpOAuth] issuer rotated for ${server.name} (stored ${row.issuer} → discovered ${discovered.server.issuer}); dropping the stored token`, + ); + await this.deps.graph.deleteMcpOAuthToken(server.id, userKey); + return null; + } + const client = await this.loadClient(discovered.server.issuer); + if (!client) return null; + const tok = await this.client.refresh({ server: discovered.server, client, refreshToken }); + await this.persistToken(server.id, userKey, tok, discovered.server.issuer); + return tok.accessToken; + } catch (err) { + // D5: an OAuth error body routinely echoes the token back — never let + // `String(err)` reach a log line unredacted. + this.deps.log?.( + `[mcpOAuth] refresh failed for ${server.name}: ${redactedErrorText(err, [refreshToken])}`, + ); + return (await this.deps.vault.get(VAULT_NS, accessRefKey)) ?? null; + } + })().finally(() => { + this.refreshInFlight.delete(key); + }); + + this.refreshInFlight.set(key, attempt); + return attempt; } /** Start the authorization flow: returns the URL to send the user to. */ @@ -156,6 +241,11 @@ export class McpOAuthService { // that a malicious server could have switched in the meantime. tokenEndpoint: discovered.server.tokenEndpoint, authorizationEndpoint: discovered.server.authorizationEndpoint, + // RFC 9207 (W0-1, D1): remember whether THIS authorization server + // promised to send `iss`, captured now rather than re-discovered at the + // callback — a server that could flip the flag in between would simply + // opt itself out of the check. + issRequired: discovered.server.issParameterSupported, }); return { authorizeUrl: url }; } @@ -163,11 +253,35 @@ export class McpOAuthService { /** Finish the flow at the callback: exchange the code and store the token. * Uses the endpoints captured when the flow started — NOT a fresh discovery * (codex W9 critical fold: a malicious server could otherwise switch its - * token endpoint to steal the code + PKCE verifier + client secret). */ - async completeAuthorization(state: string, code: string): Promise<{ serverId: string }> { + * token endpoint to steal the code + PKCE verifier + client secret). + * + * @param iss the RFC 9207 `iss` authorization-response parameter, or null + * when the provider sent none. Validated against the issuer bound to the + * flow BEFORE the code is exchanged, so a mismatch persists nothing. */ + async completeAuthorization( + state: string, + code: string, + iss?: string | null, + ): Promise<{ serverId: string }> { const flow = await this.deps.graph.takeMcpOAuthFlow(state); if (!flow) throw new Error('unknown or expired authorization state'); if (!flow.tokenEndpoint) throw new Error('flow is missing its bound token endpoint'); + // ── RFC 9207 issuer validation (W0-1, D1) ─────────────────────────────── + // `state` alone proves only that the response came back to a flow we + // started; it does NOT prove WHICH authorization server issued the code. + // A malicious MCP server can steer the browser so a code minted by one AS + // is redeemed at another. Runs before the exchange — a rejected callback + // must leave no token behind. + const received = typeof iss === 'string' && iss.trim() !== '' ? iss.trim() : null; + if (received !== null) { + if (!sameIssuer(flow.issuer, received)) { + throw new McpOAuthIssuerMismatchError(flow.issuer, received); + } + } else if (flow.issRequired) { + // The AS advertised RFC 9207 support and then did not send `iss` — + // either a stripped parameter or a response that never came from it. + throw new McpOAuthIssuerMismatchError(flow.issuer, null); + } const client = await this.loadClient(flow.issuer); if (!client) throw new McpOAuthNeedsClientError(flow.issuer); // Reconstruct the minimal server metadata from the FLOW-BOUND values. @@ -179,6 +293,9 @@ export class McpOAuthService { codeChallengeMethods: [] as string[], grantTypes: [] as string[], scopesSupported: [] as string[], + // Irrelevant for the exchange itself; the `iss` decision was already + // made above from the flow's persisted `issRequired`. + issParameterSupported: flow.issRequired, }; const tok = await this.client.exchangeCode({ server: boundServer, @@ -187,7 +304,7 @@ export class McpOAuthService { codeVerifier: flow.codeVerifier, redirectUri: flow.redirectUri, }); - await this.persistToken(flow.serverId, flow.userKey, tok); + await this.persistToken(flow.serverId, flow.userKey, tok, flow.issuer); return { serverId: flow.serverId }; } @@ -328,6 +445,9 @@ export class McpOAuthService { serverId: string, userKey: string, tok: { accessToken: string; refreshToken: string | null; expiresInSec: number | null; scope: string | null }, + /** Issuer that minted this token (W0-1) — recorded so a later issuer + * rotation invalidates it instead of replaying it at a different AS. */ + issuer?: string | null, ): Promise { const accessRef = this.tokenRef(serverId, userKey, 'access'); await this.deps.vault.set(VAULT_NS, accessRef, tok.accessToken); @@ -343,6 +463,7 @@ export class McpOAuthService { refreshTokenRef: refreshRef, expiresAt: tok.expiresInSec ? new Date(Date.now() + tok.expiresInSec * 1000) : null, scopes: tok.scope, + issuer: issuer ?? null, }); } diff --git a/middleware/src/services/secretRedaction.ts b/middleware/src/services/secretRedaction.ts new file mode 100644 index 00000000..b47b47a4 --- /dev/null +++ b/middleware/src/services/secretRedaction.ts @@ -0,0 +1,77 @@ +/** + * Redaction for anything on its way to a log line in the OAuth path (W0-1, D5). + * + * An OAuth error is one of the most secret-dense strings in the system: a + * provider's error body routinely echoes the `code`, the `code_verifier`, or a + * whole token JSON back at you, and `fetch` failures embed the request URL with + * its query string. `String(err)` therefore cannot go to a log untouched. + * + * Two layers, because either alone leaks: + * 1. EXACT values we already hold (the token we just sent, the verifier we + * generated) — caught wherever they appear, in any encoding shape. + * 2. PATTERNS for values we do NOT hold, because the error came from a server + * that minted them (a rotated refresh token in a JSON error body). + */ + +const REDACTED = '[redacted]'; + +/** Sensitive parameter/field names, matched in JSON bodies and query strings. */ +const SECRET_KEYS = [ + 'access_token', + 'refresh_token', + 'id_token', + 'code_verifier', + 'code_challenge', + 'client_secret', + 'assertion', + 'code', +] as const; + +const KEY_ALTERNATION = SECRET_KEYS.join('|'); + +/** `"access_token":"…"` / `"access_token": '…'` in a JSON-ish body. */ +const JSON_FIELD_RE = new RegExp(`("?(?:${KEY_ALTERNATION})"?\\s*:\\s*)("[^"]*"|'[^']*'|[^,}\\s]+)`, 'gi'); +/** `code=…` / `&refresh_token=…` in a query string or form body. */ +const QUERY_PARAM_RE = new RegExp(`\\b(${KEY_ALTERNATION})=([^&\\s"'}\\]]+)`, 'gi'); +/** `Authorization: Bearer …` echoed back in an error. */ +const BEARER_RE = /\b(bearer\s+)[A-Za-z0-9._~+/-]{8,}=*/gi; + +/** Escape a literal for safe use inside a RegExp. */ +function escapeRe(s: string): string { + return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} + +/** + * Redact secrets from arbitrary text before it reaches a log. + * + * @param text the text to sanitize (already stringified). + * @param secrets exact secret values known to the caller (token, verifier, …). + * Short values (< 8 chars) are ignored — redacting them would + * shred unrelated text without protecting anything meaningful. + */ +export function redactSecrets(text: string, secrets: readonly (string | null | undefined)[] = []): string { + let out = text; + for (const secret of secrets) { + if (typeof secret !== 'string' || secret.length < 8) continue; + out = out.replace(new RegExp(escapeRe(secret), 'g'), REDACTED); + // Providers frequently echo the value URL-encoded rather than raw. + const encoded = encodeURIComponent(secret); + if (encoded !== secret) out = out.replace(new RegExp(escapeRe(encoded), 'g'), REDACTED); + } + out = out.replace(JSON_FIELD_RE, (_m, key: string) => `${key}"${REDACTED}"`); + out = out.replace(QUERY_PARAM_RE, (_m, key: string) => `${key}=${REDACTED}`); + out = out.replace(BEARER_RE, (_m, prefix: string) => `${prefix}${REDACTED}`); + return out; +} + +/** + * `String(err)` for a log line, with redaction applied. Use this instead of + * `String(err)` anywhere an OAuth error can reach a logger. + */ +export function redactedErrorText( + err: unknown, + secrets: readonly (string | null | undefined)[] = [], +): string { + const raw = err instanceof Error ? `${err.name}: ${err.message}` : String(err); + return redactSecrets(raw, secrets); +} diff --git a/middleware/test/mcpOAuth.test.ts b/middleware/test/mcpOAuth.test.ts index f938c329..dd051290 100644 --- a/middleware/test/mcpOAuth.test.ts +++ b/middleware/test/mcpOAuth.test.ts @@ -5,6 +5,14 @@ import { createHash } from 'node:crypto'; import { McpAuthDiscovery } from '../src/services/mcpAuthDiscovery.js'; import { McpOAuthClient, type OAuthClientCredentials } from '../src/services/mcpOAuthClient.js'; import type { AuthServerMetadata } from '../src/services/mcpAuthDiscovery.js'; +import { + SERVICE_USER_KEY, + UNRESOLVED_IDENTITY, + auditIdentity, + parseDelegation, + resolveMcpUserKey, +} from '../src/services/mcpDelegation.js'; +import { redactSecrets, redactedErrorText } from '../src/services/secretRedaction.js'; function jsonResponder(routes: Record): typeof fetch { return (async (input: RequestInfo | URL) => { @@ -112,6 +120,7 @@ const AS: AuthServerMetadata = { codeChallengeMethods: ['S256'], grantTypes: ['authorization_code'], scopesSupported: ['read'], + issParameterSupported: false, }; const CLIENT: OAuthClientCredentials = { clientId: 'cid', clientSecret: 'sec' }; @@ -258,3 +267,567 @@ describe('McpOAuthService.describeAuth (broker classification)', () => { assert.equal((await svc.describeAuth(server)).protected, false); }); }); + +// ───────────────────────────────────────────────────────────────────────────── +// W0-1 — three live defects in the MCP OAuth path +// D1 no RFC 9207 `iss` validation at the callback +// D2 silent 'operator' fallback (confused deputy) +// D3 unbounded concurrent refreshes for the same (server, user) +// ───────────────────────────────────────────────────────────────────────────── + +/** An in-memory stand-in for the parts of AgentGraphStore the OAuth service + * touches. Records writes so a test can assert that a REJECTED callback + * persisted nothing. */ +function fakeGraph(opts?: { + flow?: Record; + token?: Record | undefined; + client?: { clientId: string; clientSecretRef: string | null } | null; +}): { + graph: never; + tokenWrites: Record[]; + tokenDeletes: { serverId: string; userKey: string }[]; + flowCreates: Record[]; +} { + const tokenWrites: Record[] = []; + const tokenDeletes: { serverId: string; userKey: string }[] = []; + const flowCreates: Record[] = []; + let flow = opts?.flow; + const graph = { + // One-shot, like the real DELETE … RETURNING. + takeMcpOAuthFlow: async (state: string) => { + if (!flow || flow['state'] !== state) return undefined; + const taken = flow; + flow = undefined; + return taken; + }, + createMcpOAuthFlow: async (input: Record) => { + flowCreates.push(input); + }, + getMcpOAuthToken: async () => opts?.token, + upsertMcpOAuthToken: async (input: Record) => { + tokenWrites.push(input); + }, + deleteMcpOAuthToken: async (serverId: string, userKey: string) => { + tokenDeletes.push({ serverId, userKey }); + }, + getMcpOAuthClient: async () => + opts?.client === undefined ? { clientId: 'cid', clientSecretRef: null } : opts.client, + upsertMcpOAuthClient: async () => {}, + } as never; + return { graph, tokenWrites, tokenDeletes, flowCreates }; +} + +const FLOW_BASE = { + state: 'ST', + serverId: 'srv-1', + userKey: 'user-a', + issuer: 'https://as.example', + codeVerifier: 'VERIFIER-VALUE', + redirectUri: 'https://host/cb', + scopes: 'read', + tokenEndpoint: 'https://as.example/token', + authorizationEndpoint: 'https://as.example/authorize', + issRequired: false, +}; + +describe('W0-1 D1 — RFC 9207 iss validation at the OAuth callback', () => { + it('parses authorization_response_iss_parameter_supported from AS metadata', async () => { + const fetchImpl = jsonResponder({ + '/.well-known/oauth-protected-resource': { + authorization_servers: ['https://as.example'], + }, + '/.well-known/oauth-authorization-server': { + issuer: 'https://as.example', + authorization_endpoint: 'https://as.example/authorize', + token_endpoint: 'https://as.example/token', + authorization_response_iss_parameter_supported: true, + }, + }); + const out = await new McpAuthDiscovery({ fetchImpl }).discover('https://as.example/mcp'); + assert.equal(out?.server.issParameterSupported, true); + }); + + it('defaults issParameterSupported to false when the AS does not advertise it', async () => { + const fetchImpl = jsonResponder({ + '/.well-known/oauth-protected-resource': { + authorization_servers: ['https://as.example'], + }, + '/.well-known/oauth-authorization-server': { + issuer: 'https://as.example', + authorization_endpoint: 'https://as.example/authorize', + token_endpoint: 'https://as.example/token', + }, + }); + const out = await new McpAuthDiscovery({ fetchImpl }).discover('https://as.example/mcp'); + assert.equal(out?.server.issParameterSupported, false); + }); + + it('records the AS iss support on the flow at authorize time (not re-discovered later)', async () => { + const { McpOAuthService } = await import('../src/services/mcpOAuthService.js'); + const { graph, flowCreates } = fakeGraph(); + const discovery = { + discover: async () => ({ + resource: { + resource: 'https://srv.example', + authorizationServers: ['https://as.example'], + scopesSupported: ['read'], + bearerMethods: ['header'], + }, + server: { ...AS, issParameterSupported: true }, + }), + } as never; + const svc = new McpOAuthService({ + graph, + vault: { get: async () => undefined, set: async () => {} } as never, + redirectUri: 'https://host/cb', + discovery, + }); + await svc.beginAuthorization( + { id: 'srv-1', name: 'srv', endpoint: 'https://srv.example/mcp', transport: 'http' } as never, + 'user-a', + ); + assert.equal(flowCreates.length, 1); + assert.equal(flowCreates[0]?.['issRequired'], true); + assert.equal(flowCreates[0]?.['issuer'], 'https://as.example'); + }); + + it('accepts a matching iss and stores the token', async () => { + const { McpOAuthService } = await import('../src/services/mcpOAuthService.js'); + const { graph, tokenWrites } = fakeGraph({ flow: { ...FLOW_BASE, issRequired: true } }); + const client = { + exchangeCode: async () => ({ + accessToken: 'AT', + refreshToken: 'RT', + expiresInSec: 3600, + scope: 'read', + }), + } as never; + const svc = new McpOAuthService({ + graph, + vault: { get: async () => undefined, set: async () => {} } as never, + redirectUri: 'https://host/cb', + client, + }); + const out = await svc.completeAuthorization('ST', 'CODE', 'https://as.example'); + assert.equal(out.serverId, 'srv-1'); + assert.equal(tokenWrites.length, 1, 'a valid callback stores exactly one token'); + // AC3: the token is bound to the issuer that minted it. + assert.equal(tokenWrites[0]?.['issuer'], 'https://as.example'); + }); + + it('tolerates a single trailing slash difference in the issuer (RFC 9207 §2.4)', async () => { + const { McpOAuthService } = await import('../src/services/mcpOAuthService.js'); + const { graph, tokenWrites } = fakeGraph({ flow: { ...FLOW_BASE, issRequired: true } }); + const client = { + exchangeCode: async () => ({ + accessToken: 'AT', + refreshToken: null, + expiresInSec: null, + scope: null, + }), + } as never; + const svc = new McpOAuthService({ + graph, + vault: { get: async () => undefined, set: async () => {} } as never, + redirectUri: 'https://host/cb', + client, + }); + await svc.completeAuthorization('ST', 'CODE', 'https://as.example/'); + assert.equal(tokenWrites.length, 1); + }); + + it('REJECTS a mismatched iss and persists nothing', async () => { + const { McpOAuthService, McpOAuthIssuerMismatchError } = await import( + '../src/services/mcpOAuthService.js' + ); + const { graph, tokenWrites } = fakeGraph({ flow: { ...FLOW_BASE } }); + let exchanged = false; + const client = { + exchangeCode: async () => { + exchanged = true; + return { accessToken: 'AT', refreshToken: 'RT', expiresInSec: 3600, scope: null }; + }, + } as never; + const vaultWrites: string[] = []; + const svc = new McpOAuthService({ + graph, + vault: { + get: async () => undefined, + set: async (_ns: string, k: string) => { + vaultWrites.push(k); + }, + } as never, + redirectUri: 'https://host/cb', + client, + }); + await assert.rejects( + () => svc.completeAuthorization('ST', 'CODE', 'https://evil.example'), + (err: unknown) => err instanceof McpOAuthIssuerMismatchError, + ); + // The whole point: hard rejection BEFORE the exchange, so no code leaves + // and no credential lands anywhere. + assert.equal(exchanged, false, 'the code must never be exchanged on a mismatch'); + assert.equal(tokenWrites.length, 0, 'no token row may be written'); + assert.deepEqual(vaultWrites, [], 'no secret may be written to the vault'); + }); + + it('REJECTS an absent iss when the AS advertised support for it', async () => { + const { McpOAuthService, McpOAuthIssuerMismatchError } = await import( + '../src/services/mcpOAuthService.js' + ); + const { graph, tokenWrites } = fakeGraph({ flow: { ...FLOW_BASE, issRequired: true } }); + let exchanged = false; + const client = { + exchangeCode: async () => { + exchanged = true; + return { accessToken: 'AT', refreshToken: null, expiresInSec: null, scope: null }; + }, + } as never; + const svc = new McpOAuthService({ + graph, + vault: { get: async () => undefined, set: async () => {} } as never, + redirectUri: 'https://host/cb', + client, + }); + await assert.rejects( + () => svc.completeAuthorization('ST', 'CODE', null), + (err: unknown) => + err instanceof McpOAuthIssuerMismatchError && err.received === null, + ); + assert.equal(exchanged, false); + assert.equal(tokenWrites.length, 0); + }); + + it('accepts an absent iss when the AS never advertised support (backward compatible)', async () => { + const { McpOAuthService } = await import('../src/services/mcpOAuthService.js'); + const { graph, tokenWrites } = fakeGraph({ flow: { ...FLOW_BASE, issRequired: false } }); + const client = { + exchangeCode: async () => ({ + accessToken: 'AT', + refreshToken: null, + expiresInSec: null, + scope: null, + }), + } as never; + const svc = new McpOAuthService({ + graph, + vault: { get: async () => undefined, set: async () => {} } as never, + redirectUri: 'https://host/cb', + client, + }); + await svc.completeAuthorization('ST', 'CODE', null); + assert.equal(tokenWrites.length, 1, 'pre-RFC-9207 providers keep working'); + }); + + it('treats a blank iss as absent rather than as a mismatch', async () => { + const { McpOAuthService } = await import('../src/services/mcpOAuthService.js'); + const { graph, tokenWrites } = fakeGraph({ flow: { ...FLOW_BASE, issRequired: false } }); + const client = { + exchangeCode: async () => ({ + accessToken: 'AT', + refreshToken: null, + expiresInSec: null, + scope: null, + }), + } as never; + const svc = new McpOAuthService({ + graph, + vault: { get: async () => undefined, set: async () => {} } as never, + redirectUri: 'https://host/cb', + client, + }); + await svc.completeAuthorization('ST', 'CODE', ' '); + assert.equal(tokenWrites.length, 1); + }); +}); + +describe('W0-1 D2 — delegation: fail closed instead of borrowing the operator identity', () => { + it('per_user + resolvable identity → that identity', () => { + assert.equal(resolveMcpUserKey({ delegation: 'per_user' }, 'alice@example.com'), 'alice@example.com'); + }); + + it('per_user + UNRESOLVABLE identity → null (never the operator)', () => { + for (const candidate of [null, undefined, '', ' ']) { + const resolved = resolveMcpUserKey({ delegation: 'per_user' }, candidate); + assert.equal(resolved, null, `candidate ${JSON.stringify(candidate)} must not resolve`); + assert.notEqual(resolved, SERVICE_USER_KEY); + assert.notEqual(resolved, 'operator'); + } + }); + + it('service delegation is the explicit opt-in that keeps a shared identity', () => { + assert.equal(resolveMcpUserKey({ delegation: 'service' }, null), SERVICE_USER_KEY); + // Grandfathered rows must keep resolving to the historical literal, or + // migration 0031 would silently orphan their stored tokens. + assert.equal(SERVICE_USER_KEY, 'operator'); + }); + + it('service delegation ignores a caller identity (one shared token by design)', () => { + assert.equal(resolveMcpUserKey({ delegation: 'service' }, 'alice@example.com'), SERVICE_USER_KEY); + }); + + it('audit identity is never blank — an unattributable call is recorded as such', () => { + assert.equal(auditIdentity({ delegation: 'per_user' }, null), UNRESOLVED_IDENTITY); + assert.equal(auditIdentity({ delegation: 'per_user' }, 'bob'), 'bob'); + assert.equal(auditIdentity({ delegation: 'service' }, null), SERVICE_USER_KEY); + }); + + it('parseDelegation rejects anything outside the CHECK constraint', () => { + assert.equal(parseDelegation('per_user'), 'per_user'); + assert.equal(parseDelegation('service'), 'service'); + for (const bad of ['operator', 'PER_USER', '', null, undefined, 1, {}]) { + assert.equal(parseDelegation(bad), null, `${JSON.stringify(bad)} must not parse`); + } + }); + + it('an unresolved per_user identity yields NO token from the service', async () => { + const { McpOAuthService } = await import('../src/services/mcpOAuthService.js'); + // A token DOES exist under the shared key — the old code would have found + // and used it. Resolution must never reach this call. + let lookups = 0; + const graph = { + getMcpOAuthToken: async () => { + lookups += 1; + return { accessTokenRef: 'ref', refreshTokenRef: null, expiresAt: null, scopes: null, issuer: null }; + }, + } as never; + const svc = new McpOAuthService({ + graph, + vault: { get: async () => 'SHARED-OPERATOR-TOKEN', set: async () => {} } as never, + redirectUri: 'https://host/cb', + }); + const server = { id: 'srv-1', name: 'srv', delegation: 'per_user' as const }; + const userKey = resolveMcpUserKey(server, undefined); + assert.equal(userKey, null); + // The production call sites short-circuit on null, so the operator's token + // is never even looked up. + const token = userKey === null ? null : await svc.getValidAccessToken(server as never, userKey); + assert.equal(token, null); + assert.equal(lookups, 0, 'the shared token must not be consulted at all'); + }); +}); + +describe('W0-1 D3 — concurrent refresh is single-flight (MUTATION-CHECKED)', () => { + /** Build a service whose refresh path goes over a real McpOAuthClient, so the + * assertion counts genuine HTTP requests to the token endpoint rather than + * mock invocations. */ + async function refreshHarness(): Promise<{ + svc: import('../src/services/mcpOAuthService.js').McpOAuthService; + server: never; + tokenPosts: () => number; + tokenWrites: Record[]; + }> { + const { McpOAuthService } = await import('../src/services/mcpOAuthService.js'); + let tokenPosts = 0; + const fetchImpl: typeof fetch = (async (input: RequestInfo | URL) => { + const url = String(input); + if (url === 'https://as.example/token') { + tokenPosts += 1; + // Rotating refresh token, as OAuth 2.1 recommends — this is precisely + // what makes a lost race destructive. + return new Response( + JSON.stringify({ + access_token: `AT-${String(tokenPosts)}`, + refresh_token: `RT-${String(tokenPosts)}`, + expires_in: 3600, + }), + { status: 200, headers: { 'content-type': 'application/json' } }, + ); + } + return new Response('not found', { status: 404 }); + }) as typeof fetch; + + const tokenWrites: Record[] = []; + const graph = { + // Expired 60s ago → inside the refresh margin, so every caller wants a refresh. + getMcpOAuthToken: async () => ({ + serverId: 'srv-1', + userKey: 'user-a', + accessTokenRef: 'token/srv-1/user-a/access', + refreshTokenRef: 'token/srv-1/user-a/refresh', + expiresAt: new Date(Date.now() - 60_000), + scopes: 'read', + issuer: 'https://as.example', + }), + upsertMcpOAuthToken: async (input: Record) => { + tokenWrites.push(input); + }, + getMcpOAuthClient: async () => ({ clientId: 'cid', clientSecretRef: null }), + deleteMcpOAuthToken: async () => {}, + } as never; + const vaultStore = new Map([ + ['token/srv-1/user-a/access', 'STALE-AT'], + ['token/srv-1/user-a/refresh', 'RT-0'], + ]); + const vault = { + get: async (_ns: string, k: string) => vaultStore.get(k), + set: async (_ns: string, k: string, v: string) => { + vaultStore.set(k, v); + }, + } as never; + const discovery = { + discover: async () => ({ + resource: { + resource: 'https://srv.example', + authorizationServers: ['https://as.example'], + scopesSupported: ['read'], + bearerMethods: ['header'], + }, + server: AS, + }), + } as never; + const svc = new McpOAuthService({ + graph, + vault, + redirectUri: 'https://host/cb', + discovery, + client: new McpOAuthClient({ fetchImpl }), + }); + const server = { + id: 'srv-1', + name: 'srv', + endpoint: 'https://srv.example/mcp', + transport: 'http', + delegation: 'per_user', + } as never; + return { svc, server, tokenPosts: () => tokenPosts, tokenWrites }; + } + + it('issues exactly ONE token-endpoint HTTP request for N concurrent callers', async () => { + const { svc, server, tokenPosts, tokenWrites } = await refreshHarness(); + const N = 8; + const results = await Promise.all( + Array.from({ length: N }, () => svc.getValidAccessToken(server, 'user-a')), + ); + // THE mutation check: remove the in-flight map and this becomes 8. + // Counting mock calls would not prove this — the count is of real HTTP + // requests made through fetch to the token endpoint. + assert.equal(tokenPosts(), 1, `expected exactly 1 token request, got ${String(tokenPosts())}`); + // One refresh ⇒ one persisted rotation. N writes would mean N-1 of them + // stored a refresh token the AS had already retired. + assert.equal(tokenWrites.length, 1, 'exactly one token rotation may be persisted'); + // Every caller gets the same live token — nobody is handed a loser's result. + assert.deepEqual(new Set(results), new Set(['AT-1'])); + }); + + it('a later refresh is not blocked by the completed one (the map is cleared)', async () => { + const { svc, server, tokenPosts } = await refreshHarness(); + await svc.getValidAccessToken(server, 'user-a'); + await svc.getValidAccessToken(server, 'user-a'); + assert.equal(tokenPosts(), 2, 'sequential refreshes must each do their own request'); + }); + + it('different users do not share one refresh', async () => { + const { svc, server, tokenPosts } = await refreshHarness(); + await Promise.all([ + svc.getValidAccessToken(server, 'user-a'), + svc.getValidAccessToken(server, 'user-b'), + ]); + assert.equal(tokenPosts(), 2, 'the in-flight key must include the user'); + }); + + it('drops a stored token whose issuer has rotated instead of replaying it', async () => { + const { McpOAuthService } = await import('../src/services/mcpOAuthService.js'); + let tokenPosts = 0; + const fetchImpl: typeof fetch = (async () => { + tokenPosts += 1; + return new Response(JSON.stringify({ access_token: 'AT' }), { + status: 200, + headers: { 'content-type': 'application/json' }, + }); + }) as typeof fetch; + const deletes: { serverId: string; userKey: string }[] = []; + const graph = { + getMcpOAuthToken: async () => ({ + accessTokenRef: 'a', + refreshTokenRef: 'r', + expiresAt: new Date(Date.now() - 60_000), + scopes: null, + // Minted by the OLD issuer. + issuer: 'https://old-as.example', + }), + upsertMcpOAuthToken: async () => {}, + getMcpOAuthClient: async () => ({ clientId: 'cid', clientSecretRef: null }), + deleteMcpOAuthToken: async (serverId: string, userKey: string) => { + deletes.push({ serverId, userKey }); + }, + } as never; + const svc = new McpOAuthService({ + graph, + vault: { + get: async (_ns: string, k: string) => (k === 'r' ? 'RT' : 'STALE-AT'), + set: async () => {}, + } as never, + redirectUri: 'https://host/cb', + // Discovery now reports a DIFFERENT issuer. + discovery: { + discover: async () => ({ + resource: { + resource: 'https://srv.example', + authorizationServers: ['https://as.example'], + scopesSupported: [], + bearerMethods: [], + }, + server: AS, + }), + } as never, + client: new McpOAuthClient({ fetchImpl }), + }); + const token = await svc.getValidAccessToken( + { id: 'srv-1', name: 'srv', endpoint: 'https://srv.example/mcp', transport: 'http' } as never, + 'user-a', + ); + assert.equal(token, null, 'a token from a rotated issuer must not be usable'); + assert.equal(tokenPosts, 0, 'the old refresh token must not be sent to the new issuer'); + assert.deepEqual(deletes, [{ serverId: 'srv-1', userKey: 'user-a' }]); + }); +}); + +describe('W0-1 D5 — no token, code, or code_verifier can reach a log line', () => { + it('redacts an exact secret value wherever it appears', () => { + const out = redactSecrets('refresh failed for RT-abcdefgh12345 (retry)', ['RT-abcdefgh12345']); + assert.ok(!out.includes('RT-abcdefgh12345'), out); + assert.ok(out.includes('[redacted]')); + }); + + it('redacts a secret the provider echoed back URL-encoded', () => { + const secret = 'tok/with+special=chars'; + const out = redactSecrets(`error: value=${encodeURIComponent(secret)}`, [secret]); + assert.ok(!out.includes(encodeURIComponent(secret)), out); + }); + + it('redacts token fields in a JSON error body we did not mint', () => { + const body = '{"error":"invalid_grant","access_token":"AT-SECRET-1","refresh_token":"RT-SECRET-2"}'; + const out = redactSecrets(body); + assert.ok(!out.includes('AT-SECRET-1'), out); + assert.ok(!out.includes('RT-SECRET-2'), out); + // Non-secret diagnostics must survive, or the log becomes useless. + assert.ok(out.includes('invalid_grant'), out); + }); + + it('redacts code and code_verifier from a query string or form body', () => { + const out = redactSecrets( + 'POST https://as.example/token?code=THE-AUTH-CODE&code_verifier=THE-VERIFIER&client_id=cid', + ); + assert.ok(!out.includes('THE-AUTH-CODE'), out); + assert.ok(!out.includes('THE-VERIFIER'), out); + assert.ok(out.includes('client_id=cid'), 'client_id is not a secret'); + }); + + it('redacts a bearer token echoed in an error', () => { + const out = redactSecrets('upstream said: Authorization: Bearer eyJhbGciOi.SECRET.PART'); + assert.ok(!out.includes('eyJhbGciOi.SECRET.PART'), out); + }); + + it('redactedErrorText never leaks the refresh token from a thrown Error', () => { + const err = new Error('token endpoint rejected refresh_token=RT-LEAKY-VALUE for client cid'); + const out = redactedErrorText(err, ['RT-LEAKY-VALUE']); + assert.ok(!out.includes('RT-LEAKY-VALUE'), out); + assert.ok(out.includes('Error:'), 'the error class stays visible for debugging'); + }); + + it('leaves short values alone rather than shredding unrelated text', () => { + // A 3-char "secret" would otherwise redact every occurrence of those chars. + assert.equal(redactSecrets('the cat sat', ['cat']), 'the cat sat'); + }); +}); diff --git a/web-ui/app/_components/mcp/McpAuthSection.tsx b/web-ui/app/_components/mcp/McpAuthSection.tsx index e2e22d0d..72da9a28 100644 --- a/web-ui/app/_components/mcp/McpAuthSection.tsx +++ b/web-ui/app/_components/mcp/McpAuthSection.tsx @@ -9,6 +9,7 @@ import { disconnectMcpServer, getMcpAuthStatus, setMcpOAuthClient, + setMcpServerDelegation, type McpAuthStatus, } from '@/app/_lib/agentBuilder'; @@ -87,6 +88,26 @@ export function McpAuthSection({ } } + /** Flip the delegation mode (W0-1). Surfaced here because it decides WHOSE + * authorization every call to this server uses — the same question the rest + * of this panel is about. */ + async function toggleDelegation(): Promise { + if (!status?.delegation) return; + setBusy(true); + setError(null); + try { + await setMcpServerDelegation( + serverId, + status.delegation === 'per_user' ? 'service' : 'per_user', + ); + await refresh(); + } catch (err) { + setError(errText(err)); + } finally { + setBusy(false); + } + } + async function saveClient(): Promise { if (!status?.issuer || clientId.trim() === '') return; setBusy(true); @@ -143,6 +164,33 @@ export function McpAuthSection({ : t('auth.hintDelegated', { host: status.issuerHost ?? status.issuer ?? '?' })}
) : null} + {status.delegation ? ( +
+
+ {t('auth.delegationLabel')}: + + {status.delegation === 'per_user' + ? t('auth.delegationPerUser') + : t('auth.delegationService')} + + +
+
+ {status.delegation === 'per_user' + ? t('auth.delegationPerUserWhy') + : t('auth.delegationServiceWhy')} +
+ {status.delegation === 'per_user' && status.identityResolved === false ? ( +
+ {t('auth.delegationIdentityMissing')} +
+ ) : null} +
+ ) : null} {showClientForm ? (
diff --git a/web-ui/app/_lib/agentBuilder.ts b/web-ui/app/_lib/agentBuilder.ts index 433a3769..8b34ef76 100644 --- a/web-ui/app/_lib/agentBuilder.ts +++ b/web-ui/app/_lib/agentBuilder.ts @@ -856,6 +856,11 @@ export async function importMcpServerFromRegistry( // ── Generic MCP OAuth (issue #459 W9) ──────────────────────────────────────── +/** Whose authority MCP calls to a server act under (W0-1). `per_user` requires + * each caller to have its own identity and fails closed without one; + * `service` is the explicit opt-in to one shared identity. */ +export type McpDelegation = 'per_user' | 'service'; + export interface McpAuthStatus { protected: boolean; connected: boolean; @@ -865,6 +870,23 @@ export interface McpAuthStatus { brokered?: boolean; needsClient: boolean; redirectUri?: string; + /** W0-1 — the server's delegation mode. */ + delegation?: McpDelegation; + /** W0-1 — whether this session has an identity to act as. False on a + * `per_user` server means every call fails closed until an identity is + * available or the operator opts into `service` delegation. */ + identityResolved?: boolean; +} + +/** Switch a server's delegation mode (W0-1). */ +export async function setMcpServerDelegation( + serverId: string, + delegation: McpDelegation, +): Promise<{ id: string; delegation: McpDelegation }> { + return callJson(`/v1/operator/mcp-servers/${encodeURIComponent(serverId)}/delegation`, { + method: 'PUT', + body: JSON.stringify({ delegation }), + }); } export async function getMcpAuthStatus(serverId: string): Promise { diff --git a/web-ui/messages/de.json b/web-ui/messages/de.json index 70eaf4af..af15a18d 100644 --- a/web-ui/messages/de.json +++ b/web-ui/messages/de.json @@ -2862,7 +2862,15 @@ "readyToLogin": "Client gespeichert. Melde dich jetzt bei {host} an, um den Zugriff zu autorisieren.", "continueLogin": "Weiter zum Login bei {host} ↗", "afterLoginHint": "Öffnet in einem neuen Tab. Nach der Bestätigung hierher zurückkehren — der Status aktualisiert sich automatisch.", - "checkStatus": "Status prüfen" + "checkStatus": "Status prüfen", + "delegationLabel": "Handelnde Identität", + "delegationPerUser": "Jeder Nutzer handelt als er selbst", + "delegationService": "Eine gemeinsame Service-Identität", + "delegationPerUserWhy": "Jeder Aufrufer nutzt an diesem Server seine eigene Autorisierung. Eine Unterhaltung ohne zugeordnete Nutzeridentität wird abgewiesen und greift nicht auf deine zurück.", + "delegationServiceWhy": "Alle Aufrufer teilen eine Autorisierung — deine. Wer einen Orchestrator mit diesem Server erreicht, handelt mit dieser Berechtigung. Behalte das nur für Server, bei denen eine gemeinsame Identität gewollt ist.", + "delegationSwitchToPerUser": "Identität pro Nutzer verlangen", + "delegationSwitchToService": "Gemeinsame Service-Identität nutzen", + "delegationIdentityMissing": "Diese Sitzung hat keine zugeordnete Nutzeridentität — Aufrufe pro Nutzer an diesen Server schlagen deshalb bewusst fehl. Es wird nichts an den Server gesendet." }, "marketplace": { "registry": "Registry", diff --git a/web-ui/messages/en.json b/web-ui/messages/en.json index e6399178..31dffaae 100644 --- a/web-ui/messages/en.json +++ b/web-ui/messages/en.json @@ -2862,7 +2862,15 @@ "readyToLogin": "Client saved. Now sign in at {host} to authorize access.", "continueLogin": "Continue to {host} login ↗", "afterLoginHint": "Opens in a new tab. After you approve, come back here — the status refreshes automatically.", - "checkStatus": "Check status" + "checkStatus": "Check status", + "delegationLabel": "Acting identity", + "delegationPerUser": "Each user acts as themselves", + "delegationService": "One shared service identity", + "delegationPerUserWhy": "Every caller uses their own authorization at this server. A conversation with no mapped user identity is refused rather than falling back to yours.", + "delegationServiceWhy": "All callers share one authorization — yours. Anyone who can reach an orchestrator with this server granted acts with that authority. Only keep this for servers where a shared identity is intended.", + "delegationSwitchToPerUser": "Require per-user identity", + "delegationSwitchToService": "Use a shared service identity", + "delegationIdentityMissing": "This session has no mapped user identity, so per-user calls to this server fail closed. Nothing is sent to the server." }, "marketplace": { "registry": "Registry", From 2575db6526b8760a0820387ad70cd029b42676cc Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 09:00:10 +0200 Subject: [PATCH 05/90] feat(orchestrator): race every tool dispatch against a per-tool deadline One hung sub-agent used to pin the whole Promise.allSettled batch for the rest of the turn: domainQueryTool awaits agent.ask() with no abort and no timeout, and there was no per-tool deadline anywhere in the orchestrator. dispatchTool now races an AbortSignal-backed deadline (default 120s, OMADIA_TOOL_DISPATCH_TIMEOUT_MS, 0 disables) and returns a structured Error: result on timeout. The abandoned dispatch is marked aborted, so a late result is discarded before the first write into turn state (raw-result capture, canvas sentinel, KG ingestion, privacy interning) and late sub-agent events are dropped by an abort-guarded observer. --- .../harness-orchestrator/src/orchestrator.ts | 144 ++++++++++++++++++ 1 file changed, 144 insertions(+) diff --git a/middleware/packages/harness-orchestrator/src/orchestrator.ts b/middleware/packages/harness-orchestrator/src/orchestrator.ts index 8f0691f2..7e98146f 100644 --- a/middleware/packages/harness-orchestrator/src/orchestrator.ts +++ b/middleware/packages/harness-orchestrator/src/orchestrator.ts @@ -1292,6 +1292,85 @@ function mcpObservationDigest(raw: string): string { return `(${String(Buffer.byteLength(raw, 'utf8'))} bytes, values masked)`; } +/** + * Per-tool dispatch deadline (W0-2). Every tool of an iteration is dispatched + * into one `Promise.allSettled` (non-streaming) / race loop (streaming), so a + * single sub-agent that never returns used to pin the WHOLE parallel batch for + * the rest of the turn — there was no per-tool timeout anywhere. + * + * 120s is deliberately generous: a domain sub-agent runs its own multi-iteration + * LLM loop with its own tool calls, so p99 legitimately reaches tens of seconds. + * Operators whose Odoo/Confluence sub-agents run longer raise it via + * `OMADIA_TOOL_DISPATCH_TIMEOUT_MS`; `0` disables the deadline entirely. + */ +const DEFAULT_TOOL_DISPATCH_TIMEOUT_MS = 120_000; +const TOOL_DISPATCH_TIMEOUT_ENV = 'OMADIA_TOOL_DISPATCH_TIMEOUT_MS'; + +/** Resolved per dispatch (not cached at module load) so an operator env change + * applies to the next turn without a restart. */ +function resolveToolDispatchTimeoutMs(): number { + const raw = process.env[TOOL_DISPATCH_TIMEOUT_ENV]; + if (raw === undefined || raw.trim() === '') { + return DEFAULT_TOOL_DISPATCH_TIMEOUT_MS; + } + const parsed = Number(raw); + if (!Number.isFinite(parsed) || parsed < 0) { + console.warn( + `[orchestrator] ${TOOL_DISPATCH_TIMEOUT_ENV}="${raw}" is not a non-negative number — using the ${String(DEFAULT_TOOL_DISPATCH_TIMEOUT_MS)}ms default.`, + ); + return DEFAULT_TOOL_DISPATCH_TIMEOUT_MS; + } + return parsed; +} + +/** Model-facing result for a tool that blew its deadline. `Error:`-prefixed so + * both dispatch loops key `is_error` off it exactly like any other failure. */ +function toolDeadlineError(name: string, timeoutMs: number): string { + const seconds = (timeoutMs / 1000).toFixed(timeoutMs % 1000 === 0 ? 0 : 1); + return `Error: tool \`${name}\` was aborted after exceeding its ${seconds}s dispatch deadline. Its result (if it ever arrives) is discarded. Continue without it or retry with a narrower request.`; +} + +/** Returned by the abandoned dispatch when it finally settles. Never reaches + * the model — the turn already took {@link toolDeadlineError} for this slot. */ +const TOOL_DISPATCH_DISCARDED = '__omadia_tool_dispatch_discarded__'; + +/** + * Wrap a slot observer so sub-agent events emitted AFTER the deadline are + * dropped. A sub-agent that keeps running past its abort would otherwise keep + * pushing `sub_tool_use`/`sub_tool_result` events into a turn that already + * moved on — the same late-write class the discarded result guards against. + */ +function abortGuardedObserver( + observer: AskObserver | undefined, + signal: AbortSignal, +): AskObserver | undefined { + if (observer === undefined) return undefined; + const gate = + (fn: ((ev: E) => void) | undefined): ((ev: E) => void) | undefined => + fn === undefined + ? undefined + : (ev: E): void => { + if (signal.aborted) return; + fn.call(observer, ev); + }; + const onIteration = gate(observer.onIteration); + const onSubToolUse = gate(observer.onSubToolUse); + const onSubToolResult = gate(observer.onSubToolResult); + const onIterationPhase = gate(observer.onIterationPhase); + const onTokenChunk = gate(observer.onTokenChunk); + const onIterationUsage = gate(observer.onIterationUsage); + const onIterationEnd = gate(observer.onIterationEnd); + return { + ...(onIteration ? { onIteration } : {}), + ...(onSubToolUse ? { onSubToolUse } : {}), + ...(onSubToolResult ? { onSubToolResult } : {}), + ...(onIterationPhase ? { onIterationPhase } : {}), + ...(onTokenChunk ? { onTokenChunk } : {}), + ...(onIterationUsage ? { onIterationUsage } : {}), + ...(onIterationEnd ? { onIterationEnd } : {}), + }; +} + export class Orchestrator { /** The Agent (orchestrator instance) this object serves. */ readonly agentId: string; @@ -4772,10 +4851,63 @@ export class Orchestrator { } } + /** + * W0-2 — every tool dispatch runs under a per-tool deadline. Without it a + * single hung sub-agent (`domainQueryTool` awaits `agent.ask()` with no + * abort) blocks the entire `Promise.allSettled` batch for the whole turn. + * + * On timeout the slot resolves with a structured `Error:` string and the + * abandoned dispatch is marked aborted, so when it eventually settles its + * result is DISCARDED instead of being written into a turn that moved on + * (raw-result capture, privacy interning, KG ingestion, sub-events). + * + * The deadline is per tool, not per batch: sibling tools in the same + * `allSettled` keep running and resolve normally. + */ private async dispatchTool( name: string, input: unknown, observer?: AskObserver, + ): Promise { + const timeoutMs = resolveToolDispatchTimeoutMs(); + if (timeoutMs === 0) { + // Deadline explicitly disabled by the operator — legacy behaviour. + return this.dispatchToolDeadlined(name, input, observer); + } + const controller = new AbortController(); + const work = this.dispatchToolDeadlined( + name, + input, + abortGuardedObserver(observer, controller.signal), + controller.signal, + ); + // A dispatch that rejects AFTER the deadline already resolved the race + // would otherwise surface as an unhandled rejection and kill the process. + work.catch(() => undefined); + let timer: ReturnType | undefined; + const deadline = new Promise((resolve) => { + timer = setTimeout(() => { + controller.abort(); + console.warn( + `[orchestrator.dispatchTool:${name}] exceeded the ${String(timeoutMs)}ms dispatch deadline — aborting this slot; siblings are unaffected.`, + ); + resolve(toolDeadlineError(name, timeoutMs)); + }, timeoutMs); + // Never hold the event loop open just to police a deadline. + timer.unref?.(); + }); + try { + return await Promise.race([work, deadline]); + } finally { + if (timer !== undefined) clearTimeout(timer); + } + } + + private async dispatchToolDeadlined( + name: string, + input: unknown, + observer?: AskObserver, + deadlineSignal?: AbortSignal, ): Promise { // Privacy Shield v4 — Data-Plane Boundary. The privacy handle is // threaded through `turnContext.privacyHandle`; absent ⇒ no privacy @@ -4828,6 +4960,18 @@ export class Orchestrator { } else { result = await this.dispatchToolInner(name, input, observer); } + // W0-2 — late-result firewall. The deadline already fired for this slot: + // the turn took `toolDeadlineError` and moved on. Everything below this + // line WRITES this result into turn state (raw-result capture, canvas + // sentinel tap, KG ingestion, privacy interning/bypass receipts), so a + // late arrival must be dropped HERE — before the first side effect — + // rather than merely being ignored by the caller. + if (deadlineSignal?.aborted === true) { + console.warn( + `[orchestrator.dispatchTool:${name}] result arrived after the dispatch deadline — discarded.`, + ); + return TOOL_DISPATCH_DISCARDED; + } // Phase C.2 — Raw tool-result capture. Outer scope (routine runner) // may install a callback that stashes the raw result keyed by tool // name; later template rendering uses it as the source of truth for From 20c3b993f752f15d8265b8febdf132a2e8e0da72 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 09:37:59 +0200 Subject: [PATCH 06/90] fix(mcp): state the callTool request policy and stop retrying Unauthorized callTool passed no RequestOptions and silently inherited the SDK's 60s default, so the real ceiling was undocumented and un-tunable. It now passes an explicit { timeout, resetTimeoutOnProgress, maxTotalTimeout } (env-tunable), where resetTimeoutOnProgress keeps long streaming calls alive and maxTotalTimeout is the absolute ceiling. looksTransient() also matched a bare -32001, contradicting its own contract: the code is implementation-defined and servers legitimately use it for Unauthorized (omadia's LoopbackMcpServer does), so a genuine auth failure got one doomed retry before surfacing. Auth now wins; a real SDK request timeout still retries via its "Request timed out" message. --- .../harness-orchestrator/src/mcp/mcpClient.ts | 42 ++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts index d3ed9d0c..5257b412 100644 --- a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts +++ b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts @@ -164,8 +164,15 @@ function looksUnauthorized(text: string): boolean { * (request timeout, dropped/closed connection, socket reset) — NOT an auth or * application-level tool error. */ function looksTransient(text: string): boolean { + // W0-5 — auth wins. `-32001` used to be matched as a bare numeric code here, + // which contradicted this function's own contract: the code is only + // *implementation-defined*, and servers legitimately use it for Unauthorized + // (omadia's own LoopbackMcpServer does, see its 401 branch). A genuine + // Unauthorized was therefore retried once — an extra doomed round trip that + // delayed the auth prompt. A real SDK request timeout still retries: it + // carries "Request timed out" and matches the timeout pattern below. + if (looksUnauthorized(text)) return false; return ( - /-?32001\b/.test(text) || /timed?\s*out|timeout/i.test(text) || /connection closed|connection reset|econnreset|socket hang ?up|network error|fetch failed|und_err/i.test( text, @@ -181,6 +188,29 @@ interface Pooled { const CLIENT_INFO = { name: 'omadia-agent-builder', version: '0.1.0' } as const; +/** + * W0-2 — explicit per-call MCP request policy. `callTool` used to pass no + * `RequestOptions` at all and silently inherited the SDK's 60s default, so the + * real ceiling was undocumented and un-tunable. Stated here instead: + * - `timeout`: idle budget for one request. + * - `resetTimeoutOnProgress`: a server streaming progress notifications keeps + * its budget alive (long Odoo/Confluence reports do exactly this)… + * - `maxTotalTimeout`: …but never past this absolute ceiling, so a chatty + * server cannot extend a call forever. + * Both are env-tunable per deployment; the orchestrator's own per-tool dispatch + * deadline (`OMADIA_TOOL_DISPATCH_TIMEOUT_MS`) is the outer bound. + */ +const DEFAULT_MCP_CALL_TIMEOUT_MS = 60_000; +const DEFAULT_MCP_CALL_MAX_TOTAL_TIMEOUT_MS = 180_000; + +function envMs(name: string, fallback: number): number { + const raw = process.env[name]; + if (raw === undefined || raw.trim() === '') return fallback; + const parsed = Number(raw); + if (!Number.isFinite(parsed) || parsed <= 0) return fallback; + return parsed; +} + export class McpManager { private readonly pool = new Map(); private readonly connecting = new Map>(); @@ -324,6 +354,16 @@ export class McpManager { // the failure surfaces to the model as "-32000 Connection closed", // making every tool call on that server look like a transport failure. LENIENT_CALL_TOOL_RESULT_SCHEMA, + // Stated request policy instead of the SDK's implicit 60s default — + // see DEFAULT_MCP_CALL_TIMEOUT_MS. + { + timeout: envMs('OMADIA_MCP_CALL_TIMEOUT_MS', DEFAULT_MCP_CALL_TIMEOUT_MS), + resetTimeoutOnProgress: true, + maxTotalTimeout: envMs( + 'OMADIA_MCP_CALL_MAX_TOTAL_TIMEOUT_MS', + DEFAULT_MCP_CALL_MAX_TOTAL_TIMEOUT_MS, + ), + }, ); const rendered = renderToolResult(res); // MCP protocol errors resolve (isError result) instead of throwing — From 5f24a5f42b71554a545adbc78235777f3d3d549b Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 09:43:32 +0200 Subject: [PATCH 07/90] test(orchestrator): prove the dispatch deadline discards late results MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The mutation check is captureRawToolResult: a real turn-state write the routine runner reads back. Verified by temporarily removing the deadlineSignal guard — the suite then fails on the capture assertion, not on a missing error string. Also covers batch siblings resolving normally, the 0-disables path, and a bad env value falling back to the default. --- .../orchestrator/toolDispatchDeadline.test.ts | 370 ++++++++++++++++++ 1 file changed, 370 insertions(+) create mode 100644 middleware/test/orchestrator/toolDispatchDeadline.test.ts diff --git a/middleware/test/orchestrator/toolDispatchDeadline.test.ts b/middleware/test/orchestrator/toolDispatchDeadline.test.ts new file mode 100644 index 00000000..a6277a60 --- /dev/null +++ b/middleware/test/orchestrator/toolDispatchDeadline.test.ts @@ -0,0 +1,370 @@ +import { describe, it, afterEach } from 'node:test'; +import { strict as assert } from 'node:assert'; + +import type { + LlmProvider, + LlmResponse, + LlmStreamEvent, +} from '@omadia/llm-provider'; +import type { ChatStreamEvent } from '@omadia/channel-sdk'; +import { + NativeToolRegistry, + Orchestrator, + turnContext, + type AskObserver, + type DomainTool, +} from '@omadia/orchestrator'; + +/** + * W0-2 — per-tool dispatch deadline. + * + * Before this, `dispatchTool` had no timeout anywhere: `domainQueryTool` awaits + * `agent.ask()` with no abort, and every tool of an iteration is dispatched into + * one `Promise.allSettled` / race loop. One hung sub-agent therefore pinned the + * whole batch for the rest of the turn. + * + * The load-bearing test here is the MUTATION CHECK: it is not enough that the + * timed-out slot returns an error — the abandoned dispatch's LATE result must + * never be written into the turn afterwards. `captureRawToolResult` is a real + * turn-state write (the routine runner reads it back as the source of truth for + * template data sections), so a late write is observable. Delete the + * `deadlineSignal?.aborted` guard in `dispatchToolDeadlined` and this test fails + * on the capture assertion, not merely on a missing error string. + */ + +const DEADLINE_MS = 150; +const LATE_VALUE = 'LATE-VALUE-from-abandoned-subagent'; +const FAST_VALUE = 'fast-tool-output'; + +const providerCapabilities = { + tools: true, + vision: true, + streaming: true, + promptCaching: true, + forcedToolChoice: true, + parallelToolCalls: true, +} as const; + +const usage = { + inputTokens: 10, + outputTokens: 2, + cacheReadTokens: 0, + cacheWriteTokens: 0, +} as const; + +function toolCallResponse( + toolUses: ReadonlyArray<{ id: string; name: string }>, +): LlmResponse { + return { + content: toolUses.map((u) => ({ + type: 'tool_call', + id: u.id, + name: u.name, + input: {}, + })), + finishReason: 'tool_calls', + providerFinishReason: 'tool_use', + model: 'test', + usage, + } as unknown as LlmResponse; +} + +function textResponse(text: string): LlmResponse { + return { + content: [{ type: 'text', text }], + finishReason: 'stop', + providerFinishReason: 'end_turn', + model: 'test', + usage, + } as unknown as LlmResponse; +} + +const sleep = (ms: number): Promise => + new Promise((r) => setTimeout(r, ms)); + +/** + * Scripted provider. `completeDelays[i]` / `streamDelays[i]` hold the i-th + * call open, which keeps the turn LIVE while the abandoned dispatch settles — + * without that window a late write could not be observed at all and the + * mutation check would be vacuous. + */ +function fakeProvider( + responses: readonly LlmResponse[], + delaysMs: readonly number[] = [], +): LlmProvider { + let idx = 0; + const next = async (): Promise => { + const i = idx; + idx += 1; + const response = responses[i]; + if (!response) { + throw new Error(`fakeProvider: no scripted response for call ${String(i + 1)}`); + } + const delay = delaysMs[i] ?? 0; + if (delay > 0) await sleep(delay); + return response; + }; + const provider = { + id: 'anthropic', + capabilities: providerCapabilities, + complete: next, + stream: (): AsyncIterable => ({ + async *[Symbol.asyncIterator]() { + const response = await next(); + yield { type: 'final', response } as LlmStreamEvent; + }, + }), + classifyError: () => ({ retryable: false, kind: 'other' as const }), + }; + return provider as unknown as LlmProvider; +} + +const minimalSpec = (name: string): Record => ({ + name, + description: `${name} for testing`, + input_schema: { type: 'object' as const, properties: {}, required: [] }, +}); + +interface SlowToolProbe { + readonly settledLate: () => boolean; + readonly lateObserverCalls: () => number; + readonly tool: DomainTool; +} + +/** + * A sub-agent that ignores the deadline entirely — the real-world case this + * unit exists for. It resolves long after the deadline AND emits a sub-agent + * event on its way out, exercising both late-write vectors. + */ +function slowDomainTool(name: string, latencyMs: number): SlowToolProbe { + let settled = false; + let lateEmits = 0; + const tool: DomainTool = { + name, + domain: 'test.slow', + spec: minimalSpec(name) as unknown as DomainTool['spec'], + async handle(_input: unknown, observer?: AskObserver): Promise { + observer?.onIteration?.({ iteration: 1 }); + await sleep(latencyMs); + // Everything below happens AFTER the deadline fired for this slot. + lateEmits += 1; + observer?.onSubToolResult?.({ + id: 'late-sub-call', + output: LATE_VALUE, + durationMs: latencyMs, + isError: false, + }); + settled = true; + return LATE_VALUE; + }, + }; + return { + settledLate: () => settled, + lateObserverCalls: () => lateEmits, + tool, + }; +} + +function buildOrchestrator( + provider: LlmProvider, + registry: NativeToolRegistry, + domainTools: DomainTool[], +): Orchestrator { + return new Orchestrator({ + provider, + model: 'test', + maxTokens: 1024, + maxToolIterations: 5, + domainTools, + nativeToolRegistry: registry, + }); +} + +function fastToolRegistry(): NativeToolRegistry { + const registry = new NativeToolRegistry(); + registry.register('fast_tool', { + handler: async (): Promise => { + await sleep(10); + return FAST_VALUE; + }, + // eslint-disable-next-line @typescript-eslint/no-explicit-any + spec: minimalSpec('fast_tool') as any, + }); + return registry; +} + +const originalTimeout = process.env['OMADIA_TOOL_DISPATCH_TIMEOUT_MS']; + +afterEach(() => { + if (originalTimeout === undefined) { + delete process.env['OMADIA_TOOL_DISPATCH_TIMEOUT_MS']; + } else { + process.env['OMADIA_TOOL_DISPATCH_TIMEOUT_MS'] = originalTimeout; + } +}); + +describe('Orchestrator per-tool dispatch deadline (W0-2)', () => { + it('times out the hung tool, keeps its batch siblings, and DISCARDS the late result', async () => { + process.env['OMADIA_TOOL_DISPATCH_TIMEOUT_MS'] = String(DEADLINE_MS); + const probe = slowDomainTool('query_slow_agent', DEADLINE_MS * 4); + const orchestrator = buildOrchestrator( + // Second call is held open past the slow tool's late settle, so the turn + // is still live when the abandoned dispatch resolves. + fakeProvider( + [ + toolCallResponse([ + { id: 'use-slow', name: 'query_slow_agent' }, + { id: 'use-fast', name: 'fast_tool' }, + ]), + textResponse('done'), + ], + [0, DEADLINE_MS * 6], + ), + fastToolRegistry(), + [probe.tool], + ); + + const captured: Array<{ name: string; result: string }> = []; + const events: ChatStreamEvent[] = []; + await turnContext.run( + { + turnId: 'outer-turn', + turnDate: '2026-07-30', + captureRawToolResult: (name, result) => { + captured.push({ name, result }); + }, + }, + async () => { + // `sessionScope` switches the run-trace collector on, so the late + // sub-agent event has a real turn-state sink to corrupt: without the + // abort-guarded observer it lands in the `done` event's runTrace. + for await (const ev of orchestrator.chatStream({ + userMessage: 'go', + sessionScope: 'test::deadline', + })) { + events.push(ev); + } + }, + ); + + // The late path must actually have run, or this test proves nothing. + assert.equal( + probe.settledLate(), + true, + 'the abandoned sub-agent must have settled during the turn for this test to be meaningful', + ); + + const results = events.filter((e) => e.type === 'tool_result'); + const slow = results.find((e) => e.type === 'tool_result' && e.id === 'use-slow'); + const fast = results.find((e) => e.type === 'tool_result' && e.id === 'use-fast'); + assert.ok(slow && slow.type === 'tool_result', 'the slow slot must produce a tool_result'); + assert.ok(fast && fast.type === 'tool_result', 'the fast slot must produce a tool_result'); + + // 1. Structured error, not a hang. + assert.equal(slow.isError, true); + assert.match(slow.output, /^Error: tool `query_slow_agent` was aborted/); + assert.match(slow.output, /dispatch deadline/); + + // 2. Batch siblings are unaffected by another slot's deadline. + assert.equal(fast.isError, false); + assert.equal(fast.output, FAST_VALUE); + + // 3. MUTATION CHECK — the late result is never written into the turn. + assert.deepEqual( + captured, + [{ name: 'fast_tool', result: FAST_VALUE }], + 'only the sibling tool may reach captureRawToolResult; a late write from the aborted slot is a corruption bug', + ); + const transcript = JSON.stringify(events); + assert.equal( + transcript.includes(LATE_VALUE), + false, + 'the abandoned dispatch\'s value must not appear anywhere in the turn transcript', + ); + assert.equal( + probe.lateObserverCalls(), + 1, + 'the sub-agent still emitted its late event (so the observer guard, not the sub-agent, is what suppresses it)', + ); + // Invariant (belt-and-braces): the abort-guarded observer drops the late + // sub-event at the boundary. Downstream layers happen to ignore it too (the + // slot left the race loop, its invocation is already finished), so this + // assertion documents the boundary contract rather than being the only + // thing standing between a late event and the turn. + assert.equal( + events.some( + (e) => e.type === 'sub_tool_result' && e.id === 'late-sub-call', + ), + false, + 'a post-deadline sub-agent event must be dropped, not streamed into the turn', + ); + }); + + it('non-streaming Promise.allSettled batch: one deadline does not stop the siblings', async () => { + process.env['OMADIA_TOOL_DISPATCH_TIMEOUT_MS'] = String(DEADLINE_MS); + const probe = slowDomainTool('query_slow_agent', DEADLINE_MS * 3); + const orchestrator = buildOrchestrator( + fakeProvider([ + toolCallResponse([ + { id: 'use-slow', name: 'query_slow_agent' }, + { id: 'use-fast', name: 'fast_tool' }, + ]), + textResponse('answered'), + ]), + fastToolRegistry(), + [probe.tool], + ); + + const started = Date.now(); + const result = await orchestrator.runTurn({ userMessage: 'go' }); + const elapsed = Date.now() - started; + + assert.equal(result.answer, 'answered'); + // The turn must not wait for the hung tool (3× the deadline). + assert.ok( + elapsed < DEADLINE_MS * 3, + `turn should finish on the deadline, not on the hung tool; took ${String(elapsed)}ms`, + ); + assert.equal(probe.settledLate(), false, 'the hung tool must still be in flight'); + }); + + it('honours a 0 deadline as "disabled" (legacy behaviour)', async () => { + process.env['OMADIA_TOOL_DISPATCH_TIMEOUT_MS'] = '0'; + const probe = slowDomainTool('query_slow_agent', 60); + const orchestrator = buildOrchestrator( + fakeProvider([ + toolCallResponse([{ id: 'use-slow', name: 'query_slow_agent' }]), + textResponse('answered'), + ]), + fastToolRegistry(), + [probe.tool], + ); + + const result = await orchestrator.runTurn({ userMessage: 'go' }); + assert.equal(result.answer, 'answered'); + assert.equal( + probe.settledLate(), + true, + 'with the deadline disabled the dispatch must be awaited to completion', + ); + }); + + it('falls back to the 120s default when the env value is not a number', async () => { + process.env['OMADIA_TOOL_DISPATCH_TIMEOUT_MS'] = 'not-a-number'; + const probe = slowDomainTool('query_slow_agent', 20); + const orchestrator = buildOrchestrator( + fakeProvider([ + toolCallResponse([{ id: 'use-slow', name: 'query_slow_agent' }]), + textResponse('answered'), + ]), + fastToolRegistry(), + [probe.tool], + ); + + const result = await orchestrator.runTurn({ userMessage: 'go' }); + // A bad env value must not degrade into "no deadline" or "0ms deadline": + // the tool completes normally well inside the 120s default. + assert.equal(result.answer, 'answered'); + assert.equal(probe.settledLate(), true); + }); +}); From fdd293667b62d3f68d79a06f93bd3203e8d965d7 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 09:45:28 +0200 Subject: [PATCH 08/90] test(mcp): isolate the re-apply check in a schema, not a database MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The re-apply-under-data suite originally built its private copy of the domain in a throwaway database. That isolates correctly, but CREATE/DROP DATABASE is a cluster-wide operation: run inside the full suite with a test Postgres reachable, it stalled the concurrently executing dev-platform pg suites long enough that 29 of their tests were cancelled with "test did not finish before its parent". Reproduced deterministically against appStore.pg.test.ts and devJobStore.pg.test.ts, and confirmed absent from the same run with this file removed. It now runs against a dedicated schema on one pinned connection with `public` off the search_path. The migrations name every object unqualified, so they build a private copy there and never touch — or take ACCESS EXCLUSIVE on — the shared tables. Cancellations: 29 -> 0. The test asserts the isolation itself (table count in the schema), because a leaked search_path would turn the migrations into no-ops against the shared tables and make every later assertion pass vacuously. Both suites now share the file's single capped pool, closed once in a file-level after hook. --- docs/CHANGELOG.md | 15 ++- middleware/test/mcpRegistrySchema.pg.test.ts | 131 +++++++++---------- 2 files changed, 71 insertions(+), 75 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index ef913c0e..12221b76 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -59,12 +59,19 @@ entry. See `CONTRIBUTING.md` § Releases & changelog. - A second suite covers what the CI gate structurally cannot: the CI idempotency check re-applies against an **empty** database, so it can never catch a migration that only breaks once rows exist. That suite re-applies - all 30 files with MCP rows in place, in its own throwaway database — - re-running `0001`/`0003` drops and recreates the NOTIFY triggers, which - must not happen underneath a concurrently running suite. + all 30 files with MCP rows in place. It runs against a dedicated schema on + a pinned connection with `public` off the `search_path`, so the migrations + build a private copy of the domain: re-running `0001`/`0003` drops and + recreates the NOTIFY triggers and takes ACCESS EXCLUSIVE on shared tables, + which must not happen underneath a concurrently running suite. A scratch + *database* isolates just as well but `CREATE`/`DROP DATABASE` is a + cluster-wide operation — it stalled the dev-platform pg suites long enough + to cancel 29 of their tests, so the schema is the cheaper boundary. The + test asserts the isolation itself, since a leaked `search_path` would make + every later assertion pass vacuously. - Both suites skip when no test Postgres is reachable, and scope every row they write to a `w04-mcp-` tenant prefix, matching the existing pg-suite - convention. Their pools are capped: the runner executes test files + convention. They share one capped pool: the runner executes test files concurrently and ~16 other pg suites each hold a default-sized (max 10) pool, so an uncapped extra pool in one file exhausts `max_connections` and cancels an unrelated suite mid-run. diff --git a/middleware/test/mcpRegistrySchema.pg.test.ts b/middleware/test/mcpRegistrySchema.pg.test.ts index f5725c20..ac20e4c0 100644 --- a/middleware/test/mcpRegistrySchema.pg.test.ts +++ b/middleware/test/mcpRegistrySchema.pg.test.ts @@ -4,7 +4,7 @@ import { dirname, join, resolve } from 'node:path'; import { fileURLToPath } from 'node:url'; import { after, before, describe, it } from 'node:test'; -import { Pool } from 'pg'; +import { Pool, type PoolClient } from 'pg'; import { runMultiOrchestratorMigrations } from '@omadia/orchestrator'; @@ -23,9 +23,12 @@ import { runMultiOrchestratorMigrations } from '@omadia/orchestrator'; * * Isolation: every row this suite writes carries the `w04-mcp-` tenant * prefix so it cannot collide with the other pg suites sharing the database. - * The destructive re-apply check runs in its own scratch database for the - * same reason — re-running 0001/0003 drops and recreates NOTIFY triggers, - * which must never happen underneath a concurrently running suite. + * The destructive re-apply check needs more than a prefix — re-running + * 0001/0003 drops and recreates the NOTIFY triggers, taking ACCESS EXCLUSIVE + * on shared tables — so it runs against its own schema with `public` off the + * search_path. A scratch *database* would also isolate it, but CREATE/DROP + * DATABASE is a cluster-wide operation: it stalled the concurrently running + * dev-platform pg suites long enough to cancel 29 of their tests. * Skips when no test Postgres is reachable, mirroring the other pg tests. */ const PG_URL = @@ -36,19 +39,23 @@ const PG_URL = /** Tenant prefix — unique to this suite, see the isolation note above. */ const TENANT = 'w04-mcp-'; -const SCRATCH_DB = 'w04_mcp_schema_scratch'; +/** Schema the re-apply check builds its own copy of the domain in. */ +const REAPPLY_SCHEMA = 'w04_mcp_reapply'; const migrationsDir = resolve(dirname(fileURLToPath(import.meta.url)), '..', 'migrations'); /** - * Pools here are deliberately capped. The suite is fully sequential, but the - * test runner executes files concurrently and ~16 other pg suites each hold a - * default-sized (max 10) pool — an uncapped third pool in this file is enough - * to exhaust `max_connections` and cancel an unrelated suite mid-flight. + * One capped pool for the whole file. The test runner executes files + * concurrently and ~16 other pg suites each hold a default-sized (max 10) + * pool, so an uncapped extra pool here is enough to exhaust + * `max_connections` and cancel an unrelated suite mid-flight. */ -const POOL = { connectionTimeoutMillis: 2000, max: 2, idleTimeoutMillis: 1000 } as const; - -const probePool = new Pool({ connectionString: PG_URL, ...POOL }); +const probePool = new Pool({ + connectionString: PG_URL, + connectionTimeoutMillis: 2000, + max: 2, + idleTimeoutMillis: 1000, +}); let pgAvailable = true; try { await probePool.query('SELECT 1'); @@ -90,10 +97,9 @@ describe('MCP registry + OAuth schema (pg)', { skip: !pgAvailable }, () => { await cleanup(); }); - after(async () => { - await cleanup(); - await pool.end(); - }); + // The pool is shared with the re-apply suite below, so it is closed by the + // file-level `after` hook rather than here. + after(cleanup); it('seeds the official and smithery registries with their catalog kinds (0010 + 0013)', async () => { const { rows } = await pool.query<{ name: string; kind: string; auth_kind: string }>( @@ -262,99 +268,77 @@ describe('MCP registry + OAuth schema (pg)', { skip: !pgAvailable }, () => { }); describe('middleware/migrations idempotency under data (pg)', { skip: !pgAvailable }, () => { - // Runs in a throwaway database so the destructive re-apply (0001/0003 drop - // and recreate the NOTIFY triggers) cannot disturb a concurrently running - // pg suite on the shared test database. - const adminUrl = new URL(PG_URL); - adminUrl.pathname = '/postgres'; - const scratchUrl = new URL(PG_URL); - scratchUrl.pathname = `/${SCRATCH_DB}`; - - let scratchPool: Pool | undefined; - let scratchReady = false; - /** - * Admin connections are opened per operation and closed immediately — - * holding one open for the suite's lifetime is exactly the connection - * pressure the POOL cap above exists to avoid. + * Runs against a dedicated schema on a single pinned connection, with + * `public` deliberately absent from the search_path: the migrations name + * every object unqualified, so they build a private copy of the domain here + * and never touch — or lock — the shared tables the other pg suites use. */ - async function withAdmin(fn: (pool: Pool) => Promise): Promise { - const admin = new Pool({ connectionString: adminUrl.toString(), ...POOL, max: 1 }); - try { - return await fn(admin); - } finally { - await admin.end().catch(() => undefined); - } - } + let client: PoolClient | undefined; before(async () => { - try { - await withAdmin(async (admin) => { - // DROP/CREATE DATABASE cannot run inside a transaction block, so these - // must stay separate statements. - await admin.query(`DROP DATABASE IF EXISTS ${SCRATCH_DB} WITH (FORCE)`); - await admin.query(`CREATE DATABASE ${SCRATCH_DB}`); - }); - scratchReady = true; - } catch { - // No CREATEDB privilege (or no `postgres` database) — skip rather than - // fail, matching how the pg suites degrade when Postgres is absent. - scratchReady = false; - return; - } - scratchPool = new Pool({ connectionString: scratchUrl.toString(), ...POOL, max: 1 }); + client = await probePool.connect(); + await client.query(`DROP SCHEMA IF EXISTS ${REAPPLY_SCHEMA} CASCADE`); + await client.query(`CREATE SCHEMA ${REAPPLY_SCHEMA}`); + await client.query(`SET search_path = ${REAPPLY_SCHEMA}`); }); after(async () => { - await scratchPool?.end().catch(() => undefined); - if (scratchReady) { - await withAdmin((admin) => - admin.query(`DROP DATABASE IF EXISTS ${SCRATCH_DB} WITH (FORCE)`), - ).catch(() => undefined); - } + if (!client) return; + await client.query('RESET search_path').catch(() => undefined); + await client.query(`DROP SCHEMA IF EXISTS ${REAPPLY_SCHEMA} CASCADE`).catch(() => undefined); + client.release(); }); - it('re-applies every migration cleanly with rows present', async (t) => { - if (!scratchReady || !scratchPool) { - t.skip('no CREATEDB privilege on the test Postgres'); - return; - } - const pool = scratchPool; + it('re-applies every migration cleanly with rows present', async () => { + const pool = client!; const files = await migrationFiles(); assert.ok(files.length > 0, 'expected migrations to be discovered'); - // Pass 1 — virgin database. `middleware/migrations` needs no extensions + // Pass 1 — virgin schema. `middleware/migrations` needs no extensions // (gen_random_uuid is core since pg13) and has no cross-domain FKs, which // is why this domain can be applied standalone. for (const file of files) { await pool.query(await readFile(join(migrationsDir, file), 'utf8')); } + // Guard the isolation itself: if search_path had leaked to `public` the + // migrations would have been no-ops against the shared tables and every + // assertion below would pass vacuously. + const built = await pool.query<{ count: string }>( + `SELECT count(*)::text AS count FROM information_schema.tables WHERE table_schema = $1`, + [REAPPLY_SCHEMA], + ); + assert.ok( + Number(built.rows[0]!.count) > 25, + `expected the domain to be built inside ${REAPPLY_SCHEMA}, saw ${built.rows[0]!.count} tables`, + ); + // Seed the MCP + dev-platform surfaces so the re-apply runs against real // rows — the case the CI gate cannot reach, since it re-applies empty. const server = await pool.query<{ id: string }>( `INSERT INTO mcp_servers (name, transport, endpoint) VALUES ($1, 'http', 'https://srv.invalid/mcp') RETURNING id`, - [`${TENANT}scratch-server`], + [`${TENANT}reapply-server`], ); const agent = await pool.query<{ id: string }>( `INSERT INTO agents (slug, name) VALUES ($1, 'W0-4 Scratch') RETURNING id`, - [`${TENANT}scratch-agent`], + [`${TENANT}reapply-agent`], ); await pool.query( `INSERT INTO mcp_registries (name, url, auth_kind, kind) VALUES ($1, 'https://reg.invalid', 'none', 'generic')`, - [`${TENANT}scratch-registry`], + [`${TENANT}reapply-registry`], ); await pool.query( `INSERT INTO agent_tool_grants (agent_id, tool_kind, tool_ref, mcp_server_id) VALUES ($1, 'mcp', $2, $3)`, - [agent.rows[0]!.id, `${TENANT}scratch-server:ping`, server.rows[0]!.id], + [agent.rows[0]!.id, `${TENANT}reapply-server:ping`, server.rows[0]!.id], ); await pool.query( `INSERT INTO mcp_oauth_clients (issuer, client_id, registered_via) VALUES ($1, 'cid', 'manual')`, - [`${TENANT}https://scratch-issuer.invalid`], + [`${TENANT}https://reapply-issuer.invalid`], ); await pool.query( `INSERT INTO mcp_oauth_tokens (server_id, user_key, access_token_ref) @@ -385,3 +369,8 @@ describe('middleware/migrations idempotency under data (pg)', { skip: !pgAvailab assert.equal(tokens.rowCount, 1, 're-applying must not disturb stored OAuth token refs'); }); }); + +// Both suites share the single capped pool, so it is closed once, here. +after(async () => { + if (pgAvailable) await probePool.end().catch(() => undefined); +}); From 904b17c21a6e711b8ed5405445ed7bf90d0d10b7 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 09:46:39 +0200 Subject: [PATCH 09/90] perf(orchestrator): deterministic tool ordering + stateless loopback MCP server MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit W0-3 — sort the dynamic tool segments by name so the Anthropic prompt-cache tool block is byte-stable across machines and deploys. `buildToolsList()` stamps `cache_control: {type:'ephemeral'}` on the last tool spec, which makes the whole tool block a single cacheable chunk. The cache keys on a byte-exact prefix, but two of the segments feeding that block were iterated straight out of Maps — plugin load order for the native tool registry, `created_at` row order for domain tools — with no sort anywhere. Stable within one process, divergent across Fly machines and across deploys: a silent, signal-free cache miss for the tool block and everything after it. - new `toolOrdering.ts`: `compareToolNames` (locale-pinned `localeCompare(b,'en')` so the result does not depend on the host's LANG/LC_COLLATE), `sortByToolName`, `sortBySpecName`, `normalizeDiscoveredToolOrder` - `buildToolsList()`: native + domain segments sorted; the deliberate fixed-literal prefix (memory, knowledge-graph, ...) keeps its existing order - `ToolDispatchService.listDispatchableToolSpecs()`: sorted, so the loopback server and the CLI bridge inherit it - `LoopbackMcpServer` tools/list: sorted independently, because `deps.tools` is caller-supplied - `resolveSubAgentTools()`: sorted (grants arrive in `created_at` order) - `setMcpDiscoveredTools()`: normalizes by name before persisting, so a server that returns `tools/list` in a different order each call stops churning the JSONB column and any grant-epoch diff derived from it Ordering is advertisement-only. Collision resolution is unchanged — native tools still win a duplicate name, decided by Map insertion and never by array position — and that is now pinned by a test whose colliding name deliberately sorts last. W1-2 — make the loopback MCP server stateless. `sessionIdGenerator: undefined` is the SDK's stateless mode: no session id is issued and no session validation is performed, so the CLI bridge needs neither the `initialize` handshake nor `Mcp-Session-Id`. The previous comment claiming session ids "remain required by the protocol" was wrong. SDK 1.29.0 enforces the other half of that contract — a stateless transport throws "Stateless transport cannot be reused across requests" on its second use. The MCP server + transport pair is therefore built per request (matching the SDK's own stateless example) and torn down in a `finally`. Both are in-memory handler tables with no I/O, and this server sees a handful of requests per CLI turn. `enableJsonResponse` stays on, which also guarantees the response is fully written before teardown. Non-POST is now declined with 405, which the MCP spec explicitly allows for the optional GET SSE stream. Without it the per-request transport leaks: a GET stream never ends, so `handleRequest` never resolves and the request scope never tears down. Under the old stateful transport a session-less GET was rejected with 400, so nothing regresses. Tests were written first for W1-2 and observed to fail (HTTP 400) before the production change. The wire test is parameterized over replaying vs never sending the session header, plus a case with no `initialize` at all. The 401 `-32001` body and the 413 oversized-POST case still hold. Deliberately NOT implemented: the `ttlMs` tool-list cache also proposed in #545. Its premise is false — `subAgentToolHydration` reads `mcp_servers.discovered_tools` and never calls `listTools`, so steady state is already ~4 wire calls per server per day — and it would re-advertise removed or repurposed tools inside exactly the window the #454 scan-verdict gate exists to close. MANUAL VERIFICATION REQUIRED BEFORE MERGE: the loopback server's only consumer is the Claude CLI bridge, spawned with `--strict-mcp-config --mcp-config --allowedTools mcp__omadia__*`. If the installed CLI refuses to proceed when the server issues no `mcp-session-id`, the bridge yields a server with ZERO tools and the turn silently degrades to a toolless answer rather than erroring — no automated test catches that. A pass against the real `claude` CLI with the stubbed `createLoopbackServer` bypassed is needed. Not performed here: spawning a nested `claude` session is blocked in this environment. Installed CLI version on this machine is 2.1.220. --- .../src/loopbackMcpServer.ts | 163 +++++--- .../harness-orchestrator/src/orchestrator.ts | 33 +- .../src/registry/agentGraphStore.ts | 9 +- .../src/registry/subAgentTools.ts | 13 +- .../src/toolDispatchService.ts | 11 +- .../harness-orchestrator/src/toolOrdering.ts | 81 ++++ .../test/cliBridge/loopbackMcpServer.test.ts | 367 ++++++++++++++---- .../cliBridge/toolDispatchService.test.ts | 14 +- .../deterministicToolOrder.test.ts | 222 +++++++++++ .../toolOrderingInvariants.test.ts | 240 ++++++++++++ 10 files changed, 1016 insertions(+), 137 deletions(-) create mode 100644 middleware/packages/harness-orchestrator/src/toolOrdering.ts create mode 100644 middleware/test/orchestrator/deterministicToolOrder.test.ts create mode 100644 middleware/test/orchestrator/toolOrderingInvariants.test.ts diff --git a/middleware/packages/harness-orchestrator/src/loopbackMcpServer.ts b/middleware/packages/harness-orchestrator/src/loopbackMcpServer.ts index 3d37f4a7..7ef7f0e1 100644 --- a/middleware/packages/harness-orchestrator/src/loopbackMcpServer.ts +++ b/middleware/packages/harness-orchestrator/src/loopbackMcpServer.ts @@ -7,7 +7,6 @@ * the dispatch service and the MCP SDK this package already depends on. */ -import { randomUUID } from 'node:crypto'; import { createServer, type IncomingMessage, @@ -28,6 +27,7 @@ import type { DispatchableToolSpec, ToolDispatchService, } from './toolDispatchService.js'; +import { sortByToolName } from './toolOrdering.js'; const MAX_REQUEST_BYTES = 8 * 1024 * 1024; @@ -49,8 +49,6 @@ export interface LoopbackMcpServerHandle { export class LoopbackMcpServer { private http?: HttpServer; - private transport?: StreamableHTTPServerTransport; - private mcp?: McpServer; private started = false; constructor(private readonly deps: LoopbackMcpServerDeps) { @@ -68,39 +66,6 @@ export class LoopbackMcpServer { throw new Error('LoopbackMcpServer: already started'); } - this.mcp = new McpServer( - { - name: this.deps.serverName ?? 'omadia-loopback', - version: this.deps.serverVersion ?? '0.0.0', - }, - { capabilities: { tools: {} } }, - ); - - this.mcp.setRequestHandler(ListToolsRequestSchema, async () => ({ - tools: this.deps.tools.map((tool) => ({ - name: tool.name, - description: tool.description, - inputSchema: tool.input_schema, - })), - })); - - this.mcp.setRequestHandler(CallToolRequestSchema, async (request) => { - const { name, arguments: args } = request.params; - const result = await this.deps.dispatch.dispatch(name, args ?? {}); - return { - content: [{ type: 'text' as const, text: result.content }], - ...(result.isError ? { isError: true } : {}), - }; - }); - - // Stateless-ish loopback transport; JSON responses simplify the client and - // session IDs remain required by the protocol. - this.transport = new StreamableHTTPServerTransport({ - sessionIdGenerator: () => randomUUID(), - enableJsonResponse: true, - }); - await this.mcp.connect(this.transport); - this.http = createServer((req, res) => { void this.handleHttp(req, res); }); @@ -134,9 +99,6 @@ export class LoopbackMcpServer { return; } - await this.mcp?.close().catch(() => {}); - await this.transport?.close().catch(() => {}); - if (this.http) { await new Promise((resolve) => { this.http?.close(() => resolve()); @@ -144,11 +106,68 @@ export class LoopbackMcpServer { } this.http = undefined; - this.transport = undefined; - this.mcp = undefined; this.started = false; } + /** + * Builds a fresh MCP server + transport pair for a single HTTP request. + * + * W1-2 — `sessionIdGenerator: undefined` selects the SDK's stateless mode: + * no session id is issued and no session validation happens, so a client may + * skip the `initialize` handshake and never send `Mcp-Session-Id`. The SDK + * enforces the other half of that contract — a stateless transport throws + * "Stateless transport cannot be reused across requests" on its second use — + * so the transport (and the `Server` bound to it) is per-request by + * construction, matching the SDK's own stateless example. + * + * Cost is negligible: both objects are pure in-memory handler tables with no + * I/O, and this server sees a handful of requests per CLI turn. Nothing here + * held cross-request state worth keeping — the tool list and the dispatch + * service are owned by `deps`, and the bearer token is what actually scopes + * access. `enableJsonResponse` stays on: JSON replies keep the client simple + * and also guarantee the response is fully written by the time + * `handleRequest` resolves, which is what makes per-request teardown safe. + */ + private createRequestScopedServer(): { + mcp: McpServer; + transport: StreamableHTTPServerTransport; + } { + const mcp = new McpServer( + { + name: this.deps.serverName ?? 'omadia-loopback', + version: this.deps.serverVersion ?? '0.0.0', + }, + { capabilities: { tools: {} } }, + ); + + // W0-3 — advertise name-sorted. `ToolDispatchService` already sorts, but + // `deps.tools` is caller-supplied, so sorting here makes the wire order a + // property of this server rather than a convention every caller must know. + mcp.setRequestHandler(ListToolsRequestSchema, async () => ({ + tools: sortByToolName(this.deps.tools).map((tool) => ({ + name: tool.name, + description: tool.description, + inputSchema: tool.input_schema, + })), + })); + + mcp.setRequestHandler(CallToolRequestSchema, async (request) => { + const { name, arguments: args } = request.params; + const result = await this.deps.dispatch.dispatch(name, args ?? {}); + return { + content: [{ type: 'text' as const, text: result.content }], + ...(result.isError ? { isError: true } : {}), + }; + }); + + const transport = new StreamableHTTPServerTransport({ + sessionIdGenerator: undefined, + enableJsonResponse: true, + }); + + return { mcp, transport }; + } + private async handleHttp( req: IncomingMessage, res: ServerResponse, @@ -173,20 +192,45 @@ export class LoopbackMcpServer { return; } - try { - const transport = this.transport; - if (!transport) { - throw new McpError(ErrorCode.InternalError, 'Transport not started'); - } + if (!this.started) { + throw new McpError(ErrorCode.InternalError, 'Transport not started'); + } - if (req.method === 'POST') { - const rawBody = await this.readBody(req); - const parsedBody = rawBody.length > 0 ? JSON.parse(rawBody) : undefined; - await transport.handleRequest(req, res, parsedBody); - return; - } + // W1-2 — POST only. The MCP spec makes the GET standalone SSE stream + // optional and blesses 405 when a server does not offer one, and this + // server has nothing to deliver over it: `enableJsonResponse` answers every + // request inline and there are no server-initiated notifications. + // + // Declining it explicitly also avoids a leak introduced by the per-request + // transport: a GET opens a stream that never ends, so `handleRequest` never + // resolves, so the `finally` below never runs and the request-scoped + // server/transport pair stays alive until the client disconnects. Under the + // old stateful transport a session-less GET was simply rejected with 400, + // so nothing regresses here. + if (req.method !== 'POST') { + res.writeHead(405, { + 'Content-Type': 'application/json', + Allow: 'POST', + }); + res.end( + JSON.stringify({ + jsonrpc: '2.0', + error: { code: -32000, message: 'Method Not Allowed' }, + id: null, + }), + ); + return; + } - await transport.handleRequest(req, res); + // Read the body BEFORE building the per-request server so an oversized + // POST still fails with 413 without paying for the handler wiring. + let session: ReturnType | undefined; + try { + const parsedBody = await this.parsePostBody(req); + + session = this.createRequestScopedServer(); + await session.mcp.connect(session.transport); + await session.transport.handleRequest(req, res, parsedBody); } catch (error) { if (res.headersSent) { res.end(); @@ -215,9 +259,22 @@ export class LoopbackMcpServer { id: null, }), ); + } finally { + // A stateless transport is single-use; dropping it here is what keeps + // the next request from hitting the SDK's reuse guard. Safe at this + // point because `enableJsonResponse` means the response is already + // fully written when `handleRequest` resolves. + await session?.transport.close().catch(() => {}); + await session?.mcp.close().catch(() => {}); } } + /** Reads and JSON-parses a POST body, or `undefined` when the body is empty. */ + private async parsePostBody(req: IncomingMessage): Promise { + const rawBody = await this.readBody(req); + return rawBody.length > 0 ? JSON.parse(rawBody) : undefined; + } + private async readBody(req: IncomingMessage): Promise { const chunks: Buffer[] = []; let totalBytes = 0; diff --git a/middleware/packages/harness-orchestrator/src/orchestrator.ts b/middleware/packages/harness-orchestrator/src/orchestrator.ts index 8f0691f2..1446e409 100644 --- a/middleware/packages/harness-orchestrator/src/orchestrator.ts +++ b/middleware/packages/harness-orchestrator/src/orchestrator.ts @@ -80,6 +80,7 @@ import { QueryDatasetTool, queryDatasetToolSpec, } from './tools/queryDatasetTool.js'; +import { sortByToolName } from './toolOrdering.js'; import { parseAttachmentsInfo } from './attachmentsInfo.js'; import { checkVisionEmbeddable, @@ -5652,22 +5653,45 @@ export class Orchestrator { // Issue #474 — a plugin that hasn't finished its own connection/auth // setup is excluded here so the orchestrator never offers a tool it // knows will fail, instead of discovering that via a wasted round-trip. + // + // W0-3 — sorted by name. `listWithHandler()` iterates a Map, so raw order + // is plugin LOAD order, which differs between Fly machines and between + // deploys. That silently invalidated the `cache_control` block stamped at + // the end of this method. Sorting makes the block a function of the tool + // set, not of registration timing. Advertisement order only — dispatch + // still resolves by name, so precedence is unaffected. + const nativeSpecs: unknown[] = []; for (const entry of this.nativeTools.listWithHandler()) { if (entry.spec && this.isToolAvailable(entry.agentId)) { - tools.push(entry.spec); + nativeSpecs.push(entry.spec); } } + for (const spec of sortByToolName( + nativeSpecs as ReadonlyArray<{ readonly name: string }>, + )) { + tools.push(spec); + } // DomainTools dynamically from the map — so hot-registered uploaded // agents become visible from the next iteration without reboot. // Issue #474 — same gate as the native-tools loop above: a domain tool // whose owning plugin hasn't completed its connection/auth setup must // not be offered either, otherwise the model discovers the missing // access via a failing dispatch instead of the tool being absent. + // + // W0-3 — sorted for the same reason as the native segment above; this map + // is populated in `created_at` row order, which is not stable across + // machines that hydrated their registry at different times. + const domainSpecs: unknown[] = []; for (const tool of this.domainToolsByName.values()) { if (this.isToolAvailable(tool.agentId)) { - tools.push(tool.spec); + domainSpecs.push(tool.spec); } } + for (const spec of sortByToolName( + domainSpecs as ReadonlyArray<{ readonly name: string }>, + )) { + tools.push(spec); + } // Privacy-Shield v4 — verb + render tools, offered only when the v4 // data-plane boundary is active for this turn. const v4ToolSpecs = turnContext.current()?.privacyHandle?.v4ToolSpecs(); @@ -5679,6 +5703,11 @@ export class Orchestrator { // marking the final tool makes the whole list a single cacheable chunk. // 5-minute TTL comfortably covers a multi-iteration orchestrator turn, // so iter 2..N skip re-reading the tool definitions on the server side. + // + // W0-3 — the cache keys on a byte-exact prefix, so this only pays off + // because the dynamic segments above are name-sorted. Do not reorder or + // append unsorted segments before this point without re-reading + // `toolOrdering.ts`; a reordered block is a silent, signal-free cache miss. const last = tools[tools.length - 1]; if (last) { tools[tools.length - 1] = { diff --git a/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts b/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts index 4ea2f5bd..bba0e6f2 100644 --- a/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts +++ b/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts @@ -2,6 +2,7 @@ import type { Pool } from 'pg'; import { ConfigValidationError, validateModelRef } from './configStore.js'; import { computeSkillHash } from './skillHash.js'; +import { normalizeDiscoveredToolOrder } from '../toolOrdering.js'; /** * Agent Builder graph store (P0). @@ -2152,11 +2153,17 @@ export class AgentGraphStore { id: string, tools: readonly unknown[], ): Promise { + // W0-3 — normalize by name before persisting. An MCP server may return + // `tools/list` in a different order on every call; storing that raw makes + // each rediscovery rewrite the JSONB with semantically identical content, + // churning the row and any grant-epoch diff computed from it. It also + // leaks the server's arbitrary ordering into the tool block that + // `subAgentToolHydration` later builds from this column. await this.pool.query( `UPDATE mcp_servers SET discovered_tools = $2::jsonb, last_discovered_at = now(), updated_at = now() WHERE id = $1`, - [id, JSON.stringify(tools)], + [id, JSON.stringify(normalizeDiscoveredToolOrder(tools))], ); } diff --git a/middleware/packages/harness-orchestrator/src/registry/subAgentTools.ts b/middleware/packages/harness-orchestrator/src/registry/subAgentTools.ts index 67c9a416..841c0f45 100644 --- a/middleware/packages/harness-orchestrator/src/registry/subAgentTools.ts +++ b/middleware/packages/harness-orchestrator/src/registry/subAgentTools.ts @@ -4,6 +4,7 @@ import type { LocalSubAgentTool } from '@omadia/plugin-api'; import { createCliSubAgent } from '../cliSubAgent.js'; import { LocalSubAgent } from '../localSubAgent.js'; import { resolveModelIdForProvider } from './agentRuntime.js'; +import { sortBySpecName } from '../toolOrdering.js'; import type { McpManager} from '../mcp/mcpClient.js'; import { @@ -176,7 +177,11 @@ export function resolveCliSubAgentModel( ); } -function resolveSubAgentTools( +/** + * Exported for the W0-3 determinism test; production callers should go + * through `buildSubAgentDomainTools`. + */ +export function resolveSubAgentTools( grants: readonly ToolGrantRow[], deps: SubAgentToolDeps, ): LocalSubAgentTool[] { @@ -209,7 +214,11 @@ function resolveSubAgentTools( mcpToolToLocalSubAgentTool(deps.mcpManager, cfg, { name: toolName }), ); } - return out; + // W0-3 — sort by name. Grants arrive in `created_at` row order, so two + // machines that were seeded at different times would hand the sub-agent an + // identical tool SET in a different sequence, defeating prompt caching on + // the sub-agent's own tool block for no behavioural gain. + return sortBySpecName(out); } /** `toolRef` for an mcp grant is ":"; fall back to the diff --git a/middleware/packages/harness-orchestrator/src/toolDispatchService.ts b/middleware/packages/harness-orchestrator/src/toolDispatchService.ts index 4a23fc2c..c713833b 100644 --- a/middleware/packages/harness-orchestrator/src/toolDispatchService.ts +++ b/middleware/packages/harness-orchestrator/src/toolDispatchService.ts @@ -9,6 +9,7 @@ import type { DomainTool } from './tools/domainQueryTool.js'; import type { NativeToolRegistry } from './nativeToolRegistry.js'; +import { sortByToolName } from './toolOrdering.js'; export interface ToolDispatchResult { readonly content: string; @@ -135,7 +136,15 @@ export class ToolDispatchService { }); } - return Array.from(advertised.values()); + // W0-3 — sort by name so every consumer of this list (the loopback MCP + // server, the CLI bridge) advertises a byte-stable order. Both source + // iterations above are Map-ordered — plugin load order and `created_at` + // row order — which differ across machines and deploys. + // + // Collision resolution is NOT affected: which spec wins a duplicate name + // was already decided by the `advertised.has(...)` guard above (native + // tools first), and sorting only reorders the surviving entries. + return sortByToolName(Array.from(advertised.values())); } private errMsg(error: unknown): string { diff --git a/middleware/packages/harness-orchestrator/src/toolOrdering.ts b/middleware/packages/harness-orchestrator/src/toolOrdering.ts new file mode 100644 index 00000000..8f6bef0d --- /dev/null +++ b/middleware/packages/harness-orchestrator/src/toolOrdering.ts @@ -0,0 +1,81 @@ +/** + * Deterministic tool ordering (W0-3). + * + * Anthropic prompt caching keys on a byte-exact prefix: `buildToolsList()` + * stamps `cache_control: { type: 'ephemeral' }` on the LAST tool spec, which + * makes the whole tool block one cacheable chunk. That only pays off if the + * block serializes identically every time. + * + * Several of the segments feeding that block are iterated straight out of a + * `Map`, so their order is insertion order: plugin load order for the native + * tool registry, `created_at` row order for domain tools. Both are stable + * *within* one process but diverge across Fly machines and across deploys — + * silently dropping the cache for the entire tool block and everything after + * it, with no error and no signal other than the cache-read token counter. + * + * Sorting the dynamic segments by name makes the serialized block a pure + * function of the tool SET rather than of registration timing. + * + * Ordering is advertisement-only. Collision resolution (native tools win over + * domain tools on a duplicate name) is decided by `Map` insertion in + * `ToolDispatchService`, never by array position, so sorting the resulting + * array cannot change which handler a name resolves to. + */ + +/** + * Locale-pinned name comparison. The explicit `'en'` locale keeps the result + * independent of the host's `LANG`/`LC_COLLATE`, which is the whole point — + * two Fly machines with different environments must produce the same bytes. + */ +export function compareToolNames(left: string, right: string): number { + return left.localeCompare(right, 'en'); +} + +/** Returns a new array sorted by `name`; never mutates the input. */ +export function sortByToolName( + items: readonly T[], +): T[] { + return [...items].sort((a, b) => compareToolNames(a.name, b.name)); +} + +/** + * Same ordering for tool shapes that carry their name on a nested `spec` + * (`LocalSubAgentTool` has no top-level `name`). + */ +export function sortBySpecName< + T extends { readonly spec: { readonly name: string } }, +>(items: readonly T[]): T[] { + return [...items].sort((a, b) => compareToolNames(a.spec.name, b.spec.name)); +} + +/** + * Normalizes the order of an MCP server's discovered-tool list before it is + * persisted to `mcp_servers.discovered_tools`. + * + * A server is free to return `tools/list` in any order it likes, and some + * return a different order per call. Without normalization each rediscovery + * rewrites the JSONB column with semantically identical content, which churns + * the row and any grant-epoch diff computed from it. + * + * Entries without a usable string `name` keep their relative order and are + * placed after named entries, so a malformed payload degrades rather than + * throws. + */ +export function normalizeDiscoveredToolOrder( + tools: readonly unknown[], +): unknown[] { + const named: Array<{ name: string; value: unknown }> = []; + const unnamed: unknown[] = []; + + for (const tool of tools) { + const name = + typeof tool === 'object' && tool !== null && 'name' in tool + ? (tool as { name: unknown }).name + : undefined; + if (typeof name === 'string') named.push({ name, value: tool }); + else unnamed.push(tool); + } + + named.sort((a, b) => compareToolNames(a.name, b.name)); + return [...named.map((entry) => entry.value), ...unnamed]; +} diff --git a/middleware/test/cliBridge/loopbackMcpServer.test.ts b/middleware/test/cliBridge/loopbackMcpServer.test.ts index 12a79425..ffa237d5 100644 --- a/middleware/test/cliBridge/loopbackMcpServer.test.ts +++ b/middleware/test/cliBridge/loopbackMcpServer.test.ts @@ -19,6 +19,13 @@ function parseMcpJson(text: string): unknown { return JSON.parse(dataLines.join('\n')); } +const MCP_ACCEPT = 'application/json, text/event-stream'; + +/** True when the sandbox refuses loopback listeners, so the test self-skips. */ +function isSandboxListenDenied(error: unknown): boolean { + return error instanceof Error && 'code' in error && error.code === 'EPERM'; +} + describe('LoopbackMcpServer', () => { let server: LoopbackMcpServer | undefined; @@ -27,7 +34,181 @@ describe('LoopbackMcpServer', () => { server = undefined; }); - it('serves initialize, tools/list, and tools/call over loopback HTTP', async (t) => { + /** + * W1-2 — the loopback transport is stateless (`sessionIdGenerator: + * undefined`), so the wire contract must hold both for a client that + * replays whatever session id the server hands out and for one that never + * sends the header at all. Before the stateless switch the second variant + * failed with HTTP 400 "Mcp-Session-Id header is required". + */ + for (const variant of [ + { + label: 'replaying the session id when the server issues one', + replaySession: true, + }, + { label: 'never sending a session header', replaySession: false }, + ] as const) { + it(`serves initialize, tools/list, and tools/call over loopback HTTP — ${variant.label}`, async (t) => { + const seenCalls: Array<{ name: string; input: unknown }> = []; + const fakeDispatch = { + async dispatch(name: string, input: unknown) { + seenCalls.push({ name, input }); + return { content: `dispatch:${name}:${JSON.stringify(input)}` }; + }, + } as unknown as ToolDispatchService; + + server = new LoopbackMcpServer({ + dispatch: fakeDispatch, + bearer: 'secret-token', + tools: [ + { + name: 'ping', + description: 'p', + input_schema: { type: 'object', properties: {} }, + }, + ], + }); + + let handle: Awaited>; + try { + handle = await server.start(); + } catch (error) { + if (isSandboxListenDenied(error)) { + t.skip('sandbox blocks loopback listeners on 127.0.0.1'); + return; + } + throw error; + } + + const initializeResponse = await fetch(handle.url, { + method: 'POST', + headers: { + Authorization: `Bearer ${handle.bearer}`, + 'Content-Type': 'application/json', + Accept: MCP_ACCEPT, + }, + body: JSON.stringify({ + jsonrpc: '2.0', + method: 'initialize', + params: { + protocolVersion: '2025-06-18', + capabilities: {}, + clientInfo: { name: 'test', version: '0' }, + }, + id: 1, + }), + }); + assert.equal(initializeResponse.status, 200); + const issuedSessionId = initializeResponse.headers.get('mcp-session-id'); + const initializePayload = parseMcpJson( + await initializeResponse.text(), + ) as { + result?: { protocolVersion?: string }; + }; + assert.ok(initializePayload.result); + + // A stateless transport issues no session id at all. Only the replaying + // variant forwards one, and only when the server actually handed it out. + const sessionHeaders: Record = + variant.replaySession && issuedSessionId + ? { 'mcp-session-id': issuedSessionId } + : {}; + + const initializedResponse = await fetch(handle.url, { + method: 'POST', + headers: { + Authorization: `Bearer ${handle.bearer}`, + 'Content-Type': 'application/json', + Accept: MCP_ACCEPT, + ...sessionHeaders, + }, + body: JSON.stringify({ + jsonrpc: '2.0', + method: 'notifications/initialized', + }), + }); + assert.equal(initializedResponse.status, 202); + + const listResponse = await fetch(handle.url, { + method: 'POST', + headers: { + Authorization: `Bearer ${handle.bearer}`, + 'Content-Type': 'application/json', + Accept: MCP_ACCEPT, + ...sessionHeaders, + }, + body: JSON.stringify({ + jsonrpc: '2.0', + method: 'tools/list', + params: {}, + id: 2, + }), + }); + assert.equal(listResponse.status, 200); + const listPayload = parseMcpJson(await listResponse.text()) as { + result?: { tools?: Array<{ name: string }> }; + }; + assert.ok( + listPayload.result?.tools?.some((tool) => tool.name === 'ping'), + ); + + const callResponse = await fetch(handle.url, { + method: 'POST', + headers: { + Authorization: `Bearer ${handle.bearer}`, + 'Content-Type': 'application/json', + Accept: MCP_ACCEPT, + ...sessionHeaders, + }, + body: JSON.stringify({ + jsonrpc: '2.0', + method: 'tools/call', + params: { name: 'ping', arguments: {} }, + id: 3, + }), + }); + assert.equal(callResponse.status, 200); + const callPayload = parseMcpJson(await callResponse.text()) as { + result?: { + content?: Array<{ type: string; text: string }>; + isError?: boolean; + }; + }; + assert.equal(callPayload.result?.content?.[0]?.text, 'dispatch:ping:{}'); + assert.equal(callPayload.result?.isError, undefined); + assert.deepEqual(seenCalls, [{ name: 'ping', input: {} }]); + + const badBearerResponse = await fetch(handle.url, { + method: 'POST', + headers: { + Authorization: 'Bearer wrong', + 'Content-Type': 'application/json', + Accept: MCP_ACCEPT, + }, + body: JSON.stringify({ + jsonrpc: '2.0', + method: 'tools/list', + params: {}, + id: 4, + }), + }); + assert.equal(badBearerResponse.status, 401); + const badBearerPayload = parseMcpJson(await badBearerResponse.text()) as { + error?: { code?: number; message?: string }; + result?: unknown; + }; + assert.equal(badBearerPayload.result, undefined); + assert.equal(badBearerPayload.error?.code, -32001); + assert.equal(badBearerPayload.error?.message, 'Unauthorized'); + }); + } + + /** + * W1-2 criterion — the stateless transport must serve a cold client that + * skips the handshake entirely. Under the previous stateful transport both + * calls below were rejected before ever reaching a request handler. + */ + it('serves tools/list and tools/call with no prior initialize and no session header', async (t) => { const seenCalls: Array<{ name: string; input: unknown }> = []; const fakeDispatch = { async dispatch(name: string, input: unknown) { @@ -52,126 +233,168 @@ describe('LoopbackMcpServer', () => { try { handle = await server.start(); } catch (error) { - if ( - error instanceof Error && - 'code' in error && - error.code === 'EPERM' - ) { + if (isSandboxListenDenied(error)) { t.skip('sandbox blocks loopback listeners on 127.0.0.1'); return; } throw error; } - const initializeResponse = await fetch(handle.url, { + const listResponse = await fetch(handle.url, { method: 'POST', headers: { Authorization: `Bearer ${handle.bearer}`, 'Content-Type': 'application/json', - Accept: 'application/json, text/event-stream', + Accept: MCP_ACCEPT, }, body: JSON.stringify({ jsonrpc: '2.0', - method: 'initialize', - params: { - protocolVersion: '2025-06-18', - capabilities: {}, - clientInfo: { name: 'test', version: '0' }, - }, + method: 'tools/list', + params: {}, id: 1, }), }); - assert.equal(initializeResponse.status, 200); - const sessionId = initializeResponse.headers.get('mcp-session-id'); - assert.ok(sessionId); - const initializePayload = parseMcpJson(await initializeResponse.text()) as { - result?: { protocolVersion?: string }; + assert.equal(listResponse.status, 200); + const listPayload = parseMcpJson(await listResponse.text()) as { + result?: { tools?: Array<{ name: string }> }; }; - assert.ok(initializePayload.result); + assert.deepEqual( + listPayload.result?.tools?.map((tool) => tool.name), + ['ping'], + ); - const initializedResponse = await fetch(handle.url, { + const callResponse = await fetch(handle.url, { method: 'POST', headers: { Authorization: `Bearer ${handle.bearer}`, 'Content-Type': 'application/json', - Accept: 'application/json, text/event-stream', - 'mcp-session-id': sessionId, + Accept: MCP_ACCEPT, }, body: JSON.stringify({ jsonrpc: '2.0', - method: 'notifications/initialized', + method: 'tools/call', + params: { name: 'ping', arguments: { a: 1 } }, + id: 2, }), }); - assert.equal(initializedResponse.status, 202); + assert.equal(callResponse.status, 200); + const callPayload = parseMcpJson(await callResponse.text()) as { + result?: { content?: Array<{ type: string; text: string }> }; + }; + assert.equal( + callPayload.result?.content?.[0]?.text, + 'dispatch:ping:{"a":1}', + ); + assert.deepEqual(seenCalls, [{ name: 'ping', input: { a: 1 } }]); + }); + + /** W0-3 — the loopback tool list is advertised name-sorted regardless of + * the order the dispatch service handed the specs over in. */ + it('advertises tools sorted by name', async (t) => { + const fakeDispatch = { + async dispatch() { + return { content: 'ok' }; + }, + } as unknown as ToolDispatchService; + + server = new LoopbackMcpServer({ + dispatch: fakeDispatch, + bearer: 'secret-token', + tools: ['zebra_tool', 'alpha_tool', 'mango_tool'].map((name) => ({ + name, + description: name, + input_schema: { type: 'object', properties: {} }, + })), + }); + + let handle: Awaited>; + try { + handle = await server.start(); + } catch (error) { + if (isSandboxListenDenied(error)) { + t.skip('sandbox blocks loopback listeners on 127.0.0.1'); + return; + } + throw error; + } const listResponse = await fetch(handle.url, { method: 'POST', headers: { Authorization: `Bearer ${handle.bearer}`, 'Content-Type': 'application/json', - Accept: 'application/json, text/event-stream', - 'mcp-session-id': sessionId, + Accept: MCP_ACCEPT, }, body: JSON.stringify({ jsonrpc: '2.0', method: 'tools/list', params: {}, - id: 2, + id: 1, }), }); assert.equal(listResponse.status, 200); const listPayload = parseMcpJson(await listResponse.text()) as { result?: { tools?: Array<{ name: string }> }; }; - assert.ok(listPayload.result?.tools?.some((tool) => tool.name === 'ping')); + assert.deepEqual( + listPayload.result?.tools?.map((tool) => tool.name), + ['alpha_tool', 'mango_tool', 'zebra_tool'], + ); + }); - const callResponse = await fetch(handle.url, { - method: 'POST', - headers: { - Authorization: `Bearer ${handle.bearer}`, - 'Content-Type': 'application/json', - Accept: 'application/json, text/event-stream', - 'mcp-session-id': sessionId, + /** + * W1-2 — the optional GET standalone SSE stream is declined with 405 (which + * the MCP spec allows). Without this the per-request transport would leak: + * a GET stream never ends, so its request scope never tears down. + */ + it('declines the standalone SSE stream with HTTP 405', async (t) => { + const fakeDispatch = { + async dispatch() { + return { content: 'ok' }; }, - body: JSON.stringify({ - jsonrpc: '2.0', - method: 'tools/call', - params: { name: 'ping', arguments: {} }, - id: 3, - }), + } as unknown as ToolDispatchService; + + server = new LoopbackMcpServer({ + dispatch: fakeDispatch, + bearer: 'secret-token', + tools: [], }); - assert.equal(callResponse.status, 200); - const callPayload = parseMcpJson(await callResponse.text()) as { - result?: { - content?: Array<{ type: string; text: string }>; - isError?: boolean; - }; - }; - assert.equal(callPayload.result?.content?.[0]?.text, 'dispatch:ping:{}'); - assert.equal(callPayload.result?.isError, undefined); - assert.deepEqual(seenCalls, [{ name: 'ping', input: {} }]); - const badBearerResponse = await fetch(handle.url, { - method: 'POST', + let handle: Awaited>; + try { + handle = await server.start(); + } catch (error) { + if (isSandboxListenDenied(error)) { + t.skip('sandbox blocks loopback listeners on 127.0.0.1'); + return; + } + throw error; + } + + const response = await fetch(handle.url, { + method: 'GET', headers: { - Authorization: 'Bearer wrong', - 'Content-Type': 'application/json', - Accept: 'application/json, text/event-stream', + Authorization: `Bearer ${handle.bearer}`, + Accept: 'text/event-stream', }, - body: JSON.stringify({ - jsonrpc: '2.0', - method: 'tools/list', - params: {}, - id: 4, - }), + // A hanging SSE stream would blow this, which is the regression guard. + signal: AbortSignal.timeout(5000), + }); + + assert.equal(response.status, 405); + assert.equal(response.headers.get('allow'), 'POST'); + + // Auth is still checked before the method gate. + const unauthorized = await fetch(handle.url, { + method: 'GET', + headers: { Authorization: 'Bearer wrong' }, + signal: AbortSignal.timeout(5000), }); - assert.equal(badBearerResponse.status, 401); - const badBearerPayload = parseMcpJson(await badBearerResponse.text()) as { - error?: { message?: string }; - result?: unknown; + assert.equal(unauthorized.status, 401); + const payload = parseMcpJson(await unauthorized.text()) as { + error?: { code?: number }; }; - assert.equal(badBearerPayload.result, undefined); - assert.equal(badBearerPayload.error?.message, 'Unauthorized'); + assert.equal(payload.error?.code, -32001); }); it('rejects oversized POST bodies with HTTP 413', async (t) => { @@ -191,11 +414,7 @@ describe('LoopbackMcpServer', () => { try { handle = await server.start(); } catch (error) { - if ( - error instanceof Error && - 'code' in error && - error.code === 'EPERM' - ) { + if (isSandboxListenDenied(error)) { t.skip('sandbox blocks loopback listeners on 127.0.0.1'); return; } diff --git a/middleware/test/cliBridge/toolDispatchService.test.ts b/middleware/test/cliBridge/toolDispatchService.test.ts index 25345f98..b53f377c 100644 --- a/middleware/test/cliBridge/toolDispatchService.test.ts +++ b/middleware/test/cliBridge/toolDispatchService.test.ts @@ -181,13 +181,19 @@ describe('ToolDispatchService', () => { }); const specs = service.listDispatchableToolSpecs(); + // W0-3 — advertised name-sorted (this used to be registration order: + // natives in Map order, then domain tools). Order is the only thing the + // sort changed; the precedence assertion below is unchanged. assert.deepEqual( specs.map((spec) => spec.name), - ['echo_native', 'shared_name', 'domain_ping'], + ['domain_ping', 'echo_native', 'shared_name'], ); - assert.equal(specs[0]?.input_schema.type, 'object'); - assert.equal(specs[2]?.input_schema.type, 'object'); - assert.equal(specs[1]?.description, 'native shared'); + // Look specs up by name so this stays honest if the ordering ever moves. + const byName = new Map(specs.map((spec) => [spec.name, spec])); + assert.equal(byName.get('echo_native')?.input_schema.type, 'object'); + assert.equal(byName.get('domain_ping')?.input_schema.type, 'object'); + // Native still wins the `shared_name` collision. + assert.equal(byName.get('shared_name')?.description, 'native shared'); }); it('issue #474: refuses to dispatch a not-ready plugin tool and excludes it from the list', async () => { diff --git a/middleware/test/orchestrator/deterministicToolOrder.test.ts b/middleware/test/orchestrator/deterministicToolOrder.test.ts new file mode 100644 index 00000000..8a1480ee --- /dev/null +++ b/middleware/test/orchestrator/deterministicToolOrder.test.ts @@ -0,0 +1,222 @@ +/** + * W0-3 — the tool block handed to the provider must serialize identically for + * a given tool SET, regardless of the order the tools were registered in. + * + * `buildToolsList()` stamps `cache_control: { type: 'ephemeral' }` on the last + * tool spec, which makes the whole block one Anthropic prompt-cache chunk. The + * cache keys on a byte-exact prefix, so any reordering is a silent, total cache + * miss for the tool block and everything after it. The dynamic segments are + * iterated out of Maps — plugin load order for the native registry, `created_at` + * row order for domain tools — so before the name sort, two Fly machines could + * legitimately produce different byte streams for identical configuration. + */ +import { describe, it } from 'node:test'; +import { strict as assert } from 'node:assert'; + +import type { + LlmProvider, + LlmRequest, + LlmResponse, + LlmStreamEvent, +} from '@omadia/llm-provider'; +import type { DomainTool } from '../../packages/harness-orchestrator/src/tools/domainQueryTool.js'; +import { NativeToolRegistry, Orchestrator } from '@omadia/orchestrator'; + +const providerCapabilities = { + tools: true, + vision: true, + streaming: true, + promptCaching: true, + forcedToolChoice: true, + parallelToolCalls: true, +} as const; + +const finalTextStream: LlmStreamEvent[] = [ + { type: 'text_delta', text: 'done' }, + { + type: 'final', + response: { + content: [{ type: 'text', text: 'done' }], + finishReason: 'stop', + providerFinishReason: 'end_turn', + model: 'test', + usage: { + inputTokens: 100, + outputTokens: 1, + cacheReadTokens: 0, + cacheWriteTokens: 0, + }, + }, + }, +]; + +function recordingProvider(seenRequests: LlmRequest[]): LlmProvider { + return { + id: 'anthropic', + capabilities: providerCapabilities, + complete: async (): Promise => { + throw new Error('complete() not scripted'); + }, + stream: (req: LlmRequest): AsyncIterable => { + seenRequests.push(req); + return { + async *[Symbol.asyncIterator]() { + for (const ev of finalTextStream) yield ev; + }, + }; + }, + classifyError: () => ({ retryable: false, kind: 'other' as const }), + } as unknown as LlmProvider; +} + +const minimalSpec = (name: string): Record => ({ + name, + description: `${name} for testing`, + input_schema: { type: 'object' as const, properties: {}, required: [] }, +}); + +function domainTool(name: string): DomainTool { + return { + name, + // eslint-disable-next-line @typescript-eslint/no-explicit-any + spec: minimalSpec(name) as any, + domain: `domain.${name}`, + async handle() { + return `${name}-output`; + }, + } as DomainTool; +} + +const NATIVE_NAMES = ['n_yankee', 'n_alpha', 'n_mike', 'n_bravo'] as const; +const DOMAIN_NAMES = ['d_zulu', 'd_charlie', 'd_papa', 'd_delta'] as const; + +/** + * Builds an orchestrator with the given registration orders, runs one turn, + * and returns the request the provider actually saw. + * + * `buildToolsList()` output is not observable directly: `llmProviderSeam` + * translates the Anthropic-shaped specs into the neutral `LlmRequest.tools` + * (`input_schema` → `inputSchema`, `type` → `serverType`) and collapses the + * per-tool `cache_control` into the request-level `cacheHints.tools` flag. The + * Anthropic adapter then re-stamps `cache_control` on the LAST tool — which + * `test/llmProviderAnthropicAdapter.test.ts` already covers. `LlmRequest.tools` + * is therefore the ordered payload the wire tool block is built from, and the + * right place to pin ordering. + */ +async function buildRequest( + nativeOrder: readonly string[], + domainOrder: readonly string[], +): Promise { + const registry = new NativeToolRegistry(); + for (const name of nativeOrder) { + registry.register(name, { + handler: async () => `${name}-output`, + // eslint-disable-next-line @typescript-eslint/no-explicit-any + spec: minimalSpec(name) as any, + agentId: `plugin-${name}`, + }); + } + + const seenRequests: LlmRequest[] = []; + const orchestrator = new Orchestrator({ + provider: recordingProvider(seenRequests), + model: 'test', + maxTokens: 1024, + maxToolIterations: 5, + domainTools: domainOrder.map(domainTool), + nativeToolRegistry: registry, + }); + + for await (const _ev of orchestrator.chatStream({ userMessage: 'go' })) { + // drain + } + + const request = seenRequests[0]; + assert.ok(request, 'provider received no request'); + return request; +} + +const toolNames = (request: LlmRequest): string[] => + (request.tools ?? []).map((tool) => tool.name); + +/** A deterministic shuffle, so a failure is reproducible rather than flaky. */ +function rotate(items: readonly T[], by: number): T[] { + const offset = ((by % items.length) + items.length) % items.length; + return [...items.slice(offset), ...items.slice(0, offset)]; +} + +describe('W0-3 — deterministic tool ordering', () => { + it('produces an identical name sequence for shuffled registration orders', async () => { + const runA = await buildRequest(NATIVE_NAMES, DOMAIN_NAMES); + const runB = await buildRequest( + rotate(NATIVE_NAMES, 3), + rotate(DOMAIN_NAMES, 2), + ); + const runC = await buildRequest( + [...NATIVE_NAMES].reverse(), + [...DOMAIN_NAMES].reverse(), + ); + + assert.deepEqual(toolNames(runA), toolNames(runB)); + assert.deepEqual(toolNames(runA), toolNames(runC)); + + // The dynamic segments are name-sorted; natives precede domain tools, and + // the deliberate fixed-literal prefix (memory, …) keeps its own order. + const names = toolNames(runA); + assert.deepEqual( + names.filter((n) => n.startsWith('n_')), + ['n_alpha', 'n_bravo', 'n_mike', 'n_yankee'], + ); + assert.deepEqual( + names.filter((n) => n.startsWith('d_')), + ['d_charlie', 'd_delta', 'd_papa', 'd_zulu'], + ); + assert.ok( + names.indexOf('n_yankee') < names.indexOf('d_charlie'), + 'native segment must stay ahead of the domain segment', + ); + }); + + it('keeps the fixed-literal prefix ahead of the sorted dynamic segments', async () => { + const names = toolNames(await buildRequest(NATIVE_NAMES, DOMAIN_NAMES)); + + // `memory` comes from the deliberate fixed prefix and sorts *after* every + // `d_*`/`n_*` name alphabetically — so finding it first proves the prefix + // was not swept into the sort. + assert.equal(names[0], 'memory'); + }); + + it('marks the tool block cacheable with a deterministic last element', async () => { + const runA = await buildRequest(NATIVE_NAMES, DOMAIN_NAMES); + const runB = await buildRequest( + rotate(NATIVE_NAMES, 2), + rotate(DOMAIN_NAMES, 1), + ); + + // `buildToolsList()` stamps `cache_control` on its last spec; the seam + // collapses that to `cacheHints.tools`, and the Anthropic adapter re-stamps + // the last tool (covered by test/llmProviderAnthropicAdapter.test.ts). + assert.equal(runA.cacheHints?.tools, true); + + // Which tool receives the stamp must not depend on registration order — + // that is exactly what used to drift between machines. + const lastA = toolNames(runA).at(-1); + assert.equal(lastA, toolNames(runB).at(-1)); + assert.equal(lastA, 'd_zulu'); + }); + + it('golden-snapshots byte-identically across two orchestrator rebuilds', async () => { + const first = await buildRequest(NATIVE_NAMES, DOMAIN_NAMES); + const second = await buildRequest( + rotate(NATIVE_NAMES, 1), + rotate(DOMAIN_NAMES, 3), + ); + + // The whole point of the sort: the serialized tool block — the exact bytes + // the prompt cache keys on — must match for the same tool set. + assert.equal( + JSON.stringify(first.tools), + JSON.stringify(second.tools), + ); + }); +}); diff --git a/middleware/test/orchestrator/toolOrderingInvariants.test.ts b/middleware/test/orchestrator/toolOrderingInvariants.test.ts new file mode 100644 index 00000000..88971f89 --- /dev/null +++ b/middleware/test/orchestrator/toolOrderingInvariants.test.ts @@ -0,0 +1,240 @@ +/** + * W0-3 — ordering invariants for the other three surfaces that feed a tool + * block: the standalone dispatch service (advertised to the loopback MCP + * server / CLI bridge), sub-agent tool lists, and the persisted MCP + * discovered-tools column. + * + * The load-bearing assertion is the negative one: sorting changes the + * advertised ARRAY ORDER only. Which spec wins a duplicate name — native tools + * take precedence over domain tools — is decided by Map insertion, and must be + * unchanged. + */ +import { describe, it } from 'node:test'; +import { strict as assert } from 'node:assert'; + +import type { LocalSubAgentTool } from '@omadia/plugin-api'; +import type { DomainTool } from '../../packages/harness-orchestrator/src/tools/domainQueryTool.js'; +import type { ToolGrantRow } from '../../packages/harness-orchestrator/src/registry/agentGraphStore.js'; +import { NativeToolRegistry } from '../../packages/harness-orchestrator/src/nativeToolRegistry.js'; +import { ToolDispatchService } from '../../packages/harness-orchestrator/src/toolDispatchService.js'; +import { resolveSubAgentTools } from '../../packages/harness-orchestrator/src/registry/subAgentTools.js'; +import { + normalizeDiscoveredToolOrder, + sortByToolName, +} from '../../packages/harness-orchestrator/src/toolOrdering.js'; + +const schema = { + type: 'object' as const, + properties: {}, + required: [] as string[], +}; + +function domainTool(name: string, description = name): DomainTool { + return { + name, + spec: { name, description, input_schema: schema }, + domain: `domain.${name}`, + async handle() { + return `${name}-output`; + }, + } as unknown as DomainTool; +} + +describe('W0-3 — ToolDispatchService.listDispatchableToolSpecs', () => { + it('advertises name-sorted regardless of registration order', () => { + const nativeTools = new NativeToolRegistry(); + for (const name of ['n_zulu', 'n_alpha', 'n_mike']) { + nativeTools.register(name, { + handler: async () => name, + spec: { name, description: name, input_schema: schema }, + domain: 'test.x', + }); + } + + const service = new ToolDispatchService({ + nativeTools, + domainTools: [domainTool('d_yankee'), domainTool('d_bravo')], + }); + + assert.deepEqual( + service.listDispatchableToolSpecs().map((spec) => spec.name), + ['d_bravo', 'd_yankee', 'n_alpha', 'n_mike', 'n_zulu'], + ); + }); + + it('keeps native precedence on a name collision — only the order changes', () => { + const nativeTools = new NativeToolRegistry(); + // `zzz_shared` sorts last, so if sorting were driving collision resolution + // the domain spec (registered later) could plausibly win. It must not. + nativeTools.register('zzz_shared', { + handler: async () => 'native wins', + spec: { + name: 'zzz_shared', + description: 'native', + input_schema: schema, + }, + domain: 'test.x', + }); + nativeTools.register('aaa_native_only', { + handler: async () => 'ok', + spec: { + name: 'aaa_native_only', + description: 'native-only', + input_schema: schema, + }, + domain: 'test.x', + }); + + const service = new ToolDispatchService({ + nativeTools, + domainTools: [ + domainTool('zzz_shared', 'domain'), + domainTool('mmm_domain_only', 'domain-only'), + ], + }); + + const advertised = service.listDispatchableToolSpecs(); + + // Sorted… + assert.deepEqual( + advertised.map((spec) => spec.name), + ['aaa_native_only', 'mmm_domain_only', 'zzz_shared'], + ); + // …deduplicated to one entry for the colliding name… + assert.equal( + advertised.filter((spec) => spec.name === 'zzz_shared').length, + 1, + ); + // …and it is still the NATIVE spec that survives. + assert.equal( + advertised.find((spec) => spec.name === 'zzz_shared')?.description, + 'native', + ); + }); + + it('dispatch still resolves a collision to the native handler', async () => { + const nativeTools = new NativeToolRegistry(); + nativeTools.register('zzz_shared', { + handler: async () => 'native wins', + spec: { + name: 'zzz_shared', + description: 'native', + input_schema: schema, + }, + domain: 'test.x', + }); + + const service = new ToolDispatchService({ + nativeTools, + domainTools: [domainTool('zzz_shared', 'domain')], + }); + + const result = await service.dispatch('zzz_shared', {}); + assert.equal(result.content, 'native wins'); + }); +}); + +describe('W0-3 — resolveSubAgentTools', () => { + const grant = (toolRef: string, index: number): ToolGrantRow => ({ + id: `grant-${String(index)}`, + agentId: null, + subAgentId: 'sub-1', + toolKind: 'native', + toolRef, + mcpServerId: null, + config: {}, + // Grants arrive in `created_at` order — deliberately the inverse of + // alphabetical here, so an unsorted implementation is visible. + createdAt: new Date(2026, 0, 100 - index), + }); + + const nativeTool = (toolRef: string): LocalSubAgentTool => + ({ + spec: { name: toolRef, description: toolRef, input_schema: schema }, + async handle() { + return `${toolRef}-output`; + }, + }) as unknown as LocalSubAgentTool; + + it('returns the granted tools name-sorted', () => { + const grants = ['s_zulu', 's_alpha', 's_papa', 's_bravo'].map(grant); + + const resolved = resolveSubAgentTools(grants, { nativeTool }); + + assert.deepEqual( + resolved.map((tool) => tool.spec.name), + ['s_alpha', 's_bravo', 's_papa', 's_zulu'], + ); + }); + + it('drops unresolvable grants without disturbing the order', () => { + const grants = ['s_zulu', 's_missing', 's_alpha'].map(grant); + + const resolved = resolveSubAgentTools(grants, { + nativeTool: (ref) => (ref === 's_missing' ? undefined : nativeTool(ref)), + }); + + assert.deepEqual( + resolved.map((tool) => tool.spec.name), + ['s_alpha', 's_zulu'], + ); + }); +}); + +describe('W0-3 — normalizeDiscoveredToolOrder', () => { + it('sorts discovered tools by name so rediscovery does not churn the JSONB', () => { + const fromServer = [ + { name: 'search', description: 'b' }, + { name: 'create', description: 'a' }, + { name: 'update', description: 'c' }, + ]; + + // Same set, different wire order — must normalize to identical bytes. + const shuffled = [fromServer[2], fromServer[0], fromServer[1]]; + + assert.equal( + JSON.stringify(normalizeDiscoveredToolOrder(fromServer)), + JSON.stringify(normalizeDiscoveredToolOrder(shuffled)), + ); + assert.deepEqual( + normalizeDiscoveredToolOrder(fromServer).map( + (tool) => (tool as { name: string }).name, + ), + ['create', 'search', 'update'], + ); + }); + + it('degrades rather than throwing on entries without a usable name', () => { + const malformed = [ + { name: 'beta' }, + null, + { name: 42 }, + 'not-an-object', + { name: 'alpha' }, + ]; + + const normalized = normalizeDiscoveredToolOrder(malformed); + + // Named entries sort first, unnamed keep their relative order. + assert.equal(normalized.length, malformed.length); + assert.deepEqual(normalized.slice(0, 2), [ + { name: 'alpha' }, + { name: 'beta' }, + ]); + assert.deepEqual(normalized.slice(2), [null, { name: 42 }, 'not-an-object']); + }); + + it('sortByToolName does not mutate its input', () => { + const input = [{ name: 'b' }, { name: 'a' }]; + const sorted = sortByToolName(input); + + assert.deepEqual( + input.map((item) => item.name), + ['b', 'a'], + ); + assert.deepEqual( + sorted.map((item) => item.name), + ['a', 'b'], + ); + }); +}); From 6516f7efe2d6864b574925e85713b89d59890897 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 09:47:02 +0200 Subject: [PATCH 10/90] test(mcp): first real McpManager round-trip against a live MCP server MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Nothing in the repo proved the client could complete an initialize → tools/list → tools/call sequence: mcpCallAudit dials a refused port, mcpRescan stubs listTools, and the cliBridge tests stub the server. These drive a live in-process LoopbackMcpServer, sometimes through a recording proxy that injects one transport-level failure, so retry and pool behaviour are observed rather than inferred: - listTools + callTool succeed over the wire; a second call reuses the pool - a successful call is audited as ok (previously uncovered) - a genuine Unauthorized surfaces immediately: exactly one POST, exactly one pool invalidation (fails if -32001 is classified transient again) - the shipped once-retry still fires exactly once for -32000 and then succeeds, dropping the pooled connection once - a stale token invalidates the pool and the next call reconnects --- middleware/test/mcpClient.test.ts | 361 ++++++++++++++++++++++++++++++ 1 file changed, 361 insertions(+) create mode 100644 middleware/test/mcpClient.test.ts diff --git a/middleware/test/mcpClient.test.ts b/middleware/test/mcpClient.test.ts new file mode 100644 index 00000000..3eef928b --- /dev/null +++ b/middleware/test/mcpClient.test.ts @@ -0,0 +1,361 @@ +import { strict as assert } from 'node:assert'; +import { afterEach, describe, it } from 'node:test'; +import { + createServer, + type IncomingHttpHeaders, + type IncomingMessage, + type Server, + type ServerResponse, +} from 'node:http'; +import type { AddressInfo } from 'node:net'; + +import { + LoopbackMcpServer, + McpManager, + type McpServerConfig, +} from '@omadia/orchestrator'; +import type { ToolDispatchService } from '../packages/harness-orchestrator/src/toolDispatchService.js'; + +/** + * W0-5 — the first REAL `McpManager` → MCP-server round trip in the repo. + * + * Everything that existed before only ever exercised failure paths or stubs: + * `mcpCallAudit.test.ts` dials 127.0.0.1:9 (connection refused, no handshake + * possible), `mcpRescan.test.ts` stubs `listTools`, and the cliBridge tests stub + * the loopback server. So no test ever proved the client can complete an + * `initialize` → `tools/list` → `tools/call` sequence over the wire. Everything + * later (including the eventual SDK v2 port) leans on this file. + * + * The tests below drive a live in-process `LoopbackMcpServer` — a real + * Streamable-HTTP MCP server — sometimes through a thin recording proxy that can + * inject one transport-level failure so the retry/pool behaviour is observable + * instead of inferred. + */ + +const BEARER = 'loopback-secret-token'; +const TOOL = 'ping'; + +function serverConfig(url: string, overrides: Partial = {}): McpServerConfig { + return { + id: '00000000-0000-4000-8000-00000000c0de', + name: 'loopback', + transport: 'http', + endpoint: url, + ...overrides, + }; +} + +function fakeDispatch(seen: Array<{ name: string; input: unknown }>): ToolDispatchService { + return { + async dispatch(name: string, input: unknown) { + seen.push({ name, input }); + return { content: `dispatch:${name}:${JSON.stringify(input)}` }; + }, + } as unknown as ToolDispatchService; +} + +function isSandboxListenError(error: unknown): boolean { + return ( + error instanceof Error && 'code' in error && (error as { code?: string }).code === 'EPERM' + ); +} + +/** Count how many times the manager dropped a pooled connection. `close` is the + * single invalidation point (connect failure, call failure, stale token). */ +function recordPoolInvalidations(manager: McpManager): () => readonly string[] { + const closed: string[] = []; + const original = manager.close.bind(manager); + manager.close = async (id: string): Promise => { + closed.push(id); + await original(id); + }; + return () => closed; +} + +const HOP_BY_HOP = new Set([ + 'connection', + 'content-length', + 'host', + 'keep-alive', + 'transfer-encoding', + 'upgrade', +]); + +function forwardableHeaders(headers: IncomingHttpHeaders): Record { + const out: Record = {}; + for (const [key, value] of Object.entries(headers)) { + if (HOP_BY_HOP.has(key.toLowerCase()) || value === undefined) continue; + out[key] = Array.isArray(value) ? value.join(', ') : value; + } + return out; +} + +interface RecordingProxy { + readonly url: string; + readonly postCount: () => number; + readonly toolCallCount: () => number; + readonly stop: () => Promise; +} + +/** + * Transparent HTTP proxy in front of the real MCP server(s). Records POSTs (so a + * doomed extra attempt is countable) and can answer the FIRST `tools/call` with + * a JSON-RPC transport error, which is the only way to observe the deliberate + * once-retry without stubbing the client. + * + * `targets` may hold more than one upstream: after the injected failure the + * proxy advances to the next one. The retry legitimately reconnects (the manager + * drops the pooled connection first), and a `LoopbackMcpServer` holds exactly + * one Streamable-HTTP session — a second `initialize` against the same instance + * is rejected with "Server already initialized". Two instances model the hosted + * proxy this mitigation exists for, where the reconnect lands on a healthy node. + */ +async function startRecordingProxy( + targets: readonly string[], + options: { failFirstToolCall?: boolean } = {}, +): Promise { + let posts = 0; + let toolCalls = 0; + let targetIdx = 0; + + const handle = async (req: IncomingMessage, res: ServerResponse): Promise => { + const chunks: Buffer[] = []; + for await (const chunk of req) { + chunks.push(typeof chunk === 'string' ? Buffer.from(chunk, 'utf8') : chunk); + } + const body = Buffer.concat(chunks); + if (req.method === 'POST') { + posts += 1; + const text = body.toString('utf8'); + if (text.includes('"tools/call"')) { + toolCalls += 1; + if (options.failFirstToolCall === true && toolCalls === 1) { + const id = (JSON.parse(text) as { id?: unknown }).id ?? null; + targetIdx = Math.min(targetIdx + 1, targets.length - 1); + res.writeHead(200, { 'content-type': 'application/json' }); + // -32000 "Connection closed" is what a flaky hosted proxy actually + // returns; `looksTransient` must classify it as retry-worthy. + res.end( + JSON.stringify({ + jsonrpc: '2.0', + id, + error: { code: -32000, message: 'Connection closed' }, + }), + ); + return; + } + } + } + + const upstream = await fetch(targets[targetIdx] ?? targets[0]!, { + method: req.method ?? 'GET', + headers: forwardableHeaders(req.headers), + ...(body.length > 0 ? { body } : {}), + }); + const responseHeaders: Record = {}; + upstream.headers.forEach((value, key) => { + if (!HOP_BY_HOP.has(key.toLowerCase())) responseHeaders[key] = value; + }); + res.writeHead(upstream.status, responseHeaders); + if (upstream.body) { + for await (const chunk of upstream.body) { + res.write(Buffer.from(chunk as Uint8Array)); + } + } + res.end(); + }; + + const server: Server = createServer((req, res) => { + void handle(req, res).catch(() => { + if (!res.headersSent) res.writeHead(502); + res.end(); + }); + }); + await new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(0, '127.0.0.1', () => { + server.removeListener('error', reject); + resolve(); + }); + }); + const port = (server.address() as AddressInfo).port; + return { + url: `http://127.0.0.1:${String(port)}/mcp`, + postCount: () => posts, + toolCallCount: () => toolCalls, + stop: () => + new Promise((resolve) => { + server.close(() => resolve()); + }), + }; +} + +describe('McpManager against a live MCP server (W0-5)', () => { + const servers: LoopbackMcpServer[] = []; + let proxy: RecordingProxy | undefined; + let manager: McpManager | undefined; + + afterEach(async () => { + await manager?.closeAll(); + await proxy?.stop(); + for (const s of servers.splice(0)) await s.stop(); + manager = undefined; + proxy = undefined; + }); + + async function startServer( + t: { skip: (reason: string) => void }, + seen: Array<{ name: string; input: unknown }> = [], + ): Promise { + const server = new LoopbackMcpServer({ + dispatch: fakeDispatch(seen), + bearer: BEARER, + tools: [ + { + name: TOOL, + description: 'echo the input back', + input_schema: { + type: 'object', + properties: { value: { type: 'string' } }, + }, + }, + ], + }); + try { + const handle = await server.start(); + servers.push(server); + return handle.url; + } catch (error) { + if (isSandboxListenError(error)) { + t.skip('sandbox blocks loopback listeners on 127.0.0.1'); + return undefined; + } + throw error; + } + } + + it('completes listTools AND callTool over the wire', async (t) => { + const seen: Array<{ name: string; input: unknown }> = []; + const url = await startServer(t, seen); + if (!url) return; + + manager = new McpManager(); + const cfg = serverConfig(url, { headers: { Authorization: `Bearer ${BEARER}` } }); + + const tools = await manager.listTools(cfg); + assert.equal(tools.length, 1); + assert.equal(tools[0]?.name, TOOL); + assert.equal(tools[0]?.description, 'echo the input back'); + assert.equal(tools[0]?.inputSchema?.['type'], 'object'); + + const result = await manager.callTool(cfg, TOOL, { value: 'hello' }); + assert.equal(result, 'dispatch:ping:{"value":"hello"}'); + assert.deepEqual(seen, [{ name: TOOL, input: { value: 'hello' } }]); + + // A second call must reuse the pooled connection, not reconnect. + const again = await manager.callTool(cfg, TOOL, { value: 'again' }); + assert.equal(again, 'dispatch:ping:{"value":"again"}'); + }); + + it('audits a successful call as ok (the audit path had no success coverage)', async (t) => { + const url = await startServer(t); + if (!url) return; + + const entries: Array<{ ok: boolean; toolName: string; error: string | null }> = []; + manager = new McpManager({ + onToolCall: (e) => entries.push({ ok: e.ok, toolName: e.toolName, error: e.error }), + }); + const cfg = serverConfig(url, { headers: { Authorization: `Bearer ${BEARER}` } }); + + await manager.callTool(cfg, TOOL, {}); + assert.deepEqual(entries, [{ ok: true, toolName: TOOL, error: null }]); + }); + + it('surfaces a genuine Unauthorized immediately — no retry, one pool invalidation', async (t) => { + const url = await startServer(t); + if (!url) return; + proxy = await startRecordingProxy([url]); + + // The loopback server answers a bad bearer with JSON-RPC code -32001, which + // `looksTransient` used to match as a bare number — so a real auth failure + // got one doomed retry before the user saw the authorize prompt. + manager = new McpManager({ + auth: { + getToken: async () => 'stale-token', + onAuthFailure: async () => '🔒 authorize here: https://auth.example/authorize', + }, + }); + const closed = recordPoolInvalidations(manager); + + const result = await manager.callTool(serverConfig(proxy.url), TOOL, {}); + + assert.match(result, /authorize here/); + assert.equal( + proxy.postCount(), + 1, + 'an Unauthorized must not be retried — a second POST means -32001 is still classified transient', + ); + assert.equal( + closed().length, + 1, + 'the stale-token connection must be invalidated exactly once (handleFailure), not twice', + ); + }); + + it('retries a genuinely transient failure exactly once, then succeeds', async (t) => { + const seen: Array<{ name: string; input: unknown }> = []; + const first = await startServer(t, seen); + if (!first) return; + const second = await startServer(t, seen); + if (!second) return; + proxy = await startRecordingProxy([first, second], { failFirstToolCall: true }); + + manager = new McpManager(); + const closed = recordPoolInvalidations(manager); + const cfg = serverConfig(proxy.url, { + headers: { Authorization: `Bearer ${BEARER}` }, + }); + + const result = await manager.callTool(cfg, TOOL, { value: 'retry-me' }); + + // The shipped once-retry mitigation (flaky hosted proxy) must stay: the + // first tools/call fails transiently, the second one succeeds. + assert.equal(result, 'dispatch:ping:{"value":"retry-me"}'); + assert.equal(proxy.toolCallCount(), 2, 'exactly one retry — no more, no fewer'); + assert.equal( + closed().length, + 1, + 'the retry must drop the pooled connection once so it reconnects fresh', + ); + assert.deepEqual(seen, [{ name: TOOL, input: { value: 'retry-me' } }]); + }); + + it('a stale token invalidates the pool and the next call reconnects and succeeds', async (t) => { + const url = await startServer(t); + if (!url) return; + + const tokens = ['stale-token', BEARER]; + manager = new McpManager({ + auth: { + getToken: async () => tokens.shift() ?? BEARER, + // Not an OAuth-protected server: the raw failure must stand so the + // pooling behaviour is what the assertions observe. + onAuthFailure: async () => null, + }, + }); + const closed = recordPoolInvalidations(manager); + + const failed = await manager.callTool(serverConfig(url), TOOL, {}); + assert.match(failed, /^Error: could not connect to MCP server "loopback"/); + assert.equal(closed().length, 1, 'the rejected token must be evicted from the pool'); + + // Same server, fresh (valid) token — a reconnect must happen and succeed. + const recovered = await manager.callTool(serverConfig(url), TOOL, { value: 'after-refresh' }); + assert.equal(recovered, 'dispatch:ping:{"value":"after-refresh"}'); + assert.equal( + closed().length, + 1, + 'a successful reconnect must not invalidate anything further', + ); + }); +}); From 3b87ffe6d48bd0bc537b214e0a8bc52a7d758c5e Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 09:51:42 +0200 Subject: [PATCH 11/90] feat(mcp): mark the legacy HTTP+SSE transport deprecated (MCP 2026-07-28) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MCP 2026-07-28 reclassifies the legacy HTTP+SSE transport as Deprecated with a minimum 12-month removal window. Discourage 'sse' for NEW registrations while keeping every existing SSE server fully working — no protocol work, SSEClientTransport stays wired and the DB CHECK is untouched. - DEPRECATED_MCP_TRANSPORTS + isDeprecatedMcpTransport in mcpClient.ts as the single source of truth, re-exported from @omadia/orchestrator. - mcpNode() gains an additive transportDeprecated flag derived from it (no migration); exported for unit tests. - Marketplace import path (the second way an sse row can be minted): prefer an http remote when a catalog entry offers both, still allow an sse-only entry but flag it via McpCatalogEntry.transportDeprecated. - web-ui McpServerNode gains optional transportDeprecated; McpTransport keeps 'sse' (published plugin contract stays as-is). Refs #541 --- .../harness-orchestrator/src/index.ts | 3 + .../harness-orchestrator/src/mcp/mcpClient.ts | 33 +++++++ middleware/src/routes/agentBuilder.ts | 13 ++- middleware/src/services/mcpRegistryClient.ts | 86 +++++++++++++------ web-ui/app/_lib/agentBuilder.ts | 10 +++ 5 files changed, 120 insertions(+), 25 deletions(-) diff --git a/middleware/packages/harness-orchestrator/src/index.ts b/middleware/packages/harness-orchestrator/src/index.ts index 75acc579..8dd1232e 100644 --- a/middleware/packages/harness-orchestrator/src/index.ts +++ b/middleware/packages/harness-orchestrator/src/index.ts @@ -125,6 +125,8 @@ export type { ToolGrantRow, } from './registry/agentGraphStore.js'; export { + DEPRECATED_MCP_TRANSPORTS, + isDeprecatedMcpTransport, McpManager, mcpNativeHandler, mcpNativeToolName, @@ -132,6 +134,7 @@ export { mcpToolToNativeSpec, } from './mcp/mcpClient.js'; export type { + DeprecatedMcpTransport, McpAuthProvider, McpCallerKind, McpCallGuard, diff --git a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts index d3ed9d0c..73b4d1c6 100644 --- a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts +++ b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts @@ -56,6 +56,39 @@ const LENIENT_CALL_TOOL_RESULT_SCHEMA = CallToolResultSchema.extend({ export type McpTransportKind = 'stdio' | 'http' | 'sse'; +/** + * Transports the MCP specification has formally deprecated (issue #541). + * + * The MCP 2026-07-28 revision reclassifies the legacy HTTP+SSE transport + * (two endpoints: `GET /sse` for the event stream plus a separate POST + * endpoint for messages) as **Deprecated**, with a minimum 12-month removal + * window. Streamable HTTP (our `'http'`) is the migration target. + * + * omadia therefore *discourages* `'sse'` for NEW registrations — the operator + * picker hides it behind a "show deprecated transports" toggle, and the + * marketplace importer prefers an `http` remote when a catalog entry offers + * both. Nothing is hard-blocked: the removal window is open, existing rows + * keep working unchanged (`SSEClientTransport` stays wired in + * `McpManager.transportFor`), and the `agent_mcp_servers.transport` CHECK + * constraint still accepts `'sse'`, so a legacy server can be re-created. + * + * This array is the single source of truth for "which transports are + * deprecated" — the API serializer, the marketplace importer, and the web-ui + * all derive from it rather than hard-coding `'sse'`. + */ +export const DEPRECATED_MCP_TRANSPORTS = ['sse'] as const; + +/** A transport listed in {@link DEPRECATED_MCP_TRANSPORTS}. */ +export type DeprecatedMcpTransport = (typeof DEPRECATED_MCP_TRANSPORTS)[number]; + +/** + * True when `transport` is deprecated by the MCP spec. Takes a plain `string` + * so callers holding an unvalidated DB/catalog value can ask without casting. + */ +export function isDeprecatedMcpTransport(transport: string): boolean { + return (DEPRECATED_MCP_TRANSPORTS as readonly string[]).includes(transport); +} + export interface McpServerConfig { readonly id: string; readonly name: string; diff --git a/middleware/src/routes/agentBuilder.ts b/middleware/src/routes/agentBuilder.ts index bec4a697..fc7a73da 100644 --- a/middleware/src/routes/agentBuilder.ts +++ b/middleware/src/routes/agentBuilder.ts @@ -30,6 +30,7 @@ import { type ToolGrantRow, } from '@omadia/orchestrator'; import { + isDeprecatedMcpTransport, McpManager, mcpToolNameFromRef, turnContext, @@ -2478,11 +2479,21 @@ async function withToolVerdicts( })); } -function mcpNode(s: McpServerRow) { +/** + * Row → API node for an MCP server. Exported for unit tests (issue #541). + * + * `transportDeprecated` is derived from `DEPRECATED_MCP_TRANSPORTS`, never + * hard-coded: the web-ui uses it to badge legacy rows without duplicating the + * spec's deprecation list. Purely additive — the row's transport is returned + * unchanged and no DB constraint moved. + */ +export function mcpNode(s: McpServerRow) { return { id: s.id, name: s.name, transport: s.transport, + /** MCP 2026-07-28 deprecated this transport (see DEPRECATED_MCP_TRANSPORTS). */ + transportDeprecated: isDeprecatedMcpTransport(s.transport), endpoint: s.endpoint, status: s.status, lastDiscoveredAt: s.lastDiscoveredAt ? s.lastDiscoveredAt.toISOString() : null, diff --git a/middleware/src/services/mcpRegistryClient.ts b/middleware/src/services/mcpRegistryClient.ts index 52042a43..3d13556b 100644 --- a/middleware/src/services/mcpRegistryClient.ts +++ b/middleware/src/services/mcpRegistryClient.ts @@ -12,7 +12,7 @@ * official API. */ -import type { McpConfigField } from '@omadia/orchestrator'; +import { isDeprecatedMcpTransport, type McpConfigField } from '@omadia/orchestrator'; export interface McpRegistryConfig { readonly id: string; @@ -33,6 +33,10 @@ export interface McpCatalogEntry { /** Derived connection candidate; null when the entry only ships packages * we cannot translate into a transport (then it is browse-only). */ readonly transport: 'http' | 'sse' | 'stdio' | null; + /** Issue #541 — the derived transport is deprecated by MCP 2026-07-28. Only + * true when the entry offers no non-deprecated alternative; the import is + * still allowed (removal window open), the operator just gets warned. */ + readonly transportDeprecated: boolean; readonly endpoint: string | null; readonly license: string | null; readonly author: string | null; @@ -100,6 +104,56 @@ function deriveAuthor(name: string, repoUrl: string | null): string | null { return ghRepo?.[1] ?? null; } +type RemoteCandidate = { + readonly transport: 'http' | 'sse'; + readonly endpoint: string; +}; + +/** + * Pick the connection candidate from a catalog entry's `remotes[]`. + * + * Second registration path for issue #541: a marketplace/catalog import is the + * other way an `sse` row can be minted, so the deprecation has to be enforced + * here too — a UI-only change would keep importing legacy SSE servers. + * + * When an entry advertises BOTH a Streamable-HTTP and a legacy HTTP+SSE remote + * we now take the `http` one (MCP 2026-07-28 deprecated HTTP+SSE, Streamable + * HTTP is the migration target). `sse` is still returned when it is the only + * remote offered — nothing is hard-blocked while the removal window is open; + * the row is flagged `transportDeprecated` instead. + * + * Every candidate must clear the same UNTRUSTED-remote validation as before + * (https only, host not internal/metadata); unlike the previous version this + * scans all remotes rather than only the first, which is what makes the + * preference possible and also rescues entries whose first remote is malformed. + */ +function pickRemoteCandidate(remotes: readonly unknown[]): RemoteCandidate | null { + const candidates: RemoteCandidate[] = []; + for (const r of remotes) { + if (!r || typeof r !== 'object') continue; + const remote = r as Record; + const kind = str(remote['type'] ?? remote['transport_type'] ?? remote['transport']); + const url = str(remote['url']); + // Catalog entries are UNTRUSTED (codex fold): only well-formed https + // remotes become endpoints — a catalog must not be able to point the + // middleware at plain-http, custom schemes, or metadata addresses. + if (!url || !kind) continue; + try { + const parsed = new URL(url); + // https only, and the host must clear the untrusted-remote block — + // an untrusted catalog must not yield an internal/metadata endpoint. + if (parsed.protocol !== 'https:' || !isUntrustedRemoteHostSafe(parsed.hostname)) continue; + candidates.push({ transport: kind.includes('sse') ? 'sse' : 'http', endpoint: url }); + } catch { + /* malformed remote URL → not a candidate */ + } + } + // Prefer the first non-deprecated candidate; fall back to the first overall. + return ( + candidates.find((c) => !isDeprecatedMcpTransport(c.transport)) ?? candidates[0] ?? null + ); +} + function normalizeEntry(raw: Record): McpCatalogEntry | null { // Official API wraps the server.json under `server`; accept both. const server = (raw['server'] ?? raw) as Record; @@ -111,28 +165,10 @@ function normalizeEntry(raw: Record): McpCatalogEntry | null { let transport: McpCatalogEntry['transport'] = null; let endpoint: string | null = null; const remotes = Array.isArray(server['remotes']) ? server['remotes'] : []; - const remote = remotes.find( - (r): r is Record => !!r && typeof r === 'object', - ); - if (remote) { - const kind = str(remote['type'] ?? remote['transport_type'] ?? remote['transport']); - const url = str(remote['url']); - // Catalog entries are UNTRUSTED (codex fold): only well-formed https - // remotes become endpoints — a catalog must not be able to point the - // middleware at plain-http, custom schemes, or metadata addresses. - if (url && kind) { - try { - const parsed = new URL(url); - // https only, and the host must clear the untrusted-remote block — - // an untrusted catalog must not yield an internal/metadata endpoint. - if (parsed.protocol === 'https:' && isUntrustedRemoteHostSafe(parsed.hostname)) { - transport = kind.includes('sse') ? 'sse' : 'http'; - endpoint = url; - } - } catch { - /* malformed remote URL → browse-only entry */ - } - } + const picked = pickRemoteCandidate(remotes); + if (picked) { + transport = picked.transport; + endpoint = picked.endpoint; } if (!endpoint) { const packages = Array.isArray(server['packages']) ? server['packages'] : []; @@ -174,6 +210,7 @@ function normalizeEntry(raw: Record): McpCatalogEntry | null { str(server['version']) ?? str((server['version_detail'] as Record | undefined)?.['version']), transport, + transportDeprecated: transport !== null && isDeprecatedMcpTransport(transport), endpoint, license: str(server['license']) ?? str(raw['license']), author: deriveAuthor(name, repoUrl), @@ -221,6 +258,7 @@ function normalizeSmitheryEntry(raw: Record): McpCatalogEntry | version: null, // Remote Smithery servers are streamable-http; endpoint deferred to connect. transport: remote ? 'http' : null, + transportDeprecated: false, endpoint: null, license: null, author: str(raw['owner']) ?? str(raw['namespace']), @@ -516,7 +554,7 @@ export class McpRegistryClient { if (registry.kind === 'smithery') { // A minimal entry; resolveSmitheryEndpoint fills the endpoint + enriches // name/description from the detail doc. - return { id: entryId, name: entryId, description: null, version: null, transport: 'http', endpoint: null, license: null, author: null, sourceUrl: null }; + return { id: entryId, name: entryId, description: null, version: null, transport: 'http', transportDeprecated: false, endpoint: null, license: null, author: null, sourceUrl: null }; } const results = await this.fetchCatalog(registry, entryId); const exact = results.find((e) => e.id === entryId); diff --git a/web-ui/app/_lib/agentBuilder.ts b/web-ui/app/_lib/agentBuilder.ts index 433a3769..ae489bef 100644 --- a/web-ui/app/_lib/agentBuilder.ts +++ b/web-ui/app/_lib/agentBuilder.ts @@ -207,10 +207,20 @@ export interface McpDiscoveredTool { export type McpTransport = 'stdio' | 'http' | 'sse'; +/** + * Transports MCP 2026-07-28 deprecated (issue #541). Mirrors the middleware's + * `DEPRECATED_MCP_TRANSPORTS`; the union above deliberately keeps `'sse'` — + * legacy servers stay fully usable during the 12-month removal window, they are + * only discouraged for new registrations. + */ +export const DEPRECATED_MCP_TRANSPORTS: readonly McpTransport[] = ['sse']; + export interface McpServerNode { id: string; name: string; transport: McpTransport; + /** Issue #541 — server-derived deprecation flag; absent on older middleware. */ + transportDeprecated?: boolean; endpoint: string | null; status: NodeStatus; lastDiscoveredAt: string | null; From 6090fcb9291981028b551d09923b78e2e6c12076 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 09:54:20 +0200 Subject: [PATCH 12/90] feat(web-ui): gate the deprecated sse transport behind an operator toggle Issue #541 acceptance 4 + 6. http (Streamable HTTP) stays the default and the only remote option shown; 'sse' appears in the picker only after ticking 'Show deprecated transports', labelled '(deprecated)'. Existing sse rows get a Deprecated badge in the transport column with a hint pointing at Streamable HTTP as the migration target. Nothing is hard-blocked: the MCP removal window is at least 12 months, so an operator can still deliberately register a legacy SSE server. i18n: adminMcp.servers.{transportDeprecated,transportDeprecatedHint, showDeprecatedTransports,deprecatedOption} in both en.json and de.json. Refs #541 --- web-ui/app/_lib/agentBuilder.ts | 2 ++ web-ui/app/admin/mcp/page.tsx | 53 +++++++++++++++++++++++++++++++-- web-ui/messages/de.json | 4 +++ web-ui/messages/en.json | 4 +++ 4 files changed, 61 insertions(+), 2 deletions(-) diff --git a/web-ui/app/_lib/agentBuilder.ts b/web-ui/app/_lib/agentBuilder.ts index ae489bef..c0749150 100644 --- a/web-ui/app/_lib/agentBuilder.ts +++ b/web-ui/app/_lib/agentBuilder.ts @@ -814,6 +814,8 @@ export interface McpCatalogEntry { description: string | null; version: string | null; transport: McpTransport | null; + /** Issue #541 — the catalog only offered a deprecated (HTTP+SSE) remote. */ + transportDeprecated?: boolean; endpoint: string | null; license: string | null; author: string | null; diff --git a/web-ui/app/admin/mcp/page.tsx b/web-ui/app/admin/mcp/page.tsx index dc1906a9..e9639557 100644 --- a/web-ui/app/admin/mcp/page.tsx +++ b/web-ui/app/admin/mcp/page.tsx @@ -14,6 +14,7 @@ import { ackMcpToolVerdict, addMcpRegistry, createMcpServer, + DEPRECATED_MCP_TRANSPORTS, rescanAllMcpServers, testCallMcpTool, deleteGraphEdge, @@ -201,6 +202,15 @@ function worstSeverityOf(server: McpServerNode): SkillVerdictSeverity { return worst; } +/** + * Issue #541 — badge an existing row whose transport MCP 2026-07-28 deprecated. + * Trusts the middleware's `transportDeprecated` when present and falls back to + * the local list, so the badge still shows against an older middleware build. + */ +function isDeprecatedTransport(server: McpServerNode): boolean { + return server.transportDeprecated ?? DEPRECATED_MCP_TRANSPORTS.includes(server.transport); +} + function ServersPane({ onAssign }: { onAssign: (serverId: string) => void }): React.ReactElement { const t = useTranslations('adminMcp'); const [servers, setServers] = useState(null); @@ -212,6 +222,11 @@ function ServersPane({ onAssign }: { onAssign: (serverId: string) => void }): Re const [busy, setBusy] = useState(null); const [name, setName] = useState(''); const [transport, setTransport] = useState('http'); + // Issue #541 — MCP 2026-07-28 deprecated the legacy HTTP+SSE transport. It is + // hidden from the picker by default (http/Streamable HTTP stays the default + // choice) but never blocked: the removal window is open, so an operator must + // still be able to register a legacy SSE server on purpose. + const [showDeprecated, setShowDeprecated] = useState(false); const [endpoint, setEndpoint] = useState(''); const refresh = useCallback(async () => { @@ -288,13 +303,35 @@ function ServersPane({ onAssign }: { onAssign: (serverId: string) => void }): Re +
+ + ); +} diff --git a/web-ui/app/_components/chat/__tests__/McpInputCard.test.tsx b/web-ui/app/_components/chat/__tests__/McpInputCard.test.tsx new file mode 100644 index 00000000..8a3255d6 --- /dev/null +++ b/web-ui/app/_components/chat/__tests__/McpInputCard.test.tsx @@ -0,0 +1,141 @@ +import { screen } from '@testing-library/react'; +import userEvent from '@testing-library/user-event'; +import { describe, expect, it, vi } from 'vitest'; + +import type { PendingMcpInput } from '../../../_lib/chatSessions'; +import { renderWithIntl } from '../../../_lib/test-utils'; +import { + MCP_INPUT_REPLY_PREFIX, + McpInputCard, + formatMcpInputReply, +} from '../McpInputCard'; + +const REQUEST: PendingMcpInput = { + correlationId: 'corr-abc', + serverName: 'Kunden-CRM', + serverId: 'srv-1', + toolName: 'create_ticket', + prompt: 'Bitte Kundennummer und PIN angeben.', + fields: [ + { name: 'customerNumber', label: 'Kundennummer', required: true }, + { name: 'pin', label: 'PIN', secret: true }, + { name: 'note', label: 'Notiz', description: 'Optional' }, + ], +}; + +describe('#544 W2-1 McpInputCard', () => { + it('MUTATION CHECK: names the asking server', () => { + renderWithIntl( + {}} />, + ); + // The security control: a hostile MCP server must not be able to render a + // credential prompt that reads as omadia's own UI. Removing `serverName` + // from the heading turns this red. + expect( + screen.getByText(/“Kunden-CRM” needs additional details for “create_ticket”/), + ).toBeInTheDocument(); + }); + + it('warns that the values leave for an external server', () => { + renderWithIntl( + {}} />, + ); + // Two independent mentions: the heading and the explicit warning line. + expect(screen.getAllByText(/Kunden-CRM/).length).toBeGreaterThan(1); + }); + + it("renders the server's prompt as quoted, attributed text", () => { + renderWithIntl( + {}} />, + ); + const quote = screen.getByText('Bitte Kundennummer und PIN angeben.'); + expect(quote.tagName.toLowerCase()).toBe('blockquote'); + }); + + it('renders one labelled input per field and masks secrets', () => { + renderWithIntl( + {}} />, + ); + expect(screen.getByLabelText(/Kundennummer/)).toHaveAttribute('type', 'text'); + expect(screen.getByLabelText(/PIN/)).toHaveAttribute('type', 'password'); + expect(screen.getByLabelText(/Notiz/)).toBeInTheDocument(); + }); + + it('MUTATION CHECK: submit stays disabled until every required field is filled', async () => { + renderWithIntl( + {}} />, + ); + const submit = screen.getByRole('button'); + expect(submit).toBeDisabled(); + // Filling an OPTIONAL field must not unlock it — a required-field check that + // merely counted non-empty inputs would pass without this step. + await userEvent.type(screen.getByLabelText(/Notiz/), 'egal'); + expect(submit).toBeDisabled(); + await userEvent.type(screen.getByLabelText(/Kundennummer/), 'K-1234'); + expect(submit).toBeEnabled(); + }); + + it('MUTATION CHECK: submits the envelope with the correlation id and only filled fields', async () => { + const onSubmit = vi.fn(); + renderWithIntl( + , + ); + await userEvent.type(screen.getByLabelText(/Kundennummer/), 'K-1234'); + await userEvent.type(screen.getByLabelText(/PIN/), '9876'); + await userEvent.click(screen.getByRole('button')); + + expect(onSubmit).toHaveBeenCalledTimes(1); + const message = onSubmit.mock.calls[0]![0] as string; + // Asserting the PARSED payload, not that a callback fired: the orchestrator + // resolves this exact shape, so a malformed envelope would be a silent + // no-op in production. + expect(message.startsWith(MCP_INPUT_REPLY_PREFIX)).toBe(true); + const parsed = JSON.parse(message.slice(MCP_INPUT_REPLY_PREFIX.length)) as { + correlationId: string; + inputResponses: Record; + }; + expect(parsed.correlationId).toBe('corr-abc'); + // The untouched optional field is ABSENT, not an empty string, so the server + // can tell "skipped" from "explicitly empty". + expect(parsed.inputResponses).toEqual({ customerNumber: 'K-1234', pin: '9876' }); + }); + + it('MUTATION CHECK: cannot be submitted twice', async () => { + const onSubmit = vi.fn(); + renderWithIntl( + , + ); + await userEvent.type(screen.getByLabelText(/Kundennummer/), 'K-1'); + const submit = screen.getByRole('button'); + await userEvent.click(submit); + await userEvent.click(submit); + // The correlation id is single-use server-side, so a second submit could + // only ever fail. Dropping the `submitted` latch turns this into 2. + expect(onSubmit).toHaveBeenCalledTimes(1); + expect(submit).toBeDisabled(); + }); + + it('is inert while a turn is in flight', async () => { + const onSubmit = vi.fn(); + renderWithIntl( + , + ); + expect(screen.getByRole('button')).toBeDisabled(); + expect(screen.getByLabelText(/Kundennummer/)).toBeDisabled(); + }); + + it('renders without a server prompt', () => { + const { prompt: _drop, ...noPrompt } = REQUEST; + renderWithIntl( + {}} />, + ); + // Attribution survives even when the server sent no prose at all. + expect(screen.getAllByText(/Kunden-CRM/).length).toBeGreaterThan(0); + expect(screen.queryByRole('blockquote')).not.toBeInTheDocument(); + }); + + it('formatMcpInputReply is a stable, parseable envelope', () => { + const wire = formatMcpInputReply('c1', { a: 'b' }); + expect(wire).toBe(`${MCP_INPUT_REPLY_PREFIX} {"correlationId":"c1","inputResponses":{"a":"b"}}`); + }); +}); diff --git a/web-ui/app/_lib/__tests__/stripStaleInteractives.test.ts b/web-ui/app/_lib/__tests__/stripStaleInteractives.test.ts new file mode 100644 index 00000000..6cf26057 --- /dev/null +++ b/web-ui/app/_lib/__tests__/stripStaleInteractives.test.ts @@ -0,0 +1,94 @@ +import { describe, expect, it } from 'vitest'; + +import { + stripStaleInteractives, + type Message, + type PendingMcpInput, +} from '../chatSessions'; + +/** + * #544 W2-1 — a mutation run found this logic completely uncovered while it was + * inline in `chat/page.tsx`, including the pre-existing `pendingUserChoice` + * half. A stale MCP input form is the worst case: its `correlationId` is + * single-use server-side, so re-submitting an old form can only fail. + */ + +const MCP_INPUT: PendingMcpInput = { + correlationId: 'corr-1', + serverName: 'Kunden-CRM', + serverId: 'srv-1', + toolName: 'create_ticket', + fields: [{ name: 'customerNumber', required: true }], +}; + +function assistant(over?: Partial): Message { + return { + id: 'm1', + role: 'assistant', + content: 'answer', + tools: [], + ...over, + } as Message; +} + +describe('#544 stripStaleInteractives', () => { + it('MUTATION CHECK: strips a stale MCP input form', () => { + const out = stripStaleInteractives([assistant({ pendingMcpInput: MCP_INPUT })]); + expect(out[0]!.pendingMcpInput).toBeUndefined(); + // The key must be GONE, not merely falsy — the render guard is a truthiness + // check, but an `undefined`-valued key would still serialize into storage. + expect('pendingMcpInput' in out[0]!).toBe(false); + }); + + it('MUTATION CHECK: strips a stale choice card and follow-ups (pre-existing behaviour)', () => { + const out = stripStaleInteractives([ + assistant({ + pendingUserChoice: { question: 'Q?', options: [{ label: 'A', value: 'a' }] }, + followUpOptions: [{ label: 'more', prompt: 'more' }], + }), + ]); + expect('pendingUserChoice' in out[0]!).toBe(false); + expect('followUpOptions' in out[0]!).toBe(false); + }); + + it('strips all three at once', () => { + const out = stripStaleInteractives([ + assistant({ + pendingMcpInput: MCP_INPUT, + pendingUserChoice: { question: 'Q?', options: [{ label: 'A', value: 'a' }] }, + followUpOptions: [{ label: 'more', prompt: 'more' }], + }), + ]); + expect('pendingMcpInput' in out[0]!).toBe(false); + expect('pendingUserChoice' in out[0]!).toBe(false); + expect('followUpOptions' in out[0]!).toBe(false); + }); + + it('keeps everything else on the message', () => { + const out = stripStaleInteractives([ + assistant({ pendingMcpInput: MCP_INPUT, content: 'keep me', turnId: 't1' }), + ]); + expect(out[0]!.content).toBe('keep me'); + expect(out[0]!.turnId).toBe('t1'); + expect(out[0]!.role).toBe('assistant'); + }); + + it('MUTATION CHECK: returns untouched messages by IDENTITY', () => { + const plain = assistant(); + const out = stripStaleInteractives([plain]); + // Reference equality matters: React short-circuits re-renders on it, so a + // version that always spread every message would re-render the whole + // transcript on every send. `toEqual` would not catch that. + expect(out[0]).toBe(plain); + }); + + it('does not mutate the input', () => { + const input = assistant({ pendingMcpInput: MCP_INPUT }); + stripStaleInteractives([input]); + expect(input.pendingMcpInput).toEqual(MCP_INPUT); + }); + + it('handles an empty transcript', () => { + expect(stripStaleInteractives([])).toEqual([]); + }); +}); diff --git a/web-ui/app/_lib/chatSessions.ts b/web-ui/app/_lib/chatSessions.ts index eb76133e..d8dd3b5b 100644 --- a/web-ui/app/_lib/chatSessions.ts +++ b/web-ui/app/_lib/chatSessions.ts @@ -95,6 +95,66 @@ export interface PendingUserChoice { options: Array<{ label: string; value: string }>; } +/** + * Strip every one-shot interactive affordance from older assistant messages. + * + * Called when a new user message is sent, so button rows and input forms + * disappear from the history instead of inviting a second click. Extracted from + * `chat/page.tsx` to be testable: a mutation run showed the inline version was + * completely uncovered, and a stale MCP input form is worse than a stale button + * row — its `correlationId` is single-use server-side, so re-submitting it can + * only fail. + * + * Pure and non-mutating: messages with nothing to strip are returned by + * identity, so React's reference equality still short-circuits their re-render. + */ +export function stripStaleInteractives(messages: readonly Message[]): Message[] { + return messages.map((m) => { + if (!m.pendingUserChoice && !m.followUpOptions && !m.pendingMcpInput) return m; + const { + pendingUserChoice: _dropChoice, + pendingMcpInput: _dropMcpInput, + followUpOptions: _dropFollowUps, + ...rest + } = m; + return rest; + }); +} + +/** One free-text field an MCP server asked for (#544 W2-1). */ +export interface McpInputCardField { + name: string; + label?: string; + description?: string; + /** + * Render the input masked. ADVISORY: the value still travels to the + * third-party MCP server verbatim, so the UI must not imply it is protected. + */ + secret?: boolean; + required?: boolean; +} + +/** + * Mid-call input request from an MCP tool (#544 W2-1, MRTR + * `resultType: "input_required"`). The turn ended so the user can fill the + * fields in; submitting sends a fresh turn carrying the reply envelope, and the + * orchestrator replays the parked tool call. + * + * Mirrors the backend's `PendingMcpInputCard`. A SIBLING of + * {@link PendingUserChoice}, not a variant of it: free-text fields, not buttons. + * + * `serverName` MUST be rendered — see `McpInputCard.tsx`. + */ +export interface PendingMcpInput { + correlationId: string; + serverName: string; + serverId: string; + toolName: string; + /** Server-supplied prose. UNTRUSTED text — render as text, never as markup. */ + prompt?: string; + fields: McpInputCardField[]; +} + /** * Non-blocking 1-click refinement options attached below an answer. Each * click submits `prompt` as a fresh user message. Mirrors the backend's @@ -408,6 +468,13 @@ export interface Message { * the buttons disappear on re-renders of the conversation history. */ pendingUserChoice?: PendingUserChoice; + /** + * #544 W2-1 — set when the turn ended because an MCP tool needs mid-call user + * input. Cleared once the user submits (or types a fresh message) so the form + * disappears on re-renders of the conversation history, exactly like + * `pendingUserChoice`. + */ + pendingMcpInput?: PendingMcpInput; /** * Refinement buttons attached below the answer (from `suggest_follow_ups`). * Cleared once the user commits to a follow-up or types a fresh message, diff --git a/web-ui/app/_lib/chatStreamEvents.ts b/web-ui/app/_lib/chatStreamEvents.ts index cc28907c..086c6f49 100644 --- a/web-ui/app/_lib/chatStreamEvents.ts +++ b/web-ui/app/_lib/chatStreamEvents.ts @@ -14,6 +14,7 @@ import type { NudgeEvent, OutgoingFileAttachment, PalaiaExcerpt, + PendingMcpInput, PendingUserChoice, PlanSnapshot, PrivacyReceipt, @@ -129,6 +130,8 @@ export type ChatStreamEvent = attachments?: DiagramAttachment[]; fileAttachments?: OutgoingFileAttachment[]; pendingUserChoice?: PendingUserChoice; + /** #544 W2-1 — see `Message.pendingMcpInput`. Sibling of the above. */ + pendingMcpInput?: PendingMcpInput; followUpOptions?: FollowUpOption[]; privacyReceipt?: PrivacyReceipt; maskedValues?: readonly string[]; @@ -358,6 +361,9 @@ function foldIntoMessage(m: Message, event: ChatStreamEvent): Message { ...(event.pendingUserChoice ? { pendingUserChoice: event.pendingUserChoice } : {}), + ...(event.pendingMcpInput + ? { pendingMcpInput: event.pendingMcpInput } + : {}), ...(event.followUpOptions && event.followUpOptions.length > 0 ? { followUpOptions: event.followUpOptions } : {}), diff --git a/web-ui/app/chat/page.tsx b/web-ui/app/chat/page.tsx index 127d7f85..eebf9f43 100644 --- a/web-ui/app/chat/page.tsx +++ b/web-ui/app/chat/page.tsx @@ -39,6 +39,7 @@ import { resetChatSession, steerActiveTurn } from '../_lib/api'; import { deriveTitle, newSessionId, + stripStaleInteractives, type ChatSession, type DiagramAttachment, type OutgoingFileAttachment, @@ -51,6 +52,7 @@ import { import { useChatSessionsCtx } from '../_lib/chatSessionsContext'; import { useStreamStore } from '../_lib/streamStore'; import { ChoiceCard } from '../_components/ChoiceCard'; +import { McpInputCard } from '../_components/chat/McpInputCard'; import { DevJobChatCard } from '../_components/devjobs/DevJobChatCard'; import { parseDevJobStartResult } from '../_components/devjobs/devJobChatCardState'; import { KgWalkPane } from '../_components/KgWalkPane'; @@ -292,18 +294,12 @@ export default function ChatPage(): React.ReactElement { mutateActive((session) => { if (session.id !== targetSessionId) return session; const isFirst = session.messages.length === 0; - // Strip pendingUserChoice AND followUpOptions from older assistant - // messages so the button rows disappear as soon as the user commits - // to a choice or types a fresh message. - const cleanedMessages = session.messages.map((m) => { - if (!m.pendingUserChoice && !m.followUpOptions) return m; - const { - pendingUserChoice: _dropChoice, - followUpOptions: _dropFollowUps, - ...rest - } = m; - return rest; - }); + // Strip pendingUserChoice, pendingMcpInput AND followUpOptions from + // older assistant messages so the button rows / input forms disappear as + // soon as the user commits to a choice or types a fresh message. Lives + // in `chatSessions.ts` so it is actually covered by tests — see + // `stripStaleInteractives`. + const cleanedMessages = stripStaleInteractives(session.messages); return { ...session, title: isFirst ? deriveTitle(trimmed) : session.title, @@ -971,6 +967,17 @@ function MessageRow({ onChoose={onChoose} /> )} + {/* #544 W2-1 — MCP mid-call input form. Mutually exclusive with the + choice card server-side (the choice card wins), so the two can + never render together. `onChoose` submits the returned envelope + as a fresh user turn, exactly like a choice-card click. */} + {message.pendingMcpInput && ( + + )} {message.attachments && message.attachments.length > 0 && ( )} diff --git a/web-ui/messages/de.json b/web-ui/messages/de.json index dc990a73..76874e86 100644 --- a/web-ui/messages/de.json +++ b/web-ui/messages/de.json @@ -913,6 +913,13 @@ "modalBody": "Autorisiere {server}, damit diese und künftige Anfragen es nutzen können.", "close": "Schließen" }, + "mcpInput": { + "kicker": "Externe Eingabe-Anfrage", + "heading": "„{server}“ braucht für „{tool}“ noch weitere Angaben.", + "secretHint": "Bei der Eingabe verborgen — an den Server geht der Wert unverändert.", + "warning": "Was du hier eingibst, geht an den externen Server „{server}“. Gib nur Daten ein, die dieser Server bekommen darf.", + "submit": "An Server senden" + }, "iterationLabel": "── iteration {n}", "attachmentTitle": "{kind}{cachedSuffix} — klick für Originalgröße", "attachmentCachedSuffix": " · cached", diff --git a/web-ui/messages/en.json b/web-ui/messages/en.json index 49531639..5ec2e7c2 100644 --- a/web-ui/messages/en.json +++ b/web-ui/messages/en.json @@ -913,6 +913,13 @@ "modalBody": "Authorize {server} so this and future requests can use it.", "close": "Close" }, + "mcpInput": { + "kicker": "External input request", + "heading": "“{server}” needs additional details for “{tool}”.", + "secretHint": "Hidden while typing — the value is still sent to the server as entered.", + "warning": "Whatever you enter is sent to the external server “{server}”. Only provide details you want that server to have.", + "submit": "Send to server" + }, "iterationLabel": "── iteration {n}", "attachmentTitle": "{kind}{cachedSuffix} — click for full size", "attachmentCachedSuffix": " · cached", From 1d6c2f7a6c3da99e70e74393d88b6012fa71ed67 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 12:29:19 +0200 Subject: [PATCH 34/90] feat(orchestrator): idempotency primitive for write-capable tool dispatch Adds the process-local dedupe store plus an AsyncLocalStorage channel that carries the active idempotency key down to the MCP transport layer without touching the published NativeToolHandler contract. Also adds isWriteCapableTool() to the existing (previously unwired) WriteCapability contract in plugin-api. --- .../src/toolIdempotency.ts | Bin 0 -> 11149 bytes .../plugin-api/src/writeCapabilities.ts | 23 ++++++++++++++++++ 2 files changed, 23 insertions(+) create mode 100644 middleware/packages/harness-orchestrator/src/toolIdempotency.ts diff --git a/middleware/packages/harness-orchestrator/src/toolIdempotency.ts b/middleware/packages/harness-orchestrator/src/toolIdempotency.ts new file mode 100644 index 0000000000000000000000000000000000000000..701885c3995fed0312afc282eb833ae1fa2d1eed GIT binary patch literal 11149 zcmbVS-Etep(aklVVj`7LnLkf7p9DkT$(*_oc6)2C1OV0UL{%j}rr*e#ZM<+5mHW_e+5illPG z$S&!P}HCeNHn^UEZ%sjiDZ@YhWexjjFjF)v4E zXJ=g4tX$@WpK)i${QK|!GLw^Nd15nr>536w9mDpCteLpdoShyXVF_bR#kUjKL%ZdF z@^F)nptYER%V`;m5ZC{;Dp;% ze`F3!l`LFdR|aF_B9|$mJj+}pQzRw5lay7GUBaDn@%XIB7v@A6Wrk@H2Z)7DN+Y&A;t7kAdH@2#9wv~BBUpLbgZWugn&7!V!JJsA)#-@d{G4^9& zYUW}dE*$iVK0ky-jBz4rla#2eN9U@tNz|AD zNG$UttNNI7SmIM@CPZ7g%XcTn&MGX2(^40X1YQji1w*EEu52c(0JhRCEl;e0tSbXR z-^qD#e+sT?Ud@e#U2|J3QdgEEqeCPBbWCq@7U%h70BHU8o3AH*W4T&fFR>A>f&+}| zvdR~(7{;~&X6yp+8hvx40X_8$TBEA#P7ZujJm)Nj;C&Dg=X%7Da+}MS~ zF+~8AoPCt1sa=-Aq3YmQdxExEQj}HWQ&S_Fc{G>Fwad(vTLo;E0|YmZEu2%BP2V7J zFAh$Q6wYm8qPazMf^_Kd$QA{egJxSq?6nia-=6(&3V0}unU{*cV0D$( z(Hs_=$tmLM+VMo*BTNxG6GE8N<{9`V^6b-_^pb2@S9|d4RhHjmee-7Dd;|$3rTez5 z3QV>S1UKm^sCxTjfGwt&xh zeF=af-pgVI}!yOK(-m05szv37*;o+r$m@Qy|fP6>GrW~T&EZ<;e6+G8^Tw{MT$9enyQ{(SuQ z=;Y(s_~`WT&!5NR51&si?&ZvVm~R?(a`30mN2lZSSvf(UEF6WisEfXAgbPhxy6g(IV9!1!*L?eBAfFPA1W~7KWta*F808nEjKsx>5Z2UgJcVtfN72PL!q`O(sa#%-`8}$&B zNC7V(u^}(@J_6R4_8x>u}qL z^H0WI13)x`J|sw6eb!A_25gpOot56H0E89TJE>64Hs(?nE$%iY+7a@SPqt0KxT1g$ zV>v04I(rQ_XSoqRP>CjAA$;w-j?(A;Zd<#Ryjs)?IL8}=Qp4=(H*3Q6dGfFn}v&V z>>MswadQk*asmpD-7P9&Scq(Zyt*mpX=*fY%*(IA^WVI{KQCXrcoEj7OHs!H$;1=F z9Bb=(055P2Y8`@xasD(45}lNRKP6}XG7CCcmu57J^K8a|wU><@=$MjPbpS(bEbKR3 z;go$*44O~jF$P4VRSKRL=bQM*JVRhkXfKDIoqqVUDU(a4gN^Y){9!0Cx{JDOO6AJOl)9 zSGroakghYT?n|vUn^Q++=!%(zn0m;y^q;xgB1zboS(q7V4-leMe%*aeqbTCKUjmgF zLU&e04L|<-AO92+$zKR$TV4?X%)Ql+f^H~+&Z`d~Nh;bO&X31Op+}F=DE>VfsLklrUB=_)(N%#?F?C<(v*y(H;p5gc1Z(aO-4Gekw|9QL@R)a zIAMaKL~4UKf|e=@YrwE|y@Eno$`L6oN(;wveSsBgYo zM;E<()-~u0%?Z^^jJ;oEV3>BzLMR0E4IVjKH$y(4K|b1V=Frd+eFN1#OIJyC#H)(< zprbYLzyq1&R8*s4VJEzTtX$uH>s&!zd?cmcn!&&H)Tt)eCeWxrZjG zlkA-oYH<2T-z0btqq_0^wYPM~cxU6-CO9rCNlE{Aadx_A@+oUFk}I+T5;5WovO>cH zEnS2Kg!|;q4yO0$)-BydnCqW}7OrK7qeSvg>e^Rdf3w{m)k}11-NU2R>H7UqoLoX1_O|ElcDv=< zL+O>m@R51jt!2?ZOCc^jL26mRq#v3I63$>3s!%NkHg2$!M`fk&-KvMh!k*feRCt1% zp+N5@4jgRdRnPsep5I}D_HcqIm7)|l@sbBgl6`6rBZ@#V5%ng+UQ;9zN;s#`(ixef zXpShj6y*o%={tuE+l3Wx<9$OW&w)GpJxTQJ@p_)@o!vp}@l>2Pk$f zUK8=yaT-fDz-iP33B3mH?(~wHa*^HmNBUab8$?tn2iT3Eqy3e@9AX&yl`o6?QRtwm zRS^;tS<_X8*w5JXt&*rtZK1~qA&LL9Th>Ky=N#4!DoSlxK1?T)@&u0329~CS*XS)j zQfQ(x)(~MQPl~cMs8!KJ?!H`A=}Fm5hmS@q?Cp^<@E3=n!5&ms$gP3Fi#mc-?B&w& zE|DtlgqWMeD>mS9c#o{zeAzDlU<9?tamGX{nKi=g%auT>)3JirD*yx}8Ez3UZ;HD- zbZ*dXRaM_v44c3zsvTQv?)%ne_;evno;N@bUT^o)oul*giHSP( zaA!6c+?|FW`96+#gugiu*+X<1E!Z(f{z4cPuZy59x~+9I|ICEM*b*Xo^lihPIBQz% zS`_J|L-T>n_qGioBT=%FwnDF()u8WbvuEQC@L?f*$vaWn=%@3OJnnjPrjXH$Fzzpz zuC1A#vd$6~65>r8SeXb-vc@!YzT~DsogtfI+2h>_sirxJM4`&C>nL{heK#ItGVvmR z(i9oq8_2KPQjvcA8Vm9M!55>_Ju<9d_=x-%SerEv(B_lX`qT9rF&#MfqUuew|My4_Z@LMin}<7aQ)KIl9a(~RJDzI zq5Z=r(r+K-ycU4CBt|dA-0MMwIkoAo-?iV|tM?f4IPSZ;?hSG+BG{@2lX?%_8no$~MxS*Bhb^D_N>GODP zgzvz9I+9&*gbTArcRsLzmZ+CdUg+%^+#!2$5-33&9nQURoia(|z~GDj=0c(BfaLHJ zkj(gGpM{G_4k7uVA{wEi*noS0esc$xU|+UpV%ECy4$?u7xbj7=x%U&oZwKuSrV-N_ z7n3lORVOc;p<%oW8mIAg3YLR#M)4%(B*ZOPo>4q{xGSJu9m*Q6|T{^ zj!G(#Y{G4gh4`Tnei+r$WZXumk74w?8+QvKwD*eV5_8^N$XFpBeFVJGJvQs;z~3_3~Aj5N^H^LQpJ-bTASA}^3(zkK-lYd#bUdLB+9LJ%MV5xB-JF!i{yf#^VnLg`PU1 zP=qmQU5kPM1$ngv>7KdH6YYu{FO7BU;iSvtZw_rDaYrla+f9?V!fhB-4I@pRh_*FQ z+*9qN{Uj|eQ8W0-+gVhfTz}yNub((Q%8(?hn?*O%(N)7kreiHxCKzkX6s6Qv_}G;k zzxMkJT|XVe5zPUe2Zie|DI@#{p_1Z_<=U?13_DHIqyoPtu|E_#m$w#Lnt5XPC|GAG L4CJ!+XrBEC0Wy}> literal 0 HcmV?d00001 diff --git a/middleware/packages/plugin-api/src/writeCapabilities.ts b/middleware/packages/plugin-api/src/writeCapabilities.ts index 4da6ba0b..d0f9b597 100644 --- a/middleware/packages/plugin-api/src/writeCapabilities.ts +++ b/middleware/packages/plugin-api/src/writeCapabilities.ts @@ -49,6 +49,29 @@ export interface WriteCapability { }; } +/** + * True when a tool declares at least one write capability — i.e. dispatching it + * may MUTATE data in a downstream system. + * + * This is the single predicate the dispatch layer uses to decide whether a call + * needs at-most-once protection (idempotency dedupe + transport-retry + * suppression). It is deliberately declaration-driven rather than name-derived: + * guessing "write" from a free-form tool name is exactly the silent-rollback + * failure mode the `WriteCapability` contract exists to avoid. + * + * A tool that declares NOTHING is treated as read-only. That default is safe for + * the canvas/inline-edit consumer (no affordances offered) but note the opposite + * asymmetry here: an UNANNOTATED write tool gets no idempotency protection. The + * annotation is the only signal available without an LLM call, so a plugin that + * mutates data and ships no `writeCapabilities` is a plugin bug — see the + * `writeCapabilities` field docs on `NativeToolRegistration`. + */ +export function isWriteCapableTool( + capabilities: readonly WriteCapability[] | undefined, +): boolean { + return capabilities !== undefined && capabilities.length > 0; +} + /** Deterministic Tier-2 derivation of mutability from a tool's write capabilities. */ export interface DerivedMutability { canAddItems: boolean; From 282b44655148b3710e99726b41ba10150a833995 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 12:38:08 +0200 Subject: [PATCH 35/90] feat(orchestrator): close the privacy/trace seam in ToolDispatchService and add idempotency MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 1 — ToolDispatchService now applies the privacy data-plane boundary (intern-exemption, operator bypass + receipt, internToolResultV4 masking) and raw-result capture in the same order as Orchestrator.dispatchToolDeadlined, so a caller reaching tools via the loopback/public path no longer bypasses the PII masking the chat path enforces. Adds an optional caller-context carrier (principal/scopes/tenantId/userId/requestId) propagated ambiently. SEAM comment rewritten to state what is closed and what stays orchestrator-only. Task 2 — write-capability is declared via the existing WriteCapability contract, now wired onto its intended non-model-facing carriers (NativeToolRegistration, DomainTool). A write-capable dispatch with an idempotency key dedupes on the key and clamps McpManager.callTool to a single attempt; reads keep the flaky-proxy retry unchanged. --- .../harness-orchestrator/src/index.ts | 21 ++ .../harness-orchestrator/src/mcp/mcpClient.ts | 30 +- .../src/nativeToolRegistry.ts | 30 ++ .../src/toolCallerContext.ts | 38 +++ .../src/toolDispatchService.ts | 276 +++++++++++++++++- .../src/tools/domainQueryTool.ts | 13 +- 6 files changed, 391 insertions(+), 17 deletions(-) create mode 100644 middleware/packages/harness-orchestrator/src/toolCallerContext.ts diff --git a/middleware/packages/harness-orchestrator/src/index.ts b/middleware/packages/harness-orchestrator/src/index.ts index a31faf0b..946d7e03 100644 --- a/middleware/packages/harness-orchestrator/src/index.ts +++ b/middleware/packages/harness-orchestrator/src/index.ts @@ -310,7 +310,28 @@ export { ToolDispatchService } from './toolDispatchService.js'; export type { DispatchableToolSpec, ToolDispatchResult, + ToolDispatchCallerContext, + ToolDispatchOptions, } from './toolDispatchService.js'; +export { + ToolIdempotencyStore, + currentIdempotencyScope, + runWithIdempotencyScope, + fingerprintToolInput, + idempotencyCacheKey, + idempotencyConflictMessage, + DEFAULT_IDEMPOTENCY_TTL_MS, + DEFAULT_IDEMPOTENCY_MAX_ENTRIES, +} from './toolIdempotency.js'; +export type { + ToolIdempotencyScope, + ToolIdempotencyResult, + ToolIdempotencyOutcome, +} from './toolIdempotency.js'; +export { + currentDispatchCaller, + runWithDispatchCaller, +} from './toolCallerContext.js'; export { LoopbackMcpServer } from './loopbackMcpServer.js'; export type { LoopbackMcpServerDeps, diff --git a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts index bfa04934..722d7a26 100644 --- a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts +++ b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts @@ -36,6 +36,7 @@ import type { } from '@omadia/plugin-api'; import { turnContext } from '../turnContext.js'; +import { currentIdempotencyScope } from '../toolIdempotency.js'; import { MCP_INPUT_MAX_REPLAY_DEPTH, extractMcpInputPrompt, @@ -677,8 +678,21 @@ export class McpManager { // that intermittently returns "-32001 Request timed out" or drops the // connection). The retry drops the pooled connection first so it reconnects // fresh; auth-looking and real tool errors are NOT retried. + // + // #542 prerequisite — EXCEPT under an exactly-once idempotency scope. A + // transient failure is indistinguishable from "the server executed the write + // and the response was lost", so retrying a WRITE-capable call can duplicate + // a mutation (a second Odoo/M365 write = customer-data damage). When + // `ToolDispatchService` dispatched this call as write-capable with an + // idempotency key it publishes `exactlyOnce`, and this loop then makes ONE + // attempt: at-most-once beats at-least-once for writes. + // + // The mitigation itself is untouched for everything else — read tools, and + // write tools dispatched without an idempotency key, still get the retry. + const idempotency = currentIdempotencyScope(); + const maxAttempts = idempotency?.exactlyOnce === true ? 1 : 2; let lastFailure = `Error: MCP tool "${toolName}" on "${cfg.name}" failed.`; - for (let attempt = 1; attempt <= 2; attempt += 1) { + for (let attempt = 1; attempt <= maxAttempts; attempt += 1) { let pooled: Pooled; try { pooled = await this.getOrConnect(cfg, token); @@ -687,7 +701,7 @@ export class McpManager { // (streamable-HTTP surfaces the 401 as "-32000 Connection closed"), so // this path must also offer the auth prompt — not just tool-level errors. const failure = `Error: could not connect to MCP server "${cfg.name}": ${msg(err)}`; - if (attempt < 2 && looksTransient(failure) && token !== null) { + if (attempt < maxAttempts && looksTransient(failure) && token !== null) { await this.close(this.poolKey(cfg, token)); lastFailure = failure; continue; @@ -699,6 +713,16 @@ export class McpManager { { name: toolName, arguments: args, + // #542 prerequisite — advertise the idempotency key so a server that + // implements dedupe can recognise a duplicate as the SAME call. + // Advisory only: MCP defines no standard idempotency field and no + // server is obliged to honour this, so it is never the protection — + // the `maxAttempts` clamp above and the dispatcher's dedupe store + // are. Rides `_meta`, the spec's extension channel, so a server that + // ignores it sees byte-identical arguments. + ...(idempotency !== undefined + ? { _meta: { idempotencyKey: idempotency.key } } + : {}), }, // Tolerate off-spec `structuredContent` (some third-party MCP servers — // e.g. the hosted Strava proxy — return it as a JSON array instead of an @@ -754,7 +778,7 @@ export class McpManager { // Drop the connection so the next call reconnects (server may have died). await this.close(this.poolKey(cfg, token)); const failure = `Error: MCP tool "${toolName}" on "${cfg.name}" failed: ${msg(err)}`; - if (attempt < 2 && looksTransient(failure)) { + if (attempt < maxAttempts && looksTransient(failure)) { lastFailure = failure; continue; } diff --git a/middleware/packages/harness-orchestrator/src/nativeToolRegistry.ts b/middleware/packages/harness-orchestrator/src/nativeToolRegistry.ts index fe9f038c..86d638ce 100644 --- a/middleware/packages/harness-orchestrator/src/nativeToolRegistry.ts +++ b/middleware/packages/harness-orchestrator/src/nativeToolRegistry.ts @@ -27,6 +27,7 @@ import type { NativeToolAttachmentSink, NativeToolHandler, NativeToolSpec, + WriteCapability, } from '@omadia/plugin-api'; export interface NativeToolRegistration { @@ -68,6 +69,23 @@ export interface NativeToolRegistration { * without restart. Absent for marker-only kernel registrations. */ readonly readConfig?: (key: string) => unknown | undefined; + /** + * #542 prerequisite — the tool's declared write capabilities, i.e. the + * assertion "dispatching me may MUTATE data". + * + * This is the carrier the `WriteCapability` contract was always meant to land + * on. `plugin-api`'s `pluginContext.ts` notes that the annotation deliberately + * does NOT go on `NativeToolSpec`, because the whole spec is forwarded verbatim + * into the Anthropic tools list and unknown fields are rejected there — it + * belongs on "a non-model-facing carrier (manifest annotation / registration + * metadata)". This registration IS that carrier. + * + * Read by `ToolDispatchService` to decide whether a dispatch needs at-most-once + * protection. Absent or empty ⇒ treated as read-only: no idempotency dedupe, + * and the MCP transient-retry mitigation stays fully in force. A plugin that + * mutates data MUST declare this or it forfeits duplicate-write protection. + */ + readonly writeCapabilities?: readonly WriteCapability[]; } export interface NativeToolRegistrationOptions { @@ -84,6 +102,8 @@ export interface NativeToolRegistrationOptions { /** Slice 2.5 — see `NativeToolRegistration.readConfig`. Set by * `ToolsAccessor.register` as `(k) => config.get(k)`. */ readConfig?: (key: string) => unknown | undefined; + /** #542 — see `NativeToolRegistration.writeCapabilities`. */ + writeCapabilities?: readonly WriteCapability[]; } /** @@ -104,6 +124,10 @@ export interface NativeToolHandlerRegistrationOptions { * such an entry stays always-available, matching `register()`'s * kernel-internal (marker-only) convention. */ agentId?: string; + /** #542 — see `NativeToolRegistration.writeCapabilities`. Honoured on this + * path too: a handler-only registration is dispatchable by name, so it needs + * the same duplicate-write protection as a `register()`-contributed tool. */ + writeCapabilities?: readonly WriteCapability[]; } export class NativeToolRegistry { @@ -143,6 +167,9 @@ export class NativeToolRegistry { ...(options.readConfig !== undefined ? { readConfig: options.readConfig } : {}), + ...(options.writeCapabilities !== undefined + ? { writeCapabilities: options.writeCapabilities } + : {}), } : { name }; this.entries.set(name, entry); @@ -182,6 +209,9 @@ export class NativeToolRegistry { ? { attachmentSink: options.attachmentSink } : {}), ...(options.agentId !== undefined ? { agentId: options.agentId } : {}), + ...(options.writeCapabilities !== undefined + ? { writeCapabilities: options.writeCapabilities } + : {}), }; this.entries.set(name, entry); return () => { diff --git a/middleware/packages/harness-orchestrator/src/toolCallerContext.ts b/middleware/packages/harness-orchestrator/src/toolCallerContext.ts new file mode 100644 index 00000000..e761ed1e --- /dev/null +++ b/middleware/packages/harness-orchestrator/src/toolCallerContext.ts @@ -0,0 +1,38 @@ +/** + * Ambient caller identity for the standalone tool-dispatch path (#542 prerequisite). + * + * `ToolDispatchService` historically carried no caller identity at all. That is + * fine for the loopback bridge — the caller is the local CLI acting as the + * session's own user — but a public endpoint receives every call with an API key + * or token behind it, and the layers beneath dispatch (MCP audit rows, plugin + * handlers, downstream integrations) need to be able to attribute the call. + * + * Propagated ambiently via `AsyncLocalStorage` rather than as a handler parameter + * for the same reason the idempotency key is: `NativeToolHandler` is + * `(input: unknown) => Promise`, a published contract implemented by + * out-of-tree plugins. Widening it is not an option, and it is the same mechanism + * the privacy handle already uses via `turnContext`. + * + * This is a CARRIER, not an authorization boundary. Reading a principal here says + * who claims to be calling — it does not mean anything checked their scopes. + * Enforcement belongs with the endpoint that authenticated the credential. + */ + +import { AsyncLocalStorage } from 'node:async_hooks'; + +import type { ToolDispatchCallerContext } from './toolDispatchService.js'; + +const callerStorage = new AsyncLocalStorage(); + +/** Caller identity of the in-flight dispatch, if the entry point supplied one. */ +export function currentDispatchCaller(): ToolDispatchCallerContext | undefined { + return callerStorage.getStore(); +} + +/** Run `fn` with `caller` visible to every layer beneath it. */ +export function runWithDispatchCaller( + caller: ToolDispatchCallerContext, + fn: () => T, +): T { + return callerStorage.run(caller, fn); +} diff --git a/middleware/packages/harness-orchestrator/src/toolDispatchService.ts b/middleware/packages/harness-orchestrator/src/toolDispatchService.ts index c713833b..bfa5e8b9 100644 --- a/middleware/packages/harness-orchestrator/src/toolDispatchService.ts +++ b/middleware/packages/harness-orchestrator/src/toolDispatchService.ts @@ -1,15 +1,27 @@ /** - * M1 library code for #309 Shape-3 OpenClaw. + * Standalone tool dispatcher — the entry point that is NOT the Orchestrator turn + * loop. Serves the loopback MCP server (subscription-CLI provider), the CLI + * bridge, and CLI sub-agents; it is also the path any future public MCP endpoint + * (#542) would dispatch through. * - * This standalone dispatcher executes tools outside the Orchestrator turn loop. - * It intentionally replicates only the native-handler and DomainTool branches - * of `Orchestrator.dispatchToolInner`; kernel-tool branches plus privacy/trace - * seams are deferred to M2. + * It replicates the native-handler and DomainTool branches of + * `Orchestrator.dispatchToolInner`, and — since #542's prerequisite work — the + * privacy data-plane boundary and raw-result capture that `dispatchToolDeadlined` + * applies around them. See the SEAM note at the bottom of this file for what is + * closed and what is still deliberately orchestrator-only. */ +import { isInternExemptTool } from './privacyInternPolicy.js'; +import { isWriteCapableTool } from '@omadia/plugin-api'; +import type { WriteCapability } from '@omadia/plugin-api'; +import type { PrivacyTurnHandle } from './privacyHandle.js'; import type { DomainTool } from './tools/domainQueryTool.js'; import type { NativeToolRegistry } from './nativeToolRegistry.js'; import { sortByToolName } from './toolOrdering.js'; +import { turnContext } from './turnContext.js'; +import { runWithDispatchCaller } from './toolCallerContext.js'; +import { runWithIdempotencyScope } from './toolIdempotency.js'; +import type { ToolIdempotencyStore } from './toolIdempotency.js'; export interface ToolDispatchResult { readonly content: string; @@ -26,6 +38,54 @@ export interface DispatchableToolSpec { }; } +/** + * Identity of whoever asked for this dispatch. + * + * The dispatch path historically carried NO caller identity at all — no tenant, + * no user, no principal — which is fine for the loopback bridge (the caller is + * the local CLI, acting as the session's own user) but is the missing seam for a + * public endpoint, where every call arrives with an API key or token that has to + * be attributable and scope-checked. + * + * Optional by construction: the loopback and CLI-sub-agent paths pass nothing and + * behave exactly as before. When #438/#439's `harness-api-key-auth` lands, the + * public endpoint fills this in from the verified credential; nothing downstream + * has to change shape again. + * + * NOTE: this is a CARRIER, not an enforcement point. `ToolDispatchService` does + * not currently authorize against `scopes` — a per-principal tool allowlist is + * the public endpoint's own job (#542) and belongs where the allowlist policy + * lives, not here. Do not read the presence of this field as "the dispatch path + * is now access-controlled". + */ +export interface ToolDispatchCallerContext { + /** Stable id of the acting principal (API-key id, service account, user id). */ + readonly principal?: string; + /** Scopes/permissions the credential carries, for the caller's own policy check. */ + readonly scopes?: readonly string[]; + /** Tenant the call is acting within. */ + readonly tenantId?: string; + /** End user on whose behalf the call runs, when distinct from `principal`. */ + readonly userId?: string; + /** Correlation id for logs/traces. */ + readonly requestId?: string; +} + +/** Per-dispatch options. All optional — omitting the whole argument is legacy behaviour. */ +export interface ToolDispatchOptions { + readonly caller?: ToolDispatchCallerContext; + /** + * Caller-supplied idempotency key. Applied ONLY to write-capable tools (see + * `isWriteCapableTool`): two dispatches sharing a key execute the tool at most + * once while the record is live, and the MCP transport layer suppresses its + * transient retry for the call. + * + * Read tools ignore this: deduping reads would serve stale data, and the + * flaky-proxy retry mitigation must stay in force for them. + */ + readonly idempotencyKey?: string; +} + export class ToolDispatchService { constructor( private readonly deps: { @@ -46,6 +106,42 @@ export class ToolDispatchService { * ungated. Absent ⇒ every plugin's tools are always available. */ readonly isPluginToolsReady?: (agentId: string) => boolean; + /** + * #542 prerequisite — the privacy data-plane boundary for this path. + * + * The chat path reads its handle from `turnContext`, which this dispatcher + * runs entirely outside of: the loopback MCP server and any public endpoint + * are not inside `turnContext.run(...)`, so `turnContext.current()` is + * `undefined` and a tool result would reach the caller with PII intact. + * That was the open half of the privacy seam. + * + * Resolution order is explicit-dep first, ambient turn context second, so + * a host that DOES dispatch from inside a turn still inherits that turn's + * handle. Absent from both ⇒ no privacy provider installed and results flow + * through unchanged, matching the orchestrator. + */ + readonly privacy?: () => PrivacyTurnHandle | undefined; + /** + * #542 prerequisite — raw-result capture (the orchestrator's Phase C.2 + * `captureRawToolResult`). Receives the tool result BEFORE masking, so a + * trace/audit consumer sees ground truth while the caller gets the digest. + * Must not throw; a throw is caught and logged rather than failing the call. + * + * Receives the dispatch's caller context so an audit consumer can attribute + * the result to the principal that caused it. + */ + readonly captureRawToolResult?: ( + name: string, + result: string, + caller?: ToolDispatchCallerContext, + ) => void; + /** + * #542 prerequisite — dedupe store for write-capable dispatches. Absent ⇒ + * `idempotencyKey` is inert and every dispatch executes (legacy behaviour). + * Process-local: see `toolIdempotency.ts` for the exact limits of the + * guarantee — it is NOT distributed idempotency. + */ + readonly idempotency?: ToolIdempotencyStore; }, ) {} @@ -60,7 +156,66 @@ export class ToolDispatchService { return this.deps.isPluginToolsReady(agentId); } - async dispatch(name: string, input: unknown): Promise { + /** Explicit dep wins; ambient turn handle is the fallback. */ + private privacyHandle(): PrivacyTurnHandle | undefined { + return this.deps.privacy?.() ?? turnContext.current()?.privacyHandle; + } + + /** Declared write capabilities for `name`, from whichever carrier owns it. */ + private writeCapabilities(name: string): readonly WriteCapability[] | undefined { + const native = this.deps.nativeTools.get(name); + if (native?.writeCapabilities !== undefined) return native.writeCapabilities; + return this.domainTools().find((t) => t.name === name)?.writeCapabilities; + } + + /** True when dispatching `name` may mutate data. */ + isWriteCapable(name: string): boolean { + return isWriteCapableTool(this.writeCapabilities(name)); + } + + async dispatch( + name: string, + input: unknown, + options?: ToolDispatchOptions, + ): Promise { + const caller = options?.caller; + // Publish caller identity for every layer beneath this dispatch. Omitted + // entirely when the entry point supplied none, so the loopback path runs with + // an empty store exactly as before. + return caller === undefined + ? this.dispatchIdempotent(name, input, options) + : runWithDispatchCaller(caller, () => + this.dispatchIdempotent(name, input, options), + ); + } + + private async dispatchIdempotent( + name: string, + input: unknown, + options?: ToolDispatchOptions, + ): Promise { + const key = options?.idempotencyKey; + const store = this.deps.idempotency; + // Idempotency applies to write-capable tools only. A read tool keeps the + // transport-retry mitigation and never replays a cached body. + if (key !== undefined && store !== undefined && this.isWriteCapable(name)) { + const outcome = await store.run(key, name, input, () => + // The scope must wrap the EXECUTION, not the cache lookup, so the MCP + // transport layer beneath the handler can read it and suppress its retry. + runWithIdempotencyScope({ key, toolName: name, exactlyOnce: true }, () => + this.dispatchInner(name, input, options), + ), + ); + return outcome.result; + } + return this.dispatchInner(name, input, options); + } + + private async dispatchInner( + name: string, + input: unknown, + options?: ToolDispatchOptions, + ): Promise { const nativeRegistration = this.deps.nativeTools.get(name); // Mirrors Orchestrator ordering: plugin/native handlers win first. if (nativeRegistration?.handler) { @@ -71,7 +226,8 @@ export class ToolDispatchService { }; } try { - return { content: await nativeRegistration.handler(input) }; + const raw = await nativeRegistration.handler(input); + return { content: await this.afterDispatch(name, raw, options) }; } catch (error) { return { content: this.errMsg(error), isError: true }; } @@ -89,7 +245,8 @@ export class ToolDispatchService { }; } try { - return { content: await domainTool.handle(input) }; + const raw = await domainTool.handle(input); + return { content: await this.afterDispatch(name, raw, options) }; } catch (error) { return { content: this.errMsg(error), isError: true }; } @@ -98,6 +255,82 @@ export class ToolDispatchService { return { content: `Error: unknown tool \`${name}\`.`, isError: true }; } + /** + * Post-dispatch pipeline: raw capture, then the privacy data-plane boundary. + * + * Ordering mirrors `Orchestrator.dispatchToolDeadlined` deliberately, because a + * divergence here is a privacy divergence: + * 1. raw capture — trace/audit consumers must see ground truth + * 2. intern-exemption — the agent's own infra tools are never masked + * 3. operator bypass (+ receipt entry) — explicit opt-out stays auditable + * 4. intern — the caller receives the identity-free digest + */ + private async afterDispatch( + name: string, + result: string, + options?: ToolDispatchOptions, + ): Promise { + const capture = this.deps.captureRawToolResult; + if (capture !== undefined && typeof result === 'string') { + try { + capture(name, result, options?.caller); + } catch (err) { + console.warn( + `[toolDispatchService:${name}] captureRawToolResult threw — continuing without capture:`, + err, + ); + } + } + + const privacy = this.privacyHandle(); + if (privacy === undefined || typeof result !== 'string') return result; + + // Interning-exemption: the agent's own infrastructure/self tools (memory, + // stored-process CRUD, self-produced meta output) are never interned — + // masking them blinds the agent to its own operational state. Same + // auditable allowlist the orchestrator uses. + if (isInternExemptTool(name)) return result; + + // Operator-owned per-plugin bypass (Slice 2.5). Raw passthrough, but the + // receipt entry keeps it transparent. + const bypass = privacy.checkBypass(name); + if (bypass !== undefined) { + try { + await privacy.recordBypassedTool({ + toolName: name, + pluginId: bypass.pluginId, + reason: 'operator_setting', + bytes: Buffer.byteLength(result, 'utf8'), + }); + } catch (err) { + console.warn( + `[toolDispatchService:${name}] privacy.recordBypassedTool threw — bypass still applied:`, + err, + ); + } + return result; + } + + try { + const v4 = await privacy.internToolResultV4({ + toolName: name, + rawResult: result, + }); + return v4.digestText; + } catch (err) { + // Fail-OPEN, matching `Orchestrator.dispatchToolDeadlined` exactly. This is + // parity, not an endorsement: for a PUBLIC endpoint a masking failure that + // emits raw rows is a leak, and a fail-CLOSED policy for untrusted callers + // is worth its own decision (#542) — but making this path stricter than the + // chat path would be a silent behaviour change beyond closing the seam. + console.warn( + `[toolDispatchService:${name}] privacy.internToolResultV4 threw — sending raw result:`, + err, + ); + return result; + } + } + listDispatchableToolSpecs(): readonly DispatchableToolSpec[] { const advertised = new Map(); @@ -152,8 +385,25 @@ export class ToolDispatchService { } } -// SEAM (M2): kernel-tool branches (knowledge_graph, chat_participants, -// ask_user_choice, suggest_follow_ups, find_free_slots, book_meeting, -// read_attachment) and scoped-memory shadowing, plus privacy interning / -// trace capture, are intentionally NOT replicated here — see -// Orchestrator.dispatchToolInner. +// SEAM — divergence from `Orchestrator.dispatchToolInner` / +// `dispatchToolDeadlined`, kept current deliberately. +// +// CLOSED (#542 prerequisite): the privacy data-plane boundary — intern-exemption, +// operator bypass with its receipt entry, and `internToolResultV4` masking — plus +// raw-result capture, now run on this path in the same order as the chat path. A +// caller reaching tools here no longer bypasses the PII masking chat enforces. +// Caller identity is carried by `ToolDispatchCallerContext` (a carrier, not an +// enforcement point — see its docs). +// +// STILL ORCHESTRATOR-ONLY, because each needs turn-scoped state this path has no +// access to (an unconditional copy would throw or silently no-op): +// - kernel-tool branches: scoped-memory shadowing, knowledge_graph, +// query_dataset, chat_participants, ask_user_choice, suggest_follow_ups, +// read_attachment, find_free_slots, book_meeting +// - `v4_*` verb/render tool routing via `privacy.runV4Tool` (needs the turn's +// data-plane engine; here such a name resolves to "unknown tool") +// - sub-agent dataset bridging (`subAgentDatasetSink` / `subAgentResultV4`) +// and the Slice-2.5 sub-agent bypass flag +// - MCP → Knowledge-Graph ingestion (needs `knowledgeGraph` + turn user id) +// - canvas sentinel tap (`canvasSentinelSink`) +// - the W0-2 per-tool dispatch deadline and its late-result firewall diff --git a/middleware/packages/harness-orchestrator/src/tools/domainQueryTool.ts b/middleware/packages/harness-orchestrator/src/tools/domainQueryTool.ts index 35f3498a..6ade52ce 100644 --- a/middleware/packages/harness-orchestrator/src/tools/domainQueryTool.ts +++ b/middleware/packages/harness-orchestrator/src/tools/domainQueryTool.ts @@ -77,7 +77,7 @@ export interface Askable { * orchestrator threads it onto every emitted `ReadonlyToolTraceEntry` so * the Nudge-Pipeline's multi-domain trigger can count distinct domains. */ -import type { ToolPIIField } from '@omadia/plugin-api'; +import type { ToolPIIField, WriteCapability } from '@omadia/plugin-api'; export interface DomainTool { name: string; @@ -119,6 +119,17 @@ export interface DomainTool { * See `@omadia/plugin-api`'s `piiAnnotation.ts` for the full schema. */ piiFields?: readonly ToolPIIField[]; + /** + * #542 prerequisite — declared write capabilities, i.e. "dispatching me may + * MUTATE data". Same contract and same rationale as + * `NativeToolRegistration.writeCapabilities`: it lives on the wrapper rather + * than on `spec` because Anthropic rejects unknown fields on a tool spec, so + * mutability is a harness-side concern exactly like `piiFields` above. + * + * Read by `ToolDispatchService` to decide whether a dispatch needs + * at-most-once protection. Absent or empty ⇒ treated as read-only. + */ + writeCapabilities?: readonly WriteCapability[]; } export interface DomainToolSpec { From 792260367853bf8f25eb1f8f60785770d12eed33 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 12:41:24 +0200 Subject: [PATCH 36/90] test(orchestrator): mutation-checked privacy masking on the dispatch path 11 tests asserting masked CONTENT, never call counts. Verified empirically by breaking three invariants and confirming failures: masking removed (5 fail), intern-exemption dropped (1 fail), raw capture fed the masked value (1 fail). --- .../cliBridge/toolDispatchPrivacySeam.test.ts | 363 ++++++++++++++++++ 1 file changed, 363 insertions(+) create mode 100644 middleware/test/cliBridge/toolDispatchPrivacySeam.test.ts diff --git a/middleware/test/cliBridge/toolDispatchPrivacySeam.test.ts b/middleware/test/cliBridge/toolDispatchPrivacySeam.test.ts new file mode 100644 index 00000000..9aa8be5b --- /dev/null +++ b/middleware/test/cliBridge/toolDispatchPrivacySeam.test.ts @@ -0,0 +1,363 @@ +import { strict as assert } from 'node:assert'; +import { describe, it } from 'node:test'; + +import { NativeToolRegistry } from '../../packages/harness-orchestrator/src/nativeToolRegistry.js'; +import type { PrivacyTurnHandle } from '../../packages/harness-orchestrator/src/privacyHandle.js'; +import { + ToolDispatchService, + type ToolDispatchCallerContext, +} from '../../packages/harness-orchestrator/src/toolDispatchService.js'; +import { currentDispatchCaller } from '../../packages/harness-orchestrator/src/toolCallerContext.js'; +import { turnContext } from '../../packages/harness-orchestrator/src/turnContext.js'; +import type { DomainTool } from '../../packages/harness-orchestrator/src/tools/domainQueryTool.js'; + +/** + * #542 prerequisite — the privacy/trace seam in `ToolDispatchService`. + * + * `ToolDispatchService` is what the loopback MCP server dispatches through, and + * what a public MCP endpoint would dispatch through. Before this work it applied + * NO privacy masking: the chat path masks tool results via + * `Orchestrator.dispatchToolDeadlined`, but that code reads its handle from + * `turnContext`, which this dispatcher runs entirely outside of. A caller reaching + * tools here got PII in clear. + * + * MUTATION-CHECK DISCIPLINE: every assertion below inspects the CONTENT that + * leaves the dispatcher. None of them assert "a masking function was called" — + * a call-count assertion stays green over a masking function that returns its + * input unchanged, which is exactly the class of false-green this repo has been + * burned by. The fake handle performs a REAL redaction and the tests assert the + * raw PII is absent from the output. + */ + +const EMAIL = 'erika.mustermann@example.com'; +const IBAN = 'DE89370400440532013000'; +const PII_RESULT = `{"name":"Erika Mustermann","email":"${EMAIL}","iban":"${IBAN}"}`; + +interface RecordedBypass { + readonly toolName: string; + readonly pluginId: string; + readonly bytes: number; +} + +/** + * A privacy handle that genuinely redacts. `internToolResultV4` strips the email + * and IBAN and returns a digest — so if the dispatcher fails to call it, the raw + * values survive into the output and the assertions below fail. + */ +function redactingPrivacyHandle(options?: { + readonly bypassTools?: ReadonlySet; + readonly bypassReceipts?: RecordedBypass[]; + readonly internThrows?: boolean; +}): PrivacyTurnHandle { + return { + async internToolResultV4({ toolName, rawResult }) { + if (options?.internThrows === true) { + throw new Error('privacy provider unavailable'); + } + const redacted = rawResult + .replaceAll(EMAIL, '[masked:email]') + .replaceAll(IBAN, '[masked:iban]') + .replaceAll('Erika Mustermann', '[masked:person]'); + return { + digestText: `«dataset:${toolName}» ${redacted}`, + datasetId: `ds-${toolName}`, + }; + }, + async recordBypassedTool({ toolName, pluginId, bytes }) { + options?.bypassReceipts?.push({ toolName, pluginId, bytes }); + }, + checkBypass(toolName) { + return options?.bypassTools?.has(toolName) === true + ? { pluginId: `plugin-for-${toolName}` } + : undefined; + }, + async runV4Tool() { + throw new Error('not used on this path'); + }, + async subAgentResultV4() { + throw new Error('not used on this path'); + }, + async takeRenderedAnswerV4() { + return undefined; + }, + v4ToolSpecs() { + return []; + }, + async maskUserPrompt() { + return { outcome: 'disabled' }; + }, + async restorePromptPseudonyms(text) { + return text; + }, + snapshotPromptRestorer() { + return undefined; + }, + async finalize() { + return undefined; + }, + }; +} + +function registryWith( + name: string, + result: string, + extra?: { readonly agentId?: string }, +): NativeToolRegistry { + const nativeTools = new NativeToolRegistry(); + nativeTools.register(name, { + handler: async () => result, + spec: { + name, + description: 'returns a PII-bearing payload', + input_schema: { type: 'object', properties: {} }, + }, + domain: 'test.pii', + ...(extra?.agentId !== undefined ? { agentId: extra.agentId } : {}), + }); + return nativeTools; +} + +describe('ToolDispatchService — privacy data-plane boundary (#542 prerequisite)', () => { + it('MASKS a PII-bearing native tool result — the raw values never leave the dispatcher', async () => { + const service = new ToolDispatchService({ + nativeTools: registryWith('odoo_read_partner', PII_RESULT), + domainTools: [], + privacy: () => redactingPrivacyHandle(), + }); + + const result = await service.dispatch('odoo_read_partner', {}); + + // The load-bearing assertions: the actual PII is GONE from the output. + assert.equal( + result.content.includes(EMAIL), + false, + 'the email address reached the caller in clear — masking did not happen', + ); + assert.equal( + result.content.includes(IBAN), + false, + 'the IBAN reached the caller in clear — masking did not happen', + ); + assert.equal( + result.content.includes('Erika Mustermann'), + false, + 'the person name reached the caller in clear — masking did not happen', + ); + // And the masked substitutes ARE present, so this is masking rather than + // the result having been dropped or emptied. + assert.match(result.content, /\[masked:email\]/); + assert.match(result.content, /\[masked:iban\]/); + assert.match(result.content, /«dataset:odoo_read_partner»/); + assert.equal(result.isError, undefined); + }); + + it('MASKS a PII-bearing DOMAIN tool result too (both dispatch branches, not just native)', async () => { + const domainTool: DomainTool = { + name: 'ask_hr', + spec: { + name: 'ask_hr', + description: 'sub-agent', + input_schema: { type: 'object', properties: {}, required: [] }, + }, + domain: 'domain.hr', + async handle() { + return PII_RESULT; + }, + }; + const service = new ToolDispatchService({ + nativeTools: new NativeToolRegistry(), + domainTools: [domainTool], + privacy: () => redactingPrivacyHandle(), + }); + + const result = await service.dispatch('ask_hr', {}); + + assert.equal(result.content.includes(EMAIL), false, 'domain-tool branch leaked the email'); + assert.equal(result.content.includes(IBAN), false, 'domain-tool branch leaked the IBAN'); + assert.match(result.content, /\[masked:email\]/); + }); + + it('inherits an AMBIENT turn privacy handle when no explicit dep is wired', async () => { + const service = new ToolDispatchService({ + nativeTools: registryWith('odoo_read_partner', PII_RESULT), + domainTools: [], + }); + + const result = await turnContext.run( + { + privacyHandle: redactingPrivacyHandle(), + } as unknown as Parameters[0], + () => service.dispatch('odoo_read_partner', {}), + ); + + assert.equal( + result.content.includes(EMAIL), + false, + 'a dispatch inside a turn must inherit that turn privacy handle', + ); + assert.match(result.content, /\[masked:email\]/); + }); + + it('leaves the result UNCHANGED when no privacy provider is installed (parity with the orchestrator)', async () => { + const service = new ToolDispatchService({ + nativeTools: registryWith('odoo_read_partner', PII_RESULT), + domainTools: [], + }); + + const result = await service.dispatch('odoo_read_partner', {}); + + assert.equal(result.content, PII_RESULT); + }); + + it('honours the intern EXEMPTION list — a self/infra tool is not masked', async () => { + // `memory` is on `INTERN_EXEMPT_TOOLS`: masking it would blind the agent to + // its own operational state. The chat path exempts it, so this path must too. + const service = new ToolDispatchService({ + nativeTools: registryWith('memory', PII_RESULT), + domainTools: [], + privacy: () => redactingPrivacyHandle(), + }); + + const result = await service.dispatch('memory', {}); + + assert.equal(result.content, PII_RESULT, 'an intern-exempt tool must pass through raw'); + }); + + it('honours the operator BYPASS and records the receipt entry', async () => { + const receipts: RecordedBypass[] = []; + const service = new ToolDispatchService({ + nativeTools: registryWith('odoo_read_partner', PII_RESULT), + domainTools: [], + privacy: () => + redactingPrivacyHandle({ + bypassTools: new Set(['odoo_read_partner']), + bypassReceipts: receipts, + }), + }); + + const result = await service.dispatch('odoo_read_partner', {}); + + // Bypass means the operator explicitly opted this plugin out — raw is correct. + assert.equal(result.content, PII_RESULT); + // But it must stay auditable, exactly as on the chat path. + assert.deepEqual(receipts, [ + { + toolName: 'odoo_read_partner', + pluginId: 'plugin-for-odoo_read_partner', + bytes: Buffer.byteLength(PII_RESULT, 'utf8'), + }, + ]); + }); + + it('fails OPEN when the privacy provider throws — documented parity with the chat path', async () => { + const service = new ToolDispatchService({ + nativeTools: registryWith('odoo_read_partner', PII_RESULT), + domainTools: [], + privacy: () => redactingPrivacyHandle({ internThrows: true }), + }); + + const result = await service.dispatch('odoo_read_partner', {}); + + // `Orchestrator.dispatchToolDeadlined` logs and sends the raw result when + // interning throws. This path matches it deliberately rather than silently + // diverging; a fail-CLOSED policy for untrusted callers is its own decision. + assert.equal(result.content, PII_RESULT); + assert.equal(result.isError, undefined); + }); +}); + +describe('ToolDispatchService — raw-result capture (#542 prerequisite)', () => { + it('captures the RAW result before masking, while the caller gets the MASKED one', async () => { + const captured: Array<{ name: string; result: string; caller?: ToolDispatchCallerContext }> = []; + const service = new ToolDispatchService({ + nativeTools: registryWith('odoo_read_partner', PII_RESULT), + domainTools: [], + privacy: () => redactingPrivacyHandle(), + captureRawToolResult: (name, result, caller) => { + captured.push({ name, result, ...(caller !== undefined ? { caller } : {}) }); + }, + }); + + const result = await service.dispatch('odoo_read_partner', {}); + + // The trace consumer sees ground truth … + assert.equal(captured.length, 1); + assert.equal(captured[0]?.result, PII_RESULT); + // … and the caller does NOT. Both halves matter: capturing the masked value + // would make traces useless, returning the raw value would be the leak. + assert.equal(result.content.includes(EMAIL), false); + }); + + it('survives a throwing capture callback without failing the tool call', async () => { + const service = new ToolDispatchService({ + nativeTools: registryWith('odoo_read_partner', PII_RESULT), + domainTools: [], + captureRawToolResult: () => { + throw new Error('audit sink exploded'); + }, + }); + + const result = await service.dispatch('odoo_read_partner', {}); + + assert.equal(result.content, PII_RESULT); + assert.equal(result.isError, undefined); + }); +}); + +describe('ToolDispatchService — caller context seam (#542 prerequisite)', () => { + it('propagates the caller identity to layers BENEATH the handler', async () => { + const nativeTools = new NativeToolRegistry(); + let seenInsideHandler: ToolDispatchCallerContext | undefined; + nativeTools.register('whoami', { + // A plugin handler cannot receive identity as a parameter — the + // `NativeToolHandler` contract is published — so it must be readable + // ambiently, which is what this asserts. + handler: async () => { + seenInsideHandler = currentDispatchCaller(); + return 'ok'; + }, + spec: { + name: 'whoami', + description: 'd', + input_schema: { type: 'object', properties: {} }, + }, + domain: 'test.x', + }); + const service = new ToolDispatchService({ nativeTools, domainTools: [] }); + + const caller: ToolDispatchCallerContext = { + principal: 'apikey_123', + scopes: ['tools:write'], + tenantId: 'tenant-a', + userId: 'user-7', + requestId: 'req-abc', + }; + await service.dispatch('whoami', {}, { caller }); + + assert.deepEqual(seenInsideHandler, caller); + }); + + it('leaves the ambient caller EMPTY on the loopback path (no caller supplied)', async () => { + const nativeTools = new NativeToolRegistry(); + let seenInsideHandler: ToolDispatchCallerContext | undefined = { + principal: 'sentinel', + }; + nativeTools.register('whoami', { + handler: async () => { + seenInsideHandler = currentDispatchCaller(); + return 'ok'; + }, + spec: { + name: 'whoami', + description: 'd', + input_schema: { type: 'object', properties: {} }, + }, + domain: 'test.x', + }); + const service = new ToolDispatchService({ nativeTools, domainTools: [] }); + + await service.dispatch('whoami', {}); + + assert.equal(seenInsideHandler, undefined); + }); +}); From 4b26167a78cb0369c8b9b1d918a13b186add38b9 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 12:44:32 +0200 Subject: [PATCH 37/90] fix(orchestrator): propagate turnContext into the streaming tool loop MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `chatStream` established the turn scope with `turnContext.enter` (`AsyncLocalStorage.enterWith`). That binding survives only until the generator's first real suspension: from the first yielded event onward every continuation is resumed in the async context of whoever called `.next()`, so the store is gone. Anything read BEFORE the first yield (the privacy handle for prompt masking) accidentally worked; everything read at or after tool dispatch did not. Consequences, all silent and all on every streaming turn (web-ui + every channel): MCP audit rows degraded to `callerKind: 'unattributed'`, `turnId: null`, `callerAgent: null`; `mcpUserKey` was unreachable so `resolveIdentity` recorded `unresolved` and a `per_user` server got no token; the skill-binding persona gate refused every skill-bound MCP tool; chat-launched dev jobs failed closed on a missing `userId`; plugin memory writes landed under the `default` Agent namespace instead of the acting one. - Add `turnContext.runGenerator`, which wraps every advance of an inner generator in `storage.run` — the `run()` equivalent that composes with `yield`. The context value is passed BY REFERENCE per step so documented live-store writes (`activePersonaSkillId`, `mcpInputReplayNote`) keep working, and an abandoned stream's teardown runs inside the scope. - Split `chatStream` into a context-establishing wrapper plus `chatStreamInContext`, and document why `enter` must not be used from a generator. - Carry `mcpUserKey` through the three nested scopes that deliberately inherit the turn but dropped it: both orchestrator entry points, `runWithChatParticipants`, the plugin MCP accessor and the skill-tool hydration wrapper. Tests assert the OBSERVABLE audit row, not just the context object. All six were red before the fix; three deliberate mutations (dropped carry-over, per-step context copy, skipped inner teardown) each turn a real assertion red. --- .../harness-orchestrator/src/orchestrator.ts | 62 ++- .../harness-orchestrator/src/turnContext.ts | 71 ++- .../src/agents/subAgentToolHydration.ts | 4 + middleware/src/platform/pluginContext.ts | 4 + .../turnContextPropagation.test.ts | 500 ++++++++++++++++++ 5 files changed, 631 insertions(+), 10 deletions(-) create mode 100644 middleware/test/orchestrator/turnContextPropagation.test.ts diff --git a/middleware/packages/harness-orchestrator/src/orchestrator.ts b/middleware/packages/harness-orchestrator/src/orchestrator.ts index c9b9c492..26fb0752 100644 --- a/middleware/packages/harness-orchestrator/src/orchestrator.ts +++ b/middleware/packages/harness-orchestrator/src/orchestrator.ts @@ -168,7 +168,12 @@ import type { } from './llmProviderSeam.js'; import { streamMessageEvents } from './streaming.js'; import { steeringBus } from './steeringBus.js'; -import { buildDateHeader, today, turnContext } from './turnContext.js'; +import { + buildDateHeader, + today, + turnContext, + type TurnContextValue, +} from './turnContext.js'; import { resolveTurnOwnerIdentity } from './resolveTurnOwnerIdentity.js'; import { isMcpServerPrivacyBypassed } from './mcpPrivacyBypass.js'; import { isMcpServerKgIngest } from './mcpKgIngest.js'; @@ -2587,6 +2592,11 @@ export class Orchestrator { ...(parent?.chatParticipants ? { chatParticipants: parent.chatParticipants } : {}), + // W3-A — MCP OAuth caller identity. Set by an outer scope (channel + // adapter / route) and read by the auth provider's `getToken` + + // `resolveIdentity`. Without the carry-over a `per_user` server audits + // every call as `unresolved` and then fails closed. + ...(parent?.mcpUserKey ? { mcpUserKey: parent.mcpUserKey } : {}), ...(privacyHandle ? { privacyHandle } : {}), ...(parent?.captureRawToolResult ? { captureRawToolResult: parent.captureRawToolResult } @@ -4016,10 +4026,16 @@ export class Orchestrator { if (mcpInputReply) { input = { ...input, userMessage: mcpInputReplyLabel(mcpInputReply) }; } - // `enter` (not `run`) because AsyncLocalStorage.run doesn't compose with - // async generators. `enter` binds turnId to the current async resource, - // which the generator's awaits inherit; scope ends when the HTTP request - // resource is cleaned up. + // W3-A — this used to be `turnContext.enter` (AsyncLocalStorage.enterWith). + // That does NOT survive a generator's first `yield`: the generator is + // resumed in the async context of whoever called `.next()`, so by the time + // the tool loop ran, `turnContext.current()` was empty (or, worse, bound to + // the consumer's ambient scope). Everything that reads the turn context at + // dispatch time was therefore broken on every streaming turn — MCP audit + // attribution (`callerKind`/`turnId`/`callerAgent`/`mcpUserKey`), the + // skill-binding persona gate, the privacy handle, the KG-ingest owner. + // The body now runs through `turnContext.runGenerator`, which wraps every + // advance of the inner generator in `storage.run`. const parent = turnContext.current(); // Privacy-Proxy Slice 2.1: same handle pattern as `runTurn`. The handle @@ -4042,7 +4058,7 @@ export class Orchestrator { input, ); - turnContext.enter({ + const context: TurnContextValue = { turnId, turnDate: today(), // Per-orchestrator isolation: see the matching `turnContext.run` above. @@ -4057,6 +4073,8 @@ export class Orchestrator { ...(parent?.chatParticipants ? { chatParticipants: parent.chatParticipants } : {}), + // W3-A — see the matching `turnContext.run` above. + ...(parent?.mcpUserKey ? { mcpUserKey: parent.mcpUserKey } : {}), ...(privacyHandle ? { privacyHandle } : {}), ...(parent?.captureRawToolResult ? { captureRawToolResult: parent.captureRawToolResult } @@ -4066,7 +4084,37 @@ export class Orchestrator { ...(parent?.canvasSentinelSink ? { canvasSentinelSink: parent.canvasSentinelSink } : {}), - }); + }; + // `input` is re-bound above (envelope normalisation); capture the final + // value so the body cannot observe the pre-normalisation message. + const turnInput = input; + yield* turnContext.runGenerator(context, () => + this.chatStreamInContext({ + input: turnInput, + turnId, + sessionId, + mcpInputReply, + ...(privacyHandle ? { privacyHandle } : {}), + ...(observer ? { observer } : {}), + }), + ); + } + + /** + * The body of {@link chatStream}, run inside the turn's AsyncLocalStorage + * scope by `turnContext.runGenerator`. Split out purely so the context can be + * established with `run()` semantics instead of `enterWith` — see the comment + * at the top of `chatStream`. + */ + private async *chatStreamInContext(args: { + readonly input: ChatTurnInput; + readonly turnId: string; + readonly sessionId: string; + readonly mcpInputReply: McpInputReply | undefined; + readonly privacyHandle?: PrivacyTurnHandle; + readonly observer?: AskObserver; + }): AsyncGenerator { + const { input, turnId, sessionId, mcpInputReply, privacyHandle, observer } = args; this.applyTurnAuthContext(input); // W2-1 (#544) — forced replay before the model runs. Mirror of `runTurn`. diff --git a/middleware/packages/harness-orchestrator/src/turnContext.ts b/middleware/packages/harness-orchestrator/src/turnContext.ts index 25e1566d..48cfc1f1 100644 --- a/middleware/packages/harness-orchestrator/src/turnContext.ts +++ b/middleware/packages/harness-orchestrator/src/turnContext.ts @@ -214,13 +214,39 @@ export const turnContext = { }, /** * Sets the turn context for the current async resource and its descendants. - * Used from async generators (`chatStream`) because AsyncLocalStorage.run() - * doesn't compose with `yield`. Scope is bounded by the enclosing HTTP - * request — a new request creates a fresh async resource chain. + * + * ⚠️ NOT usable from an async generator. `enterWith` binds the store to the + * async resource that is executing at that instant, but a generator is + * resumed in the async context of whoever called `.next()` — so the store is + * gone the moment the generator yields, and every continuation after that + * point sees either nothing or the CONSUMER's ambient scope. The streaming + * orchestrator entry point used to do exactly this, which silently broke MCP + * audit attribution (`callerKind`/`turnId`/`mcpUserKey`) on every streaming + * turn. Use {@link runGenerator} from generators. + * + * Correct uses are plain async functions whose own async chain bounds the + * scope — e.g. an Express route handler establishing a per-request identity. */ enter(value: TurnContextValue): void { storage.enterWith(value); }, + /** + * Establishes `value` for the entire lifetime of an async generator — the + * `run()` equivalent that composes with `yield`. + * + * Every advance of the inner generator is performed inside `storage.run`, so + * the context is active for exactly the spans that execute generator body + * code, and is NOT active while the consumer processes a yielded value. + * `value` is passed by reference on every step, so writes onto the live store + * (`activePersonaSkillId`, `mcpInputReplayNote`) stay visible to later steps + * — same semantics `run()` gives a plain async function. + */ + runGenerator( + value: TurnContextValue, + makeGenerator: () => AsyncGenerator, + ): AsyncGenerator { + return runGeneratorInContext(value, makeGenerator); + }, /** * Runs `fn` in an outer scope that only installs a `chatParticipants` * provider — turnId/turnDate are left as placeholders the orchestrator @@ -239,6 +265,10 @@ export const turnContext = { turnDate: prev?.turnDate ?? today(), ...(prev?.agentSlug ? { agentSlug: prev.agentSlug } : {}), chatParticipants, + // W3-A: the caller identity MCP OAuth tokens are keyed to must survive + // an adapter-established outer scope, or every audited MCP call on a + // channel turn records `unresolved` and a `per_user` server fails closed. + ...(prev?.mcpUserKey ? { mcpUserKey: prev.mcpUserKey } : {}), ...(prev?.privacyHandle ? { privacyHandle: prev.privacyHandle } : {}), ...(prev?.captureRawToolResult ? { captureRawToolResult: prev.captureRawToolResult } @@ -275,6 +305,41 @@ export const turnContext = { }, }; +/** + * Implementation of {@link turnContext.runGenerator}. Kept as a module-level + * generator function (rather than inline) so it can `yield` while still owning + * the `storage.run` wrapping of every `next()`. + */ +async function* runGeneratorInContext( + value: TurnContextValue, + makeGenerator: () => AsyncGenerator, +): AsyncGenerator { + // Create inside the scope too: a factory that reads the context eagerly + // (before its first yield) then behaves the same as one that reads it later. + const inner = storage.run(value, makeGenerator); + let exhausted = false; + try { + for (;;) { + const step = await storage.run(value, () => inner.next()); + if (step.done) { + exhausted = true; + return; + } + yield step.value; + } + } finally { + // The consumer broke out of its loop or threw. Drive the inner generator's + // own `finally` blocks (steering-bus teardown, privacy finalisation) INSIDE + // the turn scope — outside it they would run context-less, which is the + // very bug this helper exists to prevent. + if (!exhausted) { + await storage.run(value, async () => { + await inner.return(undefined); + }); + } + } +} + /** `YYYY-MM-DD` in Europe/Berlin. Single place this computation lives. */ export function today(): string { return new Intl.DateTimeFormat('en-CA', { diff --git a/middleware/src/agents/subAgentToolHydration.ts b/middleware/src/agents/subAgentToolHydration.ts index fca2fb8d..f6ed9db9 100644 --- a/middleware/src/agents/subAgentToolHydration.ts +++ b/middleware/src/agents/subAgentToolHydration.ts @@ -401,6 +401,10 @@ export function registerDbSubAgentTools( turnDate: current.turnDate, ...(current.agentSlug ? { agentSlug: current.agentSlug } : {}), ...(current.privacyHandle ? { privacyHandle: current.privacyHandle } : {}), + // W3-A — same carry-over as the plugin accessor: without it a + // skill-bound MCP call reaches a `per_user` server with no + // identity, audits as `unresolved` and fails closed. + ...(current.mcpUserKey ? { mcpUserKey: current.mcpUserKey } : {}), activePersonaSkillId: skillId, mcpCallerKind: 'skill', mcpCallerId: skillSlug, diff --git a/middleware/src/platform/pluginContext.ts b/middleware/src/platform/pluginContext.ts index 9e92b390..f4a2a70e 100644 --- a/middleware/src/platform/pluginContext.ts +++ b/middleware/src/platform/pluginContext.ts @@ -896,6 +896,10 @@ export function createPluginMcpAccessor( turnDate: current?.turnDate ?? new Date().toISOString().slice(0, 10), ...(current?.agentSlug ? { agentSlug: current.agentSlug } : {}), ...(current?.privacyHandle ? { privacyHandle: current.privacyHandle } : {}), + // W3-A — the turn's MCP OAuth identity. Dropping it here made every + // plugin-attributed call resolve as an unknown caller: `unresolved` in + // the audit trail, no token, and a `per_user` server failing closed. + ...(current?.mcpUserKey ? { mcpUserKey: current.mcpUserKey } : {}), mcpCallerKind: 'plugin', mcpCallerId: pluginId, }, diff --git a/middleware/test/orchestrator/turnContextPropagation.test.ts b/middleware/test/orchestrator/turnContextPropagation.test.ts new file mode 100644 index 00000000..5dc310e2 --- /dev/null +++ b/middleware/test/orchestrator/turnContextPropagation.test.ts @@ -0,0 +1,500 @@ +/** + * W3-A — `turnContext` must reach tool handlers on BOTH orchestrator entry + * points. + * + * The streaming entry point used to establish the turn scope with + * `AsyncLocalStorage.enterWith` (`turnContext.enter`). `enterWith` binds the + * store to the async resource that is executing at that instant; an async + * generator, however, is resumed in the async context of whoever called + * `.next()`. So the moment `chatStream` yielded its first event the store was + * gone, and every tool handler further down ran with `turnContext.current()` + * either `undefined` or — worse — bound to whatever OUTER scope the consumer + * happened to be iterating from. + * + * That silently broke the MCP audit trail on every streaming turn (which is + * every web-ui and every channel turn): `callerKind` degraded to + * `unattributed`, `turnId` to `null`, `callerAgent` to `null`, and the + * per-user OAuth identity (`mcpUserKey`) was unreachable, so `resolveIdentity` + * recorded `unresolved`. + * + * These tests therefore assert the OBSERVABLE audit row, not just the context + * object. Tests labelled MUTATION CHECK were verified by breaking the + * invariant, rebuilding, and confirming the assertion turns red. + */ +import { after, describe, it } from 'node:test'; +import { strict as assert } from 'node:assert'; +import { + createServer, + type IncomingMessage, + type Server as HttpServer, + type ServerResponse, +} from 'node:http'; +import type { AddressInfo } from 'node:net'; + +import { Server as McpSdkServer } from '@modelcontextprotocol/sdk/server/index.js'; +import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js'; +import { + CallToolRequestSchema, + ListToolsRequestSchema, +} from '@modelcontextprotocol/sdk/types.js'; + +import type { + LlmProvider, + LlmRequest, + LlmResponse, + LlmStreamEvent, +} from '@omadia/llm-provider'; +import { + McpManager, + NativeToolRegistry, + Orchestrator, + steeringBus, + turnContext, + type McpCallLogEntry, + type McpServerConfig, + type TurnContextValue, +} from '@omadia/orchestrator'; + +import { UNRESOLVED_IDENTITY, auditIdentity } from '../../src/services/mcpDelegation.js'; + +// ── fake MCP server ───────────────────────────────────────────────────────── + +function buildMcpServerInstance(): McpSdkServer { + const mcp = new McpSdkServer( + { name: 'fake-crm', version: '0.0.0' }, + { capabilities: { tools: {} } }, + ); + mcp.setRequestHandler(ListToolsRequestSchema, async () => ({ + tools: [{ name: 'ping', inputSchema: { type: 'object' as const } }], + })); + mcp.setRequestHandler(CallToolRequestSchema, async () => ({ + content: [{ type: 'text' as const, text: 'pong' }], + })); + return mcp; +} + +async function startFakeMcpServer(): Promise<{ url: string; close(): Promise }> { + const handle = async (req: IncomingMessage, res: ServerResponse): Promise => { + const mcp = buildMcpServerInstance(); + const transport = new StreamableHTTPServerTransport({ + sessionIdGenerator: undefined, + enableJsonResponse: true, + }); + res.on('close', () => { + void transport.close().catch(() => {}); + void mcp.close().catch(() => {}); + }); + await mcp.connect(transport); + if (req.method === 'POST') { + const chunks: Buffer[] = []; + for await (const c of req) chunks.push(c as Buffer); + const raw = Buffer.concat(chunks).toString('utf8'); + await transport.handleRequest(req, res, raw.length > 0 ? JSON.parse(raw) : undefined); + return; + } + await transport.handleRequest(req, res); + }; + const http: HttpServer = createServer((req, res) => { + void handle(req, res).catch(() => { + if (!res.headersSent) res.writeHead(500); + res.end(); + }); + }); + const sockets = new Set(); + http.on('connection', (socket) => { + sockets.add(socket); + socket.on('close', () => sockets.delete(socket)); + }); + await new Promise((resolve) => http.listen(0, '127.0.0.1', () => resolve())); + const { port } = http.address() as AddressInfo; + return { + url: `http://127.0.0.1:${port}/mcp`, + async close() { + for (const socket of sockets) socket.destroy(); + sockets.clear(); + await new Promise((resolve) => http.close(() => resolve())); + }, + }; +} + +const fake = await startFakeMcpServer(); +after(() => fake.close()); + +const CFG: McpServerConfig = { + id: '00000000-0000-4000-8000-0000000003a0', + name: 'Kunden-CRM', + transport: 'http', + endpoint: fake.url, +}; + +// ── scripted provider ─────────────────────────────────────────────────────── + +const providerCapabilities = { + tools: true, + vision: true, + streaming: true, + promptCaching: true, + forcedToolChoice: true, + parallelToolCalls: true, +} as const; + +function fakeProvider(streams: LlmStreamEvent[][]): LlmProvider { + let idx = 0; + const take = (): LlmStreamEvent[] => { + if (idx >= streams.length) { + throw new Error(`no scripted stream for provider call ${String(idx + 1)}`); + } + const events = streams[idx]!; + idx += 1; + return events; + }; + return { + id: 'anthropic', + capabilities: providerCapabilities, + complete: async (): Promise => { + const events = take(); + const final = events.at(-1) as { type: string; response: LlmResponse }; + return final.response; + }, + stream: (): AsyncIterable => { + const events = take(); + return { + async *[Symbol.asyncIterator]() { + for (const ev of events) yield ev; + }, + }; + }, + classifyError: () => ({ retryable: false, kind: 'other' as const }), + } as unknown as LlmProvider; +} + +function toolCallStream( + calls: Array<{ id: string; name: string; input: unknown }>, +): LlmStreamEvent[] { + return [ + { + type: 'final', + response: { + content: calls.map((c) => ({ + type: 'tool_call', + id: c.id, + name: c.name, + input: c.input, + })), + finishReason: 'tool_calls', + providerFinishReason: 'tool_use', + model: 'test', + usage: { inputTokens: 50, outputTokens: 4, cacheReadTokens: 0, cacheWriteTokens: 0 }, + }, + } as LlmStreamEvent, + ]; +} + +function textStream(text: string): LlmStreamEvent[] { + return [ + { type: 'text_delta', text }, + { + type: 'final', + response: { + content: [{ type: 'text', text }], + finishReason: 'stop', + providerFinishReason: 'end_turn', + model: 'test', + usage: { inputTokens: 100, outputTokens: 1, cacheReadTokens: 0, cacheWriteTokens: 0 }, + }, + } as LlmStreamEvent, + ]; +} + +const PROBE_TOOL = 'probe_turn_context'; +const AGENT_SLUG = 'probe-agent'; + +/** A snapshot of what a tool handler saw, so assertions read the HANDLER's + * view of the turn rather than the test's. */ +interface Seen { + readonly defined: boolean; + readonly turnId: string | undefined; + readonly agentSlug: string | undefined; + readonly userId: string | undefined; + readonly sessionScope: string | undefined; + readonly mcpUserKey: string | undefined; +} + +function snapshot(ctx: TurnContextValue | undefined): Seen { + return { + defined: ctx !== undefined, + turnId: ctx?.turnId, + agentSlug: ctx?.agentSlug, + userId: ctx?.userId, + sessionScope: ctx?.sessionScope, + mcpUserKey: ctx?.mcpUserKey, + }; +} + +interface Harness { + readonly orchestrator: Orchestrator; + /** One entry per `probe_turn_context` dispatch, in dispatch order. */ + readonly seen: Seen[]; + /** Every audit row the McpManager emitted. */ + readonly audit: McpCallLogEntry[]; +} + +/** + * @param callMcp when true the probe handler ALSO makes a real MCP call, so + * the audit row is produced from inside the tool-dispatch + * call tree — exactly where production makes it. + */ +function harness( + streams: LlmStreamEvent[][], + opts?: { readonly callMcp?: boolean }, +): Harness { + const seen: Seen[] = []; + const audit: McpCallLogEntry[] = []; + const manager = new McpManager({ + onToolCall: (entry) => audit.push(entry), + auth: { + // Mirrors the production wiring in `src/index.ts`: the identity is + // resolved from the TURN CONTEXT, per server delegation mode. + getToken: async () => null, + onAuthFailure: async () => null, + resolveIdentity: async () => + auditIdentity({ delegation: 'per_user' }, turnContext.current()?.mcpUserKey), + }, + }); + const registry = new NativeToolRegistry(); + registry.register(PROBE_TOOL, { + handler: async () => { + seen.push(snapshot(turnContext.current())); + if (opts?.callMcp) return manager.callTool(CFG, 'ping', {}); + return 'ok'; + }, + spec: { + name: PROBE_TOOL, + description: 'Records the ambient turn context.', + input_schema: { type: 'object' as const, properties: {}, required: [] }, + // eslint-disable-next-line @typescript-eslint/no-explicit-any + } as any, + agentId: 'probe', + }); + const orchestrator = new Orchestrator({ + provider: fakeProvider(streams), + model: 'test', + maxTokens: 1024, + maxToolIterations: 5, + domainTools: [], + nativeToolRegistry: registry, + agentId: AGENT_SLUG, + }); + return { orchestrator, seen, audit }; +} + +const oneToolTurn = (): LlmStreamEvent[][] => [ + toolCallStream([{ id: 'tu-1', name: PROBE_TOOL, input: {} }]), + textStream('fertig'), +]; + +async function drain(orchestrator: Orchestrator, userId = 'u1'): Promise { + for await (const _ of orchestrator.chatStream({ + userMessage: 'los', + sessionScope: 'sess-w3a', + userId, + })) { + // drain + } +} + +// ── 1. the context reaches a tool handler on BOTH paths ───────────────────── + +describe('turnContext reaches tool handlers (W3-A)', () => { + it('MUTATION CHECK: buffered path (runTurn) — handler sees the full turn context', async () => { + const h = harness(oneToolTurn()); + await h.orchestrator.runTurn({ + userMessage: 'los', + sessionScope: 'sess-w3a', + userId: 'u1', + }); + assert.equal(h.seen.length, 1, 'the probe tool never ran'); + const s = h.seen[0]!; + assert.equal(s.defined, true, 'turnContext.current() was undefined in the tool handler'); + assert.ok(s.turnId && s.turnId.length > 0, 'no turnId in the tool handler'); + assert.equal(s.agentSlug, AGENT_SLUG); + assert.equal(s.userId, 'u1'); + assert.equal(s.sessionScope, 'sess-w3a'); + }); + + it('MUTATION CHECK: streaming path (chatStream) — handler sees the full turn context', async () => { + // This is THE regression. `turnContext.enter` (enterWith) does not survive + // the generator's first `yield`, so before the fix `defined` was false. + const h = harness(oneToolTurn()); + await drain(h.orchestrator); + assert.equal(h.seen.length, 1, 'the probe tool never ran'); + const s = h.seen[0]!; + assert.equal(s.defined, true, 'turnContext.current() was undefined in the tool handler'); + assert.ok(s.turnId && s.turnId.length > 0, 'no turnId in the tool handler'); + assert.equal(s.agentSlug, AGENT_SLUG); + assert.equal(s.userId, 'u1'); + assert.equal(s.sessionScope, 'sess-w3a'); + }); + + it('MUTATION CHECK: streaming turns do not leak each other`s context', async () => { + // Two turns on the SAME orchestrator, interleaved at the generator level: + // both streams are advanced alternately, so a single shared store (or an + // `enterWith` that bleeds across async resources) shows up as a duplicate + // turnId here. + const a = harness(oneToolTurn()); + const b = harness(oneToolTurn()); + const genA = a.orchestrator.chatStream({ userMessage: 'a', sessionScope: 's-a', userId: 'ua' }); + const genB = b.orchestrator.chatStream({ userMessage: 'b', sessionScope: 's-b', userId: 'ub' }); + let doneA = false; + let doneB = false; + while (!doneA || !doneB) { + if (!doneA) doneA = (await genA.next()).done === true; + if (!doneB) doneB = (await genB.next()).done === true; + } + assert.equal(a.seen[0]?.userId, 'ua'); + assert.equal(b.seen[0]?.userId, 'ub'); + assert.notEqual(a.seen[0]?.turnId, b.seen[0]?.turnId, 'both turns shared one turnId'); + assert.equal(a.seen[0]?.sessionScope, 's-a'); + assert.equal(b.seen[0]?.sessionScope, 's-b'); + }); + + it('MUTATION CHECK: a write onto the LIVE store survives to a later tool iteration', async () => { + // `activePersonaSkillId` and `mcpInputReplayNote` are documented to be + // MUTATED onto the live store inside the turn scope. A propagation fix that + // re-created the context object per generator step would silently drop + // those writes, so pin the behaviour: iteration 1 writes, iteration 2 reads. + const observed: Array = []; + const registry = new NativeToolRegistry(); + registry.register(PROBE_TOOL, { + handler: async () => { + const ctx = turnContext.current(); + observed.push(ctx?.activePersonaSkillId); + if (ctx) ctx.activePersonaSkillId = 'persona-x'; + return 'ok'; + }, + spec: { + name: PROBE_TOOL, + description: 'Mutates the live turn context.', + input_schema: { type: 'object' as const, properties: {}, required: [] }, + // eslint-disable-next-line @typescript-eslint/no-explicit-any + } as any, + agentId: 'probe', + }); + const orchestrator = new Orchestrator({ + provider: fakeProvider([ + toolCallStream([{ id: 'tu-1', name: PROBE_TOOL, input: {} }]), + toolCallStream([{ id: 'tu-2', name: PROBE_TOOL, input: {} }]), + textStream('fertig'), + ]), + model: 'test', + maxTokens: 1024, + maxToolIterations: 5, + domainTools: [], + nativeToolRegistry: registry, + agentId: AGENT_SLUG, + }); + for await (const _ of orchestrator.chatStream({ userMessage: 'los', sessionScope: 's' })) { + // drain + } + assert.deepEqual(observed, [undefined, 'persona-x'], 'live-store mutation was lost'); + }); + + it('MUTATION CHECK: an abandoned stream still tears the turn down INSIDE the scope', async () => { + // A web-ui client disconnecting mid-turn `break`s out of the `for await`. + // The body's own `finally` (steering-bus teardown, privacy finalisation) + // must still run, and must still see the turn context — outside it those + // handlers would operate on the wrong (or no) turn. + const registry = new NativeToolRegistry(); + registry.register(PROBE_TOOL, { + handler: async () => 'ok', + spec: { + name: PROBE_TOOL, + description: 'noop', + input_schema: { type: 'object' as const, properties: {}, required: [] }, + // eslint-disable-next-line @typescript-eslint/no-explicit-any + } as any, + agentId: 'probe', + }); + const orchestrator = new Orchestrator({ + provider: fakeProvider(oneToolTurn()), + model: 'test', + maxTokens: 1024, + maxToolIterations: 5, + domainTools: [], + nativeToolRegistry: registry, + agentId: AGENT_SLUG, + }); + const seenByConsumer: Array = []; + for await (const event of orchestrator.chatStream({ + userMessage: 'los', + sessionScope: 's-abandon', + })) { + seenByConsumer.push(turnContext.currentTurnId()); + if (event.type === 'tool_result') break; // client disconnected + } + // 1. The body's `finally` ran: the steering bus released this scope. An + // abandoned turn that stays "live" would accept steers forever. + assert.equal( + steeringBus.enqueue('s-abandon', 'nachtrag').live, + false, + 'the turn was never torn down — steeringBus still reports it live', + ); + // 2. The CONSUMER must never inherit the turn scope. That leak is exactly + // what `enterWith` produced, and it is why an audit row could be + // attributed to the consumer's ambient turn instead of this one. + assert.deepEqual( + [...new Set(seenByConsumer)], + [undefined], + 'the turn scope leaked into the consumer', + ); + }); +}); + +// ── 2. the DOWNSTREAM consequence: the MCP audit row ──────────────────────── + +describe('MCP audit attribution on a streaming turn (W3-A)', () => { + it('MUTATION CHECK: the audit row names the agent and the turn, not `unattributed`', async () => { + const h = harness(oneToolTurn(), { callMcp: true }); + await drain(h.orchestrator); + assert.equal(h.audit.length, 1, 'no mcp_call_log row was emitted'); + const row = h.audit[0]!; + // Before the fix: 'unattributed' / null / null — on EVERY streaming turn. + assert.equal(row.callerKind, 'agent'); + assert.ok(row.turnId !== null && row.turnId.length > 0, 'audit row has no turnId'); + assert.equal(row.callerAgent, AGENT_SLUG); + assert.equal(row.turnId, h.seen[0]?.turnId, 'audit row and handler disagree on the turn'); + assert.equal(row.outcome, 'ok'); + }); + + it('MUTATION CHECK: a per-user identity established by an outer scope reaches the audit row', async () => { + // A channel adapter (Teams) establishes the caller identity in an OUTER ALS + // scope and then consumes `chatStream` inside it — the same shape + // `runWithChatParticipants` uses. Both the propagation fix AND the + // `mcpUserKey` carry-over are required for this to be anything but + // `unresolved`. + const h = harness(oneToolTurn(), { callMcp: true }); + await turnContext.run( + { + turnId: 'outer-adapter-turn', + turnDate: '2026-07-30', + mcpUserKey: 'alice@example.com', + }, + async () => { + await drain(h.orchestrator); + }, + ); + assert.equal(h.audit.length, 1, 'no mcp_call_log row was emitted'); + const row = h.audit[0]!; + assert.notEqual( + row.actingIdentity, + UNRESOLVED_IDENTITY, + 'per_user delegation recorded `unresolved` despite a resolvable caller', + ); + assert.equal(row.actingIdentity, 'alice@example.com'); + // The row must be attributed to THIS turn, never to the adapter's outer + // placeholder scope — which is exactly what a leaked `enterWith` produced. + assert.notEqual(row.turnId, 'outer-adapter-turn'); + assert.equal(row.callerAgent, AGENT_SLUG); + assert.equal(h.seen[0]?.mcpUserKey, 'alice@example.com'); + }); +}); From 1a352b9c263cc8c93215af92faf9cfc2d8ce88e8 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 12:46:34 +0200 Subject: [PATCH 38/90] test(orchestrator): mutation-checked exactly-once semantics for write tool dispatch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 20 tests. The end-to-end suite drives a real LoopbackMcpServer behind a proxy that forwards the first tools/call upstream (the server really executes) and only then loses the response — the shape the transport retry cannot distinguish from a pre-execution failure. Side effects are counted on the server itself. A CONTROL test proves the hazard is real (2 writes without a key) alongside the protected case (1 write with one). Verified by breaking three more invariants: retry clamp removed (1 fail), dispatch dedupe bypassed (4 fail), every tool treated as write-capable so reads lose the mitigation (4 fail). Records a dual-module-graph gotcha: mixing dist and src imports yields two AsyncLocalStorage instances and the scope silently never arrives. --- middleware/test/mcpWriteIdempotency.test.ts | 389 ++++++++++++++++++++ middleware/test/toolIdempotency.test.ts | 317 ++++++++++++++++ 2 files changed, 706 insertions(+) create mode 100644 middleware/test/mcpWriteIdempotency.test.ts create mode 100644 middleware/test/toolIdempotency.test.ts diff --git a/middleware/test/mcpWriteIdempotency.test.ts b/middleware/test/mcpWriteIdempotency.test.ts new file mode 100644 index 00000000..4fb40644 --- /dev/null +++ b/middleware/test/mcpWriteIdempotency.test.ts @@ -0,0 +1,389 @@ +import { strict as assert } from 'node:assert'; +import { afterEach, describe, it } from 'node:test'; +import { + createServer, + type IncomingHttpHeaders, + type IncomingMessage, + type Server, + type ServerResponse, +} from 'node:http'; +import type { AddressInfo } from 'node:net'; + +// IMPORTANT: every symbol here comes from `packages/.../src`, never from the +// built `@omadia/orchestrator` entry point. The idempotency scope is carried by +// an `AsyncLocalStorage` instance that lives in a module — importing `McpManager` +// from `dist` while importing `ToolDispatchService` from `src` gives two separate +// module graphs, hence two separate ALS instances, and the scope silently never +// reaches the transport layer. That failure looks exactly like a broken feature. +import { LoopbackMcpServer } from '../packages/harness-orchestrator/src/loopbackMcpServer.js'; +import { + McpManager, + mcpNativeHandler, + type McpServerConfig, +} from '../packages/harness-orchestrator/src/mcp/mcpClient.js'; +import { NativeToolRegistry } from '../packages/harness-orchestrator/src/nativeToolRegistry.js'; +import { ToolDispatchService } from '../packages/harness-orchestrator/src/toolDispatchService.js'; +import { ToolIdempotencyStore } from '../packages/harness-orchestrator/src/toolIdempotency.js'; +import type { WriteCapability } from '../packages/plugin-api/src/writeCapabilities.js'; + +/** + * #542 prerequisite — duplicate-write protection across the MCP transport retry. + * + * `McpManager.callTool` retries ONCE on a transient transport failure. That is a + * deliberate, shipped mitigation for a flaky hosted proxy and it stays. But a + * transient failure is indistinguishable from "the server executed the write and + * the response was lost on the way back", so for a write-capable tool the retry + * can duplicate a mutation. + * + * THE MUTATION CHECK: the proxy below forwards the first `tools/call` UPSTREAM — + * so the server really executes and really records a side effect — and only THEN + * replaces the response with a transport error. `writes()` counts side effects + * observed by the server itself, not mock invocations. The control test proves + * the hazard is real (2 writes); the protected test proves the fix (1 write). + */ + +const BEARER = 'loopback-secret-token'; +const REMOTE_TOOL = 'create_invoice'; +const LOCAL_TOOL = 'odoo_create_invoice'; +const CREATE_INVOICE: readonly WriteCapability[] = [ + { dataClass: 'odoo.invoice', operation: 'create' }, +]; + +const HOP_BY_HOP = new Set([ + 'connection', + 'content-length', + 'host', + 'keep-alive', + 'transfer-encoding', + 'upgrade', +]); + +function forwardableHeaders(headers: IncomingHttpHeaders): Record { + const out: Record = {}; + for (const [key, value] of Object.entries(headers)) { + if (HOP_BY_HOP.has(key.toLowerCase()) || value === undefined) continue; + out[key] = Array.isArray(value) ? value.join(', ') : value; + } + return out; +} + +function isSandboxListenError(error: unknown): boolean { + return ( + error instanceof Error && 'code' in error && (error as { code?: string }).code === 'EPERM' + ); +} + +function serverConfig(url: string): McpServerConfig { + return { + id: '00000000-0000-4000-8000-0000000wr1te'.replace('wr1te', 'c0ded'), + name: 'loopback-write', + transport: 'http', + endpoint: url, + headers: { Authorization: `Bearer ${BEARER}` }, + }; +} + +interface LosingProxy { + readonly url: string; + readonly toolCallCount: () => number; + readonly stop: () => Promise; +} + +/** + * Proxy that loses the RESPONSE to the first `tools/call` after the upstream + * server already handled it. This is the dangerous shape the retry cannot + * distinguish: the write happened, the caller only saw a dropped connection. + * + * `targets` holds two upstreams because a `LoopbackMcpServer` accepts exactly one + * Streamable-HTTP session — the retry legitimately reconnects, so it must land on + * a second instance, modelling the hosted proxy failing over to a healthy node. + */ +async function startLosingProxy(targets: readonly string[]): Promise { + let toolCalls = 0; + let targetIdx = 0; + + const handle = async (req: IncomingMessage, res: ServerResponse): Promise => { + const chunks: Buffer[] = []; + for await (const chunk of req) { + chunks.push(typeof chunk === 'string' ? Buffer.from(chunk, 'utf8') : chunk); + } + const body = Buffer.concat(chunks); + const text = body.toString('utf8'); + const isToolCall = req.method === 'POST' && text.includes('"tools/call"'); + if (isToolCall) toolCalls += 1; + const loseResponse = isToolCall && toolCalls === 1; + + const target = targets[targetIdx] ?? targets[0]!; + const upstream = await fetch(target, { + method: req.method ?? 'GET', + headers: forwardableHeaders(req.headers), + ...(body.length > 0 ? { body } : {}), + }); + // Drain upstream so the server completes the call (and its side effect). + const upstreamText = await upstream.text(); + + if (loseResponse) { + // The write DID happen upstream. Now drop the answer on the floor and hand + // back the transport error a flaky hosted proxy actually returns. + targetIdx = Math.min(targetIdx + 1, targets.length - 1); + const id = (() => { + try { + return (JSON.parse(text) as { id?: unknown }).id ?? null; + } catch { + return null; + } + })(); + res.writeHead(200, { 'content-type': 'application/json' }); + res.end( + JSON.stringify({ + jsonrpc: '2.0', + id, + error: { code: -32000, message: 'Connection closed' }, + }), + ); + return; + } + + const responseHeaders: Record = {}; + upstream.headers.forEach((value, key) => { + if (!HOP_BY_HOP.has(key.toLowerCase())) responseHeaders[key] = value; + }); + res.writeHead(upstream.status, responseHeaders); + res.end(upstreamText); + }; + + const server: Server = createServer((req, res) => { + void handle(req, res).catch(() => { + if (!res.headersSent) res.writeHead(502); + res.end(); + }); + }); + await new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(0, '127.0.0.1', () => { + server.removeListener('error', reject); + resolve(); + }); + }); + const port = (server.address() as AddressInfo).port; + return { + url: `http://127.0.0.1:${String(port)}/mcp`, + toolCallCount: () => toolCalls, + stop: () => + new Promise((resolve) => { + server.close(() => resolve()); + }), + }; +} + +describe('write-capable MCP tool — duplicate-write protection (#542 prerequisite)', () => { + const servers: LoopbackMcpServer[] = []; + let proxy: LosingProxy | undefined; + let manager: McpManager | undefined; + + afterEach(async () => { + await manager?.closeAll(); + await proxy?.stop(); + for (const s of servers.splice(0)) await s.stop(); + manager = undefined; + proxy = undefined; + }); + + /** + * A real `LoopbackMcpServer` whose tool records a side effect per execution. + * `writes` is shared across instances so a retry landing on the second server + * still increments the SAME counter — that is what makes it a true count of + * effects rather than a per-connection statistic. + */ + async function startWriteServer( + t: { skip: (reason: string) => void }, + writes: string[], + ): Promise { + const remote = new NativeToolRegistry(); + remote.register(REMOTE_TOOL, { + handler: async (input) => { + writes.push(JSON.stringify(input)); + return `invoice #${String(writes.length)} created`; + }, + spec: { + name: REMOTE_TOOL, + description: 'creates an invoice (side effect)', + input_schema: { type: 'object', properties: { amount: { type: 'number' } } }, + }, + domain: 'test.odoo', + }); + const server = new LoopbackMcpServer({ + dispatch: new ToolDispatchService({ + nativeTools: remote, + domainTools: [], + }), + bearer: BEARER, + tools: [ + { + name: REMOTE_TOOL, + description: 'creates an invoice (side effect)', + input_schema: { + type: 'object', + properties: { amount: { type: 'number' } }, + }, + }, + ], + }); + try { + const handle = await server.start(); + servers.push(server); + return handle.url; + } catch (error) { + if (isSandboxListenError(error)) { + t.skip('sandbox blocks loopback listeners on 127.0.0.1'); + return undefined; + } + throw error; + } + } + + /** Local dispatcher whose write tool forwards to the remote MCP server. */ + function localDispatcher( + mgr: McpManager, + cfg: McpServerConfig, + options: { readonly declareWrite: boolean; readonly store?: ToolIdempotencyStore }, + ): ToolDispatchService { + const nativeTools = new NativeToolRegistry(); + nativeTools.register(LOCAL_TOOL, { + handler: mcpNativeHandler(mgr, cfg, REMOTE_TOOL), + spec: { + name: LOCAL_TOOL, + description: 'creates an invoice', + input_schema: { type: 'object', properties: { amount: { type: 'number' } } }, + }, + domain: 'test.odoo', + ...(options.declareWrite ? { writeCapabilities: CREATE_INVOICE } : {}), + }); + return new ToolDispatchService({ + nativeTools, + domainTools: [], + ...(options.store !== undefined ? { idempotency: options.store } : {}), + }); + } + + it('CONTROL — without an idempotency key the lost response duplicates the write', async (t) => { + const writes: string[] = []; + const first = await startWriteServer(t, writes); + if (!first) return; + const second = await startWriteServer(t, writes); + if (!second) return; + proxy = await startLosingProxy([first, second]); + + manager = new McpManager(); + const dispatcher = localDispatcher(manager, serverConfig(proxy.url), { + declareWrite: true, + }); + + await dispatcher.dispatch(LOCAL_TOOL, { amount: 100 }); + + // This is the hazard, demonstrated rather than asserted in prose: the server + // executed the write, the response was lost, the retry executed it AGAIN. + assert.equal(proxy.toolCallCount(), 2, 'the shipped once-retry must still fire here'); + assert.equal( + writes.length, + 2, + 'baseline: the write really does happen twice when the response is lost', + ); + }); + + it('executes the write EXACTLY ONCE under an idempotency key, despite the lost response', async (t) => { + const writes: string[] = []; + const first = await startWriteServer(t, writes); + if (!first) return; + const second = await startWriteServer(t, writes); + if (!second) return; + proxy = await startLosingProxy([first, second]); + + manager = new McpManager(); + const dispatcher = localDispatcher(manager, serverConfig(proxy.url), { + declareWrite: true, + store: new ToolIdempotencyStore(), + }); + + const result = await dispatcher.dispatch( + LOCAL_TOOL, + { amount: 100 }, + { idempotencyKey: 'req-invoice-1' }, + ); + + // THE load-bearing assertion: one real side effect on the server. + assert.equal( + writes.length, + 1, + 'the write executed more than once — duplicate customer data is exactly what this prevents', + ); + assert.equal( + proxy.toolCallCount(), + 1, + 'a write-capable call under an exactly-once scope must make a single attempt', + ); + // The caller still learns it failed — at-most-once means the caller may have + // to ask again with the same key, not that failure is hidden. `McpManager` + // never throws, it returns the failure as `Error: …` TEXT, so this surfaces as + // content rather than an `isError` flag. + assert.match( + result.content, + /Error:/, + 'suppressing the retry must not silently report success', + ); + }); + + it('a READ tool keeps the once-retry mitigation intact', async (t) => { + const writes: string[] = []; + const first = await startWriteServer(t, writes); + if (!first) return; + const second = await startWriteServer(t, writes); + if (!second) return; + proxy = await startLosingProxy([first, second]); + + manager = new McpManager(); + // No `writeCapabilities` ⇒ read-only, so the flaky-proxy mitigation applies + // even with a key present. Removing the retry outright would regress this. + const dispatcher = localDispatcher(manager, serverConfig(proxy.url), { + declareWrite: false, + store: new ToolIdempotencyStore(), + }); + + const result = await dispatcher.dispatch( + LOCAL_TOOL, + { amount: 100 }, + { idempotencyKey: 'req-read-1' }, + ); + + assert.equal(proxy.toolCallCount(), 2, 'the read must still be retried once'); + assert.equal(result.isError, undefined, 'and the retry must succeed'); + assert.match(result.content, /invoice #2 created/); + }); + + it('a duplicate dispatch under the SAME key does not reach the server again', async (t) => { + const writes: string[] = []; + const url = await startWriteServer(t, writes); + if (!url) return; + + manager = new McpManager(); + const dispatcher = localDispatcher(manager, serverConfig(url), { + declareWrite: true, + store: new ToolIdempotencyStore(), + }); + + const a = await dispatcher.dispatch( + LOCAL_TOOL, + { amount: 100 }, + { idempotencyKey: 'req-invoice-2' }, + ); + const b = await dispatcher.dispatch( + LOCAL_TOOL, + { amount: 100 }, + { idempotencyKey: 'req-invoice-2' }, + ); + + assert.equal(writes.length, 1, 'the caller retry must not produce a second invoice'); + assert.equal(a.content, b.content, 'the retry must receive the original result'); + assert.match(a.content, /invoice #1 created/); + }); +}); diff --git a/middleware/test/toolIdempotency.test.ts b/middleware/test/toolIdempotency.test.ts new file mode 100644 index 00000000..c079ecb5 --- /dev/null +++ b/middleware/test/toolIdempotency.test.ts @@ -0,0 +1,317 @@ +import { strict as assert } from 'node:assert'; +import { describe, it } from 'node:test'; + +import { NativeToolRegistry } from '../packages/harness-orchestrator/src/nativeToolRegistry.js'; +import { ToolDispatchService } from '../packages/harness-orchestrator/src/toolDispatchService.js'; +import { + ToolIdempotencyStore, + currentIdempotencyScope, + fingerprintToolInput, +} from '../packages/harness-orchestrator/src/toolIdempotency.js'; +import type { WriteCapability } from '../packages/plugin-api/src/writeCapabilities.js'; +import { isWriteCapableTool } from '../packages/plugin-api/src/writeCapabilities.js'; + +/** + * #542 prerequisite — idempotency for write-capable tool dispatch. + * + * MUTATION-CHECK DISCIPLINE: the assertions count REAL EXECUTIONS of the + * underlying handler (a counter the handler itself increments), never mock + * invocation counts on a dedupe helper. A test that asserted "the store was + * consulted" would stay green over a store that always misses. + */ + +const CREATE_INVOICE: readonly WriteCapability[] = [ + { dataClass: 'odoo.invoice', operation: 'create' }, +]; + +/** A write-capable tool whose handler counts how many times it really ran. */ +function writeToolService(options?: { + readonly capabilities?: readonly WriteCapability[]; + readonly store?: ToolIdempotencyStore; + readonly failWith?: () => never; +}): { service: ToolDispatchService; executions: () => number } { + let executions = 0; + const nativeTools = new NativeToolRegistry(); + nativeTools.register('odoo_create_invoice', { + handler: async (input) => { + executions += 1; + options?.failWith?.(); + return `invoice-created:${JSON.stringify(input)}`; + }, + spec: { + name: 'odoo_create_invoice', + description: 'creates an invoice — mutates data', + input_schema: { type: 'object', properties: {} }, + }, + domain: 'test.odoo', + ...(options?.capabilities !== undefined + ? { writeCapabilities: options.capabilities } + : {}), + }); + return { + service: new ToolDispatchService({ + nativeTools, + domainTools: [], + ...(options?.store !== undefined ? { idempotency: options.store } : {}), + }), + executions: () => executions, + }; +} + +describe('write-capability declaration', () => { + it('treats a tool with declared write capabilities as write-capable', () => { + assert.equal(isWriteCapableTool(CREATE_INVOICE), true); + }); + + it('treats an unannotated or empty declaration as read-only', () => { + assert.equal(isWriteCapableTool(undefined), false); + assert.equal(isWriteCapableTool([]), false); + }); + + it('surfaces the declaration through the registry onto the dispatcher', () => { + const { service } = writeToolService({ capabilities: CREATE_INVOICE }); + assert.equal(service.isWriteCapable('odoo_create_invoice'), true); + assert.equal(service.isWriteCapable('nope'), false); + }); +}); + +describe('ToolIdempotencyStore', () => { + it('executes once and REPLAYS the stored result for a duplicate key', async () => { + const store = new ToolIdempotencyStore(); + let runs = 0; + const exec = async () => { + runs += 1; + return { content: `run-${String(runs)}` }; + }; + + const a = await store.run('k1', 'tool', { x: 1 }, exec); + const b = await store.run('k1', 'tool', { x: 1 }, exec); + + assert.equal(runs, 1, 'the executor must run exactly once'); + assert.equal(a.result.content, 'run-1'); + assert.equal(b.result.content, 'run-1', 'the duplicate must see the FIRST result'); + assert.equal(b.replayed, true); + }); + + it('COLLAPSES concurrent duplicates onto one execution', async () => { + const store = new ToolIdempotencyStore(); + let runs = 0; + let release: (() => void) | undefined; + const gate = new Promise((resolve) => { + release = resolve; + }); + const exec = async () => { + runs += 1; + await gate; + return { content: 'once' }; + }; + + const both = Promise.all([ + store.run('k1', 'tool', { x: 1 }, exec), + store.run('k1', 'tool', { x: 1 }, exec), + ]); + release?.(); + const [a, b] = await both; + + assert.equal(runs, 1, 'a concurrent duplicate must not start a second execution'); + assert.equal(a.result.content, 'once'); + assert.equal(b.result.content, 'once'); + }); + + it('REJECTS a reused key carrying a different payload instead of executing', async () => { + const store = new ToolIdempotencyStore(); + let runs = 0; + const exec = async () => { + runs += 1; + return { content: 'ok' }; + }; + + await store.run('k1', 'tool', { amount: 100 }, exec); + const conflict = await store.run('k1', 'tool', { amount: 999 }, exec); + + assert.equal(runs, 1, 'a conflicting payload must NOT execute'); + assert.equal(conflict.result.isError, true); + assert.match(conflict.result.content, /idempotency key reused/); + }); + + it('treats key-equal payloads with reordered object keys as the SAME call', async () => { + assert.equal( + fingerprintToolInput({ a: 1, b: 2 }), + fingerprintToolInput({ b: 2, a: 1 }), + 'key order must not change the fingerprint, or a benign re-serialisation looks like a conflict', + ); + const store = new ToolIdempotencyStore(); + let runs = 0; + const exec = async () => { + runs += 1; + return { content: 'ok' }; + }; + await store.run('k1', 'tool', { a: 1, b: 2 }, exec); + await store.run('k1', 'tool', { b: 2, a: 1 }, exec); + assert.equal(runs, 1); + }); + + it('re-executes after the TTL window expires (bounded, not permanent)', async () => { + let now = 1_000; + const store = new ToolIdempotencyStore({ ttlMs: 500, now: () => now }); + let runs = 0; + const exec = async () => { + runs += 1; + return { content: `run-${String(runs)}` }; + }; + + await store.run('k1', 'tool', {}, exec); + now += 499; + await store.run('k1', 'tool', {}, exec); + assert.equal(runs, 1, 'still inside the window — must replay'); + + now += 2; + const after = await store.run('k1', 'tool', {}, exec); + assert.equal(runs, 2, 'past the window — must execute again'); + assert.equal(after.result.content, 'run-2'); + }); + + it('does NOT retain an isError outcome, so a caller may legitimately retry', async () => { + const store = new ToolIdempotencyStore(); + let runs = 0; + const exec = async () => { + runs += 1; + return runs === 1 + ? { content: 'Error: downstream refused', isError: true } + : { content: 'ok' }; + }; + + const first = await store.run('k1', 'tool', {}, exec); + assert.equal(first.result.isError, true); + const second = await store.run('k1', 'tool', {}, exec); + + assert.equal(runs, 2, 'a failed call must not be cached as the final answer'); + assert.equal(second.result.content, 'ok'); + }); + + it('does NOT retain a thrown outcome', async () => { + const store = new ToolIdempotencyStore(); + let runs = 0; + const exec = async (): Promise<{ content: string }> => { + runs += 1; + if (runs === 1) throw new Error('boom'); + return { content: 'ok' }; + }; + + await assert.rejects(() => store.run('k1', 'tool', {}, exec), /boom/); + const second = await store.run('k1', 'tool', {}, exec); + + assert.equal(runs, 2); + assert.equal(second.result.content, 'ok'); + assert.equal(store.size(), 1, 'the rejected entry must not linger alongside the good one'); + }); + + it('bounds retained records', async () => { + const store = new ToolIdempotencyStore({ maxEntries: 3 }); + for (let i = 0; i < 10; i += 1) { + await store.run(`k${String(i)}`, 'tool', {}, async () => ({ content: 'ok' })); + } + assert.equal(store.size(), 3); + }); +}); + +describe('ToolDispatchService — idempotent write dispatch', () => { + it('executes a write tool ONCE across duplicate dispatches sharing a key', async () => { + const store = new ToolIdempotencyStore(); + const { service, executions } = writeToolService({ + capabilities: CREATE_INVOICE, + store, + }); + + const a = await service.dispatch( + 'odoo_create_invoice', + { amount: 100 }, + { idempotencyKey: 'req-1' }, + ); + const b = await service.dispatch( + 'odoo_create_invoice', + { amount: 100 }, + { idempotencyKey: 'req-1' }, + ); + + assert.equal(executions(), 1, 'the write executed twice — duplicate customer data'); + assert.equal(a.content, b.content); + }); + + it('executes a write tool TWICE under DIFFERENT keys (dedupe is per key, not per tool)', async () => { + const store = new ToolIdempotencyStore(); + const { service, executions } = writeToolService({ + capabilities: CREATE_INVOICE, + store, + }); + + await service.dispatch('odoo_create_invoice', { amount: 1 }, { idempotencyKey: 'req-1' }); + await service.dispatch('odoo_create_invoice', { amount: 2 }, { idempotencyKey: 'req-2' }); + + assert.equal(executions(), 2, 'two distinct requests must both run'); + }); + + it('does NOT dedupe a READ tool — a cached read would serve stale data', async () => { + const store = new ToolIdempotencyStore(); + // Same tool, no write-capability declaration ⇒ read-only. + const { service, executions } = writeToolService({ store }); + + await service.dispatch('odoo_create_invoice', {}, { idempotencyKey: 'req-1' }); + await service.dispatch('odoo_create_invoice', {}, { idempotencyKey: 'req-1' }); + + assert.equal(executions(), 2, 'a read tool must not be deduplicated'); + }); + + it('is INERT without a store (legacy behaviour preserved)', async () => { + const { service, executions } = writeToolService({ capabilities: CREATE_INVOICE }); + + await service.dispatch('odoo_create_invoice', {}, { idempotencyKey: 'req-1' }); + await service.dispatch('odoo_create_invoice', {}, { idempotencyKey: 'req-1' }); + + assert.equal(executions(), 2); + }); + + it('publishes an exactlyOnce scope to layers beneath the handler — for writes only', async () => { + const store = new ToolIdempotencyStore(); + const seen: Array<{ key: string; exactlyOnce: boolean } | undefined> = []; + const nativeTools = new NativeToolRegistry(); + const record = async (): Promise => { + const scope = currentIdempotencyScope(); + seen.push( + scope === undefined + ? undefined + : { key: scope.key, exactlyOnce: scope.exactlyOnce }, + ); + return 'ok'; + }; + nativeTools.register('write_tool', { + handler: record, + spec: { + name: 'write_tool', + description: 'd', + input_schema: { type: 'object', properties: {} }, + }, + domain: 'test.x', + writeCapabilities: CREATE_INVOICE, + }); + nativeTools.register('read_tool', { + handler: record, + spec: { + name: 'read_tool', + description: 'd', + input_schema: { type: 'object', properties: {} }, + }, + domain: 'test.x', + }); + const service = new ToolDispatchService({ + nativeTools, + domainTools: [], + idempotency: store, + }); + + await service.dispatch('write_tool', {}, { idempotencyKey: 'req-1' }); + await service.dispatch('read_tool', {}, { idempotencyKey: 'req-2' }); + + assert.deepEqual(seen, [{ key: 'req-1', exactlyOnce: true }, undefined]); + }); +}); From 7be34439f4f7c0f11d1281820103bec7e2558bdc Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 12:52:04 +0200 Subject: [PATCH 39/90] fix(orchestrator): make the tool-dispatch and MCP timeout bounds coherent MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `OMADIA_TOOL_DISPATCH_TIMEOUT_MS` defaulted to 120 s, which sits INSIDE the MCP call ceiling (60 s idle budget per request, 180 s absolute via `OMADIA_MCP_CALL_MAX_TOTAL_TIMEOUT_MS`). The outer schranke was therefore tighter than the inner one: an MCP-backed sub-agent legitimately streaming progress notifications for its full 180 s allowance was aborted by the per-tool dispatch deadline first, and the model saw a generic dispatch-deadline error instead of the MCP layer's own diagnosis — with no `mcp_call_log` failure row naming the slow server. - Raise the dispatch default to 240 s so the outer bound is genuinely looser. - Document the three-level ordering next to BOTH defaults, each pointing at the other and at the guard test. - Export `resolveToolDispatchTimeoutMs` and a new `resolveMcpCallTimeouts` so the invariant is asserted against the real resolvers (env overrides included) rather than copies of the literals; `callTool` now uses the same helper. - Add `test/orchestrator/timeoutHierarchy.test.ts` asserting `dispatchDeadline > mcpAbsoluteCeiling > mcpRequestBudget`, plus the shipped numbers so lowering both together is not silently green. Mutation-checked in both directions: restoring the 120 s dispatch default and independently raising the MCP ceiling to 300 s each turn the ordering assertion red with the exact diagnosis. --- .../harness-orchestrator/src/index.ts | 11 ++- .../harness-orchestrator/src/mcp/mcpClient.ts | 49 +++++++-- .../harness-orchestrator/src/orchestrator.ts | 26 ++++- .../src/tasks/longRunningTool.ts | 4 +- .../orchestrator/timeoutHierarchy.test.ts | 99 +++++++++++++++++++ .../orchestrator/toolDispatchDeadline.test.ts | 4 +- 6 files changed, 174 insertions(+), 19 deletions(-) create mode 100644 middleware/test/orchestrator/timeoutHierarchy.test.ts diff --git a/middleware/packages/harness-orchestrator/src/index.ts b/middleware/packages/harness-orchestrator/src/index.ts index a31faf0b..f2650e30 100644 --- a/middleware/packages/harness-orchestrator/src/index.ts +++ b/middleware/packages/harness-orchestrator/src/index.ts @@ -139,6 +139,9 @@ export { mcpToolToLocalSubAgentTool, mcpToolToNativeSpec, renderToolResult, + // W3-A — the inner half of the timeout hierarchy; see the ORDERING INVARIANT + // block next to `DEFAULT_MCP_CALL_MAX_TOTAL_TIMEOUT_MS`. + resolveMcpCallTimeouts, } from './mcp/mcpClient.js'; export type { DeprecatedMcpTransport, @@ -223,7 +226,13 @@ export type { } from './buildOrchestrator.js'; // Orchestrator class + options -export { Orchestrator, parseToolEmittedChoice } from './orchestrator.js'; +export { + Orchestrator, + parseToolEmittedChoice, + // W3-A — the outer half of the timeout hierarchy; see the ORDERING INVARIANT + // block next to `DEFAULT_TOOL_DISPATCH_TIMEOUT_MS`. + resolveToolDispatchTimeoutMs, +} from './orchestrator.js'; export type { OrchestratorOptions } from './orchestrator.js'; // #332 Layer 2 — Direct Line directive parsing & target resolution (exported diff --git a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts index bfa04934..23f5d4d1 100644 --- a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts +++ b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts @@ -393,8 +393,18 @@ const CLIENT_INFO = { name: 'omadia-agent-builder', version: '0.1.0' } as const; * its budget alive (long Odoo/Confluence reports do exactly this)… * - `maxTotalTimeout`: …but never past this absolute ceiling, so a chatty * server cannot extend a call forever. - * Both are env-tunable per deployment; the orchestrator's own per-tool dispatch - * deadline (`OMADIA_TOOL_DISPATCH_TIMEOUT_MS`) is the outer bound. + * Both are env-tunable per deployment. + * + * ── ORDERING INVARIANT (W3-A) ─────────────────────────────────────────────── + * These are the INNER bounds. The orchestrator's per-tool dispatch deadline + * (`OMADIA_TOOL_DISPATCH_TIMEOUT_MS`, see `DEFAULT_TOOL_DISPATCH_TIMEOUT_MS` in + * `orchestrator.ts`) is the OUTER bound and must stay strictly LOOSER than + * `maxTotalTimeout` here. It used to default to 120 s — i.e. INSIDE this 180 s + * ceiling — so an MCP-backed sub-agent legitimately streaming progress for its + * full allowance was killed by the outer bound first, and the model saw a + * generic dispatch-deadline error instead of the MCP layer's own diagnosis. + * `test/orchestrator/timeoutHierarchy.test.ts` fails loudly if a future edit to + * either knob re-creates the inversion. */ const DEFAULT_MCP_CALL_TIMEOUT_MS = 60_000; const DEFAULT_MCP_CALL_MAX_TOTAL_TIMEOUT_MS = 180_000; @@ -407,6 +417,25 @@ function envMs(name: string, fallback: number): number { return parsed; } +/** + * The MCP request policy as it would be applied to the NEXT `callTool` — the + * same resolution `callTool` performs, exposed so the timeout-hierarchy + * invariant can be asserted against the real numbers (including env overrides) + * rather than against a copy of the defaults. + */ +export function resolveMcpCallTimeouts(): { + readonly timeoutMs: number; + readonly maxTotalTimeoutMs: number; +} { + return { + timeoutMs: envMs('OMADIA_MCP_CALL_TIMEOUT_MS', DEFAULT_MCP_CALL_TIMEOUT_MS), + maxTotalTimeoutMs: envMs( + 'OMADIA_MCP_CALL_MAX_TOTAL_TIMEOUT_MS', + DEFAULT_MCP_CALL_MAX_TOTAL_TIMEOUT_MS, + ), + }; +} + export class McpManager { private readonly pool = new Map(); private readonly connecting = new Map>(); @@ -708,14 +737,14 @@ export class McpManager { LENIENT_CALL_TOOL_RESULT_SCHEMA, // Stated request policy instead of the SDK's implicit 60s default — // see DEFAULT_MCP_CALL_TIMEOUT_MS. - { - timeout: envMs('OMADIA_MCP_CALL_TIMEOUT_MS', DEFAULT_MCP_CALL_TIMEOUT_MS), - resetTimeoutOnProgress: true, - maxTotalTimeout: envMs( - 'OMADIA_MCP_CALL_MAX_TOTAL_TIMEOUT_MS', - DEFAULT_MCP_CALL_MAX_TOTAL_TIMEOUT_MS, - ), - }, + (() => { + const policy = resolveMcpCallTimeouts(); + return { + timeout: policy.timeoutMs, + resetTimeoutOnProgress: true, + maxTotalTimeout: policy.maxTotalTimeoutMs, + }; + })(), ); const rendered = renderToolResult(res); // MCP protocol errors resolve (isError result) instead of throwing — diff --git a/middleware/packages/harness-orchestrator/src/orchestrator.ts b/middleware/packages/harness-orchestrator/src/orchestrator.ts index 26fb0752..eb16e226 100644 --- a/middleware/packages/harness-orchestrator/src/orchestrator.ts +++ b/middleware/packages/harness-orchestrator/src/orchestrator.ts @@ -1386,17 +1386,35 @@ function mcpObservationDigest(raw: string): string { * single sub-agent that never returns used to pin the WHOLE parallel batch for * the rest of the turn — there was no per-tool timeout anywhere. * - * 120s is deliberately generous: a domain sub-agent runs its own multi-iteration + * 240s is deliberately generous: a domain sub-agent runs its own multi-iteration * LLM loop with its own tool calls, so p99 legitimately reaches tens of seconds. * Operators whose Odoo/Confluence sub-agents run longer raise it via * `OMADIA_TOOL_DISPATCH_TIMEOUT_MS`; `0` disables the deadline entirely. + * + * ── ORDERING INVARIANT (W3-A) ─────────────────────────────────────────────── + * This is the OUTER bound. It must stay strictly LOOSER than the innermost MCP + * bound — `OMADIA_MCP_CALL_MAX_TOTAL_TIMEOUT_MS` (default 180 s, see + * `DEFAULT_MCP_CALL_MAX_TOTAL_TIMEOUT_MS` in `mcp/mcpClient.ts`), which itself + * sits above the 60 s per-request idle budget. + * + * The default was 120 s, i.e. INSIDE the 180 s MCP ceiling. An MCP-backed + * sub-agent legitimately streaming progress notifications for its full + * allowance was therefore killed by the OUTER bound first: the tighter schranke + * was the outer one, which is backwards, and the model got a generic + * dispatch-deadline error instead of the MCP layer's own diagnosis (which the + * audit trail records as an `fail`/`timeout` row against the server). + * + * `test/orchestrator/timeoutHierarchy.test.ts` asserts + * `dispatchDeadline > mcpAbsoluteCeiling`, so a future edit to EITHER knob fails + * loudly rather than silently re-creating the inversion. */ -const DEFAULT_TOOL_DISPATCH_TIMEOUT_MS = 120_000; +const DEFAULT_TOOL_DISPATCH_TIMEOUT_MS = 240_000; const TOOL_DISPATCH_TIMEOUT_ENV = 'OMADIA_TOOL_DISPATCH_TIMEOUT_MS'; /** Resolved per dispatch (not cached at module load) so an operator env change - * applies to the next turn without a restart. */ -function resolveToolDispatchTimeoutMs(): number { + * applies to the next turn without a restart. Exported so the timeout-hierarchy + * invariant test reads the REAL resolved value, env overrides included. */ +export function resolveToolDispatchTimeoutMs(): number { const raw = process.env[TOOL_DISPATCH_TIMEOUT_ENV]; if (raw === undefined || raw.trim() === '') { return DEFAULT_TOOL_DISPATCH_TIMEOUT_MS; diff --git a/middleware/packages/harness-orchestrator/src/tasks/longRunningTool.ts b/middleware/packages/harness-orchestrator/src/tasks/longRunningTool.ts index 0bae7a21..5b7a586c 100644 --- a/middleware/packages/harness-orchestrator/src/tasks/longRunningTool.ts +++ b/middleware/packages/harness-orchestrator/src/tasks/longRunningTool.ts @@ -23,10 +23,10 @@ * * ## Interaction with the per-tool dispatch deadline * - * A separate unit adds `OMADIA_TOOL_DISPATCH_TIMEOUT_MS` (default 120 s) around + * A separate unit adds `OMADIA_TOOL_DISPATCH_TIMEOUT_MS` (default 240 s) around * tool dispatch. It does not interact with this path in any harmful way, BY * CONSTRUCTION: every handler here is bounded by a store round-trip, so - * `_start` returns in milliseconds and can never approach a 120 s deadline. The + * `_start` returns in milliseconds and can never approach that deadline. The * long work runs in a DETACHED runner (see `startRunner`) that is not inside the * dispatch call at all, so the deadline has nothing to cancel. That is the point * of the seam — a deadline and a long-running tool stop being in conflict once diff --git a/middleware/test/orchestrator/timeoutHierarchy.test.ts b/middleware/test/orchestrator/timeoutHierarchy.test.ts new file mode 100644 index 00000000..ac027807 --- /dev/null +++ b/middleware/test/orchestrator/timeoutHierarchy.test.ts @@ -0,0 +1,99 @@ +/** + * W3-A — the two tool-timeout knobs must stay COHERENT. + * + * There are three nested bounds around one MCP-backed tool dispatch: + * + * inner `OMADIA_MCP_CALL_TIMEOUT_MS` 60 s idle budget / request + * middle `OMADIA_MCP_CALL_MAX_TOTAL_TIMEOUT_MS` 180 s absolute MCP ceiling + * outer `OMADIA_TOOL_DISPATCH_TIMEOUT_MS` 240 s per-tool dispatch + * + * The dispatch deadline defaulted to 120 s, i.e. INSIDE the 180 s MCP ceiling. + * A server streaming progress notifications (`resetTimeoutOnProgress`) for its + * full allowance was therefore aborted by the OUTER bound first — the outer + * schranke was tighter than the inner one, which is backwards. The model then + * saw a generic dispatch-deadline error rather than the MCP layer's own + * diagnosis, and no `mcp_call_log` failure row named the slow server. + * + * These assertions read the REAL resolvers (not copies of the literals), so an + * env override that re-creates the inversion is caught too. + */ +import { afterEach, describe, it } from 'node:test'; +import { strict as assert } from 'node:assert'; + +import { + resolveMcpCallTimeouts, + resolveToolDispatchTimeoutMs, +} from '@omadia/orchestrator'; + +const DISPATCH_ENV = 'OMADIA_TOOL_DISPATCH_TIMEOUT_MS'; +const MCP_TOTAL_ENV = 'OMADIA_MCP_CALL_MAX_TOTAL_TIMEOUT_MS'; +const MCP_REQUEST_ENV = 'OMADIA_MCP_CALL_TIMEOUT_MS'; + +const originals = { + [DISPATCH_ENV]: process.env[DISPATCH_ENV], + [MCP_TOTAL_ENV]: process.env[MCP_TOTAL_ENV], + [MCP_REQUEST_ENV]: process.env[MCP_REQUEST_ENV], +}; + +afterEach(() => { + for (const [name, value] of Object.entries(originals)) { + if (value === undefined) delete process.env[name]; + else process.env[name] = value; + } +}); + +/** The invariant itself, so both the default case and the override case assert + * the SAME rule rather than two hand-mirrored copies of it. */ +function assertOuterIsLooser(): void { + const dispatchDeadlineMs = resolveToolDispatchTimeoutMs(); + const { timeoutMs, maxTotalTimeoutMs } = resolveMcpCallTimeouts(); + // `0` means "no dispatch deadline", which is looser than any finite ceiling. + if (dispatchDeadlineMs !== 0) { + assert.ok( + dispatchDeadlineMs > maxTotalTimeoutMs, + `the OUTER tool-dispatch deadline (${String(dispatchDeadlineMs)}ms) must be strictly ` + + `looser than the absolute MCP ceiling (${String(maxTotalTimeoutMs)}ms) — otherwise an ` + + `MCP call that legitimately uses its full allowance is killed by the outer bound first`, + ); + } + assert.ok( + maxTotalTimeoutMs > timeoutMs, + `the absolute MCP ceiling (${String(maxTotalTimeoutMs)}ms) must be looser than the ` + + `per-request idle budget (${String(timeoutMs)}ms)`, + ); +} + +describe('tool-timeout hierarchy (W3-A)', () => { + it('MUTATION CHECK: the shipped defaults order outer > mcp-absolute > mcp-request', () => { + for (const name of [DISPATCH_ENV, MCP_TOTAL_ENV, MCP_REQUEST_ENV]) { + delete process.env[name]; + } + assertOuterIsLooser(); + // Pin the actual shipped numbers too: the ordering assertion alone would + // stay green if BOTH knobs were lowered together, which would silently + // shrink the allowance every long-running Odoo/Confluence report depends on. + assert.equal(resolveToolDispatchTimeoutMs(), 240_000); + assert.equal(resolveMcpCallTimeouts().maxTotalTimeoutMs, 180_000); + assert.equal(resolveMcpCallTimeouts().timeoutMs, 60_000); + }); + + it('MUTATION CHECK: an operator raising the MCP ceiling past the dispatch deadline is caught', () => { + // The failure mode this guard exists for, reproduced through the env knobs: + // raising only the inner ceiling re-creates the inversion. + process.env[MCP_TOTAL_ENV] = '300000'; + assert.throws( + () => assertOuterIsLooser(), + /must be strictly looser than the absolute MCP ceiling/, + 'raising the MCP ceiling above the dispatch deadline was not rejected', + ); + // …and raising the outer bound with it restores coherence. + process.env[DISPATCH_ENV] = '360000'; + assertOuterIsLooser(); + }); + + it('a disabled dispatch deadline (0) is treated as looser than any ceiling', () => { + process.env[DISPATCH_ENV] = '0'; + assert.equal(resolveToolDispatchTimeoutMs(), 0); + assertOuterIsLooser(); + }); +}); diff --git a/middleware/test/orchestrator/toolDispatchDeadline.test.ts b/middleware/test/orchestrator/toolDispatchDeadline.test.ts index a6277a60..88c403aa 100644 --- a/middleware/test/orchestrator/toolDispatchDeadline.test.ts +++ b/middleware/test/orchestrator/toolDispatchDeadline.test.ts @@ -349,7 +349,7 @@ describe('Orchestrator per-tool dispatch deadline (W0-2)', () => { ); }); - it('falls back to the 120s default when the env value is not a number', async () => { + it('falls back to the 240s default when the env value is not a number', async () => { process.env['OMADIA_TOOL_DISPATCH_TIMEOUT_MS'] = 'not-a-number'; const probe = slowDomainTool('query_slow_agent', 20); const orchestrator = buildOrchestrator( @@ -363,7 +363,7 @@ describe('Orchestrator per-tool dispatch deadline (W0-2)', () => { const result = await orchestrator.runTurn({ userMessage: 'go' }); // A bad env value must not degrade into "no deadline" or "0ms deadline": - // the tool completes normally well inside the 120s default. + // the tool completes normally well inside the 240s default. assert.equal(result.answer, 'answered'); assert.equal(probe.settledLate(), true); }); From 98e9e01aade0aa20a865abd17512091274ec938e Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 12:55:49 +0200 Subject: [PATCH 40/90] test(embeddings): stop the gate-fence fixture from reaching into public MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `embeddingGateWriteFence.pg.test.ts` failed on every with-pg run after the first. Not the repo's known flake pattern (which produces disjoint failure sets) — a deterministic fixture bug, and the FK that surfaced it was luck. `freshSchema()` ran `DROP TABLE IF EXISTS graph_nodes, processes, process_history, graph_embedding_model` UNQUALIFIED. A DROP resolves an unqualified name through the search_path, and this suite is the only one in the cluster whose pool has `public` on it (it needs `public.vector` for the bare `::vector` casts the writers emit). On the first test its own schema is still empty, so every name fell through to `public`: - run 1, pristine database: `public.graph_nodes` does not exist yet → no-op → suite green. - run 2+: the real KG suites have since created `public.graph_nodes` and `public.graph_edges`, which survive in the container. The DROP now hits `public.graph_nodes` and errors 2BP01 on `graph_edges`' foreign keys, so the CREATE never runs, so the next test's DROP falls through again — self-perpetuating across all six tests. The FK is the ONLY reason this was loud. `public.processes` and `public.graph_embedding_model` have no dependents, so the same fall-through was silently dropping sibling suites' tables. - `freshSchema` now recreates the SCHEMA instead of enumerating tables, so it can never name an object outside its own namespace. - All DDL, DML and assertion reads are schema-qualified; an unqualified READ would not error, it would quietly assert against another suite's rows. - Added `assertFixtureIsIsolated()` after every fixture build, turning a future search_path regression into an immediate named failure. `embeddingGateReevaluation`, `embeddingModelGateMigration` and `embeddingModelGateMigrationGuards` use `search_path=` with no `public`, so they cannot fall through and are left unchanged. Mutation check: restoring only the unqualified DROP against the same dirty container reproduces 6/6 failures with the identical 2BP01 error; the fix takes it back to 6/6 green. --- .../test/embeddingGateWriteFence.pg.test.ts | 63 ++++++++++++++----- 1 file changed, 49 insertions(+), 14 deletions(-) diff --git a/middleware/test/embeddingGateWriteFence.pg.test.ts b/middleware/test/embeddingGateWriteFence.pg.test.ts index 780f9705..ac95364e 100644 --- a/middleware/test/embeddingGateWriteFence.pg.test.ts +++ b/middleware/test/embeddingGateWriteFence.pg.test.ts @@ -160,11 +160,25 @@ describe('#440 gate-epoch write fence (real Postgres, real backfill handle)', { pendingTurns?: number; pendingProcesses?: number; }): Promise { - await real.query( - 'DROP TABLE IF EXISTS graph_nodes, processes, process_history, graph_embedding_model', - ); + // Recreate the SCHEMA, and qualify every name below — never `DROP TABLE IF + // EXISTS graph_nodes` unqualified. + // + // A DROP resolves an unqualified name THROUGH the search_path. On the first + // test this suite's own schema is still empty, so `graph_nodes` fell through + // to `public.graph_nodes` — a table the real KG suites create and which + // survives in the container between runs. That is why this suite passed on a + // pristine database and then failed on EVERY subsequent with-pg run: the + // second run's DROP hit `public.graph_nodes` and errored 2BP01 on + // `public.graph_edges`' foreign keys, so the CREATE never ran, so the next + // test's DROP fell through again — self-perpetuating, all six tests. + // + // The FK is the only reason this surfaced as a loud error. `public.processes` + // and `public.graph_embedding_model` have no dependents, so the same + // fall-through was SILENTLY DROPPING a sibling suite's tables. + await real.query(`DROP SCHEMA IF EXISTS ${SCHEMA} CASCADE`); + await real.query(`CREATE SCHEMA ${SCHEMA}`); await real.query(` - CREATE TABLE graph_nodes ( + CREATE TABLE ${SCHEMA}.graph_nodes ( id TEXT PRIMARY KEY, tenant_id TEXT NOT NULL, type TEXT NOT NULL, @@ -177,7 +191,7 @@ describe('#440 gate-epoch write fence (real Postgres, real backfill handle)', { created_at TIMESTAMPTZ NOT NULL DEFAULT now() )`); await real.query(` - CREATE TABLE processes ( + CREATE TABLE ${SCHEMA}.processes ( id TEXT NOT NULL, tenant_id TEXT NOT NULL, scope TEXT NOT NULL DEFAULT 'team', @@ -191,7 +205,7 @@ describe('#440 gate-epoch write fence (real Postgres, real backfill handle)', { PRIMARY KEY (tenant_id, id) )`); await real.query(` - CREATE TABLE process_history ( + CREATE TABLE ${SCHEMA}.process_history ( id TEXT NOT NULL, tenant_id TEXT NOT NULL, version INTEGER NOT NULL, @@ -201,7 +215,7 @@ describe('#440 gate-epoch write fence (real Postgres, real backfill handle)', { superseded_at TIMESTAMPTZ NOT NULL )`); await real.query(` - CREATE TABLE graph_embedding_model ( + CREATE TABLE ${SCHEMA}.graph_embedding_model ( tenant_id TEXT PRIMARY KEY, model_id TEXT NOT NULL, dimensions INTEGER NOT NULL, @@ -211,7 +225,7 @@ describe('#440 gate-epoch write fence (real Postgres, real backfill handle)', { )`); for (let i = 0; i < (opts.pendingTurns ?? 0); i++) { await real.query( - `INSERT INTO graph_nodes (id, tenant_id, type, external_id, properties) + `INSERT INTO ${SCHEMA}.graph_nodes (id, tenant_id, type, external_id, properties) VALUES ($1, $2, 'Turn', $3, $4::jsonb)`, [ `00000000-0000-4000-8000-00000000000${String(i)}`, @@ -223,34 +237,55 @@ describe('#440 gate-epoch write fence (real Postgres, real backfill handle)', { } for (let i = 0; i < (opts.pendingProcesses ?? 0); i++) { await real.query( - `INSERT INTO processes (id, tenant_id, title, steps) + `INSERT INTO ${SCHEMA}.processes (id, tenant_id, title, steps) VALUES ($1, $2, $3, $4::jsonb)`, [`proc:${String(i)}`, TENANT, `Backend: step ${String(i)}`, JSON.stringify(['do it'])], ); } // Three days old: the switch cooldown guards a rolling deploy, not a test. await real.query( - `INSERT INTO graph_embedding_model (tenant_id, model_id, dimensions, updated_at) + `INSERT INTO ${SCHEMA}.graph_embedding_model (tenant_id, model_id, dimensions, updated_at) VALUES ($1, 'ollama:nomic-embed-text', 768, now() - interval '3 days')`, [TENANT], ); + await assertFixtureIsIsolated(); } + // Assertions read SCHEMA-QUALIFIED names for the same reason the DDL writes + // them: an unqualified read that fell through to `public` would not error, it + // would quietly assert against a sibling suite's rows. const storedVectors = async (table: string): Promise => { const r = await real.query<{ v: string }>( - `SELECT embedding::text AS v FROM ${table} WHERE embedding IS NOT NULL`, + `SELECT embedding::text AS v FROM ${SCHEMA}.${table} WHERE embedding IS NOT NULL`, ); return r.rows.map((row) => row.v); }; const attemptCounters = async (): Promise => { const r = await real.query<{ n: number }>( - 'SELECT embedding_attempts AS n FROM graph_nodes WHERE tenant_id = $1', + `SELECT embedding_attempts AS n FROM ${SCHEMA}.graph_nodes WHERE tenant_id = $1`, [TENANT], ); return r.rows.map((row) => Number(row.n)); }; + /** + * Guard for the fixture itself: every table this suite drives must live in + * THIS suite's schema. Cheap, and it turns a silent cross-suite collision + * (the fall-through above) into an immediate, named failure. + */ + async function assertFixtureIsIsolated(): Promise { + const r = await real.query<{ tablename: string }>( + `SELECT tablename FROM pg_tables WHERE schemaname = $1 ORDER BY tablename`, + [SCHEMA], + ); + assert.deepEqual( + r.rows.map((row) => row.tablename), + ['graph_embedding_model', 'graph_nodes', 'process_history', 'processes'], + `the fixture did not land in ${SCHEMA} — it resolved through the search_path`, + ); + } + /** * The plugin's own wiring, verbatim in shape: ONE `syncBackfill` that stops * the outgoing handle and constructs a real replacement, handed to @@ -452,7 +487,7 @@ describe('#440 gate-epoch write fence (real Postgres, real backfill handle)', { // embedder for MemorableKnowledge / PalaiaExcerpt. Same shape, same window. await freshSchema({}); await real.query( - `INSERT INTO graph_nodes (id, tenant_id, type, external_id) + `INSERT INTO ${SCHEMA}.graph_nodes (id, tenant_id, type, external_id) VALUES ('11111111-0000-4000-8000-000000000000', $1, 'MemorableKnowledge', 'mk:1')`, [TENANT], ); @@ -598,7 +633,7 @@ describe('#440 gate-epoch write fence (real Postgres, real backfill handle)', { // it; the ROLLBACK makes the fenced write a clean no-op on both counts. await freshSchema({}); await real.query( - `INSERT INTO processes (id, tenant_id, title, steps, embedding) + `INSERT INTO ${SCHEMA}.processes (id, tenant_id, title, steps, embedding) VALUES ('proc:edit', $1, 'Backend: deploy to staging', $2::jsonb, NULL)`, [TENANT, JSON.stringify(['build'])], ); From 9cb374fd3be49f2a24fae3d04d6cad554534c40e Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 12:56:13 +0200 Subject: [PATCH 41/90] feat(mcp): add mcp:list / mcp:invoke / mcp:write: scopes and the public-MCP key binding migration MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per-tool write scopes are unreachable via WILDCARD_SCOPE — the exception lives inside hasScope itself so a parallel matcher cannot be forgotten. Migration 0033 adds public_mcp_key_bindings (allowlist per KEY, one agent per key) and widens mcp_call_log.caller_kind with 'api_key'. --- .../migrations/0033_public_mcp_keys.sql | 123 ++++++++++++++++++ .../harness-api-key-auth/src/apiKeyScopes.ts | 89 ++++++++++++- .../harness-api-key-auth/src/index.ts | 6 + 3 files changed, 216 insertions(+), 2 deletions(-) create mode 100644 middleware/migrations/0033_public_mcp_keys.sql diff --git a/middleware/migrations/0033_public_mcp_keys.sql b/middleware/migrations/0033_public_mcp_keys.sql new file mode 100644 index 00000000..c25cebc6 --- /dev/null +++ b/middleware/migrations/0033_public_mcp_keys.sql @@ -0,0 +1,123 @@ +-- ── W2-3: the public stateless MCP endpoint's per-key authorization ───────── +-- (issue #542) +-- +-- Correcting the issue body before anything else, because the schema below is +-- shaped by the correction: #542 claims "the delta is transport exposure + +-- auth, not new tool plumbing". That is false. `ToolDispatchService` (the +-- dispatcher the loopback MCP server already uses) carries an explicit SEAM +-- comment recording that kernel-tool branches, scoped-memory shadowing, +-- privacy interning and trace capture are deliberately NOT replicated versus +-- `Orchestrator.dispatchToolInner`, and dispatch carries no tenant, user or +-- principal at all. Closing that seam is a SIBLING unit; this migration +-- provides the authorization data the endpoint needs either way. +-- +-- Two things live here: +-- 1. `public_mcp_key_bindings` — the per-key allowlist and agent binding. +-- 2. a widened `mcp_call_log.caller_kind`, so a public MCP call is auditable +-- as what it actually is rather than mislabelled as one of the five +-- in-process caller kinds. + +-- ── 1. Per-key tool allowlist + agent binding ─────────────────────────────── +-- WHY A TABLE RATHER THAN MORE SCOPES ON THE KEY RECORD +-- +-- Scopes (`@omadia/api-key-auth`) answer "what class of thing may this key +-- do": list, invoke, write-this-tool. They are vault-resident, per key, and +-- deliberately free-form so plugins can mint their own. What they cannot +-- answer is "WHICH tools, on WHICH agent" — a set that an operator edits, that +-- wants to be inspectable in a query, and that must default to nothing. +-- +-- Both halves are required for a call to succeed, and they are checked +-- independently: +-- - the SCOPE says the key holds the capability; +-- - this ROW says the key reaches that specific tool on that specific agent. +-- Neither is sufficient. A key whose scopes say `mcp:write:create_lead` but +-- whose row does not list `create_lead` reaches nothing, and vice versa. That +-- redundancy is deliberate: the two live in different stores (vault vs. DB) +-- with different write paths, so a mistake in one is not a mistake in both. +-- +-- ALLOWLIST, NEVER DENYLIST. A key with no row here reaches ZERO tools — it +-- authenticates and is authorized for nothing. That is what makes +-- integration-backed and write-capable tools (Odoo, M365, Confluence) excluded +-- by DEFAULT: they are excluded because nothing is included until an operator +-- names it. A denylist would have made every future tool reachable the moment +-- it was registered, which is a privilege escalation delivered by a deploy. +-- +-- KEY → EXACTLY ONE AGENT. `agent_id` is scalar, not an array, and is the +-- primary-key-adjacent fact of this table. omadia had no seam for "which +-- agent's tools does this caller see" — the native tool registry is a process +-- -wide singleton with unique names, and per-agent scoping existed only for +-- DomainTools. This column IS that seam. A key bound to agent A cannot reach +-- agent B's tools even when both agents' tools sit in the same registry, +-- because the endpoint resolves the dispatcher from THIS column and filters to +-- THIS row's allowlist. +CREATE TABLE IF NOT EXISTS public_mcp_key_bindings ( + -- `ApiKeyRecord.id` from `@omadia/api-key-auth`. Not a foreign key: those + -- records live in the secret vault, not in Postgres, so the database cannot + -- enforce the reference. The endpoint verifies the key FIRST (constant-time + -- hash compare against the vault) and only then reads this row, so an + -- orphaned row grants nothing — it is unreachable without a live key whose + -- id matches. + key_id TEXT PRIMARY KEY, + + -- The ONE agent (orchestrator slug) whose tools this key reaches. + agent_id TEXT NOT NULL CHECK (length(agent_id) > 0), + + -- Exact tool names, no patterns. A pattern would reintroduce the "I thought + -- `odoo_*` didn't cover `odoo_delete_invoice`" mistake that per-tool scopes + -- exist to prevent, and would silently widen on every newly-registered tool. + read_tools TEXT[] NOT NULL DEFAULT '{}', + + -- Write-capable subset, named separately rather than inferred. omadia has no + -- per-tool "is this a write" metadata today: `DispatchableToolSpec` carries + -- name/description/input_schema and nothing about effects. Inferring from the + -- name ("does it start with create_/update_/delete_") would be a guess that + -- fails open on the first tool named `submit_expense`. So the operator + -- declares it, and a tool listed here additionally requires the key to hold + -- `mcp:write:` AND spends the tighter write rate-limit budget. + write_tools TEXT[] NOT NULL DEFAULT '{}', + + -- Separate, tighter budget than the key's general `rateLimitPerMinute`. + -- Reads are cheap and idempotent; a write is neither. Sharing one budget + -- would let a read-heavy integration's unused headroom fund a write burst. + write_rate_limit_per_minute INTEGER NOT NULL DEFAULT 5 + CHECK (write_rate_limit_per_minute BETWEEN 0 AND 600), + + -- 0 disables the binding without deleting it (and without revoking the key, + -- which may still be used for chat). Distinct from "no row": an operator can + -- see that this key WAS configured and is currently parked. + enabled BOOLEAN NOT NULL DEFAULT true, + + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + updated_at TIMESTAMPTZ NOT NULL DEFAULT now() +); + +-- Answers "which keys can reach this agent" without a sequential scan once an +-- install has more than a handful of integrations. +CREATE INDEX IF NOT EXISTS public_mcp_key_bindings_agent_idx + ON public_mcp_key_bindings (agent_id); + +-- ── 2. Audit a public MCP call as what it is ──────────────────────────────── +-- 0009 constrained `caller_kind` to the five IN-PROCESS caller kinds (agent, +-- subagent, skill, plugin, unattributed). A public MCP call is none of them: +-- there is no orchestrator turn, no sub-agent, no plugin — there is an API key +-- held by a third party. Squeezing it into `plugin` or `unattributed` would +-- make the one question this row exists to answer ("was this an internal turn +-- or the internet?") unanswerable from the data. +-- +-- `acting_identity` (added by 0031 for the confused-deputy fix) carries the +-- key: `apikey:`, or the literal `unresolved` when the identity could +-- not be established — the SAME vocabulary 0031 established, reused rather than +-- reinvented, so one operator query covers both sources. +-- +-- 0009 created the constraint inline, so Postgres named it +-- `mcp_call_log_caller_kind_check`. Drop whichever name is present and re-add +-- an explicitly named one, so a future migration has a stable handle. +ALTER TABLE mcp_call_log + DROP CONSTRAINT IF EXISTS mcp_call_log_caller_kind_check; +ALTER TABLE mcp_call_log + DROP CONSTRAINT IF EXISTS mcp_call_log_caller_kind_chk; +ALTER TABLE mcp_call_log + ADD CONSTRAINT mcp_call_log_caller_kind_chk + CHECK (caller_kind IN ('agent', 'subagent', 'skill', 'plugin', 'unattributed', 'api_key')); + +-- rollback: DELETE FROM mcp_call_log WHERE caller_kind = 'api_key'; ALTER TABLE mcp_call_log DROP CONSTRAINT mcp_call_log_caller_kind_chk, ADD CONSTRAINT mcp_call_log_caller_kind_check CHECK (caller_kind IN ('agent', 'subagent', 'skill', 'plugin', 'unattributed')); DROP TABLE public_mcp_key_bindings; diff --git a/middleware/packages/harness-api-key-auth/src/apiKeyScopes.ts b/middleware/packages/harness-api-key-auth/src/apiKeyScopes.ts index 4ce83105..3c630705 100644 --- a/middleware/packages/harness-api-key-auth/src/apiKeyScopes.ts +++ b/middleware/packages/harness-api-key-auth/src/apiKeyScopes.ts @@ -25,6 +25,48 @@ export const WILDCARD_SCOPE = '*'; /** The capability the public chat ingress requires (`@omadia/channel-api`). */ export const CHAT_WRITE_SCOPE = 'chat:write'; +/** W2-3 (issue #542) — enumerate the tools the public MCP endpoint exposes to + * this key. Seeing a tool name is itself a disclosure, so listing is its own + * capability rather than a free side effect of authenticating. */ +export const MCP_LIST_SCOPE = 'mcp:list'; + +/** W2-3 — call a READ tool over the public MCP endpoint. Deliberately NOT + * sufficient for a write: see `MCP_WRITE_SCOPE_PREFIX`. */ +export const MCP_INVOKE_SCOPE = 'mcp:invoke'; + +/** + * W2-3 — prefix of the per-tool write capability, `mcp:write:`. + * + * Marcel's decision to expose write tools (not just reads) over a PUBLIC + * endpoint is what makes this granularity a requirement rather than a nicety. + * Three properties hold, and each exists because the coarser alternative is a + * real escalation: + * + * - It is PER TOOL. `mcp:invoke` authorizes reads as a class; there is no + * equivalent class-wide write scope, because "this integration may write" + * is never the sentence an operator means — they mean "this integration may + * call `create_lead`", and nothing else. + * - It is NOT reachable via `WILDCARD_SCOPE`. `*` is a convenience for an + * operator's own tooling; silently including "delete every Odoo invoice via + * an internet-facing endpoint" in that convenience is not a trade anyone + * consciously makes. `hasScope` enforces this for every caller — see there. + * - It is THREE segments, so it cannot collide with, or be satisfied by, any + * two-segment scope an operator or plugin already minted. + */ +export const MCP_WRITE_SCOPE_PREFIX = 'mcp:write:'; + +/** Builds the write capability for one tool. Use this rather than + * concatenating, so the prefix has exactly one definition. */ +export function mcpWriteScope(toolName: string): ApiKeyScope { + return `${MCP_WRITE_SCOPE_PREFIX}${toolName}`; +} + +/** True for a `mcp:write:` scope. Drives the wildcard exclusion in + * `hasScope`, so it must stay a pure shape test with no allow-list. */ +export function isMcpWriteScope(scope: ApiKeyScope): boolean { + return scope.startsWith(MCP_WRITE_SCOPE_PREFIX); +} + /** * What a key with no persisted `scopes` field is treated as. * @@ -41,9 +83,27 @@ export const LEGACY_DEFAULT_SCOPES: readonly ApiKeyScope[] = [CHAT_WRITE_SCOPE]; /** `:`, lowercase, or the bare global wildcard. */ const SCOPE_PATTERN = /^[a-z][a-z0-9_-]*:[a-z][a-z0-9_-]*$/; +/** + * W2-3 — the ONLY three-segment shape admitted: `mcp:write:`. + * + * Written as a literal `mcp:write:` prefix rather than a generic + * `::` rule on purpose. A generic three-segment rule would quietly + * legalize every `foo:bar:baz` string an operator mistypes, and each such + * string would then be a scope that validates, persists, and grants nothing — + * indistinguishable from a revoked key at debug time. `` reuses the same + * character class the other segments use, so a tool name that cannot appear + * here cannot be granted at all (fail closed, not fail open). + */ +const MCP_WRITE_SCOPE_PATTERN = /^mcp:write:[a-z][a-z0-9_-]*$/; + export function isValidScope(value: unknown): value is ApiKeyScope { if (typeof value !== 'string') return false; - return value === WILDCARD_SCOPE || SCOPE_PATTERN.test(value); + if (value === WILDCARD_SCOPE) return true; + if (MCP_WRITE_SCOPE_PATTERN.test(value)) return true; + // Checked LAST and unchanged: a `mcp:write:x` string has two colons and + // never matched `SCOPE_PATTERN` anyway, so nothing that used to validate + // stops validating and nothing new slips through the two-segment rule. + return SCOPE_PATTERN.test(value); } /** Grants nothing. Every `hasScope` check against it is false. */ @@ -143,11 +203,36 @@ export function assertValidScopes(scopes: readonly unknown[]): readonly ApiKeySc return Array.from(new Set(scopes as readonly ApiKeyScope[])); } -/** True when `granted` covers `required` — exact match, or the global `*`. */ +/** + * True when `granted` covers `required` — exact match, or the global `*`. + * + * W2-3 carves ONE exception out of the wildcard: a `mcp:write:` scope is + * satisfied by an exact match and by nothing else. The exception lives HERE, + * inside the single scope-matching primitive, rather than in a second + * `hasWriteScope` function the public-MCP route is expected to remember to + * call. A parallel matcher is a matcher someone eventually forgets: the wrong + * call would still compile, still typecheck, and still return `true` for `*` — + * quietly granting an internet-facing write. There is one matcher, and it is + * correct for every caller including `requireApiKey`'s own `opts.scope` gate. + * + * `hasWriteScope` below exists only as an intention-revealing alias; it adds no + * behavior, so using the wrong one of the two is not a security event. + */ export function hasScope( granted: readonly ApiKeyScope[] | undefined, required: ApiKeyScope, ): boolean { if (!granted) return false; + if (isMcpWriteScope(required)) return granted.includes(required); return granted.includes(WILDCARD_SCOPE) || granted.includes(required); } + +/** True when `granted` explicitly names the write capability for `toolName`. + * Intention-revealing alias for `hasScope(granted, mcpWriteScope(tool))` — + * see the wildcard note on `hasScope`. */ +export function hasWriteScope( + granted: readonly ApiKeyScope[] | undefined, + toolName: string, +): boolean { + return hasScope(granted, mcpWriteScope(toolName)); +} diff --git a/middleware/packages/harness-api-key-auth/src/index.ts b/middleware/packages/harness-api-key-auth/src/index.ts index 45216852..e8d6dde9 100644 --- a/middleware/packages/harness-api-key-auth/src/index.ts +++ b/middleware/packages/harness-api-key-auth/src/index.ts @@ -22,8 +22,14 @@ export { CHAT_WRITE_SCOPE, DENY_ALL_SCOPES, hasScope, + hasWriteScope, + isMcpWriteScope, isValidScope, LEGACY_DEFAULT_SCOPES, + MCP_INVOKE_SCOPE, + MCP_LIST_SCOPE, + MCP_WRITE_SCOPE_PREFIX, + mcpWriteScope, normalizeScopes, WILDCARD_SCOPE, type ApiKeyScope, From 9b29790aa9bb4a8c8c838dae2303ea139360753f Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 12:57:10 +0200 Subject: [PATCH 42/90] fix(orchestrator): remove a literal NUL byte and disambiguate the idempotency cache key The cache-key template carried a raw NUL (git classified the source file as binary). Replaced with a length-prefixed ASCII composition, which also closes a real collision: a naive `${toolName}:${key}` maps both ("a:b","t") and ("b","t:a") to the same entry, letting one caller key replay another tool's stored write result. Regression test added and mutation-verified. --- .../src/toolIdempotency.ts | Bin 11149 -> 11459 bytes middleware/test/toolIdempotency.test.ts | 25 ++++++++++++++++++ 2 files changed, 25 insertions(+) diff --git a/middleware/packages/harness-orchestrator/src/toolIdempotency.ts b/middleware/packages/harness-orchestrator/src/toolIdempotency.ts index 701885c3995fed0312afc282eb833ae1fa2d1eed..2b9851f4a135210cf7eaa3d40d065a78b7affb10 100644 GIT binary patch delta 371 zcmXw#!AiqG5QgtL>CFhj5ENQT<2^U&sURZsAR7-fQ>@K?#Y$>I_fT#KvUIZV( zH|di&Yv^tH_y7O-WT>{rI#;PHBr8U&n1Hx<|?;O}#eUB{-XW&rzLafn! zM`p>*$a$*OiWXoEu*9PqjMyRAUU;)m+16P@h67Y+jAfXUh^Fp>BRz!ir`QJ4Nla5~JF7fgu7^ zR%^AO0v@!~C2%_(PbM+6H_7=)F7z>tP* bWv|R~a6DF_|IahrmY18Owk>}S>DTEWh7f&& delta 88 zcmX>c*&Dt=Us6ZGIWajSRUtdIQcuApH77GEwJ5P9HK$S`QK29wF*8r0xTGjEFI}NH sUm { assert.equal(store.size(), 1, 'the rejected entry must not linger alongside the good one'); }); + it('does not let a key containing the separator collide with another tool', async () => { + // `("a:b", "t")` and `("b", "t:a")` must stay distinct. A naive + // `${toolName}:${key}` composition maps BOTH to `t:a:b`, which would let one + // caller's key replay another tool's stored write result. + assert.notEqual( + idempotencyCacheKey('a:b', 't'), + idempotencyCacheKey('b', 't:a'), + 'cache-key composition is ambiguous — one tool could replay another tool result', + ); + + const store = new ToolIdempotencyStore(); + let runs = 0; + const exec = async () => { + runs += 1; + return { content: `run-${String(runs)}` }; + }; + const a = await store.run('a:b', 't', {}, exec); + const b = await store.run('b', 't:a', {}, exec); + + assert.equal(runs, 2, 'two distinct (key, tool) pairs must both execute'); + assert.equal(a.result.content, 'run-1'); + assert.equal(b.result.content, 'run-2'); + }); + it('bounds retained records', async () => { const store = new ToolIdempotencyStore({ maxEntries: 3 }); for (let i = 0; i < 10; i += 1) { From 23f9667bf2c00c6f9ae66fd33fa04d2d6c5a8311 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 13:00:43 +0200 Subject: [PATCH 43/90] feat(plugin-api): expose writeCapabilities on the plugin-facing tool accessor ToolRegistrationOptions gains writeCapabilities and the kernel shim forwards it on both register() and registerHandler(). Without this hop only kernel-internal registrations could declare themselves write-capable, so every real plugin (Odoo, M365) would have stayed unprotected while the unit tests passed. Test walks the actual ctx.tools.register shim; mutation-verified by dropping the forward. --- .../packages/plugin-api/src/pluginContext.ts | 19 +++++++ middleware/src/platform/pluginContext.ts | 12 ++++ middleware/test/toolIdempotency.test.ts | 55 +++++++++++++++++++ 3 files changed, 86 insertions(+) diff --git a/middleware/packages/plugin-api/src/pluginContext.ts b/middleware/packages/plugin-api/src/pluginContext.ts index cbfb8526..fe79077e 100644 --- a/middleware/packages/plugin-api/src/pluginContext.ts +++ b/middleware/packages/plugin-api/src/pluginContext.ts @@ -17,6 +17,8 @@ import type { Socket } from 'node:net'; +import type { WriteCapability } from './writeCapabilities.js'; + import type { EntityCapturedTurnsHit, EntityCapturedTurnsOptions, @@ -572,6 +574,23 @@ export interface ToolRegistrationOptions { readonly promptDoc?: string; /** Per-turn attachment collector. See NativeToolAttachmentSink docs. */ readonly attachmentSink?: NativeToolAttachmentSink; + /** + * #542 prerequisite — declare that dispatching this tool may MUTATE data. + * + * This is the plugin-facing end of the `WriteCapability` contract in + * `./writeCapabilities.ts` (see the NOTE under `NativeToolSpec` for why it + * rides the options bag rather than the spec: the spec is forwarded verbatim + * to Anthropic, which rejects unknown fields). The kernel stores it on the + * registry entry, where `ToolDispatchService` reads it. + * + * Declaring it opts the tool into duplicate-write protection: a dispatch that + * carries an idempotency key is deduplicated, and the MCP transport's + * transient retry is suppressed for it (a retry cannot tell "failed before + * writing" from "wrote, then lost the response"). A tool that mutates data and + * omits this gets no such protection — for an Odoo or M365 write reachable from + * a public endpoint, that means a duplicate is possible. + */ + readonly writeCapabilities?: readonly WriteCapability[]; } /** diff --git a/middleware/src/platform/pluginContext.ts b/middleware/src/platform/pluginContext.ts index 9e92b390..694aeb6b 100644 --- a/middleware/src/platform/pluginContext.ts +++ b/middleware/src/platform/pluginContext.ts @@ -444,6 +444,13 @@ export function createPluginContext( ...(options?.attachmentSink ? { attachmentSink: options.attachmentSink } : {}), + // #542 — carry the plugin's declared write capabilities into the registry + // so `ToolDispatchService` can give this tool duplicate-write protection. + // Without this forward, only kernel-internal registrations could declare + // themselves and no real plugin (Odoo, M365) would ever be protected. + ...(options?.writeCapabilities !== undefined + ? { writeCapabilities: options.writeCapabilities } + : {}), }); }, registerHandler(name, handler, options) { @@ -464,6 +471,11 @@ export function createPluginContext( ...(options?.attachmentSink ? { attachmentSink: options.attachmentSink } : {}), + // #542 — same forward as `register()` above; a handler-only tool is + // dispatchable by name, so it needs the same protection. + ...(options?.writeCapabilities !== undefined + ? { writeCapabilities: options.writeCapabilities } + : {}), }); }, async invoke(name, input) { diff --git a/middleware/test/toolIdempotency.test.ts b/middleware/test/toolIdempotency.test.ts index 758e2f67..58f5e6be 100644 --- a/middleware/test/toolIdempotency.test.ts +++ b/middleware/test/toolIdempotency.test.ts @@ -74,6 +74,61 @@ describe('write-capability declaration', () => { assert.equal(service.isWriteCapable('odoo_create_invoice'), true); assert.equal(service.isWriteCapable('nope'), false); }); + + it('reaches the dispatcher through the PLUGIN-facing accessor, not just kernel calls', async () => { + // The declaration is worthless if a real plugin cannot make it. This walks + // the actual `ctx.tools.register(spec, handler, options)` shim the kernel + // gives plugins and asserts the capability survives the hop into the + // registry — a shim that silently drops the field would leave every real + // Odoo/M365 write unprotected while all the unit tests stayed green. + const { createPluginContext } = await import('../src/platform/pluginContext.js'); + const { ServiceRegistry } = await import('../src/platform/serviceRegistry.js'); + type Opts = Parameters[0]; + const stub = (): (() => void) => (): void => {}; + const nativeTools = new NativeToolRegistry(); + const ctx = createPluginContext({ + agentId: '@omadia/integration-odoo', + vault: { + get: async () => undefined, + listKeys: async () => [], + } as unknown as Opts['vault'], + registry: { + has: () => true, + list: () => [], + get: () => undefined, + } as unknown as Opts['registry'], + catalog: new Map() as unknown as Opts['catalog'], + serviceRegistry: new ServiceRegistry(), + nativeToolRegistry: nativeTools, + routeRegistry: { + register: stub, + disposeBySource: () => 0, + } as unknown as Opts['routeRegistry'], + jobScheduler: { + register: stub, + stopForPlugin: () => {}, + } as unknown as Opts['jobScheduler'], + logger: () => {}, + }); + + ctx.tools.register( + { + name: 'odoo_post_invoice', + description: 'posts an invoice', + input_schema: { type: 'object', properties: {} }, + }, + async () => 'posted', + { writeCapabilities: CREATE_INVOICE }, + ); + + assert.deepEqual( + nativeTools.get('odoo_post_invoice')?.writeCapabilities, + CREATE_INVOICE, + 'the plugin-facing shim dropped writeCapabilities', + ); + const service = new ToolDispatchService({ nativeTools, domainTools: [] }); + assert.equal(service.isWriteCapable('odoo_post_invoice'), true); + }); }); describe('ToolIdempotencyStore', () => { From c11778f4f829f8e8b0c22557af808560abe306c6 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 13:04:24 +0200 Subject: [PATCH 44/90] feat(mcp): PublicMcpServer, per-key binding store and the /api/v1/mcp router New class rather than a flag on LoopbackMcpServer: the loopback's trust boundary (any local process that can read the 0600 bearer) does not transfer. Per-request Server+transport with sessionIdGenerator: undefined torn down in a finally, 405 on non-POST, 8MB cap, per-tool timeout, concurrency ceiling. tools/list returns exactly the callable set so a name is never leaked. --- .../harness-orchestrator/src/mcp/mcpClient.ts | 23 +- .../src/registry/agentGraphStore.ts | 5 +- middleware/src/auth/publicPaths.ts | 15 + middleware/src/mcp/publicMcpKeyBindings.ts | 213 +++++++ middleware/src/mcp/publicMcpPath.ts | 29 + middleware/src/mcp/publicMcpRouter.ts | 61 ++ middleware/src/mcp/publicMcpServer.ts | 574 ++++++++++++++++++ 7 files changed, 915 insertions(+), 5 deletions(-) create mode 100644 middleware/src/mcp/publicMcpKeyBindings.ts create mode 100644 middleware/src/mcp/publicMcpPath.ts create mode 100644 middleware/src/mcp/publicMcpRouter.ts create mode 100644 middleware/src/mcp/publicMcpServer.ts diff --git a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts index bfa04934..379b26c1 100644 --- a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts +++ b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts @@ -160,10 +160,25 @@ export interface McpToolDescriptor { readonly outputSchema?: Record; } -/** Caller taxonomy for the MCP call audit log (epic #459 W2, issue #462). - * Defined once here; skill (#456) and plugin (#458) surfaces identify - * themselves via `turnContext.mcpCallerKind`. */ -export type McpCallerKind = 'agent' | 'subagent' | 'skill' | 'plugin' | 'unattributed'; +/** + * Caller taxonomy for the MCP call audit log (epic #459 W2, issue #462). + * Defined once here; skill (#456) and plugin (#458) surfaces identify + * themselves via `turnContext.mcpCallerKind`. + * + * W2-3 (issue #542) adds `api_key`: a call arriving over the public MCP + * endpoint from a third party holding an API key. It is none of the other five + * — no orchestrator turn, no sub-agent, no plugin — and squeezing it into + * `plugin` or `unattributed` would make the one question that row exists to + * answer ("internal turn, or the internet?") unanswerable from the data. + * Migration 0033 widens the matching `mcp_call_log.caller_kind` CHECK. + */ +export type McpCallerKind = + | 'agent' + | 'subagent' + | 'skill' + | 'plugin' + | 'unattributed' + | 'api_key'; /** * Issue #544 (W2-1) — what actually happened on a call. diff --git a/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts b/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts index 8fbe5715..9fcca6f4 100644 --- a/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts +++ b/middleware/packages/harness-orchestrator/src/registry/agentGraphStore.ts @@ -2,6 +2,7 @@ import type { Pool } from 'pg'; import { ConfigValidationError, validateModelRef } from './configStore.js'; import { computeSkillHash } from './skillHash.js'; +import type { McpCallerKind } from '../mcp/mcpClient.js'; import { normalizeDiscoveredToolOrder } from '../toolOrdering.js'; /** @@ -534,7 +535,9 @@ export interface McpCallLogRow { readonly serverId: string | null; readonly serverName: string; readonly toolName: string; - readonly callerKind: 'agent' | 'subagent' | 'skill' | 'plugin' | 'unattributed'; + /** W2-3 (#542) — reuses the shared `McpCallerKind` union rather than a + * retyped copy, which is how the previous copy fell one member behind. */ + readonly callerKind: McpCallerKind; readonly callerAgent: string | null; readonly turnId: string | null; readonly ok: boolean; diff --git a/middleware/src/auth/publicPaths.ts b/middleware/src/auth/publicPaths.ts index 1d204dbe..86cf03a1 100644 --- a/middleware/src/auth/publicPaths.ts +++ b/middleware/src/auth/publicPaths.ts @@ -13,6 +13,7 @@ */ import { CIMD_METADATA_PATH } from '../services/mcpCimd.js'; +import { PUBLIC_MCP_PATH } from '../mcp/publicMcpPath.js'; /** Escape a literal path for embedding in a RegExp, so the shared constant — * not a hand-retyped pattern — is what the allowlist actually matches. */ @@ -79,6 +80,20 @@ export const STATIC_PUBLIC_PATHS: readonly RegExp[] = [ // narrowest regex that covers the one route, never a prefix that also // catches its siblings. /^\/api\/public\/v1\/chat(?:\/|$|\?)/, + // W2-3 (issue #542) — the public, stateless MCP endpoint. Follows the NOTE + // directly above to the letter: built from the SHARED `PUBLIC_MCP_PATH` + // constant (so the express mount and this allowlist cannot drift), and via + // `pathPrefixPattern`, which anchors on `$` or `?` and therefore matches the + // ONE path — not `/api/v1/mcp/anything` and not a sibling like + // `/api/v1/mcp-servers`. No regex bypass, no prefix that catches neighbours. + // + // Its authentication is `requireApiKey` from `@omadia/api-key-auth`, mounted + // by `mcp/publicMcpRouter.ts`. That is necessary but NOT the whole gate: the + // key must additionally hold `mcp:list`/`mcp:invoke` (and the exact + // `mcp:write:` for a write), AND have an enabled + // `public_mcp_key_bindings` row naming the one agent and the exact tools it + // reaches. A key with no row authenticates and reaches nothing. + pathPrefixPattern(PUBLIC_MCP_PATH), ]; /** diff --git a/middleware/src/mcp/publicMcpKeyBindings.ts b/middleware/src/mcp/publicMcpKeyBindings.ts new file mode 100644 index 00000000..e5741fe9 --- /dev/null +++ b/middleware/src/mcp/publicMcpKeyBindings.ts @@ -0,0 +1,213 @@ +/** + * W2-3 (issue #542) — the per-key authorization record for the public MCP + * endpoint: which agent a key is bound to, and exactly which of that agent's + * tools it may read and write. + * + * This is the seam the issue assumed already existed. omadia's native tool + * registry is a process-wide singleton with unique tool names; per-agent + * scoping existed only for DomainTools (`scopeDomainToolsToPlugins`), and the + * loopback MCP server's own security note says as much: "the subscription CLI + * sees the FULL native tool registry via the loopback MCP server, with no + * allowlist beyond MCP server scoping; a per-agent tool allowlist is a + * follow-up." An internet-facing endpoint cannot ship on that footing, so the + * allowlist is built here, per KEY rather than per server. + * + * Everything in this module fails CLOSED. Absent row, disabled row, malformed + * row, unreadable column: all resolve to "this key reaches no tools". There is + * deliberately no code path that turns a read problem into a grant — the + * asymmetry mirrors `normalizeScopes` in `@omadia/api-key-auth`, which denies + * everything on a malformed persisted `scopes` field for the same reason. + */ + +import type { Pool } from 'pg'; + +/** The resolved authorization for one API key. */ +export interface PublicMcpKeyBinding { + /** `ApiKeyRecord.id` — the key this binding belongs to. */ + readonly keyId: string; + /** The ONE agent (orchestrator slug) whose tools this key reaches. */ + readonly agentId: string; + /** Exact names of read-only tools this key may call. No patterns. */ + readonly readTools: readonly string[]; + /** Exact names of write-capable tools this key may call. Calling one + * additionally requires the `mcp:write:` scope and spends the write + * rate-limit budget. */ + readonly writeTools: readonly string[]; + /** Tighter per-minute budget for writes, independent of the key's general + * `rateLimitPerMinute`. */ + readonly writeRateLimitPerMinute: number; +} + +/** + * Reads bindings. Read-only by design: bindings are operator-managed + * configuration, and this endpoint — the internet-facing one — has no business + * holding a writer for its own authorization data. + */ +export interface PublicMcpKeyBindingStore { + /** The binding for `keyId`, or `undefined` when the key reaches nothing. + * `undefined` covers absent, disabled, and malformed alike: a caller that + * cannot distinguish them cannot accidentally treat one as permissive. */ + get(keyId: string): Promise; +} + +/** A binding that grants nothing, for the "row exists but says no" case. */ +export function denyAllBinding(keyId: string, agentId: string): PublicMcpKeyBinding { + return { keyId, agentId, readTools: [], writeTools: [], writeRateLimitPerMinute: 0 }; +} + +/** + * Shapes a raw row into a binding, or `undefined` when the row cannot be + * trusted. + * + * Exported so both store implementations and the tests share ONE normalization + * rule. The pg driver returns `TEXT[]` as a JS array, but a hand-edited row, a + * future column-type change, or a NULL where the schema promises NOT NULL would + * all arrive here as something else — and each of those must deny, not partly + * grant. + */ +export function normalizeBindingRow(raw: { + key_id?: unknown; + agent_id?: unknown; + read_tools?: unknown; + write_tools?: unknown; + write_rate_limit_per_minute?: unknown; + enabled?: unknown; +}): PublicMcpKeyBinding | undefined { + const keyId = nonEmptyString(raw.key_id); + const agentId = nonEmptyString(raw.agent_id); + if (!keyId || !agentId) { + warnMalformed('key_id or agent_id missing/empty', keyId ?? ''); + return undefined; + } + + // `enabled` is NOT NULL DEFAULT true in the schema, so anything other than a + // boolean is corruption. Treat it as disabled rather than guessing `true`: + // guessing wrong in that direction reopens an endpoint an operator parked. + if (typeof raw.enabled !== 'boolean') { + warnMalformed('enabled is not a boolean', keyId); + return undefined; + } + if (!raw.enabled) return undefined; + + const readTools = normalizeToolList(raw.read_tools, 'read_tools', keyId); + const writeTools = normalizeToolList(raw.write_tools, 'write_tools', keyId); + if (!readTools || !writeTools) return undefined; + + // A tool named in BOTH lists is ambiguous about whether it needs + // `mcp:write:`. Resolve toward the STRICTER reading — it is a write — + // rather than rejecting the whole row, because an operator adding a write + // capability to a tool they had listed as a read is a plausible edit and + // silently downgrading it to a read would be the dangerous resolution. + const writeSet = new Set(writeTools); + const readOnly = readTools.filter((t) => !writeSet.has(t)); + + const writeLimit = normalizeRateLimit(raw.write_rate_limit_per_minute); + if (writeLimit === undefined) { + warnMalformed('write_rate_limit_per_minute is not a usable integer', keyId); + return undefined; + } + + return { + keyId, + agentId, + readTools: readOnly, + writeTools: Array.from(writeSet), + writeRateLimitPerMinute: writeLimit, + }; +} + +function nonEmptyString(value: unknown): string | undefined { + return typeof value === 'string' && value.length > 0 ? value : undefined; +} + +/** `undefined` signals "deny the whole row"; `[]` is a legitimate empty list. */ +function normalizeToolList( + raw: unknown, + column: string, + keyId: string, +): readonly string[] | undefined { + if (raw === null || raw === undefined) { + // Schema says NOT NULL DEFAULT '{}', so NULL here is a foreign writer. + warnMalformed(`${column} is null`, keyId); + return undefined; + } + if (!Array.isArray(raw)) { + warnMalformed(`${column} is not an array`, keyId); + return undefined; + } + const invalid = raw.filter((entry) => nonEmptyString(entry) === undefined); + if (invalid.length > 0) { + // Partially-valid arrays deny rather than narrowing to the valid subset — + // same rule and same reasoning as `normalizeScopes`: a record we cannot + // read faithfully is one we must not guess at. + warnMalformed(`${column} holds ${String(invalid.length)} non-string entr(y|ies)`, keyId); + return undefined; + } + return Array.from(new Set(raw as readonly string[])); +} + +function normalizeRateLimit(raw: unknown): number | undefined { + if (typeof raw === 'number' && Number.isInteger(raw) && raw >= 0) return raw; + // pg returns some integer types as strings depending on the type OID; accept + // a clean integer string rather than denying a perfectly good row. + if (typeof raw === 'string' && /^\d+$/.test(raw)) return Number(raw); + return undefined; +} + +/** A malformed binding silently stops a key from reaching anything; without + * this line an operator cannot tell that from a deliberate revoke. Only the + * key id and the reason are logged — never the row, which names the tools an + * integration is trusted with. */ +function warnMalformed(reason: string, keyId: string): void { + console.warn( + `[public-mcp] unusable key binding (${reason}) for key ${keyId} — key reaches no tools until the row is repaired`, + ); +} + +/** + * Postgres-backed store. + * + * No cache. A binding is read once per MCP request against a primary-key + * lookup, and an operator revoking a tool from an integration expects that to + * take effect on the next call rather than after a TTL. If this ever becomes + * hot, the fix is a short negative cache — never a positive one, because a + * cached grant is a grant that outlives its revocation. + */ +export function createPublicMcpKeyBindingStore(pool: Pool): PublicMcpKeyBindingStore { + return { + async get(keyId) { + if (nonEmptyString(keyId) === undefined) return undefined; + const { rows } = await pool.query( + `SELECT key_id, agent_id, read_tools, write_tools, write_rate_limit_per_minute, enabled + FROM public_mcp_key_bindings + WHERE key_id = $1`, + [keyId], + ); + const row = rows[0] as Parameters[0] | undefined; + return row ? normalizeBindingRow(row) : undefined; + }, + }; +} + +/** + * In-memory store, for tests and for a DATABASE_URL-less install. + * + * Takes RAW row shapes rather than ready-made `PublicMcpKeyBinding` values on + * purpose: a test that hands over a well-formed object bypasses + * `normalizeBindingRow` entirely and therefore proves nothing about the + * fail-closed rules the pg path relies on. Same normalization, same denials. + */ +export function createInMemoryPublicMcpKeyBindingStore( + rows: readonly Parameters[0][], +): PublicMcpKeyBindingStore { + const byKey = new Map[0]>(); + for (const row of rows) { + if (typeof row.key_id === 'string') byKey.set(row.key_id, row); + } + return { + async get(keyId) { + const row = byKey.get(keyId); + return row ? normalizeBindingRow(row) : undefined; + }, + }; +} diff --git a/middleware/src/mcp/publicMcpPath.ts b/middleware/src/mcp/publicMcpPath.ts new file mode 100644 index 00000000..88952a6b --- /dev/null +++ b/middleware/src/mcp/publicMcpPath.ts @@ -0,0 +1,29 @@ +/** + * W2-3 (issue #542) — the ONE definition of the public MCP endpoint's path. + * + * Its own module, dependency-free on purpose. `auth/publicPaths.ts` imports it + * to build the requireAuth exemption, and `index.ts` imports it to mount the + * router; if this constant lived next to the server implementation, importing + * it would drag the MCP SDK into `publicPaths.ts`'s import graph (and into + * every test that asserts against the allowlist) for the sake of one string. + * + * The reason it is a shared constant at all is recorded at the top of + * `auth/publicPaths.ts`: epic #470's runner router was mounted without a + * session guard and its e2e test built a bare `express()` app to prove it, so + * the test passed while the route 401'd in production behind the blanket `/api` + * guard. A retyped path is that same bug with a different name. + */ + +/** Where the public, stateless, API-key-authenticated MCP server is mounted. */ +export const PUBLIC_MCP_PATH = '/api/v1/mcp'; + +/** + * Denormalized `server_name` for the `mcp_call_log` rows this endpoint writes. + * + * Public MCP calls have no upstream MCP server — omadia IS the server here, and + * the call goes inward to a local tool rather than outward to a vendor. The + * `server_id` FK stays NULL (0009 made it nullable precisely so audit rows + * survive without a server row) and this literal is what an operator sees in + * the call-log UI's Server column. + */ +export const PUBLIC_MCP_SERVER_NAME = 'omadia-public-mcp'; diff --git a/middleware/src/mcp/publicMcpRouter.ts b/middleware/src/mcp/publicMcpRouter.ts new file mode 100644 index 00000000..b0cd9f1f --- /dev/null +++ b/middleware/src/mcp/publicMcpRouter.ts @@ -0,0 +1,61 @@ +/** + * W2-3 (issue #542) — mounts the public MCP server as an express router. + * + * Split from `PublicMcpServer` so the protocol/authorization logic is testable + * without an HTTP stack, and so the WIRING (which middleware, in which order) + * is one short readable file. The order below is load-bearing: + * + * 1. `bodyCapMiddleware` — cheapest rejection, and it must run before any + * work is attributed to a key. + * 2. `requireApiKey` — authentication + the general per-key rate limit + + * the `mcp:invoke`-class scope floor. Answers 401 on + * a missing/invalid key and 403 on a scope miss. + * 3. the MCP handler — per-request stateless transport, per-key allowlist. + * + * `requireApiKey` is given NO `scope` option on purpose. A single scope check + * here would have to be either `mcp:list` or `mcp:invoke`, and whichever were + * chosen would 403 the other legitimate request shape. Both are checked inside + * the JSON-RPC handlers, where the method being invoked is known. The 403-on- + * scope-miss behavior is still exercised — see `requireApiKey`'s own tests and + * the endpoint tests for the per-method gates. + */ + +import { Router } from 'express'; + +import type { ApiKeyStore, AuditLog, RateLimiter } from '@omadia/api-key-auth'; +import { requireApiKey } from '@omadia/api-key-auth'; + +import { PUBLIC_MCP_PATH } from './publicMcpPath.js'; +import { PublicMcpServer, type PublicMcpServerDeps } from './publicMcpServer.js'; + +export interface PublicMcpRouterDeps extends PublicMcpServerDeps { + /** The SAME store the operator mints keys with. Reused rather than + * duplicated: a second key store would be a second place to revoke. */ + readonly apiKeys: ApiKeyStore; + /** General per-key budget, applied by `requireApiKey` to every request + * including `tools/list`. Distinct from `writeRateLimiter`. */ + readonly rateLimiter?: RateLimiter; + /** `@omadia/api-key-auth`'s own vault-backed usage trail. Complementary to + * the `mcp_call_log` rows the audit sink writes: this one records HTTP + * outcomes per key, that one records tool calls. */ + readonly keyAuditLog?: AuditLog; +} + +export function createPublicMcpRouter(deps: PublicMcpRouterDeps): Router { + const server = new PublicMcpServer(deps); + const router = Router(); + + router.use( + PUBLIC_MCP_PATH, + server.bodyCapMiddleware(), + requireApiKey({ + apiKeys: deps.apiKeys, + ...(deps.rateLimiter ? { rateLimiter: deps.rateLimiter } : {}), + ...(deps.keyAuditLog ? { auditLog: deps.keyAuditLog } : {}), + routeLabel: PUBLIC_MCP_PATH, + }), + server.handler(), + ); + + return router; +} diff --git a/middleware/src/mcp/publicMcpServer.ts b/middleware/src/mcp/publicMcpServer.ts new file mode 100644 index 00000000..9ebe2210 --- /dev/null +++ b/middleware/src/mcp/publicMcpServer.ts @@ -0,0 +1,574 @@ +/** + * W2-3 (issue #542) — the public, stateless, API-key-authenticated MCP server. + * + * ─── Why this is a new class and not a flag on `LoopbackMcpServer` ─────────── + * + * `LoopbackMcpServer` binds `127.0.0.1` on an ephemeral port and authenticates + * ONE static bearer with a constant-time compare. Its own security note states + * the trust boundary it was designed for: "any local process that can read the + * 0600 mcp-config bearer can call omadia's tools — a local-process trust + * boundary." Not one clause of that transfers to an internet-facing route. + * There is no single bearer, no local-process assumption, no "the token is the + * whole authorization", and no acceptable version of "sees the FULL native tool + * registry" (which is what the loopback path documents itself as doing). A flag + * would leave both behaviors in one class where the dangerous default is one + * boolean away from every caller. + * + * What IS shared is the stateless-transport lifecycle, and that part is copied + * deliberately rather than reinvented — see `createRequestScopedServer`. + * + * ─── The authorization model ──────────────────────────────────────────────── + * + * Four independent gates, all default-deny, in this order: + * + * 1. AUTHENTICATION — `requireApiKey` (mounted by the router, not here) does + * the constant-time hash compare and answers 401. It deliberately does not + * populate `req.session`; that is preserved, and nothing here reads it. + * 2. BINDING — the key must have an enabled `public_mcp_key_bindings` row. + * No row ⇒ zero tools. The row names ONE agent, which is what makes key A + * unable to reach agent B's tools even though the native tool registry is + * process-wide. + * 3. ALLOWLIST — the tool must be named in that row. Enforced on `tools/call` + * AND on `tools/list`, because a tool name the key cannot call is itself a + * disclosure (it tells a third party which integrations this install runs). + * 4. SCOPE — `mcp:list` to enumerate, `mcp:invoke` to call, and additionally + * the exact `mcp:write:` for anything the row lists as a write. + * `WILDCARD_SCOPE` does not satisfy a write scope; `hasScope` enforces that + * for every caller. + * + * `tools/list` returns exactly the set the key could successfully CALL — not + * "everything it may see". A key holding `mcp:list` but not `mcp:invoke` gets + * an empty list, and a write tool appears only when its per-tool write scope is + * present. Any looser rule turns the list into an inventory of what to attack. + */ + +import type { Request, RequestHandler, Response } from 'express'; +import { Server as McpServer } from '@modelcontextprotocol/sdk/server/index.js'; +import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js'; +import { + CallToolRequestSchema, + ErrorCode, + ListToolsRequestSchema, + McpError, +} from '@modelcontextprotocol/sdk/types.js'; + +import type { ApiKeyPrincipal, ApiKeyScope, RateLimiter } from '@omadia/api-key-auth'; +import { hasScope, hasWriteScope, MCP_INVOKE_SCOPE, MCP_LIST_SCOPE } from '@omadia/api-key-auth'; +import type { DispatchableToolSpec, ToolDispatchResult } from '@omadia/orchestrator'; + +import type { PublicMcpKeyBinding, PublicMcpKeyBindingStore } from './publicMcpKeyBindings.js'; + +/** Mirrors `LoopbackMcpServer`'s ceiling. See `enforceBodyCap` for why it is + * re-checked here instead of being handed to `express.json`. */ +export const MAX_REQUEST_BYTES = 8 * 1024 * 1024; + +/** Per-tool wall clock. A public caller must not be able to pin a dispatch + * slot open indefinitely; without this, `maxConcurrentCalls` below is a + * denial-of-service budget rather than a protection. */ +export const DEFAULT_TOOL_TIMEOUT_MS = 30_000; + +/** Process-wide ceiling on tool calls in flight from this endpoint. Tools reach + * Odoo/M365/Confluence and the LLM providers; an unbounded public fan-in + * starves the operator-facing chat path that shares those pools. */ +export const DEFAULT_MAX_CONCURRENT_CALLS = 4; + +/** + * The subset of `ToolDispatchService` this server uses. + * + * Structural rather than the concrete class so the wiring can supply a + * per-agent dispatcher without this module importing the orchestrator's + * construction path — and so tests exercise the real gates against a fake + * dispatcher instead of a whole orchestrator. + */ +export interface PublicMcpDispatcher { + dispatch(name: string, input: unknown): Promise; + listDispatchableToolSpecs(): readonly DispatchableToolSpec[]; +} + +/** + * WHO is calling — assembled here and handed to dispatch. + * + * ─── WHERE THIS BRANCH MEETS `feat/w3-b-dispatch-privacy-seam-and-idempotency` + * + * `ToolDispatchService.dispatch(name, input)` today carries no tenant, no user + * and no principal, and its trailing SEAM comment records that privacy + * interning and trace capture are NOT replicated versus + * `Orchestrator.dispatchToolInner`. The sibling unit closes that seam and adds + * an optional caller-context parameter. This type is the shape this endpoint + * offers it; `PublicMcpServerDeps.dispatchWithContext` is the single injection + * point where the two branches join. Nothing in `toolDispatchService.ts` is + * touched by this branch. + */ +export interface PublicMcpCallerContext { + /** Stable id of the API key. Becomes `apikey:` in the audit trail. */ + readonly keyId: string; + readonly label?: string; + readonly scopes: readonly ApiKeyScope[]; + /** The agent whose tools this call runs against. */ + readonly agentId: string; + /** True when the tool is declared write-capable by the key's binding. */ + readonly write: boolean; +} + +/** One audit row per call, written by the wiring. Mirrors the vocabulary the + * base branch established on `mcp_call_log` (`actingIdentity`, with the + * literal `unresolved` for an identity that could not be established). */ +export interface PublicMcpAuditEntry { + readonly keyId: string; + readonly agentId: string; + readonly toolName: string; + readonly ok: boolean; + readonly error: string | null; + readonly durationMs: number; + readonly calledAt: Date; + readonly actingIdentity: string; + readonly write: boolean; +} + +/** Fire-and-forget. Implementations MUST NOT throw — an audit failure must + * never fail a caller's request, and must never be the reason a call + * succeeds either. */ +export type PublicMcpAuditSink = (entry: PublicMcpAuditEntry) => void; + +export interface PublicMcpServerDeps { + /** + * Resolves the dispatcher for ONE agent. `undefined` when that agent is not + * currently active — which fails the call closed rather than falling back to + * any other agent's dispatcher. + */ + readonly resolveDispatcher: (agentId: string) => PublicMcpDispatcher | undefined; + readonly bindings: PublicMcpKeyBindingStore; + /** + * Budget for WRITES only, separate from the general per-key limiter + * `requireApiKey` already applies. Two limiter instances, not one shared + * bucket: reads are cheap and idempotent, writes are neither, and a + * read-heavy integration's unused read headroom must not fund a write burst. + */ + readonly writeRateLimiter: RateLimiter; + readonly audit?: PublicMcpAuditSink; + /** + * Where the sibling privacy-seam branch plugs in. When present, EVERY tool + * call goes through it instead of calling `dispatch` directly. + */ + readonly dispatchWithContext?: ( + dispatcher: PublicMcpDispatcher, + name: string, + input: unknown, + caller: PublicMcpCallerContext, + ) => Promise; + /** + * Whether a call is refused when `dispatchWithContext` is absent. + * + * DEFAULTS TO TRUE, i.e. the endpoint refuses to serve tool calls until the + * privacy/trace seam is closed. `ToolDispatchService` applies no PII masking + * — the chat path's masking lives in `Orchestrator.dispatchToolInner`, which + * this dispatcher explicitly does not replicate — so serving without the + * seam means a public HTTP response can carry unmasked personal data straight + * out of Odoo or M365. That is not a trade to make silently, so the default + * is to fail closed and say why. Set false ONLY with a deliberate, + * documented operator decision. + */ + readonly requirePrivacySeam?: boolean; + readonly serverName?: string; + readonly serverVersion?: string; + readonly toolTimeoutMs?: number; + readonly maxConcurrentCalls?: number; +} + +/** Deliberately identical for "no such tool" and "not allowlisted for this + * key". Distinguishing them would confirm a tool's existence to a caller not + * entitled to know, which is the same disclosure `tools/list` filtering + * exists to prevent. */ +function unavailableToolMessage(name: string): string { + return `Tool \`${name}\` is not available to this API key.`; +} + +export class PublicMcpServer { + private inFlight = 0; + + constructor(private readonly deps: PublicMcpServerDeps) {} + + private get toolTimeoutMs(): number { + return this.deps.toolTimeoutMs ?? DEFAULT_TOOL_TIMEOUT_MS; + } + + private get maxConcurrentCalls(): number { + return this.deps.maxConcurrentCalls ?? DEFAULT_MAX_CONCURRENT_CALLS; + } + + private get privacySeamRequired(): boolean { + return this.deps.requirePrivacySeam ?? true; + } + + /** + * Enforces the 8 MB ceiling. + * + * NOT `express.json({ limit })`. The kernel mounts a global + * `express.json({ limit: '10mb' })` before every `/api` router (index.ts), so + * by the time a request reaches this one the stream is already consumed and + * parsed: a route-level parser would be a silent no-op and the loopback + * server's 8 MB ceiling would quietly become the kernel's 10 MB one. So the + * check runs on the two signals still available after parsing — + * `Content-Length` (the only pre-parse number, sent by any well-behaved + * client) and the re-serialized body length (which covers a chunked upload + * that carries no `Content-Length` at all). + * + * Mounting this router BEFORE `express.json` would allow a real streaming + * cap, but would also put it in front of the `/api` requireAuth mount and + * throw away the `publicPaths` half of the defense this route is required to + * use. The cap is the cheaper thing to reimplement. + */ + bodyCapMiddleware(): RequestHandler { + return (req: Request, res: Response, next): void => { + const declared = Number(req.headers['content-length']); + const declaredTooLarge = Number.isFinite(declared) && declared > MAX_REQUEST_BYTES; + const actualTooLarge = + req.body !== undefined && Buffer.byteLength(JSON.stringify(req.body) ?? '', 'utf8') > MAX_REQUEST_BYTES; + if (declaredTooLarge || actualTooLarge) { + res.status(413).json({ + jsonrpc: '2.0', + error: { code: 413, message: 'Payload Too Large' }, + id: null, + }); + return; + } + next(); + }; + } + + /** The express handler. Mount behind `requireApiKey`, which is what + * guarantees `req.apiKey` is present. */ + handler(): RequestHandler { + return (req: Request, res: Response): void => { + void this.handleHttp(req, res); + }; + } + + private async handleHttp(req: Request, res: Response): Promise { + // POST only, for the same two reasons `LoopbackMcpServer` gives: the MCP + // spec makes the standalone GET SSE stream optional and blesses 405 when a + // server does not offer one, and — decisive here — a per-request transport + // LEAKS on GET, because an SSE stream never ends, so `handleRequest` never + // resolves and the `finally` that tears the pair down never runs. + if (req.method !== 'POST') { + res.status(405).set('Allow', 'POST').json({ + jsonrpc: '2.0', + error: { code: -32000, message: 'Method Not Allowed' }, + id: null, + }); + return; + } + + const principal = req.apiKey; + if (!principal) { + // Unreachable behind `requireApiKey`. Answering 401 rather than throwing + // means a future mis-mount degrades to "authentication required" instead + // of to an unauthenticated 500 that still ran the handler. + res.status(401).json({ + jsonrpc: '2.0', + error: { code: -32001, message: 'Unauthorized' }, + id: null, + }); + return; + } + + let session: ReturnType | undefined; + try { + session = this.createRequestScopedServer(principal); + await session.mcp.connect(session.transport); + await session.transport.handleRequest(req, res, req.body); + } catch (error) { + if (res.headersSent) { + res.end(); + return; + } + // No error detail on the wire: this is a public surface, and a dispatch + // stack trace names internal tools, plugins and hosts. + console.warn(`[public-mcp] request failed: ${String(error)}`); + res.status(500).json({ + jsonrpc: '2.0', + error: { code: -32603, message: 'Internal server error' }, + id: null, + }); + } finally { + // A stateless transport is SINGLE-USE — the SDK throws "Stateless + // transport cannot be reused across requests" on its second use — so + // dropping it here is what makes the next request work at all. Safe at + // this point because `enableJsonResponse` means the response is fully + // written by the time `handleRequest` resolves. + await session?.transport.close().catch(() => {}); + await session?.mcp.close().catch(() => {}); + } + } + + /** + * A fresh `Server` + transport pair for ONE HTTP request. + * + * `sessionIdGenerator: undefined` selects the SDK's stateless mode: no + * session id is issued, no session validation happens, and a client may skip + * the `initialize` handshake and never send `Mcp-Session-Id`. That is the + * whole premise of the issue — horizontal scalability requires that any + * process can answer any request — and it is why the pair is per-request by + * construction rather than by convention: a shared transport makes only the + * FIRST request work and 500s every one after it. + */ + private createRequestScopedServer(principal: ApiKeyPrincipal): { + mcp: McpServer; + transport: StreamableHTTPServerTransport; + } { + const mcp = new McpServer( + { + name: this.deps.serverName ?? 'omadia-public-mcp', + version: this.deps.serverVersion ?? '0.0.0', + }, + { capabilities: { tools: {} } }, + ); + + mcp.setRequestHandler(ListToolsRequestSchema, async () => ({ + tools: await this.listToolsFor(principal), + })); + + mcp.setRequestHandler(CallToolRequestSchema, async (request) => { + const { name, arguments: args } = request.params; + const result = await this.callToolFor(principal, name, args ?? {}); + return { + content: [{ type: 'text' as const, text: result.content }], + ...(result.isError ? { isError: true } : {}), + }; + }); + + const transport = new StreamableHTTPServerTransport({ + sessionIdGenerator: undefined, + enableJsonResponse: true, + }); + + return { mcp, transport }; + } + + /** + * The tools this key can actually CALL, name-sorted. + * + * Not "the tools it may see" — the two must be the same set. A name the + * caller cannot invoke is a free hint about which integrations this install + * runs, which is exactly the enumeration the issue's own security notes warn + * about. + */ + private async listToolsFor( + principal: ApiKeyPrincipal, + ): Promise<{ name: string; description: string; inputSchema: unknown }[]> { + if (!hasScope(principal.scopes, MCP_LIST_SCOPE)) { + // An error, not an empty list: "not scoped for mcp:list" leaks no tool + // names, and an integrator debugging a misconfigured key deserves to be + // able to tell a scope problem from an empty allowlist. + throw new McpError( + ErrorCode.InvalidRequest, + `this API key is not scoped for '${MCP_LIST_SCOPE}'`, + ); + } + + const binding = await this.deps.bindings.get(principal.keyId); + if (!binding) return []; + + const dispatcher = this.deps.resolveDispatcher(binding.agentId); + if (!dispatcher) return []; + + const callable = this.callableToolNames(principal, binding); + if (callable.size === 0) return []; + + // Filter the AGENT's advertised specs by the KEY's callable set. Both + // directions matter: a tool in the binding that the agent does not + // advertise cannot be described (and must not be invented), and a tool the + // agent advertises that the binding omits must not appear. + return dispatcher + .listDispatchableToolSpecs() + .filter((spec) => callable.has(spec.name)) + .map((spec) => ({ + name: spec.name, + description: spec.description, + inputSchema: spec.input_schema, + })) + .sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0)); + } + + /** + * The exact set of tool names this key could successfully invoke. + * + * ONE function, used by both `tools/list` and `tools/call`, so the list can + * never advertise something the call path would refuse (or vice versa — + * which would be the security-relevant direction). + */ + private callableToolNames( + principal: ApiKeyPrincipal, + binding: PublicMcpKeyBinding, + ): ReadonlySet { + if (!hasScope(principal.scopes, MCP_INVOKE_SCOPE)) return new Set(); + const callable = new Set(binding.readTools); + for (const tool of binding.writeTools) { + // Per-tool, and wildcard-proof: `hasWriteScope` routes through `hasScope`, + // which refuses to let `*` satisfy a `mcp:write:` scope. + if (hasWriteScope(principal.scopes, tool)) callable.add(tool); + } + return callable; + } + + private async callToolFor( + principal: ApiKeyPrincipal, + name: string, + input: unknown, + ): Promise { + const startedAt = Date.now(); + const binding = await this.deps.bindings.get(principal.keyId); + + // No binding ⇒ nothing is callable, and there is no agent to attribute the + // attempt to. Audited with the key id so a probing key is still visible. + if (!binding) { + this.record(principal, 'unbound', name, false, 'no public MCP binding', startedAt, false); + throw new McpError(ErrorCode.InvalidParams, unavailableToolMessage(name)); + } + + const isWrite = binding.writeTools.includes(name); + const callable = this.callableToolNames(principal, binding); + if (!callable.has(name)) { + this.record(principal, binding.agentId, name, false, 'not allowlisted', startedAt, isWrite); + throw new McpError(ErrorCode.InvalidParams, unavailableToolMessage(name)); + } + + // Stricter budget for writes, on its own limiter instance. Checked AFTER + // authorization so a caller cannot map the allowlist by watching which + // names cost quota, and BEFORE the concurrency slot so an over-budget + // caller cannot occupy one. + if (isWrite && !this.deps.writeRateLimiter.tryConsume(principal.keyId, binding.writeRateLimitPerMinute)) { + this.record(principal, binding.agentId, name, false, 'write rate limited', startedAt, true); + throw new McpError( + ErrorCode.InvalidRequest, + `write rate limit exceeded: this key is limited to ${String(binding.writeRateLimitPerMinute)} write calls/minute`, + ); + } + + const dispatcher = this.deps.resolveDispatcher(binding.agentId); + if (!dispatcher) { + this.record(principal, binding.agentId, name, false, 'agent not active', startedAt, isWrite); + throw new McpError(ErrorCode.InternalError, unavailableToolMessage(name)); + } + + if (this.privacySeamRequired && !this.deps.dispatchWithContext) { + // See `requirePrivacySeam`. Refusing here rather than at boot keeps the + // endpoint's `tools/list` honest (an integrator can still discover the + // contract) while making it impossible to move unmasked data. + this.record(principal, binding.agentId, name, false, 'privacy seam absent', startedAt, isWrite); + throw new McpError( + ErrorCode.InternalError, + 'public MCP tool calls are disabled: the dispatch privacy/trace seam is not wired, so a response could carry unmasked personal data', + ); + } + + if (this.inFlight >= this.maxConcurrentCalls) { + this.record(principal, binding.agentId, name, false, 'concurrency ceiling', startedAt, isWrite); + throw new McpError(ErrorCode.InternalError, 'public MCP endpoint is at capacity — retry shortly'); + } + + const caller: PublicMcpCallerContext = { + keyId: principal.keyId, + ...(principal.label ? { label: principal.label } : {}), + scopes: principal.scopes, + agentId: binding.agentId, + write: isWrite, + }; + + this.inFlight += 1; + try { + const result = await this.withTimeout(name, () => + this.deps.dispatchWithContext + ? this.deps.dispatchWithContext(dispatcher, name, input, caller) + : dispatcher.dispatch(name, input), + ); + this.record( + principal, + binding.agentId, + name, + result.isError !== true, + result.isError === true ? 'tool reported an error' : null, + startedAt, + isWrite, + ); + return result; + } catch (error) { + this.record(principal, binding.agentId, name, false, String(error), startedAt, isWrite); + throw error; + } finally { + this.inFlight -= 1; + } + } + + /** + * Bounds one dispatch. + * + * The timer is cleared on BOTH paths. A dangling timer would keep the + * process's event loop alive per call, and — worse for a public endpoint — + * the losing side of the race is the only thing that ever clears it, so a + * fast tool would leak one timer per successful call. + * + * Note the dispatch itself is not cancelled (nothing in `ToolDispatchService` + * accepts an AbortSignal); the SLOT is released, which is what the + * concurrency ceiling needs. Recorded here so nobody reads this as + * cancellation. + */ + private async withTimeout( + toolName: string, + run: () => Promise, + ): Promise { + let timer: NodeJS.Timeout | undefined; + try { + return await Promise.race([ + run(), + new Promise((_resolve, reject) => { + timer = setTimeout( + () => + reject( + new McpError( + ErrorCode.InternalError, + `tool \`${toolName}\` exceeded the ${String(this.toolTimeoutMs)}ms public MCP timeout`, + ), + ), + this.toolTimeoutMs, + ); + }), + ]); + } finally { + if (timer) clearTimeout(timer); + } + } + + /** One audit row per call ATTEMPT, including every refusal. A refusal that + * leaves no trace is the one an operator cannot investigate. */ + private record( + principal: ApiKeyPrincipal, + agentId: string, + toolName: string, + ok: boolean, + error: string | null, + startedAt: number, + write: boolean, + ): void { + if (!this.deps.audit) return; + try { + this.deps.audit({ + keyId: principal.keyId, + agentId, + toolName, + ok, + error, + durationMs: Math.max(0, Date.now() - startedAt), + calledAt: new Date(startedAt), + // Same vocabulary the base branch established: a resolved identity, or + // the literal `unresolved` when there is none to name. + actingIdentity: principal.keyId ? `apikey:${principal.keyId}` : 'unresolved', + write, + }); + } catch (err) { + // An audit sink that throws must not turn a successful call into a + // failure — nor a refusal into a success. + console.warn(`[public-mcp] audit sink threw: ${String(err)}`); + } + } +} From 3445dbb3fa08604182e5523bf1c6ae3611d92bbd Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 13:05:06 +0200 Subject: [PATCH 45/90] feat(devplatform): make the stalled-job sweep narrowable, and restore its pg test MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `DevJobStore.findStalled` had no scope predicate at all: `WHERE status IN (provisioning, running, applying) AND COALESCE(last_heartbeat_at, started_at, claimed_at) < $1`. Database-global is CORRECT production behaviour for the single-tenant deployment — the reaper must reach every abandoned job whoever launched it — but it made the sweep untestable in a shared cluster: a forward-dated cutoff in one suite finalized a sibling suite's in-flight jobs as `stalled` and broke `devPlatformPipeline.wire.pg.test.ts`, so the pg-level test was removed and downgraded to a fake-`findStalled` unit test. - `findStalled(cutoff, scope?)` takes an optional `DevJobSweepScope`. `dev_jobs` has no tenant column, so `repo_id` is the tenancy axis: `AND repo_id = ANY($2::uuid[])`. Omitting the scope emits byte-identical SQL to before, so production behaviour is unchanged. - An EXPLICITLY EMPTY `repoIds` returns no rows. It must never widen back to global — that would turn a caller who computed an empty entitlement set into a caller who sweeps everything. - `DevJobTaskStoreDeps.sweepScope` threads it into `reapOrphans` at CONSTRUCTION, not per call, so the generic `TaskReapOptions` contract stays free of dev-platform concepts. - Restored the pg-level sweep test, now with the assertion whose absence forced the downgrade: a sibling repo's in-flight job is left `provisioning`. Setup activates jobs by stamping the columns rather than `claimNextQueued`, which pops the oldest queued row DATABASE-WIDE and cannot be aimed at a repo with concurrent suites running. Kept the fake-based unit test for its driven clock. Mutation-checked: ignoring the predicate, letting an empty scope widen to global, and dropping the forwarding in `reapOrphans` each turn a real assertion red. `devPlatformPipeline.wire.pg.test.ts` passes alongside the restored sweep. --- middleware/src/devplatform/devJobStore.ts | 32 ++++- middleware/src/devplatform/devJobTaskStore.ts | 18 ++- .../devplatform/devJobTaskStore.pg.test.ts | 121 ++++++++++++++++-- .../test/tasks/devJobTaskStoreReap.test.ts | 7 +- 4 files changed, 156 insertions(+), 22 deletions(-) diff --git a/middleware/src/devplatform/devJobStore.ts b/middleware/src/devplatform/devJobStore.ts index f5d65aff..399bbd93 100644 --- a/middleware/src/devplatform/devJobStore.ts +++ b/middleware/src/devplatform/devJobStore.ts @@ -165,6 +165,25 @@ export interface DevJobStoreOptions { artifactCeiling?: ArtifactCeilingOptions; } +/** + * Narrows a reaper sweep (W3-A). `dev_jobs` has no tenant column — `repo_id` IS + * the tenancy axis here, and every suite/deployment owns its own `dev_repos` + * rows — so a repo-id set is the scope key. + * + * OMITTING this (the production call) keeps the sweep DATABASE-GLOBAL, which is + * correct for the single-tenant deployment: the reaper must reach every + * abandoned job whoever launched it. It exists because a global sweep is + * untestable in a shared cluster — a forward-dated cutoff in one suite finalized + * a sibling suite's in-flight jobs as `stalled`. + * + * An EXPLICITLY EMPTY `repoIds` means "nothing is in scope" and returns no rows. + * It must never widen back to global: that would turn a caller who computed an + * empty entitlement set into a caller who sweeps everything. + */ +export interface DevJobSweepScope { + readonly repoIds?: readonly string[]; +} + export interface ListJobsFilter { repoId?: string; /** Scope to a SET of repos IN SQL (before LIMIT). Use this — not a post-query @@ -876,13 +895,18 @@ export class DevJobStore { // --- reaper / enforcement reads (worker calls finalizeDevJob on these) ---- /** Active jobs whose last sign of life is older than `cutoff` — stalled - * candidates for the worker/reaper. */ - async findStalled(cutoff: Date): Promise { + * candidates for the worker/reaper. Unscoped ⇒ database-global (production); + * pass `scope` to constrain it. See {@link DevJobSweepScope}. */ + async findStalled(cutoff: Date, scope?: DevJobSweepScope): Promise { + // An explicitly empty scope means "nothing", never "everything". + if (scope?.repoIds !== undefined && scope.repoIds.length === 0) return []; + const scoped = scope?.repoIds !== undefined; const r = await this.pool.query( `SELECT ${JOB_COLS} FROM dev_jobs WHERE status IN (${ACTIVE_SET_SQL}) - AND COALESCE(last_heartbeat_at, started_at, claimed_at) < $1`, - [cutoff], + AND COALESCE(last_heartbeat_at, started_at, claimed_at) < $1 + ${scoped ? 'AND repo_id = ANY($2::uuid[])' : ''}`, + scoped ? [cutoff, scope.repoIds] : [cutoff], ); return r.rows.map(toJob); } diff --git a/middleware/src/devplatform/devJobTaskStore.ts b/middleware/src/devplatform/devJobTaskStore.ts index 41bb1dfd..013e42c1 100644 --- a/middleware/src/devplatform/devJobTaskStore.ts +++ b/middleware/src/devplatform/devJobTaskStore.ts @@ -60,7 +60,7 @@ import { type TerminalTaskPatch, } from '@omadia/orchestrator'; -import type { ListJobsFilter } from './devJobStore.js'; +import type { DevJobSweepScope, ListJobsFilter } from './devJobStore.js'; import type { DevJob, DevJobEvent, @@ -145,7 +145,7 @@ export interface DevJobTaskJobStore { provision: number, events: readonly { seq: number; type: string; payload: Record }[], ): Promise; - findStalled(cutoff: Date): Promise; + findStalled(cutoff: Date, scope?: DevJobSweepScope): Promise; /** Optional so a unit-test fake need not implement it; absent ⇒ empty tail. */ listEvents?( jobId: string, @@ -177,6 +177,15 @@ export interface DevJobTaskStoreDeps { olderThanDays: number, now: Date, ) => Promise; + /** + * W3-A — narrows `reapOrphans`' stalled sweep. OMITTED in production, where the + * reaper is deliberately database-global (see {@link DevJobSweepScope}). + * + * Set at CONSTRUCTION rather than per call because `reapOrphans` implements the + * generic {@link TaskStore} contract, whose `TaskReapOptions` must stay free of + * dev-platform concepts like a repo id. + */ + readonly sweepScope?: DevJobSweepScope; } const MS_PER_DAY = 86_400_000; @@ -323,7 +332,10 @@ export function createDevJobTaskStore(deps: DevJobTaskStoreDeps): TaskStore { async reapOrphans(opts: TaskReapOptions): Promise { const now = opts.now ?? new Date(); const cutoff = new Date(now.getTime() - opts.staleAfterMs); - const stalled = await jobStore.findStalled(cutoff); + const stalled = await jobStore.findStalled( + cutoff, + ...(deps.sweepScope ? ([deps.sweepScope] as const) : ([] as const)), + ); let staleFailed = 0; for (const job of stalled) { const finished = await deps.finalize(job.id, 'stalled', { diff --git a/middleware/test/devplatform/devJobTaskStore.pg.test.ts b/middleware/test/devplatform/devJobTaskStore.pg.test.ts index ae70d9ff..a10f06cf 100644 --- a/middleware/test/devplatform/devJobTaskStore.pg.test.ts +++ b/middleware/test/devplatform/devJobTaskStore.pg.test.ts @@ -12,6 +12,7 @@ import { DevJobEventBus } from '../../src/devplatform/devJobEventBus.js'; import { DevJobStore, TERMINAL_FINISH_BRAND, + type DevJobSweepScope, } from '../../src/devplatform/devJobStore.js'; import { createDevJobTaskStore, @@ -74,7 +75,14 @@ describe('devplatform/devJobTaskStore — seam conformance (pg)', { skip: !pgAva return jobStore.finishTerminal(TERMINAL_FINISH_BRAND, jobId, status, patch); } - function seam(overrides: { createJob?: (input: unknown) => Promise } = {}) { + function seam( + overrides: { + createJob?: (input: unknown) => Promise | Promise; + /** W3-A — constrains `reapOrphans`' sweep to this run's repos. */ + sweepScope?: DevJobSweepScope; + purgeTerminalJobs?: (olderThanDays: number, now: Date) => Promise; + } = {}, + ) { return createDevJobTaskStore({ jobStore, createJob: @@ -83,6 +91,10 @@ describe('devplatform/devJobTaskStore — seam conformance (pg)', { skip: !pgAva throw new Error('createJob not wired in this test'); }), finalize, + ...(overrides.sweepScope ? { sweepScope: overrides.sweepScope } : {}), + ...(overrides.purgeTerminalJobs + ? { purgeTerminalJobs: overrides.purgeTerminalJobs } + : {}), }); } @@ -314,18 +326,99 @@ describe('devplatform/devJobTaskStore — seam conformance (pg)', { skip: !pgAva assert.ok(working.every((d) => d.status === 'working')); }); - // NOTE — `reapOrphans` is deliberately NOT exercised here. + // ── reapOrphans, at the pg level (W3-A) ─────────────────────────────────── // - // It delegates to `DevJobStore.findStalled`, whose query is DATABASE-GLOBAL: - // `WHERE status IN (provisioning, running, applying) AND - // COALESCE(last_heartbeat_at, started_at, claimed_at) < $1`, with no tenant - // predicate. A sweep in a shared test cluster therefore finalizes OTHER - // suites' in-flight jobs as `stalled` — which is exactly what happened the - // first time this suite ran it with a forward-dated cutoff: it broke - // `devPlatformPipeline.wire.pg.test.ts`. That is correct production behaviour - // (the real reaper IS global) and an untenable test, so the sweep is covered - // where it can be isolated: `test/tasks/devJobTaskStoreReap.test.ts` drives it - // against a controlled fake `findStalled`, and - // `test/tasks/inMemoryTaskStore.test.ts` pins the reaper semantics against a - // real store with a driven clock. + // This used to be deliberately absent. `DevJobStore.findStalled` was + // database-global with no scope predicate, so a forward-dated cutoff here + // finalized OTHER suites' in-flight jobs as `stalled` — it broke + // `devPlatformPipeline.wire.pg.test.ts` the first time it ran. Global IS the + // correct production behaviour, so the fix was to make the sweep NARROWABLE + // (`DevJobSweepScope`) rather than to change what production does. With the + // scope bound to this run's own repos the sweep is finally testable against + // real rows, so the fake-`findStalled` unit test in + // `test/tasks/devJobTaskStoreReap.test.ts` is no longer the only coverage. + + /** + * A job in the ACTIVE set (`provisioning`) on a KNOWN repo. + * + * NOT via `claimNextQueued`: that pops the oldest queued row DATABASE-WIDE, so + * with concurrent dev-platform pg suites (and this suite's own earlier jobs) it + * cannot be aimed at a repo. The reaper only reads `status` + the heartbeat + * columns, so stamping them is a faithful and deterministic setup. + */ + async function newActiveJob(repoId: string): Promise { + const job = await newQueuedJob(repoId); + await pool.query( + `UPDATE dev_jobs + SET status = 'provisioning', claimed_by = $2, claimed_at = now(), started_at = now() + WHERE id = $1`, + [job.id, randomUUID()], + ); + const active = await jobStore.getJob(job.id); + assert.ok(active && active.status === 'provisioning', 'setup failed to activate the job'); + return active; + } + + it('MUTATION CHECK: reapOrphans finalizes a real stalled job as `stalled`, scoped to its own repos', async () => { + const localRepo = await newRepo(); + const otherRepo = await newRepo(); + const inScope = await newActiveJob(localRepo.id); + const outOfScope = await newActiveJob(otherRepo.id); + + const store = seam({ sweepScope: { repoIds: [localRepo.id] } }); + // Forward-dated `now` ⇒ every active job is past the cutoff. Before the scope + // predicate existed this is precisely what reached across suites. + const result = await store.reapOrphans({ + now: new Date(Date.now() + 3_600_000), + staleAfterMs: 1_000, + purgeTerminalAfterMs: 30 * 86_400_000, + }); + + assert.equal(result.staleFailed, 1, 'exactly the in-scope job was reaped'); + const reaped = await jobStore.getJob(inScope.id); + assert.equal(reaped?.status, 'stalled'); + assert.match(String(reaped?.error), /no worker heartbeat/); + // The load-bearing half: the sibling repo's in-flight job is UNTOUCHED. This + // is the assertion whose absence forced the whole test to be downgraded. + const untouched = await jobStore.getJob(outOfScope.id); + assert.equal(untouched?.status, 'provisioning', 'the sweep escaped its scope'); + assert.equal(untouched?.error, null); + }); + + it('MUTATION CHECK: an EMPTY scope reaps nothing — it must never widen to global', async () => { + const localRepo = await newRepo(); + const active = await newActiveJob(localRepo.id); + + const result = await seam({ sweepScope: { repoIds: [] } }).reapOrphans({ + now: new Date(Date.now() + 3_600_000), + staleAfterMs: 1_000, + purgeTerminalAfterMs: 30 * 86_400_000, + }); + + assert.equal(result.staleFailed, 0, 'an empty scope swept something'); + // A caller who computed an empty entitlement set must not become a caller who + // sweeps everything — `= ANY('{}')` semantics, made explicit. + assert.equal((await jobStore.getJob(active.id))?.status, 'provisioning'); + }); + + it('findStalled without a scope stays DATABASE-GLOBAL (production behaviour)', async () => { + // Production passes no scope and must keep reaching every abandoned job. Both + // repos' active jobs are visible to an unscoped sweep. + const a = await newRepo(); + const b = await newRepo(); + const one = await newActiveJob(a.id); + const two = await newActiveJob(b.id); + + const global = await jobStore.findStalled(new Date(Date.now() + 3_600_000)); + const ids = new Set(global.map((j) => j.id)); + assert.ok(ids.has(one.id) && ids.has(two.id), 'the unscoped sweep lost a repo'); + // …and the scoped call over the same cutoff sees only its own. + const scoped = await jobStore.findStalled(new Date(Date.now() + 3_600_000), { + repoIds: [a.id], + }); + assert.deepEqual( + scoped.map((j) => j.repoId), + [a.id], + ); + }); }); diff --git a/middleware/test/tasks/devJobTaskStoreReap.test.ts b/middleware/test/tasks/devJobTaskStoreReap.test.ts index e78e175f..60e20f35 100644 --- a/middleware/test/tasks/devJobTaskStoreReap.test.ts +++ b/middleware/test/tasks/devJobTaskStoreReap.test.ts @@ -14,7 +14,12 @@ import type { DevJob, DevJobStatus } from '../../src/devplatform/types.js'; /** * W2-2 — the dev_job adapter's orphan sweep, isolated. * - * Why not in the pg suite: `DevJobStore.findStalled` is DATABASE-GLOBAL (no + * W3-A UPDATE: the pg-level sweep is now covered too — + * `test/devplatform/devJobTaskStore.pg.test.ts` drives `reapOrphans` against real + * rows with `DevJobSweepScope` bound to its own repos. This file keeps the + * driven-clock and error-shape coverage the fake makes cheap. Historical note: + * + * Why this was not in the pg suite: `DevJobStore.findStalled` was DATABASE-GLOBAL (no * tenant predicate), so triggering a real sweep in a shared test cluster * finalizes other suites' in-flight jobs as `stalled`. That is correct * production behaviour and an untenable test, so the sweep is driven here From ec04f9adb4cb1887c13b20838328bee29cfb6578 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 13:18:30 +0200 Subject: [PATCH 46/90] feat(mcp): mount /api/v1/mcp behind publicPaths + requireApiKey, dark by default PUBLIC_MCP_ENABLED=false mounts no router at all. Mounted after the /api requireAuth line so the publicPaths entry is load-bearing: losing it makes the route go dark (401), not open. Widens McpCallerKind with 'api_key' and points McpCallLogRow at the shared union instead of a retyped copy. --- middleware/src/config.ts | 17 +++ middleware/src/index.ts | 18 +++ middleware/src/mcp/wirePublicMcp.ts | 229 ++++++++++++++++++++++++++++ 3 files changed, 264 insertions(+) create mode 100644 middleware/src/mcp/wirePublicMcp.ts diff --git a/middleware/src/config.ts b/middleware/src/config.ts index f4afeaef..2d08517f 100644 --- a/middleware/src/config.ts +++ b/middleware/src/config.ts @@ -494,6 +494,23 @@ const ConfigSchema = z.object({ .positive() .default(15_000), + // --- Public MCP endpoint (W2-3, issue #542) ------------------------------ + // omadia's own tools, exposed over a stateless Streamable-HTTP MCP server at + // /api/v1/mcp to third parties holding an API key. This is the highest-blast + // -radius surface in the MCP cluster — an internet-facing route that reaches + // the tool layer, including WRITE tools by operator allowlist — so the whole + // thing is dark by default: false mounts NO router at all, which is a + // stronger guarantee than mounting one that answers 403. + PUBLIC_MCP_ENABLED: devFlag(), + // Whether tool calls are served while the dispatch privacy/trace seam is + // still open. `ToolDispatchService` applies no PII masking — the chat path's + // masking lives in `Orchestrator.dispatchToolInner`, which that dispatcher + // explicitly does not replicate — so serving without the seam means a public + // HTTP response can carry unmasked personal data out of Odoo or M365. + // Default false ⇒ tools/list works, tools/call refuses and says why. + // Set true ONLY on a deliberate, documented operator decision. + PUBLIC_MCP_ALLOW_WITHOUT_PRIVACY_SEAM: devFlag(), + // --- Dev platform (epic #470 W0) ---------------------------------------- // Isolated per-job code runners (clone → agent-edit → diff → server-side PR). // The whole subsystem is dark by default: DEV_PLATFORM_ENABLED=false mounts diff --git a/middleware/src/index.ts b/middleware/src/index.ts index dfb9aabb..bd2bd471 100644 --- a/middleware/src/index.ts +++ b/middleware/src/index.ts @@ -162,6 +162,7 @@ import { type MdnsAdvertisement, } from './pairing/mdns.js'; import { publicPaths } from './auth/publicPaths.js'; +import { mountPublicMcp } from './mcp/wirePublicMcp.js'; import { createRequireAuth } from './auth/requireAuth.js'; import { createOperatorAuthAccessor } from './auth/operatorAuthAccessor.js'; import { assembleDevPlatform, mountDevPlatform } from './devplatform/wireDevPlatform.js'; @@ -2422,6 +2423,23 @@ async function main(): Promise { }), ); + // W2-3 (issue #542) — the public, stateless MCP endpoint. + // + // Mounted AFTER the `/api` requireAuth line above ON PURPOSE. That mount runs + // for every `/api/*` request whichever router answers it, so being listed in + // `auth/publicPaths.ts` is what makes this route reachable at all — and + // losing that entry makes it go DARK (401) rather than open. `requireApiKey` + // inside the router is the actual authentication; the per-key tool allowlist + // and the per-tool write scopes are the actual authorization. + mountPublicMcp(app, requireAuth, { + enabled: config.PUBLIC_MCP_ENABLED, + allowWithoutPrivacySeam: config.PUBLIC_MCP_ALLOW_WITHOUT_PRIVACY_SEAM, + vault: secretVault, + graphPool, + getRegistry, + nativeToolRegistry, + }); + // Chat-sessions CRUD behind `requireAuth` — sessions may contain // PII / tool outputs / code snippets and must not be readable anonymously. // The `/api` mount above already gates this, but the explicit middleware diff --git a/middleware/src/mcp/wirePublicMcp.ts b/middleware/src/mcp/wirePublicMcp.ts new file mode 100644 index 00000000..800f1ba5 --- /dev/null +++ b/middleware/src/mcp/wirePublicMcp.ts @@ -0,0 +1,229 @@ +/** + * W2-3 (issue #542) — assembles and mounts the public MCP endpoint. + * + * A separate wire module (mirroring `devplatform/wireDevPlatform.ts`) for two + * reasons: `index.ts` is already ~4500 lines, and — more usefully — the whole + * assembly becomes testable end-to-end against the real router chain instead of + * a bare `express()` app. Mounting a bare app is exactly the epic #470 bug the + * doc comment at the top of `auth/publicPaths.ts` records: the runner router's + * e2e test built its own app, so the test passed while production 401'd behind + * the blanket `/api` gate. + */ + +import type { Express, RequestHandler } from 'express'; +import type { Pool } from 'pg'; + +import type { ApiKeySecretStorage, ApiKeyStore, AuditLog, RateLimiter } from '@omadia/api-key-auth'; +import { createApiKeyStore, createRateLimiter } from '@omadia/api-key-auth'; +import { AgentGraphStore, ToolDispatchService } from '@omadia/orchestrator'; +import type { NativeToolRegistry, OrchestratorRegistry } from '@omadia/orchestrator'; + +import type { SecretVault } from '../secrets/vault.js'; +import { + createPublicMcpKeyBindingStore, + type PublicMcpKeyBindingStore, +} from './publicMcpKeyBindings.js'; +import { PUBLIC_MCP_PATH, PUBLIC_MCP_SERVER_NAME } from './publicMcpPath.js'; +import { createPublicMcpRouter } from './publicMcpRouter.js'; +import type { PublicMcpAuditEntry, PublicMcpDispatcher } from './publicMcpServer.js'; + +/** + * The vault namespace holding API-key records. + * + * Deliberately the SAME namespace `@omadia/channel-api` writes to (a plugin's + * `ctx.secrets` is its own manifest id), so there is ONE key list and ONE place + * to revoke. A key's SCOPES decide what it can reach: `chat:write` gets the + * chat ingress, `mcp:list`/`mcp:invoke`/`mcp:write:` get this endpoint, + * and a key holding only the former reaches nothing here. A second key store + * would have meant a second revoke an operator can forget. + */ +export const API_KEY_VAULT_NAMESPACE = '@omadia/channel-api'; + +/** + * Adapts the kernel's namespaced vault to the flat `ApiKeySecretStorage` shape. + * + * Read/list ONLY — no `set`, no `delete`. `createApiKeyStore` requires a + * write-capable accessor and throws without one, which is the point: minting + * and revoking keys stays with `@omadia/channel-api`'s operator-session-gated + * `/admin/keys` routes. This module builds a store for VERIFICATION, and giving + * an internet-facing route the ability to write its own credentials is not a + * capability it has any use for. + * + * @see createVerifyOnlyApiKeyStore for how the read-only store is obtained. + */ +function readOnlyVaultStorage(vault: SecretVault): ApiKeySecretStorage { + return { + get: (key) => vault.get(API_KEY_VAULT_NAMESPACE, key), + keys: () => vault.listKeys(API_KEY_VAULT_NAMESPACE), + }; +} + +/** + * An `ApiKeyStore` usable for `verify()` only. + * + * `createApiKeyStore` demands a writer up front, so it cannot be handed the + * read-only accessor above. Rather than widen the endpoint's vault access to + * satisfy a constructor, the two write methods are stubbed to throw: reaching + * either is a programmer error (nothing on this path calls them), and a throw + * is a louder, more debuggable failure than a silent no-op that appears to have + * minted a key. + */ +export function createVerifyOnlyApiKeyStore(vault: SecretVault): ApiKeyStore { + const storage = readOnlyVaultStorage(vault); + return createApiKeyStore({ + ...storage, + set: () => { + throw new Error( + 'public MCP endpoint must not mint API keys — use @omadia/channel-api /admin/keys', + ); + }, + delete: () => { + throw new Error( + 'public MCP endpoint must not revoke API keys — use @omadia/channel-api /admin/keys', + ); + }, + }); +} + +export interface WirePublicMcpDeps { + readonly enabled: boolean; + /** See `PUBLIC_MCP_ALLOW_WITHOUT_PRIVACY_SEAM`. */ + readonly allowWithoutPrivacySeam: boolean; + readonly vault: SecretVault; + /** Bindings and the audit trail both live in the graph DB. Absent ⇒ the + * endpoint is NOT mounted: without bindings every key reaches nothing, and + * without the audit trail a public write would be unattributable. */ + readonly graphPool: Pool | undefined; + /** Resolved LIVE, not captured: the orchestrator plugin republishes the + * registry on reactivation, so a boot-time value would pin a stale set. */ + readonly getRegistry: () => OrchestratorRegistry | undefined; + /** The process-wide native tool registry. Shared across agents by design — + * which is precisely why per-agent reach is decided by the binding row and + * the agent's OWN `listDomainTools()`, not by this registry. */ + readonly nativeToolRegistry: NativeToolRegistry; + readonly log?: (msg: string) => void; + /** Test seams. Production passes neither. */ + readonly bindings?: PublicMcpKeyBindingStore; + readonly apiKeys?: ApiKeyStore; + readonly rateLimiter?: RateLimiter; + readonly keyAuditLog?: AuditLog; +} + +/** + * Builds the per-agent dispatcher. + * + * Mirrors the `ToolDispatchService` construction in `buildOrchestrator.ts`'s + * claude-cli branch, with ONE difference that carries all the isolation: + * `domainToolsProvider` reads THIS agent's orchestrator. Native tools come from + * the process-wide registry (there is no per-agent native registry in omadia), + * so a shared native tool is reachable by any agent — and is kept out of reach + * of a given KEY by the binding allowlist, which is checked before dispatch is + * ever consulted. + * + * Returns `undefined` for an unknown or inactive slug, which fails the call + * closed rather than falling back to the default agent. + */ +function makeDispatcherResolver(deps: WirePublicMcpDeps): (agentId: string) => PublicMcpDispatcher | undefined { + return (agentId) => { + const entry = deps.getRegistry()?.get(agentId); + if (!entry) return undefined; + return new ToolDispatchService({ + nativeTools: deps.nativeToolRegistry, + domainToolsProvider: () => entry.built.orchestrator.listDomainTools(), + }); + }; +} + +/** + * Maps a public MCP call onto an `mcp_call_log` row. + * + * `serverId` is NULL and `serverName` is the `omadia-public-mcp` literal: + * omadia IS the server here, so there is no `mcp_servers` row to point at (0009 + * made the FK nullable for exactly this "no server row" case). `callerKind` is + * the `api_key` member migration 0033 added, and `actingIdentity` reuses 0031's + * vocabulary — `apikey:`, or the literal `unresolved`. + * + * Fire-and-forget: an audit write must never fail a caller's request. It must + * also never be the reason one succeeds, which is why the endpoint is not + * mounted at all without a pool. + */ +export function createPublicMcpAuditSink( + graph: AgentGraphStore, + log: (msg: string) => void, +): (entry: PublicMcpAuditEntry) => void { + return (entry) => { + void graph + .insertMcpCallLog({ + serverId: null, + serverName: PUBLIC_MCP_SERVER_NAME, + toolName: entry.toolName, + callerKind: 'api_key', + callerAgent: entry.agentId, + turnId: null, + ok: entry.ok, + error: entry.error, + durationMs: entry.durationMs, + calledAt: entry.calledAt, + actingIdentity: entry.actingIdentity, + }) + .catch((err: unknown) => { + log(`[public-mcp] audit write failed: ${String(err)}`); + }); + }; +} + +/** + * Mounts the endpoint, or explains why it stayed dark. + * + * `requireAuth` runs FIRST and is load-bearing in both directions: it short- + * circuits to `next()` because `PUBLIC_MCP_PATH` is in `publicPaths.ts`, and if + * that entry were ever removed this route would 401 before `requireApiKey` ran. + * That is the intended failure mode — a path that loses its exemption goes dark + * rather than open — and it is what makes the `publicPaths` test meaningful + * rather than decorative. + */ +export function mountPublicMcp(app: Express, requireAuth: RequestHandler, deps: WirePublicMcpDeps): boolean { + const log = deps.log ?? ((msg: string) => console.log(msg)); + + if (!deps.enabled) { + log('[public-mcp] DISABLED (PUBLIC_MCP_ENABLED=false) — no router mounted'); + return false; + } + if (!deps.graphPool && !deps.bindings) { + log('[public-mcp] NOT mounted — no DATABASE_URL, so there are no key bindings and no audit trail'); + return false; + } + + const bindings = + deps.bindings ?? createPublicMcpKeyBindingStore(deps.graphPool as Pool); + const audit = deps.graphPool + ? createPublicMcpAuditSink(new AgentGraphStore(deps.graphPool), log) + : undefined; + + app.use( + requireAuth, + createPublicMcpRouter({ + apiKeys: deps.apiKeys ?? createVerifyOnlyApiKeyStore(deps.vault), + rateLimiter: deps.rateLimiter ?? createRateLimiter(), + ...(deps.keyAuditLog ? { keyAuditLog: deps.keyAuditLog } : {}), + bindings, + // A SECOND limiter instance, not the one above. Writes get their own + // budget so a read-heavy integration's unused read headroom cannot fund a + // write burst. + writeRateLimiter: createRateLimiter(), + resolveDispatcher: makeDispatcherResolver(deps), + ...(audit ? { audit } : {}), + requirePrivacySeam: !deps.allowWithoutPrivacySeam, + serverName: PUBLIC_MCP_SERVER_NAME, + }), + ); + + log( + `[public-mcp] mounted at POST ${PUBLIC_MCP_PATH} (API-key auth, per-key tool allowlist)${ + deps.allowWithoutPrivacySeam + ? ' ⚠ tool calls ENABLED WITHOUT the dispatch privacy seam — responses may carry unmasked PII' + : ' — tool calls REFUSED until the dispatch privacy seam is wired (tools/list works)' + }`, + ); + return true; +} From e1a5647f2332c55b345bed90370ed94d8ad82e6c Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 13:38:10 +0200 Subject: [PATCH 47/90] test(mcp): 102 tests for the public MCP endpoint against the real middleware chain MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Harness reproduces index.ts's chain (express.json 10mb, the OB-106 /api requireAuth line, mountPublicMcp, the shared publicPaths array) rather than a bare express() app — the epic #470 bug publicPaths.ts documents. Also rejects the bare 'mcp:write' scope, which validated and granted nothing. --- .../harness-api-key-auth/src/apiKeyScopes.ts | 14 + middleware/src/mcp/publicMcpRouter.ts | 13 +- middleware/src/mcp/publicMcpServer.ts | 20 +- middleware/src/mcp/wirePublicMcp.ts | 71 +- middleware/test/publicMcp/harness.ts | 285 +++++++ .../test/publicMcp/publicMcpBodyCap.test.ts | 128 ++++ .../publicMcp/publicMcpEndpoint.e2e.test.ts | 723 ++++++++++++++++++ .../publicMcp/publicMcpKeyBindings.test.ts | 134 ++++ .../test/publicMcp/publicMcpScopes.test.ts | 178 +++++ middleware/test/publicPaths.test.ts | 60 ++ 10 files changed, 1606 insertions(+), 20 deletions(-) create mode 100644 middleware/test/publicMcp/harness.ts create mode 100644 middleware/test/publicMcp/publicMcpBodyCap.test.ts create mode 100644 middleware/test/publicMcp/publicMcpEndpoint.e2e.test.ts create mode 100644 middleware/test/publicMcp/publicMcpKeyBindings.test.ts create mode 100644 middleware/test/publicMcp/publicMcpScopes.test.ts diff --git a/middleware/packages/harness-api-key-auth/src/apiKeyScopes.ts b/middleware/packages/harness-api-key-auth/src/apiKeyScopes.ts index 3c630705..7a0b8195 100644 --- a/middleware/packages/harness-api-key-auth/src/apiKeyScopes.ts +++ b/middleware/packages/harness-api-key-auth/src/apiKeyScopes.ts @@ -96,9 +96,23 @@ const SCOPE_PATTERN = /^[a-z][a-z0-9_-]*:[a-z][a-z0-9_-]*$/; */ const MCP_WRITE_SCOPE_PATTERN = /^mcp:write:[a-z][a-z0-9_-]*$/; +/** + * The bare two-segment `mcp:write`, rejected outright. + * + * It is a perfectly well-formed two-segment scope, so `SCOPE_PATTERN` accepts + * it — and it is the single most likely thing an operator types when they mean + * "let this key write". It would validate, persist, and grant NOTHING (no write + * check ever asks for it), which is indistinguishable from a revoked key at + * debug time. Rejecting it turns a silent misconfiguration into an error at the + * moment of the mistake. There is deliberately no class-wide write scope to + * point them at instead: writes are per tool, by design. + */ +const REJECTED_SCOPES: readonly string[] = ['mcp:write']; + export function isValidScope(value: unknown): value is ApiKeyScope { if (typeof value !== 'string') return false; if (value === WILDCARD_SCOPE) return true; + if (REJECTED_SCOPES.includes(value)) return false; if (MCP_WRITE_SCOPE_PATTERN.test(value)) return true; // Checked LAST and unchanged: a `mcp:write:x` string has two colons and // never matched `SCOPE_PATTERN` anyway, so nothing that used to validate diff --git a/middleware/src/mcp/publicMcpRouter.ts b/middleware/src/mcp/publicMcpRouter.ts index b0cd9f1f..4bc1efb6 100644 --- a/middleware/src/mcp/publicMcpRouter.ts +++ b/middleware/src/mcp/publicMcpRouter.ts @@ -28,6 +28,17 @@ import { requireApiKey } from '@omadia/api-key-auth'; import { PUBLIC_MCP_PATH } from './publicMcpPath.js'; import { PublicMcpServer, type PublicMcpServerDeps } from './publicMcpServer.js'; +/** + * The router handles its mount root, NOT an absolute path. + * + * The caller mounts it at `PUBLIC_MCP_PATH`, so this is `'/'`. Baking the + * absolute path in here as well would make `app.use(PUBLIC_MCP_PATH, router)` + * resolve to `/api/v1/mcp/api/v1/mcp` — and the version that "works", + * `app.use(router)`, would apply the router's own middleware (including + * `requireAuth`, which the caller pairs it with) to EVERY request on the app. + */ +const ROUTER_ROOT = '/'; + export interface PublicMcpRouterDeps extends PublicMcpServerDeps { /** The SAME store the operator mints keys with. Reused rather than * duplicated: a second key store would be a second place to revoke. */ @@ -46,7 +57,7 @@ export function createPublicMcpRouter(deps: PublicMcpRouterDeps): Router { const router = Router(); router.use( - PUBLIC_MCP_PATH, + ROUTER_ROOT, server.bodyCapMiddleware(), requireApiKey({ apiKeys: deps.apiKeys, diff --git a/middleware/src/mcp/publicMcpServer.ts b/middleware/src/mcp/publicMcpServer.ts index 9ebe2210..a0176f3c 100644 --- a/middleware/src/mcp/publicMcpServer.ts +++ b/middleware/src/mcp/publicMcpServer.ts @@ -207,16 +207,26 @@ export class PublicMcpServer { * `express.json({ limit: '10mb' })` before every `/api` router (index.ts), so * by the time a request reaches this one the stream is already consumed and * parsed: a route-level parser would be a silent no-op and the loopback - * server's 8 MB ceiling would quietly become the kernel's 10 MB one. So the - * check runs on the two signals still available after parsing — - * `Content-Length` (the only pre-parse number, sent by any well-behaved - * client) and the re-serialized body length (which covers a chunked upload - * that carries no `Content-Length` at all). + * server's 8 MB ceiling would quietly become the kernel's 10 MB one. * * Mounting this router BEFORE `express.json` would allow a real streaming * cap, but would also put it in front of the `/api` requireAuth mount and * throw away the `publicPaths` half of the defense this route is required to * use. The cap is the cheaper thing to reimplement. + * + * Two checks, and they are NOT two independent gates — be precise about which + * does the work: + * + * - The re-serialized body length is the ACTUAL enforcement. It catches + * every oversized body, including a chunked upload carrying no + * `Content-Length` at all. + * - `Content-Length` is a COST optimization in front of it: re-serializing + * an 8 MB body to measure it is itself expensive, and a client that + * honestly declares an oversized payload can be refused without paying + * that. It is not a security control on its own — a lying header cannot be + * trusted, and one that declares 9 MB while sending 100 bytes never + * reaches this middleware anyway (`express.json` is still waiting for the + * rest of the body). Treat it as the fast path, not as the gate. */ bodyCapMiddleware(): RequestHandler { return (req: Request, res: Response, next): void => { diff --git a/middleware/src/mcp/wirePublicMcp.ts b/middleware/src/mcp/wirePublicMcp.ts index 800f1ba5..e80ce8a7 100644 --- a/middleware/src/mcp/wirePublicMcp.ts +++ b/middleware/src/mcp/wirePublicMcp.ts @@ -85,28 +85,51 @@ export function createVerifyOnlyApiKeyStore(vault: SecretVault): ApiKeyStore { }); } +/** The vault is only needed when the caller did not supply an `ApiKeyStore`; + * throwing here keeps that a wiring error rather than a runtime 500. */ +function requireVault(deps: WirePublicMcpDeps): SecretVault { + if (!deps.vault) { + throw new Error('mountPublicMcp requires a vault (or an explicit apiKeys store)'); + } + return deps.vault; +} + export interface WirePublicMcpDeps { readonly enabled: boolean; /** See `PUBLIC_MCP_ALLOW_WITHOUT_PRIVACY_SEAM`. */ readonly allowWithoutPrivacySeam: boolean; - readonly vault: SecretVault; + /** Only read when `apiKeys` is not supplied. */ + readonly vault?: SecretVault; /** Bindings and the audit trail both live in the graph DB. Absent ⇒ the * endpoint is NOT mounted: without bindings every key reaches nothing, and * without the audit trail a public write would be unattributable. */ readonly graphPool: Pool | undefined; /** Resolved LIVE, not captured: the orchestrator plugin republishes the - * registry on reactivation, so a boot-time value would pin a stale set. */ - readonly getRegistry: () => OrchestratorRegistry | undefined; + * registry on reactivation, so a boot-time value would pin a stale set. + * Only read when `resolveDispatcher` is not supplied. */ + readonly getRegistry?: () => OrchestratorRegistry | undefined; /** The process-wide native tool registry. Shared across agents by design — * which is precisely why per-agent reach is decided by the binding row and - * the agent's OWN `listDomainTools()`, not by this registry. */ - readonly nativeToolRegistry: NativeToolRegistry; + * the agent's OWN `listDomainTools()`, not by this registry. Only read when + * `resolveDispatcher` is not supplied. */ + readonly nativeToolRegistry?: NativeToolRegistry; readonly log?: (msg: string) => void; - /** Test seams. Production passes neither. */ + + // ── Test seams. Production supplies none of these. ──────────────────────── + // Each one substitutes an INFRASTRUCTURE dependency (a pool, a vault, the + // orchestrator registry), never a GATE: the allowlist check, the scope + // checks, the rate limits and the audit calls all run exactly as they do in + // production, which is what lets the e2e tests assert on real refusals. readonly bindings?: PublicMcpKeyBindingStore; readonly apiKeys?: ApiKeyStore; readonly rateLimiter?: RateLimiter; readonly keyAuditLog?: AuditLog; + readonly resolveDispatcher?: (agentId: string) => PublicMcpDispatcher | undefined; + readonly audit?: (entry: PublicMcpAuditEntry) => void; + readonly toolTimeoutMs?: number; + readonly maxConcurrentCalls?: number; + /** Second limiter instance. Defaults to a fresh one — never the read one. */ + readonly writeRateLimiter?: RateLimiter; } /** @@ -123,12 +146,21 @@ export interface WirePublicMcpDeps { * Returns `undefined` for an unknown or inactive slug, which fails the call * closed rather than falling back to the default agent. */ -function makeDispatcherResolver(deps: WirePublicMcpDeps): (agentId: string) => PublicMcpDispatcher | undefined { +function makeDispatcherResolver( + deps: WirePublicMcpDeps, +): (agentId: string) => PublicMcpDispatcher | undefined { + if (deps.resolveDispatcher) return deps.resolveDispatcher; + const { getRegistry, nativeToolRegistry } = deps; + if (!getRegistry || !nativeToolRegistry) { + throw new Error( + 'mountPublicMcp requires getRegistry + nativeToolRegistry (or an explicit resolveDispatcher)', + ); + } return (agentId) => { - const entry = deps.getRegistry()?.get(agentId); + const entry = getRegistry()?.get(agentId); if (!entry) return undefined; return new ToolDispatchService({ - nativeTools: deps.nativeToolRegistry, + nativeTools: nativeToolRegistry, domainToolsProvider: () => entry.built.orchestrator.listDomainTools(), }); }; @@ -196,25 +228,36 @@ export function mountPublicMcp(app: Express, requireAuth: RequestHandler, deps: const bindings = deps.bindings ?? createPublicMcpKeyBindingStore(deps.graphPool as Pool); - const audit = deps.graphPool - ? createPublicMcpAuditSink(new AgentGraphStore(deps.graphPool), log) - : undefined; + const audit = + deps.audit ?? + (deps.graphPool + ? createPublicMcpAuditSink(new AgentGraphStore(deps.graphPool), log) + : undefined); + const apiKeys = deps.apiKeys ?? createVerifyOnlyApiKeyStore(requireVault(deps)); + // Scoped to the ONE path, not `app.use(requireAuth, …)`. An unscoped mount + // would run `requireAuth` for every request on the whole app — including the + // non-`/api` surfaces (`/health`, static assets) that were never behind it. app.use( + PUBLIC_MCP_PATH, requireAuth, createPublicMcpRouter({ - apiKeys: deps.apiKeys ?? createVerifyOnlyApiKeyStore(deps.vault), + apiKeys, rateLimiter: deps.rateLimiter ?? createRateLimiter(), ...(deps.keyAuditLog ? { keyAuditLog: deps.keyAuditLog } : {}), bindings, // A SECOND limiter instance, not the one above. Writes get their own // budget so a read-heavy integration's unused read headroom cannot fund a // write burst. - writeRateLimiter: createRateLimiter(), + writeRateLimiter: deps.writeRateLimiter ?? createRateLimiter(), resolveDispatcher: makeDispatcherResolver(deps), ...(audit ? { audit } : {}), requirePrivacySeam: !deps.allowWithoutPrivacySeam, serverName: PUBLIC_MCP_SERVER_NAME, + ...(deps.toolTimeoutMs !== undefined ? { toolTimeoutMs: deps.toolTimeoutMs } : {}), + ...(deps.maxConcurrentCalls !== undefined + ? { maxConcurrentCalls: deps.maxConcurrentCalls } + : {}), }), ); diff --git a/middleware/test/publicMcp/harness.ts b/middleware/test/publicMcp/harness.ts new file mode 100644 index 00000000..8f2edb49 --- /dev/null +++ b/middleware/test/publicMcp/harness.ts @@ -0,0 +1,285 @@ +/** + * W2-3 (issue #542) — shared harness for the public MCP endpoint's e2e tests. + * + * ─── Why this reproduces the real chain instead of a bare `express()` app ──── + * + * The doc comment at the top of `src/auth/publicPaths.ts` records the bug this + * exists to avoid: epic #470's runner router was mounted without a session + * guard, and its e2e test built its OWN bare `express()` app to prove it — so + * the test passed while the route 401'd in production behind the blanket `/api` + * guard. A test app that omits the guard proves nothing about a route whose + * reachability depends on it. + * + * So this harness assembles, in order, exactly what `src/index.ts` assembles: + * + * 1. `express.json({ limit: '10mb' })` — the same global parser, which is why + * the 8 MB cap cannot be an `express.json` limit (see `bodyCapMiddleware`). + * 2. `app.use('/api', requireAuth, )` — the OB-106 line. It runs for + * EVERY `/api/*` request whichever router answers, which is what makes the + * `publicPaths` entry load-bearing. + * 3. `mountPublicMcp(app, requireAuth, …)` — the SAME function index.ts calls, + * not a hand-rolled equivalent. + * 4. `createRequireAuth({ publicPaths: publicPaths({ … }) })` — the SAME + * shared allowlist production runs. + * + * `withoutPublicPathEntry` drives the negative half: strip the entry and the + * route must go DARK (401), never open. + */ + +import type { AddressInfo } from 'node:net'; +import { createServer, type Server } from 'node:http'; + +import express, { type Express } from 'express'; + +import type { ApiKeyRecord, ApiKeyStore, ApiKeyScope } from '@omadia/api-key-auth'; +import { createRateLimiter, sha256Hex } from '@omadia/api-key-auth'; +import type { DispatchableToolSpec, ToolDispatchResult } from '@omadia/orchestrator'; + +import { publicPaths, STATIC_PUBLIC_PATHS } from '../../src/auth/publicPaths.js'; +import { createRequireAuth } from '../../src/auth/requireAuth.js'; +import { EmailWhitelist } from '../../src/auth/whitelist.js'; +import { createInMemoryPublicMcpKeyBindingStore } from '../../src/mcp/publicMcpKeyBindings.js'; +import { PUBLIC_MCP_PATH } from '../../src/mcp/publicMcpPath.js'; +import { mountPublicMcp } from '../../src/mcp/wirePublicMcp.js'; +import type { + PublicMcpAuditEntry, + PublicMcpDispatcher, +} from '../../src/mcp/publicMcpServer.js'; + +export const MCP_ACCEPT = 'application/json, text/event-stream'; + +/** Reads either a plain JSON body or the SSE framing the transport may use. */ +export function parseMcpJson(text: string): Record { + const trimmed = text.trim(); + if (!trimmed.startsWith('event:') && !trimmed.startsWith('data:')) { + return JSON.parse(trimmed) as Record; + } + const data = trimmed + .split(/\r?\n/) + .filter((line) => line.startsWith('data:')) + .map((line) => line.slice('data:'.length).trim()) + .filter(Boolean); + return JSON.parse(data.join('\n')) as Record; +} + +/** True when the sandbox refuses loopback listeners, so callers can self-skip. */ +export function isSandboxListenDenied(error: unknown): boolean { + return error instanceof Error && 'code' in error && error.code === 'EPERM'; +} + +export interface FakeKey { + readonly token: string; + readonly id: string; + readonly scopes: readonly ApiKeyScope[]; + readonly rateLimitPerMinute?: number; +} + +/** + * An `ApiKeyStore` over an in-memory key list. + * + * Only `verify` is reachable from the endpoint; the mutators throw so a test + * that accidentally exercises a write path fails loudly rather than silently + * "succeeding". Verification hashes the token the same way `apiKeyToken.ts` + * does, so a token/record mismatch fails here the way it would in production. + */ +export function fakeApiKeyStore(keys: readonly FakeKey[]): ApiKeyStore { + const records: ApiKeyRecord[] = keys.map((k) => ({ + id: k.id, + hash: sha256Hex(k.token), + rateLimitPerMinute: k.rateLimitPerMinute ?? 60, + scopes: k.scopes, + createdAt: Date.now(), + })); + return { + create: () => Promise.reject(new Error('not used')), + list: () => Promise.reject(new Error('not used')), + revoke: () => Promise.reject(new Error('not used')), + verify: (token) => + Promise.resolve(records.find((r) => r.hash === sha256Hex(String(token)))), + }; +} + +export interface FakeTool { + readonly name: string; + /** Called on dispatch. Default returns a deterministic marker. */ + readonly handle?: (input: unknown) => Promise; +} + +/** A dispatcher for ONE agent, advertising exactly `tools`. */ +export function fakeDispatcher( + tools: readonly FakeTool[], + seen?: { name: string; input: unknown }[], +): PublicMcpDispatcher { + const specs: DispatchableToolSpec[] = tools.map((t) => ({ + name: t.name, + description: `desc:${t.name}`, + input_schema: { type: 'object' as const, properties: {} }, + })); + return { + listDispatchableToolSpecs: () => specs, + async dispatch(name, input) { + seen?.push({ name, input }); + const tool = tools.find((t) => t.name === name); + if (!tool) return { content: `Error: unknown tool \`${name}\`.`, isError: true }; + if (tool.handle) return tool.handle(input); + return { content: `dispatched:${name}` }; + }, + }; +} + +export interface HarnessOptions { + readonly keys: readonly FakeKey[]; + /** Raw binding rows — normalized by the production code path, not bypassed. */ + readonly bindingRows: readonly Record[]; + /** agentId → dispatcher. An agent absent here is "not active". */ + readonly dispatchers: Readonly>; + /** Default true, matching production's fail-closed default. */ + readonly allowWithoutPrivacySeam?: boolean; + /** + * Strips `PUBLIC_MCP_PATH` from the allowlist handed to `requireAuth`, to + * prove the entry is load-bearing rather than decorative. + */ + readonly withoutPublicPathEntry?: boolean; + readonly audit?: PublicMcpAuditEntry[]; + readonly toolTimeoutMs?: number; + readonly maxConcurrentCalls?: number; +} + +export interface Harness { + readonly url: string; + readonly app: Express; + readonly mounted: boolean; + post(body: unknown, opts?: { token?: string; headers?: Record }): Promise; + rpc( + body: unknown, + opts?: { token?: string }, + ): Promise<{ status: number; payload: Record }>; + close(): Promise; +} + +/** A real session-cookie signer, so the "no cookie" 401 is the production one. */ +const SESSION_KEY = new TextEncoder().encode('harness-session-signing-key-32bytes!!'); + +export async function startHarness(opts: HarnessOptions): Promise { + const app = express(); + app.set('trust proxy', true); + + // (1) The SAME global parser index.ts installs. Its 10mb limit is why the + // endpoint's own 8 MB ceiling is enforced post-parse. + app.use(express.json({ limit: '10mb' })); + + const allowlist = opts.withoutPublicPathEntry + ? STATIC_PUBLIC_PATHS.filter((re) => !re.test(PUBLIC_MCP_PATH)) + : publicPaths({ devEndpointsEnabled: false }); + + const requireAuth = createRequireAuth({ + signingKey: SESSION_KEY, + whitelist: new EmailWhitelist('operator@example.com'), + publicPaths: allowlist, + }); + + // (2) The OB-106 line: requireAuth for every /api/* request, whichever router + // ultimately answers. The trailing router is a stand-in for createChatRouter — + // what matters is that the guard runs first for the whole prefix. + app.use('/api', requireAuth, express.Router()); + + // (3) The production wire function, not a re-implementation. + const audit = opts.audit; + const mounted = mountPublicMcp(app, requireAuth, { + enabled: true, + allowWithoutPrivacySeam: opts.allowWithoutPrivacySeam ?? true, + graphPool: undefined, + log: () => {}, + apiKeys: fakeApiKeyStore(opts.keys), + rateLimiter: createRateLimiter(), + bindings: createInMemoryPublicMcpKeyBindingStore( + opts.bindingRows as Parameters[0], + ), + resolveDispatcher: (agentId) => opts.dispatchers[agentId], + ...(audit ? { audit: (entry: PublicMcpAuditEntry) => audit.push(entry) } : {}), + ...(opts.toolTimeoutMs !== undefined ? { toolTimeoutMs: opts.toolTimeoutMs } : {}), + ...(opts.maxConcurrentCalls !== undefined + ? { maxConcurrentCalls: opts.maxConcurrentCalls } + : {}), + }); + + const server: Server = createServer(app); + await new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(0, '127.0.0.1', () => { + server.removeListener('error', reject); + resolve(); + }); + }); + const { port } = server.address() as AddressInfo; + const url = `http://127.0.0.1:${String(port)}${PUBLIC_MCP_PATH}`; + + async function post( + body: unknown, + o?: { token?: string; headers?: Record }, + ): Promise { + return fetch(url, { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + Accept: MCP_ACCEPT, + ...(o?.token ? { Authorization: `Bearer ${o.token}` } : {}), + ...(o?.headers ?? {}), + }, + body: typeof body === 'string' ? body : JSON.stringify(body), + }); + } + + return { + url, + app, + mounted, + post, + async rpc(body, o) { + const res = await post(body, o); + const text = await res.text(); + return { + status: res.status, + payload: text.length > 0 ? parseMcpJson(text) : {}, + }; + }, + close: () => + new Promise((resolve) => { + server.close(() => resolve()); + }), + }; +} + +/** A `tools/list` JSON-RPC request. Stateless: no `initialize`, no session id. */ +export function listToolsRequest(id = 1): unknown { + return { jsonrpc: '2.0', method: 'tools/list', params: {}, id }; +} + +/** A `tools/call` JSON-RPC request. */ +export function callToolRequest(name: string, args: unknown = {}, id = 2): unknown { + return { jsonrpc: '2.0', method: 'tools/call', params: { name, arguments: args }, id }; +} + +/** Tool names from a `tools/list` reply, or `undefined` when it errored. */ +export function toolNames(payload: Record): string[] | undefined { + const result = payload['result'] as { tools?: { name: string }[] } | undefined; + return result?.tools?.map((t) => t.name); +} + +/** The JSON-RPC error message, or undefined when the reply succeeded. */ +export function rpcErrorMessage(payload: Record): string | undefined { + const err = payload['error'] as { message?: string } | undefined; + if (err?.message !== undefined) return err.message; + // A tool-level failure surfaces as a successful RPC with isError set. + const result = payload['result'] as + | { isError?: boolean; content?: { text?: string }[] } + | undefined; + if (result?.isError) return result.content?.[0]?.text; + return undefined; +} + +/** The text content of a successful `tools/call` reply. */ +export function callResultText(payload: Record): string | undefined { + const result = payload['result'] as { content?: { text?: string }[] } | undefined; + return result?.content?.[0]?.text; +} diff --git a/middleware/test/publicMcp/publicMcpBodyCap.test.ts b/middleware/test/publicMcp/publicMcpBodyCap.test.ts new file mode 100644 index 00000000..22a408eb --- /dev/null +++ b/middleware/test/publicMcp/publicMcpBodyCap.test.ts @@ -0,0 +1,128 @@ +import { strict as assert } from 'node:assert'; +import { describe, it } from 'node:test'; + +import type { Request, Response } from 'express'; + +import { + MAX_REQUEST_BYTES, + PublicMcpServer, + type PublicMcpServerDeps, +} from '../../src/mcp/publicMcpServer.js'; +import { createInMemoryPublicMcpKeyBindingStore } from '../../src/mcp/publicMcpKeyBindings.js'; + +/** + * W2-3 (issue #542) — the 8 MB body cap, at the honest boundary. + * + * The `Content-Length` half of `bodyCapMiddleware` cannot be driven end-to-end: + * a header declaring 8-10 MB while sending a small body leaves the kernel's + * global `express.json` waiting for the rest of the body, and `fetch` refuses to + * send a mismatched `Content-Length` at all. So it is exercised here against the + * middleware itself, which is exactly the unit that owns the decision. The + * actual-size half — the real enforcement — is additionally driven end-to-end in + * `publicMcpEndpoint.e2e.test.ts`. + */ + +function server(): PublicMcpServer { + const deps: PublicMcpServerDeps = { + resolveDispatcher: () => undefined, + bindings: createInMemoryPublicMcpKeyBindingStore([]), + writeRateLimiter: { tryConsume: () => true }, + }; + return new PublicMcpServer(deps); +} + +interface Captured { + status?: number; + body?: unknown; + nextCalled: boolean; +} + +function run(req: Partial): Captured { + const captured: Captured = { nextCalled: false }; + const res = { + status(code: number) { + captured.status = code; + return this as unknown as Response; + }, + json(body: unknown) { + captured.body = body; + return this as unknown as Response; + }, + } as unknown as Response; + server().bodyCapMiddleware()( + { headers: {}, ...req } as Request, + res, + () => { + captured.nextCalled = true; + }, + ); + return captured; +} + +describe('public MCP body cap', () => { + it('passes a small body through', () => { + const out = run({ headers: { 'content-length': '42' }, body: { jsonrpc: '2.0' } }); + assert.equal(out.nextCalled, true); + assert.equal(out.status, undefined); + }); + + it('passes a body with no Content-Length at all', () => { + const out = run({ body: { jsonrpc: '2.0' } }); + assert.equal(out.nextCalled, true); + }); + + it('passes a request with no body at all', () => { + const out = run({}); + assert.equal(out.nextCalled, true); + }); + + it('413s on a declared Content-Length over the cap', () => { + const out = run({ + headers: { 'content-length': String(MAX_REQUEST_BYTES + 1) }, + body: { jsonrpc: '2.0' }, + }); + assert.equal(out.nextCalled, false); + assert.equal(out.status, 413); + assert.deepEqual(out.body, { + jsonrpc: '2.0', + error: { code: 413, message: 'Payload Too Large' }, + id: null, + }); + }); + + it('allows a declared Content-Length exactly AT the cap', () => { + const out = run({ + headers: { 'content-length': String(MAX_REQUEST_BYTES) }, + body: { jsonrpc: '2.0' }, + }); + assert.equal(out.nextCalled, true); + }); + + /** The real enforcement: an oversized body with NO honest header. */ + it('413s on an oversized body even when Content-Length is absent', () => { + const out = run({ body: { blob: 'x'.repeat(MAX_REQUEST_BYTES + 1024) } }); + assert.equal(out.nextCalled, false); + assert.equal(out.status, 413); + }); + + /** A lying header must not be able to smuggle an oversized body through. */ + it('413s on an oversized body that declares a small Content-Length', () => { + const out = run({ + headers: { 'content-length': '10' }, + body: { blob: 'x'.repeat(MAX_REQUEST_BYTES + 1024) }, + }); + assert.equal(out.nextCalled, false); + assert.equal(out.status, 413); + }); + + it('ignores a non-numeric Content-Length and falls back to the body size', () => { + assert.equal(run({ headers: { 'content-length': 'banana' }, body: { a: 1 } }).nextCalled, true); + assert.equal( + run({ + headers: { 'content-length': 'banana' }, + body: { blob: 'x'.repeat(MAX_REQUEST_BYTES + 1024) }, + }).status, + 413, + ); + }); +}); diff --git a/middleware/test/publicMcp/publicMcpEndpoint.e2e.test.ts b/middleware/test/publicMcp/publicMcpEndpoint.e2e.test.ts new file mode 100644 index 00000000..0d7f121f --- /dev/null +++ b/middleware/test/publicMcp/publicMcpEndpoint.e2e.test.ts @@ -0,0 +1,723 @@ +import { strict as assert } from 'node:assert'; +import { afterEach, describe, it } from 'node:test'; + +import { MCP_INVOKE_SCOPE, MCP_LIST_SCOPE, mcpWriteScope, WILDCARD_SCOPE } from '@omadia/api-key-auth'; + +import { MAX_REQUEST_BYTES } from '../../src/mcp/publicMcpServer.js'; +import { PUBLIC_MCP_PATH } from '../../src/mcp/publicMcpPath.js'; +import type { PublicMcpAuditEntry } from '../../src/mcp/publicMcpServer.js'; +import { + callResultText, + callToolRequest, + fakeDispatcher, + isSandboxListenDenied, + listToolsRequest, + rpcErrorMessage, + startHarness, + toolNames, + type Harness, + type HarnessOptions, +} from './harness.js'; + +/** + * W2-3 (issue #542) — end-to-end proof for the public, stateless MCP endpoint. + * + * Driven against the REAL middleware chain (`express.json` at 10mb → the OB-106 + * `/api` requireAuth line → `mountPublicMcp`, the same function index.ts calls + * → the shared `publicPaths` allowlist). See `harness.ts` for why a bare + * `express()` app would have proven nothing. + * + * Every request below is stateless: no `initialize`, no `notifications/ + * initialized`, no `Mcp-Session-Id`. That is the premise of the issue — any + * process must be able to answer any request — and it is asserted directly in + * the "statelessness" block. + */ + +const READ_TOOL = 'query_crm'; +const WRITE_TOOL = 'create_lead'; +const OTHER_TOOL = 'query_hr'; + +const KEY_TOKEN = 'omadia_ak_test_token_aaaaaaaaaaaaaaaa'; +const KEY_ID = 'key-sales'; + +/** + * The endpoint advertises name-SORTED, so the wire order is a property of the + * server rather than of whichever order the binding or the dispatcher happened + * to iterate in (plugin load order and `created_at` row order differ across + * machines). Derived here rather than hand-listed, so this stays correct if the + * tool names above are ever renamed. + */ +const SORTED_TOOLS = [READ_TOOL, WRITE_TOOL].slice().sort(); + +function baseOptions(overrides: Partial = {}): HarnessOptions { + return { + keys: [ + { + token: KEY_TOKEN, + id: KEY_ID, + scopes: [MCP_LIST_SCOPE, MCP_INVOKE_SCOPE, mcpWriteScope(WRITE_TOOL)], + }, + ], + bindingRows: [ + { + key_id: KEY_ID, + agent_id: 'sales', + read_tools: [READ_TOOL], + write_tools: [WRITE_TOOL], + write_rate_limit_per_minute: 5, + enabled: true, + }, + ], + dispatchers: { + sales: fakeDispatcher([{ name: READ_TOOL }, { name: WRITE_TOOL }]), + }, + ...overrides, + }; +} + +describe('public MCP endpoint', () => { + let harness: Harness | undefined; + + afterEach(async () => { + await harness?.close(); + harness = undefined; + }); + + async function start(opts: HarnessOptions, t: { skip: (m: string) => void }): Promise { + try { + harness = await startHarness(opts); + return harness; + } catch (error) { + if (isSandboxListenDenied(error)) { + t.skip('sandbox blocks loopback listeners on 127.0.0.1'); + return undefined; + } + throw error; + } + } + + // ── authentication ──────────────────────────────────────────────────────── + + it('401s with no Authorization header', async (t) => { + const h = await start(baseOptions(), t); + if (!h) return; + const res = await h.post(listToolsRequest()); + assert.equal(res.status, 401); + const body = (await res.json()) as { error?: string }; + assert.equal(body.error, 'unauthorized'); + }); + + it('401s with an unknown bearer token', async (t) => { + const h = await start(baseOptions(), t); + if (!h) return; + const res = await h.post(listToolsRequest(), { token: 'omadia_ak_wrong_token_bbbbbbbbbbbb' }); + assert.equal(res.status, 401); + }); + + /** + * The `publicPaths` entry is LOAD-BEARING, and its failure mode must be dark + * rather than open. Strip it and the blanket `/api` requireAuth line answers + * 401 with the SESSION gate's error shape (`{code}`), never reaching + * `requireApiKey` (`{error}`) — which is how we know the two are distinct + * layers and not one duplicated check. + */ + it('goes DARK (session 401), not open, when the publicPaths entry is removed', async (t) => { + const h = await start(baseOptions({ withoutPublicPathEntry: true }), t); + if (!h) return; + const res = await h.post(listToolsRequest(), { token: KEY_TOKEN }); + assert.equal(res.status, 401); + const body = (await res.json()) as { code?: string; error?: string }; + assert.equal(body.error, undefined, 'must not have reached requireApiKey'); + assert.ok(body.code?.startsWith('auth.'), `expected a session-gate code, got ${JSON.stringify(body)}`); + }); + + // ── method + payload limits ─────────────────────────────────────────────── + + /** A per-request transport LEAKS on GET: an SSE stream never ends, so + * `handleRequest` never resolves and the teardown `finally` never runs. */ + it('405s a non-POST request and advertises Allow: POST', async (t) => { + const h = await start(baseOptions(), t); + if (!h) return; + const res = await fetch(h.url, { + method: 'GET', + headers: { Authorization: `Bearer ${KEY_TOKEN}`, Accept: 'text/event-stream' }, + }); + assert.equal(res.status, 405); + assert.equal(res.headers.get('allow'), 'POST'); + await res.text(); + }); + + it('413s a body over the 8 MB cap', async (t) => { + const h = await start(baseOptions(), t); + if (!h) return; + // Over 8 MB, under the kernel's 10 MB express.json limit, so this exercises + // THIS cap rather than express's. + const big = 'x'.repeat(MAX_REQUEST_BYTES + 1024); + const res = await h.post(callToolRequest(READ_TOOL, { blob: big }), { token: KEY_TOKEN }); + assert.equal(res.status, 413); + await res.text(); + }); + + // The `Content-Length` fast path is not reachable end-to-end: a header + // declaring 8-10 MB while sending a small body leaves `express.json` waiting + // for the rest, and `fetch` will not send a mismatched one anyway. It is unit- + // tested directly in `publicMcpBodyCap.test.ts`, where the honest boundary is. + + it('serves a normal-sized body', async (t) => { + const h = await start(baseOptions(), t); + if (!h) return; + const { status, payload } = await h.rpc(listToolsRequest(), { token: KEY_TOKEN }); + assert.equal(status, 200); + assert.deepEqual(toolNames(payload), SORTED_TOOLS); + }); + + // ── statelessness ───────────────────────────────────────────────────────── + + /** + * THE headline guarantee. Two sequential `tools/call`s, each answered by a + * FRESH `Server` + transport pair, with no session header on either. A shared + * transport makes only the first work — the SDK throws "Stateless transport + * cannot be reused across requests" — so a regression here shows up as the + * SECOND call failing while the first still passes. + */ + it('answers two sequential tools/call requests with no session header', async (t) => { + const h = await start(baseOptions(), t); + if (!h) return; + + const first = await h.rpc(callToolRequest(READ_TOOL, { q: 'one' }, 10), { token: KEY_TOKEN }); + assert.equal(first.status, 200, `first call failed: ${JSON.stringify(first.payload)}`); + assert.equal(callResultText(first.payload), `dispatched:${READ_TOOL}`); + + const second = await h.rpc(callToolRequest(READ_TOOL, { q: 'two' }, 11), { token: KEY_TOKEN }); + assert.equal(second.status, 200, `second call failed: ${JSON.stringify(second.payload)}`); + assert.equal(callResultText(second.payload), `dispatched:${READ_TOOL}`); + }); + + it('never issues an Mcp-Session-Id', async (t) => { + const h = await start(baseOptions(), t); + if (!h) return; + const res = await h.post(listToolsRequest(), { token: KEY_TOKEN }); + assert.equal(res.headers.get('mcp-session-id'), null); + await res.text(); + }); + + // ── tools/list must leak nothing ────────────────────────────────────────── + + it('errors tools/list for a key without mcp:list, leaking no names', async (t) => { + const h = await start( + baseOptions({ + keys: [{ token: KEY_TOKEN, id: KEY_ID, scopes: [MCP_INVOKE_SCOPE] }], + }), + t, + ); + if (!h) return; + const { payload } = await h.rpc(listToolsRequest(), { token: KEY_TOKEN }); + const message = rpcErrorMessage(payload) ?? ''; + assert.match(message, new RegExp(MCP_LIST_SCOPE)); + assert.doesNotMatch(message, new RegExp(READ_TOOL)); + assert.doesNotMatch(message, new RegExp(WRITE_TOOL)); + }); + + /** `tools/list` returns the CALLABLE set, not the visible-in-principle set. + * A key that can list but not invoke can call nothing, so it sees nothing. */ + it('returns an EMPTY list for a key with mcp:list but no mcp:invoke', async (t) => { + const h = await start( + baseOptions({ keys: [{ token: KEY_TOKEN, id: KEY_ID, scopes: [MCP_LIST_SCOPE] }] }), + t, + ); + if (!h) return; + const { payload } = await h.rpc(listToolsRequest(), { token: KEY_TOKEN }); + assert.deepEqual(toolNames(payload), []); + }); + + it('hides a write tool from tools/list when the per-tool write scope is absent', async (t) => { + const h = await start( + baseOptions({ + keys: [{ token: KEY_TOKEN, id: KEY_ID, scopes: [MCP_LIST_SCOPE, MCP_INVOKE_SCOPE] }], + }), + t, + ); + if (!h) return; + const { payload } = await h.rpc(listToolsRequest(), { token: KEY_TOKEN }); + assert.deepEqual(toolNames(payload), [READ_TOOL]); + }); + + /** `*` grants list and invoke, and must NOT reveal a write tool. */ + it('WILDCARD_SCOPE does not reveal a write tool in tools/list', async (t) => { + const h = await start( + baseOptions({ keys: [{ token: KEY_TOKEN, id: KEY_ID, scopes: [WILDCARD_SCOPE] }] }), + t, + ); + if (!h) return; + const { payload } = await h.rpc(listToolsRequest(), { token: KEY_TOKEN }); + assert.deepEqual(toolNames(payload), [READ_TOOL]); + }); + + it('never lists a tool the agent advertises but the binding omits', async (t) => { + const h = await start( + baseOptions({ + dispatchers: { + sales: fakeDispatcher([{ name: READ_TOOL }, { name: WRITE_TOOL }, { name: 'secret_tool' }]), + }, + }), + t, + ); + if (!h) return; + const { payload } = await h.rpc(listToolsRequest(), { token: KEY_TOKEN }); + assert.deepEqual(toolNames(payload), SORTED_TOOLS); + }); + + it('never lists a binding entry the agent does not advertise', async (t) => { + const h = await start( + baseOptions({ + bindingRows: [ + { + key_id: KEY_ID, + agent_id: 'sales', + read_tools: [READ_TOOL, 'tool_that_does_not_exist'], + write_tools: [WRITE_TOOL], + write_rate_limit_per_minute: 5, + enabled: true, + }, + ], + }), + t, + ); + if (!h) return; + const { payload } = await h.rpc(listToolsRequest(), { token: KEY_TOKEN }); + assert.deepEqual(toolNames(payload), SORTED_TOOLS); + }); + + it('returns an empty list for a key with no binding at all', async (t) => { + const h = await start(baseOptions({ bindingRows: [] }), t); + if (!h) return; + const { payload } = await h.rpc(listToolsRequest(), { token: KEY_TOKEN }); + assert.deepEqual(toolNames(payload), []); + }); + + // ── tools/call authorization ────────────────────────────────────────────── + + it('calls an allowlisted read tool', async (t) => { + const seen: { name: string; input: unknown }[] = []; + const h = await start( + baseOptions({ + dispatchers: { sales: fakeDispatcher([{ name: READ_TOOL }, { name: WRITE_TOOL }], seen) }, + }), + t, + ); + if (!h) return; + const { payload } = await h.rpc(callToolRequest(READ_TOOL, { q: 'x' }), { token: KEY_TOKEN }); + assert.equal(callResultText(payload), `dispatched:${READ_TOOL}`); + assert.deepEqual(seen, [{ name: READ_TOOL, input: { q: 'x' } }]); + }); + + it('calls a write tool when the per-tool write scope is present', async (t) => { + const h = await start(baseOptions(), t); + if (!h) return; + const { payload } = await h.rpc(callToolRequest(WRITE_TOOL), { token: KEY_TOKEN }); + assert.equal(callResultText(payload), `dispatched:${WRITE_TOOL}`); + }); + + /** THE merge blocker: `mcp:invoke` is not sufficient for a write. */ + it('refuses a write tool for a key holding only mcp:invoke — and never dispatches it', async (t) => { + const seen: { name: string; input: unknown }[] = []; + const h = await start( + baseOptions({ + keys: [{ token: KEY_TOKEN, id: KEY_ID, scopes: [MCP_LIST_SCOPE, MCP_INVOKE_SCOPE] }], + dispatchers: { sales: fakeDispatcher([{ name: READ_TOOL }, { name: WRITE_TOOL }], seen) }, + }), + t, + ); + if (!h) return; + const { payload } = await h.rpc(callToolRequest(WRITE_TOOL), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(payload) ?? '', /not available to this API key/); + assert.deepEqual(seen, [], 'the tool must never have been dispatched'); + }); + + /** THE second merge blocker: `*` does not grant a write. */ + it('refuses a write tool for a WILDCARD_SCOPE key — and never dispatches it', async (t) => { + const seen: { name: string; input: unknown }[] = []; + const h = await start( + baseOptions({ + keys: [{ token: KEY_TOKEN, id: KEY_ID, scopes: [WILDCARD_SCOPE] }], + dispatchers: { sales: fakeDispatcher([{ name: READ_TOOL }, { name: WRITE_TOOL }], seen) }, + }), + t, + ); + if (!h) return; + const { payload } = await h.rpc(callToolRequest(WRITE_TOOL), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(payload) ?? '', /not available to this API key/); + assert.deepEqual(seen, []); + }); + + it('refuses a write scope for a DIFFERENT tool', async (t) => { + const h = await start( + baseOptions({ + keys: [ + { + token: KEY_TOKEN, + id: KEY_ID, + scopes: [MCP_LIST_SCOPE, MCP_INVOKE_SCOPE, mcpWriteScope('some_other_tool')], + }, + ], + }), + t, + ); + if (!h) return; + const { payload } = await h.rpc(callToolRequest(WRITE_TOOL), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(payload) ?? '', /not available to this API key/); + }); + + /** A non-allowlisted tool and a nonexistent tool must be INDISTINGUISHABLE — + * distinguishing them confirms a tool's existence to a caller not entitled + * to know it. */ + it('gives the same answer for a non-allowlisted tool and a nonexistent one', async (t) => { + const h = await start( + baseOptions({ + dispatchers: { + sales: fakeDispatcher([{ name: READ_TOOL }, { name: WRITE_TOOL }, { name: OTHER_TOOL }]), + }, + }), + t, + ); + if (!h) return; + const existing = await h.rpc(callToolRequest(OTHER_TOOL), { token: KEY_TOKEN }); + const absent = await h.rpc(callToolRequest('no_such_tool_anywhere'), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(existing.payload) ?? '', /not available to this API key/); + assert.match(rpcErrorMessage(absent.payload) ?? '', /not available to this API key/); + assert.equal( + (rpcErrorMessage(existing.payload) ?? '').replace(OTHER_TOOL, 'T'), + (rpcErrorMessage(absent.payload) ?? '').replace('no_such_tool_anywhere', 'T'), + ); + }); + + it('refuses every call for a key with no binding', async (t) => { + const h = await start(baseOptions({ bindingRows: [] }), t); + if (!h) return; + const { payload } = await h.rpc(callToolRequest(READ_TOOL), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(payload) ?? '', /not available to this API key/); + }); + + it('refuses every call for a DISABLED binding', async (t) => { + const h = await start( + baseOptions({ + bindingRows: [ + { + key_id: KEY_ID, + agent_id: 'sales', + read_tools: [READ_TOOL], + write_tools: [], + write_rate_limit_per_minute: 5, + enabled: false, + }, + ], + }), + t, + ); + if (!h) return; + const { payload } = await h.rpc(callToolRequest(READ_TOOL), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(payload) ?? '', /not available to this API key/); + }); + + // ── key → agent isolation ───────────────────────────────────────────────── + + /** + * omadia's native tool registry is process-wide with unique names, so "agent + * isolation" cannot come from the registry. It comes from the binding row + * naming ONE agent plus the allowlist. Two keys, two agents, two tool sets: + * neither key may reach the other's tool, and the refusal must happen before + * the other agent's dispatcher is ever consulted. + */ + it('key A cannot reach agent B\'s tools', async (t) => { + const seenA: { name: string; input: unknown }[] = []; + const seenB: { name: string; input: unknown }[] = []; + const tokenB = 'omadia_ak_test_token_cccccccccccccccc'; + const h = await start( + baseOptions({ + keys: [ + { token: KEY_TOKEN, id: 'key-a', scopes: [MCP_LIST_SCOPE, MCP_INVOKE_SCOPE] }, + { token: tokenB, id: 'key-b', scopes: [MCP_LIST_SCOPE, MCP_INVOKE_SCOPE] }, + ], + bindingRows: [ + { + key_id: 'key-a', + agent_id: 'sales', + read_tools: [READ_TOOL], + write_tools: [], + write_rate_limit_per_minute: 5, + enabled: true, + }, + { + key_id: 'key-b', + agent_id: 'hr', + read_tools: [OTHER_TOOL], + write_tools: [], + write_rate_limit_per_minute: 5, + enabled: true, + }, + ], + dispatchers: { + sales: fakeDispatcher([{ name: READ_TOOL }], seenA), + hr: fakeDispatcher([{ name: OTHER_TOOL }], seenB), + }, + }), + t, + ); + if (!h) return; + + // Each key sees only its own agent's tool. + assert.deepEqual(toolNames((await h.rpc(listToolsRequest(), { token: KEY_TOKEN })).payload), [ + READ_TOOL, + ]); + assert.deepEqual(toolNames((await h.rpc(listToolsRequest(), { token: tokenB })).payload), [ + OTHER_TOOL, + ]); + + // Key A naming agent B's tool is refused, and B's dispatcher is untouched. + const crossed = await h.rpc(callToolRequest(OTHER_TOOL), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(crossed.payload) ?? '', /not available to this API key/); + assert.deepEqual(seenB, [], "agent B's dispatcher must never have been reached"); + + // And each key's own call still works, so the isolation is not just breakage. + assert.equal( + callResultText((await h.rpc(callToolRequest(READ_TOOL), { token: KEY_TOKEN })).payload), + `dispatched:${READ_TOOL}`, + ); + assert.equal( + callResultText((await h.rpc(callToolRequest(OTHER_TOOL), { token: tokenB })).payload), + `dispatched:${OTHER_TOOL}`, + ); + }); + + it('fails closed when the bound agent is not active', async (t) => { + const h = await start(baseOptions({ dispatchers: {} }), t); + if (!h) return; + const { payload } = await h.rpc(callToolRequest(READ_TOOL), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(payload) ?? '', /not available to this API key/); + }); + + // ── rate limits: reads and writes on SEPARATE budgets ───────────────────── + + it('exhausts the write budget while READS keep working', async (t) => { + const h = await start( + baseOptions({ + bindingRows: [ + { + key_id: KEY_ID, + agent_id: 'sales', + read_tools: [READ_TOOL], + write_tools: [WRITE_TOOL], + write_rate_limit_per_minute: 2, + enabled: true, + }, + ], + }), + t, + ); + if (!h) return; + + for (const attempt of [1, 2]) { + const ok = await h.rpc(callToolRequest(WRITE_TOOL, {}, attempt), { token: KEY_TOKEN }); + assert.equal(callResultText(ok.payload), `dispatched:${WRITE_TOOL}`, `write ${String(attempt)}`); + } + const over = await h.rpc(callToolRequest(WRITE_TOOL, {}, 3), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(over.payload) ?? '', /write rate limit exceeded/); + + // The READ budget is a different bucket entirely — it must be untouched. + const read = await h.rpc(callToolRequest(READ_TOOL, {}, 4), { token: KEY_TOKEN }); + assert.equal(callResultText(read.payload), `dispatched:${READ_TOOL}`); + }); + + /** The general per-key budget `requireApiKey` applies is separate again, and + * exhausting it stops EVERYTHING including tools/list — with an HTTP 429, + * not a JSON-RPC error, because it is enforced before the transport. */ + it('exhausts the general per-key budget with an HTTP 429', async (t) => { + const h = await start( + baseOptions({ + keys: [ + { + token: KEY_TOKEN, + id: KEY_ID, + scopes: [MCP_LIST_SCOPE, MCP_INVOKE_SCOPE], + rateLimitPerMinute: 2, + }, + ], + }), + t, + ); + if (!h) return; + assert.equal((await h.post(listToolsRequest(), { token: KEY_TOKEN })).status, 200); + assert.equal((await h.post(listToolsRequest(), { token: KEY_TOKEN })).status, 200); + const over = await h.post(listToolsRequest(), { token: KEY_TOKEN }); + assert.equal(over.status, 429); + await over.text(); + }); + + it('a write budget of 0 refuses every write while reads still work', async (t) => { + const h = await start( + baseOptions({ + bindingRows: [ + { + key_id: KEY_ID, + agent_id: 'sales', + read_tools: [READ_TOOL], + write_tools: [WRITE_TOOL], + write_rate_limit_per_minute: 0, + enabled: true, + }, + ], + }), + t, + ); + if (!h) return; + const write = await h.rpc(callToolRequest(WRITE_TOOL), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(write.payload) ?? '', /write rate limit exceeded/); + const read = await h.rpc(callToolRequest(READ_TOOL, {}, 9), { token: KEY_TOKEN }); + assert.equal(callResultText(read.payload), `dispatched:${READ_TOOL}`); + }); + + // ── timeout + concurrency ───────────────────────────────────────────────── + + it('times out a hanging tool without hanging the request', async (t) => { + const h = await start( + baseOptions({ + toolTimeoutMs: 60, + dispatchers: { + sales: fakeDispatcher([ + { name: READ_TOOL, handle: () => new Promise(() => {/* never settles */}) }, + ]), + }, + }), + t, + ); + if (!h) return; + const { payload } = await h.rpc(callToolRequest(READ_TOOL), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(payload) ?? '', /exceeded the 60ms public MCP timeout/); + }); + + it('refuses a call once the concurrency ceiling is reached', async (t) => { + let release: (() => void) | undefined; + const gate = new Promise((resolve) => { + release = resolve; + }); + const h = await start( + baseOptions({ + maxConcurrentCalls: 1, + toolTimeoutMs: 5_000, + dispatchers: { + sales: fakeDispatcher([ + { + name: READ_TOOL, + handle: async () => { + await gate; + return { content: 'slow' }; + }, + }, + ]), + }, + }), + t, + ); + if (!h) return; + + const first = h.rpc(callToolRequest(READ_TOOL, {}, 20), { token: KEY_TOKEN }); + // Give the first call time to occupy the single slot. + await new Promise((resolve) => setTimeout(resolve, 80)); + const second = await h.rpc(callToolRequest(READ_TOOL, {}, 21), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(second.payload) ?? '', /at capacity/); + + release?.(); + assert.equal(callResultText((await first).payload), 'slow'); + + // The slot is released, so a later call succeeds — the ceiling is a gate, + // not a permanent lockout. + const third = await h.rpc(callToolRequest(READ_TOOL, {}, 22), { token: KEY_TOKEN }); + assert.equal(callResultText(third.payload), 'slow'); + }); + + // ── the privacy-seam gate ───────────────────────────────────────────────── + + /** + * `ToolDispatchService` applies NO PII masking — its own trailing SEAM comment + * records that privacy interning and trace capture are not replicated versus + * `Orchestrator.dispatchToolInner`. Until the sibling branch closes that seam, + * serving a tool call could put unmasked personal data on a public HTTP + * response, so the default refuses and says so. `tools/list` still works, so + * an integrator can discover the contract meanwhile. + */ + it('refuses tool calls while the dispatch privacy seam is open, but still lists tools', async (t) => { + const seen: { name: string; input: unknown }[] = []; + const h = await start( + baseOptions({ + allowWithoutPrivacySeam: false, + dispatchers: { sales: fakeDispatcher([{ name: READ_TOOL }, { name: WRITE_TOOL }], seen) }, + }), + t, + ); + if (!h) return; + assert.deepEqual( + toolNames((await h.rpc(listToolsRequest(), { token: KEY_TOKEN })).payload), + SORTED_TOOLS, + ); + const { payload } = await h.rpc(callToolRequest(READ_TOOL), { token: KEY_TOKEN }); + assert.match(rpcErrorMessage(payload) ?? '', /privacy\/trace seam is not wired/); + assert.deepEqual(seen, [], 'nothing may be dispatched while the seam is open'); + }); + + // ── audit ───────────────────────────────────────────────────────────────── + + it('records one audit row per call with the acting identity', async (t) => { + const audit: PublicMcpAuditEntry[] = []; + const h = await start(baseOptions({ audit }), t); + if (!h) return; + await h.rpc(callToolRequest(READ_TOOL), { token: KEY_TOKEN }); + assert.equal(audit.length, 1); + assert.equal(audit[0]?.toolName, READ_TOOL); + assert.equal(audit[0]?.agentId, 'sales'); + assert.equal(audit[0]?.ok, true); + assert.equal(audit[0]?.write, false); + assert.equal(audit[0]?.actingIdentity, `apikey:${KEY_ID}`); + }); + + it('records a REFUSED call too — a refusal with no trace is uninvestigable', async (t) => { + const audit: PublicMcpAuditEntry[] = []; + const h = await start( + baseOptions({ + audit, + keys: [{ token: KEY_TOKEN, id: KEY_ID, scopes: [MCP_LIST_SCOPE, MCP_INVOKE_SCOPE] }], + }), + t, + ); + if (!h) return; + await h.rpc(callToolRequest(WRITE_TOOL), { token: KEY_TOKEN }); + assert.equal(audit.length, 1); + assert.equal(audit[0]?.ok, false); + assert.equal(audit[0]?.error, 'not allowlisted'); + assert.equal(audit[0]?.write, true, 'the attempt was against a write tool'); + assert.equal(audit[0]?.actingIdentity, `apikey:${KEY_ID}`); + }); + + it('marks a write call as a write in the audit row', async (t) => { + const audit: PublicMcpAuditEntry[] = []; + const h = await start(baseOptions({ audit }), t); + if (!h) return; + await h.rpc(callToolRequest(WRITE_TOOL), { token: KEY_TOKEN }); + assert.equal(audit[0]?.write, true); + assert.equal(audit[0]?.ok, true); + }); + + it('does not audit tools/list — only tool calls', async (t) => { + const audit: PublicMcpAuditEntry[] = []; + const h = await start(baseOptions({ audit }), t); + if (!h) return; + await h.rpc(listToolsRequest(), { token: KEY_TOKEN }); + assert.deepEqual(audit, []); + }); + + it('mounts at the shared PUBLIC_MCP_PATH constant', async (t) => { + const h = await start(baseOptions(), t); + if (!h) return; + assert.equal(h.mounted, true); + assert.ok(h.url.endsWith(PUBLIC_MCP_PATH)); + }); +}); diff --git a/middleware/test/publicMcp/publicMcpKeyBindings.test.ts b/middleware/test/publicMcp/publicMcpKeyBindings.test.ts new file mode 100644 index 00000000..f1370114 --- /dev/null +++ b/middleware/test/publicMcp/publicMcpKeyBindings.test.ts @@ -0,0 +1,134 @@ +import { strict as assert } from 'node:assert'; +import { describe, it } from 'node:test'; + +import { + createInMemoryPublicMcpKeyBindingStore, + normalizeBindingRow, +} from '../../src/mcp/publicMcpKeyBindings.js'; + +/** + * W2-3 (issue #542) — the binding half of the public MCP endpoint's + * authorization: which agent a key is bound to, and which of that agent's tools + * it reaches. + * + * Every case here asks the same question: does an unreadable row GRANT + * anything? It must not. The endpoint is internet-facing and exposes write + * tools, so a normalization bug that fails open is a remote write. + */ +const GOOD_ROW = { + key_id: 'key-1', + agent_id: 'sales', + read_tools: ['query_crm'], + write_tools: ['create_lead'], + write_rate_limit_per_minute: 5, + enabled: true, +}; + +describe('public MCP key bindings — the happy path', () => { + it('resolves a well-formed row', () => { + const binding = normalizeBindingRow(GOOD_ROW); + assert.ok(binding); + assert.equal(binding.keyId, 'key-1'); + assert.equal(binding.agentId, 'sales'); + assert.deepEqual(binding.readTools, ['query_crm']); + assert.deepEqual(binding.writeTools, ['create_lead']); + assert.equal(binding.writeRateLimitPerMinute, 5); + }); + + it('accepts a rate limit that pg returned as a string', () => { + const binding = normalizeBindingRow({ ...GOOD_ROW, write_rate_limit_per_minute: '12' }); + assert.equal(binding?.writeRateLimitPerMinute, 12); + }); + + it('de-duplicates repeated tool names', () => { + const binding = normalizeBindingRow({ + ...GOOD_ROW, + read_tools: ['query_crm', 'query_crm'], + }); + assert.deepEqual(binding?.readTools, ['query_crm']); + }); + + /** + * A tool in BOTH lists is ambiguous about whether it needs + * `mcp:write:`. Resolve toward WRITE — the stricter reading. Resolving + * toward read would silently drop the per-tool write scope requirement from a + * tool the operator just marked as a write. + */ + it('a tool listed as both read and write is treated as a WRITE', () => { + const binding = normalizeBindingRow({ + ...GOOD_ROW, + read_tools: ['create_lead'], + write_tools: ['create_lead'], + }); + assert.deepEqual(binding?.readTools, []); + assert.deepEqual(binding?.writeTools, ['create_lead']); + }); +}); + +describe('public MCP key bindings — fail closed', () => { + for (const [label, row] of [ + ['missing key_id', { ...GOOD_ROW, key_id: undefined }], + ['empty key_id', { ...GOOD_ROW, key_id: '' }], + ['missing agent_id', { ...GOOD_ROW, agent_id: undefined }], + ['empty agent_id', { ...GOOD_ROW, agent_id: '' }], + ['non-string agent_id', { ...GOOD_ROW, agent_id: 7 }], + ['read_tools is null', { ...GOOD_ROW, read_tools: null }], + ['read_tools is not an array', { ...GOOD_ROW, read_tools: 'query_crm' }], + ['read_tools holds a non-string', { ...GOOD_ROW, read_tools: ['ok', 3] }], + ['write_tools is null', { ...GOOD_ROW, write_tools: null }], + ['write_tools holds a non-string', { ...GOOD_ROW, write_tools: [{}] }], + ['enabled is not a boolean', { ...GOOD_ROW, enabled: 'true' }], + ['enabled is missing', { ...GOOD_ROW, enabled: undefined }], + ['rate limit is negative', { ...GOOD_ROW, write_rate_limit_per_minute: -1 }], + ['rate limit is fractional', { ...GOOD_ROW, write_rate_limit_per_minute: 1.5 }], + ['rate limit is not a number', { ...GOOD_ROW, write_rate_limit_per_minute: 'many' }], + ] as const) { + it(`denies the whole row: ${label}`, () => { + assert.equal(normalizeBindingRow(row), undefined); + }); + } + + /** + * Partially-valid lists deny rather than narrowing to the valid subset — the + * same rule `normalizeScopes` applies, for the same reason: a record we cannot + * read faithfully is one we must not guess at, and "half its tools" is worse + * to debug than "none". + */ + it('a partially-valid read_tools list does NOT narrow to its valid subset', () => { + assert.equal( + normalizeBindingRow({ ...GOOD_ROW, read_tools: ['query_crm', null] }), + undefined, + ); + }); + + it('a disabled row grants nothing, and is indistinguishable from absent to callers', () => { + assert.equal(normalizeBindingRow({ ...GOOD_ROW, enabled: false }), undefined); + }); + + it('a row may legitimately grant nothing without being malformed', () => { + const binding = normalizeBindingRow({ ...GOOD_ROW, read_tools: [], write_tools: [] }); + assert.ok(binding); + assert.deepEqual(binding.readTools, []); + assert.deepEqual(binding.writeTools, []); + }); +}); + +describe('public MCP key bindings — the in-memory store', () => { + it('returns undefined for an unknown key', async () => { + const store = createInMemoryPublicMcpKeyBindingStore([GOOD_ROW]); + assert.equal(await store.get('nope'), undefined); + }); + + it('returns the binding for a known key', async () => { + const store = createInMemoryPublicMcpKeyBindingStore([GOOD_ROW]); + assert.equal((await store.get('key-1'))?.agentId, 'sales'); + }); + + /** The in-memory store takes RAW rows on purpose: a store that accepted + * ready-made binding objects would bypass `normalizeBindingRow` and every + * test above would prove nothing about the pg path. */ + it('applies the same fail-closed normalization as the pg store', async () => { + const store = createInMemoryPublicMcpKeyBindingStore([{ ...GOOD_ROW, enabled: 'yes' }]); + assert.equal(await store.get('key-1'), undefined); + }); +}); diff --git a/middleware/test/publicMcp/publicMcpScopes.test.ts b/middleware/test/publicMcp/publicMcpScopes.test.ts new file mode 100644 index 00000000..42846c4b --- /dev/null +++ b/middleware/test/publicMcp/publicMcpScopes.test.ts @@ -0,0 +1,178 @@ +import { strict as assert } from 'node:assert'; +import { describe, it } from 'node:test'; + +import { + assertValidScopes, + DENY_ALL_SCOPES, + hasScope, + hasWriteScope, + isMcpWriteScope, + isValidScope, + LEGACY_DEFAULT_SCOPES, + MCP_INVOKE_SCOPE, + MCP_LIST_SCOPE, + mcpWriteScope, + normalizeScopes, + WILDCARD_SCOPE, +} from '../../packages/harness-api-key-auth/src/apiKeyScopes.js'; + +/** + * W2-3 (issue #542) — the scope half of the public MCP endpoint's + * authorization. + * + * Marcel chose to expose WRITE tools over an internet-facing endpoint (I + * recommended read-only). That decision is what makes the wildcard exclusion + * below a merge blocker rather than a refinement: with `*` satisfying a write + * scope, any operator key minted with `*` for convenience would silently carry + * "delete every Odoo invoice, over the internet, with no session". + */ +describe('MCP scopes — shape', () => { + it('admits the three-segment mcp:write: form', () => { + assert.equal(isValidScope('mcp:write:create_lead'), true); + assert.equal(isValidScope(mcpWriteScope('book-meeting')), true); + }); + + it('admits the two-segment list/invoke scopes', () => { + assert.equal(isValidScope(MCP_LIST_SCOPE), true); + assert.equal(isValidScope(MCP_INVOKE_SCOPE), true); + }); + + /** + * The three-segment rule is a literal `mcp:write:` prefix, not a generic + * `::`. A generic rule would legalize every mistyped triple, and + * each such string would validate, persist, and grant nothing — which reads + * exactly like a revoked key at debug time. + */ + it('does NOT admit an arbitrary three-segment scope', () => { + assert.equal(isValidScope('odoo:write:invoice'), false); + assert.equal(isValidScope('mcp:read:thing'), false); + assert.equal(isValidScope('mcp:write:'), false); + assert.equal(isValidScope('mcp:write:Create_Lead'), false); + assert.equal(isValidScope('mcp:write:1tool'), false); + }); + + /** + * `mcp:write` is a well-formed TWO-segment scope, so the generic pattern + * accepts it — and it is the likeliest thing an operator types meaning "this + * key may write". It would grant nothing (no check ever asks for it), which + * reads exactly like a revoked key. Rejected outright so the mistake surfaces + * at mint time. There is no class-wide write scope by design. + */ + it('rejects the bare mcp:write, which would validate and grant nothing', () => { + assert.equal(isValidScope('mcp:write'), false); + assert.throws(() => assertValidScopes(['mcp:write']), /invalid API-key scope/); + assert.deepEqual(normalizeScopes(['mcp:write']), DENY_ALL_SCOPES); + }); + + it('still admits and rejects exactly what it did before, for two-segment scopes', () => { + assert.equal(isValidScope('chat:write'), true); + assert.equal(isValidScope('memory:read'), true); + assert.equal(isValidScope(WILDCARD_SCOPE), true); + assert.equal(isValidScope('Chat:Write'), false); + assert.equal(isValidScope('nonsense'), false); + assert.equal(isValidScope(42), false); + assert.equal(isValidScope(null), false); + }); + + it('identifies write scopes by prefix', () => { + assert.equal(isMcpWriteScope('mcp:write:x'), true); + assert.equal(isMcpWriteScope(MCP_INVOKE_SCOPE), false); + assert.equal(isMcpWriteScope(WILDCARD_SCOPE), false); + }); +}); + +describe('MCP scopes — the wildcard exclusion', () => { + /** + * THE load-bearing assertion of this file. `*` grants every other capability + * and must grant no write. + */ + it('WILDCARD_SCOPE does NOT grant a per-tool write', () => { + const granted = [WILDCARD_SCOPE]; + assert.equal(hasScope(granted, mcpWriteScope('create_lead')), false); + assert.equal(hasWriteScope(granted, 'create_lead'), false); + }); + + it('WILDCARD_SCOPE still grants every non-write scope', () => { + const granted = [WILDCARD_SCOPE]; + assert.equal(hasScope(granted, MCP_LIST_SCOPE), true); + assert.equal(hasScope(granted, MCP_INVOKE_SCOPE), true); + assert.equal(hasScope(granted, 'chat:write'), true); + }); + + it('grants a write ONLY on an exact per-tool match', () => { + const granted = [MCP_INVOKE_SCOPE, mcpWriteScope('create_lead')]; + assert.equal(hasWriteScope(granted, 'create_lead'), true); + // A sibling write tool is a different capability. + assert.equal(hasWriteScope(granted, 'delete_invoice'), false); + // And the write scope does not backfill the invoke scope for another tool. + assert.equal(hasWriteScope([mcpWriteScope('create_lead')], 'create_lead'), true); + }); + + it('mcp:invoke alone does NOT grant any write', () => { + assert.equal(hasWriteScope([MCP_INVOKE_SCOPE], 'create_lead'), false); + }); + + it('hasWriteScope and hasScope agree — so calling the wrong one is not a security event', () => { + for (const granted of [ + [WILDCARD_SCOPE], + [MCP_INVOKE_SCOPE], + [mcpWriteScope('t')], + [], + ]) { + assert.equal( + hasWriteScope(granted, 't'), + hasScope(granted, mcpWriteScope('t')), + `disagreement for granted=${JSON.stringify(granted)}`, + ); + } + }); + + it('denies everything when nothing is granted', () => { + assert.equal(hasScope(undefined, MCP_LIST_SCOPE), false); + assert.equal(hasScope(DENY_ALL_SCOPES, MCP_INVOKE_SCOPE), false); + assert.equal(hasWriteScope(DENY_ALL_SCOPES, 'create_lead'), false); + }); +}); + +describe('MCP scopes — persisted-record normalization', () => { + it('round-trips a valid persisted write scope', () => { + const normalized = normalizeScopes([MCP_INVOKE_SCOPE, 'mcp:write:create_lead']); + assert.deepEqual([...normalized].sort(), ['mcp:invoke', 'mcp:write:create_lead']); + assert.equal(hasWriteScope(normalized, 'create_lead'), true); + }); + + /** A malformed persisted `scopes` field must deny EVERYTHING, including the + * write scopes that happen to sit next to the malformed entry. */ + it('malformed persisted scopes deny all — a valid write scope alongside garbage grants nothing', () => { + const normalized = normalizeScopes(['mcp:write:create_lead', 'NOT A SCOPE']); + assert.deepEqual(normalized, DENY_ALL_SCOPES); + assert.equal(hasWriteScope(normalized, 'create_lead'), false); + assert.equal(hasScope(normalized, MCP_LIST_SCOPE), false); + }); + + it('a non-array persisted scopes field denies all', () => { + assert.deepEqual(normalizeScopes('mcp:invoke'), DENY_ALL_SCOPES); + }); + + it('an empty persisted scopes array denies all', () => { + assert.deepEqual(normalizeScopes([]), DENY_ALL_SCOPES); + }); + + /** An absent field is a genuine pre-#439 key and keeps its old capability — + * it must NOT be widened to the new MCP surface by an upgrade. */ + it('an absent scopes field stays chat-only and reaches no MCP capability', () => { + const normalized = normalizeScopes(undefined); + assert.deepEqual(normalized, LEGACY_DEFAULT_SCOPES); + assert.equal(hasScope(normalized, MCP_LIST_SCOPE), false); + assert.equal(hasScope(normalized, MCP_INVOKE_SCOPE), false); + assert.equal(hasWriteScope(normalized, 'create_lead'), false); + }); + + it('accepts MCP scopes at creation time and rejects a malformed one', () => { + assert.deepEqual( + [...assertValidScopes([MCP_LIST_SCOPE, MCP_INVOKE_SCOPE, 'mcp:write:create_lead'])].sort(), + ['mcp:invoke', 'mcp:list', 'mcp:write:create_lead'], + ); + assert.throws(() => assertValidScopes(['mcp:write:Create_Lead']), /invalid API-key scope/); + }); +}); diff --git a/middleware/test/publicPaths.test.ts b/middleware/test/publicPaths.test.ts index 2ed8d920..dc722b36 100644 --- a/middleware/test/publicPaths.test.ts +++ b/middleware/test/publicPaths.test.ts @@ -3,6 +3,7 @@ import { describe, it } from 'node:test'; import { publicPaths, STATIC_PUBLIC_PATHS } from '../src/auth/publicPaths.js'; import { CIMD_METADATA_PATH } from '../src/services/mcpCimd.js'; +import { PUBLIC_MCP_PATH } from '../src/mcp/publicMcpPath.js'; /** * Regression guard for the MCP-OAuth-callback 401 bug: the epic #459 W9 @@ -102,3 +103,62 @@ describe('publicPaths — MCP client-ID metadata document allowlist', () => { ); }); }); + +/** + * W2-3 (issue #542) — the public, stateless MCP endpoint. + * + * Asserted against the SHARED `PUBLIC_MCP_PATH` constant, not a retyped + * literal, for the reason in this module's own doc comment: a hand-written + * pattern next to a hand-written mount is exactly the epic #470 drift the + * constant exists to make impossible. + * + * The entry is what makes the route reachable at all — the OB-106 `/api` + * requireAuth line runs for every `/api/*` request — so its removal makes the + * endpoint go DARK rather than open. That failure direction is asserted + * end-to-end in `test/publicMcp/publicMcpEndpoint.e2e.test.ts` + * ("goes DARK (session 401), not open"); this block covers the allowlist half. + */ +describe('publicPaths — public MCP endpoint allowlist', () => { + const allowlist = publicPaths({ devEndpointsEnabled: false }); + const isPublic = (path: string): boolean => allowlist.some((p) => p.test(path)); + + it('exempts the public MCP endpoint from the session gate', () => { + assert.equal( + isPublic(PUBLIC_MCP_PATH), + true, + `${PUBLIC_MCP_PATH} must be exempt — it authenticates via requireApiKey`, + ); + }); + + it('exempts it with a query string appended', () => { + assert.equal(isPublic(`${PUBLIC_MCP_PATH}?v=1`), true); + }); + + it('is present in STATIC_PUBLIC_PATHS regardless of devEndpointsEnabled', () => { + assert.equal( + STATIC_PUBLIC_PATHS.some((p) => p.test(PUBLIC_MCP_PATH)), + true, + ); + }); + + /** + * Narrowest possible entry. Every additional character this matched would be + * a new unauthenticated-until-the-handler-says-otherwise surface, and the + * NOTE in `publicPaths.ts` asks for exactly one route, never a prefix. + */ + it('does NOT widen the bypass to sub-paths under the endpoint', () => { + assert.equal(isPublic(`${PUBLIC_MCP_PATH}/admin`), false); + assert.equal(isPublic(`${PUBLIC_MCP_PATH}/`), false); + }); + + it('does NOT widen the bypass to sibling /api/v1/mcp* routes', () => { + assert.equal(isPublic('/api/v1/mcp-servers'), false); + assert.equal(isPublic('/api/v1/mcp-oauth/callback'), false); + assert.equal(isPublic('/api/v1/mcpsecret'), false); + }); + + it('does NOT exempt the operator MCP admin surfaces', () => { + assert.equal(isPublic('/api/v1/operator/mcp-servers'), false); + assert.equal(isPublic('/api/v1/operator/mcp-call-log'), false); + }); +}); From c8284f8a82dfc988c8b300879575612fea90618c Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Thu, 30 Jul 2026 14:34:02 +0200 Subject: [PATCH 48/90] test(mcp): mutation harness for the public MCP endpoint MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 27 mutations that each break one invariant with a real source edit and require a real assertion failure. A mutation that leaves the suite green means the invariant is untested — the check fails in that direction. --- middleware/test/publicMcp/mutation-check.sh | 140 ++++++++++++++++++++ 1 file changed, 140 insertions(+) create mode 100644 middleware/test/publicMcp/mutation-check.sh diff --git a/middleware/test/publicMcp/mutation-check.sh b/middleware/test/publicMcp/mutation-check.sh new file mode 100644 index 00000000..a48ed9e0 --- /dev/null +++ b/middleware/test/publicMcp/mutation-check.sh @@ -0,0 +1,140 @@ +#!/usr/bin/env bash +# W2-3 (issue #542) — mutation harness for the public MCP endpoint. +# +# Counting mock invocations proves nothing about whether a gate works. Each entry +# below deliberately BREAKS one invariant with a real source edit, re-runs the +# suite, and requires a real assertion failure. A mutation that leaves the suite +# green means the invariant is untested — the check FAILS in that direction, which +# is the whole point. +# +# Usage: bash test/publicMcp/mutation-check.sh +# Run from middleware/. Reverts every edit via `git checkout --` afterwards. +set -uo pipefail + +cd "$(dirname "$0")/../.." || exit 1 +SCOPES=packages/harness-api-key-auth/src/apiKeyScopes.ts +SERVER=src/mcp/publicMcpServer.ts +BINDINGS=src/mcp/publicMcpKeyBindings.ts +PATHS=src/auth/publicPaths.ts + +TESTS=('test/publicMcp/publicMcpScopes.test.ts' + 'test/publicMcp/publicMcpKeyBindings.test.ts' + 'test/publicMcp/publicMcpBodyCap.test.ts' + 'test/publicMcp/publicMcpEndpoint.e2e.test.ts' + 'test/publicPaths.test.ts') + +pass=0; fail=0 + +revert() { git checkout -- "$SCOPES" "$SERVER" "$BINDINGS" "$PATHS" 2>/dev/null; } + +# run_mutation
@@ -1782,3 +1787,259 @@ function AuditPane(): React.ReactElement {
); } + +// ── Public MCP key bindings (W5-1) ─────────────────────────────────────────── + +/** + * The operator surface for `public_mcp_key_bindings` — the per-API-key + * allowlist that decides what the public MCP endpoint lets a third-party + * integration reach. Before this pane existed the endpoint could only be + * configured by hand in psql, which made it inert as shipped. + * + * KEY PICKER: the operator PASTES a key id. A binding is keyed on + * `ApiKeyRecord.id`, and the only place that lists those ids is the channel-api + * plugin's admin router under a different prefix (`/api/public/v1/admin/keys`), + * which is mounted only when that plugin is active and has no web-ui client at + * all. Rendering a dropdown from it would couple the MCP Control Center to a + * plugin-owned endpoint and give this tab a second way to fail on installs that + * do not run the plugin. Building the missing channel-API key UI is its own + * unit; this pane's job is to make the endpoint configurable. + */ +function BindingsPane(): React.ReactElement { + const t = useTranslations('adminMcp'); + const [bindings, setBindings] = useState(null); + const [orchestrators, setOrchestrators] = useState([]); + const [error, setError] = useState(null); + const [busy, setBusy] = useState(null); + const [confirmRevoke, setConfirmRevoke] = useState(null); + + const [keyId, setKeyId] = useState(''); + const [agentId, setAgentId] = useState(''); + const [readTools, setReadTools] = useState(''); + const [writeTools, setWriteTools] = useState(''); + const [writeRate, setWriteRate] = useState('5'); + + const refresh = useCallback(async () => { + try { + setBindings((await listPublicMcpKeyBindings()).bindings); + setError(null); + } catch (err) { + setError(errText(err)); + } + }, []); + + useEffect(() => { + void refresh(); + // A missing orchestrator list must not break the pane — the agent field + // falls back to free text below. + void listMcpOrchestrators() + .then((r) => setOrchestrators(r.orchestrators)) + .catch(() => setOrchestrators([])); + }, [refresh]); + + async function save(): Promise { + setBusy('save'); + setError(null); + try { + await upsertPublicMcpKeyBinding({ + keyId: keyId.trim(), + agentId: agentId.trim(), + readTools: parseToolList(readTools), + writeTools: parseToolList(writeTools), + writeRateLimitPerMinute: Number(writeRate), + }); + setKeyId(''); + setReadTools(''); + setWriteTools(''); + await refresh(); + } catch (err) { + setError(errText(err)); + } finally { + setBusy(null); + } + } + + async function revoke(binding: PublicMcpKeyBinding): Promise { + setBusy(`revoke:${binding.keyId}`); + setError(null); + try { + await revokePublicMcpKeyBinding(binding.keyId); + await refresh(); + } catch (err) { + setError(errText(err)); + } finally { + setBusy(null); + } + } + + const inputCls = + 'rounded-md border border-[color:var(--border)] bg-transparent px-3 py-2 text-sm outline-none focus:border-[color:var(--accent)]'; + const canSave = keyId.trim().length > 0 && agentId.trim().length > 0; + + return ( +
+

{t('bindings.intro')}

+

{t('bindings.keyIdHint')}

+ {error ?
{error}
: null} + +
+ + + + + + +
+

{t('bindings.writeToolsHint')}

+ + {!bindings ? ( +
{t('loading')}
+ ) : null} + {bindings && bindings.length === 0 ? ( +
{t('bindings.empty')}
+ ) : null} + + {bindings?.map((b) => ( +
+
+
+
{b.keyId}
+
+ {t('bindings.boundTo', { agent: b.agentId })} +
+
+ + {b.enabled ? t('bindings.enabled') : t('bindings.parked')} + +
+
+ {t('bindings.readToolsLabel')}:{' '} + {b.readTools.length > 0 ? b.readTools.join(', ') : t('bindings.none')} +
+
+ {t('bindings.writeToolsLabel')}:{' '} + {b.writeTools.length > 0 ? b.writeTools.join(', ') : t('bindings.none')} +
+
+ {t('bindings.meta', { + rate: b.writeRateLimitPerMinute, + updated: b.updatedAt.slice(0, 19).replace('T', ' '), + })} +
+ {b.enabled ? ( +
+ +
+ ) : null} +
+ ))} + + setConfirmRevoke(null)} + onConfirm={() => { + const target = confirmRevoke; + setConfirmRevoke(null); + if (target) void revoke(target); + }} + /> +
+ ); +} + +/** Comma- or newline-separated, trimmed, blanks dropped. The server rejects an + * empty string inside a tool list (the reader denies the whole row for one), + * so a trailing comma must not become one. */ +function parseToolList(raw: string): string[] { + return raw + .split(/[,\n]/) + .map((s) => s.trim()) + .filter((s) => s.length > 0); +} diff --git a/web-ui/messages/de.json b/web-ui/messages/de.json index b337dcb1..d6d3783a 100644 --- a/web-ui/messages/de.json +++ b/web-ui/messages/de.json @@ -2795,8 +2795,36 @@ "marketplace": "Marketplace", "grants": "Grants", "plugins": "Plugin-Zugriff", + "bindings": "Öffentliche API-Keys", "audit": "Audit-Log" }, + "bindings": { + "intro": "Über den öffentlichen MCP-Endpunkt ruft eine externe Integration Tools mit einem API-Key auf. Jeder Key erreicht genau einen Agenten und genau die Tools, die hier eingetragen sind — ein Key ohne Binding erreicht gar nichts.", + "keyIdHint": "Trag die Key-ID ein, die beim Anlegen des API-Keys zurückkam. Eine Auswahlliste gibt es noch nicht: Keys verwaltet die Channel-API über einen eigenen Endpunkt.", + "keyId": "Key-ID", + "keyIdPlaceholder": "z. B. 8f3c1a90-…", + "agentId": "Agent", + "agentIdPlaceholder": "Agent wählen", + "readTools": "Lese-Tools", + "writeTools": "Schreib-Tools", + "toolsPlaceholder": "Tool-Namen, per Komma getrennt", + "writeRate": "Writes/Min.", + "writeToolsHint": "Ein Schreib-Tool verlangt zusätzlich den passenden mcp:write-Scope auf dem Key und geht auf das engere Schreib-Budget. Steht ein Tool in beiden Listen, zählt es als Schreib-Tool.", + "save": "Binding speichern", + "saving": "Speichert", + "empty": "Noch keine Bindings. Solange keines existiert, erreicht über den öffentlichen Endpunkt kein API-Key irgendein Tool.", + "boundTo": "gebunden an {agent}", + "enabled": "Aktiv", + "parked": "Widerrufen", + "readToolsLabel": "Lesen", + "writeToolsLabel": "Schreiben", + "none": "keine", + "meta": "{rate} Writes/Min. · zuletzt geändert {updated}", + "revoke": "Widerrufen", + "revokeTitle": "Dieses Binding widerrufen?", + "revokeBody": "Der Key {keyId} erreicht ab sofort kein Tool mehr. Das Binding bleibt erhalten und wird nicht gelöscht — du siehst weiterhin, was es erlaubt hat.", + "revokeConfirm": "Widerrufen" + }, "plugins": { "intro": "Plugins, die im Manifest MCP-Zugriff deklarieren. Weise jedem gezielt die Server zu, die es erreichen darf — nichts ist implizit. Die Per-Tool-Sicherheit greift weiterhin: ein ungescanntes oder hochriskantes Tool wird zur Aufrufzeit abgelehnt.", "empty": "Noch kein installiertes Plugin deklariert MCP-Zugriff.", diff --git a/web-ui/messages/en.json b/web-ui/messages/en.json index d70908a1..f3c567fe 100644 --- a/web-ui/messages/en.json +++ b/web-ui/messages/en.json @@ -2795,8 +2795,36 @@ "marketplace": "Marketplace", "grants": "Grants", "plugins": "Plugin access", + "bindings": "Public API keys", "audit": "Audit log" }, + "bindings": { + "intro": "The public MCP endpoint lets an external integration call tools with an API key. Each key reaches exactly one agent and exactly the tools you name here — a key with no binding reaches nothing at all.", + "keyIdHint": "Paste the key id you got back when the API key was created. There is no picker yet: keys are managed by the channel API, on a separate endpoint.", + "keyId": "Key ID", + "keyIdPlaceholder": "e.g. 8f3c1a90-…", + "agentId": "Agent", + "agentIdPlaceholder": "Pick an agent", + "readTools": "Read tools", + "writeTools": "Write tools", + "toolsPlaceholder": "Tool names, comma-separated", + "writeRate": "Writes/min", + "writeToolsHint": "A write tool additionally requires the key to hold the matching mcp:write scope and spends the tighter write budget. A tool named in both lists counts as a write.", + "save": "Save binding", + "saving": "Saving", + "empty": "No key bindings yet. Until you create one, no API key reaches any tool through the public endpoint.", + "boundTo": "bound to {agent}", + "enabled": "Active", + "parked": "Revoked", + "readToolsLabel": "Read", + "writeToolsLabel": "Write", + "none": "none", + "meta": "{rate} writes/min · last changed {updated}", + "revoke": "Revoke", + "revokeTitle": "Revoke this binding?", + "revokeBody": "The key {keyId} stops reaching any tool immediately. The binding is kept, not deleted, so you can still see what it granted.", + "revokeConfirm": "Revoke" + }, "plugins": { "intro": "Plugins that declare MCP access in their manifest. Grant each the specific servers it may reach — nothing is ambient. Per-tool safety still applies: an unscanned or high-risk tool is refused at call time.", "empty": "No installed plugin declares MCP access yet.", From b7e55e443c429d62f36398353e38599b29dc24fa Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 13:15:18 +0200 Subject: [PATCH 75/90] test(mcp): close throwaway servers in a finally so a failed assertion cannot hang MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found by the W5-1 mutation check. Three tests mounted an ad-hoc express app and closed it AFTER their assertions, so removing the auth gate turned a red test into an indefinite hang: the assertion threw, the close was skipped, and the listening handle kept the event loop alive. A test that hangs when the invariant breaks is not coverage — and under a mutation run it reads as a timeout rather than as the specific assertion that was supposed to trip. withRouter() now closes the listener in a finally. --- .../publicMcpKeyBindingsAdmin.test.ts | 83 ++++++++++++------- 1 file changed, 55 insertions(+), 28 deletions(-) diff --git a/middleware/test/publicMcp/publicMcpKeyBindingsAdmin.test.ts b/middleware/test/publicMcp/publicMcpKeyBindingsAdmin.test.ts index 461e093c..ab2ee406 100644 --- a/middleware/test/publicMcp/publicMcpKeyBindingsAdmin.test.ts +++ b/middleware/test/publicMcp/publicMcpKeyBindingsAdmin.test.ts @@ -69,6 +69,28 @@ function mountRouter(opts: { return { server, baseUrl: `http://127.0.0.1:${String(addr.port)}/public-mcp-bindings` }; } +/** + * Runs `fn` against a throwaway mount and ALWAYS closes the listener. + * + * Not incidental hygiene — found by the mutation check. Closing after the + * assertions means a failing assertion skips the close, the listening handle + * keeps the event loop alive, and `node --test` hangs instead of reporting the + * failure. A test that hangs when the invariant breaks is not a red test, and + * under a mutation run it looks like a timeout rather than the specific + * assertion the mutation was supposed to trip. + */ +async function withRouter( + opts: Parameters[0], + fn: (baseUrl: string) => Promise, +): Promise { + const { server, baseUrl } = mountRouter(opts); + try { + await fn(baseUrl); + } finally { + await new Promise((r) => server.close(() => r())); + } +} + // ── The gate ──────────────────────────────────────────────────────────────── describe('publicMcpBindingsRouter — fails closed without operatorAuth', () => { @@ -94,15 +116,15 @@ describe('publicMcpBindingsRouter — fails closed without operatorAuth', () => * authorization table open to anyone who can reach the port. */ it('POST / → 503, and creates NOTHING', async () => { const store = createInMemoryPublicMcpKeyBindingAdminStore(); - const bare = mountRouter({ store }); - const res = await fetch(bare.baseUrl, { - method: 'POST', - headers: { 'content-type': 'application/json' }, - body: JSON.stringify(VALID_INPUT), + await withRouter({ store }, async (url) => { + const res = await fetch(url, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify(VALID_INPUT), + }); + assert.equal(res.status, 503); + assert.deepEqual(await store.list(), [], 'the refused POST must not have written a row'); }); - assert.equal(res.status, 503); - assert.deepEqual(await store.list(), [], 'the refused POST must not have written a row'); - await new Promise((r) => bare.server.close(() => r())); }); it('POST /:keyId/revoke → 503', async () => { @@ -147,17 +169,19 @@ describe('publicMcpBindingsRouter — operator-session gate', () => { }); it('an accessor that THROWS is treated as invalid, never as valid', async () => { - const throwing = mountRouter({ - store: createInMemoryPublicMcpKeyBindingAdminStore(), - operatorAuth: { - async hasValidSession() { - throw new Error('accessor blew up'); + await withRouter( + { + store: createInMemoryPublicMcpKeyBindingAdminStore(), + operatorAuth: { + async hasValidSession() { + throw new Error('accessor blew up'); + }, }, }, - }); - const res = await fetch(throwing.baseUrl); - assert.equal(res.status, 401); - await new Promise((r) => throwing.server.close(() => r())); + async (url) => { + assert.equal((await fetch(url)).status, 401); + }, + ); }); }); @@ -286,18 +310,21 @@ describe('publicMcpBindingsRouter — CRUD (auth stubbed valid)', () => { }); it('503s when there is no store (no graph pool), rather than pretending to save', async () => { - const poolless = mountRouter({ store: undefined, operatorAuth: alwaysValidOperatorAuth() }); - const res = await fetch(poolless.baseUrl, { - method: 'POST', - headers: { 'content-type': 'application/json' }, - body: JSON.stringify(VALID_INPUT), - }); - assert.equal(res.status, 503); - assert.equal( - ((await res.json()) as { code: string }).code, - 'public_mcp_bindings.unavailable', + await withRouter( + { store: undefined, operatorAuth: alwaysValidOperatorAuth() }, + async (url) => { + const res = await fetch(url, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify(VALID_INPUT), + }); + assert.equal(res.status, 503); + assert.equal( + ((await res.json()) as { code: string }).code, + 'public_mcp_bindings.unavailable', + ); + }, ); - await new Promise((r) => poolless.server.close(() => r())); }); }); From ac0186dd54e3a5b6efbd0ef06d2aee07f0a0d21d Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 13:35:20 +0200 Subject: [PATCH 76/90] test(mcp): prove #547 structured sidecar bypasses the privacy boundary (W5-2) The #547 producer emits structuredContent into an optional sink. Wiring that sink onto the terminal done event -- the next step toward a chat card -- would ship raw MCP tool output to the browser on turns where the equivalent TEXT is interned by Privacy Shield v4. Proven over a real MCP connection through the real production chain (ToolDispatchService -> NativeToolRegistry -> mcpNativeHandler -> McpManager.callTool -> structuredSink): BASELINE the text result reaching the caller IS masked LEAK the structured sidecar carries the same PII in clear The sidecar fires inside callTool, strictly below every dispatcher, so the leak is dispatcher-independent. PrivacyTurnHandle is string-in/string-out, so masking a structured payload while preserving its structure needs a new method on the published @omadia/plugin-api surface -- an issue, not a commit. Also locks in that outputSchema and turnId already reach the sidecar, so the generic renderer is buildable the moment masking exists. --- .../test/mcpStructuredOutputPrivacy.test.ts | 402 ++++++++++++++++++ 1 file changed, 402 insertions(+) create mode 100644 middleware/test/mcpStructuredOutputPrivacy.test.ts diff --git a/middleware/test/mcpStructuredOutputPrivacy.test.ts b/middleware/test/mcpStructuredOutputPrivacy.test.ts new file mode 100644 index 00000000..3bee2019 --- /dev/null +++ b/middleware/test/mcpStructuredOutputPrivacy.test.ts @@ -0,0 +1,402 @@ +/** + * Issue #547 (W5-2) — WHY the structured-output sidecar is NOT wired to the + * chat client. + * + * #547 landed the producer (`McpManager.structuredSink`) as plumbing only. The + * obvious next step is to wire that sink through the orchestrator onto the + * terminal `done` stream event so the UI can render a card. This file is the + * evidence that doing so, as the seam stands today, would be a PII leak — and + * it is a regression guard: if someone later makes the sidecar mask, the + * `LEAK` test below turns red and this file must be revisited on purpose. + * + * The asymmetry, stated exactly: + * + * - A tool's TEXT result is interned at the dispatch seam. `dispatchTool` + * returns `internToolResultV4(...).digestText`, and the client-facing + * `tool_result` stream event carries precisely that return value + * (`orchestrator.ts:5330` builds the slot promise from `dispatchTool`; + * `:4934` resolves it; `:4977` puts it on the wire as `output`). So the + * browser sees the digest, not the rows. + * + * - The STRUCTURED payload is emitted from inside `McpManager.callTool` + * (`mcpClient.ts:~880`), which sits strictly BELOW every dispatcher. It + * never crosses the privacy handle at all. `extractStructured` documents + * this intent outright: "Returns the payload exactly as the server sent + * it ... never a re-parse of the rendered string." + * + * That "strictly below every dispatcher" is why this file proves the property + * using `ToolDispatchService` rather than a full `Orchestrator` turn: the sink + * fires beneath the dispatcher, so the leak is dispatcher-independent. The + * interning contract asserted here is the same one the chat path uses and is + * documented as parity in `toolDispatchPrivacySeam.test.ts`. + * + * There is also no way to fix this inside W5-2's scope. The whole privacy + * contract (`PrivacyTurnHandle`) is string-in/string-out: + * `internToolResultV4({rawResult: string}) -> {digestText: string}`. Feeding a + * structured payload through it returns a digest STRING — the structure the + * card exists to render is destroyed, leaving something strictly worse than + * the `ToolRow` that already shows that digest. Masking structure while + * preserving it needs a NEW method on the published `@omadia/plugin-api` + * surface plus a privacy-guard implementation and boot wiring. That is an + * issue, not a commit. + * + * MUTATION-CHECK DISCIPLINE (same as `toolDispatchPrivacySeam.test.ts`): every + * assertion inspects CONTENT that crosses a boundary. None asserts "a masking + * function was called" — a call-count assertion stays green over a masking + * function that returns its input unchanged, the exact false-green this repo + * has been burned by. The privacy handle here performs a REAL redaction. + */ + +import { after, describe, it } from 'node:test'; +import { strict as assert } from 'node:assert'; +import { + createServer, + type IncomingMessage, + type Server as HttpServer, + type ServerResponse, +} from 'node:http'; +import type { AddressInfo } from 'node:net'; + +import { Server as McpSdkServer } from '@modelcontextprotocol/sdk/server/index.js'; +import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js'; +import { + CallToolRequestSchema, + ListToolsRequestSchema, +} from '@modelcontextprotocol/sdk/types.js'; + +import { + McpManager, + NativeToolRegistry, + ToolDispatchService, + mcpNativeHandler, + turnContext, + type McpServerConfig, + type McpSidecarPayload, + type McpStructuredOutputSidecar, +} from '@omadia/orchestrator'; +import type { PrivacyTurnHandle } from '../packages/harness-orchestrator/src/privacyHandle.js'; + +// ── fixtures ──────────────────────────────────────────────────────────────── + +const EMAIL = 'erika.mustermann@example.com'; +const IBAN = 'DE89370400440532013000'; +const PERSON = 'Erika Mustermann'; + +/** The tool's declared output shape — what a generic renderer would key off. */ +const OUTPUT_SCHEMA = { + type: 'object', + properties: { + name: { type: 'string' }, + email: { type: 'string' }, + iban: { type: 'string' }, + }, +} as const; + +/** The server answers with the SAME PII in both channels: the text block (which + * the dispatcher interns) and `structuredContent` (which nothing interns). */ +const STRUCTURED_PAYLOAD = { + name: PERSON, + email: EMAIL, + iban: IBAN, +} as const; + +const TEXT_PAYLOAD = `{"name":"${PERSON}","email":"${EMAIL}","iban":"${IBAN}"}`; + +const TOOL = 'crm_lookup_customer'; + +/** + * A privacy handle that genuinely redacts, so a missing masking call shows up + * as surviving raw PII rather than as an unmet expectation. + */ +function redactingPrivacyHandle(): PrivacyTurnHandle { + return { + async internToolResultV4({ toolName, rawResult }) { + const redacted = rawResult + .replaceAll(EMAIL, '[masked:email]') + .replaceAll(IBAN, '[masked:iban]') + .replaceAll(PERSON, '[masked:person]'); + return { digestText: `«dataset:${toolName}» ${redacted}`, datasetId: `ds-${toolName}` }; + }, + async recordBypassedTool() { + /* no bypass configured in this file */ + }, + checkBypass() { + return undefined; + }, + async runV4Tool() { + throw new Error('not used on this path'); + }, + async subAgentResultV4() { + throw new Error('not used on this path'); + }, + async takeRenderedAnswerV4() { + return undefined; + }, + v4ToolSpecs() { + return []; + }, + async maskUserPrompt() { + return { outcome: 'disabled' }; + }, + async restorePromptPseudonyms(text) { + return text; + }, + snapshotPromptRestorer() { + return undefined; + }, + async finalize() { + return undefined; + }, + }; +} + +// ── a real MCP server over a real socket ──────────────────────────────────── + +interface FakeServerHandle { + readonly url: string; + close(): Promise; +} + +function buildMcpServerInstance(): McpSdkServer { + const mcp = new McpSdkServer( + { name: 'crm', version: '1.0.0' }, + { capabilities: { tools: {} } }, + ); + mcp.setRequestHandler(ListToolsRequestSchema, async () => ({ + tools: [ + { + name: TOOL, + description: 'look up a customer record', + inputSchema: { type: 'object' as const, properties: {} }, + outputSchema: OUTPUT_SCHEMA, + }, + ], + })); + mcp.setRequestHandler(CallToolRequestSchema, async () => ({ + content: [{ type: 'text' as const, text: TEXT_PAYLOAD }], + structuredContent: STRUCTURED_PAYLOAD, + })); + return mcp; +} + +async function startFakeMcpServer(): Promise { + const handle = async (req: IncomingMessage, res: ServerResponse): Promise => { + const mcp = buildMcpServerInstance(); + const transport = new StreamableHTTPServerTransport({ + sessionIdGenerator: undefined, + enableJsonResponse: true, + }); + res.on('close', () => { + void transport.close().catch(() => {}); + void mcp.close().catch(() => {}); + }); + await mcp.connect(transport); + if (req.method === 'POST') { + const chunks: Buffer[] = []; + for await (const c of req) chunks.push(c as Buffer); + const raw = Buffer.concat(chunks).toString('utf8'); + await transport.handleRequest(req, res, raw.length > 0 ? JSON.parse(raw) : undefined); + return; + } + await transport.handleRequest(req, res); + }; + const http: HttpServer = createServer((req, res) => { + void handle(req, res).catch(() => { + if (!res.headersSent) res.writeHead(500); + res.end(); + }); + }); + const sockets = new Set(); + http.on('connection', (socket) => { + sockets.add(socket); + socket.on('close', () => sockets.delete(socket)); + }); + await new Promise((resolve) => http.listen(0, '127.0.0.1', () => resolve())); + const { port } = http.address() as AddressInfo; + return { + url: `http://127.0.0.1:${port}/mcp`, + async close() { + for (const socket of sockets) socket.destroy(); + sockets.clear(); + await new Promise((resolve) => http.close(() => resolve())); + }, + }; +} + +const fake = await startFakeMcpServer(); +const managers: McpManager[] = []; + +// Teardown runs regardless of assertion outcome — a server closed only after a +// passing assertion turns a red run into a HANG, which is how a sibling agent's +// mutation check failed to report in this wave. +after(async () => { + for (const m of managers) { + try { + await m.closeAll(); + } catch { + /* teardown must not mask a test failure */ + } + } + await fake.close(); +}); + +const CFG: McpServerConfig = { + id: '00000000-0000-4000-8000-000000000547', + name: 'Kunden-CRM', + transport: 'http', + endpoint: fake.url, +}; + +interface Harness { + readonly manager: McpManager; + readonly sidecars: McpSidecarPayload[]; + readonly service: ToolDispatchService; +} + +/** Wire the REAL production chain: `ToolDispatchService` -> `NativeToolRegistry` + * -> `mcpNativeHandler` -> `McpManager.callTool` -> `structuredSink`. */ +function harness(): Harness { + const sidecars: McpSidecarPayload[] = []; + const manager = new McpManager({ structuredSink: (p) => sidecars.push(p) }); + managers.push(manager); + const nativeTools = new NativeToolRegistry(); + nativeTools.register(TOOL, { + handler: mcpNativeHandler(manager, CFG, TOOL), + spec: { + name: TOOL, + description: 'look up a customer record', + input_schema: { type: 'object', properties: {} }, + }, + domain: 'mcp.kunden-crm', + }); + const service = new ToolDispatchService({ + nativeTools, + domainTools: [], + privacy: () => redactingPrivacyHandle(), + }); + return { manager, sidecars, service }; +} + +function structuredSidecars( + sidecars: readonly McpSidecarPayload[], +): McpStructuredOutputSidecar[] { + return sidecars.filter( + (p): p is McpStructuredOutputSidecar => p.kind === 'structured_output', + ); +} + +// ── the finding ───────────────────────────────────────────────────────────── + +describe('#547 structured-output sidecar vs. the privacy boundary (W5-2)', () => { + it('BASELINE — the TEXT result a client receives IS masked at the dispatch seam', async () => { + const h = harness(); + + const result = await h.service.dispatch(TOOL, {}); + + // This is the benchmark the brief calls "the same terms as text output". + assert.equal( + result.content.includes(EMAIL), + false, + 'the email reached the caller in clear — the text seam did not mask', + ); + assert.equal(result.content.includes(IBAN), false, 'the IBAN reached the caller in clear'); + assert.equal( + result.content.includes(PERSON), + false, + 'the person name reached the caller in clear', + ); + // Masked rather than dropped. + assert.match(result.content, /\[masked:email\]/); + assert.match(result.content, /«dataset:crm_lookup_customer»/); + }); + + it('LEAK — the STRUCTURED sidecar carries the same PII in CLEAR on the same call', async () => { + const h = harness(); + + const result = await h.service.dispatch(TOOL, {}); + + // Same dispatch, same privacy handle installed, same PII. + assert.equal(result.content.includes(EMAIL), false, 'precondition: the text WAS masked'); + + const structured = structuredSidecars(h.sidecars); + assert.equal(structured.length, 1, 'exactly one structured sidecar for one call'); + const payload = structured[0]!.structured as Record; + + // The load-bearing assertions: raw values, byte-identical to what the + // server sent. Wiring this payload onto the `done` event would put every + // one of these into the browser on a turn where the text was masked. + assert.equal(payload['email'], EMAIL); + assert.equal(payload['iban'], IBAN); + assert.equal(payload['name'], PERSON); + assert.deepEqual(payload, STRUCTURED_PAYLOAD); + }); + + it('the sidecar is emitted BENEATH the dispatcher, so no dispatcher can mask it', async () => { + // Proves the leak is structural rather than a property of one dispatcher: + // the payload is already in the sink by the time `dispatch` returns, and + // the value in the sink is unaffected by the masking that produced + // `result.content`. + const h = harness(); + + const result = await h.service.dispatch(TOOL, {}); + + const structured = structuredSidecars(h.sidecars); + assert.equal(structured.length, 1); + assert.match(result.content, /\[masked:person\]/, 'the string path was masked'); + assert.equal( + (structured[0]!.structured as Record)['name'], + PERSON, + 'the sidecar payload was NOT masked by the same dispatch', + ); + }); + + it('carries the declared `outputSchema`, so a generic renderer is buildable once masking exists', async () => { + // Not a leak assertion — it records that the ONLY blocker is masking. The + // renderer contract the brief specifies (render from `outputSchema`, never + // by tool name) is already satisfiable end-to-end over a real wire. + const h = harness(); + await h.manager.listTools(CFG); // discovery caches the schema + + await h.service.dispatch(TOOL, {}); + + const structured = structuredSidecars(h.sidecars); + assert.equal(structured.length, 1); + assert.deepEqual(structured[0]!.outputSchema, OUTPUT_SCHEMA); + }); + + it('attaches the ambient `turnId`, so correlation onto a done event is already possible', async () => { + const h = harness(); + + await turnContext.run( + { turnId: 't-547', turnDate: '2026-07-31', agentSlug: 'main', userId: 'u1' } as never, + () => h.service.dispatch(TOOL, {}), + ); + + const structured = structuredSidecars(h.sidecars); + assert.equal(structured.length, 1); + assert.equal(structured[0]!.turnId, 't-547'); + }); + + it('emits NO sidecar for a tool whose result has no structuredContent', async () => { + // The `ToolRow` fallback case the brief requires to survive untouched. + const sidecars: McpSidecarPayload[] = []; + const manager = new McpManager({ structuredSink: (p) => sidecars.push(p) }); + managers.push(manager); + const nativeTools = new NativeToolRegistry(); + nativeTools.register('plain_tool', { + handler: async () => 'just text', + spec: { + name: 'plain_tool', + description: 'no structured output', + input_schema: { type: 'object', properties: {} }, + }, + domain: 'test.plain', + }); + const service = new ToolDispatchService({ nativeTools, domainTools: [] }); + + const result = await service.dispatch('plain_tool', {}); + + assert.equal(result.content, 'just text'); + assert.deepEqual(structuredSidecars(sidecars), []); + }); +}); From b6988912bdd414a23d7be9de7fbe90e1c0bec9e6 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 13:36:33 +0200 Subject: [PATCH 77/90] test(mcp): import the #547 privacy proof from source, not the dist barrel The first revision imported ToolDispatchService/McpManager from '@omadia/orchestrator', whose main field is dist/index.js. A mutation applied to src/ was therefore invisible and the mutation check reported GREEN over deliberately broken production code -- the exact false-green class this repo has been burned by. Source imports also guarantee a single module instance, so the turnContext AsyncLocalStorage the sidecar reads is the one this file writes. --- .../test/mcpStructuredOutputPrivacy.test.ts | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/middleware/test/mcpStructuredOutputPrivacy.test.ts b/middleware/test/mcpStructuredOutputPrivacy.test.ts index 3bee2019..b929a38b 100644 --- a/middleware/test/mcpStructuredOutputPrivacy.test.ts +++ b/middleware/test/mcpStructuredOutputPrivacy.test.ts @@ -64,16 +64,23 @@ import { ListToolsRequestSchema, } from '@modelcontextprotocol/sdk/types.js'; +// Imported from SOURCE, not from the `@omadia/orchestrator` barrel. The barrel +// resolves to `dist/`, so a mutation applied to `src/` would not be visible +// without a rebuild — and a mutation check that silently exercises a stale +// artifact reports GREEN over broken production code. Same convention as +// `toolDispatchPrivacySeam.test.ts`. Keeping every orchestrator import on the +// source path also guarantees ONE module instance, so the `turnContext` +// AsyncLocalStorage the sidecar reads is the one this file writes. import { McpManager, - NativeToolRegistry, - ToolDispatchService, mcpNativeHandler, - turnContext, type McpServerConfig, type McpSidecarPayload, type McpStructuredOutputSidecar, -} from '@omadia/orchestrator'; +} from '../packages/harness-orchestrator/src/mcp/mcpClient.js'; +import { NativeToolRegistry } from '../packages/harness-orchestrator/src/nativeToolRegistry.js'; +import { ToolDispatchService } from '../packages/harness-orchestrator/src/toolDispatchService.js'; +import { turnContext } from '../packages/harness-orchestrator/src/turnContext.js'; import type { PrivacyTurnHandle } from '../packages/harness-orchestrator/src/privacyHandle.js'; // ── fixtures ──────────────────────────────────────────────────────────────── From 3c5c53c4519566c6aca378bb00d6be4e7636594e Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 13:42:03 +0200 Subject: [PATCH 78/90] docs(changelog): record the wave 4-6 fixes and the #547 privacy limit --- docs/CHANGELOG.md | 89 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index fb3f2e38..d15aeb18 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -18,6 +18,95 @@ entry. See `CONTRIBUTING.md` § Releases & changelog. ## [Unreleased] +### Fixed — `per_user` MCP delegation was unreachable from chat + +- Migration `0031` made delegation explicit per MCP server and gave new servers a + fail-closed `per_user` default. `resolveMcpUserKey` reads + `turnContext.current()?.mcpUserKey` — but **the only thing that ever set it was + the operator discover route.** `routes/chat.ts` did not so much as import + `turnContext`. Every newly created `per_user` server was therefore dead from + chat out of the box: no token sent, the audit row recording the literal + `unresolved`, and the turn failing closed. Existing installs were masked only + because `0031` backfills token-holding servers to `service`. +- Both HTTP chat entries now open a turn scope carrying `mcpUserKey`. The + streaming entry uses `turnContext.runGenerator`, not `enter`: `enterWith` binds + to the async resource executing at that instant, and an async generator resumes + in the caller's context, so the identity would be gone by the orchestrator's + first yield — before any tool, and therefore before any MCP call, runs. +- The value is `sessionIdentity(req)` (`session.sub || session.email`), extracted + from `routes/agentBuilder.ts` into `src/auth/sessionIdentity.ts`. Deliberately + **not** `resolveUserId(req)`, which falls through to the client-sent + `x-user-id` header — keying MCP tokens on a client-controlled header would let + any caller act as any user. When nothing resolves, `mcpUserKey` stays unset and + a `per_user` server fails closed exactly as intended; there is no fallback. +- Channel turns set `mcpUserKey` inside the orchestrator from the already-resolved + `resolvedOmadiaUserId`, gated on `channelIdentity` — which only the dispatcher + mints, from the adapter's authenticated `userRef`, so it is server-attested end + to end. ⚠️ **Known limit:** channel turns key on the canonical omadia uuid while + `/authorize` stores tokens under the session-shaped key, so an affected user + still fails closed rather than reaching their server. Closing that needs a new + method on the `KnowledgeGraph` contract. Narrower than it sounds: a `per_user` + token can only exist for someone who completed `/authorize`, which requires a + session, so a channel-only user has no token and failing closed is correct. + +### Fixed — migration `0031` built neither of its guards reliably + +- The CHECK guard looked up `pg_constraint` by `conname` alone. `conname` is + unique per `(connamespace, conrelid)`, not cluster-wide, so a same-named + constraint in **any** other schema made the guard true and the `ALTER TABLE` was + silently skipped — the migration did not build the constraint it claims to. Now + anchored on `conrelid = 'mcp_servers'::regclass`. +- The backfill guard hardcoded `to_regclass('public.mcp_oauth_tokens')` in a file + that is otherwise entirely unqualified, so wherever the domain is applied outside + `public` it answered about a table the statement never touches. Demonstrated on a + database with an empty `public`: the old guard left an operator-token server on + `per_user`, losing its grandfathering and breaking it fail-closed. +- The backfill test previously **rewrote** the migration to make it apply; it now + applies verbatim, with a guard that fails if a schema-qualified reference is ever + reintroduced, plus the assertion the suite had dropped as a known flake. + +### Fixed — the middleware suite had no per-test timeout + +- `--test-timeout=120000`. Previously unset, so Node's default of `Infinity` + applied and a hung test burned the CI job's 15-minute wall with no attribution. + Note the ceiling is **per file**, not per leaf — a file whose total exceeds it is + killed as a unit — so the value is sized on the slowest file (18.4 s), not the + slowest test (7.8 s). `web-ui` needs no change; vitest already bounds at 5 s. + +### Added — operator surface for public MCP key bindings + +- The public MCP endpoint's authorization is driven entirely by rows in + `public_mcp_key_bindings`, and there was **no way to create one** except + hand-written SQL — the endpoint was inert as shipped. A Public API keys tab in + the MCP Control Center now lists, creates and revokes bindings. +- The public endpoint's dependency bag is unchanged and still receives the + read-only store: it gains no write path to its own authorization table. The + admin path validates through the same `normalizeBindingRow` the enforcement path + uses, so the two cannot drift. Revoke parks the row rather than deleting it. + +### Fixed — raw NUL bytes made ripgrep silently truncate eight source files + +- Fifteen literal `0x00` bytes, used as composite map-key separators, are now + written as `\0`. Provably a no-op — none is followed by an ASCII digit, the only + case where the escape would change meaning. Behaviour is bit-identical; what + changes is that `rg` no longer classifies these files as binary and stops + searching partway through, silently truncating every audit that crosses them. + +### Known limitation — #547 structured content cannot be rendered yet + +- `emitStructured` fires inside `McpManager.callTool`, strictly beneath every + dispatcher, while both client-facing paths take their tool text from + `dispatchTool` → `internToolResultV4`. The sidecar therefore never crosses the + privacy handle, and wiring it onto the `done` event — which is the client wire — + would put raw MCP tool output in the browser on turns where the equivalent text + is interned by Privacy Shield v4. +- `PrivacyTurnHandle` is string-in/string-out, so masking a structured payload + while preserving its structure needs a new method on the published + `@omadia/plugin-api` surface. `middleware/test/mcpStructuredOutputPrivacy.test.ts` + pins the bypass over a real MCP socket so it cannot silently widen, and confirms + `outputSchema` and `turnId` already reach the sidecar — the renderer is buildable + the moment masking exists. **Do not wire `structuredSink` until then.** + ### Added — public, stateless MCP endpoint (`POST /api/v1/mcp`) - omadia can now expose **its own tools** over a stateless Streamable-HTTP MCP From 03ce9674bbf5ef246c0e46aed2c806ef9dbc21f5 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 13:42:12 +0200 Subject: [PATCH 79/90] chore: escape raw NUL bytes as \0 in composite map-key separators Fifteen literal 0x00 bytes across eight source files caused ripgrep to classify those files as binary and silently stop searching after the first match. Every grep-based audit over them has been truncated without notice. Replaced each raw byte with the two-character escape sequence \0. Verified byte-exact and behaviour-neutral: no occurrence is followed by an ASCII digit (which would make it a legacy octal escape and a SyntaxError under strict mode), all fifteen sit inside string or template literals rather than regex literals or JSON, and every construct still evaluates to a real U+0000. Includes the two pathological-input defences the separators exist for -- the daemonProtocol NUL-injection rejection fixture and the postgres virtual path guard -- both unchanged in behaviour. --- .../src/postgresMemoryStore.ts | Bin 9514 -> 9515 bytes .../src/recallRelevanceJudge.ts | Bin 8657 -> 8659 bytes .../src/registry/index.ts | 2 +- .../llm-provider/src/modelRegistry.ts | Bin 14333 -> 14335 bytes middleware/src/routes/agentBuilder.ts | 8 ++++---- .../test/devplatform/daemonProtocol.test.ts | Bin 7382 -> 7383 bytes middleware/test/profileStorage.test.ts | Bin 14397 -> 14399 bytes web-ui/app/_lib/chatStreamEvents.ts | 2 +- 8 files changed, 6 insertions(+), 6 deletions(-) diff --git a/middleware/packages/harness-memory-postgres/src/postgresMemoryStore.ts b/middleware/packages/harness-memory-postgres/src/postgresMemoryStore.ts index 3536ee806160a75907d9e203e5c78efed0a2b1e2..5c7c74556615ca98f6dab841a1e6708ed70d727b 100644 GIT binary patch delta 24 gcmZ4Gwc2aLb}8l<1NF&*eA1I+rNlSCma^vp0B*ks5C8xG delta 23 fcmZ4OwaRP5b}1$X^~r*K(vxGQ#5cc@vgZN-Vz&o` diff --git a/middleware/packages/harness-orchestrator-extras/src/recallRelevanceJudge.ts b/middleware/packages/harness-orchestrator-extras/src/recallRelevanceJudge.ts index 2fec9e0da3d6283c3f3b1ba0fa9ec00050bc4a54..fcbcbd86f9a84cec161a758160c37fa5d65314eb 100644 GIT binary patch delta 38 tcmccUeA#(}8lPm0fl760acYroYH@L5da7PYQD&}&W^IhYW?Q}~!T=fh4Zr{Z delta 36 rcmccYe9?J>8lO0WN_A;*YLRbhadBdLs$NM^X0C>2EyHGezA3^0_q7bD diff --git a/middleware/packages/harness-orchestrator/src/registry/index.ts b/middleware/packages/harness-orchestrator/src/registry/index.ts index f97e9ec7..00650a2a 100644 --- a/middleware/packages/harness-orchestrator/src/registry/index.ts +++ b/middleware/packages/harness-orchestrator/src/registry/index.ts @@ -624,7 +624,7 @@ export function validateSnapshot( const seenBindings = new Set(); for (const binding of snap.channelBindings) { - const key = `${binding.channelType}${binding.channelKey}`; + const key = `${binding.channelType}\0${binding.channelKey}`; if (seenBindings.has(key)) { throw new ConfigValidationError( `duplicate channel binding (${binding.channelType}, ${binding.channelKey})`, diff --git a/middleware/packages/llm-provider/src/modelRegistry.ts b/middleware/packages/llm-provider/src/modelRegistry.ts index 646d7986c30a8014465972b0a420ae1fbc2d4dc5..494d924b468db7865e163c0c4fd68c50365a2ba9 100644 GIT binary patch delta 34 qcmeyH|3802hY(YY!RAh(&&;eb2I`vDlNCk9C%+Sr+PqQBM-c$_H4KFS delta 21 bcmeyL|2Ka_hY%yf<}RVn%s{GMtU(a~Y#Rtw diff --git a/middleware/src/routes/agentBuilder.ts b/middleware/src/routes/agentBuilder.ts index c459ebb0..ea04e255 100644 --- a/middleware/src/routes/agentBuilder.ts +++ b/middleware/src/routes/agentBuilder.ts @@ -2629,15 +2629,15 @@ async function withToolVerdicts( l.graph.listMcpToolVerdicts(CURRENT_VERIFIER_VERSION), l.graph.listMcpToolVerdictAcks(CURRENT_VERIFIER_VERSION), ]); - const vmap = new Map(verdicts.map((v) => [`${v.serverId}${v.toolName}`, v])); - const amap = new Map(acks.map((a) => [`${a.serverId}${a.toolName}`, a])); + const vmap = new Map(verdicts.map((v) => [`${v.serverId}\0${v.toolName}`, v])); + const amap = new Map(acks.map((a) => [`${a.serverId}\0${a.toolName}`, a])); return servers.map((s) => ({ ...s, discoveredTools: (s.discoveredTools as ReadonlyArray>).map( (tool) => { const name = typeof tool['name'] === 'string' ? (tool['name'] as string) : ''; - const v = vmap.get(`${s.id}${name}`); - const a = amap.get(`${s.id}${name}`); + const v = vmap.get(`${s.id}\0${name}`); + const a = amap.get(`${s.id}\0${name}`); const ackValid = v !== undefined && a !== undefined && a.contentHash === v.contentHash; const verdict: McpToolVerdictField = v ? { diff --git a/middleware/test/devplatform/daemonProtocol.test.ts b/middleware/test/devplatform/daemonProtocol.test.ts index c6e6c3dfe8f04d1ffe6165c00dbae6d5ee0bb892..15a7bbe5c6a26042c5d25ce48b61554d240abbe3 100644 GIT binary patch delta 15 Wcmca+dEIiu2|lJ6gUu)Tc=-S~)dkrA delta 14 Vcmca^dChXe2|h-K&8PTy`2aA81x)|| diff --git a/middleware/test/profileStorage.test.ts b/middleware/test/profileStorage.test.ts index fd8f884830a019c44e93bc23d1747106fe81efe4..bc89c8e0cd4c2ad0cd2e442e877fadaccd697771 100644 GIT binary patch delta 23 bcmdm6u)ko#6gH+9gUwUf)WGx}o `${e.from}${e.to}${e.type}`; + const edgeKey = (e: KgWalkEdge): string => `${e.from}\0${e.to}\0${e.type}`; const insertedEdgeKeys = new Set(insert.edges.map(edgeKey)); const edges: KgWalkEdge[] = prior.edges.map((e) => insertedEdgeKeys.has(edgeKey(e)) ? { ...e, inserted: true } : e, From 652ae116deff8a2ec411efa9af2ab4946dbeb42e Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 14:00:50 +0200 Subject: [PATCH 80/90] fix(memory): the NUL guard blamed a space, and had no coverage at all postgresMemoryStore.normalize() rejects a virtual path containing a NUL byte and then reports 'Path contains a space.' -- almost certainly because the raw 0x00 in the source was invisible to whoever wrote it. An error that misnames the input it rejected sends the reader looking for a bug that is not there. Found while escaping that same NUL (W6-1). The guard also turned out to have no effective coverage: disabling it outright left the entire middleware suite green. Its only test is memoryStoreConformance.pg.test.ts, which skips without a postgres -- and CI has no postgres service on the middleware job, so it never runs there at all -- and it asserts nothing about NUL regardless. The new tests need no postgres by construction: normalize() runs before any query, so a pool that THROWS when queried is itself the assertion. If validation ever moved after the first query, the fake fires and the test fails with the wrong error. A negative control pins that a clean path survives validation and reaches the pool, so a normalize() that rejected everything could not pass. Mutation-checked with a rebuild, because the test imports through the package barrel and a src-only mutation would have been invisible: removing the guard fails 3 of 4, negative control still green. --- .../src/postgresMemoryStore.ts | 2 +- .../postgresMemoryStorePathValidation.test.ts | 103 ++++++++++++++++++ 2 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 middleware/test/postgresMemoryStorePathValidation.test.ts diff --git a/middleware/packages/harness-memory-postgres/src/postgresMemoryStore.ts b/middleware/packages/harness-memory-postgres/src/postgresMemoryStore.ts index 5c7c7455..d663b75b 100644 --- a/middleware/packages/harness-memory-postgres/src/postgresMemoryStore.ts +++ b/middleware/packages/harness-memory-postgres/src/postgresMemoryStore.ts @@ -204,7 +204,7 @@ export class PostgresMemoryStore implements MemoryStore { throw new MemoryInvalidPathError('Path must be a non-empty string.'); } if (virtualPath.includes('\0')) { - throw new MemoryInvalidPathError('Path contains a space.'); + throw new MemoryInvalidPathError('Path contains a NUL byte.'); } const lowered = virtualPath.toLowerCase(); if ( diff --git a/middleware/test/postgresMemoryStorePathValidation.test.ts b/middleware/test/postgresMemoryStorePathValidation.test.ts new file mode 100644 index 00000000..1196bda5 --- /dev/null +++ b/middleware/test/postgresMemoryStorePathValidation.test.ts @@ -0,0 +1,103 @@ +import assert from 'node:assert/strict'; +import { describe, it } from 'node:test'; + +import { MemoryInvalidPathError } from '@omadia/memory'; +import { PostgresMemoryStore } from '@omadia/memory-postgres'; +import type { Pool } from 'pg'; + +/** + * W6-1 follow-through — coverage for `PostgresMemoryStore`'s NUL-byte guard. + * + * The guard existed but was provably untested: disabling it entirely left the + * whole middleware suite green. Its only coverage was + * `memoryStoreConformance.pg.test.ts`, which skips without a Postgres and + * asserts nothing about NUL anyway — and CI has no Postgres service on the + * middleware job, so that suite never runs there at all. + * + * These tests need no Postgres by construction. `normalize()` runs before any + * query, so a pool that THROWS when queried is the assertion: if validation + * ever moved after the first query, the fake would fire and the test would fail + * with the wrong error. + */ + +/** A pool whose only behaviour is to fail loudly if anyone reaches it. */ +function poolThatMustNotBeQueried(): Pool { + return { + query() { + throw new Error('the store queried the database before validating the path'); + }, + connect() { + throw new Error('the store took a client before validating the path'); + }, + } as unknown as Pool; +} + +describe('PostgresMemoryStore path validation', () => { + it('rejects a path containing a NUL byte, before touching the pool', async () => { + const store = new PostgresMemoryStore(poolThatMustNotBeQueried()); + + await assert.rejects( + () => store.fileExists('/memories/core/no\0tes.md'), + (err: unknown) => { + assert.ok( + err instanceof MemoryInvalidPathError, + `expected MemoryInvalidPathError, got ${String(err)}`, + ); + return true; + }, + ); + }); + + it('names the NUL byte in the error, rather than reporting a space', async () => { + // The message read 'Path contains a space.' until W6-1 — almost certainly + // because the raw 0x00 in the source was invisible to whoever wrote it. + // An error that misnames the input it rejected sends the reader looking for + // a bug that is not there. + const store = new PostgresMemoryStore(poolThatMustNotBeQueried()); + + await assert.rejects( + () => store.fileExists('/memories/core/no\0tes.md'), + (err: unknown) => { + const message = err instanceof Error ? err.message : String(err); + assert.match(message, /NUL/i, `error should name the NUL byte, got: ${message}`); + assert.doesNotMatch( + message, + /space/i, + `error must not blame a space for a NUL byte, got: ${message}`, + ); + return true; + }, + ); + }); + + it('applies the guard on every entry point, not just one', async () => { + const store = new PostgresMemoryStore(poolThatMustNotBeQueried()); + const bad = '/memories/core/no\0tes.md'; + + // Each of these normalises before its first query. A future refactor that + // validates in only one of them is the failure this pins. + await assert.rejects(() => store.list(bad), MemoryInvalidPathError); + await assert.rejects(() => store.fileExists(bad), MemoryInvalidPathError); + await assert.rejects(() => store.readFile(bad), MemoryInvalidPathError); + await assert.rejects(() => store.writeFile(bad, 'x'), MemoryInvalidPathError); + }); + + it('accepts an ordinary path far enough to reach the pool', async () => { + // The negative control. Without it, a `normalize` that rejected EVERY path + // would satisfy all three tests above. + const store = new PostgresMemoryStore(poolThatMustNotBeQueried()); + + await assert.rejects( + () => store.fileExists('/memories/core/notes.md'), + (err: unknown) => { + const message = err instanceof Error ? err.message : String(err); + assert.match( + message, + /before validating the path/, + `a clean path must survive validation and reach the pool, got: ${message}`, + ); + return true; + }, + ); + }); +}); From a502db3b02a5703eb4ffb8eb797ab80cbdfa6ac9 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 14:12:46 +0200 Subject: [PATCH 81/90] fix(web-ui): raise the vitest timeout above the honest cost of the RTL suites vitest defaults testTimeout to 5000ms and web-ui never overrode it, but the heavier React Testing Library suites -- template proposals, slot pickers, the publish flow -- measure 5-13s unloaded. A ceiling below a test's real cost does not catch hangs, it manufactures them. Demonstrated rather than assumed: four runs of an unchanged tree gave 0, 9, 25 and 0 failures purely as machine load varied, and every failure was a timeout at the 5000ms mark rather than an assertion. The 25-failure run happened while the middleware suite was running beside it. 30s, and the same for hooks. Deliberately generous, matching the middleware --test-timeout added in this wave: the job of the number is to stop a hung test from burning the CI wall with no attribution, not to police tests that are slow but honest. Verified against the condition that produced the failures: 427/427 pass while the full middleware suite runs concurrently. This corrects an earlier finding in the same wave that read web-ui as already adequately bounded. It was bounded, but in the wrong direction. --- web-ui/vitest.config.ts | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/web-ui/vitest.config.ts b/web-ui/vitest.config.ts index 88c473be..011ec707 100644 --- a/web-ui/vitest.config.ts +++ b/web-ui/vitest.config.ts @@ -19,6 +19,18 @@ export default defineConfig({ setupFiles: ['./vitest.setup.ts'], include: ['app/**/*.{test,spec}.{ts,tsx}'], globals: true, + // vitest's default is 5000ms, which sits BELOW the honest runtime of the + // heavier React Testing Library suites here — the template-proposal and + // slot-picker renders measure 5-13s unloaded. A ceiling under a test's real + // cost does not catch hangs, it manufactures them: four runs of an + // unchanged tree gave 0, 9, 25 and 0 failures, purely as machine load + // varied, every one of them a timeout rather than an assertion. + // + // 30s is deliberately generous. The job of this number is to stop a hung + // test from burning the CI wall with no attribution, not to police tests + // that are slow but honest. Raise it rather than trimming a real suite. + testTimeout: 30_000, + hookTimeout: 30_000, }, resolve: { alias: { From f628625f01054b1697701fd81cece3ccfb711d7c Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 14:19:20 +0200 Subject: [PATCH 82/90] chore(470): correct the decoupling baseline that raw NUL bytes had understated The ratchet reported middleware/test rising 1036 -> 1043 in this wave. None of it is new coupling: no test file added here contains a single dev-platform token, and no changed test file's count moves when measured with rg --text. The cause is the metric itself. scripts/check-core-decoupling.mjs invokes rg without --text and traverses directories, which is exactly the mode where ripgrep applies binary detection -- and two files in this zone carried raw NUL bytes, so rg classified them as binary and stopped reading partway through. Seven dev-platform references sat past that cut-off and were never counted. Proven by reversal: put the raw NULs back and the zone reports 1036; escape them and it reports 1043. Nothing about the code changed between those two measurements. So this is not a baseline being raised to accommodate new debt -- it is a baseline being corrected upward to the number that was always true. The decoupling ratchet has been under-counting since those NULs were introduced, which also means it would not have noticed new coupling added past a NUL in either file. --- specs/470-dev-platform-plugin/decoupling-baseline.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/470-dev-platform-plugin/decoupling-baseline.json b/specs/470-dev-platform-plugin/decoupling-baseline.json index 61d78159..7763ab17 100644 --- a/specs/470-dev-platform-plugin/decoupling-baseline.json +++ b/specs/470-dev-platform-plugin/decoupling-baseline.json @@ -1,8 +1,8 @@ { - "total": 3441, + "total": 3448, "zones": { "middleware/src": 1702, - "middleware/test": 1036, + "middleware/test": 1043, "middleware/packages": 97, "middleware/scripts": 8, "middleware/sidecars": 195, From a290566dd48d5465612c72299e509385694f34c9 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 14:20:49 +0200 Subject: [PATCH 83/90] fix(mcp): close three findings from the cross-vendor review of W4-1 1. routineRunner's two branches inherited opposite identities. The templated branch opens a fresh turn scope; the untemplated one opened none, so orchestrator.runTurn read whatever ambient scope was open as its parent and inherited its mcpUserKey. A routine fired from inside a chat turn would then run under the INVOKING user's MCP identity rather than its owner's -- and the same routine would behave differently depending only on whether an output template happens to be configured. Both branches now open the scope. 2. 'server-attested end to end' overclaimed. The dispatcher copies userRef.id verbatim and verifies nothing itself, so the guarantee is exactly as strong as the inbound-webhook authentication in the Teams/Telegram/Slack adapters -- which live outside this repo and cannot be checked from here. The comment now says adapter-attested, and records the bound: resolveOrCreateChannelIdentity creates on miss, so a forged id matching no known identity mints a fresh uuid holding no token and fails closed. Impersonation needs an already-known channel user id. 3. A lone ?? in an otherwise truthiness-based chain. Every other link guards on truthiness, so a parent carrying an empty string would short-circuit the ??, suppress the valid key the channel branch would have produced, and then be dropped by the truthy spread -- silently downgrading a resolvable turn to unresolved. Unreachable today; defect-in-waiting. The review's central question -- can a client-controlled value reach mcpUserKey -- came back no across all nine enumerated callers of runTurn and chatStream, including the server-to-server API-key channel from #438/#439, which sets kind:'custom', gets no channelIdentity, and fails closed. --- .../harness-orchestrator/src/orchestrator.ts | 38 ++++++++++++++++--- .../src/plugins/routines/routineRunner.ts | 22 ++++++++--- 2 files changed, 49 insertions(+), 11 deletions(-) diff --git a/middleware/packages/harness-orchestrator/src/orchestrator.ts b/middleware/packages/harness-orchestrator/src/orchestrator.ts index 7ab59cf6..b462d29b 100644 --- a/middleware/packages/harness-orchestrator/src/orchestrator.ts +++ b/middleware/packages/harness-orchestrator/src/orchestrator.ts @@ -2659,10 +2659,23 @@ export class Orchestrator { // Keying MCP tokens on it would let any caller act as any user — W0-1's // confused deputy, re-opened one door along. A `channelIdentity` is minted // only by `createOrchestratorDispatcher` from the adapter's authenticated - // `userRef` and is resolved through the KG, so it is server-attested end to - // end. + // `userRef` and is resolved through the KG. + // + // Precisely how far that attestation reaches: the dispatcher copies + // `userRef.id` verbatim and verifies nothing itself, so the guarantee is + // exactly as strong as the inbound-webhook authentication in the Teams / + // Telegram / Slack adapters — which live outside this repo. It is + // adapter-attested, not attested here. Bounded, though: + // `resolveOrCreateChannelIdentity` creates on miss, so a forged id matching + // no known identity mints a fresh uuid holding no token and fails closed. + // Impersonation needs an already-known channel user id. + // `||`, not `??`: every other link in this chain guards on truthiness (the + // spread below, `chat.ts`'s producer, `turnContext`'s carry-over). With + // `??`, a parent carrying an empty string would short-circuit, suppress the + // valid key this branch would have produced, and then be dropped by the + // truthy spread — silently downgrading a resolvable turn to `unresolved`. const mcpUserKey = - parent?.mcpUserKey ?? + parent?.mcpUserKey || (input.channelIdentity ? resolvedOmadiaUserId : undefined); return turnContext.run( @@ -4168,10 +4181,23 @@ export class Orchestrator { // Keying MCP tokens on it would let any caller act as any user — W0-1's // confused deputy, re-opened one door along. A `channelIdentity` is minted // only by `createOrchestratorDispatcher` from the adapter's authenticated - // `userRef` and is resolved through the KG, so it is server-attested end to - // end. + // `userRef` and is resolved through the KG. + // + // Precisely how far that attestation reaches: the dispatcher copies + // `userRef.id` verbatim and verifies nothing itself, so the guarantee is + // exactly as strong as the inbound-webhook authentication in the Teams / + // Telegram / Slack adapters — which live outside this repo. It is + // adapter-attested, not attested here. Bounded, though: + // `resolveOrCreateChannelIdentity` creates on miss, so a forged id matching + // no known identity mints a fresh uuid holding no token and fails closed. + // Impersonation needs an already-known channel user id. + // `||`, not `??`: every other link in this chain guards on truthiness (the + // spread below, `chat.ts`'s producer, `turnContext`'s carry-over). With + // `??`, a parent carrying an empty string would short-circuit, suppress the + // valid key this branch would have produced, and then be dropped by the + // truthy spread — silently downgrading a resolvable turn to `unresolved`. const mcpUserKey = - parent?.mcpUserKey ?? + parent?.mcpUserKey || (input.channelIdentity ? resolvedOmadiaUserId : undefined); const context: TurnContextValue = { diff --git a/middleware/src/plugins/routines/routineRunner.ts b/middleware/src/plugins/routines/routineRunner.ts index f640a622..40446b2a 100644 --- a/middleware/src/plugins/routines/routineRunner.ts +++ b/middleware/src/plugins/routines/routineRunner.ts @@ -654,11 +654,23 @@ export class RoutineRunner { readonly cardBody?: readonly unknown[]; }> { if (routine.outputTemplate === null) { - const result = await orchestrator.runTurn({ - userMessage: routine.prompt, - userId: routine.userId, - sessionScope: `routine:${routine.id}`, - }); + // The scope is deliberate, and it must match the templated branch below. + // `orchestrator.runTurn` reads `turnContext.current()` as its parent and + // inherits `mcpUserKey` from it. Without a fresh scope here, a routine + // fired while an ambient turn is open would run under the INVOKING user's + // MCP identity rather than its own owner's (`routine.userId`) — and the + // templated branch, which does open one, would behave differently for the + // same routine. Same routine, same owner, two identities, decided by + // whether an output template happens to be configured. + const result = await turnContext.run( + { turnId: '', turnDate: today() }, + () => + orchestrator.runTurn({ + userMessage: routine.prompt, + userId: routine.userId, + sessionScope: `routine:${routine.id}`, + }), + ); return { result }; } From 81addfca648945e6b659d87305962caff75169c1 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 14:32:16 +0200 Subject: [PATCH 84/90] docs(changelog): correct the #547 privacy framing and record the replay bypass An earlier entry in this wave claimed the #547 structured-content sidecar leaks PII to the browser. Cross-vendor review refuted it: Privacy Shield v4's boundary is server<->LLM, not server<->browser. internToolResultV4 returns a digest for the tool_result block while real rows stay server-side behind a datasetId, and the browser receives real values by design -- PrivacyRenderedAnswer.text carries them and the UI highlights maskedValues so the user can see what the server resolved behind the boundary. Getting that wrong blocked a correct feature and proposed a plugin-api contract change that is not needed for the stated reason. Attacking the premise is what surfaced the real defect, which is on the actual boundary and live: the MCP input replay calls callTool directly, skipping dispatchTool and therefore interning, and interpolates the raw result into the note that goes to the model. #547's renderer stays deferred, now for honest reasons: it is a full-stack change on an already-large PR, and the sidecar bypasses receipt and dataset accounting even where masking is not owed. --- docs/CHANGELOG.md | 44 ++++++++++++++++++++++++++++++-------------- 1 file changed, 30 insertions(+), 14 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index d15aeb18..e2426fc0 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -92,20 +92,36 @@ entry. See `CONTRIBUTING.md` § Releases & changelog. changes is that `rg` no longer classifies these files as binary and stops searching partway through, silently truncating every audit that crosses them. -### Known limitation — #547 structured content cannot be rendered yet - -- `emitStructured` fires inside `McpManager.callTool`, strictly beneath every - dispatcher, while both client-facing paths take their tool text from - `dispatchTool` → `internToolResultV4`. The sidecar therefore never crosses the - privacy handle, and wiring it onto the `done` event — which is the client wire — - would put raw MCP tool output in the browser on turns where the equivalent text - is interned by Privacy Shield v4. -- `PrivacyTurnHandle` is string-in/string-out, so masking a structured payload - while preserving its structure needs a new method on the published - `@omadia/plugin-api` surface. `middleware/test/mcpStructuredOutputPrivacy.test.ts` - pins the bypass over a real MCP socket so it cannot silently widen, and confirms - `outputSchema` and `turnId` already reach the sidecar — the renderer is buildable - the moment masking exists. **Do not wire `structuredSink` until then.** +### Fixed — the MCP input-replay path put raw tool output on the LLM wire + +- Privacy Shield v4's boundary is **server ↔ LLM provider**, not server ↔ browser: + `internToolResultV4` returns an identity-free digest for the `tool_result` block + while the real rows stay server-side behind a `datasetId`, and the browser + legitimately receives real values (`PrivacyRenderedAnswer.text`, highlighted via + `maskedValues` so the user can see what the server resolved). +- The replay that runs after a user answers an MCP input card called + `mcpManager.callTool` **directly** rather than going through `dispatchTool`, so + the result was never interned — and was then interpolated verbatim into the note + folded into the turn's ingested text. A replayed HR or accounting tool returning a + personnel row sent that row to the model in cleartext, where the identical tool on + an ordinary turn would have yielded only a digest. +- The comment above the interpolation shows this was a near-miss rather than a + decision: it reasons explicitly about the LLM wire, but only about the user's + typed values, and overlooks the tool result two lines below. Found by + cross-vendor review, live in any deployment with a graph pool. + +### Known limitation — #547 structured content still has no renderer + +- `emitStructured` fires inside `McpManager.callTool`, beneath every dispatcher, so + the sidecar is not interned. `middleware/test/mcpStructuredOutputPrivacy.test.ts` + pins that mechanism over a real MCP socket, and confirms `outputSchema` and + `turnId` already reach the sidecar. +- **This is not a leak to the browser** — an earlier reading of it as one was + corrected by cross-vendor review; the browser is the trusted side. The renderer is + deferred for two ordinary reasons instead: it is a full-stack change across eight + web-ui files on an already-large PR, and the sidecar bypasses Privacy Shield's + receipt and dataset *accounting* even where masking is not owed, which wants a + decision before anything renders from it. ### Added — public, stateless MCP endpoint (`POST /api/v1/mcp`) From d68245dbb72d1baaa81f2fe6a7a8f0ab8cb76602 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 14:32:31 +0200 Subject: [PATCH 85/90] fix(mcp): an omitted field must never widen a public MCP key binding MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-vendor review of the W5-1 admin surface. `public_mcp_key_bindings` rows are the entire authorization model for an internet-facing, API-key-authenticated endpoint, so three of these are grant-widening bugs rather than hygiene. FINDING 1 (HIGH) — revoke was silently undone by any later save. `validateBindingInput` defaulted `enabled: input.enabled ?? true`, and the upsert wrote that manufactured `true` over the stored row. An operator revoked a key after an incident; the next save from a stale tab, a second operator, a config replay, or the admin UI's own form (which does not round-trip `enabled`) handed the third-party key its whole allowlist back, silently, answering 201 Created. Fixed by preserving on conflict: an omitted `enabled` now stays absent through validation and resolves in the store — against the stored column on conflict, against `true` only for a row that does not exist yet. Re-arming requires saying so. Because that makes revoke sticky, un-parking gets its own explicit route (`POST /:keyId/restore`) and a confirmed UI affordance, so the fix does not strand an operator with no way back except psql. POST / now answers 201 only for a row it actually created. "Created" over an existing binding is the operator's only per-request signal that they landed on somebody else's row. FINDING 2 (MEDIUM) — `Number(null)` is 0, and 0 is a valid write budget. The `=== undefined` guard let JSON `null` through, so a client sending `null` to mean "use the default" got a key that authenticates, resolves its binding, and is throttled to nothing on every write while the UI shows write tools listed. `[]`, `false`, `""` coerced identically; `true` became 1. Both optional fields are now type-checked and a bad value is a 400. The same silence on `enabled` (present-but-non-boolean was dropped, which under the old default meant "activate") is rejected too. FINDING 3 (LOW) — `String(err)` put pg table, column, constraint and sometimes connection detail into 500 bodies that land in browser devtools and UI logs. Logged server-side, generic on the wire. --- .../src/mcp/publicMcpKeyBindingsAdmin.ts | 74 +++- .../src/routes/publicMcpBindingsRouter.ts | 117 ++++-- .../publicMcpKeyBindingsAdmin.test.ts | 386 +++++++++++++++++- web-ui/app/_lib/agentBuilder.ts | 15 + web-ui/app/admin/mcp/page.tsx | 65 ++- web-ui/messages/de.json | 7 +- web-ui/messages/en.json | 7 +- 7 files changed, 620 insertions(+), 51 deletions(-) diff --git a/middleware/src/mcp/publicMcpKeyBindingsAdmin.ts b/middleware/src/mcp/publicMcpKeyBindingsAdmin.ts index 61429b49..9f748a17 100644 --- a/middleware/src/mcp/publicMcpKeyBindingsAdmin.ts +++ b/middleware/src/mcp/publicMcpKeyBindingsAdmin.ts @@ -66,16 +66,40 @@ export interface PublicMcpKeyBindingAdminRow { readonly updatedAt: string; } -/** What an operator submits. Optional fields take the migration's defaults. */ +/** + * What an operator submits. Optional fields take the migration's defaults. + * + * `enabled` is the exception, and the asymmetry is the whole point: OMITTING it + * means "do not touch the parked/active state", not "activate". A binding is the + * entire authorization model for an internet-facing endpoint, and revoke is the + * incident response — a field the operator never mentioned must not be able to + * undo it. On a NEW row there is no prior state to preserve, so it starts + * `true`; on an existing row the stored value survives. Re-arming a revoked key + * therefore requires an explicit `enabled: true` (or the `/restore` route). + */ export interface PublicMcpKeyBindingInput { readonly keyId: string; readonly agentId: string; readonly readTools: readonly string[]; readonly writeTools: readonly string[]; readonly writeRateLimitPerMinute?: number; + /** Absent ⇒ preserve whatever the row says today (new rows start enabled). */ readonly enabled?: boolean; } +/** + * The stored row plus whether this call CREATED it. + * + * The router needs the distinction to answer `201 Created` honestly. Returning + * "Created" for a write that overwrote an existing binding is not merely a + * cosmetic lie: it is the operator's only per-request signal that they landed on + * a row somebody else had already configured — or parked. + */ +export interface PublicMcpKeyBindingUpsertResult { + readonly binding: PublicMcpKeyBindingAdminRow; + readonly created: boolean; +} + export interface BindingValidationFailure { readonly code: string; readonly message: string; @@ -94,8 +118,9 @@ export interface PublicMcpKeyBindingAdminStore { /** Every row, parked ones included. An operator reviewing what a key may do * needs to see the disabled rows; the endpoint never does. */ list(): Promise; - /** Creates or replaces the row for `input.keyId`. */ - upsert(input: PublicMcpKeyBindingInput): Promise; + /** Creates or replaces the row for `input.keyId`. An absent `input.enabled` + * PRESERVES the stored flag rather than defaulting it — see the input type. */ + upsert(input: PublicMcpKeyBindingInput): Promise; /** Parks (or un-parks) a binding without losing what it was configured to * grant. `undefined` when there is no such row. */ setEnabled(keyId: string, enabled: boolean): Promise; @@ -119,6 +144,13 @@ export interface PublicMcpKeyBindingAdminStore { * for: a parked row is a row the reader denies BY DESIGN, so running the check * with the operator's `false` would reject every attempt to save a parked * binding. + * + * It is also carried through UNTOUCHED — absent stays absent. The previous + * `input.enabled ?? true` here is what made revoke undoable: it turned "the + * submission said nothing about enabled" into "the submission asked for + * enabled", and the upsert then wrote that manufactured `true` over a row an + * operator had deliberately parked. Only the store knows the current state, so + * only the store may decide what "unspecified" resolves to. */ export function validateBindingInput(input: PublicMcpKeyBindingInput): BindingValidationResult { const writeRateLimitPerMinute = @@ -166,7 +198,7 @@ export function validateBindingInput(input: PublicMcpKeyBindingInput): BindingVa readTools: normalized.readTools, writeTools: normalized.writeTools, writeRateLimitPerMinute: normalized.writeRateLimitPerMinute, - enabled: input.enabled ?? true, + ...(input.enabled === undefined ? {} : { enabled: input.enabled }), }, }; } @@ -238,28 +270,45 @@ export function createPublicMcpKeyBindingAdminStore(pool: Pool): PublicMcpKeyBin }, async upsert(input) { + // `enabled` binds NULL when the operator did not mention it, and the two + // branches then resolve that NULL differently — `true` on insert (a new + // binding has no prior state), the CURRENT COLUMN on conflict. + // + // Note it is `public_mcp_key_bindings.enabled` and NOT `EXCLUDED.enabled` + // in the DO UPDATE branch: EXCLUDED holds the row this statement PROPOSED, + // so coalescing against it would resolve back to the insert's `true` and + // re-arm the very binding this is meant to leave parked. + // + // `(created_at = updated_at)` is the created/updated discriminator. Both + // columns resolve to `now()` — the transaction timestamp — on the insert + // branch, while the conflict branch moves only `updated_at` and leaves + // `created_at` at an earlier transaction's clock. That uses documented + // `now()` semantics rather than the usual `xmax = 0` idiom, which reads a + // storage-layer detail that also moves when an unrelated transaction holds + // a row lock. const { rows } = await pool.query( `INSERT INTO public_mcp_key_bindings (key_id, agent_id, read_tools, write_tools, write_rate_limit_per_minute, enabled, updated_at) - VALUES ($1, $2, $3, $4, $5, $6, now()) + VALUES ($1, $2, $3, $4, $5, COALESCE($6::boolean, true), now()) ON CONFLICT (key_id) DO UPDATE SET agent_id = EXCLUDED.agent_id, read_tools = EXCLUDED.read_tools, write_tools = EXCLUDED.write_tools, write_rate_limit_per_minute = EXCLUDED.write_rate_limit_per_minute, - enabled = EXCLUDED.enabled, + enabled = COALESCE($6::boolean, public_mcp_key_bindings.enabled), updated_at = now() - RETURNING ${SELECT_COLUMNS}`, + RETURNING ${SELECT_COLUMNS}, (created_at = updated_at) AS inserted`, [ input.keyId, input.agentId, input.readTools, input.writeTools, input.writeRateLimitPerMinute ?? DEFAULT_WRITE_RATE_LIMIT_PER_MINUTE, - input.enabled ?? true, + input.enabled ?? null, ], ); - return toAdminRow(rows[0] as AdminRowShape); + const raw = rows[0] as AdminRowShape & { inserted?: unknown }; + return { binding: toAdminRow(raw), created: raw.inserted === true }; }, async setEnabled(keyId, enabled) { @@ -310,12 +359,15 @@ export function createInMemoryPublicMcpKeyBindingAdminStore( writeTools: [...input.writeTools], writeRateLimitPerMinute: input.writeRateLimitPerMinute ?? DEFAULT_WRITE_RATE_LIMIT_PER_MINUTE, - enabled: input.enabled ?? true, + // Mirrors the SQL's `COALESCE($6, public_mcp_key_bindings.enabled)`: an + // unspecified flag preserves the stored state, and only a row that does + // not exist yet falls through to `true`. + enabled: input.enabled ?? existing?.enabled ?? true, createdAt: existing?.createdAt ?? stamp, updatedAt: stamp, }; byKey.set(row.keyId, row); - return row; + return { binding: row, created: existing === undefined }; }, async setEnabled(keyId, enabled) { const existing = byKey.get(keyId); diff --git a/middleware/src/routes/publicMcpBindingsRouter.ts b/middleware/src/routes/publicMcpBindingsRouter.ts index cfecc8f1..dd6831f9 100644 --- a/middleware/src/routes/publicMcpBindingsRouter.ts +++ b/middleware/src/routes/publicMcpBindingsRouter.ts @@ -86,6 +86,20 @@ export function createPublicMcpBindingsRouter( ); }); + /** + * Logs the real error and answers with a fixed string. + * + * The operator gate runs first, so nothing here reaches an anonymous caller — + * but pg errors name tables, columns and constraints, sometimes carry the + * connection host, and land verbatim in browser devtools and whatever ships + * the UI's logs. None of that helps the operator and all of it helps whoever + * reads those logs next. + */ + function fail(res: Response, code: string, err: unknown): void { + console.error('[public-mcp-bindings]', code, err); + res.status(500).json({ code, message: 'the request could not be completed' }); + } + function storeOr503(res: Response): PublicMcpKeyBindingAdminStore | undefined { const store = options.getStore(); if (!store) { @@ -105,7 +119,7 @@ export function createPublicMcpBindingsRouter( try { res.json({ bindings: await store.list() }); } catch (err) { - res.status(500).json({ code: 'public_mcp_bindings.list_failed', message: String(err) }); + fail(res, 'public_mcp_bindings.list_failed', err); } }); @@ -115,6 +129,39 @@ export function createPublicMcpBindingsRouter( if (!store) return; const body = (req.body ?? {}) as Record; + + // TYPE-CHECK, NEVER COERCE, on both optional fields. + // + // The previous `body[x] === undefined ? {} : Number(body[x])` guard let JSON + // `null` through — `null` is not `undefined` — and `Number(null)` is `0`, + // which is a VALID write budget. A client sending `null` to mean "use the + // default" got a key that authenticates, resolves its binding, and is + // throttled to nothing on every write while the UI shows write tools listed. + // `[]`, `false` and `""` coerce to `0` identically; `true` coerces to `1`. + // Both fields decide what an internet-facing key may do, so a value we + // cannot read at face value is a 400, not a guess. + const rawRate = body['writeRateLimitPerMinute']; + if (rawRate !== undefined && typeof rawRate !== 'number') { + res.status(400).json({ + error: 'invalid_request', + code: 'write_rate_limit_invalid_type', + message: 'writeRateLimitPerMinute must be a number, or omitted to take the default', + }); + return; + } + // Same class of silence on the other side: a present-but-non-boolean + // `enabled` used to be dropped on the floor, and under the old + // `?? true` default "dropped" meant "activate". + const rawEnabled = body['enabled']; + if (rawEnabled !== undefined && typeof rawEnabled !== 'boolean') { + res.status(400).json({ + error: 'invalid_request', + code: 'enabled_invalid_type', + message: 'enabled must be a boolean, or omitted to leave the current state untouched', + }); + return; + } + const input: PublicMcpKeyBindingInput = { keyId: typeof body['keyId'] === 'string' ? body['keyId'].trim() : '', agentId: typeof body['agentId'] === 'string' ? body['agentId'].trim() : '', @@ -122,10 +169,10 @@ export function createPublicMcpBindingsRouter( writeTools: Array.isArray(body['writeTools']) ? (body['writeTools'] as readonly string[]) : [], - ...(body['writeRateLimitPerMinute'] === undefined - ? {} - : { writeRateLimitPerMinute: Number(body['writeRateLimitPerMinute']) }), - ...(typeof body['enabled'] === 'boolean' ? { enabled: body['enabled'] } : {}), + ...(rawRate === undefined ? {} : { writeRateLimitPerMinute: rawRate }), + // Absent stays absent all the way to the store — that is what keeps a + // revoked binding revoked across a save that never mentions it. + ...(rawEnabled === undefined ? {} : { enabled: rawEnabled }), }; // The reader's own rules decide. See `validateBindingInput`. @@ -136,40 +183,54 @@ export function createPublicMcpBindingsRouter( } try { - res.status(201).json({ binding: await store.upsert(validated.value) }); + const { binding, created } = await store.upsert(validated.value); + // 201 only for a row that did not exist. "Created" over an existing + // binding is the operator's only per-request hint that they landed on + // somebody else's row — spending it on every save makes it worthless. + res.status(created ? 201 : 200).json({ binding }); } catch (err) { - res.status(500).json({ code: 'public_mcp_bindings.upsert_failed', message: String(err) }); + fail(res, 'public_mcp_bindings.upsert_failed', err); } }); - // ── Revoke (park, never delete) ───────────────────────────────────────── + // ── Revoke / restore (park and un-park, never delete) ─────────────────── // A revoked binding keeps its configured tool lists so an operator can see // what the integration USED to reach, and can restore it without // reconstructing the allowlist from memory. `DELETE` exists on the store for // completeness but is deliberately not exposed here: the destructive path // wants a deliberate decision, and parking already stops every call. - router.post('/:keyId/revoke', async (req: Request, res: Response) => { - const store = storeOr503(res); - if (!store) return; - - const rawKeyId = req.params['keyId']; - const keyId = Array.isArray(rawKeyId) ? rawKeyId[0] : rawKeyId; - if (!keyId) { - res.status(400).json({ error: 'invalid_request', message: 'missing key id' }); - return; - } - - try { - const binding = await store.setEnabled(keyId, false); - if (!binding) { - res.status(404).json({ error: 'not_found', keyId }); + // + // RESTORE IS ITS OWN ROUTE rather than a side effect of saving. Since an + // upsert now preserves `enabled`, re-arming a key had to become something an + // operator does ON PURPOSE — and a dedicated route makes that intent legible + // in an access log, where `POST /:keyId` would not be. + function setEnabledRoute(enabled: boolean, code: string) { + return async (req: Request, res: Response): Promise => { + const store = storeOr503(res); + if (!store) return; + + const rawKeyId = req.params['keyId']; + const keyId = Array.isArray(rawKeyId) ? rawKeyId[0] : rawKeyId; + if (!keyId) { + res.status(400).json({ error: 'invalid_request', message: 'missing key id' }); return; } - res.json({ binding }); - } catch (err) { - res.status(500).json({ code: 'public_mcp_bindings.revoke_failed', message: String(err) }); - } - }); + + try { + const binding = await store.setEnabled(keyId, enabled); + if (!binding) { + res.status(404).json({ error: 'not_found', keyId }); + return; + } + res.json({ binding }); + } catch (err) { + fail(res, code, err); + } + }; + } + + router.post('/:keyId/revoke', setEnabledRoute(false, 'public_mcp_bindings.revoke_failed')); + router.post('/:keyId/restore', setEnabledRoute(true, 'public_mcp_bindings.restore_failed')); return router; } diff --git a/middleware/test/publicMcp/publicMcpKeyBindingsAdmin.test.ts b/middleware/test/publicMcp/publicMcpKeyBindingsAdmin.test.ts index ab2ee406..9a0ab0d0 100644 --- a/middleware/test/publicMcp/publicMcpKeyBindingsAdmin.test.ts +++ b/middleware/test/publicMcp/publicMcpKeyBindingsAdmin.test.ts @@ -91,6 +91,16 @@ async function withRouter( } } +/** POSTs a binding body verbatim — `unknown` on purpose, because several tests + * submit values the TypeScript input type forbids and the wire allows. */ +async function postBinding(baseUrl: string, body: unknown): Promise { + return fetch(baseUrl, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify(body), + }); +} + // ── The gate ──────────────────────────────────────────────────────────────── describe('publicMcpBindingsRouter — fails closed without operatorAuth', () => { @@ -384,6 +394,33 @@ describe('validateBindingInput — the admin path cannot drift from the reader', assert.equal(validated.ok && validated.value.enabled, false); }); + /** + * The FINDING-1 root cause, at the unit that caused it. `enabled: input.enabled + * ?? true` turned "the operator said nothing about enabled" into "the operator + * asked for enabled", and the upsert then wrote that `true` over a parked row. + * An omitted field must stay omitted all the way to the store, which is the + * only place that knows what the row currently says. + */ + it('leaves enabled UNSET when omitted — it must never become an implicit true', () => { + const validated = validateBindingInput(VALID_INPUT); + assert.equal(validated.ok, true); + assert.equal( + validated.ok && validated.value.enabled, + undefined, + 'omitting enabled must not be silently upgraded to enabled:true', + ); + assert.equal( + validated.ok && 'enabled' in validated.value, + false, + 'the key itself must be absent, so `?? existing` downstream can see the difference', + ); + }); + + it('carries an explicit enabled:true through — re-arming is allowed when asked for', () => { + const validated = validateBindingInput({ ...VALID_INPUT, enabled: true }); + assert.equal(validated.ok && validated.value.enabled, true); + }); + it('defaults the write rate limit to the migration default when omitted', () => { const validated = validateBindingInput({ keyId: 'k', @@ -395,6 +432,227 @@ describe('validateBindingInput — the admin path cannot drift from the reader', }); }); +// ── FINDING 1: revoke must not be undone by a save that never mentions it ──── + +describe('a revoked binding survives an upsert that omits `enabled`', () => { + /** + * The composition nobody tested. Two green tests already proved that revoke + * parks rather than deletes, and that the writer defaults sensibly. Neither + * asked the only question that matters operationally: after an incident + * revoke, does the NEXT save — from a stale tab, a second operator, a config + * replay, or the admin UI's own form, which does not round-trip `enabled` — + * silently hand the key its whole allowlist back? + */ + it('store: revoke, then re-save the same binding — still parked', async () => { + const store = createInMemoryPublicMcpKeyBindingAdminStore(); + await store.upsert(VALID_INPUT); + await store.setEnabled('key-1', false); + + const resaved = await store.upsert({ ...VALID_INPUT, readTools: ['query_crm', 'read_notes'] }); + + assert.equal(resaved.binding.enabled, false, 'an omitted `enabled` must not re-arm the key'); + assert.deepEqual(resaved.binding.readTools, ['query_crm', 'read_notes'], 'the edit still lands'); + }); + + it('store: an explicit enabled:true is the only way back — and it works', async () => { + const store = createInMemoryPublicMcpKeyBindingAdminStore(); + await store.upsert(VALID_INPUT); + await store.setEnabled('key-1', false); + + const restored = await store.upsert({ ...VALID_INPUT, enabled: true }); + assert.equal(restored.binding.enabled, true); + }); + + it('store: a fresh row still defaults to enabled — preservation is not "always parked"', async () => { + const store = createInMemoryPublicMcpKeyBindingAdminStore(); + const created = await store.upsert(VALID_INPUT); + assert.equal(created.binding.enabled, true); + assert.equal(created.created, true); + }); + + it('HTTP: POST / over a revoked binding leaves it revoked, and answers 200 not 201', async () => { + const store = createInMemoryPublicMcpKeyBindingAdminStore(); + await withRouter({ store, operatorAuth: alwaysValidOperatorAuth() }, async (url) => { + const created = await postBinding(url, VALID_INPUT); + assert.equal(created.status, 201, 'a genuinely new binding is Created'); + + assert.equal((await fetch(`${url}/key-1/revoke`, { method: 'POST' })).status, 200); + + const resaved = await postBinding(url, { ...VALID_INPUT, writeTools: ['create_lead'] }); + assert.equal( + resaved.status, + 200, + '201 Created over an existing row is the operator’s only hint they overwrote one', + ); + const { binding } = (await resaved.json()) as { binding: { enabled: boolean } }; + assert.equal(binding.enabled, false, 'the parked row must still be parked'); + + const listed = (await (await fetch(url)).json()) as { + bindings: { keyId: string; enabled: boolean }[]; + }; + assert.equal(listed.bindings.find((b) => b.keyId === 'key-1')?.enabled, false); + }); + }); + + it('HTTP: an operator who SAYS enabled:true gets the key back', async () => { + const store = createInMemoryPublicMcpKeyBindingAdminStore(); + await withRouter({ store, operatorAuth: alwaysValidOperatorAuth() }, async (url) => { + await postBinding(url, VALID_INPUT); + await fetch(`${url}/key-1/revoke`, { method: 'POST' }); + + const res = await postBinding(url, { ...VALID_INPUT, enabled: true }); + assert.equal(res.status, 200); + assert.equal(((await res.json()) as { binding: { enabled: boolean } }).binding.enabled, true); + }); + }); + + /** Un-parking through its own route, so the UI has an affordance that does not + * depend on re-submitting the whole binding. */ + it('HTTP: POST /:keyId/restore un-parks, and 404s on an unknown key', async () => { + const store = createInMemoryPublicMcpKeyBindingAdminStore(); + await withRouter({ store, operatorAuth: alwaysValidOperatorAuth() }, async (url) => { + await postBinding(url, VALID_INPUT); + await fetch(`${url}/key-1/revoke`, { method: 'POST' }); + + const res = await fetch(`${url}/key-1/restore`, { method: 'POST' }); + assert.equal(res.status, 200); + assert.equal(((await res.json()) as { binding: { enabled: boolean } }).binding.enabled, true); + + assert.equal((await fetch(`${url}/nope/restore`, { method: 'POST' })).status, 404); + }); + }); + + it('HTTP: restore is gated exactly like every other route', async () => { + await withRouter( + { + store: createInMemoryPublicMcpKeyBindingAdminStore(), + operatorAuth: neverValidOperatorAuth(), + }, + async (url) => { + assert.equal((await fetch(`${url}/key-1/restore`, { method: 'POST' })).status, 401); + }, + ); + }); +}); + +// ── FINDING 2: JSON null must not coerce to a zero write budget ────────────── + +describe('writeRateLimitPerMinute is type-checked, never coerced', () => { + /** + * `Number(null)` is `0`, and `0` is a perfectly valid write budget — so a + * client sending `"writeRateLimitPerMinute": null` to mean "use the default" + * was stored as a key that authenticates, resolves its binding, and is + * throttled to nothing on every write, while the UI shows write tools listed. + * `[]`, `false` and `""` coerce to `0` the same way; `true` coerces to `1`. + */ + for (const [label, value] of [ + ['null', null], + ['an empty array', []], + ['false', false], + ['an empty string', ''], + ['true', true], + ['a numeric string', '5'], + ['an object', { valueOf: 3 }], + ] as const) { + it(`rejects ${label} with 400 rather than coercing it to a budget`, async () => { + const store = createInMemoryPublicMcpKeyBindingAdminStore(); + await withRouter({ store, operatorAuth: alwaysValidOperatorAuth() }, async (url) => { + const res = await postBinding(url, { ...VALID_INPUT, writeRateLimitPerMinute: value }); + assert.equal(res.status, 400, `${label} must not be coerced`); + assert.equal( + ((await res.json()) as { code: string }).code, + 'write_rate_limit_invalid_type', + ); + assert.deepEqual(await store.list(), [], 'nothing may be written for a rejected body'); + }); + }); + } + + it('an omitted rate limit still takes the migration default', async () => { + const store = createInMemoryPublicMcpKeyBindingAdminStore(); + await withRouter({ store, operatorAuth: alwaysValidOperatorAuth() }, async (url) => { + const res = await postBinding(url, { + keyId: 'key-1', + agentId: 'sales', + readTools: ['query_crm'], + writeTools: [], + }); + assert.equal(res.status, 201); + const { binding } = (await res.json()) as { binding: { writeRateLimitPerMinute: number } }; + assert.equal(binding.writeRateLimitPerMinute, 5); + }); + }); + + it('an explicit 0 is still honoured — a deliberate zero budget is legitimate', async () => { + const store = createInMemoryPublicMcpKeyBindingAdminStore(); + await withRouter({ store, operatorAuth: alwaysValidOperatorAuth() }, async (url) => { + const res = await postBinding(url, { ...VALID_INPUT, writeRateLimitPerMinute: 0 }); + assert.equal(res.status, 201); + const { binding } = (await res.json()) as { binding: { writeRateLimitPerMinute: number } }; + assert.equal(binding.writeRateLimitPerMinute, 0); + }); + }); + + /** Same class of bug on the other security-relevant field: a present-but-wrong + * `enabled` used to be silently DROPPED, which under the old default meant + * "enable it". Silence is not an option for either field. */ + it('rejects a non-boolean `enabled` rather than silently ignoring it', async () => { + const store = createInMemoryPublicMcpKeyBindingAdminStore(); + await withRouter({ store, operatorAuth: alwaysValidOperatorAuth() }, async (url) => { + for (const bad of ['false', 0, null, []] as const) { + const res = await postBinding(url, { ...VALID_INPUT, enabled: bad }); + assert.equal(res.status, 400, `enabled: ${JSON.stringify(bad)} must be refused`); + assert.equal(((await res.json()) as { code: string }).code, 'enabled_invalid_type'); + } + assert.deepEqual(await store.list(), []); + }); + }); +}); + +// ── FINDING 3: driver text must not reach the operator's browser ───────────── + +describe('a store failure returns a generic 500, not the driver message', () => { + function explodingStore(): PublicMcpKeyBindingAdminStore { + const boom = (): never => { + throw new Error( + 'relation "public_mcp_key_bindings" does not exist at 10.0.0.7:5432 (constraint pk_key_id)', + ); + }; + return { + list: boom, + upsert: boom, + setEnabled: boom, + remove: boom, + } as unknown as PublicMcpKeyBindingAdminStore; + } + + for (const [label, call] of [ + ['GET /', (url: string): Promise => fetch(url)], + ['POST /', (url: string): Promise => postBinding(url, VALID_INPUT)], + [ + 'POST /:keyId/revoke', + (url: string): Promise => fetch(`${url}/key-1/revoke`, { method: 'POST' }), + ], + ] as const) { + it(`${label} leaks neither table, column, constraint nor host`, async () => { + await withRouter( + { store: explodingStore(), operatorAuth: alwaysValidOperatorAuth() }, + async (url) => { + const res = await call(url); + assert.equal(res.status, 500); + const body = await res.text(); + for (const secret of ['public_mcp_key_bindings', '10.0.0.7', 'pk_key_id', 'relation']) { + assert.ok( + !body.includes(secret), + `the 500 body must not carry ${secret} — it lands in devtools and UI logs: ${body}`, + ); + } + }, + ); + }); + } +}); + // ── updated_at: no trigger exists, so the writer must set it ──────────────── describe('the writer sets updated_at explicitly (migration 0033 has no trigger)', () => { @@ -406,11 +664,26 @@ describe('the writer sets updated_at explicitly (migration 0033 has no trigger)' * integration's reach last changed?" wrong forever. A fake pool is the only * way to see the statement without a live Postgres. */ - function recordingPool(): { pool: Pool; statements: string[] } { - const statements: string[] = []; + interface RecordedStatement { + readonly text: string; + readonly params: readonly unknown[]; + } + + /** + * Records the SQL **and its bound parameters**. + * + * Recording only the text is how this harness was fake-shaped: swapping + * `setEnabled`'s placeholders to `SET enabled = $1 WHERE key_id = $2` while + * still passing `[keyId, enabled]` left every text-only assertion green and + * shipped a revoke that writes the key id into a boolean column and matches + * rows on `false`. The statement and the array are only meaningful together, + * so both are captured and `boundTo` below resolves one against the other. + */ + function recordingPool(): { pool: Pool; statements: RecordedStatement[] } { + const statements: RecordedStatement[] = []; const pool = { - query(text: string) { - statements.push(text); + query(text: string, params?: readonly unknown[]) { + statements.push({ text, params: params ?? [] }); return Promise.resolve({ rows: [ { @@ -422,6 +695,7 @@ describe('the writer sets updated_at explicitly (migration 0033 has no trigger)' enabled: true, created_at: new Date('2026-01-01T00:00:00.000Z'), updated_at: new Date('2026-02-02T00:00:00.000Z'), + inserted: false, }, ], rowCount: 1, @@ -436,11 +710,30 @@ describe('the writer sets updated_at explicitly (migration 0033 has no trigger)' return sql.replace(/\s+/g, ' '); } + /** + * Resolves what a statement ACTUALLY binds to a named position. + * + * `pattern` must capture a `$n` placeholder number; the value returned is the + * argument the driver would substitute there. This is the assertion the + * text-only harness could not make: it follows the placeholder the SQL names + * into the parameter array, so a swap of either side is caught by the other. + */ + function boundTo(stmt: RecordedStatement, pattern: RegExp): unknown { + const match = pattern.exec(flat(stmt.text)); + assert.ok(match, `no match for ${String(pattern)} in: ${flat(stmt.text)}`); + const position = Number(match[1]); + assert.ok( + position >= 1 && position <= stmt.params.length, + `$${String(position)} is out of range for ${String(stmt.params.length)} bound params`, + ); + return stmt.params[position - 1]; + } + it('upsert sets updated_at on the CONFLICT branch, not only via the INSERT default', async () => { const { pool, statements } = recordingPool(); await createPublicMcpKeyBindingAdminStore(pool).upsert(VALID_INPUT); assert.equal(statements.length, 1); - const sql = flat(statements[0] ?? ''); + const sql = flat(statements[0]?.text ?? ''); assert.match(sql, /ON CONFLICT \(key_id\) DO UPDATE SET/); assert.match( sql.slice(sql.indexOf('DO UPDATE SET')), @@ -452,7 +745,81 @@ describe('the writer sets updated_at explicitly (migration 0033 has no trigger)' it('setEnabled sets updated_at too — revoking is a change worth timestamping', async () => { const { pool, statements } = recordingPool(); await createPublicMcpKeyBindingAdminStore(pool).setEnabled('key-1', false); - assert.match(flat(statements[0] ?? ''), /SET enabled = \$2, updated_at = now\(\)/); + assert.match(flat(statements[0]?.text ?? ''), /SET enabled = \$\d+, updated_at = now\(\)/); + }); + + /** + * The mutation that proved the old harness fake: `SET enabled = $1 WHERE + * key_id = $2` with the parameter array untouched. Text-only assertions stay + * green; production revoke breaks. Following each placeholder into the array + * is what makes the swap visible. + */ + it('setEnabled binds the key id and the flag to the placeholders its SQL names', async () => { + const { pool, statements } = recordingPool(); + await createPublicMcpKeyBindingAdminStore(pool).setEnabled('key-1', false); + const stmt = statements[0]; + assert.ok(stmt); + assert.equal( + boundTo(stmt, /SET enabled = \$(\d+)/), + false, + 'the placeholder the SET clause names must carry the enabled flag', + ); + assert.equal( + boundTo(stmt, /WHERE key_id = \$(\d+)/), + 'key-1', + 'the placeholder the WHERE clause names must carry the key id', + ); + }); + + it('upsert binds every column to the placeholder its VALUES list names', async () => { + const { pool, statements } = recordingPool(); + await createPublicMcpKeyBindingAdminStore(pool).upsert({ ...VALID_INPUT, enabled: false }); + const stmt = statements[0]; + assert.ok(stmt); + // The VALUES list is positional, so resolve it once and check the row it + // would actually write rather than trusting the argument order. + const values = /VALUES \(([^)]*)\)/.exec(flat(stmt.text))?.[1] ?? ''; + const positions = [...values.matchAll(/\$(\d+)/g)].map((m) => Number(m[1])); + const bound = positions.map((p) => stmt.params[p - 1]); + assert.deepEqual(bound.slice(0, 5), ['key-1', 'sales', ['query_crm'], ['create_lead'], 5]); + assert.equal(bound[5], false, 'the enabled column must bind the operator flag'); + }); + + /** + * FINDING 1 in SQL. The conflict branch must resolve an unspecified `enabled` + * against the STORED column. Two ways to get this wrong and both are checked: + * binding `true` instead of NULL (nothing left to preserve), and coalescing + * against `EXCLUDED.enabled` — which holds the row this very statement + * proposed, so it resolves straight back to the insert branch's `true` and + * re-arms the binding it was supposed to leave parked. + */ + it('upsert binds NULL for an omitted enabled and coalesces it against the STORED column', async () => { + const { pool, statements } = recordingPool(); + await createPublicMcpKeyBindingAdminStore(pool).upsert(VALID_INPUT); + const stmt = statements[0]; + assert.ok(stmt); + + const doUpdate = flat(stmt.text).slice(flat(stmt.text).indexOf('DO UPDATE SET')); + // Everything the conflict branch assigns to `enabled`, up to the next + // ` = ` assignment. A plain `[^,]+` would stop at the comma INSIDE + // `COALESCE(a, b)` and read the preservation as if it were absent. + const conflictEnabled = /\benabled = (.+?)(?=, [a-z_]+ = |RETURNING)/.exec(doUpdate)?.[1] ?? ''; + assert.match( + conflictEnabled, + /public_mcp_key_bindings\.enabled/, + 'the conflict branch must fall back to the stored column, not to EXCLUDED', + ); + assert.ok( + !/EXCLUDED\.enabled/.test(conflictEnabled), + 'EXCLUDED.enabled is the proposed row — coalescing against it preserves nothing', + ); + + const position = Number(/\$(\d+)/.exec(conflictEnabled)?.[1]); + assert.equal( + stmt.params[position - 1], + null, + 'an omitted enabled must bind NULL, so COALESCE has something to fall through', + ); }); it('the in-memory writer moves updatedAt on an upsert and keeps createdAt', async () => { @@ -460,8 +827,9 @@ describe('the writer sets updated_at explicitly (migration 0033 has no trigger)' const clock = (): Date => new Date(1_700_000_000_000 + tick++ * 60_000); const store = createInMemoryPublicMcpKeyBindingAdminStore([], clock); - const first = await store.upsert(VALID_INPUT); - const second = await store.upsert({ ...VALID_INPUT, readTools: ['query_crm', 'read_notes'] }); + const first = (await store.upsert(VALID_INPUT)).binding; + const second = (await store.upsert({ ...VALID_INPUT, readTools: ['query_crm', 'read_notes'] })) + .binding; assert.equal(second.createdAt, first.createdAt, 'createdAt must survive an update'); assert.notEqual(second.updatedAt, first.updatedAt, 'updatedAt must move'); @@ -472,7 +840,7 @@ describe('the writer sets updated_at explicitly (migration 0033 has no trigger)' let tick = 0; const clock = (): Date => new Date(1_700_000_000_000 + tick++ * 60_000); const store = createInMemoryPublicMcpKeyBindingAdminStore([], clock); - const created = await store.upsert(VALID_INPUT); + const created = (await store.upsert(VALID_INPUT)).binding; const parked = await store.setEnabled('key-1', false); assert.ok(parked); assert.ok(Date.parse(parked.updatedAt) > Date.parse(created.updatedAt)); diff --git a/web-ui/app/_lib/agentBuilder.ts b/web-ui/app/_lib/agentBuilder.ts index 407f35c2..d3160d7a 100644 --- a/web-ui/app/_lib/agentBuilder.ts +++ b/web-ui/app/_lib/agentBuilder.ts @@ -1069,6 +1069,9 @@ export interface UpsertPublicMcpKeyBindingInput { readTools: string[]; writeTools: string[]; writeRateLimitPerMinute?: number; + /** OMIT to leave the revoked/active state exactly as stored. Sending `true` + * RE-ARMS a revoked key, so this pane never sends it implicitly — un-parking + * goes through `restorePublicMcpKeyBinding` instead. */ enabled?: boolean; } @@ -1096,6 +1099,18 @@ export async function revokePublicMcpKeyBinding( ); } +/** Un-parks a revoked binding, restoring the reach it already had on the row. + * Its own call, not a side effect of saving: a save that never mentions + * `enabled` deliberately CANNOT re-arm a key an operator revoked. */ +export async function restorePublicMcpKeyBinding( + keyId: string, +): Promise<{ binding: PublicMcpKeyBinding }> { + return callJson<{ binding: PublicMcpKeyBinding }>( + `/v1/operator/public-mcp-bindings/${encodeURIComponent(keyId)}/restore`, + { method: 'POST' }, + ); +} + // ----------------------------------------------------------------------------- // MCP servers // ----------------------------------------------------------------------------- diff --git a/web-ui/app/admin/mcp/page.tsx b/web-ui/app/admin/mcp/page.tsx index 0aa6df05..432e5719 100644 --- a/web-ui/app/admin/mcp/page.tsx +++ b/web-ui/app/admin/mcp/page.tsx @@ -32,6 +32,7 @@ import { listMcpServers, listPublicMcpKeyBindings, revokePublicMcpKeyBinding, + restorePublicMcpKeyBinding, upsertPublicMcpKeyBinding, revokeMcpGrant, revokePluginMcpServer, @@ -1812,6 +1813,7 @@ function BindingsPane(): React.ReactElement { const [error, setError] = useState(null); const [busy, setBusy] = useState(null); const [confirmRevoke, setConfirmRevoke] = useState(null); + const [confirmRestore, setConfirmRestore] = useState(null); const [keyId, setKeyId] = useState(''); const [agentId, setAgentId] = useState(''); @@ -1872,6 +1874,28 @@ function BindingsPane(): React.ReactElement { } } + /** + * The only way back from a revoke. + * + * `save()` above never sends `enabled`, and the server preserves the stored + * flag when it is absent — so re-submitting the form over a revoked binding + * edits its tool lists and leaves it parked. That is the point: un-parking a + * key after an incident must be something an operator asks for, not something + * that falls out of pressing Save on a stale tab. + */ + async function restore(binding: PublicMcpKeyBinding): Promise { + setBusy(`restore:${binding.keyId}`); + setError(null); + try { + await restorePublicMcpKeyBinding(binding.keyId); + await refresh(); + } catch (err) { + setError(errText(err)); + } finally { + setBusy(null); + } + } + const inputCls = 'rounded-md border border-[color:var(--border)] bg-transparent px-3 py-2 text-sm outline-none focus:border-[color:var(--accent)]'; const canSave = keyId.trim().length > 0 && agentId.trim().length > 0; @@ -2012,7 +2036,23 @@ function BindingsPane(): React.ReactElement { {t('bindings.revoke')} - ) : null} + ) : ( +
+ + {t('bindings.restoreHint')} + +
+ +
+
+ )} ))} @@ -2030,6 +2070,29 @@ function BindingsPane(): React.ReactElement { if (target) void revoke(target); }} /> + + {/* Un-parking hands a third-party key its whole allowlist back, so it is + confirmed exactly like the revoke that parked it. */} + setConfirmRestore(null)} + onConfirm={() => { + const target = confirmRestore; + setConfirmRestore(null); + if (target) void restore(target); + }} + /> ); } diff --git a/web-ui/messages/de.json b/web-ui/messages/de.json index d6d3783a..8abcc34d 100644 --- a/web-ui/messages/de.json +++ b/web-ui/messages/de.json @@ -2823,7 +2823,12 @@ "revoke": "Widerrufen", "revokeTitle": "Dieses Binding widerrufen?", "revokeBody": "Der Key {keyId} erreicht ab sofort kein Tool mehr. Das Binding bleibt erhalten und wird nicht gelöscht — du siehst weiterhin, was es erlaubt hat.", - "revokeConfirm": "Widerrufen" + "revokeConfirm": "Widerrufen", + "restore": "Zugriff wiederherstellen", + "restoreHint": "Erneutes Speichern ändert die Tools, hebt den Widerruf aber nicht auf. Nur Wiederherstellen macht den Key wieder nutzbar.", + "restoreTitle": "Dieses Binding wiederherstellen?", + "restoreBody": "Der Key {keyId} erhält sofort alles zurück, was auf dieser Zeile steht: {tools}.", + "restoreConfirm": "Wiederherstellen" }, "plugins": { "intro": "Plugins, die im Manifest MCP-Zugriff deklarieren. Weise jedem gezielt die Server zu, die es erreichen darf — nichts ist implizit. Die Per-Tool-Sicherheit greift weiterhin: ein ungescanntes oder hochriskantes Tool wird zur Aufrufzeit abgelehnt.", diff --git a/web-ui/messages/en.json b/web-ui/messages/en.json index f3c567fe..7c86708b 100644 --- a/web-ui/messages/en.json +++ b/web-ui/messages/en.json @@ -2823,7 +2823,12 @@ "revoke": "Revoke", "revokeTitle": "Revoke this binding?", "revokeBody": "The key {keyId} stops reaching any tool immediately. The binding is kept, not deleted, so you can still see what it granted.", - "revokeConfirm": "Revoke" + "revokeConfirm": "Revoke", + "restore": "Restore access", + "restoreHint": "Saving this binding again edits its tools but leaves it revoked. Restoring is the only way to make the key work again.", + "restoreTitle": "Restore this binding?", + "restoreBody": "The key {keyId} immediately regains everything on this row: {tools}.", + "restoreConfirm": "Restore" }, "plugins": { "intro": "Plugins that declare MCP access in their manifest. Grant each the specific servers it may reach — nothing is ambient. Per-tool safety still applies: an unscanned or high-risk tool is refused at call time.", From e8111983b2a81ebd6dd2542260a6543f3244eb32 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 14:36:39 +0200 Subject: [PATCH 86/90] docs(changelog): record the W5-1 review fixes as operator-visible behaviour Revoke becoming sticky, 201 narrowing to created-only, and the two fields switching from coercion to type-checking all change what an operator and any config-replay client observe. The Added section for this surface said "lists, creates and revokes", which is no longer the whole story. --- docs/CHANGELOG.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index d15aeb18..d140f175 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -83,6 +83,27 @@ entry. See `CONTRIBUTING.md` § Releases & changelog. read-only store: it gains no write path to its own authorization table. The admin path validates through the same `normalizeBindingRow` the enforcement path uses, so the two cannot drift. Revoke parks the row rather than deleting it. +- **Revoke is sticky.** A cross-vendor review found that saving a binding + re-enabled it: an omitted `enabled` was defaulted to `true` and written over the + stored value, so any later save — a stale browser tab, a second operator, a + config replay, or this pane's own form, which does not round-trip the field — + silently handed a revoked key its whole allowlist back. An absent `enabled` now + preserves the stored flag (a genuinely new row still starts enabled), and + un-parking is an explicit act: `POST /:keyId/restore`, or an explicit + `enabled: true` on the upsert. The pane grew a confirmed **Restore access** + button so the stricter server does not strand an operator in psql. +- `POST /` answers **200** for a row it replaced and keeps 201 for one it created + — "Created" is the operator's only per-request signal that they landed on a + binding somebody else had already configured, or parked. +- `writeRateLimitPerMinute` and `enabled` are type-checked rather than coerced. A + JSON `null` reached `Number(null)` → `0`, a valid write budget, so a client + sending `null` to mean "use the default" got an integration that authenticates, + resolves its binding, and is throttled to nothing on every write while the UI + showed write tools listed. `[]`, `false` and `""` coerced identically; `true` + became 1. Bad values are now a 400. +- 500 bodies no longer carry `String(err)`. pg errors name tables, columns and + constraints and sometimes the connection host, and those bodies land in browser + devtools and UI logs; the detail is logged server-side instead. ### Fixed — raw NUL bytes made ripgrep silently truncate eight source files From 04ea85c31ba6e1b7dfad33e2cbb8dd59858a5de2 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 14:39:10 +0200 Subject: [PATCH 87/90] fix(privacy): intern MCP input-replay results before they reach the LLM wire MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Privacy Shield v4's boundary is server <-> LLM provider: `dispatchTool` interns every raw tool result via `internToolResultV4` and hands the model only an identity-free digest, while the real rows stay server-side behind a datasetId. The browser is on the TRUSTED side and legitimately receives real values. The MCP input-replay path (MRTR, #544/W2-1) bypassed that boundary. The replayer registered in `middleware/src/index.ts` calls `mcpManager.callTool` DIRECTLY rather than through `dispatchTool`, so nothing interned its result; `runMcpInputReplay` then interpolated that string verbatim into the note, and `withMcpInputNote` folded the note into the turn's ingested text bound for the model. A user answering an MCP credential card against an HR or accounting server therefore put the returned personnel row on the LLM wire in cleartext — where the same row from the same tool on an ordinary turn would have been a digest. The existing comment above the note proves this was a near-miss: it notes the text "goes on the LLM wire" and then justifies omitting only the user's typed values. The tool result two lines below was overlooked. Fix shape (a): intern inside `runMcpInputReplay`, where the turn's privacy handle is in scope, immediately before the note is built. Shape (b) — route the replay through `dispatchTool` — was investigated and rejected on four counts, recorded in the new method's doc comment: * `dispatchTool` keys on the hydrated, namespaced `mcpNativeToolName(...)`, while the parked record carries the RAW MCP tool name; no reverse mapping exists and one would break the moment a server is renamed between turns. * the replayer must re-resolve the server's LIVE config (endpoint, headers, Vault-resolved env), not a hydration-time closure — the reason it is registered in `index.ts` in the first place. * a replay must still complete a call the previous turn already made, even when the tool is no longer granted or hydrated for the current agent. * it must not re-enter dispatch-only deadline, audit and MRTR park semantics; a replay that re-parks is what MCP_INPUT_MAX_REPLAY_DEPTH exists to prevent. `guardReplayResult` mirrors the ordinary path exactly: no privacy handle means byte-identical legacy behaviour; an operator-flagged privacy-bypass server (`isMcpServerPrivacyBypassed`, still clamped by OMADIA_PRIVACY_FORCE_GUARDED through `resolveEffectivePrivacyMode`) passes raw and records a receipt entry; everything else is interned. Fail-open on a throwing guard is deliberate parity with `dispatchTool`, documented as such. `mcpDomainForServer` is extracted in mcpClient.ts so the bypass receipt entry carries the SAME plugin id an ordinary dispatch would, from one derivation. Regression test drives a real Orchestrator turn against a real MCP server over a real socket and asserts on the text that crossed the provider boundary, never on a call count. Verified by mutation: reverting the note to the raw result turns it red with the personnel row printed verbatim in the failure. --- .../harness-orchestrator/src/mcp/mcpClient.ts | 12 +- .../harness-orchestrator/src/orchestrator.ts | 73 ++- .../mcpInputReplayPrivacy.test.ts | 497 ++++++++++++++++++ 3 files changed, 579 insertions(+), 3 deletions(-) create mode 100644 middleware/test/orchestrator/mcpInputReplayPrivacy.test.ts diff --git a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts index 9a239e37..a3cc7b5e 100644 --- a/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts +++ b/middleware/packages/harness-orchestrator/src/mcp/mcpClient.ts @@ -1089,6 +1089,16 @@ function inputSchemaOrEmpty(tool: McpToolDescriptor): { return { type: 'object', properties: {}, required: [] }; } +/** + * The `domain` an MCP server's tools are attributed to. Single derivation: + * `mcpToolToNativeSpec` stamps it onto every hydrated DomainTool, and the + * orchestrator's MCP input-replay privacy guard reuses it so a replay's + * bypass-receipt entry carries the SAME plugin id an ordinary dispatch would. + */ +export function mcpDomainForServer(serverName: string): string { + return `mcp.${slugifyDomain(serverName)}`; +} + /** Adapt an MCP tool into a top-level orchestrator NativeToolSpec. */ export function mcpToolToNativeSpec( serverName: string, @@ -1099,7 +1109,7 @@ export function mcpToolToNativeSpec( description: tool.description ?? `MCP tool "${tool.name}" from server "${serverName}".`, input_schema: inputSchemaOrEmpty(tool), - domain: `mcp.${slugifyDomain(serverName)}`, + domain: mcpDomainForServer(serverName), }; } diff --git a/middleware/packages/harness-orchestrator/src/orchestrator.ts b/middleware/packages/harness-orchestrator/src/orchestrator.ts index b462d29b..1cadf613 100644 --- a/middleware/packages/harness-orchestrator/src/orchestrator.ts +++ b/middleware/packages/harness-orchestrator/src/orchestrator.ts @@ -1,6 +1,9 @@ import { randomUUID } from 'node:crypto'; import type { Pool } from 'pg'; -import { resolveMcpCallTimeouts } from './mcp/mcpClient.js'; +import { + mcpDomainForServer, + resolveMcpCallTimeouts, +} from './mcp/mcpClient.js'; import { deriveAgentsConsulted, toSemanticAnswer, @@ -2171,16 +2174,82 @@ export class Orchestrator { 'Sag das dem User und ruf kein Tool auf.' ); } + const guardedResult = await this.guardReplayResult(record, result); // The collected VALUES are deliberately absent from this note: they may be // secrets the user typed for the server, and this text goes on the LLM wire // and into the session log. Only the outcome travels. return ( `[MCP-Eingabe] Die Angaben des Users wurden an "${record.serverName}" ` + - `übermittelt und "${record.toolName}" erneut ausgeführt. Ergebnis:\n${result}\n` + + `übermittelt und "${record.toolName}" erneut ausgeführt. Ergebnis:\n${guardedResult}\n` + 'Formuliere daraus die Antwort für den User. Ruf das Tool nicht noch einmal auf.' ); } + /** + * Privacy Shield v4 boundary for MCP input replay: this note crosses only the + * server ↔ LLM-provider seam. The browser stays on the trusted side and is + * unaffected — it may still render the real values server-side. + * + * Shape (b) ("route replay through dispatchTool") was rejected and must stay + * rejected here: the parked record keeps the RAW MCP tool name while + * `dispatchTool` keys on the hydrated native/namespaced one; replay must use + * the server's LIVE config rather than a hydration-time closure; it must stay + * reachable even when the tool is no longer granted/hydrated; and it must not + * re-enter dispatch-only deadline/audit/park semantics. So replay resolves the + * live call where it already does today and applies the SAME privacy boundary + * here, immediately before the note is put on the LLM wire. + * + * Fail-open is deliberate parity with ordinary dispatch: if receipt recording + * or interning throws, we warn and continue with the raw result rather than + * breaking the turn after the user already supplied the requested input. + */ + private async guardReplayResult( + record: PendingMcpInput, + rawResult: string, + ): Promise { + const privacy = turnContext.current()?.privacyHandle; + if (privacy === undefined) return rawResult; + + if (isMcpServerPrivacyBypassed(record.serverId)) { + const effective = resolveEffectivePrivacyMode({ + storedMode: 'bypass', + storedScopes: undefined, + toolName: record.toolName, + env: process.env, + }); + if (effective === 'bypass') { + try { + await privacy.recordBypassedTool({ + toolName: record.toolName, + pluginId: mcpDomainForServer(record.serverName), + reason: 'operator_setting', + bytes: Buffer.byteLength(rawResult, 'utf8'), + }); + } catch (err) { + console.warn( + `[orchestrator.mcpInputReplay:${record.serverId}:${record.toolName}] privacy.recordBypassedTool threw — bypass still applied:`, + err, + ); + } + return rawResult; + } + } + + try { + const v4 = await privacy.internToolResultV4({ + toolName: record.toolName, + rawResult, + }); + return v4.digestText; + } catch (err) { + console.warn( + `[orchestrator.mcpInputReplay:${record.serverId}:${record.toolName}] privacy.internToolResultV4 threw — sending raw replay result:`, + err, + ); + return rawResult; + } + } + /** * OB-29-4 — scan plugin-tool result strings for an in-band * `_pendingUserChoice` payload. Plugins (which have no kernel-internal diff --git a/middleware/test/orchestrator/mcpInputReplayPrivacy.test.ts b/middleware/test/orchestrator/mcpInputReplayPrivacy.test.ts new file mode 100644 index 00000000..be482933 --- /dev/null +++ b/middleware/test/orchestrator/mcpInputReplayPrivacy.test.ts @@ -0,0 +1,497 @@ +/** + * Issue #544 / W2-1 — Privacy Shield v4 on the MCP input-replay note. + * + * The MCP input-replay path re-calls a parked tool in a LATER turn and folds + * the result into the note that `withMcpInputNote` puts on the model's wire. + * That result was NOT interned: the replayer calls `McpManager.callTool` + * directly rather than through `dispatchTool`, so a personnel row coming back + * from an HR/accounting MCP server reached the LLM provider in cleartext. + * + * Coverage: + * 1. the replayed MCP result is interned before the note crosses the + * server ↔ LLM-provider boundary; + * 2. an operator-flagged MCP privacy bypass still passes the replay result + * through raw on that boundary (exempt stays exempt, still functional); + * 3. with no privacy handle the note keeps byte-identical legacy behaviour. + * + * WHY THE CARD IS PRE-SEEDED INTO THE STORE rather than parked by a real first + * turn: with a privacy handle installed, `dispatchTool` interns EVERY + * non-allowlisted tool result — including the `[mcp_input_required:]` + * sentinel — and `parseMcpInputSentinel` is deliberately anchored at the start + * of the string, so the card never materialises at all. That is a SEPARATE, + * pre-existing defect (Privacy Shield v4 vs. MRTR cards; `privacyInternPolicy.ts` + * exempts by tool NAME only and no MCP tool is ever on that list), reported + * alongside this fix and deliberately NOT papered over here. Pre-seeding the + * store isolates the replay half — the code this file exists to pin — from it. + * The full two-turn parking flow is covered by `mcpInputRequired.test.ts`. + * + * Imported from SOURCE, not from the `@omadia/orchestrator` barrel. The barrel + * resolves to `dist/`, so a mutation in `src/` would otherwise be invisible + * without a rebuild and a mutation check could report GREEN over stale code. + */ + +import { after, describe, it } from 'node:test'; +import { strict as assert } from 'node:assert'; +import { + createServer, + type IncomingMessage, + type Server as HttpServer, + type ServerResponse, +} from 'node:http'; +import type { AddressInfo } from 'node:net'; + +import { Server as McpSdkServer } from '@modelcontextprotocol/sdk/server/index.js'; +import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js'; +import { + CallToolRequestSchema, + ListToolsRequestSchema, +} from '@modelcontextprotocol/sdk/types.js'; + +import type { + LlmProvider, + LlmRequest, + LlmResponse, + LlmStreamEvent, +} from '@omadia/llm-provider'; +import type { ChatStreamEvent, PendingMcpInputCard } from '@omadia/channel-sdk'; +import type { PrivacyGuardService } from '@omadia/plugin-api'; +import { + InMemoryPendingMcpInputStore, + type McpInputReplayer, + type PendingMcpInput, + formatMcpInputReply, + resetSharedMcpInputWiring, +} from '../../packages/harness-orchestrator/src/mcp/pendingMcpInput.js'; +import { + McpManager, + REPLAY_ARG_KEY, + mcpNativeHandler, + type McpServerConfig, +} from '../../packages/harness-orchestrator/src/mcp/mcpClient.js'; +import { setMcpPrivacyBypassServers } from '../../packages/harness-orchestrator/src/mcpPrivacyBypass.js'; +import { NativeToolRegistry } from '../../packages/harness-orchestrator/src/nativeToolRegistry.js'; +import { Orchestrator } from '../../packages/harness-orchestrator/src/orchestrator.js'; + +const PERSON = 'Erika Mustermann'; +const EMAIL = 'erika.mustermann@example.com'; +const IBAN = 'DE89370400440532013000'; +const RAW_ROW = + `Personalakte: ${PERSON} | Email: ${EMAIL} | IBAN: ${IBAN} | Status: aktiv`; +const DIGEST_MARKER = '«dataset:lookup_employee_record»'; +const MCP_TOOL_NAME = 'mcp__HR_Payroll__lookup_employee_record'; + +const providerCapabilities = { + tools: true, + vision: true, + streaming: true, + promptCaching: true, + forcedToolChoice: true, + parallelToolCalls: true, +} as const; + +const serverArgs: Array> = []; +const managers = new Set(); + +function clearSharedState(): void { + setMcpPrivacyBypassServers([]); + resetSharedMcpInputWiring(); +} + +function redactingPrivacyService(): PrivacyGuardService { + return { + async internToolResultV4(request: { toolName: string; rawResult: string }) { + const redacted = request.rawResult + .replaceAll(PERSON, '[masked:person]') + .replaceAll(EMAIL, '[masked:email]') + .replaceAll(IBAN, '[masked:iban]'); + return { + digestText: `${DIGEST_MARKER} ${redacted}`, + datasetId: `ds-${request.toolName}`, + }; + }, + async recordBypassedTool() {}, + async runV4Tool() { + return { resultText: '' }; + }, + async subAgentResultV4() { + return { resultText: '' }; + }, + async takeRenderedAnswerV4() { + return undefined; + }, + v4ToolSpecs() { + return []; + }, + async finalizeTurn() { + return undefined; + }, + } as unknown as PrivacyGuardService; +} + +function buildMcpServerInstance(): McpSdkServer { + const mcp = new McpSdkServer( + { name: 'payroll', version: '0.0.0' }, + { capabilities: { tools: {} } }, + ); + mcp.setRequestHandler(ListToolsRequestSchema, async () => ({ + tools: [{ name: 'lookup_employee_record', inputSchema: { type: 'object' as const } }], + })); + mcp.setRequestHandler(CallToolRequestSchema, async (request) => { + const args = (request.params.arguments ?? {}) as Record; + serverArgs.push(args); + const answers = args[REPLAY_ARG_KEY]; + if (answers !== undefined && answers !== null && typeof answers === 'object') { + return { + content: [{ type: 'text' as const, text: RAW_ROW }], + }; + } + return { + content: [{ type: 'text' as const, text: 'Bitte Personalnummer und PIN angeben.' }], + resultType: 'input_required', + inputRequests: [ + { name: 'employeeId', label: 'Personalnummer' }, + { name: 'pin', label: 'PIN', secret: true }, + ], + message: 'Bitte Personalnummer und PIN angeben.', + } as never; + }); + return mcp; +} + +async function startFakeMcpServer(): Promise<{ url: string; close(): Promise }> { + const handle = async (req: IncomingMessage, res: ServerResponse): Promise => { + const mcp = buildMcpServerInstance(); + const transport = new StreamableHTTPServerTransport({ + sessionIdGenerator: undefined, + enableJsonResponse: true, + }); + res.on('close', () => { + void transport.close().catch(() => {}); + void mcp.close().catch(() => {}); + }); + await mcp.connect(transport); + if (req.method === 'POST') { + const chunks: Buffer[] = []; + for await (const c of req) chunks.push(c as Buffer); + const raw = Buffer.concat(chunks).toString('utf8'); + await transport.handleRequest(req, res, raw.length > 0 ? JSON.parse(raw) : undefined); + return; + } + await transport.handleRequest(req, res); + }; + const http: HttpServer = createServer((req, res) => { + void handle(req, res).catch(() => { + if (!res.headersSent) res.writeHead(500); + res.end(); + }); + }); + const sockets = new Set(); + http.on('connection', (socket) => { + sockets.add(socket); + socket.on('close', () => sockets.delete(socket)); + }); + await new Promise((resolve) => http.listen(0, '127.0.0.1', () => resolve())); + const { port } = http.address() as AddressInfo; + return { + url: `http://127.0.0.1:${port}/mcp`, + async close() { + for (const socket of sockets) socket.destroy(); + sockets.clear(); + await new Promise((resolve) => http.close(() => resolve())); + }, + }; +} + +const fake = await startFakeMcpServer(); + +const CFG: McpServerConfig = { + id: '00000000-0000-4000-8000-000000000945', + name: 'HR Payroll', + transport: 'http', + endpoint: fake.url, +}; + +// Teardown runs regardless of assertion outcome, and every step is +// individually guarded: a server closed only after a passing assertion turns a +// RED run into a HANG, which is how a sibling agent's mutation check in this +// wave failed to report at all. Swallowing here is correct — teardown must +// never mask the failure that is already being reported. +after(async () => { + try { + clearSharedState(); + } catch { + /* teardown must not mask a test failure */ + } + for (const manager of managers) { + try { + await manager.closeAll(); + } catch { + /* teardown must not mask a test failure */ + } + } + try { + await fake.close(); + } catch { + /* teardown must not mask a test failure */ + } +}); + +function toolCallStream( + calls: Array<{ id: string; name: string; input: unknown }>, +): LlmStreamEvent[] { + return [ + { + type: 'final', + response: { + content: calls.map((c) => ({ + type: 'tool_call', + id: c.id, + name: c.name, + input: c.input, + })), + finishReason: 'tool_calls', + providerFinishReason: 'tool_use', + model: 'test', + usage: { inputTokens: 50, outputTokens: 4, cacheReadTokens: 0, cacheWriteTokens: 0 }, + }, + } as LlmStreamEvent, + ]; +} + +function textStream(text: string): LlmStreamEvent[] { + return [ + { type: 'text_delta', text }, + { + type: 'final', + response: { + content: [{ type: 'text', text }], + finishReason: 'stop', + providerFinishReason: 'end_turn', + model: 'test', + usage: { inputTokens: 100, outputTokens: 1, cacheReadTokens: 0, cacheWriteTokens: 0 }, + }, + } as LlmStreamEvent, + ]; +} + +function fakeStreamProvider( + streams: LlmStreamEvent[][], + seenRequests: LlmRequest[], +): LlmProvider { + let idx = 0; + return { + id: 'anthropic', + capabilities: providerCapabilities, + complete: async (req: LlmRequest): Promise => { + seenRequests.push(req); + const events = streams[idx]; + idx += 1; + if (!events) throw new Error(`no scripted stream for provider call ${String(idx)}`); + const final = events.at(-1) as { response: LlmResponse }; + return final.response; + }, + stream: (req: LlmRequest): AsyncIterable => { + seenRequests.push(req); + const events = streams[idx]; + idx += 1; + if (!events) throw new Error(`no scripted stream for provider call ${String(idx)}`); + return { + async *[Symbol.asyncIterator]() { + for (const ev of events) yield ev; + }, + }; + }, + classifyError: () => ({ retryable: false, kind: 'other' as const }), + } as unknown as LlmProvider; +} + +interface Harness { + readonly orchestrator: Orchestrator; + readonly seenRequests: LlmRequest[]; + readonly store: InMemoryPendingMcpInputStore; +} + +function harness( + streams: LlmStreamEvent[][], + options?: { readonly privacyGuard?: () => PrivacyGuardService | undefined }, +): Harness { + const store = new InMemoryPendingMcpInputStore(); + const manager = new McpManager({ pendingInput: store }); + managers.add(manager); + const registry = new NativeToolRegistry(); + registry.register(MCP_TOOL_NAME, { + handler: mcpNativeHandler(manager, CFG, 'lookup_employee_record'), + spec: { + name: MCP_TOOL_NAME, + description: 'Look up an employee record.', + input_schema: { type: 'object' as const, properties: {}, required: [] }, + } as never, + agentId: 'mcp-test', + }); + const seenRequests: LlmRequest[] = []; + const replayer: McpInputReplayer = { + replay: async (record: PendingMcpInput, inputResponses: Record) => + manager.callTool(CFG, record.toolName, { + ...record.originalArgs, + [REPLAY_ARG_KEY]: inputResponses, + }), + }; + const orchestrator = new Orchestrator({ + provider: fakeStreamProvider(streams, seenRequests), + model: 'test', + maxTokens: 1024, + maxToolIterations: 5, + domainTools: [], + nativeToolRegistry: registry, + pendingMcpInput: store, + mcpInputReplay: replayer, + ...(options?.privacyGuard ? { privacyGuard: options.privacyGuard } : {}), + }); + return { orchestrator, seenRequests, store }; +} + +async function runStream( + orchestrator: Orchestrator, + userMessage: string, + sessionScope: string, + userId: string, +): Promise { + const events: ChatStreamEvent[] = []; + for await (const ev of orchestrator.chatStream({ userMessage, sessionScope, userId })) { + events.push(ev); + } + return events; +} + +function doneEvent(events: ChatStreamEvent[]): { + answer: string; + pendingMcpInput?: PendingMcpInputCard; +} { + const done = events.find((event) => event.type === 'done'); + assert.ok(done, 'no done event'); + return done as never; +} + +function wireUserText(requests: readonly LlmRequest[]): string { + const parts: string[] = []; + for (const req of requests) { + for (const message of (req.messages ?? []) as Array<{ role: string; content: unknown }>) { + if (message.role !== 'user') continue; + if (typeof message.content === 'string') { + parts.push(message.content); + continue; + } + if (!Array.isArray(message.content)) continue; + for (const block of message.content as Array<{ text?: string }>) { + if (typeof block.text === 'string') parts.push(block.text); + } + } + } + return parts.join('\n'); +} + +const SESSION = 'sess-1'; +const USER = 'u1'; + +/** + * Park a record and bind it to `(USER, SESSION)` exactly as a real first turn + * would: `put` stores it ownerless, `claim` binds the owner. `take` during the + * replay turn then needs the full `{userId, sessionId, correlationId}` triple, + * so the #445 ownership defence is exercised rather than bypassed. + */ +function seedParkedCard(h: Harness, correlationId: string): void { + const record: PendingMcpInput = { + correlationId, + serverId: CFG.id, + serverName: CFG.name, + toolName: 'lookup_employee_record', + originalArgs: { caseId: 'HR-7' }, + inputRequests: [ + { name: 'employeeId', required: true }, + { name: 'pin', secret: true, required: true }, + ], + replayDepth: 0, + }; + assert.equal(h.store.put(record), 'stored'); + assert.ok( + h.store.claim(correlationId, { userId: USER, sessionId: SESSION }), + 'the seeded record must claim, or the replay turn cannot take it', + ); +} + +/** + * Drive the replay turn and return the user-role text the LLM provider saw. + * This is the wire the fix is about — the browser is on the trusted side and + * is deliberately not asserted on here. + */ +async function replayWire( + h: Harness, + correlationId: string, + inputResponses: Record, +): Promise { + await runStream( + h.orchestrator, + formatMcpInputReply({ correlationId, inputResponses }), + SESSION, + USER, + ); + return wireUserText(h.seenRequests); +} + +describe('MCP input replay privacy boundary (#544 / W2-1)', () => { + it('MUTATION CHECK: the replay note interns the MCP result before it reaches the LLM wire', async () => { + clearSharedState(); + serverArgs.length = 0; + const h = harness([textStream('fertig')], { + privacyGuard: () => redactingPrivacyService(), + }); + seedParkedCard(h, 'corr-intern'); + + const wire = await replayWire(h, 'corr-intern', { employeeId: 'E-42', pin: '4321' }); + + // The server DID return the row — otherwise "absent from the wire" would + // pass vacuously over a replay that never happened. + assert.ok( + serverArgs.some((a) => a[REPLAY_ARG_KEY] !== undefined), + 'the replay never reached the MCP server', + ); + assert.equal(wire.includes(PERSON), false, `person name crossed the wire: ${wire}`); + assert.equal(wire.includes(EMAIL), false, `email crossed the wire: ${wire}`); + assert.equal(wire.includes(IBAN), false, `IBAN crossed the wire: ${wire}`); + assert.ok(wire.includes(DIGEST_MARKER), `digest marker missing from the wire: ${wire}`); + assert.ok(wire.includes('[masked:person]'), `masked payload missing from the wire: ${wire}`); + }); + + it('MUTATION CHECK: an MCP privacy-bypass server keeps the replay result raw on the wire', async () => { + clearSharedState(); + serverArgs.length = 0; + setMcpPrivacyBypassServers([CFG.id]); + try { + const h = harness([textStream('fertig')], { + privacyGuard: () => redactingPrivacyService(), + }); + seedParkedCard(h, 'corr-bypass'); + + const wire = await replayWire(h, 'corr-bypass', { employeeId: 'E-77', pin: '9999' }); + + assert.ok(wire.includes(PERSON), `person name missing from bypassed wire text: ${wire}`); + assert.ok(wire.includes(EMAIL), `email missing from bypassed wire text: ${wire}`); + assert.ok(wire.includes(IBAN), `IBAN missing from bypassed wire text: ${wire}`); + assert.equal(wire.includes(DIGEST_MARKER), false, `digest should not replace bypassed raw text: ${wire}`); + } finally { + setMcpPrivacyBypassServers([]); + } + }); + + it('MUTATION CHECK: without a privacy handle the replay note stays legacy-raw byte-for-byte', async () => { + clearSharedState(); + serverArgs.length = 0; + const h = harness([textStream('fertig')]); + seedParkedCard(h, 'corr-legacy'); + + const wire = await replayWire(h, 'corr-legacy', { employeeId: 'E-99', pin: '1111' }); + + assert.ok(wire.includes(RAW_ROW), `legacy raw replay result missing from the wire: ${wire}`); + assert.equal(wire.includes(DIGEST_MARKER), false, `digest should not appear without privacy guard: ${wire}`); + }); +}); From 88cc0875fcdc82cbdf97565bcdb24ef8496fccd0 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 14:42:15 +0200 Subject: [PATCH 88/90] docs(test): name the sidecar mechanism instead of a verdict in #547 coverage MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two test names in `mcpStructuredOutputPrivacy.test.ts` encoded a conclusion the code does not support, because the author misread which side of the Privacy Shield v4 boundary the browser sits on. * `BASELINE — the TEXT result a client receives IS masked at the dispatch seam` — wrong as written. The interned digest is what the MODEL receives. The client legitimately gets real values via `takeRenderedAnswerV4` (`plugin-api/src/privacyReceipt.ts:164-174`), rendered with `highlightTerms={message.maskedValues}`. * `LEAK — the STRUCTURED sidecar carries the same PII in CLEAR on the same call` — the assertion is true, the word LEAK is not. The boundary is server <-> LLM provider, not server <-> browser, and the browser is on the trusted side. Both now describe the mechanism: the dispatched text IS interned, the sidecar is NOT. The header gains an explicit "which boundary this is about" section and records the earlier misreading, so the next reader does not re-derive it. The sidecar bypassing interning remains a real and documented property worth pinning — it makes the payload unsafe to forward across the model boundary and unsafe for any consumer to assume masked. Naming and framing only: all 20 assertions are unchanged, none weakened or removed. --- .../test/mcpStructuredOutputPrivacy.test.ts | 54 ++++++++++++------- 1 file changed, 36 insertions(+), 18 deletions(-) diff --git a/middleware/test/mcpStructuredOutputPrivacy.test.ts b/middleware/test/mcpStructuredOutputPrivacy.test.ts index b929a38b..04ab0571 100644 --- a/middleware/test/mcpStructuredOutputPrivacy.test.ts +++ b/middleware/test/mcpStructuredOutputPrivacy.test.ts @@ -1,22 +1,35 @@ /** - * Issue #547 (W5-2) — WHY the structured-output sidecar is NOT wired to the - * chat client. + * Issue #547 (W5-2) — WHY the structured-output sidecar is NOT wired onward. * * #547 landed the producer (`McpManager.structuredSink`) as plumbing only. The * obvious next step is to wire that sink through the orchestrator onto the - * terminal `done` stream event so the UI can render a card. This file is the - * evidence that doing so, as the seam stands today, would be a PII leak — and - * it is a regression guard: if someone later makes the sidecar mask, the - * `LEAK` test below turns red and this file must be revisited on purpose. + * terminal `done` stream event so the UI can render a card. This file pins the + * property that decides whether that is safe: the sidecar never crosses the + * interning seam, so it still carries exactly what the server sent. It is also + * a regression guard — if someone later makes the sidecar mask, the sidecar + * test below turns red and this file must be revisited on purpose. + * + * WHICH BOUNDARY THIS IS ABOUT — read this before calling anything a leak. + * Privacy Shield v4's data-plane boundary is server <-> LLM PROVIDER, not + * server <-> browser. The browser sits on the TRUSTED side and legitimately + * receives real values: that is precisely what `takeRenderedAnswerV4` hands + * back (`plugin-api/src/privacyReceipt.ts:164-174`), rendered with + * `highlightTerms={message.maskedValues}` in the chat page. So "the sidecar is + * not interned" is NOT by itself a leak to the browser, and this file + * deliberately does not call it one. What it IS: an asymmetry that makes the + * sidecar unsafe to forward across the MODEL boundary, and unsafe for any + * future consumer to assume masked. An earlier revision of this file named one + * test `LEAK` and described the interned text as what "a client receives"; + * both encoded a misreading of which side the browser is on. * * The asymmetry, stated exactly: * * - A tool's TEXT result is interned at the dispatch seam. `dispatchTool` - * returns `internToolResultV4(...).digestText`, and the client-facing - * `tool_result` stream event carries precisely that return value + * returns `internToolResultV4(...).digestText`, so the string the MODEL + * sees is a digest, not the rows. That same return value is what the + * client-facing `tool_result` stream event carries * (`orchestrator.ts:5330` builds the slot promise from `dispatchTool`; - * `:4934` resolves it; `:4977` puts it on the wire as `output`). So the - * browser sees the digest, not the rows. + * `:4934` resolves it; `:4977` puts it on the wire as `output`). * * - The STRUCTURED payload is emitted from inside `McpManager.callTool` * (`mcpClient.ts:~880`), which sits strictly BELOW every dispatcher. It @@ -26,11 +39,11 @@ * * That "strictly below every dispatcher" is why this file proves the property * using `ToolDispatchService` rather than a full `Orchestrator` turn: the sink - * fires beneath the dispatcher, so the leak is dispatcher-independent. The + * fires beneath the dispatcher, so the asymmetry is dispatcher-independent. The * interning contract asserted here is the same one the chat path uses and is * documented as parity in `toolDispatchPrivacySeam.test.ts`. * - * There is also no way to fix this inside W5-2's scope. The whole privacy + * There is also no way to close that asymmetry inside W5-2's scope. The whole privacy * contract (`PrivacyTurnHandle`) is string-in/string-out: * `internToolResultV4({rawResult: string}) -> {digestText: string}`. Feeding a * structured payload through it returns a digest STRING — the structure the @@ -295,12 +308,15 @@ function structuredSidecars( // ── the finding ───────────────────────────────────────────────────────────── describe('#547 structured-output sidecar vs. the privacy boundary (W5-2)', () => { - it('BASELINE — the TEXT result a client receives IS masked at the dispatch seam', async () => { + it('the TEXT result IS interned at the dispatch seam, so the MODEL sees a digest', async () => { const h = harness(); const result = await h.service.dispatch(TOOL, {}); // This is the benchmark the brief calls "the same terms as text output". + // `result.content` is the dispatcher's return value — the string bound for + // the model. What the BROWSER ultimately renders is a separate question + // (see the header): it is on the trusted side and gets real values. assert.equal( result.content.includes(EMAIL), false, @@ -317,7 +333,7 @@ describe('#547 structured-output sidecar vs. the privacy boundary (W5-2)', () => assert.match(result.content, /«dataset:crm_lookup_customer»/); }); - it('LEAK — the STRUCTURED sidecar carries the same PII in CLEAR on the same call', async () => { + it('the STRUCTURED sidecar is NOT interned, so it still carries the raw values', async () => { const h = harness(); const result = await h.service.dispatch(TOOL, {}); @@ -330,8 +346,10 @@ describe('#547 structured-output sidecar vs. the privacy boundary (W5-2)', () => const payload = structured[0]!.structured as Record; // The load-bearing assertions: raw values, byte-identical to what the - // server sent. Wiring this payload onto the `done` event would put every - // one of these into the browser on a turn where the text was masked. + // server sent. Not a leak in itself — the browser is trusted — but it + // pins that ANY future consumer of this payload must treat it as unmasked, + // and that forwarding it across the model boundary would undo the + // interning the text path just performed. assert.equal(payload['email'], EMAIL); assert.equal(payload['iban'], IBAN); assert.equal(payload['name'], PERSON); @@ -339,7 +357,7 @@ describe('#547 structured-output sidecar vs. the privacy boundary (W5-2)', () => }); it('the sidecar is emitted BENEATH the dispatcher, so no dispatcher can mask it', async () => { - // Proves the leak is structural rather than a property of one dispatcher: + // Proves the asymmetry is structural rather than a property of one dispatcher: // the payload is already in the sink by the time `dispatch` returns, and // the value in the sink is unaffected by the masking that produced // `result.content`. @@ -358,7 +376,7 @@ describe('#547 structured-output sidecar vs. the privacy boundary (W5-2)', () => }); it('carries the declared `outputSchema`, so a generic renderer is buildable once masking exists', async () => { - // Not a leak assertion — it records that the ONLY blocker is masking. The + // Not an exposure assertion — it records that the ONLY blocker is masking. The // renderer contract the brief specifies (render from `outputSchema`, never // by tool name) is already satisfiable end-to-end over a real wire. const h = harness(); From b715f1465cd4ad32035ce8c62a2c586112355705 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 15:04:48 +0200 Subject: [PATCH 89/90] test(memory): widen the path-validation coverage the review found too narrow Three gaps in a file I added earlier in this wave, all found by review: The negative control used a single tidy path, so it could not catch the mistake standing closest to the bug this file exists for. That bug was a NUL branch reporting 'Path contains a space.'; someone reading that message in a stale checkout fixes it the other way round and adds a real space rejection. With one space-free control path, every other test stays green while every memory file whose name contains a space breaks at runtime. The control now covers a space, a non-traversal dot, and non-ASCII. 'Applies the guard on every entry point' covered four of eight, omitting directoryExists, createFile, and -- worst -- delete and rename, the two destructive ones. All eight now. rename's destination was entirely unpinned: a refactor keeping normalize(from) and dropping normalize(to) passed the whole suite. pg parameterisation would still reject a NUL in a bind parameter, so this is defence in depth rather than an injection hole, but the guard should not lean on the driver. Mutation-checked with a rebuild, since the import goes through the package barrel: adding a space rejection fails the control, dropping normalize(to) fails the rename test, and a git-status guard confirmed the source returned to HEAD after each. --- .../postgresMemoryStorePathValidation.test.ts | 63 ++++++++++++++----- 1 file changed, 48 insertions(+), 15 deletions(-) diff --git a/middleware/test/postgresMemoryStorePathValidation.test.ts b/middleware/test/postgresMemoryStorePathValidation.test.ts index 1196bda5..e7c86eb1 100644 --- a/middleware/test/postgresMemoryStorePathValidation.test.ts +++ b/middleware/test/postgresMemoryStorePathValidation.test.ts @@ -74,30 +74,63 @@ describe('PostgresMemoryStore path validation', () => { const store = new PostgresMemoryStore(poolThatMustNotBeQueried()); const bad = '/memories/core/no\0tes.md'; - // Each of these normalises before its first query. A future refactor that - // validates in only one of them is the failure this pins. + // ALL EIGHT path-taking methods, not a sample. An earlier version of this + // test covered four and still called itself "every entry point"; a refactor + // that added an early return to `delete` — bypassing normalize on the most + // destructive method — would have passed it. await assert.rejects(() => store.list(bad), MemoryInvalidPathError); await assert.rejects(() => store.fileExists(bad), MemoryInvalidPathError); + await assert.rejects(() => store.directoryExists(bad), MemoryInvalidPathError); await assert.rejects(() => store.readFile(bad), MemoryInvalidPathError); + await assert.rejects(() => store.createFile(bad, 'x'), MemoryInvalidPathError); await assert.rejects(() => store.writeFile(bad, 'x'), MemoryInvalidPathError); + await assert.rejects(() => store.delete(bad), MemoryInvalidPathError); + await assert.rejects(() => store.rename(bad, '/memories/core/ok.md'), MemoryInvalidPathError); }); - it('accepts an ordinary path far enough to reach the pool', async () => { - // The negative control. Without it, a `normalize` that rejected EVERY path - // would satisfy all three tests above. + it("normalises rename's DESTINATION, not only its source", async () => { + // `rename` is the one method taking two paths, and the second was entirely + // unpinned: a refactor keeping `normalize(from)` and dropping `normalize(to)` + // passed the whole suite. Defence in depth rather than an injection hole — + // pg parameterisation rejects a NUL in a bind parameter at the driver — but + // the guard should not depend on the driver to hold. const store = new PostgresMemoryStore(poolThatMustNotBeQueried()); await assert.rejects( - () => store.fileExists('/memories/core/notes.md'), - (err: unknown) => { - const message = err instanceof Error ? err.message : String(err); - assert.match( - message, - /before validating the path/, - `a clean path must survive validation and reach the pool, got: ${message}`, - ); - return true; - }, + () => store.rename('/memories/core/from.md', '/memories/core/no\0tes.md'), + MemoryInvalidPathError, ); }); + + it('accepts ordinary paths far enough to reach the pool', async () => { + // The negative control, and it has to be wider than one tidy path. + // + // The bug this file exists for was a NUL branch reporting 'Path contains a + // space.' Someone reading that message in a stale checkout "fixes" it the + // other way round and adds a real space rejection. With a single + // space-free control path, every test above stays green while every memory + // file whose name contains a space breaks at runtime — the suite built to + // protect this line would say nothing. + const store = new PostgresMemoryStore(poolThatMustNotBeQueried()); + + for (const clean of [ + '/memories/core/notes.md', + '/memories/core/my notes.md', // a space is legal, and must stay legal + '/memories/core/notes.v2.md', // a dot that is not a traversal + '/memories/core/Ünïcödé.md', + ]) { + await assert.rejects( + () => store.fileExists(clean), + (err: unknown) => { + const message = err instanceof Error ? err.message : String(err); + assert.match( + message, + /before validating the path/, + `${clean} must survive validation and reach the pool, got: ${message}`, + ); + return true; + }, + ); + } + }); }); From ee814f705680288cd4a4a43fce5f2c1fc36fa1c6 Mon Sep 17 00:00:00 2001 From: Marcel Wege Date: Fri, 31 Jul 2026 15:16:33 +0200 Subject: [PATCH 90/90] fix(migrations): run the 0031 delegation backfill only when the column is introduced MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The backfill grandfathering operator-token servers to `delegation = 'service'` had no first-apply condition. Changing `delegation` does not delete the operator token row, so the EXISTS predicate still matched on every subsequent application: an operator who opted an existing server into `per_user` — the exact action 0031's own header instructs them to take — had it silently flipped back to `service`, handing unmapped channel users operator authority again and undoing the D2 confused-deputy fix. No log, no error. Re-application is the expected mode: migrations/README.md documents that there is no runner and no applied-migrations bookkeeping yet. The backfill was idempotent in the SQL sense but not with respect to operator intent. The ADD COLUMN and the backfill now sit inside one DO block gated on a pg_attribute lookup for mcp_servers.delegation, so both run exactly once, when the column is first added. ADD COLUMN is plain rather than IF NOT EXISTS: the gate already proved absence, and masking a broken gate would turn a logic error into silent drift. The CHECK-constraint guard stays a separate top-level block — it carries no operator intent and must still repair a database where the column exists but the constraint does not. Neither existing gate could see this: CI re-applies against an empty database, and the pg suite re-applied against rows it never flipped. Tests: add the re-apply regression (flip a grandfathered server to per_user, re-apply, assert it stays) and a constraint-independence test that drops the CHECK and proves re-apply restores it. Both are restore-safe via try/finally so they cannot perturb sibling tests. Also from the same review: - the schema-relative guard only matched `public.` inside a quoted literal, so `ALTER TABLE public.mcp_servers` slipped through; now a regex over the comment-stripped text, via a shared helper - the CHECK-constrains test asserted bare SQLSTATE 23514, which mcp_servers_transport_check also satisfies; now asserts the constraint name - the constraint-existence lookup gains the conrelid filter its title claims - header prose named assertSchemaRelative(), which does not exist --- .../0031_mcp_oauth_iss_delegation.sql | 113 ++++++++++------ .../mcpDelegationBackfillMigration.pg.test.ts | 121 ++++++++++++++---- 2 files changed, 168 insertions(+), 66 deletions(-) diff --git a/middleware/migrations/0031_mcp_oauth_iss_delegation.sql b/middleware/migrations/0031_mcp_oauth_iss_delegation.sql index f873138c..f2780153 100644 --- a/middleware/migrations/0031_mcp_oauth_iss_delegation.sql +++ b/middleware/migrations/0031_mcp_oauth_iss_delegation.sql @@ -24,40 +24,29 @@ -- A fail-closed `per_user` default for EVERY row would break installed systems -- whose channel users reach MCP servers today precisely BECAUSE of the -- 'operator' fallback. So this migration is deliberately asymmetric: --- • existing rows that already hold an operator token keep today's shared --- behaviour (delegation = 'service'), and --- • only NEWLY created servers get the safe 'per_user' default. --- Operators who want per-user delegation on an existing server must opt in via --- the MCP Control Center (or UPDATE the column directly). +-- • when `mcp_servers.delegation` is introduced, pre-existing rows that +-- already hold an operator token are grandfathered to today's shared +-- behaviour (`delegation = 'service'`), +-- • operators who later opt an existing server into `per_user` stay there on +-- every re-apply, because that grandfathering runs only on the first +-- application that adds the column, and +-- • only NEWLY created servers get the safe `per_user` default. +-- That one-time asymmetry is intentional. A new server that later acquires an +-- operator token is still `per_user` unless an operator deliberately chooses +-- shared delegation in the MCP Control Center (or UPDATEs the column directly). -- ── D2: explicit delegation mode per MCP server ───────────────────────────── -ALTER TABLE mcp_servers - ADD COLUMN IF NOT EXISTS delegation TEXT NOT NULL DEFAULT 'per_user'; - --- `conname` is unique per (connamespace, conrelid), NOT cluster-wide, so an --- unanchored lookup reports "exists" for a same-named constraint sitting in any --- other schema and the ALTER below is silently skipped. Anchoring on --- `conrelid = 'mcp_servers'::regclass` resolves the relation through --- `search_path`, matching every other unqualified reference in this file. The --- cast cannot raise here: the ALTER TABLE above already required the table to --- resolve. -DO $$ -BEGIN - IF NOT EXISTS ( - SELECT 1 FROM pg_constraint - WHERE conname = 'mcp_servers_delegation_chk' - AND conrelid = 'mcp_servers'::regclass - ) THEN - ALTER TABLE mcp_servers - ADD CONSTRAINT mcp_servers_delegation_chk - CHECK (delegation IN ('per_user', 'service')); - END IF; -END $$; - --- Backward compatibility (see the warning above): every EXISTING server that +-- The column introduction and the compatibility backfill are one decision and +-- must happen exactly once. Re-applying 0031 after an operator has opted an +-- existing server into `per_user` must preserve that operator decision: +-- changing `delegation` does NOT delete the stored operator token row, so a +-- standing backfill would silently flip the server back to `service` on the +-- next apply and reopen D2 for unmapped channel users. +-- +-- Backward compatibility on that first apply only: every EXISTING server that -- already holds an OPERATOR token keeps the shared identity it is working with --- today. Guarded by to_regclass so the migration is safe on a database where --- mcp_oauth_tokens has not been created yet. The argument is UNQUALIFIED on +-- today. Guarded by `to_regclass` so the migration is safe on a database where +-- `mcp_oauth_tokens` has not been created yet. The argument is UNQUALIFIED on -- purpose: it must resolve through `search_path` like every other reference in -- this file. A hardcoded `public.` would probe the wrong schema wherever the -- domain is applied outside `public` — the guard would then answer about a @@ -69,25 +58,67 @@ END $$; -- producing (see D2 above), and that fallback only ever applied where an -- operator token existed to borrow. A server holding only per-user tokens — -- `user_key = 'alice@corp.com'` and nothing else — was never using a shared --- identity, so flipping it to 'service' would be a silent identity change no +-- identity, so flipping it to `service` would be a silent identity change no -- operator decided on: `resolveMcpUserKey` would hand every caller the shared -- `operator` key, and once anyone completed a re-auth the minted operator token -- would be shared by every caller, including unmapped channel users. The --- narrow predicate leaves such a server on the safe 'per_user' default, which +-- narrow predicate leaves such a server on the safe `per_user` default, which -- is the choice its stored tokens already imply. -- -- The literal must stay in sync with `SERVICE_USER_KEY` in -- `src/services/mcpDelegation.ts` (a migration cannot import it). DO $$ +DECLARE + delegation_exists BOOLEAN; BEGIN - IF to_regclass('mcp_oauth_tokens') IS NOT NULL THEN - UPDATE mcp_servers s - SET delegation = 'service' - WHERE EXISTS ( - SELECT 1 FROM mcp_oauth_tokens t - WHERE t.server_id = s.id - AND t.user_key = 'operator' - ); + SELECT EXISTS ( + SELECT 1 + FROM pg_attribute + WHERE attrelid = 'mcp_servers'::regclass + AND attname = 'delegation' + AND NOT attisdropped + ) + INTO delegation_exists; + + IF NOT delegation_exists THEN + -- Plain ADD COLUMN, not IF NOT EXISTS: the catalog gate above already + -- proved absence, and masking a broken gate here would turn a logic error + -- into silent drift instead of failing loudly at the statement that broke. + ALTER TABLE mcp_servers + ADD COLUMN delegation TEXT NOT NULL DEFAULT 'per_user'; + + IF to_regclass('mcp_oauth_tokens') IS NOT NULL THEN + UPDATE mcp_servers s + SET delegation = 'service' + WHERE EXISTS ( + SELECT 1 FROM mcp_oauth_tokens t + WHERE t.server_id = s.id + AND t.user_key = 'operator' + ); + END IF; + END IF; +END $$; + +-- `conname` is unique per (connamespace, conrelid), NOT cluster-wide, so an +-- unanchored lookup reports "exists" for a same-named constraint sitting in any +-- other schema and the ALTER below is silently skipped. Anchoring on +-- `conrelid = 'mcp_servers'::regclass` resolves the relation through +-- `search_path`, matching every other unqualified reference in this file. The +-- cast cannot raise here for any new reason: this migration still requires +-- `mcp_servers` to resolve, the gate block above already resolved the same +-- regclass while checking whether `delegation` exists, and the old `ALTER TABLE +-- ... ADD COLUMN IF NOT EXISTS` would likewise have failed on a missing table +-- because `IF NOT EXISTS` guards the column name, not the relation. +DO $$ +BEGIN + IF NOT EXISTS ( + SELECT 1 FROM pg_constraint + WHERE conname = 'mcp_servers_delegation_chk' + AND conrelid = 'mcp_servers'::regclass + ) THEN + ALTER TABLE mcp_servers + ADD CONSTRAINT mcp_servers_delegation_chk + CHECK (delegation IN ('per_user', 'service')); END IF; END $$; diff --git a/middleware/test/mcpDelegationBackfillMigration.pg.test.ts b/middleware/test/mcpDelegationBackfillMigration.pg.test.ts index e66f787e..5e3ca8d5 100644 --- a/middleware/test/mcpDelegationBackfillMigration.pg.test.ts +++ b/middleware/test/mcpDelegationBackfillMigration.pg.test.ts @@ -50,9 +50,9 @@ import { SERVICE_USER_KEY } from '../src/services/mcpDelegation.js'; * answered about a table this migration never touches, and this file had to * rewrite the literal before running it. The guard is now unqualified and * resolves through `search_path` like everything else, so the file is applied - * AS SHIPPED and `assertSchemaRelative()` fails loudly if a qualified reference - * is ever reintroduced — a silent short-circuit would otherwise make every - * assertion below pass vacuously. + * AS SHIPPED and `migrationSql()` fails loudly if an executable `public.`- + * qualified reference is ever reintroduced — a silent short-circuit would + * otherwise make every assertion below pass vacuously. */ const PG_URL = @@ -75,14 +75,25 @@ const TENANT = `w4_deleg_${process.pid}_${Date.now().toString(36)}`; const MIGRATION_PATH = new URL('../migrations/0031_mcp_oauth_iss_delegation.sql', import.meta.url); +function stripWholeLineSqlComments(sql: string): string { + // Whole-line `--` stripping is sufficient for this file: the migration's + // false positives live in prose comments, and its executable statements do + // not use trailing `--` comments that would need SQL-aware parsing. + return sql + .split('\n') + .filter((line) => !line.trimStart().startsWith('--')) + .join('\n'); +} + /** The migration text, applied verbatim. See the header for why no rewrite is - * needed, and what the schema-relative guard protects. */ + * needed, and what the public-qualified guard protects. */ async function migrationSql(): Promise { const raw = await readFile(MIGRATION_PATH, 'utf8'); + const executable = stripWholeLineSqlComments(raw); assert.equal( - raw.split("'public.").length - 1, - 0, - 'migration 0031 gained a schema-qualified reference — it must resolve through search_path, ' + + /\bpublic\s*\.|"public"\s*\./i.test(executable), + false, + 'migration 0031 gained an executable public-qualified reference — it must resolve through search_path, ' + 'or this suite runs it against tables it does not own and passes vacuously', ); return raw; @@ -170,6 +181,41 @@ describe('migration 0031 — delegation backfill predicate (pg)', { skip: !pgAva return rows[0]?.delegation; } + async function applyMigration(): Promise { + await pool.query(await migrationSql()); + } + + async function hasDelegationConstraint(): Promise { + const { rows } = await pool.query<{ present: boolean }>( + `SELECT EXISTS ( + SELECT 1 + FROM pg_constraint + WHERE conname = 'mcp_servers_delegation_chk' + AND connamespace = (SELECT oid FROM pg_namespace WHERE nspname = $1) + AND conrelid = 'mcp_servers'::regclass + ) AS present`, + [TENANT], + ); + return rows[0]?.present ?? false; + } + + async function ensureDelegationConstraint(): Promise { + await pool.query(` + DO $$ + BEGIN + IF NOT EXISTS ( + SELECT 1 FROM pg_constraint + WHERE conname = 'mcp_servers_delegation_chk' + AND conrelid = 'mcp_servers'::regclass + ) THEN + ALTER TABLE mcp_servers + ADD CONSTRAINT mcp_servers_delegation_chk + CHECK (delegation IN ('per_user', 'service')); + END IF; + END $$; + `); + } + it('does NOT flip a server holding only a NON-operator token', async () => { // THE regression. The broad `EXISTS (… WHERE server_id = s.id)` predicate // matched this row and handed every future caller the shared operator key. @@ -199,14 +245,7 @@ describe('migration 0031 — delegation backfill predicate (pg)', { skip: !pgAva // The migration cannot import `SERVICE_USER_KEY`, so the two literals can // drift. If they ever do, the backfill grandfathers a different set of // servers than the runtime can actually resolve tokens for. - // - // Comments are stripped first: the file DISCUSSES the predicate in prose - // right above it, so matching the raw text would stay green over a backfill - // that no longer filters at all. - const executable = (await readFile(MIGRATION_PATH, 'utf8')) - .split('\n') - .filter((line) => !line.trimStart().startsWith('--')) - .join('\n'); + const executable = stripWholeLineSqlComments(await readFile(MIGRATION_PATH, 'utf8')); assert.equal(SERVICE_USER_KEY, 'operator'); assert.match(executable, new RegExp(`user_key\\s*=\\s*'${SERVICE_USER_KEY}'`)); }); @@ -222,15 +261,9 @@ describe('migration 0031 — delegation backfill predicate (pg)', { skip: !pgAva // assertions below are scoped to THIS suite's schema for the same reason. it('creates the delegation CHECK in THIS schema, not merely somewhere in the cluster', async () => { - const { rows } = await pool.query<{ n: string }>( - `SELECT count(*)::text AS n FROM pg_constraint - WHERE conname = 'mcp_servers_delegation_chk' - AND connamespace = (SELECT oid FROM pg_namespace WHERE nspname = $1)`, - [TENANT], - ); assert.equal( - rows[0]?.n, - '1', + await hasDelegationConstraint(), + true, 'the CHECK was skipped in this schema — the guard matched a constraint owned by another schema', ); }); @@ -244,16 +277,54 @@ describe('migration 0031 — delegation backfill predicate (pg)', { skip: !pgAva `INSERT INTO mcp_servers (name, transport, endpoint, delegation) VALUES ('bad-delegation', 'http', 'https://e.example', 'telepathy')`, ), - (err: unknown) => (err as { code?: string }).code === '23514', + (err: unknown) => { + const pgErr = err as { code?: string; constraint?: string }; + assert.equal(pgErr.code, '23514'); + assert.equal(pgErr.constraint, 'mcp_servers_delegation_chk'); + return true; + }, 'delegation accepted a value outside (per_user, service)', ); }); + it('re-applying preserves an operator opt-in from service back to per_user', async () => { + try { + await pool.query(`UPDATE mcp_servers SET delegation = 'per_user' WHERE name = 'operator-only'`); + await applyMigration(); + assert.equal( + await delegationOf('operator-only'), + 'per_user', + 're-applying 0031 silently overrode an operator opt-in back to service', + ); + } finally { + await pool.query(`UPDATE mcp_servers SET delegation = 'service' WHERE name = 'operator-only'`); + } + }); + + it('re-applying recreates the delegation CHECK even when the column already exists', async () => { + let dropped = false; + try { + await pool.query(`ALTER TABLE mcp_servers DROP CONSTRAINT mcp_servers_delegation_chk`); + dropped = true; + assert.equal(await hasDelegationConstraint(), false); + await applyMigration(); + assert.equal( + await hasDelegationConstraint(), + true, + 'the top-level CHECK guard stopped repairing a table that already had the delegation column', + ); + } finally { + if (dropped) { + await ensureDelegationConstraint(); + } + } + }); + it('is idempotent — re-applying flips nothing further', async () => { // A second run must be a no-op, not a second chance to convert a per-user // server (e.g. if one acquired an operator token in between, that is a real // change; if not, nothing may move). - await pool.query(await migrationSql()); + await applyMigration(); assert.equal(await delegationOf('per-user-only'), 'per_user'); assert.equal(await delegationOf('operator-only'), 'service'); assert.equal(await delegationOf('no-tokens'), 'per_user');