[WRONG BRANCH] chore: temporary dev delta extraction for #1206 rebase

bin/ocx.mjs

@@ -17,6 +17,10 @@ import { dirname, join, resolve } from "node:path";
 import { fileURLToPath } from "node:url";
 import { isRealBunBinary } from "../src/lib/bun-binary-validator.mjs";
 import { npmInvocation } from "../src/update/npm-invocation.mjs";
+import {
+  npmCachePreflightFailureMessage,
+  runNpmCachePreflight,
+} from "../src/update/npm-cache-preflight.mjs";
 import { handoffWindowsTrayForUpdate, planWindowsTrayUpdate } from "../src/update/tray-update-plan.mjs";
 
 const PKG = "@bitkyc08/opencodex";
@@ -136,6 +140,12 @@ function runNpmSelfUpdate() {
     process.exit(0);
   }
 
+  const cachePreflight = runNpmCachePreflight();
+  if (!cachePreflight.ok) {
+    console.error(`opencodex: ${npmCachePreflightFailureMessage(cachePreflight.reason)}. Aborting before stopping the proxy.`);
+    process.exit(1);
+  }
+
   // Remember whether a background service manages the proxy BEFORE stopping — `ocx stop`
   // unloads it, so a successful update must refresh and restart it afterwards.
   const serviceStatePath = join(configDir(), "service-state.json");

devlog/_plan/260806_overnight_triage_round2/002_issue_1065_rca.md

@@ -49,3 +49,15 @@ and default-path equivalence for the 5s error-body callers.
 Nothing on origin/dev (last 30 commits) or in any open PR touches
 `bounded-body` or this stall path. #947 is the Darwin SSE relay, a
 different path. #1069 is unrelated ladder metadata.
+
+## Supersession note (2026-08-07)
+
+The "keep bounded JSON, do not restore streaming" disposition above is
+superseded by `devlog/_plan/260807_deepseek_responses_streaming/`: fresh
+upstream probes (2026-08-07, including the tool-result replay shape behind
+#875) show DeepSeek's `/responses` stream closing on the documented
+`response.completed` terminal, and the official guide states there is no
+`data: [DONE]` sentinel — which the relay's terminal boundary already
+synthesizes. The deepseek registry opt-in is removed; the
+`firstByteTimeoutMs` bounded-body fix this RCA shipped remains valid and
+still guards the mechanism's synthetic-fixture path.

devlog/_plan/260807_deepseek_responses_streaming/000_plan.md

@@ -0,0 +1,148 @@
+# DeepSeek V4 Flash Responses upstream streaming re-enable
+
+## Problem
+
+User report: `deepseek/deepseek-v4-flash` through Codex "responds slowly / appears
+unresponsive" since the model moved to the Responses wire. Reproduced live: the
+proxy log shows a 28,387 ms turn (and the essay probe below took 46 s) during
+which the Codex client receives **zero bytes** until the whole generation
+finishes, because the #875 reliability policy forces `stream: false` upstream
+(`modelResponsesUpstreamStreaming: { "deepseek-v4-flash": false }`) and
+synthesizes the entire SSE sequence only after the bounded JSON body arrives.
+
+Correctness is fine — every logged turn is 200, tool calls work end to end via
+`codex exec` — the failure mode is pure perceived latency / no incremental
+output, which reads as a hang for long generations.
+
+## Evidence (fresh, 2026-08-07)
+
+1. Official guide (https://api-docs.deepseek.com/guides/responses_api, fetched
+   today): "Set stream: true to receive the response as a sequence of semantic
+   server-sent events (SSE). … The stream ends with a `response.completed` /
+   `response.incomplete` / `response.failed` event — **there is no `data: [DONE]`
+   message.**" Model `deepseek-v4-flash`; Codex adaptation is explicit; public
+   beta per the 2026-07-31 changelog entry.
+2. Live probe, short turn: HTTP 200, first event at 0.22 s, stream **closed** at
+   0.78 s after `response.completed`. No hang.
+3. Live probe, tool-call replay (the #875 stall scenario — turn 2 after a
+   `function_call_output`): closed at 0.72 s, last events
+   `…output_item.done → response.completed`. No stall after tool results.
+4. Live probe, 1500-word essay: 4149 events, first event 0.23 s, max inter-event
+   gap 0.29 s, `response.completed` at 46.41 s, socket closed 46.42 s. The same
+   turn under today's bounded-JSON policy delivers nothing for ~46 s.
+5. Our own relay already handles the no-`[DONE]` shape:
+   `src/server/relay.ts` (`createSseTerminalOutputBoundary`) treats a Responses
+   terminal event as the protocol boundary and appends the conventional
+   `data: [DONE]` itself when the upstream never sent one (commit 02ca79a37,
+   "close passthrough streams at terminal events"). The WS bridge
+   (`pumpResponsesSseToWebSocket`) likewise terminates on
+   `response.completed|failed|incomplete` and never waits for `[DONE]`.
+
+## Root cause of the original #875 stall (best supported reading)
+
+The 2026-07-31-era DeepSeek Responses beta stream reportedly "delivered output
+without closing on the terminal event". Whatever the historical truth, the
+CURRENT upstream (probed today, including the exact tool-result replay shape
+that stalled) emits the documented terminal and closes the socket. With
+02ca79a37's terminal-boundary relay in place, even a gateway that leaves the
+HTTP connection open after `response.completed` is cut off at the terminal
+block and `[DONE]` is synthesized. The belt-and-suspenders `stream:false`
+force is therefore no longer load-bearing for correctness, but it is now the
+direct cause of the reported UX regression.
+
+## Change map (one work-phase)
+
+- MODIFY `src/providers/registry.ts`
+  - DELETE the `modelResponsesUpstreamStreaming: { "deepseek-v4-flash": false }`
+    line from the deepseek entry (and its comment block), restoring true
+    streaming on the native Responses wire.
+  - KEEP `responsesItemIdRepair`, `responsesPath`, `statelessResponses`,
+    `preserveResponsesReasoningContent`, `supportsServiceTier` untouched.
+  - The `modelResponsesUpstreamStreaming` registry FIELD and its resolver
+    (`providerModelResponsesUpstreamStreaming`) STAY — the mechanism remains
+    available for providers that genuinely need it; only DeepSeek's entry stops
+    using it. Consumers in `src/server/responses/core.ts` short-circuit to
+    `undefined` and become inert for deepseek automatically.
+  - **Reachability disposition (audit round 1, blocker 2):** after the deletion
+    no production registry entry opts in, so the `=== false` branches at
+    core.ts:899 / :2322 / :2349 have no production activator. This is a
+    DELIBERATE retention of a rollback knob, not an oversight: DeepSeek's
+    Responses route is public beta (changelog 2026-07-31), and the #875
+    symptom class returns with a one-line registry re-add if the upstream
+    regresses. Test reachability is preserved by the synthetic-registry
+    fixture below, so the branches stay exercised by the suite even with no
+    production user.
+- MODIFY `tests/deepseek-inbound-wire.test.ts`
+  - Per-test disposition (audit round 1, blockers 2-3 — all eight pinned
+    tests):
+    | Test (current line) | Disposition |
+    |---|---|
+    | WS turn asks bounded JSON upstream (:129) | REWRITE — WS turn keeps `stream:true` upstream |
+    | WS turn keeps plain JSON downstream (:135) | REWRITE — WS turn returns an SSE body (content-type text/event-stream) that index.ts feeds to the WS pump |
+    | HTTP turns use bounded JSON (#875) (:156) | REWRITE — HTTP Responses inbound keeps `stream:true` upstream |
+    | HTTP synthesized terminal SSE (#875) (:164) | REWRITE — upstream SSE (UUID `output_item.added` → deltas → `response.completed`, NO `[DONE]`) relays through with terminal close + synthesized `[DONE]` |
+    | Synthesized-SSE id repair (:250) | MOVE to synthetic-registry fixture (mechanism coverage) |
+    | WS bounded-JSON id repair (:271) | MOVE to synthetic-registry fixture (mechanism coverage) |
+    | No-repair byte-identical bounded JSON (:290) | MOVE to synthetic-registry fixture (generic JSON path) |
+    | Bounded-body size limit (:308) | MOVE to synthetic-registry fixture (generic JSON path) |
+  - NEW streamed #938 integration case: drive `handleResponses` with a mock
+    upstream emitting UUID-bearing `response.output_item.added` + delta +
+    terminal frames WITHOUT `[DONE]`; assert canonical `msg_`/`rs_` ids reach
+    the HTTP SSE client (the relay id-repair path at core.ts:2095, already
+    unit-covered in tests/responses-item-id-repair.test.ts, gets deepseek
+    integration proof).
+  - **Synthetic-registry fixture (concrete, replaces the round-1 "provider
+    override if available" hand-wave):** `PROVIDER_REGISTRY` is an exported
+    mutable array (`src/providers/registry.ts`); the fixture pushes a
+    dedicated entry (`id: "bounded-json-fixture"`, `adapter:
+    "openai-responses"`, distinct baseUrl, `modelResponsesUpstreamStreaming:
+    { "fixture-model": false }`, plus the id-repair policy) in `beforeEach`
+    and pops it in `afterEach`, with a provider config matching the entry's
+    transport so `providerMatchesRegistryTransport` accepts it. The four
+    moved tests run against this fixture, keeping every bounded-JSON branch
+    reachable from the suite.
+- MODIFY `tests/deepseek-responses-item-id-repair.test.ts` (audit round 1,
+  blocker 1 — this file also pins the bounded-JSON contract at :119/:150)
+  - Rewrite its deepseek integration cases around a real streamed SSE
+    upstream (UUID ids in `output_item.added`/`output_item.done` frames, no
+    `[DONE]`), asserting repaired ids in the relayed stream; keep its pure
+    rewrite-unit coverage untouched.
+- MODIFY `structure/04_transports-and-sidecars.md`
+  - Update the DeepSeek bounded-JSON paragraph: policy mechanism remains,
+    deepseek entry no longer opts in; terminal handling is the relay boundary
+    (02ca79a37) + documented `response.completed` terminal.
+- MODIFY `devlog/_plan/260806_overnight_triage_round2/002_issue_1065_rca.md`
+  (audit round 1, minor 4) — append a dated supersession note: the
+  "keep bounded JSON, do not restore streaming" disposition is superseded by
+  this unit (fresh 2026-08-07 upstream probes show terminal-closing streams;
+  the first-byte-deadline fix that RCA shipped remains valid for the
+  synthetic-fixture path).
+
+## Out of scope
+
+- No change to Chat/Anthropic inbound wiring (they stay on /chat/completions).
+- No change to the bounded-body primitive, first-byte deadline, or WS bridge.
+- No change to other providers' `modelResponsesUpstreamStreaming` usage
+  (none exist today — deepseek is the only user — but the field survives).
+
+## Accept criteria
+
+1. `bun run typecheck` clean; `bun run test` green (full suite — shared
+   registry + responses core touched).
+2. Activation evidence (C-ACTIVATION-GROUNDING-01): live `curl` through the
+   running proxy with `stream:true` shows incremental `response.output_text.delta`
+   events arriving BEFORE generation completes (first delta << total time), and
+   the stream closes after `response.completed` + `[DONE]`.
+3. Codex exec end-to-end: a tool-call turn against the live proxy still
+   completes (no stall after function_call_output replay).
+4. The mechanism tests prove bounded-JSON still works when a provider opts in
+   (mechanism not dead).
+
+## Risks
+
+- DeepSeek Responses is public beta; a regression on their side would re-open
+  #875 symptoms. Mitigation: the relay's terminal boundary already defends the
+  no-close case, and the registry knob can be re-enabled in one line.
+- WS path: Codex app connects over WS when available; the WS pump terminates on
+  the terminal event, so live streaming is safe there too (426 fallback to HTTP
+  SSE observed in codex exec runs; both paths covered by tests).

devlog/_plan/260807_deepseek_responses_streaming/010_check_evidence.md

@@ -0,0 +1,38 @@
+# C-phase evidence — DeepSeek Responses streaming re-enable
+
+Commit under test: `13c81cee5` + the `content_part` cross-table id-repair fix.
+
+## Static gates
+
+- `bun run typecheck` — clean (2026-08-07).
+- Focused suites: `deepseek-inbound-wire` 24 pass, `deepseek-responses-item-id-repair`
+  5 pass, `responses-item-id-repair` 5 pass. Full-suite run recorded below.
+
+## Live activation (C-ACTIVATION-GROUNDING-01)
+
+Isolated proxy: `OPENCODEX_HOME=$(mktemp -d)` seeded with only the deepseek
+provider, `bun run src/cli/index.ts start --port 10199` from the patched tree.
+
+1. **Streaming is live again** — 300-word essay probe through the patched proxy:
+   `events=442 deltas=429 first_delta=0.53s terminal=5.85s done=True closed=5.85s`.
+   First token in half a second; the bounded-JSON build would have delivered
+   nothing until ~6 s (and 28-46 s on the turns in the original report).
+2. **Terminal + sentinel** — the relayed stream ends `response.completed` then
+   `data: [DONE]` (synthesized by the relay terminal boundary; upstream sends no
+   sentinel per the official guide).
+3. **#938 stays fixed on the streaming path** — tool-call probe: initial run
+   leaked 13 raw UUID `item_id`s via `response.content_part.*` /
+   `function_call_arguments.*` events (content parts wrap DeepSeek's streamed
+   reasoning, and the static event-type map pointed at the message table only).
+   Fixed with a cross-table fallback in `rewriteItemIdField`; re-probe:
+   `BAD msg/rs UUID leaks: 0`, `function_call call_id preserved:
+   call_00_wzzbHN9Bf0dVvM25aIhn3776` (function_call ids are intentionally
+   untouched). Regression pinned in the streamed #938 test (content_part frame).
+
+## Known pre-existing failure (not this unit)
+
+`tests/jawcode-metadata-sync.test.ts` ("regenerating reproduces the committed
+file byte for byte") fails identically on the parent commit `529646cd7` when the
+jawcode source checkout is present (verified in a clean worktree with
+`JAWCODE_MODELS_JSON` pointed at the sibling checkout; CI skips it without the
+source). Generated-metadata drift predates this unit and is out of scope.

devlog/_plan/260807_untouched_bug_stack/000_plan.md

@@ -0,0 +1,127 @@
+# 260807 — untouched-bug stack: research and roadmap
+
+Base: `codex/260807-stack-base` at `origin/dev@6d04574d0`.
+Cycle: docs-first. This unit writes the plan; no production code changes land here.
+
+## Why this unit exists
+
+A sweep over the 60 open issues and 24 open PRs found two distinct backlogs that
+the merged bug campaign did not reach.
+
+The first is a **CI admission backlog**. Eight bug-fix PRs were reported as
+"never ran CI", which reads like contributor neglect but is not: 524 workflow
+runs sat in `action_required`, waiting on maintainer approval. Thirty-nine of
+them belonged to branches with an open PR. The readiness gate cannot verify the
+`ci` check on a run that was never allowed to start, so those PRs could not
+leave draft no matter what their authors did. Approving the open-PR subset is
+the precondition for every disposition below; approving all 524 is not, because
+most belong to branches already merged or abandoned.
+
+The second is a set of **defects with no PR at all** — issues where a reporter
+filed evidence and nothing was ever opened against it.
+
+## Disposition summary
+
+Every verdict below was reached by reading the diff and the current tree, not
+the PR description.
+
+| Target | Verdict | Reason |
+|---|---|---|
+| #557 npm cache preflight | rewrite | dev is 1,220 commits past the merge base; diff mixes the useful preflight with obsolete recovery machinery |
+| #1095 DeepSeek progressive streaming | rewrite | 2,184-line diff carries an unsafe terminal-repair state machine and a raw-fragment race |
+| #1155 web-search buffered policy | adopt with changes | correct intent; `parseResponse` misuse and a lease leak must be fixed |
+| #1159 Cursor Grok wire prefix | adopt as-is | request-only helper, correctly isolated from discovery |
+| #1171 A6API unlimited quota | adopt as-is | unlimited branch ordered before finite validation, focused coverage |
+| #1163 combo catalog fallback | rewrite | resolver cannot distinguish missing rows from deliberately filtered ones |
+| #1152 account picker selectors | adopt with changes | foundations are sound but the entry point has no production caller |
+| #1169 codex-shim readiness warning | adopt with changes | advisory design is right; the probe can throw and fail a good install |
+| #1131 in-place restart identity | rewrite | 35 files with unresolved lifecycle defects; CI red was a GitHub outage, not the code |
+| #1056 desktop picker (#241) | rewrite | 54-file branch with backup poisoning and lost-alias defects |
+| #1170 unspaced SSE frames | new fix | strict `"data: "` prefix in six parsers |
+| #1100 routed reasoning effort | new fix | routed rows advertise ladders, then lose summary support |
+| #1156 Windows ACL budget | new fix | a complete ACL sequence gets only five seconds |
+
+## Two corrections to the initial triage
+
+Recording these because both changed the plan.
+
+**#1156 was described imprecisely.** The first pass said PR #1135's retry shares
+the 5-second budget. It does not — owner-level recovery at
+`src/codex/native-main-owner.ts:205-210` calls `hardenSecret` again and receives
+a fresh deadline. The real defect is narrower and still real: one complete ACL
+sequence (grant, inheritance, verify, with `/findsid` fallbacks) must finish
+inside a single 5-second envelope. The fix is the envelope size, not the retry
+structure.
+
+**#1170 has six call sites, not one.** The reporter named the OpenAI Chat
+adapter. The same strict prefix also sits in `src/chat/outbound.ts`,
+`src/web-search/parse.ts`, `src/server/claude-messages.ts`, and — twice —
+`src/claude/outbound.ts`, which contains two independent parsers (`:591-605`
+and `:864-865`). The second one was missed on our first pass and found in audit.
+Fixing only the reported site would leave five live paths broken.
+
+## Roadmap
+
+Implementation phases, one decade doc each, one PABCD cycle each:
+
+- `010` — #1170 unspaced SSE field parsing (6 call sites, 2 shared primitives)
+- `020` — #1100 routed reasoning-effort propagation
+- `030` — #1156 Windows ACL harden envelope
+- `040` — #557 replacement: npm cache preflight + log sanitization
+- `050` — adopt-as-is PR replacements (#1159, #1171)
+- `060` — adopt-with-changes PR replacements (#1155, #1152, #1169)
+
+Rewrite-class targets (#1095, #1163, #1131, #1056) are deliberately not in this
+roadmap. Each is a full unit of work with its own defect list, and folding four
+rewrites into this stack would produce a chain no reviewer can follow. They are
+recorded here so the next unit can pick them up with the audit already done.
+
+## Stack shape
+
+Sequential stacked PRs. Each targets `dev` or the previous PR's head branch, per
+the stacked-child workflow that `enforce-target` already supports.
+
+`010` and `020` and `030` touch disjoint production files, so their order is a
+review convenience rather than a dependency. `040` is independent of all three.
+`050` and `060` follow because they replace existing PRs and their originals
+must be closed with a pointer to the replacement.
+
+One real overlap: `040` and `060` both edit lifecycle locale files. Whichever
+lands first, the other rebases.
+
+## Review gates beyond CI
+
+`MAINTAINERS.md` requires explicit security review for credential/permission
+handling and for the dependency-install path. Two phases are in that class and
+cannot go ready on green CI alone:
+
+- `030` — Windows ACL permission handling
+- `040` — npm install path plus log sanitization
+
+Both run `bun run privacy:scan` and request security review before leaving
+draft.
+
+## Out of scope
+
+No promotion to `main` or `preview`, no npm publish, no release tag, and no
+merge. Merging is a separate authorization; this unit stops at open PRs with
+green CI.
+
+## Audit record
+
+This plan failed its first independent audit with six blockers, all corrected
+in place:
+
+1. `010` missed a second parser in `src/claude/outbound.ts` and did not address
+   CRLF framing or multiline `data` joining.
+2. `020` did not specify Record merge semantics; a whole-Record fill-if-undefined
+   would let one user override suppress every registry default.
+3. `030` claimed a ~60s worst case; the real load-time bound is ~90s because
+   `loadConfig()` hardens three paths sequentially (`src/config.ts:1759-1764`).
+4. `040` cited `src/update/job.ts:269-280` as the launcher invocation; that
+   builds the command, and the invocation is at `:1469`.
+5. `060` proposed wiring into a picker-enable transaction that does not exist.
+6. Security-review gates for `030` and `040` were missing.
+
+Recording this because the corrections changed what gets built, not just how it
+is described.