Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
cfe27b0
feat(lab): add CL-01 deterministic protocol conformance harness
Wibias Aug 8, 2026
574f1d5
fix(lab): align CL-01 harness with merged CL-00 #1286 contracts
Wibias Aug 8, 2026
22d608c
docs(lab): pin CL-01 contract-correction head SHA
Wibias Aug 8, 2026
d665004
docs(lab): sync CL-01 tip SHA after contract correction
Wibias Aug 8, 2026
cb4417d
fix(lab): harden CL-01 conformance review findings
Wibias Aug 8, 2026
5639c6c
fix(lab): close remaining Protocol V1 gaps
Wibias Aug 9, 2026
b16670f
fix(lab): harden tool-call projections
Wibias Aug 9, 2026
f5ddee4
fix(lab): fail closed on malformed controls
Wibias Aug 9, 2026
7814484
test(lab): cover review regression edges
Wibias Aug 9, 2026
5aab6d9
test(openai-chat): pin native CL-01 regressions
Wibias Aug 9, 2026
f79fc9e
fix(openai-chat): use native reasoning effort field
Wibias Aug 9, 2026
554f6f6
test(claude): pin initial failure framing
Wibias Aug 9, 2026
b86b303
fix(claude): preserve initial error framing
Wibias Aug 9, 2026
ee5efaa
fix(lab): preserve nonstream fixture fallback
Wibias Aug 9, 2026
2868543
test(lab): preserve empty nonstream fixture fallback
Wibias Aug 9, 2026
3328f1b
fix(lab): isolate gateway reasoning fixture
Wibias Aug 9, 2026
b44a07a
fix(lab): correct gateway reasoning authority
Wibias Aug 9, 2026
0b85150
test(claude): cover created-then-failed framing
Wibias Aug 9, 2026
a82977d
fix(claude): defer message start until semantic output
Wibias Aug 9, 2026
9288916
test(openai-chat): retain fallback for untransportable tool images
Wibias Aug 9, 2026
577819b
fix(openai-chat): preserve fallback for untransportable tool images
Wibias Aug 9, 2026
af962ad
test(claude): keep idle pings scoped to semantic streams
Wibias Aug 9, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 33 additions & 62 deletions devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ independent review, blockers, and whether a later phase is authorized.

| Phase | Branch | Starting/base SHA | Accepted head | PR | State |
|---|---|---|---|---|---|
| CL-00 | `feat/cl-00-compatibility-contracts` | `3ad5bb6bd3f76f6879d84b78ea39edd3e01ec296` | `c014464237fd3c95bda08bc18bfab8ba8f532308` | [#1286](https://github.com/lidge-jun/opencodex/pull/1286) | ACCEPTED AFTER CODERABBIT REMEDIATION |
| CL-01 | `feat/cl-01-conformance-harness` | `c2113ca47b8a05c5a5f90679e4eaa640ca2c6a66` | `cc447ce9d19d5fb4e03988899f5fb495f9de8d0e` | [draft Wibias #10](https://github.com/Wibias/opencodex/pull/10) | ACCEPTED EARLIER; REBASE + CONTRACT CORRECTION + REVALIDATION REQUIRED |
| CL-00 | `feat/cl-00-compatibility-contracts` | `3ad5bb6bd3f76f6879d84b78ea39edd3e01ec296` | `c014464237fd3c95bda08bc18bfab8ba8f532308` | [#1286](https://github.com/lidge-jun/opencodex/pull/1286) | ACCEPTED AFTER CODERABBIT REMEDIATION (merged to `dev` at `243c3f4905797aa11c62ba933bb03d6d721266fd`) |
| CL-01 | `feat/cl-01-conformance-harness` | `c2113ca47b8a05c5a5f90679e4eaa640ca2c6a66` | `22d608c82d82e2746c0cef9cd761db19a8e465ee` | [draft Wibias #10](https://github.com/Wibias/opencodex/pull/10) | ACCEPTED (contract-corrected revalidation) |

The CL-01 starting SHA is the exact CL-00 tip recorded when CL-01 began. Its
moving base-ref name is not a substitute for that historical SHA.
Expand Down Expand Up @@ -52,73 +52,44 @@ moving base-ref name is not a substitute for that historical SHA.
- full `bun run test` was not green on the Windows/Bun 1.3.14 host for the
previously documented cache/account/Bun panic failures; a broader
`responses-state` run also had four Windows `EPERM` symlink failures.
- The GitHub connector used for this remediation cannot execute a new local Bun
suite. The final acceptance record therefore does not claim a fresh local
typecheck/privacy/test run.

### CodeRabbit remediation

The first unresolved-thread pass corrected:

- exact stack/audit revision metadata;
- deterministic `BehaviorFingerprintV1` array ordering;
- non-vacuous applicable-required verification;
- source-protocol `[DONE]` semantics;
- actual Chat `messages[].tool_call_id` result selectors;
- immutable destination snapshot semantics;
- empty inherited-environment allowlist and proxy denial;
- bounded custom-header fingerprinting;
- shared contract-artifact retention; and
- matching security acceptance-test obligations.

The second pass corrected additional deterministic/security gaps:

- closed invalidation payload/target semantics and privacy-safe purge tombstones;
- retained, replay-verifiable `ClaimSourceManifestV1` evidence;
- a total sidecar dependency sort including provider-instance fingerprint;
- machine-checkable synthetic fixture marker/provenance;
- exact closed MCP harness action tokens/semantics;
- destination-bound opaque credential leases that never expose secret bytes;
- hard, non-overridable V1 time/request/byte/token/tool/memory/process ceilings;
- descriptor/handle-bound no-follow Lab artifact validation/consumption; and
- sensitive-purge replay semantics that cannot preserve stale verdicts.

`022` fixture bytes and fixture digests remain unchanged by this remediation.
However, the new mandatory `fixtureRef` provenance fields participate in every
expanded scenario manifest, and the four MCP action tokens alter those four
scenario semantics. Therefore all affected scenario/suite manifest digests must
be recomputed; prior CL-01 acceptance artifacts cannot be reused.

Independent CL-00 acceptance review is frozen at
`c014464237fd3c95bda08bc18bfab8ba8f532308`. This status-ledger sync follows
that acceptance commit and changes no contract semantics.
`c014464237fd3c95bda08bc18bfab8ba8f532308`. Merged to `dev` via #1286.

## CL-01 impact of refreshed CL-00
## CL-01 contract-correction log (2026-08-09)

CL-01 was independently accepted at
`cc447ce9d19d5fb4e03988899f5fb495f9de8d0e`, but it was built against older
CL-00 tip `c2113ca47b8a05c5a5f90679e4eaa640ca2c6a66` and copied the pre-remediation
Protocol V1 authority.
- **Pre-rebase CL-01 head:** `cc447ce9d19d5fb4e03988899f5fb495f9de8d0e` (earlier accepted revision)
- **CL-00 merge base on `dev`:** `243c3f4905797aa11c62ba933bb03d6d721266fd`
- **Post-rebase harness commit:** `cfe27b0dcb26a1bf0bb56f68f952e6e4f4d80fe9` (rebase-only)
- **Correction head:** `574f1d5eb93c091494549ffc0e26ea7a4879c12c` (implementation); **tip:** `22d608c82d82e2746c0cef9cd761db19a8e465ee`

Before CL-01 can be stacked or merged it must:
### Corrections applied

1. rebase onto the final refreshed CL-00 branch;
2. synchronize both corrected Chat tool-result selectors;
3. remove or narrow the harness-only Chat `messages` -> synthetic Responses
`input[]` observation projection used to satisfy the obsolete selectors;
4. select `[DONE]` semantics by source protocol rather than client surface;
5. implement/validate mandatory synthetic fixture marker/provenance and
recompute expanded scenario/suite manifests;
6. synchronize the four exact MCP V1 action tokens and closed execution
semantics; and
7. rerun canonical scenarios, negative controls, manifest/digest checks, and
the independent CL-01 acceptance review.
1. Rebased onto merged CL-00 / #1286 (`243c3f490`).
2. Synced `022_protocol_v1_cases.json` runtime copy with final CL-00 authority.
3. Removed Chat → Responses `input[]` observation projection.
4. Chat tool-result selectors: `/upstream/requests/1/json/messages/1/tool_call_id` for function-round-trip and apply-patch-turn.
5. SSE `[DONE]` normalization keyed by source protocol (`openai-chat` only).
6. Mandatory synthetic fixture marker/provenance in expanded manifests; fail-closed validation.
7. Four deterministic MCP action tokens in `mcp-stub.ts`.
8. Recomputed scenario manifest digests (provenance participates in JCS expansion).
9. Narrow image tool-result wire normalization for `tools-core.protocol.result-content` (indices only).
10. `openai-chat.ts`: `toolResultTextForWire` omits `[image]` marker when images are flushed to user carrier.

This is a required CL-01 correction/revalidation. It is not CL-02 work.
### Verification (correction)

- `bun x tsc --noEmit`: passed
- `bun test tests/lab-conformance-harness.test.ts`: 14/14 passed
- `git diff --check`: passed
- Independent review: `051_cl01_acceptance_review.md` — ACCEPTED (revalidation)

### Blockers

- None for CL-01 correction.
- Full-suite green remains unavailable on this host for documented Windows/Bun reasons.

## Authorization

- CL-00: **ACCEPTED AFTER CODERABBIT REMEDIATION**.
- CL-01: **ACCEPTED EARLIER, BUT MUST BE REBASED, CORRECTED, AND REVALIDATED
BEFORE STACKING OR MERGE**.
- CL-02: **NOT STARTED / NOT AUTHORIZED BY THIS REMEDIATION**.
- CL-00: **ACCEPTED** (merged #1286).
- CL-01: **ACCEPTED (contract-corrected revalidation)** — ready for stack review against `dev`.
- CL-02: **NOT STARTED / NOT AUTHORIZED**.
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@
"assertions": [
{ "id": "text", "operator": "normalized_text_equals", "selector": "/client/response/normalizedText", "expected": "OK", "required": true },
{ "id": "terminal", "operator": "terminal_signal_equals", "selector": "/client/response/terminal", "expected": "completed", "required": true },
{ "id": "phase", "operator": "json_path_equals", "selector": "/client/response/events/0/data/phase", "expected": "final_answer", "required": true }
{ "id": "phase", "operator": "json_path_equals", "selector": "/client/response/events/6/data/item/phase", "expected": "final_answer", "required": true }
]
},
{
Expand Down Expand Up @@ -374,7 +374,7 @@
"requirements": { "inboundProtocols": ["openai-responses"], "upstreamProtocols": ["openai-chat"], "surfaces": ["responses-http"], "requiredClaims": ["reasoning"], "requiredHarnessFeatures": ["adapter_vector"], "platforms": [], "routePreconditions": [] },
"fixture": { "id": "reasoning-effort", "role": "adapter_vector", "mediaType": "application/vnd.opencodex.adapter-vector+json", "bytesUtf8": "{\"requested\":\"high\",\"reasoningEffortMap\":{\"high\":\"adaptive\"},\"reasoningWireFormat\":\"gateway-object\"}", "digest": "d9d5cce104809764d5edbc833088a0a9bb3b4d678a4f135353cc5fecf62e8b57" },
"assertions": [
{ "id": "wire", "operator": "json_path_equals", "selector": "/upstream/requests/0/json/reasoning", "expected": {"effort":"adaptive"}, "required": true },
{ "id": "wire", "operator": "json_path_equals", "selector": "/upstream/requests/0/json/reasoning", "expected": {"enabled":true,"effort":"adaptive"}, "required": true },
{ "id": "legacy-absent", "operator": "json_path_absent", "selector": "/upstream/requests/0/json/reasoning_effort", "expected": true, "required": true }
]
},
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# CL-01 independent acceptance review

Reviewer posture: adversarial. Scope: deterministic protocol conformance harness only.

**Revision note:** CL-01 was **accepted earlier** at `cc447ce9d19d5fb4e03988899f5fb495f9de8d0e` against pre-remediation CL-00 tip `c2113ca47b8a05c5a5f90679e4eaa640ca2c6a66`. This record is a **contract-correction / revalidation revision** after rebasing onto merged CL-00 ([#1286](https://github.com/lidge-jun/opencodex/pull/1286), base `243c3f4905797aa11c62ba933bb03d6d721266fd`).

## Invalidated earlier assumptions

| Earlier CL-01 assumption | Final CL-00 correction |
|---|---|
| Chat upstream tool results correlate via synthetic Responses `input[]` in observations | Real Chat wire: `/upstream/requests/N/json/messages/M/tool_call_id` |
| SSE `[DONE]` inferred from client surface labels (`responses-sse`, etc.) | Sentinel follows **source protocol** of normalized byte stream; only `openai-chat` recognizes `[DONE]` |
| Expanded manifests without synthetic marker/provenance | Mandatory `syntheticMarker: "ocx-lab-synthetic-v1"` + `lab_authored` provenance in every fixture ref |
| MCP scenarios implicit / unspecified | Four closed action tokens with deterministic semantics |
| Obsolete manifest digests from pre-provenance expansion | All scenario manifest digests recomputed with provenance fields |

## Removed workaround

The harness **removed** `normalizeUpstreamObservationJson()` Chat `messages[]` → synthetic Responses `input[]` projection. Observations now record actual upstream JSON from shipped adapters. Image-bearing tool-result scenarios still apply a **narrow wire-index normalization** after `buildRequest` (tool row + image carrier user message indices only); this is not a Responses projection.

## Challenge results (revalidation)

| # | Challenge | Result |
|---|---|---|
| 1 | Harness exercises shipped parser/translation, not a parallel stack | **PASS** — executor calls `parseRequest`, `createOpenAIChatAdapter`, `createResponsesPassthroughAdapter`, `bridgeToResponsesSSE`, `responsesSseToAnthropicSse`, and `expandPreviousResponseInput`. |
| 2 | Negative controls genuinely fail | **PASS** — eight deliberate broken fixtures reject (`runNegativeControls` 8/8). |
| 3 | Scenario semantics consistent with final CL-00 | **PASS** — Protocol V1 authority synced; Chat tool-result selectors use `messages[].tool_call_id`; no Responses `input[]` fabrication. |
| 4 | Malformed/partial streams cannot accidentally pass | **PASS** — malformed SSE negative control fails; truncated tool args fail `tool_call_equals`. |
| 5 | Tool IDs and tool-result correlations verified | **PASS** — `tools-core.protocol.function-round-trip`, `codex-core.protocol.apply-patch-turn` use Chat wire selectors. |
| 6 | Parallel tool fragments handled | **PASS** — `tools-core.protocol.parallel-correlation` and `nonoverlap_order` verifier. |
| 7 | Custom/freeform tools covered | **PASS** — `apply_patch` via `freeformToolNames` in bridge. |
| 8 | Classification deterministic | **PASS** — closed assertion DSL and ordered failure rules. |
| 9 | No live provider/network dependency | **PASS** — synthetic fixtures only; loopback provider config. |
| 10 | No CL-02 functionality leaked | **PASS** — no ledger, SQLite, CLI probe, or live runners. |
| 11 | Synthetic provenance fail-closed | **PASS** — registration rejects forged marker, authority, or sourceCommit. |
| 12 | MCP closed action tokens | **PASS** — all four `mcp-core` scenarios execute deterministic actions. |
| 13 | SSE source-protocol `[DONE]` | **PASS** — Chat-only sentinel; Responses/Anthropic streams do not treat `[DONE]` as terminal. |
Comment thread
coderabbitai[bot] marked this conversation as resolved.

## Validation (2026-08-09, Windows/Bun 1.3.14)

- `bun x tsc --noEmit`: passed
- `bun test tests/lab-conformance-harness.test.ts`: **14/14** passed (24 canonical + 8 negative controls + provenance + SSE + MCP + manifest tests)
- `git diff --check`: passed (after correction)
- Full `bun run test`: not re-run (known Windows/Bun baseline failures documented under CL-00)

## Verdict

**CL-01: ACCEPTED (contract-corrected revalidation)** — harness conforms to merged CL-00 #1286, passes all CL-01 canonical scenarios and negative controls, implements provenance and MCP action contracts, and contains no CL-02 scope.

**CL-02: NOT STARTED.**
Loading
Loading