Skip to content

docs(lab): define compatibility evidence and conformance contracts - #1286

Merged
Wibias merged 20 commits into
lidge-jun:devfrom
Wibias:feat/cl-00-compatibility-contracts
Aug 8, 2026
Merged

docs(lab): define compatibility evidence and conformance contracts#1286
Wibias merged 20 commits into
lidge-jun:devfrom
Wibias:feat/cl-00-compatibility-contracts

Conversation

@Wibias

@Wibias Wibias commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Freeze the Compatibility Lab boundaries across Provider Registry, evidence production, Routing Profiles, Router Intelligence, and future Agent Fabric outcomes.
  • Define immutable evidence/verdict/failure/subject/privacy contracts plus 35 canonical protocol cases across eight initial suites.
  • Convert 21 historical compatibility incidents into provider-neutral regression specifications and record the CL-00 independent acceptance review.

Verification

  • bun run typecheck — passed.
  • bun run privacy:scan — passed.
  • bun test tests/repo-hygiene.test.ts — 11 passed, 0 failed.
  • Focused protocol/compatibility suite — 395 passed, 0 failed across 24 files.
  • Focused continuation-state semantics — 2 passed, 95 filtered, 0 failed.
  • Canonical authority validation — 35 cases, 46 fixture records, eight suites, all fixture digests valid.
  • Local Compatibility Lab link validation and git diff --check — passed.
  • Independent acceptance review — all ten challenges pass; no Critical/High/Medium findings remain.
  • Full bun run testnot green on this Windows/Bun 1.3.14 host. It exited 3 after a cache-invalidation failure, an empty Windows effective-account lookup, and a Bun index out of bounds panic. Serial isolation passed: cache invalidation 6/6; native residue 63 passed with 2 platform skips. A broader focused run had 488 pass and 4 Windows EPERM symlink-creation failures.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Summary by CodeRabbit

  • Documentation
    • Added comprehensive Compatibility Lab architecture and planning documentation.
    • Documented protocol, live-route, and task-effectiveness evidence contracts, including verification, integrity, freshness, and failure rules.
    • Added versioned scenario definitions and 29 deterministic compatibility cases covering streaming, tools, errors, multimodality, reasoning, and structured output.
    • Added 21 historical incident specifications for regression tracking.
    • Documented security, privacy, sandboxing, artifact handling, and retention requirements.
    • Recorded CL-00 as accepted; CL-01 requires corrections before progression.

Freeze the CL-00 evidence, verdict, scenario, identity, privacy, and incident contracts so conformance implementation can proceed without duplicating routing authority.
@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR adds the CL-00 Compatibility Lab plan and normative contracts. It defines evidence, protocol scenarios, manifests, fixture cases, incident specifications, security controls, privacy rules, and acceptance status. CL-00 is accepted. CL-01 remains unauthorized.

Changes

Compatibility Lab CL-00

Layer / File(s) Summary
Architecture and evidence contracts
devlog/_plan/260807_compatibility_lab/000_master_plan.md, devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md
Defines Lab scope, evidence layers, immutable evidence, verdict states, freshness, route identity, behavior fingerprints, task outcomes, and consumer boundaries.
Scenario and protocol authority
devlog/_plan/260807_compatibility_lab/020_scenario_contract_and_catalogue.md, devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md
Defines scenario schemas, execution limits, assertion rules, suite projections, manifest digests, fixture handling, SSE normalization, verifier derivations, and CL-01 limits.
Protocol cases and incident corpus
devlog/_plan/260807_compatibility_lab/022_protocol_v1_cases.json, devlog/_plan/260807_compatibility_lab/030_incident_corpus.md
Adds protocol cases and incident specifications for Responses, Chat, Anthropic, tools, Codex, vision, reasoning, MCP, streaming, errors, and state handling.
Security, privacy, and sandbox controls
devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md
Defines synthetic-only inputs, probe and process restrictions, credential isolation, artifact redaction, privacy-preserving subjects, export rules, retention, and required acceptance tests.
CL-00 status and acceptance record
devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md, devlog/_plan/260807_compatibility_lab/050_cl00_acceptance_review.md
Records stack metadata, verification results, review findings, host-specific test limitations, CL-00 acceptance, and the absence of CL-01 authorization.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

  • lidge-jun/opencodex#350: Covers Responses and Chat SSE completion, replay, and output-item behavior referenced by these compatibility scenarios.

Suggested reviewers: ingwannu, lidge-jun

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the pull request's main change: documenting compatibility evidence and conformance contracts.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Aug 8, 2026
Wibias added 2 commits August 8, 2026 14:51
Record the accepted implementation head and draft pull request so the programme ledger is complete without authorizing CL-01.
Remove Markdown hard-break spaces so the final branch diff passes repository whitespace checks.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 13

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@devlog/_plan/260807_compatibility_lab/000_master_plan.md`:
- Around line 265-269: Update the CL-01 Authorization entry in the phase table
to replace “requires accepted CL-00” with clear grammatical wording such as
“requires CL-00 acceptance” or “requires CL-00 to be accepted,” preserving the
intended acceptance gate.

In
`@devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md`:
- Around line 457-462: Update the dependency ordering contract to define a total
deterministic order for RouteDependencyV1 records. Extend the sort key after
role, provider ID, upstream model ID, and endpoint fingerprint with
clientModelId, effectiveAdapter, upstreamProtocol, and behaviorFingerprint,
using a stable documented ordering; alternatively, explicitly reject duplicate
four-field prefixes.
- Around line 14-17: Define a provider-neutral ProtocolSubjectV1 for
protocol_conformance instead of requiring RouteSubjectV1 provider fields, with
deterministic values for the OpenCodex compatibility version, adapter,
protocols, surface, and behavior fingerprint. Update subject hashing and
projection matching to use this protocol subject, and document explicit omission
or stable synthetic-value rules for route-only fields.

In `@devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md`:
- Around line 82-121: Unify negative-control verdict semantics across all three
sites: in
devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md
lines 82-121, include negative_control scenarios in verificationRule and define
that their expected rejection satisfies the control; in
devlog/_plan/260807_compatibility_lab/020_scenario_contract_and_catalogue.md
lines 177-192, remove or revise the rule allowing a negative control to prove
UNSUPPORTED; and in
devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md
lines 313-320, keep the final rule explicit that expected rejection does not
produce UNSUPPORTED.
- Around line 191-208: The Anthropic SSE parser in the vision path must conform
to the V1 normalization contract: reuse or update the shared SSE normalizer to
join repeated data fields with LF, normalize bare CR, honor explicit event
fields over inferred data.type, treat malformed nonempty JSON as terminal, and
reject [DONE] as a sentinel outside Chat surfaces. Keep data-only type inference
for valid object payloads and ensure padding values emit no event.

In `@devlog/_plan/260807_compatibility_lab/022_protocol_v1_cases.json`:
- Line 225: Restore valid JSON in
devlog/_plan/260807_compatibility_lab/022_protocol_v1_cases.json by replacing
raw line feeds with escaped \n sequences in the arguments values at lines
225-225 and 283-283; preserve the represented multiline tool arguments and all
other case data.

In `@devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md`:
- Line 13: Update the user MCP prohibition in the compatibility lab plan to
remove “by default” and make it an unconditional CL-00 rule: no override or CLI
mode may load user MCP server definitions, resources, results, or credentials.
Keep it consistent with the synthetic-only boundary and the existing statement
that user MCP configuration is not loaded.
- Around line 80-82: Update the environment inheritance policy to reject
HTTP_PROXY, HTTPS_PROXY, and ALL_PROXY by default. If proxy routing is needed,
require separately authorizing the exact proxy endpoint with the same SSRF
validation as other destinations, while keeping proxy credentials bound to their
intended destination.
- Around line 43-44: Update the redirect and DNS security requirements to define
one fail-closed policy, preferably rejecting redirects as established by the
SSRF tests. If redirects remain supported, require each hop to be authorized,
connect only to its validated IP, and preserve the intended Host and SNI values.
- Around line 121-126: Expand the normalized event limits in the security and
privacy plan to include maximum serialized event bytes, aggregate event bytes,
nesting depth, map keys, and array elements. Specify that these bounds must be
validated before normalization buffers each event, alongside the existing
event-count and string-size limits, so oversized or deeply nested structures
cannot consume excessive memory or CPU.
- Around line 149-152: Define explicit maximum retention durations and cleanup
deadlines for scenario/suite manifests, synthetic fixtures, JSONL, SQLite,
artifacts, temporary files, and export staging, rather than retaining referenced
data indefinitely under storage policy. Update the retention language at the
manifest/fixture rules and the storage-policy section, and add a test that
confirms each class is cleaned up after its maximum retention period.
- Around line 222-225: Revise the evidence-handling policy to distinguish
non-sensitive invalid evidence, which may be neutralized by appending an
invalidation, from confirmed sensitive evidence. Make sensitive-evidence purge
mandatory and fail closed: delete it from the local ledger, SQLite, logs,
temporary files, and generated exports, and ensure incident records never
contain the leaked value.
- Around line 50-54: Separate the sidecar’s trusted in-memory connection
destination from its persisted subject identity: represent the raw endpoint only
in the authorization/runtime destination record, and use a keyed opaque endpoint
identity in the composite subject. Update the requirements around the sidecar
authorization and the local subject sections (including lines 179–185) so
subjects never persist or export raw base URLs while the runner still has a
usable connection target.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 8607a26c-f137-4c47-b55f-e692eaed2452

📥 Commits

Reviewing files that changed from the base of the PR and between f5147cb and f2f2837.

📒 Files selected for processing (9)
  • devlog/_plan/260807_compatibility_lab/000_master_plan.md
  • devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md
  • devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md
  • devlog/_plan/260807_compatibility_lab/020_scenario_contract_and_catalogue.md
  • devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md
  • devlog/_plan/260807_compatibility_lab/022_protocol_v1_cases.json
  • devlog/_plan/260807_compatibility_lab/030_incident_corpus.md
  • devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md
  • devlog/_plan/260807_compatibility_lab/050_cl00_acceptance_review.md

Comment thread devlog/_plan/260807_compatibility_lab/000_master_plan.md
Comment thread devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md Outdated
Wibias added 4 commits August 8, 2026 22:32
Freeze layer-specific subjects, claim supersession, closed behavior keys,
scenario-local expectedFailure expansion, and canonical toolCall projections
so CL-01 needs no semantic invention.
Record the accepted delayed-review correction commit on the stack ledger and
acceptance review.
Keep the stack ledger and acceptance review pointed at the delayed-review
contract correction commit rather than chasing tip SHA churn.
Tighten destination/proxy sandbox wording, event bounds, retention and
sensitive purge rules, dependency total order, and CL-01 gate phrasing.
@Wibias
Wibias marked this pull request as ready for review August 8, 2026 20:56

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md (1)

589-596: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Include providerInstanceFingerprint in the dependency sort key.

Lines 590-592 make providerInstanceFingerprint part of RouteDependencyV1. Lines 593-596 omit it from the ordering. Two dependencies can therefore differ only by this fingerprint while comparing equal on every listed sort field. Their input order then changes dependencies[], RouteSubjectV1.subjectId, and derived evidence keys.

Add providerInstanceFingerprint after providerId and test this tie case.

Proposed contract fix
- role, provider ID, upstream model ID, endpoint fingerprint, client model
- ID, effective adapter, upstream protocol, then behavior fingerprint.
+ role, provider ID, provider-instance fingerprint, upstream model ID,
+ endpoint fingerprint, client model ID, effective adapter, upstream protocol,
+ then behavior fingerprint.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md`
around lines 589 - 596, Update the RouteDependencyV1 dependency ordering
contract to include providerInstanceFingerprint immediately after providerId in
the total UTF-8 string comparison key. Add or update coverage for dependencies
differing only by providerInstanceFingerprint, ensuring their sorted order and
derived subject/evidence identifiers are input-order independent.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md`:
- Around line 22-23: Update the CL-01 row so its Base SHA field is explicitly
“not started,” and move the “CL-00 must be accepted first” dependency into the
State or notes field. In the CL-00 row, rename the current implementation-head
field to “Accepted contract head,” or add and maintain a separate current
ledger-head field that is updated after every sync commit, including the
ledger-sync guidance in lines 70-72.

In
`@devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md`:
- Around line 697-700: Update the V1 behavior-fingerprint allowlist and
canonicalization contract to declare an ordering mode and element-comparison
rule for every array-valued key, covering both set-like and order-sensitive
arrays. Make canonicalization require one of these declarations and reject any
undeclared ordering mode, preserving deterministic ProtocolSubjectV1 and
RouteSubjectV1 identity.

In
`@devlog/_plan/260807_compatibility_lab/020_scenario_contract_and_catalogue.md`:
- Around line 197-199: Update the VERIFIED definition in the scenario contract
so it requires at least one applicable required scenario, preventing success
with zero observations. Explicitly define the outcome for required scenarios
that are inapplicable—UNKNOWN or BLOCKED—and align the rule with the
current-observation requirement established in the architecture and evidence
contract.

In `@devlog/_plan/260807_compatibility_lab/022_protocol_v1_cases.json`:
- Around line 218-219: Update the `result` assertions in
`tools-core.protocol.function-round-trip` and
`codex-core.protocol.apply-patch-turn` to target the Chat request’s `/messages`
structure produced by `messagesToChatFormat`, using the tool result’s
`tool_call_id` correlation field. Replace only the invalid `/input/.../call_id`
selectors for these `openai-chat` cases, and preserve `/input` selectors for
Responses cases.
- Around line 147-149: Clarify the sentinel source for the Chat fixtures in the
protocol manifest by either adding an explicit Chat-facing surface to the
affected responses-SSE cases or defining that upstreamProtocols ["openai-chat"]
selects Chat SSE [DONE] handling. Update the manifest authority and dependent
fixtures consistently so the terminal signal resolves to "completed" rather than
failing closed.

In `@devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md`:
- Around line 72-75: Update the environment inheritance policy to enumerate the
exact variable names and permitted values, and require runners to reject every
variable outside that allowlist. Explicitly include lowercase http_proxy,
https_proxy, all_proxy, and no_proxy alongside the uppercase proxy names in the
rejection rules, preserving the requirement that proxy endpoints are separately
authorized under SSRF checks.
- Around line 285-286: Revise the invalidation and secure-deletion requirement
so shared scenario, suite, and fixture artifacts remain available while any
non-invalidated observation references their digest. Delete only non-contract
artifacts, or condition deletion on confirming no valid observation still uses
the artifact, preserving reproducible VERIFIED projections.
- Around line 294-311: Add deterministic acceptance tests covering custom-header
canonicalization and unknown-classification failures, salt-rotation correlation
invalidation and re-projection, destination-record mutation between
authorization and connection, and retention expiry with cleanup retry/failure
and unavailable markers. Extend the existing canary, purge, and network test
coverage, and require these tests to pass before live-runner authorization.
- Around line 41-45: Add an immutability contract to the LabDestinationV1
authorization description: require provider destination/credential plumbing to
create a per-run immutable destination snapshot, preserve it unchanged from
fingerprinting through credential binding and connection, and reject any
attempted mutation.
- Around line 228-237: Add explicit limits for custom-header count, field-name
length, UTF-8 value length, aggregate bytes, and duplicate-value lists before
the JCS construction described in the custom-header fingerprint flow. Reuse the
request-builder limits where applicable or define broker-specific maximums, and
return harness_failure whenever any limit is exceeded before canonicalization or
HMAC generation.

---

Outside diff comments:
In
`@devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md`:
- Around line 589-596: Update the RouteDependencyV1 dependency ordering contract
to include providerInstanceFingerprint immediately after providerId in the total
UTF-8 string comparison key. Add or update coverage for dependencies differing
only by providerInstanceFingerprint, ensuring their sorted order and derived
subject/evidence identifiers are input-order independent.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 91a4fb41-d1dc-4933-b791-6de65bd62fca

📥 Commits

Reviewing files that changed from the base of the PR and between f2f2837 and df4f171.

📒 Files selected for processing (8)
  • devlog/_plan/260807_compatibility_lab/000_master_plan.md
  • devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md
  • devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md
  • devlog/_plan/260807_compatibility_lab/020_scenario_contract_and_catalogue.md
  • devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md
  • devlog/_plan/260807_compatibility_lab/022_protocol_v1_cases.json
  • devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md
  • devlog/_plan/260807_compatibility_lab/050_cl00_acceptance_review.md

Comment thread devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/020_scenario_contract_and_catalogue.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/022_protocol_v1_cases.json
Comment thread devlog/_plan/260807_compatibility_lab/022_protocol_v1_cases.json Outdated
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md`:
- Around line 301-305: Define the post-purge handling in the retention and
sensitive-purge rules: when evidence is removed, invalidate every affected
observation, exclude it from canonical projections, and emit typed unavailable
markers that contain no leaked value. Update the associated retention and purge
tests to cover shared contract artifacts and event lines, ensuring no dangling
references or unreproducible VERIFIED projections remain.
- Around line 243-247: Clarify the canonicalization contract before hashing to
explicitly group header entries by lowercase name into one values array per
name, preserving input order. Ensure every grouped value counts toward the
16-value duplicate limit, while retaining the existing bounds and
harness_failure/no-digest behavior.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 74c168a5-0c82-4b05-988f-60b1cb27cbc3

📥 Commits

Reviewing files that changed from the base of the PR and between df4f171 and 4313fb7.

📒 Files selected for processing (1)
  • devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md

Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md`:
- Line 428: Update the Markdown file containing the CL-00 ingestion
implementation text so it ends with exactly one trailing newline, resolving the
MD047 markdownlint violation without changing its content.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 7792a561-0f6b-4de6-a431-5aa4699c0acf

📥 Commits

Reviewing files that changed from the base of the PR and between 4313fb7 and 3ae5a70.

📒 Files selected for processing (3)
  • devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md
  • devlog/_plan/260807_compatibility_lab/020_scenario_contract_and_catalogue.md
  • devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md

Comment thread devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

♻️ Duplicate comments (1)
devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md (1)

250-251: ⚠️ Potential issue | 🟠 Major

Define one JCS entry per lowercase header name.

Lines 250-251 do not state whether duplicate fields are grouped into one object or emitted as separate objects. Those encodings produce different headers.nonCredentialBehaviorDigest values and route subjects.

Require grouping by lowercase name. Preserve input order inside each values array. Count every grouped value toward the 16-value limit.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md` around
lines 250 - 251, Clarify the header canonicalization step to emit exactly one
JCS entry per lowercase header name, grouping all duplicate fields under that
entry. Preserve the original input order within each values array, and count
every grouped value toward the 16-value limit.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md`:
- Around line 590-597: Update the dependency sort order specification for
RouteDependencyV1 to include providerInstanceFingerprint, preferably immediately
after provider ID, so dependencies differing only by provider instance receive a
deterministic order and duplicate detection remains based on the full key.
- Around line 117-129: Define the invalidation event payload with a required,
non-empty list of target event IDs and a closed set of reason codes. Add
validation that rejects missing or empty targets, duplicate target IDs, unknown
or invalid target references, and unsupported reasons, while preserving
append-only invalidation behavior.
- Around line 183-200: Extend the claim contract around sourceManifestDigest to
define a canonical ClaimSourceManifestV1 object, persist its bytes as a
content-addressed artifact, and retain it while referenced by any claim. During
projection replay, load the retained manifest and validate that its canonical
content digest matches sourceManifestDigest before accepting the historical
CLAIMED result.

In `@devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md`:
- Around line 203-206: Define a closed, scenario-specific action schema for
synthetic_tool covering stub registration, deterministic invocation, and
response generation, including the source of client-visible tool calls. Update
the synthetic_tool harness semantics and the affected MCP cases in
022_protocol_v1_cases.json so each asserted mcpCalls entry is produced by an
explicit action sequence rather than only a tool definition.

In `@devlog/_plan/260807_compatibility_lab/030_incident_corpus.md`:
- Line 390: Update the incident corpus entry’s terminology by replacing “falsey
error payload” with “falsy error payload,” leaving the other listed failure
cases unchanged.

In `@devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md`:
- Around line 306-311: Define the post-purge state for every observation and
canonical projection that references deleted evidence: invalidate affected
observations, remove them from projections, and emit typed unavailable markers
containing no leaked value. Update the sensitive-purge behavior and its tests to
cover shared artifacts and event lines, ensuring no dangling references or
retained VERIFIED projections remain.
- Around line 25-27: Define mandatory machine-checkable marker and provenance
fields for every Lab-authored synthetic fixture, then enforce them during
manifest validation so unmarked or incomplete fixtures are rejected. Update the
required acceptance tests to cover valid synthetic metadata and rejection of
missing or invalid markers, including the scenarios and packaged fixtures
governed by this contract.
- Around line 114-116: Update the V1 resource-limits statement near the run
enforcement requirements to define explicit maximum ceilings for wall-clock,
inactivity, byte, request, token, tool-call, memory, and process resources.
State that scenario-specific limits may only lower these maxima, and preserve
the requirement that inability to enforce any required limit results in
harness_failure.
- Around line 91-96: The credential broker API must expose only a
destination-bound opaque transport capability, keeping selected credential bytes
within request and credential plumbing rather than making them readable by Lab
or probe code. Strengthen the canary test near the existing Lab isolation checks
to verify probe code cannot inspect or serialize the selected credential, while
preserving the existing prohibitions on auth-store and unrelated-credential
access.
- Around line 184-187: Update artifact validation and consumption around
resolveArtifactPath() so validation uses a no-follow opened descriptor rather
than existsSync/statSync followed by a path read. Verify the digest and size
from the bytes read through that same descriptor before consuming them,
preventing symlink traversal and replacement after validation. For newly created
artifacts, use exclusive no-follow creation with flag "wx" or an equivalent
atomic temporary-file-and-rename flow.

---

Duplicate comments:
In `@devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md`:
- Around line 250-251: Clarify the header canonicalization step to emit exactly
one JCS entry per lowercase header name, grouping all duplicate fields under
that entry. Preserve the original input order within each values array, and
count every grouped value toward the 16-value limit.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ac365095-720a-4216-8dc9-887d4e8c8dc9

📥 Commits

Reviewing files that changed from the base of the PR and between f5147cb and 3ae5a70.

📒 Files selected for processing (9)
  • devlog/_plan/260807_compatibility_lab/000_master_plan.md
  • devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md
  • devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md
  • devlog/_plan/260807_compatibility_lab/020_scenario_contract_and_catalogue.md
  • devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md
  • devlog/_plan/260807_compatibility_lab/022_protocol_v1_cases.json
  • devlog/_plan/260807_compatibility_lab/030_incident_corpus.md
  • devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md
  • devlog/_plan/260807_compatibility_lab/050_cl00_acceptance_review.md

Comment thread devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/010_architecture_and_evidence_contract.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/021_protocol_v1_manifest_authority.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/030_incident_corpus.md
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md Outdated
Comment thread devlog/_plan/260807_compatibility_lab/040_security_and_privacy.md Outdated
@Wibias
Wibias merged commit 243c3f4 into lidge-jun:dev Aug 8, 2026
4 checks passed
Wibias added a commit to Wibias/opencodex that referenced this pull request Aug 8, 2026
Rebase onto dev CL-00 merge, remove Chat→Responses observation projection, apply source-protocol SSE [DONE] rules, synthetic provenance, MCP actions, and Chat tool_call_id selectors per final Protocol V1 authority.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant