diff --git a/devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md b/devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md index a7a230879..bcdc9a88c 100644 --- a/devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md +++ b/devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md @@ -21,7 +21,7 @@ independent review, blockers, and whether a later phase is authorized. |---|---|---|---|---|---| | CL-00 | `feat/cl-00-compatibility-contracts` | `3ad5bb6bd3f76f6879d84b78ea39edd3e01ec296` | `c014464237fd3c95bda08bc18bfab8ba8f532308` | [#1286](https://github.com/lidge-jun/opencodex/pull/1286) | ACCEPTED AFTER CODERABBIT REMEDIATION (merged to `dev` at `243c3f4905797aa11c62ba933bb03d6d721266fd`) | | CL-01 | `feat/cl-01-conformance-harness` | `c2113ca47b8a05c5a5f90679e4eaa640ca2c6a66` | `22d608c82d82e2746c0cef9cd761db19a8e465ee` | [#1320](https://github.com/lidge-jun/opencodex/pull/1320) | MERGED TO `dev` at `4bb249b756abd468c675d2d92fffe4da95ad3e2a` | -| CL-02 | `feat/cl-02-evidence-ledger` | `4bb249b756abd468c675d2d92fffe4da95ad3e2a` | (phase-2 review fixes in progress) | [draft #1333](https://github.com/lidge-jun/opencodex/pull/1333) | IMPLEMENTATION COMPLETE — PHASE-2 REVIEW FIXES — NOT INDEPENDENTLY ACCEPTED | +| CL-02 | `feat/cl-02-evidence-ledger` | `4bb249b756abd468c675d2d92fffe4da95ad3e2a` | NOT RECORDED | [#1333](https://github.com/lidge-jun/opencodex/pull/1333) | MERGED TO `dev` at `025c37916225dd685d9217e5b40190600f06d278`; POST-MERGE HARDENING [#1343](https://github.com/lidge-jun/opencodex/pull/1343) IN PROGRESS | | CL-03 | — | — | — | — | NOT STARTED | The CL-01 starting SHA is the exact CL-00 tip recorded when CL-01 began. Its @@ -99,6 +99,9 @@ Independent CL-00 acceptance review is frozen at - **Branch:** `feat/cl-02-evidence-ledger` - **Starting/base SHA:** `4bb249b756abd468c675d2d92fffe4da95ad3e2a` (CL-01 merge via #1320) +- **Merged source head:** `1eed4ffbc9772c64f4f22e37869ccb0b9efa90e1` +- **Merged to `dev`:** `025c37916225dd685d9217e5b40190600f06d278` via upstream [#1333](https://github.com/lidge-jun/opencodex/pull/1333). +- **Accepted head:** not recorded. #1333 was merged before an independent-acceptance state was observed in this programme log. - **Scope:** append-only JSONL evidence ledger with an explicit sensitive-purge exception: when the `ledger` purge action is requested, targeted evidence is physically removed by atomic ledger rewrite and a `purge_tombstone` remains as @@ -120,27 +123,25 @@ sensitive purge with shared-artifact retention, recursive event admission ceilings, and unusable-evidence exclusion from projection are implemented. Claims cannot produce `PROBED`/`VERIFIED`. -### CL-02 validation status (2026-08-09 phase-2 review fixes) +### CL-02 validation and post-merge hardening status (2026-08-09) - **Prior accepted review-fix head:** `cf626d14c823413fbcd6ac2625d1da16bbac714e` -- **Phase-2 scope:** eleven independent-review blockers (artifact dirfd I/O, - purge scratch/export + explicit sensitive artifacts, streaming JSONL replay, - zero-applicable UNKNOWN, `newest-required-observation-v1`, multi-surface - applicability, historical manifest no-substitution, closed event admission, - corrupt superseding claims, ArtifactStore lifecycle, frozen behaviour - fingerprint). -- **Previous local validation:** `bun x tsc --noEmit`, `bun run privacy:scan`, - `tests/lab-evidence-ledger.test.ts` (41/41), `tests/lab-conformance-harness.test.ts` - (17/17), `tests/repo-hygiene.test.ts` (11/11), `git diff --check` green on - Windows host before the current CodeRabbit remediation pass. -- **Current CodeRabbit remediation:** committed on draft PR #1333; current CI and - review reconciliation are required before this head may be recorded as accepted. -- **Independent acceptance:** not yet — draft PR #1333 remains open for review. +- **Final #1333 source head:** `1eed4ffbc9772c64f4f22e37869ccb0b9efa90e1` +- **Merge commit:** `025c37916225dd685d9217e5b40190600f06d278` +- A final CodeRabbit review batch arrived immediately before the #1333 merge and + identified additional hardening work in ledger replay, artifact publication, + sensitive purge, event privacy admission, contract-artifact error + classification, conformance execution timestamps, and regression coverage. +- **Post-merge hardening branch:** `fix/cl-02-post-merge-hardening` +- **Post-merge hardening PR:** [#1343](https://github.com/lidge-jun/opencodex/pull/1343), based exactly on merge commit `025c37916225dd685d9217e5b40190600f06d278`. +- The follow-up preserves frozen CL-00 semantics and does not add CL-03 work. +- Current CI/review reconciliation for #1343 must complete before the hardening + follow-up is considered closed. - **CL-03:** not started. ## Authorization - CL-00: **ACCEPTED** (merged #1286). - CL-01: **MERGED** via #1320 at `4bb249b756abd468c675d2d92fffe4da95ad3e2a`. -- CL-02: **IMPLEMENTATION COMPLETE — PHASE-2 REVIEW FIXES — NOT INDEPENDENTLY ACCEPTED** on `feat/cl-02-evidence-ledger` (draft #1333). -- CL-03: **NOT STARTED**. +- CL-02: **MERGED** via #1333 at `025c37916225dd685d9217e5b40190600f06d278`; post-merge hardening is tracked in #1343. +- CL-03: **NOT STARTED** pending completion/reconciliation of the CL-02 post-merge hardening follow-up. diff --git a/devlog/_plan/260807_compatibility_lab/CL02_POST_MERGE_HARDENING.md b/devlog/_plan/260807_compatibility_lab/CL02_POST_MERGE_HARDENING.md new file mode 100644 index 000000000..8216bbf22 --- /dev/null +++ b/devlog/_plan/260807_compatibility_lab/CL02_POST_MERGE_HARDENING.md @@ -0,0 +1,23 @@ +# CL-02 Post-Merge Hardening + +CL-02 merged to `dev` in upstream PR #1333 at merge commit `025c37916225dd685d9217e5b40190600f06d278`. + +A final CodeRabbit review batch arrived immediately before that merge and identified additional post-merge hardening work. This follow-up stays within CL-02 implementation and regression coverage; CL-03 is not started here. + +## Confirmed remediation scope + +- Preserve UTF-8 byte ordering and ownership across chunked ledger replay. +- Bound memory and corruption accounting for oversized unterminated JSONL lines. +- Keep content-addressed artifact publication idempotent under concurrent writers while preserving symlink/hardlink rejection and final digest verification. +- Classify malformed contract artifacts as artifact mismatches rather than generic harness failures. +- Reject embedded raw POSIX filesystem paths in persisted event strings. +- Include `export` in the default sensitive-evidence purge action set. +- Fail closed on unmapped conformance failure classifications. +- Make execution timestamps a typed `ScenarioRunResult` producer output instead of relying on a cast at the CL-02 persistence seam. +- Add focused regressions for each behavior above and reconcile the remaining post-merge review findings without changing frozen CL-00 semantics. + +## Base + +- Upstream base: `dev` +- Base commit: `025c37916225dd685d9217e5b40190600f06d278` +- Follow-up branch: `fix/cl-02-post-merge-hardening` diff --git a/src/lab/artifacts/secure-fs.ts b/src/lab/artifacts/secure-fs.ts index 421d0669b..80cd82d0f 100644 --- a/src/lab/artifacts/secure-fs.ts +++ b/src/lab/artifacts/secure-fs.ts @@ -271,7 +271,12 @@ function isMissingArtifactError(err: unknown): boolean { return isRawMissingError(err) || (err instanceof ArtifactFsError && err.code === "artifact_missing"); } -function assertArtifactTargetCreatable(dir: TrustedArtifactDir, name: string): void { +/** + * Returns true when a clean regular digest target appeared after the caller's + * missing read. The final readback still verifies byte count and digest, so a + * concurrent or hostile wrong-content regular file fails closed. + */ +function artifactTargetAlreadyPublished(dir: TrustedArtifactDir, name: string): boolean { revalidateDir(dir); assertRelativeName(name); try { @@ -280,9 +285,9 @@ function assertArtifactTargetCreatable(dir: TrustedArtifactDir, name: string): v harnessFailure("artifact target is a symbolic link", "artifact_unsafe_target"); } assertRegularFileStats(stats, "artifact create target"); - harnessFailure("artifact target exists but is not reusable", "artifact_unsafe_target"); + return true; } catch (err) { - if (isRawMissingError(err)) return; + if (isRawMissingError(err)) return false; if (err instanceof ArtifactFsError) throw err; harnessFailure( `artifact create target check failed: ${err instanceof Error ? err.message : String(err)}`, @@ -402,9 +407,10 @@ export function putArtifactBytes( if (!isMissingArtifactError(err)) throw err; } - assertArtifactTargetCreatable(dir, digestFileName(digest)); - const tmpName = `.tmp-${digest}-${process.pid}-${Date.now()}.partial`; - writeTempArtifact(dir, tmpName, bytes, digest, artifactBytesDigest); + if (!artifactTargetAlreadyPublished(dir, digestFileName(digest))) { + const tmpName = `.tmp-${digest}-${process.pid}-${Date.now()}.partial`; + writeTempArtifact(dir, tmpName, bytes, digest, artifactBytesDigest); + } return readArtifactBytes(dir, digest, bytes.byteLength); } @@ -429,9 +435,10 @@ export function putNamedDigestBytes( if (!isMissingArtifactError(err)) throw err; } - assertArtifactTargetCreatable(dir, digestFileName(digest)); - const tmpName = `.tmp-${digest}-${process.pid}-${Date.now()}.partial`; - writeTempArtifact(dir, tmpName, bytes, digest, contentDigest); + if (!artifactTargetAlreadyPublished(dir, digestFileName(digest))) { + const tmpName = `.tmp-${digest}-${process.pid}-${Date.now()}.partial`; + writeTempArtifact(dir, tmpName, bytes, digest, contentDigest); + } return readArtifactBytes(dir, digest, { expectedByteCount: bytes.byteLength, contentDigest }); } diff --git a/src/lab/artifacts/store.ts b/src/lab/artifacts/store.ts index 83ad5e118..1a4f07ce0 100644 --- a/src/lab/artifacts/store.ts +++ b/src/lab/artifacts/store.ts @@ -62,10 +62,21 @@ function normalizeReadOptions(value?: number | ArtifactReadOptions): ArtifactRea return typeof value === "number" ? { expectedByteCount: value } : value ?? {}; } +function parseContractJson(bytes: Uint8Array): unknown { + return JSON.parse(new TextDecoder("utf-8", { fatal: true }).decode(bytes)); +} + function jsonDigest( digest: (value: Record) => string, ): (bytes: Uint8Array) => string { - return (bytes) => digest(JSON.parse(new TextDecoder().decode(bytes)) as Record); + return (bytes) => { + try { + return digest(parseContractJson(bytes) as Record); + } catch (err) { + if (err instanceof ArtifactFsError) throw err; + throw new ArtifactFsError("artifact_mismatch", "artifact content is not valid contract JSON"); + } + }; } function digestForArtifactClass(artifactClass: ArtifactClass): (bytes: Uint8Array) => string { @@ -78,8 +89,13 @@ function digestForArtifactClass(artifactClass: ArtifactClass): (bytes: Uint8Arra return jsonDigest(suiteManifestDigest); case "claim_source_manifest": return (bytes) => { - const parsed = JSON.parse(new TextDecoder().decode(bytes)); - return claimSourceManifestDigest(validateClaimSourceManifest(parsed).manifest); + try { + const parsed = parseContractJson(bytes); + return claimSourceManifestDigest(validateClaimSourceManifest(parsed).manifest); + } catch (err) { + if (err instanceof ArtifactFsError) throw err; + throw new ArtifactFsError("artifact_mismatch", "claim-source artifact failed validation"); + } }; default: return artifactBytesDigest; @@ -154,7 +170,16 @@ export function createArtifactStore(artifactsDir: string): ArtifactStore { let stored; if (isContractClass(input.artifactClass)) { const contractClass = input.artifactClass; - const computedDigest = computeContractDigest(contractClass, bytes, redacted); + let computedDigest: string; + try { + computedDigest = computeContractDigest(contractClass, bytes, redacted); + } catch (err) { + if (err instanceof ArtifactFsError) throw err; + throw new ArtifactFsError( + "artifact_mismatch", + err instanceof Error ? err.message : "contract artifact failed validation", + ); + } if (input.expectedDigest !== undefined && computedDigest !== input.expectedDigest) { throw new ArtifactFsError("artifact_mismatch", "contract artifact digest mismatch"); } @@ -211,16 +236,16 @@ function computeContractDigest( return fixtureDigest(bytes); case "scenario_manifest": return scenarioManifestDigest( - typeof redacted === "object" && redacted ? (redacted as Record) : JSON.parse(new TextDecoder().decode(bytes)), + typeof redacted === "object" && redacted ? (redacted as Record) : parseContractJson(bytes) as Record, ); case "suite_manifest": return suiteManifestDigest( - typeof redacted === "object" && redacted ? (redacted as Record) : JSON.parse(new TextDecoder().decode(bytes)), + typeof redacted === "object" && redacted ? (redacted as Record) : parseContractJson(bytes) as Record, ); case "claim_source_manifest": { const parsed = typeof redacted === "object" && redacted ? redacted - : JSON.parse(new TextDecoder().decode(bytes)); + : parseContractJson(bytes); return claimSourceManifestDigest(validateClaimSourceManifest(parsed).manifest); } default: { @@ -266,7 +291,7 @@ export function loadClaimSourceManifest( if (!isSha256Hex(digest)) return { ok: false, manifest: null, corruption: "invalid digest" }; try { const bytes = store.get(digest, { artifactClass: "claim_source_manifest" }); - const parsed = JSON.parse(new TextDecoder().decode(bytes)); + const parsed = parseContractJson(bytes); const { manifest, digest: recomputed } = validateClaimSourceManifest(parsed); if (recomputed !== digest) return { ok: false, manifest, corruption: "claim-source digest mismatch" }; if (manifest.subjectId !== expected.subjectId) return { ok: false, manifest, corruption: "claim-source subjectId mismatch" }; diff --git a/src/lab/conformance/executor.ts b/src/lab/conformance/executor.ts index 68e17b1f5..dee649627 100644 --- a/src/lab/conformance/executor.ts +++ b/src/lab/conformance/executor.ts @@ -676,6 +676,15 @@ export async function executeScenario(caseRecord: CaseRecord): Promise { const diagnostics: string[] = []; const executionContext = resolveProtocolExecutionContext(caseRecord); + const startedAt = Date.now(); + const complete = ( + result: Omit, + ): ScenarioRunResult => ({ + ...result, + startedAt, + completedAt: Math.max(startedAt, Date.now()), + }); + try { const observation = await executeScenario(caseRecord); const assertionResults = evaluateAssertions(caseRecord.assertions, observation); @@ -688,7 +697,7 @@ export async function runScenario(caseRecord: CaseRecord): Promise assertionResults.find((r) => r.id === id)?.passed === true); const expectedFailureMatched = controlPassed && requiredFailures.length === 0; - return { + return complete({ scenarioId: caseRecord.id, suite: caseRecord.suite, passed: expectedFailureMatched, @@ -702,11 +711,11 @@ export async function runScenario(caseRecord: CaseRecord): Promise): PurgeTombstoneEvent { "targetArtifactDigests", { nonEmpty: false, max: MAX_INVALIDATION_TARGETS }, ); - if (targetEventIds.length === 0 && targetArtifactDigests.length === 0) { - throw new LabValidationError("empty_purge_targets", "at least one purge target required"); - } if (raw.reason !== "sensitive_evidence") { throw new LabValidationError("invalid_purge_reason", "reason must be sensitive_evidence"); } @@ -520,6 +517,17 @@ function validatePurge(raw: Record): PurgeTombstoneEvent { throw new LabValidationError("unsorted_purge_actions", "purgeActions must be sorted"); } } + // Directory-scoped actions (scratch/export) are meaningful without event or + // artifact ids — default sensitive purge wipes those trees and still records + // a tombstone. Ledger/sqlite/artifact-only tombstones still need a target. + const hasDirectoryPurge = purgeActions.includes("scratch") || purgeActions.includes("export"); + if ( + targetEventIds.length === 0 + && targetArtifactDigests.length === 0 + && !hasDirectoryPurge + ) { + throw new LabValidationError("empty_purge_targets", "at least one purge target required"); + } return { schemaVersion: LAB_EVENT_SCHEMA_VERSION, eventId: assertString(raw.eventId, "eventId"), diff --git a/src/lab/ledger/purge.ts b/src/lab/ledger/purge.ts index 700221b0e..dd6b95853 100644 --- a/src/lab/ledger/purge.ts +++ b/src/lab/ledger/purge.ts @@ -69,6 +69,7 @@ function atomicRewriteLedger(ledgerPath: string, events: LabEvent[]): void { const bytes = new TextEncoder().encode(body); const parent = dirname(ledgerPath); const tmpPath = join(parent, `.purge-${process.pid}-${Date.now()}.jsonl.tmp`); + let renamed = false; try { const fd = openSync(tmpPath, "wx", 0o600); try { @@ -78,21 +79,35 @@ function atomicRewriteLedger(ledgerPath: string, events: LabEvent[]): void { closeSync(fd); } renameSync(tmpPath, ledgerPath); - if (process.platform !== "win32") { + renamed = true; + } catch (err) { + if (!renamed) { + try { + unlinkSync(tmpPath); + } catch { + // Preserve the original failure. + } + } + throw err; + } + + if (process.platform !== "win32") { + try { const dirFd = openSync(parent, "r"); try { fsyncSync(dirFd); } finally { closeSync(dirFd); } + } catch (err) { + // The rename is already committed and visible. Report durability failure + // without pretending the ledger action can be rolled back. + throw new PurgeError( + "ledger_durability_failed", + `ledger rewrite committed but directory fsync failed: ${err instanceof Error ? err.message : String(err)}`, + ["ledger"], + ); } - } catch (err) { - try { - unlinkSync(tmpPath); - } catch { - // Preserve the original failure. The temp file may already have been renamed. - } - throw err; } } @@ -136,7 +151,7 @@ export function purgeSensitiveEvidence(req: SensitivePurgeRequest): PurgeTombsto const paths = ensureLabDirs(req.configDir); const targetEventIds = [...(req.targetEventIds ?? [])].sort(); const targetArtifactDigests = [...(req.targetArtifactDigests ?? [])].sort(); - const purgeActions = [...(req.purgeActions ?? ["ledger", "sqlite", "artifact", "scratch"])].sort(); + const purgeActions = [...(req.purgeActions ?? PURGE_ACTIONS)].sort(); const explicitSensitive = new Set(targetArtifactDigests); const replay = replayLabLedger(paths.ledgerPath); diff --git a/src/lab/ledger/store.ts b/src/lab/ledger/store.ts index ade8c3e3d..204d96dc5 100644 --- a/src/lab/ledger/store.ts +++ b/src/lab/ledger/store.ts @@ -97,22 +97,6 @@ function processLine( events.push(event); } -function splitIncompleteUtf8Tail(buf: Buffer): { processable: Buffer; remainder: Buffer } { - if (buf.length === 0) return { processable: buf, remainder: Buffer.alloc(0) }; - for (let back = 1; back <= 4 && back <= buf.length; back++) { - const byte = buf[buf.length - back]!; - if ((byte & 0xc0) === 0x80) continue; - const needed = byte >= 0xf0 ? 4 : byte >= 0xe0 ? 3 : byte >= 0xc0 ? 2 : 1; - const seqStart = buf.length - back; - const available = buf.length - seqStart; - if (available < needed) { - return { processable: buf.subarray(0, seqStart), remainder: buf.subarray(seqStart) }; - } - break; - } - return { processable: buf, remainder: Buffer.alloc(0) }; -} - function processBufferedLines( buf: Buffer, state: { @@ -142,7 +126,9 @@ function processBufferedLines( lineNumber: state.lineNumber, detail: `line exceeds ${MAX_SERIALIZED_EVENT_BYTES} bytes without newline`, }); - return { carry: tail, skippingOversizedLine: true }; + // Drop the oversized prefix immediately. Keeping it would defeat the + // replay memory bound and count the same line again at EOF. + return { carry: Buffer.alloc(0), skippingOversizedLine: true }; } return { carry: tail, skippingOversizedLine: false }; } @@ -157,6 +143,8 @@ function processBufferedLines( state.lineNumber += 1; state.totalLineCount += 1; + // Decode only complete JSONL lines. Keeping incomplete line bytes as Buffer + // carry naturally preserves UTF-8 code points split across read chunks. processLine(lineBytes.toString("utf8"), state.lineNumber, true, state.events, state.seenIds, state.corruptions); } @@ -209,16 +197,13 @@ export function replayLabLedger(ledgerPath: string): ReplayResult { if (n <= 0) break; offset += n; + // `chunk` is reused by readSync, so any bytes retained beyond this + // iteration must be detached from it before the next read. const combined = carry.length > 0 ? Buffer.concat([carry, chunk.subarray(0, n)]) - : chunk.subarray(0, n); - const { processable, remainder } = splitIncompleteUtf8Tail(combined); - carry = remainder; - - const result = processBufferedLines(processable, state); - carry = result.carry.length > 0 - ? Buffer.from(Buffer.concat([carry, result.carry])) - : carry; + : Buffer.from(chunk.subarray(0, n)); + const result = processBufferedLines(combined, state); + carry = result.carry.length > 0 ? Buffer.from(result.carry) : Buffer.alloc(0); skippingOversizedLine = result.skippingOversizedLine; state.skippingOversizedLine = skippingOversizedLine; lineNumber = state.lineNumber; diff --git a/src/lab/observe/from-conformance.ts b/src/lab/observe/from-conformance.ts index 5a165439c..606b95b86 100644 --- a/src/lab/observe/from-conformance.ts +++ b/src/lab/observe/from-conformance.ts @@ -33,15 +33,10 @@ import { resolveProtocolExecutionContext } from "../conformance/executor"; const COMPAT_VERSION = "protocol-v1"; -type TimedScenarioRunResult = ScenarioRunResult & { - startedAt?: number; - completedAt?: number; -}; - export interface PersistConformanceOptions { configDir?: string; recordedAt?: number; - /** Actual execution timestamps from the CL-01 runner; never fabricated. */ + /** Optional explicit override for the runner-provided execution timestamps. */ startedAt?: number; completedAt?: number; producerVersion?: string; @@ -154,7 +149,7 @@ function outcomeFromResult(result: ScenarioRunResult): ObservationOutcome { return "fail"; default: { const _never: never = result.classification; - return _never; + throw new Error(`unmapped failure classification: ${String(_never)}`); } } } @@ -163,16 +158,15 @@ function requireExecutionTimes( result: ScenarioRunResult, opts: PersistConformanceOptions, ): { startedAt: number; completedAt: number } { - const timed = result as TimedScenarioRunResult; - const startedAt = opts.startedAt ?? timed.startedAt; - const completedAt = opts.completedAt ?? timed.completedAt; + const startedAt = opts.startedAt ?? result.startedAt; + const completedAt = opts.completedAt ?? result.completedAt; if (!Number.isInteger(startedAt) || !Number.isInteger(completedAt)) { throw new Error("real startedAt/completedAt are required for persisted conformance evidence"); } - if (startedAt! < 0 || completedAt! < startedAt!) { + if (startedAt < 0 || completedAt < startedAt) { throw new Error("invalid persisted conformance execution timestamps"); } - return { startedAt: startedAt!, completedAt: completedAt! }; + return { startedAt, completedAt }; } /** diff --git a/tests/lab-post-merge-hardening.test.ts b/tests/lab-post-merge-hardening.test.ts new file mode 100644 index 000000000..f37d4808a --- /dev/null +++ b/tests/lab-post-merge-hardening.test.ts @@ -0,0 +1,302 @@ +import { afterEach, expect, test } from "bun:test"; +import { existsSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + assignEventId, + jcsStringify, + LabValidationError, + purgeSensitiveEvidence, + replayLabLedger, + subjectIdForSubject, + validateLabEvent, + LAB_EVENT_SCHEMA_VERSION, + LAB_PRODUCER, +} from "../src/lab"; +import { createArtifactStore } from "../src/lab/artifacts/store"; +import { + ArtifactFsError, + closeTrustedArtifactDir, + openTrustedArtifactDir, + putNamedDigestBytes, +} from "../src/lab/artifacts/secure-fs"; +import { suiteManifestDigest } from "../src/lab/digest"; +import { enforceEventStructureLimits } from "../src/lab/events/limits"; +import type { ObservationEvent, ProtocolSubjectV1 } from "../src/lab/events/types"; + +const HOMES: string[] = []; + +function tempHome(): string { + const dir = join(tmpdir(), `ocx-lab-hardening-${process.pid}-${Math.random().toString(16).slice(2)}`); + mkdirSync(dir, { recursive: true, mode: 0o700 }); + HOMES.push(dir); + return dir; +} + +afterEach(() => { + for (const dir of HOMES.splice(0)) { + try { + rmSync(dir, { recursive: true, force: true }); + } catch { + /* ignore */ + } + } + delete process.env.OPENCODEX_HOME; +}); + +function createHashHex(value: string): string { + return Bun.CryptoHasher.hash("sha256", value, "hex"); +} + +function protocolSubject(seed = "hardening"): ProtocolSubjectV1 { + return { + subjectSchemaVersion: 1, + subjectKind: "protocol", + opencodexCompatibilityVersion: "protocol-v1", + effectiveAdapter: "openai-chat", + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + behaviorFingerprint: createHashHex(seed), + }; +} + +function baseObservation(overrides: Partial = {}): ObservationEvent { + const subject = protocolSubject(overrides.scenarioId ?? "hardening"); + const subjectId = subjectIdForSubject(subject); + const fixtureDigest = createHashHex("fixture"); + const scenarioDigest = createHashHex("scenario"); + const suiteDigest = createHashHex("suite"); + return assignEventId({ + schemaVersion: LAB_EVENT_SCHEMA_VERSION, + eventKind: "observation" as const, + recordedAt: 1_700_000_000_000, + producer: LAB_PRODUCER, + producerVersion: "2.10.2", + evidenceLayer: "protocol_conformance" as const, + scenarioId: "responses-core.protocol.request-shape", + scenarioVersion: "1", + scenarioManifestDigest: scenarioDigest, + suiteId: "responses-core", + suiteVersion: "1", + suiteManifestDigest: suiteDigest, + fixtureDigests: [fixtureDigest], + subject, + subjectId, + startedAt: 1_700_000_000_000, + completedAt: 1_700_000_000_100, + executionMode: "fixture" as const, + attempt: 1, + limits: { totalTimeoutMs: 1000 }, + outcome: "pass" as const, + assertions: [ + { + id: "a1", + operator: "equals", + required: true, + passed: true, + expectedSummary: "ok", + observedSummary: "ok", + }, + ], + environment: { runtime: { platform: "test", arch: "x64", bunVersion: "1.0.0" } }, + artifactRefs: [ + { + digest: scenarioDigest, + mediaType: "application/json", + byteCount: 2, + redactionPolicy: "contract_canonical_v1", + relativePath: `${scenarioDigest}.bin`, + artifactClass: "scenario_manifest" as const, + }, + { + digest: suiteDigest, + mediaType: "application/json", + byteCount: 2, + redactionPolicy: "contract_canonical_v1", + relativePath: `${suiteDigest}.bin`, + artifactClass: "suite_manifest" as const, + }, + { + digest: fixtureDigest, + mediaType: "application/json", + byteCount: 2, + redactionPolicy: "contract_canonical_v1", + relativePath: `${fixtureDigest}.bin`, + artifactClass: "fixture" as const, + }, + ], + ...overrides, + }) as ObservationEvent; +} + +test("replay preserves a UTF-8 code point split at the 64 KiB read boundary", () => { + const home = tempHome(); + const ledger = join(home, "compatibility.jsonl"); + const first = baseObservation({ scenarioId: "responses-core.protocol.pad" }); + const second = baseObservation({ + scenarioId: "responses-core.protocol.utf8", + assertions: [ + { + id: "utf8", + operator: "equals", + required: true, + passed: true, + expectedSummary: "café", + observedSummary: "café", + }, + ], + }); + + const firstJson = jcsStringify(first); + const secondLine = `${jcsStringify(second)}\n`; + const secondBytes = Buffer.from(secondLine, "utf8"); + const accentIndex = secondBytes.indexOf(Buffer.from("é", "utf8")); + expect(accentIndex).toBeGreaterThan(0); + const targetFirstLineBytes = 64 * 1024 - accentIndex - 1; + const padBytes = targetFirstLineBytes - Buffer.byteLength(firstJson, "utf8") - 1; + expect(padBytes).toBeGreaterThanOrEqual(0); + const firstLine = `${" ".repeat(padBytes)}${firstJson}\n`; + expect(Buffer.byteLength(firstLine, "utf8") + accentIndex).toBe(64 * 1024 - 1); + + writeFileSync(ledger, `${firstLine}${secondLine}`, "utf8"); + const replay = replayLabLedger(ledger); + expect(replay.corruptions).toEqual([]); + expect(replay.events).toHaveLength(2); + const utf8 = replay.events.find((event) => event.eventId === second.eventId) as ObservationEvent | undefined; + expect(utf8?.assertions[0]?.observedSummary).toBe("café"); +}); + +test("replay discards an oversized unterminated line after reporting it once", () => { + const home = tempHome(); + const ledger = join(home, "compatibility.jsonl"); + writeFileSync(ledger, Buffer.alloc(128 * 1024, 0x61)); + + const replay = replayLabLedger(ledger); + expect(replay.events).toEqual([]); + expect(replay.totalLineCount).toBe(1); + expect(replay.corruptions).toHaveLength(1); + expect(replay.corruptions[0]?.kind).toBe("malformed_line"); +}); + +test("event privacy admission rejects raw POSIX path bypass forms", () => { + for (const detail of [ + "config=/home/alice/work/repo", + "cwd=/usr/local/bin", + "cwd=/tmp", + "cwd=/tmp/", + "cwd=/", + "/", + "cwd=/tmp//secret", + "cwd=/home/@alice", + "cwd=/home/josé/work", + "x-/home/alice", + ]) { + try { + enforceEventStructureLimits({ detail }); + throw new Error(`expected raw_path rejection for ${detail}`); + } catch (err) { + expect((err as { code?: string }).code).toBe("raw_path"); + } + } +}); + +test("invalid JSON contract artifacts classify as artifact_mismatch", () => { + const home = tempHome(); + const artifactsDir = join(home, "artifacts"); + const digest = createHashHex("malformed-suite-manifest"); + const dir = openTrustedArtifactDir(artifactsDir); + try { + putNamedDigestBytes(dir, digest, new TextEncoder().encode("{"), () => digest); + } finally { + closeTrustedArtifactDir(dir); + } + + const store = createArtifactStore(artifactsDir); + try { + expect(() => store.get(digest, { artifactClass: "suite_manifest" })).toThrow(ArtifactFsError); + try { + store.get(digest, { artifactClass: "suite_manifest" }); + throw new Error("expected artifact mismatch"); + } catch (err) { + expect(err).toBeInstanceOf(ArtifactFsError); + expect((err as ArtifactFsError).code).toBe("artifact_mismatch"); + } + } finally { + store.close(); + } +}); + +test("store.put converts malformed contract JSON to artifact_mismatch", () => { + const home = tempHome(); + const store = createArtifactStore(join(home, "artifacts")); + try { + try { + store.put({ + artifactClass: "suite_manifest", + payload: new TextEncoder().encode("{"), + }); + throw new Error("expected store.put to reject malformed contract JSON"); + } catch (err) { + expect(err).toBeInstanceOf(ArtifactFsError); + expect((err as ArtifactFsError).code).toBe("artifact_mismatch"); + } + } finally { + store.close(); + } +}); + +test("contract artifacts reject malformed UTF-8 instead of replacement-decoding it", () => { + const home = tempHome(); + const artifactsDir = join(home, "artifacts"); + const invalidUtf8 = Buffer.concat([ + Buffer.from('{"x":"', "utf8"), + Buffer.from([0x80]), + Buffer.from('"}', "utf8"), + ]); + const digest = suiteManifestDigest({ x: "\uFFFD" }); + const dir = openTrustedArtifactDir(artifactsDir); + try { + putNamedDigestBytes(dir, digest, invalidUtf8, () => digest); + } finally { + closeTrustedArtifactDir(dir); + } + + const store = createArtifactStore(artifactsDir); + try { + try { + store.get(digest, { artifactClass: "suite_manifest" }); + throw new Error("expected malformed UTF-8 artifact mismatch"); + } catch (err) { + expect(err).toBeInstanceOf(ArtifactFsError); + expect((err as ArtifactFsError).code).toBe("artifact_mismatch"); + } + } finally { + store.close(); + } +}); + +test("default sensitive purge removes export evidence", () => { + const home = tempHome(); + const exportDir = join(home, "lab", "export"); + mkdirSync(exportDir, { recursive: true, mode: 0o700 }); + const secret = join(exportDir, "bundle.json"); + writeFileSync(secret, "sensitive export", "utf8"); + + purgeSensitiveEvidence({ configDir: home, recordedAt: 1_700_000_000_500 }); + expect(existsSync(secret)).toBe(false); +}); + +test("purge tombstone without targets still requires a directory-scoped action", () => { + expect(() => validateLabEvent(assignEventId({ + schemaVersion: LAB_EVENT_SCHEMA_VERSION, + eventKind: "purge_tombstone", + recordedAt: 1_700_000_000_500, + producer: LAB_PRODUCER, + producerVersion: "test", + targetEventIds: [], + targetArtifactDigests: [], + reason: "sensitive_evidence", + purgeActions: ["artifact", "ledger", "sqlite"], + }))).toThrow(LabValidationError); +}); diff --git a/tests/lab-post-merge-projection.test.ts b/tests/lab-post-merge-projection.test.ts new file mode 100644 index 000000000..68793e591 --- /dev/null +++ b/tests/lab-post-merge-projection.test.ts @@ -0,0 +1,173 @@ +import { expect, test } from "bun:test"; +import { + assignEventId, + projectVerdicts, + subjectIdForSubject, + LAB_EVENT_SCHEMA_VERSION, + LAB_PRODUCER, +} from "../src/lab"; +import type { SuiteManifestV1 } from "../src/lab/conformance/suite-manifest"; +import type { ObservationEvent, ProtocolSubjectV1 } from "../src/lab/events/types"; +import { evaluateAllApplicableRequiredPassV1 } from "../src/lab/projection/verification"; + +function hash(value: string): string { + return Bun.CryptoHasher.hash("sha256", value, "hex"); +} + +function protocolSubject(seed = "projection"): ProtocolSubjectV1 { + return { + subjectSchemaVersion: 1, + subjectKind: "protocol", + opencodexCompatibilityVersion: "protocol-v1", + effectiveAdapter: "openai-chat", + inboundProtocol: "openai-responses", + upstreamProtocol: "openai-chat", + surface: "responses-http", + behaviorFingerprint: hash(seed), + }; +} + +function observation(overrides: Partial = {}): ObservationEvent { + const { + subject: subjectOverride, + subjectId: _ignoredSubjectId, + ...eventOverrides + } = overrides; + void _ignoredSubjectId; + const subject = subjectOverride?.subjectKind === "protocol" + ? subjectOverride + : protocolSubject(overrides.scenarioId ?? "projection"); + const subjectId = subjectIdForSubject(subject); + const scenarioManifestDigest = eventOverrides.scenarioManifestDigest ?? hash(`scenario:${eventOverrides.scenarioId ?? "required"}`); + const suiteManifestDigest = eventOverrides.suiteManifestDigest ?? hash("suite:projection"); + const fixtureDigest = hash("fixture:projection"); + + return assignEventId({ + schemaVersion: LAB_EVENT_SCHEMA_VERSION, + eventKind: "observation" as const, + recordedAt: 10_000, + producer: LAB_PRODUCER, + producerVersion: "2.10.2", + evidenceLayer: "protocol_conformance" as const, + scenarioId: "responses-core.protocol.required", + scenarioVersion: "1", + scenarioManifestDigest, + suiteId: "responses-core", + suiteVersion: "1", + suiteManifestDigest, + fixtureDigests: [fixtureDigest], + startedAt: 9_900, + completedAt: 10_000, + executionMode: "fixture" as const, + attempt: 1, + limits: { totalTimeoutMs: 1_000 }, + outcome: "pass" as const, + assertions: [ + { + id: "required", + operator: "equals", + required: true, + passed: true, + expectedSummary: "ok", + observedSummary: "ok", + }, + ], + environment: { runtime: { platform: "test", arch: "x64", bunVersion: "1.0.0" } }, + artifactRefs: [ + { + digest: scenarioManifestDigest, + mediaType: "application/json", + byteCount: 2, + redactionPolicy: "contract_canonical_v1", + relativePath: `${scenarioManifestDigest}.bin`, + artifactClass: "scenario_manifest" as const, + }, + { + digest: suiteManifestDigest, + mediaType: "application/json", + byteCount: 2, + redactionPolicy: "contract_canonical_v1", + relativePath: `${suiteManifestDigest}.bin`, + artifactClass: "suite_manifest" as const, + }, + { + digest: fixtureDigest, + mediaType: "application/octet-stream", + byteCount: 2, + redactionPolicy: "contract_canonical_v1", + relativePath: `${fixtureDigest}.bin`, + artifactClass: "fixture" as const, + }, + ], + ...eventOverrides, + subject, + subjectId, + }) as ObservationEvent; +} + +test("verification uses the stricter suite/scenario freshness bound", () => { + const obs = observation(); + const manifest: SuiteManifestV1 = { + schemaVersion: 1, + id: obs.suiteId, + version: obs.suiteVersion, + evidenceLayer: "protocol_conformance", + capability: "responses-core", + assertionDslVersion: "1", + evidenceSchemaVersion: "1", + freshness: { maxAgeMs: 1_000 }, + contradictionRule: "newest-required-observation-v1", + scenarios: [ + { + id: obs.scenarioId, + version: obs.scenarioVersion, + role: "required", + manifestDigest: obs.scenarioManifestDigest, + }, + ], + verificationRule: "all-applicable-required-pass-v1", + }; + const loadScenarioRequirements = () => ({ + inboundProtocols: ["openai-responses"], + upstreamProtocols: ["openai-chat"], + surfaces: ["responses-http"], + freshness: { maxAgeMs: 500 }, + }); + + const atBoundary = evaluateAllApplicableRequiredPassV1(manifest, [obs], "fixture", { + subject: obs.subject as ProtocolSubjectV1, + loadScenarioRequirements, + asOf: obs.completedAt + 500, + }); + expect(atBoundary.canVerify).toBe(true); + expect(atBoundary.notes).not.toContain(`stale_observation:${obs.scenarioId}`); + + const stale = evaluateAllApplicableRequiredPassV1(manifest, [obs], "fixture", { + subject: obs.subject as ProtocolSubjectV1, + loadScenarioRequirements, + asOf: obs.completedAt + 501, + }); + expect(stale.canVerify).toBe(false); + expect(stale.missingRequiredScenarioIds).toEqual([obs.scenarioId]); + expect(stale.notes).toContain(`stale_observation:${obs.scenarioId}`); +}); + +test("matched capability-absence control projects UNSUPPORTED without changing V1 authority", () => { + const obs = observation({ + expectedFailure: { + controlKind: "capability_absence_control", + expectedClass: "capability_failure", + expectedCode: "capability_unavailable", + assertionIds: ["required"], + onMatch: "unsupported", + onMismatch: "fail", + }, + }); + + const projected = projectVerdicts([obs]); + expect(projected.corruptions).toEqual([]); + expect(projected.verdicts).toHaveLength(1); + expect(projected.verdicts[0]!.verdict).toBe("UNSUPPORTED"); + expect(projected.verdicts[0]!.notes).toContain("capability_absence_control"); + expect(projected.verdicts[0]!.contributingEventIds).toContain(obs.eventId); +});