lazycodex-executor-verify accepts false completion from generic receipt files

[napadayte]

Summary

$start-work can accept false completion from lazycodex-executor because the shipped SubagentStop evidence gate treats any non-empty file under .omo/evidence/ as sufficient proof. In practice this allows downstream todos to be marked complete even when the task's required deliverable is missing or stale.

Environment

• LazyCodex version: latest main source synced on 2026-07-08 at 722ebec; installed OMO plugin path observed as .../sisyphuslabs/omo/4.16.0/...
• Codex version: Codex Desktop thread (exact app build not exposed in this session)
• OS: macOS
• Install method: local LazyCodex/OMO plugin install
• Relevant config: orchestration through $start-work, Boulder state in .omo/boulder.json, SubagentStop hook subagent-stop-verifying-lazycodex-executor-evidence.json

Repository Decision

• Target repository: code-yeongyu/lazycodex
• Why this belongs there: the failing behavior is implemented in LazyCodex/OMO's orchestration layer, specifically the lazycodex-executor-verify hook and the start-work / continuation contract. Upstream Codex does not ship boulder.json, lazycodex-executor-verify, or the EVIDENCE_RECORDED stop-hook protocol.
• LazyCodex evidence (runtime + /var/folders/tx/mt3tgbtj0cx7xhsh1yj4ft200000gn/T/lazycodex-sources/lazycodex-source):
  • Docs and hard rules promise artifact-backed completion and distrust worker self-reports:
    • plugins/omo/components/start-work-continuation/directive.md: "Treat every worker DoneClaim as untrusted input" and "confirmed is the only pass verdict"
    • plugins/omo/scripts/sync-skills.mjs: "No completion claim while an applicable ultraqa adversarial class was never probed" and no final completion before the Global Review and Debugging Gate
  • Shipped verifier only checks for EVIDENCE_RECORDED: <path> pointing to any non-empty file inside .omo/evidence/:
    • plugins/omo/components/lazycodex-executor-verify/src/codex-hook.ts
    • component overview at plugins/omo/components/lazycodex-executor-verify/AGENTS.md
  • Fresh runtime repro against the shipped CLI:
    1. create .omo/evidence/receipt.txt with placeholder text
    2. omit the expected deliverable entirely (for example .omo/notepads/opencode-facts.md)
    3. run node plugins/omo/components/lazycodex-executor-verify/dist/cli.js hook subagent-stop with last_assistant_message containing EVIDENCE_RECORDED: .omo/evidence/receipt.txt
    4. observed result: hook stdout is empty, so the stop passes even though the deliverable is absent
• Upstream Codex source evidence from /var/folders/tx/mt3tgbtj0cx7xhsh1yj4ft200000gn/T/lazycodex-sources/openai-codex-source:
  • no lazycodex-executor-verify
  • no .omo/boulder.json orchestration layer
  • no EVIDENCE_RECORDED stop-hook contract

Reproduction

1. Run a $start-work plan whose todo requires a concrete deliverable file, for example a fact list under .omo/notepads/....
2. Let a lazycodex-executor child stop with last_assistant_message ending in EVIDENCE_RECORDED: .omo/evidence/receipt.txt.
3. Make sure .omo/evidence/receipt.txt exists and is non-empty, but do not create the actual deliverable.
4. Observe that the SubagentStop verifier passes and the run can continue as if the step were evidenced.

Minimal local CLI repro used for confirmation:

mkdir -p "$TMP/.omo/evidence"
printf 'placeholder evidence\n' > "$TMP/.omo/evidence/receipt.txt"
# intentionally do NOT create the expected deliverable
node plugins/omo/components/lazycodex-executor-verify/dist/cli.js hook subagent-stop < payload.json

Observed on 2026-07-08:

• expected deliverable absent: .omo/notepads/opencode-facts.md
• hook stdout: empty

Expected Behavior

The verifier should keep blocking unless the recorded evidence proves the task-specific deliverable exists and matches the executor's declared outcome. A generic non-empty receipt file should not be enough to close the step.

Actual Behavior

The verifier accepts any non-empty file under .omo/evidence/ referenced by EVIDENCE_RECORDED: <path>, regardless of whether the actual deliverable exists. This creates false-positive completion and allows downstream steps to consume missing or stale outputs.

Evidence

• Real orchestration review from a live run found:
  • a todo requiring .omo/notepads/.../opencode-facts.md marked complete while that file did not exist
  • a later todo marked complete while the expected HTML hero tagline was still unchanged
• Runtime repro on 2026-07-08 against plugins/omo/components/lazycodex-executor-verify/dist/cli.js:
  • placeholder receipt present and non-empty
  • expected deliverable absent
  • CLI returned empty stdout, which is the verifier's pass condition
• Source inspection:
  • src/codex-hook.ts only validates receipt path containment and non-empty file size
  • AGENTS.md explicitly describes the valid receipt as any non-empty file inside .omo/evidence/

Root Cause

Confirmed cause: the current lazycodex-executor-verify gate validates only the existence of a non-empty receipt file inside .omo/evidence/. It does not validate task-specific deliverables, semantic verifier output, or a confirmed verdict tied to the executor's declared scope. That is weaker than the documented $start-work contract, so false completion is possible by design in the current implementation.

Proposed Fix

Strengthen the executor evidence contract so the hook or the next gate validates deliverable-specific evidence rather than a generic receipt:

• require a structured evidence payload that includes the expected deliverable path(s) and verifier verdict
• reject completion if the referenced deliverable is missing, empty, outside declared scope, or not mentioned by the verifier
• keep the current .omo/evidence/ path safety checks, but make them necessary rather than sufficient
• add a regression test where EVIDENCE_RECORDED points to a valid non-empty receipt while the task deliverable is absent; expected result should be decision: "block"

Likely files/components:

• plugins/omo/components/lazycodex-executor-verify/src/codex-hook.ts
• plugins/omo/components/lazycodex-executor-verify/test/codex-hook.test.ts
• possibly the executor-side evidence format in plugins/omo/components/ultrawork/agents/lazycodex-executor.toml
• any start-work / continuation logic that currently treats receipt existence as equivalent to confirmed

Verification Plan

• Reproduce the current false-positive case with a non-empty receipt and a missing deliverable; confirm it blocks after the fix
• Reproduce a valid executor stop with a real deliverable and evidence payload; confirm it passes
• Regression check that symlink/traversal/zero-byte receipt protections still hold
• Regression check that $start-work cannot advance a todo when the required deliverable path is absent

---

This issue or PR was generated by LazyCodex.
Tag: lazycodex-generated