Skip to content

feat(codex): add security findings workflow - #321

Open
umgbhalla wants to merge 5 commits into
steipete:mainfrom
umgbhalla:feat/codex-findings
Open

feat(codex): add security findings workflow#321
umgbhalla wants to merge 5 commits into
steipete:mainfrom
umgbhalla:feat/codex-findings

Conversation

@umgbhalla

@umgbhalla umgbhalla commented Jul 12, 2026

Copy link
Copy Markdown

Motivation

Oracle previously had no Codex Cloud security-findings workflow. This PR adds the complete path from discovery to action: it can enumerate findings, open a selected finding, extract the report and evidence needed to understand it, and then invoke the finding’s supported detail-panel actions from the CLI.

The first part is deliberately read-only. It turns the authenticated findings surface into structured CLI output instead of forcing an operator to navigate the web UI, while preserving the report sections and evidence links that explain why a finding exists. The second part adds the operational step that was missing before: an explicit finding action command that can create a PR, start a finding chat, close or adjust a finding, and copy the available patch or report material.

Design

Discovery and report extraction

  • oracle codex findings enumerates the paginated findings surface, reports severity and repository metadata, supports severity/limit filtering, and can emit JSON.
  • oracle codex findings --finding <selection-id> opens one finding and extracts its title, repository/commit reference, Summary, Validation, Evidence, and Attack-path analysis sections.
  • Detail extraction also returns deduplicated GitHub evidence links and the signed validation-artifact URL without fetching or modifying the artifact.
  • Finding IDs are validated before they are used to build detail URLs, and the browser runner uses the authenticated browser surface rather than an unauthenticated API shortcut.

Explicit finding actions

  • oracle codex finding --finding <selection-id> --action <action> provides the action layer for create-pr, chat, close, adjust, copy-content, copy-link, copy-patch, and copy-git-apply.
  • The action runner targets the accessible detail-panel controls with trusted CDP mouse events. This is required because the Radix pointer-event controls can ignore synthetic element.click() calls.
  • Chat requires --text, while mutating actions require --confirm; read-only copy actions do not require confirmation.
  • Create PR opens the git action menu and checks for View PR before clicking Create PR, so rerunning the command returns an existing PR instead of creating a duplicate.
  • Actions can be constrained to an expected GitHub repository with --repo, and the finding detail is re-read immediately before the action is dispatched.

Compatibility and safety

The original oracle codex findings command remains read-only, and adding the action command does not make list or detail inspection mutating. The runner refuses mutating actions without explicit confirmation and refuses to proceed when trusted browser input is unavailable, so a failed automation primitive cannot be reported as a successful action.

Validation

  • pnpm exec tsc --noEmit
  • pnpm exec vitest run tests/codex/findings.test.ts — 16 passed
  • pnpm run format:check
  • pnpm run lint
  • pnpm run build
  • pnpm test — 1,572 passed, 44 skipped

Exact-head live proof

Current head: 1b7d6c7cfe42dc2668ccfde8c7f6c57b51b1cbc7

  • Authenticated findings launch returned 33 findings.
  • A selected finding’s report was read successfully, including its evidence links and validation artifact reference.
  • A trusted copy action completed successfully.
  • Repeating Create PR returned that existing PR instead of creating a duplicate.

umgbhalla and others added 2 commits July 12, 2026 21:26
List ChatGPT Codex Cloud security findings and inspect a single finding's
detail sections, over the existing signed-in browser automation.

- list: scrape the SSR'd `li > button` rows in the main world and page
  through with a deny-list-guarded Next-page click; aggregate + dedupe.
- detail: direct-URL nav to `/findings/{id}?sev=` (32-hex validated), then
  read #summary/#validation/#evidence/#attack-path + github /blob/ evidence
  links + the signed validation artifact.

Read-only by construction: the CDP Input domain is never wired, so the
mutating Radix actions (Create PR / Patch / Chat / Close / Adjust / feedback)
are unreachable; the only click path is exact-aria-label Prev/Next paging.
No repo/account/id is ever hardcoded.

Mirrors the project-sources runner for the Chrome lifecycle (manual-login or
cookie-synced temp profile, lease/lock, teardown). Pure parsers live in
src/codex/* with unit tests; browser/action code follows the existing
DOM-eval idiom.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@umgbhalla umgbhalla changed the title feat(codex): add trusted finding actions feat(codex): add security findings workflow Jul 12, 2026
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. labels Jul 12, 2026
@clawsweeper

clawsweeper Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 31, 2026, 5:40 PM ET / 21:40 UTC.

ClawSweeper review

What this changes

The PR adds Oracle CLI commands to read authenticated ChatGPT Codex Cloud security findings and, with explicit confirmation, invoke actions from a finding’s detail panel.

Merge readiness

Blocked until stronger real behavior proof is added - 9 items remain

Keep this PR open: the action-result check has a concrete P1 correctness defect, and the claimed live proof is neither inspectable nor for the current PR head.

Priority: P2
Reviewed head: 840b727b8c6a241da00ff7b4687b81b96c064aab
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🧂 unranked krab (1/6) The feature has substantial implementation work, but a P1 selected-target verification bug and uninspectable stale-head proof make it unsuitable to merge.
Proof confidence 🧂 unranked krab (1/6) Needs stronger real behavior proof before merge: The body claims live proof for commit 1b7d6c7, while the current head is 840b727 and no redacted inspectable transcript, diagnostic output, recording, or artifact is supplied. Add exact-head redacted proof for the selected finding, confirmation/repository guards, and observed result; updating the PR body should trigger a fresh review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Patch quality 🦪 silver shellfish (2/6) Security review found an item that needs attention.

Verification

Check Result Evidence
Real behavior Needs proof Needs stronger real behavior proof before merge: The body claims live proof for commit 1b7d6c7, while the current head is 840b727 and no redacted inspectable transcript, diagnostic output, recording, or artifact is supplied. Add exact-head redacted proof for the selected finding, confirmation/repository guards, and observed result; updating the PR body should trigger a fresh review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed 4 items Create-PR success is not scoped to the selected finding: The action-state reader takes the first GitHub pull-request link from the entire document, and the Create PR postcondition accepts any truthy link without comparing it to the pre-action state or limiting it to the selected detail panel.
No focused action regression coverage: The PR adds only parser/URL tests under tests/codex/findings.test.ts; the browser action-state and postcondition helpers have no focused test references.
PR feature provenance: The action workflow was introduced in the branch commit chain, with the scope-hardening revision following the initial trusted-actions commit.
Findings 1 actionable finding [P1] Scope the PR postcondition to the selected finding
Security Needs attention Create PR outcome accepts an arbitrary page link: A document-wide pull-request link is accepted as successful Create PR completion, so the CLI can misreport an action outcome in an authenticated security-finding workflow.

How this fits together

Oracle’s Codex browser integration launches an authenticated ChatGPT session, reads the Codex Cloud findings interface, and presents data or action results in the CLI. This PR adds a path from CLI flags through a browser-detail panel to read-only report extraction and confirmed security-finding actions.

flowchart LR
  CLI[Oracle CLI command] --> Runner[Authenticated browser runner]
  Runner --> Finding[Selected Codex finding]
  Finding --> Reader[Report and evidence reader]
  Reader --> Output[Structured CLI output]
  Runner --> Guard[Confirmation and repository guard]
  Guard --> Action[Trusted detail-panel action]
  Action --> Result[Verified action result]
Loading

Decision needed

Question Recommendation
Should Oracle expose confirmed state-changing Codex security-finding actions after the selected-finding verification repair, or should this command surface remain read-only until a maintainer explicitly sponsors the operational boundary? Approve a narrow confirmed-action boundary: Repair the finding-local postcondition, add regression coverage, and require exact-head redacted live proof before accepting the new commands.

Why: The implementation introduces a new authenticated capability that can create PRs and alter security-finding state; source review can verify mechanical safeguards but cannot choose the intended product and authorization boundary.

Before merge

  • Add real behavior proof - Needs stronger real behavior proof before merge: The body claims live proof for commit 1b7d6c7, while the current head is 840b727 and no redacted inspectable transcript, diagnostic output, recording, or artifact is supplied. Add exact-head redacted proof for the selected finding, confirmation/repository guards, and observed result; updating the PR body should trigger a fresh review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • Scope the PR postcondition to the selected finding (P1) - The state reader returns the first a[href*="/pull/"] in the whole document, and Create PR treats any such link as completion. An unrelated pull-request link can therefore make a failed action report success; compare against the pre-action state and limit discovery to the selected finding panel.
  • Resolve security concern: Create PR outcome accepts an arbitrary page link - A document-wide pull-request link is accepted as successful Create PR completion, so the CLI can misreport an action outcome in an authenticated security-finding workflow.
  • Resolve merge risk (P1) - An unrelated pull-request link elsewhere in the ChatGPT document can make a failed Create PR action report success, leaving an operator with a false result in a security-finding workflow.
  • Resolve merge risk (P1) - The authenticated, state-changing workflow needs current-head proof that the selected finding, repository guard, confirmation gate, and observed action result remain aligned across the live UI.
  • Complete next step (P2) - A narrow mechanical repair can make Create PR verification finding-local, but contributor-supplied exact-head real behavior proof remains required before merge.
  • Improve patch quality - Bind Create PR completion to a newly observed PR link inside the selected finding panel and add focused regression coverage.
  • Improve patch quality - Post redacted exact-head terminal or browser diagnostics showing the selected finding, guard checks, action, and observed result.

Findings

  • [P1] Scope the PR postcondition to the selected finding — src/browser/actions/codexFindings.ts:420
  • [medium] Create PR outcome accepts an arbitrary page link — src/browser/actions/codexFindings.ts:420
Agent review details

Security

Needs attention: The authenticated action boundary is fail-closed for input availability and confirmation, but Create PR outcome verification is not bound to the selected security finding.

Review metrics

Metric Value Why it matters
Patch scope 1,933 added lines across 10 files The PR adds a new authenticated browser workflow spanning CLI parsing, browser lifecycle, action dispatch, and output formatting.
Focused action tests 0 added The added test file covers parsing and URL helpers, not the new browser action-result verification path.

Merge-risk options

Maintainer options:

  1. Bind action success to the selected finding (recommended)
    Require a newly observed, finding-panel-local PR link and regression coverage before merge, then obtain redacted exact-head proof for the authenticated flow.
  2. Pause the action layer
    Keep the read-only findings commands but defer mutating actions until a maintainer confirms the desired security-operational boundary.

Technical review

Best possible solution:

Keep read-only discovery intact, but merge confirmed actions only after the Create PR result is tied to a newly observed link in the selected finding panel, covered by regression tests, and demonstrated with redacted exact-head live evidence.

Do we have a high-confidence way to reproduce the issue?

Yes, from source: any pre-existing unrelated GitHub pull-request link in the document satisfies the current Create PR success condition because it is read globally and not compared with the pre-action value.

Is this the best way to solve the issue?

No: the current result check is not the safest solution because it trusts an arbitrary document-wide pull-request link; it should observe a new link scoped to the selected finding’s action panel.

Full review comments:

  • [P1] Scope the PR postcondition to the selected finding — src/browser/actions/codexFindings.ts:420
    The state reader returns the first a[href*="/pull/"] in the whole document, and Create PR treats any such link as completion. An unrelated pull-request link can therefore make a failed action report success; compare against the pre-action state and limit discovery to the selected finding panel.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.98

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 68b8c51b0ee0.

Labels

Label justifications:

  • P2: This is a substantial new operator workflow with a bounded pre-merge defect, rather than a confirmed user-facing production outage.
  • merge-risk: 🚨 security-boundary: The PR automates authenticated actions on Codex security findings, so selected-target verification and confirmed outcome reporting must fail closed.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦪 silver shellfish.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The body claims live proof for commit 1b7d6c7, while the current head is 840b727 and no redacted inspectable transcript, diagnostic output, recording, or artifact is supplied. Add exact-head redacted proof for the selected finding, confirmation/repository guards, and observed result; updating the PR body should trigger a fresh review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence

Security concerns:

  • [medium] Create PR outcome accepts an arbitrary page link — src/browser/actions/codexFindings.ts:420
    A document-wide pull-request link is accepted as successful Create PR completion, so the CLI can misreport an action outcome in an authenticated security-finding workflow.
    Confidence: 0.98

Acceptance criteria:

  • [P1] pnpm exec vitest run tests/codex/findings.test.ts tests/browser/codexFindings.test.ts.
  • [P1] pnpm exec tsc --noEmit.
  • [P1] pnpm run lint.
  • [P1] pnpm run format:check.

What I checked:

  • Create-PR success is not scoped to the selected finding: The action-state reader takes the first GitHub pull-request link from the entire document, and the Create PR postcondition accepts any truthy link without comparing it to the pre-action state or limiting it to the selected detail panel. (src/browser/actions/codexFindings.ts:420, 840b727b8c6a)
  • No focused action regression coverage: The PR adds only parser/URL tests under tests/codex/findings.test.ts; the browser action-state and postcondition helpers have no focused test references. (tests/codex/findings.test.ts:1, 840b727b8c6a)
  • PR feature provenance: The action workflow was introduced in the branch commit chain, with the scope-hardening revision following the initial trusted-actions commit. (src/browser/codexFindingsRunner.ts:243, 840b727b8c6a)
  • Adjacent browser-action ownership: Current-main history attributes the trusted CDP click precedent in the adjacent project-sources action path to Peter Steinberger; repository history also shows him as the dominant contributor to the browser-action area. (src/browser/actions/projectSources.ts:294, 5daa6ce8c352)

Likely related people:

  • Peter Steinberger: Current-main blame attributes the adjacent trusted CDP click implementation to this author, and repository shortlog history shows sustained ownership of the browser-action surface. (role: recent browser automation contributor; confidence: high; commits: 5daa6ce8c352; files: src/browser/actions/projectSources.ts, src/browser/chromeLifecycle.ts, src/browser/config.ts)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (13 earlier review cycles; latest 8 shown)
  • reviewed 2026-07-30T15:22:09.415Z sha 840b727 :: needs real behavior proof before merge. :: [P1] Require a newly observed finding-local PR link | [P2] Confirm clipboard copy before reporting success
  • reviewed 2026-07-31T00:02:35.630Z sha 840b727 :: needs real behavior proof before merge. :: [P2] Remove the release-owned changelog entry
  • reviewed 2026-07-31T01:50:07.935Z sha 840b727 :: needs real behavior proof before merge. :: [P1] Require a newly observed finding-local PR link | [P2] Confirm clipboard copy before reporting success
  • reviewed 2026-07-31T05:24:06.539Z sha 840b727 :: needs real behavior proof before merge. :: [P1] Verify that an existing PR belongs to this finding | [P2] Confirm clipboard copy before reporting success
  • reviewed 2026-07-31T11:27:26.885Z sha 840b727 :: needs real behavior proof before merge. :: [P1] Verify the created PR is local to the selected finding | [P2] Confirm clipboard completion before reporting success
  • reviewed 2026-07-31T14:27:44.704Z sha 840b727 :: needs real behavior proof before merge. :: none
  • reviewed 2026-07-31T18:43:23.548Z sha 840b727 :: needs real behavior proof before merge. :: none
  • reviewed 2026-07-31T20:16:04.323Z sha 840b727 :: needs real behavior proof before merge. :: none

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. labels Jul 12, 2026
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal priority bug or improvement with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant