Skip to content

feat(api): add GPT-5.6 Pro reasoning controls - #335

Open
enki wants to merge 2 commits into
steipete:mainfrom
enki:feat/gpt-5.6-pro-mode-pr
Open

feat(api): add GPT-5.6 Pro reasoning controls#335
enki wants to merge 2 commits into
steipete:mainfrom
enki:feat/gpt-5.6-pro-mode-pr

Conversation

@enki

@enki enki commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add --reasoning-mode standard|pro for GPT-5.6 Responses API runs
  • add --reasoning-effort none|low|medium|high|xhigh|max for the complete GPT-5.6 effort set
  • send Pro + max as reasoning: { mode: "pro", effort: "max" } without inventing a model slug
  • persist both controls across detached sessions and restarts, with Pro timeout/background defaults
  • reject unsupported models, unsupported mode routes, browser usage, and fake Pro model slugs
  • document the API/browser distinction and cover request, CLI, persistence, configuration, and live-smoke paths

Why

GPT-5.6 Pro is a Responses API reasoning mode, not a separate model slug. Oracle previously exposed a fixed reasoning effort but had no CLI path for Pro mode or GPT-5.6's new none and max effort levels. Inputs such as gpt-5.6-sol-pro could also fall through model normalization instead of failing with actionable guidance.

Mode and effort are independent: standard and pro are the complete GPT-5.6 mode enum, while none, low, medium, high, xhigh, and max are the complete GPT-5.6 effort enum. The OpenAI reasoning-mode guide documents this request shape.

Validation

After rebasing onto main at 68b8c51b:

  • full suite: 1,625 passed, 43 opt-in tests skipped
  • focused reasoning, detachment, engine, timeout, background, and model-resolution tests: 166 passed
  • formatting, lint, and typecheck passed
  • live smoke now reserves 25,000 output/reasoning tokens so Pro + max can complete instead of being constrained by a 64-token budget
  • dedicated real OpenAI GPT-5.6 Sol Pro + max background smoke passed
  • CI passed on Ubuntu, macOS, Windows, CDP disconnect proof, and GitGuardian

Live API proof

Validated against the real OpenAI Responses API on rebased head b291cdf4:

ORACLE_LIVE_TEST=1 pnpm vitest run tests/live/openai-live.test.ts \
  -t "gpt-5.6-sol Pro reasoning mode background flow eventually completes"

Test Files  1 passed (1)
Tests       1 passed | 4 skipped (5)
Duration    7.08s

The synthetic smoke completed through Oracle's background create/retrieve path with model: "gpt-5.6-sol" and reasoning: { mode: "pro", effort: "max" }. No API key or private runtime data is included.

@enki
enki force-pushed the feat/gpt-5.6-pro-mode-pr branch from 405df91 to 3d4e484 Compare July 23, 2026 12:50
@enki enki changed the title feat(api): add GPT-5.6 Pro reasoning mode feat(api): add GPT-5.6 Pro reasoning controls Jul 23, 2026
@enki
enki marked this pull request as ready for review July 23, 2026 22:26
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. labels Jul 30, 2026
@clawsweeper

clawsweeper Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs changes before merge. Reviewed July 31, 2026, 5:40 PM ET / 21:40 UTC.

ClawSweeper review

What this changes

Adds GPT-5.6 Responses API reasoning-mode and reasoning-effort CLI controls, session persistence, API-route validation, documentation, and automated coverage.

Merge readiness

⚠️ Needs maintainer review before merge - 4 items remain

Keep this PR open: its terminal live-API proof is sufficient and current main does not expose these API controls, but the unchanged head still validates reasoning options against the selected alias before user apiModel overrides determine the on-wire model. That can bypass the PR's GPT-5.6-only guard for persisted overrides and fail at the provider instead of giving local guidance.

Priority: P2
Reviewed head: b291cdf4a422a8faab0f6de0691d786191f0a740

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) Strong real API evidence supports the feature, but an unresolved P1 compatibility defect prevents merge readiness.
Proof confidence 🦞 diamond lobster (5/6) Sufficient (terminal): The PR body includes redacted terminal evidence of an after-fix real OpenAI Responses API background run completing with GPT-5.6 Sol, Pro mode, and max effort.
Patch quality 🦐 gold shrimp (3/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): The PR body includes redacted terminal evidence of an after-fix real OpenAI Responses API background run completing with GPT-5.6 Sol, Pro mode, and max effort.
Evidence reviewed 6 items Validation precedes effective-model resolution: The PR checks GPT-5.6 eligibility from options.model at line 129, then resolves overridden model configuration at line 254 and replaces the dispatched model ID at lines 324–344. A gpt-5.6-sol alias overridden to a non-GPT-5.6 API model therefore passes local validation before the incompatible request is built.
Persisted override contract: Current main supports user-config-only overrides that replace the on-wire apiModel; the resolver applies that override for known model keys. This is the configuration path the new validation must account for.
Existing negative tests do not cover overrides: The added tests reject a direct gpt-5.5 selection, but no test sets a GPT-5.6 alias plus persisted modelOverrides.apiModel to an incompatible effective model.
Findings 1 actionable finding [P1] Validate reasoning options against the resolved model
Security None None.

How this fits together

Oracle accepts CLI and persisted session options, resolves a provider route and model configuration, then sends a Responses API request and optionally tracks its background completion. User-level model overrides can replace the on-wire API model after the CLI alias has been selected.

flowchart LR
  A[CLI flags and saved session options] --> B[Run options]
  B --> C[Provider route]
  B --> D[Model configuration and user overrides]
  C --> E[Reasoning option validation]
  D --> E
  E --> F[Responses API request]
  F --> G[Background response session]
Loading

Before merge

  • Validate reasoning options against the resolved model (P1) - The GPT-5.6 check uses the requested alias before modelOverrides is resolved, while the request later replaces the on-wire model ID. A persisted override from gpt-5.6-sol to a non-GPT-5.6 target therefore sends Pro/max controls that this validation is meant to block. Validate after resolution and add the negative persisted-config case. This repeats the prior blocker on the unchanged head.
  • Resolve merge risk (P1) - A persisted modelOverrides.apiModel value can redirect a GPT-5.6 alias to an incompatible on-wire model after local reasoning validation, causing a provider-side failure for existing configured users.
  • Resolve merge risk (P1) - The PR adds public CLI controls and session persistence across 24 files; its intended API-only compatibility boundary needs the effective-model regression case before merge.
  • Complete next step (P2) - The remaining blocker is a narrow, mechanical validation-order repair on this PR branch with a clear regression test target.

Findings

  • [P1] Validate reasoning options against the resolved model — src/oracle/run.ts:129-140
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Patch surface 24 files affected; 525 added, 27 removed The change spans CLI parsing, persistence, request construction, docs, and test coverage, so the override boundary is merge-critical.
Live proof 1 real Responses API background smoke The PR body shows an after-fix GPT-5.6 Sol Pro/max run completing against the real API.

Merge-risk options

Maintainer options:

  1. Validate the dispatched model before sending reasoning options (recommended)
    Move or repeat the GPT-5.6 eligibility check after model-override resolution and add a persisted override case that proves incompatible effective models fail locally.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Validate reasoning controls after model override resolution, reject incompatible effective API model IDs before dispatch, and add a persisted modelOverrides regression test.

Technical review

Best possible solution:

Resolve the effective API model before enforcing the GPT-5.6 reasoning contract, preserve supported gateway behavior, and add a persisted-override regression test that rejects incompatible on-wire targets before any request is sent.

Do we have a high-confidence way to reproduce the issue?

Yes—source inspection gives a high-confidence path: run gpt-5.6-sol with a persisted modelOverrides entry that changes its apiModel to a non-GPT-5.6 target and request Pro or max reasoning. The current ordering accepts the alias before constructing a request for the overridden target; no write-capable test execution was performed in this read-only review.

Is this the best way to solve the issue?

No—the current branch is not yet the safest implementation because its GPT-5.6 validation runs before the documented effective-model override path. Validating the resolved dispatch target and covering that persisted-config case is the narrow maintainable repair.

Full review comments:

  • [P1] Validate reasoning options against the resolved model — src/oracle/run.ts:129-140
    The GPT-5.6 check uses the requested alias before modelOverrides is resolved, while the request later replaces the on-wire model ID. A persisted override from gpt-5.6-sol to a non-GPT-5.6 target therefore sends Pro/max controls that this validation is meant to block. Validate after resolution and add the negative persisted-config case. This repeats the prior blocker on the unchanged head.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.98

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 68b8c51b0ee0.

Labels

Label justifications:

  • P2: The P1 patch defect affects configured override users but has a bounded blast radius rather than an active core-runtime outage.
  • merge-risk: 🚨 compatibility: Existing user-level apiModel overrides can change the dispatched model after the new alias-only validation succeeds.
  • merge-risk: 🚨 auth-provider: The unresolved path sends reasoning controls to an effective provider model that the local contract intends to reject.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (terminal): The PR body includes redacted terminal evidence of an after-fix real OpenAI Responses API background run completing with GPT-5.6 Sol, Pro mode, and max effort.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes redacted terminal evidence of an after-fix real OpenAI Responses API background run completing with GPT-5.6 Sol, Pro mode, and max effort.

Evidence

Acceptance criteria:

  • [P1] pnpm vitest run tests/cli/runOracle/runOracle.request-payload.test.ts.
  • [P1] pnpm vitest run tests/oracle/modelResolver.override.test.ts.
  • [P1] pnpm vitest run tests/cli/integrationCli.test.ts.

What I checked:

  • Validation precedes effective-model resolution: The PR checks GPT-5.6 eligibility from options.model at line 129, then resolves overridden model configuration at line 254 and replaces the dispatched model ID at lines 324–344. A gpt-5.6-sol alias overridden to a non-GPT-5.6 API model therefore passes local validation before the incompatible request is built. (src/oracle/run.ts:129, b291cdf4a422)
  • Persisted override contract: Current main supports user-config-only overrides that replace the on-wire apiModel; the resolver applies that override for known model keys. This is the configuration path the new validation must account for. (src/oracle/modelResolver.ts:277, 68b8c51b0ee0)
  • Existing negative tests do not cover overrides: The added tests reject a direct gpt-5.5 selection, but no test sets a GPT-5.6 alias plus persisted modelOverrides.apiModel to an incompatible effective model. (tests/cli/runOracle/runOracle.request-payload.test.ts:120, b291cdf4a422)
  • Re-review continuity: The current head is exactly the previously reviewed SHA, so the earlier P1 remains unresolved rather than being a newly discovered late finding. (src/oracle/run.ts:129, b291cdf4a422)
  • Current-main scope check: Current main supports GPT-5.6 aliases and default effort, but does not contain --reasoning-mode or the expanded effort controls; its recent Sol Pro work is browser-side effort selection, not this Responses API feature. (src/oracle/config.ts:60, 68b8c51b0ee0)
  • Real behavior proof: The PR body supplies a redacted terminal result for an opt-in real OpenAI Responses API background smoke on this head, exercising gpt-5.6-sol with reasoning: { mode: "pro", effort: "max" } and reporting completion. (b291cdf4a422)

Likely related people:

  • Peter Steinberger: Current main attributes the GPT-5.6 model configuration and override-resolution code to this release commit; he also merged recent GPT-5.6 Sol Pro behavior work. (role: recent model-configuration contributor; confidence: high; commits: 5daa6ce8c352, a21f29c0adc8; files: src/oracle/config.ts, src/oracle/modelResolver.ts)
  • Lu Wang: Introduced the user model-override capability whose apiModel behavior exposes the unresolved validation ordering case. (role: original override feature contributor; confidence: high; commits: ea453c7b9483; files: src/oracle/modelResolver.ts, docs/configuration.md)
  • Rokurolize: Recently contributed merged GPT-5.6 Sol Pro effort handling on the related browser selection path. (role: adjacent GPT-5.6 Sol behavior contributor; confidence: medium; commits: 9468e87098ab; files: src/cli/browserConfig.ts)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Resolve the effective-model validation ordering and add the persisted modelOverrides.apiModel negative case.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (13 earlier review cycles; latest 8 shown)
  • reviewed 2026-07-30T21:32:50.006Z sha b291cdf :: needs real behavior proof before merge. :: none
  • reviewed 2026-07-30T22:10:42.219Z sha b291cdf :: needs maintainer review before merge. :: none
  • reviewed 2026-07-31T00:01:36.413Z sha b291cdf :: needs maintainer review before merge. :: none
  • reviewed 2026-07-31T01:49:25.980Z sha b291cdf :: needs maintainer review before merge. :: none
  • reviewed 2026-07-31T05:24:12.881Z sha b291cdf :: needs changes before merge. :: [P1] Validate override efforts against the resolved model
  • reviewed 2026-07-31T11:27:49.054Z sha b291cdf :: found issues before merge. :: [P1] Validate override efforts against the resolved model
  • reviewed 2026-07-31T14:27:37.435Z sha b291cdf :: needs changes before merge. :: [P1] Validate override efforts against the resolved model
  • reviewed 2026-07-31T20:16:46.712Z sha b291cdf :: needs changes before merge. :: [P1] Validate override efforts against the resolved model

@clawsweeper clawsweeper Bot added merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. labels Jul 30, 2026
@enki
enki force-pushed the feat/gpt-5.6-pro-mode-pr branch from 3d4e484 to b291cdf Compare July 30, 2026 21:25
@enki

enki commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@enki enki mentioned this pull request Jul 30, 2026
@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant