-
Notifications
You must be signed in to change notification settings - Fork 595
fix(deepseek): advertise the ladder each V4 model actually honors (#1057) #1069
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
a8e1bff
d544079
1eab635
7bf6f05
45ac9eb
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| # 000 — Plan: bug-fix stack from the 260805 triage | ||
|
|
||
| ## Objective | ||
|
|
||
| Fix the defects the 2026-08-05 triage proved at code level, one PR per defect, | ||
| each with a regression test that fails without the fix. | ||
|
|
||
| Source of candidates: `devlog/_plan/260805_issue_pr_triage/010_issue_verdicts.md`. | ||
| That unit produced seventeen verdicts; five were real open defects with no owner. | ||
| Four of them are fixable here. The fifth (#1059, the Windows suite) needs a | ||
| Windows runner this machine does not have, and stays out. | ||
|
|
||
| ## Base | ||
|
|
||
| | Fact | Value | | ||
| |------|-------| | ||
| | Base | `origin/dev` = `aaa71967a` | | ||
| | Branch | `codex/260805-bug-fix-stack` | | ||
| | Dirty files preserved | `src/usage/log.ts`, `tests/usage-log.test.ts` (user-owned, untouched) | | ||
|
|
||
| ## Layer map | ||
|
|
||
| | Layer | Issue | Files | | ||
| |-------|-------|-------| | ||
| | 010 | #1057 DeepSeek ladder | `src/providers/registry.ts:349-360` **and `:1668-1669`**, `src/config.ts`, 3 test files | | ||
| | 020 | #1043 (+ live half of #1024) | `src/providers/registry.ts:1652` **and `:1671`**, `tests/vision-sidecar-e2e.test.ts` | | ||
| | 030 | #1061 test harness | `tests/native-profile-crash-boundaries.test.ts`, `tests/helpers/native-profile-startup-child.ts` | | ||
| | 040 | #1046 startup app-server | `src/codex/app-server-processes.ts`, `src/codex/desired-state.ts`, `src/server/index.ts` | | ||
|
|
||
| ### Stack shape: 010→020 stacked, 030 and 040 independent | ||
|
|
||
| An earlier draft claimed 010 and 020 touch `registry.ts` "~1300 lines apart with | ||
| no semantic overlap" and then made all four layers a linear chain. The audit | ||
| found both halves wrong. | ||
|
|
||
| **The overlap is adjacent, not distant.** 010 rewrites `opencode-free`'s | ||
| reasoning maps at `:1668-1669`; 020 rewrites that same provider object's | ||
| `noVisionModels` at `:1671`. Two lines apart, same literal. They genuinely need | ||
| ordering — just not for the reason the draft gave. | ||
|
|
||
| **030 and 040 share nothing with anything.** Chaining them behind 020 would buy | ||
| nothing and cost two retargets after the parents land. They go straight to `dev` | ||
| as independent PRs, which `AGENTS.md` permits alongside stacked children. | ||
|
|
||
| ``` | ||
| dev ──┬── 010 (#1057) ── 020 (#1043) stacked: adjacent registry edits | ||
| ├── 030 (#1061) independent | ||
| └── 040 (#1046) independent | ||
| ``` | ||
|
|
||
| Order within the stack: 010 first, because 020's `noVisionModels` widening at | ||
| `:1671` reads more clearly once 010 has already reshaped the same object's | ||
| reasoning maps. | ||
|
|
||
| ## What the design research changed | ||
|
|
||
| Three read-only `gpt-5.6-terra`/`sol` lanes and two `gpt-5.6-luna` search lanes ran | ||
| before any code was written. Two findings materially changed the plan, and both | ||
| would have produced a wrong patch if we had gone straight from the triage anchors | ||
| to an edit. | ||
|
|
||
| **#1043: the reporter's own suggested fix is the wrong one to ship first.** The | ||
| issue proposes stripping images whenever `inputModalities` lacks `"image"`. The | ||
| control-flow lane found that modality metadata is *not reliably populated* — live | ||
| `GET /v1/models` returns `undefined` when the provider omits a recognized modality | ||
| field (`src/codex/catalog/provider-fetch.ts:719-743`), and live-discovered | ||
| modalities are never copied into the request-time provider config | ||
| (`src/router.ts:84-110`). So a modality-keyed fix would silently do nothing for | ||
| exactly the provider that motivated the issue. It also found a deliberate | ||
| regression guard asserting that unlisted models keep forwarding images | ||
| (`tests/vision-sidecar-e2e.test.ts:163-193`), which a default-on strip would | ||
| break. The narrow fix — classify the zen models explicitly — ships now; the | ||
| modality-driven default is a follow-up that needs the metadata to become canonical | ||
| first. | ||
|
|
||
| **#1057: the shared mapping table may be wrong per model.** DeepSeek's official | ||
| thinking-mode docs give a native ladder of `low / high / max`, which matches the | ||
| reporter. But the same table maps requested `xhigh` differently per model — | ||
| `xhigh -> max` for `deepseek-v4-pro` and `xhigh -> high` for `deepseek-v4-flash`. | ||
| The code currently applies one shared map to both. A confirmation lane is running | ||
| against the official table before this layer is written; if the per-model | ||
| difference holds, the fix is not a one-line constant change. | ||
|
Comment on lines
+76
to
+82
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win Make the DeepSeek requirement definitive and complete. Lines 80-82 still say that confirmation is running and identify only an State the final contract here: Flash advertises 🤖 Prompt for AI Agents |
||
|
|
||
| **#1046: the obvious fix is unsafe at boot.** The existing | ||
| `afterCatalogWriteHandleAppServers()` has a `restart: true` branch that SIGTERMs | ||
| long-lived app-servers and explicitly warns that active turns may be interrupted | ||
| (`src/codex/app-server-processes.ts:738-742`). Wiring that into unattended startup | ||
| would kill a user's in-flight turn on every service start. Only the warning path | ||
| is startup-safe. | ||
|
|
||
| ## Scope boundary | ||
|
|
||
| **IN:** `src/providers/registry.ts`, `src/config.ts`, `src/codex/app-server-processes.ts`, | ||
| `src/codex/desired-state.ts`, the named test files, and this devlog unit. | ||
|
|
||
| **OUT:** #1059 (needs a Windows runner); any change to the vision default for | ||
| unlisted models (follow-up, not this stack); process termination at startup; the | ||
| user's dirty `src/usage/log.ts` and `tests/usage-log.test.ts`; merging any PR; | ||
| closing any issue by hand. | ||
|
|
||
| ## Accept criteria, all layers | ||
|
|
||
| 1. The regression test fails on the pre-fix tree and passes after — ablation output recorded in the layer's decade doc. | ||
| 2. `bun run typecheck` exits 0. | ||
| 3. The affected test files pass. | ||
| 4. No existing test is rewritten to accommodate the change unless that test was locking the defect itself, and the decade doc says which and why. | ||
| 5. Each PR fills `.github/PULL_REQUEST_TEMPLATE.md` and links its issue. | ||
|
|
||
| Criterion 4 is the one with history: `devlog/_plan/260804_overnight_triage/000_dispositions.md` | ||
| records a PR rejected for rewriting a regression contract to make a broader change | ||
| pass. Two layers here legitimately update tests (#1057's ladder assertions, #1061's | ||
| harness) — both are tests that encode the defect, and both are named in advance. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| # 001 — Upstream evidence gathered before writing any patch | ||
|
|
||
| Two `gpt-5.6-luna` search lanes ran against primary vendor sources. Both changed | ||
| the plan, and one of them stopped a layer outright. | ||
|
|
||
| ## DeepSeek reasoning ladder (#1057) — CONFIRMED, and worse than reported | ||
|
|
||
| Source: [api-docs.deepseek.com/guides/thinking_mode](https://api-docs.deepseek.com/guides/thinking_mode/), | ||
| observed 2026-08-06. The Chinese mirror agrees verbatim. | ||
|
Comment on lines
+8
to
+9
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win Correct the evidence date before treating this source as confirmed. Line 9 records August 6, 2026, but the current review date is August 5, 2026. The document cannot report an observation from tomorrow. Lines 34-38 also use present-tense timing based on that record. Replace the date with the actual collection date, or mark the observation as planned. Align the timing language before using this document as upstream evidence. Also applies to: 34-38 🤖 Prompt for AI Agents |
||
|
|
||
| | requested | `deepseek-v4-flash` | `deepseek-v4-pro` | | ||
| |---|---|---| | ||
| | `low` | `low` | `high` | | ||
| | `high` | `high` | `high` | | ||
| | `xhigh` | `high` | `max` | | ||
| | `max` | `max` | `max` | | ||
|
|
||
| The page carries a footnote: *"We will update the actual mapped effort of | ||
| deepseek-v4-pro in early August 2026."* The Chinese page says the same. | ||
|
|
||
| Three things follow, none of which were visible from the issue alone. | ||
|
|
||
| **The two models do not share a mapping.** `xhigh` resolves to `high` on Flash and | ||
| `max` on Pro; `low` resolves to `low` on Flash and `high` on Pro. Our code applies | ||
| one shared `DEEPSEEK_THINKING_REASONING_MAP` to both | ||
| (`src/providers/registry.ts:353-360`). A single corrected constant would fix Pro | ||
| and break Flash, or the reverse. The fix has to split the map per model. | ||
|
|
||
| **The reporter's requested mapping is right for Pro and wrong for Flash.** #1057 | ||
| asks for `low -> low`. That is Flash's documented behavior. On Pro, DeepSeek | ||
| itself maps `low -> high`, so advertising `low` as a native Pro tier would promise | ||
| a level the vendor does not honor. | ||
|
|
||
| **The vendor is about to change it.** The footnote says Pro's mapping updates in | ||
| early August 2026 — which is now. Pinning Pro's map today means pinning a value | ||
| the vendor has announced it will move. That is a reason to be conservative about | ||
| Pro, not a reason to wait: the advertised *ladder* (`low/high/max`) is stable in | ||
| both columns; only Pro's internal resolution of `low` and `xhigh` is in flux. | ||
|
|
||
| `medium` has no row in either table. Our map currently sends `medium -> high` | ||
| for both models. That is a local compatibility choice, not a documented vendor | ||
| behavior, and the decade doc must say so rather than implying the vendor blessed it. | ||
|
|
||
| ## OpenCode Zen free models (#1043) — NOT verifiable, layer blocked | ||
|
|
||
| Source: [opencode.ai/docs/zen](https://opencode.ai/docs/zen), observed 2026-08-06. | ||
|
|
||
| The official page lists eight free model IDs: | ||
|
|
||
| ``` | ||
| big-pickle mimo-v2.5-free laguna-s-2.1-free | ||
| ling-3.0-flash-free longcat-2.0-free north-mini-code-free | ||
| nemotron-3-ultra-free deepseek-v4-flash-free | ||
| ``` | ||
|
|
||
| It does **not** publish input modality for any of them. The lane checked the live | ||
| `/v1/models` endpoint, a community lesson page, and a third-party catalog; none | ||
| produced an authoritative per-ID modality. Its verdict was `unknown` for all | ||
| eight, with one community report that the free MiMo model refuses images — | ||
| suggestive, not sufficient. | ||
|
|
||
| This blocks the narrow fix as designed. The narrow fix means adding zen model IDs | ||
| to `noVisionModels`, and a model on that list gets its images replaced with a | ||
| caption or an omission marker before the request goes upstream | ||
| (`src/vision/index.ts:447-471`). Guessing wrong in the text-only direction | ||
| silently degrades a working vision model — the exact failure the registry comments | ||
| warn about at `src/providers/registry.ts:542-554`. | ||
|
|
||
| What is *not* blocked: `deepseek-v4-flash-free` is already classified text-only | ||
| through `OPENCODE_FREE_DEEPSEEK_MODELS`, and the sibling `opencode-free` provider | ||
| at `src/providers/registry.ts:1655-1671` already carries a `noVisionModels` list | ||
| against the same base URL. The registry's own DeepSeek modality table also states | ||
| `"deepseek-v4-flash": ["text"]` at `src/providers/registry.ts:470`. | ||
|
|
||
| **Resolved by measurement.** Rather than ship the narrow-but-partial version, the | ||
| eight models were probed directly against the live endpoint. See `002` — six are | ||
| text-only, two accept images, and the reporter's exact error was reproduced on | ||
| `big-pickle`. The layer is no longer blocked and no ID is guessed. | ||
|
|
||
| ## What this changes in the layer map | ||
|
|
||
| | Layer | Before this research | After | | ||
| |-------|---------------------|-------| | ||
| | #1057 | one-line constant change | per-model map split + config migration | | ||
| | #1043 | add zen IDs to `noVisionModels` | 6 measured text-only IDs; 2 measured vision-capable and deliberately excluded (`002`) | | ||
| | #1061 | unchanged | unchanged | | ||
| | #1046 | call the existing handler | warning-only variant; the `restart` branch is not startup-safe | | ||
|
|
||
| The #1043 downgrade is the one worth stating plainly. The triage called it a real | ||
| open defect and it is; what the search lane established is that we cannot close it | ||
| correctly today without evidence nobody has published. Shipping a guess would | ||
| trade a loud 400 for a silent capability loss. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| # 002 — Live modality probe of the OpenCode Zen free models | ||
|
|
||
| The search lane could not classify these models: the official docs publish no | ||
| modality, and `GET /v1/models` returns only `id`, `object`, `created`, | ||
| `owned_by` — no capability field at all. That absence *is* the root cause of | ||
| #1043, so it could not also serve as its evidence. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win Keep Line [6] starts with 🧰 Tools🪛 markdownlint-cli2 (0.23.2)[warning] 6-6: No space after hash on atx style heading (MD018, no-missing-space-atx) 🤖 Prompt for AI AgentsSource: Linters/SAST tools |
||
|
|
||
| So the classification was measured instead of inferred. Probed 2026-08-05 | ||
| against `https://opencode.ai/zen/v1/chat/completions`, no credential (the public | ||
| desktop tier the `opencode-free` provider already uses), one 1x1 PNG data URL per | ||
| request. | ||
|
|
||
| ## Method | ||
|
|
||
| ```bash | ||
| IMG='{"type":"image_url","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAA...ErkJggg=="}}' | ||
| curl -s https://opencode.ai/zen/v1/chat/completions \ | ||
| -H "content-type: application/json" -H "x-opencode-client: desktop" \ | ||
| -d "{\"model\":\"$m\",\"max_tokens\":8,\"messages\":[{\"role\":\"user\", | ||
| \"content\":[{\"type\":\"text\",\"text\":\"what is in this image\"},$IMG]}]}" | ||
|
Comment on lines
+15
to
+20
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Make the probe command reproducible. The command uses an undefined 🤖 Prompt for AI Agents |
||
| ``` | ||
|
|
||
| A text-only control (`"content":"hi"`) was run first and returned `200` with a | ||
| completion, which is what makes a subsequent image failure attributable to the | ||
| image part rather than to auth, quota, or the endpoint being down. | ||
|
|
||
| Rate limiting is real on this tier: a tight loop returned `Internal server error` | ||
| for every model, and the same requests succeeded with a 12-14 second gap. An | ||
| early batch run was discarded for exactly this reason — the uniform failure was | ||
| the limiter, not the models. | ||
|
|
||
| ## Result: 8 of 8 classified, no guesses | ||
|
|
||
| | model | image request | verdict | | ||
| |---|---|---| | ||
| | `big-pickle` | `invalid_request_error … Failed to deserialize the JSON body into the target type` | **text-only** | | ||
| | `nemotron-3-ultra-free` | `[404] No endpoints found that support image input` | **text-only** | | ||
| | `ling-3.0-flash-free` | `[404] No endpoints found that support image input` | **text-only** | | ||
| | `north-mini-code-free` | `[404] No endpoints found that support image input` | **text-only** | | ||
| | `laguna-s-2.1-free` | `[404] No endpoints found that support image input` | **text-only** | | ||
| | `deepseek-v4-flash-free` | `[400] Model only supports text input; received unsupported content type 'image_url'` | **text-only** | | ||
| | `mimo-v2.5-free` | `200`, completion returned | **vision-capable** | | ||
| | `longcat-2.0-free` | `200`, completion returned | **vision-capable** | | ||
|
|
||
| ## Two findings that change the patch | ||
|
|
||
| **`big-pickle` reproduces the reported error verbatim.** The issue quotes | ||
| `Failed to deserialize the JSON body into the target type: messages[65]: unknown | ||
| variant 'image_url'`. That is `big-pickle`'s exact failure shape, and it is the | ||
| *only* one of the eight that fails this way — the others return a clean 404 or | ||
| 400. So the reporter was almost certainly on `big-pickle`, and the defect is | ||
| confirmed end to end rather than by analogy. | ||
|
|
||
| **Two of the eight accept images, and a blanket classification would have broken | ||
| them.** `mimo-v2.5-free` and `longcat-2.0-free` both returned completions for an | ||
| image request. The search lane's honest `unknown` verdict, plus one community | ||
| report that "the free MiMo model refuses images", would have led straight to a | ||
| wrong entry: MiMo is exactly the model that *does* work. Listing it in | ||
| `noVisionModels` would silently replace a user's image with a caption on a model | ||
| that never needed it. | ||
|
|
||
| This is why the probe was worth the ten minutes. The narrow fix is now | ||
| evidence-backed rather than blocked, and it covers six models instead of the two | ||
| the repository already knew about. | ||
|
|
||
| ## Caveat recorded honestly | ||
|
|
||
| This is a point-in-time measurement of a live free tier, taken once per model. | ||
| Zen's catalog is discovered live (`liveModels: true` on the sibling | ||
| `opencode-free` entry), so the free roster can change under a static registry | ||
| list. The patch should therefore classify what is measured today and leave the | ||
| list easy to amend, not claim permanence. | ||
|
|
||
| `#1024`'s remaining `TR` / `moonshotai/kimi-k3-free` half is untouched by this — | ||
| `TR` is not a built-in registry provider and depends on reporter configuration. | ||
|
|
||
| ## Follow-up: does this evidence transfer to the key-auth provider? | ||
|
|
||
| The audit raised a real objection: the probe used `x-opencode-client: desktop`, | ||
| which is the *free* tier's header, while layer 020 modifies the **key-auth** | ||
| `opencode-zen` entry. Evidence from one access tier applied to another is exactly | ||
| the kind of reasoning that silently destroys a user's image. | ||
|
|
||
| Re-probed 2026-08-05 with the header removed entirely: | ||
|
|
||
| | request | result | | ||
| |---|---| | ||
| | `big-pickle` text, no header | `200`, completion returned | | ||
| | `big-pickle` + image, no header | `invalid_request_error … unknown variant 'image_url', expected 'text'` | | ||
| | `mimo-v2.5-free` + image, no header | `200`, reasoning begins *"The user has provided an image and…"* | | ||
| | `big-pickle` text, `authorization: Bearer sk-invalid-probe` | `AuthError: Invalid API key.` | | ||
|
|
||
| Three things follow. | ||
|
|
||
| **The desktop header was never what made the probe work.** The same models answer | ||
| with no header at all, so the modality behavior is a property of the model | ||
| routing, not of the free tier's client identity. | ||
|
|
||
| **Model capability is enforced upstream of authentication.** `big-pickle` rejects | ||
| `image_url` identically with and without the header, and the rejection text names | ||
| the upstream provider's deserializer. A key does not change which content types a | ||
| text-only model accepts. | ||
|
|
||
| **`mimo-v2.5-free` is now doubly confirmed as vision-capable** — it does not merely | ||
| return 200, it narrates the image. The reviewer's own spot-check hit a transient | ||
| 502 on this model and recorded it as inconclusive rather than contrary; this run | ||
| resolves it. | ||
|
|
||
| What remains genuinely unproven is whether an *authenticated* Zen account is | ||
| served a different roster or different routing for the same IDs. That cannot be | ||
| settled without a key. The layer handles it by scope rather than by assumption: | ||
| see `020`. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
Add language tags to both fenced blocks.
markdownlint-cli2reports MD040 for both blocks. Addtextto each opening fence.devlog/_plan/260805_bug_fix_stack/000_plan.md#L45-L45: change the opening fence to```text.devlog/_plan/260805_bug_fix_stack/001_upstream_evidence.md#L50-L50: change the opening fence to```text.🧰 Tools
🪛 markdownlint-cli2 (0.23.2)
[warning] 45-45: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
📍 Affects 2 files
devlog/_plan/260805_bug_fix_stack/000_plan.md#L45-L45(this comment)devlog/_plan/260805_bug_fix_stack/001_upstream_evidence.md#L50-L50🤖 Prompt for AI Agents
Source: Linters/SAST tools