Skip to content

fix: honest status reporting, routines crash, and setup-field safety - #599

Merged
Weegy merged 1 commit into
mainfrom
fix/customer-test-report-om-blockers
Aug 3, 2026
Merged

fix: honest status reporting, routines crash, and setup-field safety#599
Weegy merged 1 commit into
mainfrom
fix/customer-test-report-om-blockers

Conversation

@Weegy

@Weegy Weegy commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Context

Our first customer test round (TE Printline, 1 Aug 2026, v0.58.0 on macOS, German UI) produced 41 findings — 3 blockers, 11 high. The tester spent 76 minutes and ended with no working AI access, a crashed section, and this note:

"sicher mache ich was falsch – aber was?"

That sentence is the actual bug. This PR fixes the three construction errors behind most of the report:

  1. Status is reported, not checked. "VERBUNDEN", "AKTIV", "1 Provider verbunden" reflected whether an entry existed, never whether it worked.
  2. Errors are shown, not explained. Raw provider JSON with request IDs, error pages without a reference, causes in transient toasts.
  3. Actions without effect or without an attainable precondition.

Corrections to the report's own root-cause analysis

The report was unusually rigorous and flagged its own hypotheses as hypotheses. Three of them turned out to be wrong, and it matters:

Finding Report's hypothesis What the code actually does
OM-14 Routines crash unguarded .filter at routines/page.tsx:28-29 Not the trigger. t.rich passed a function handler against a message declaring an ICU argument ({toolName}) instead of a tag. The raw function reaches the RSC Flight serializer, which emits an error row carrying a digest inside a 200 response — explaining the opaque digest, the "all requests are 200" observation, and why reload never helped. Verified by running the Flight serializer directly under --conditions react-server. Present at 17 call sites in 8 files; routines/page.tsx is the only Server Component among them, which is exactly why it is the only page that hard-crashes. The :28-29 defect is real and is hardened too, but no path reaches it.
OM-20 Nav dropdowns dead hydration failure in the header bundle Refuted. The "native <select> that work without React" are React onChange handlers writing a cookie + router.refresh(). If the language and theme switchers genuinely work, React is alive. A real hover/click race is fixed here, but this does not fully explain the reported symptom — see Open below.
OM-22 no loading indicator spinner missing Partly refuted. Button busy exists; it is just sub-100 ms on a local probe. The real defect: generatedAt was already on the wire with zero render sites.

Also: the report says one bootstrap site seeds ANTHROPIC_API_KEY from the environment. There are three.

Changes

Blocker · LLM provider status — OM-02/03/04/08/18/33/34/35, OM-10b, OM-10

isConnected() was a vault lookup: any non-empty string meant "connected". Meanwhile a correct, complete key probe sat in desktop/src/ipc.ts writing state.keyVerified — a value read nowhere in the entire repo.

  • New middleware/src/platform/providerCredentialVerifier.ts: no_key | unverified | verified | invalid, one wire-format-keyed probe covering anthropic/openai/mistral/minimax and every openai-compatible provider, cached (TTL 300 s), shaped after the existing cliBackendDetector.
  • Verdicts are bound to a SHA-256 fingerprint of the key, so a durable record cannot resurrect a "verified" for a key that has since been replaced — that would have reproduced the very lie being removed.
  • 2xx alone is not sufficient: a JSON content-type and a bounded model-list body are required, so a corporate proxy returning a 200 text/html block page cannot render as green.
  • Only 401 means "bad key". 403 (region/org restriction) reports unverified — telling a customer in a blocked region that their working key is invalid would be a new version of the same problem.
  • redirect: 'error': fetch strips Authorization across origins but not custom headers, so a followed redirect would have forwarded the raw x-api-key.
  • GET /admin/providers stays network-free and write-free; probing happens on an explicit POST /:id/verify or a key save. connected is retained as a derived field for wire back-compat.
  • /setup now records that its ping happened instead of throwing the result away.
  • Deterministic provider ordering — the list reshuffled because reactivate() re-registered models at the end of an append-ordered array (OM-10b).
  • GDPR notice rewritten in the present tense (OM-10).

Seeding from process.env is deliberately left in place — removing it would break existing installs. The key still lands in the vault; it now reads as ungeprüft rather than verbunden, which is the actual fix.

Blocker · Routines unreachable — OM-14/19/32, OM-41

All 17 t.rich sites converted to tag syntax; payload guards so a malformed body surfaces the page's own error card instead of crashing the render; the digest is now displayed so support and customer share a reference; chat-specific recovery ("reset local chat data") no longer offered on unrelated routes; kg-lifecycle 404s explained as "needs the Postgres backend" instead of a silent empty page. A CI guard now fails the build if a t.rich handler has no matching <tag> — in every locale, not just en.

Blocker-adjacent · Plugin status — OM-16/24/36

Google Workspace reported "Installiert · AKTIV" after every credential was deleted. install_state came from registry presence alone, while the circuit-breaker's 'errored' status never reached the UI — the source comment said so outright. Readiness is now derived from declared required fields versus stored values, plus that runtime status. PluginInstallState is deliberately not widened (20+ call sites branch on === 'installed'); readiness is additive and optional.

High · A masked field invited a real password — OM-17

The tester entered their work email and their actual Google account password into gw_sa_client_email / gw_sa_private_key, and the system accepted both with "gespeichert". They wrote: "Wenn ich an dieser Stelle ein Login sehe, dann ist das kein Anwenderfehler." They are right.

  • A standing caution under every secret input — manifest-independent, and would have prevented this on its own.
  • Manifest-declared format validation, enforced server-side in every write path (PATCH …/secrets previously validated only the shape, never the value), anchored to match HTML pattern= semantics so "my password is 1234" cannot satisfy [0-9]{4}.
  • Patterns come from untrusted manifests. An allowlist grammar rejects catastrophic shapes at load, and every match runs in a worker thread under a 50 ms budget — a regex cannot be interrupted on the main thread, so nothing weaker is a real bound.

Remaining findings

Post-install next steps modelled on the skill-import flow the report praised (OM-06/07); consistent plugin counts and a genuine mis-bucketing bug fixed (OM-27); scan verdict surfaced at import (OM-25); classified error codes instead of raw provider JSON and request IDs, with legacy rows redacted on read (OM-26); generatedAt rendered (OM-22); CLI install instructions no longer hidden precisely when the CLI is missing (OM-11); symmetric tab callback (OM-05/38); Enter sends in chat with IME handling — isComposing had zero occurrences across all five composers (OM-21/37); nav hover/click decoupled (OM-20/40); /help page (OM-09); onboarding rebuilt as visible steps with LLM access as step 1, gated on a verified key (OM-01/12); numerals, initials, truncation, theme labels (OM-39/31/30/23).

Verification

middleware   typecheck clean · 5471 pass / 0 fail / 4 skipped
web-ui       typecheck clean · lint 0 errors (38 pre-existing warnings)
             i18n:check OK — 3339 keys, en + de
             vitest 67 files / 526 tests pass

An adversarial cross-vendor review found 12 defects in the first implementation, including a bypassable ReDoS screen (^(a|a)+$ passed the filter and blocked the event loop for 1739 ms at 26 characters), validation that failed open, a proxy block page reading as "verified", and 403 mapped to "invalid". All are fixed in this branch.

Every guard was mutation-tested — reverted, observed red, restored. The clearest evidence: with the execution bound removed, a single test ran for 65.5 seconds, which is the unbounded backtracking blowup measured directly.

Open / deliberately out of scope

  • OM-20 is not closed. A provable hover/click race is fixed and covered, but the reported symptom is the menu never opening at all, which that race does not explain. Needs live reproduction against the packaged desktop app: console on load, aria-expanded after a synthetic mouseenter, header scrollWidth vs clientWidth at the 1100 px window, and whether AuthBadge ever leaves its loading skeleton.
  • OM-17 is not closed for this customer until the Google Workspace manifest declares pattern + pattern_hint. That manifest lives in neither this repo nor omadia-byte5-plugins — it ships via hub.omadia.ai. The platform half lands here first and benefits every plugin; the standing caution protects users today regardless.
  • OM-28/29 full localization — ~418 items (271 de === en, 80 jargon strings, 58 hardcoded literals, 9 window titles). Separate issues. The highest-leverage piece is already in: i18n:check now warns on de === en, making the debt CI-visible.
  • OM-15 prerequisites on the plugin card, the json_file upload field, localized manifest label/help, and the agent-based help bot (which needs a byte5-operated LLM path — it has to work when the customer's key is broken).

Review notes

  • The middleware suite is flaky at default --test-concurrency on a loaded machine: three consecutive runs each failed a different test, two with connection-level fetch failed. All pass in isolation and at --test-concurrency=4. Consistent with the pre-existing flakiness, but not proven against a stashed baseline.
  • desktop/src/ipc.ts received the same probe semantics but cannot be typechecked in this worktree (desktop/node_modules absent); it is syntax-verified and mirrored by tested middleware code.
  • @formatjs/icu-messageformat-parser (used by the new i18n guard) resolves transitively via next-intl and should be added to web-ui devDependencies in a follow-up — package.json was left untouched to avoid a merge hazard.
  • /setup behaviour change worth conscious sign-off: a bare 403 now passes with a warning instead of hard-blocking. A region-restricted operator with a valid key was previously locked out of setup entirely.

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Addresses the first customer test report (41 findings, 3 blockers) against
v0.58.0. Three construction errors accounted for most of it: status is
reported rather than checked, errors are shown rather than explained, and
actions are offered whose precondition the UI cannot establish.

LLM provider status (OM-02/03/04/08/18/33/34/35)
- New providerCredentialVerifier: 4-state no_key|unverified|verified|invalid,
  wire-format-keyed probe, cached, verdicts bound to a key fingerprint so a
  restart cannot resurrect a verdict for a replaced key.
- 2xx alone is not enough: requires a JSON content-type and a bounded
  model-list body, so a proxy block page cannot read as "verified".
- Only 401 means a bad key; 403 (region/permission) reports unverified.
- redirect: 'error' so x-api-key is never forwarded to a redirect target.
- GET /admin/providers stays network- and write-free; probing happens on an
  explicit POST /:id/verify or a key save.
- /setup now records that its ping happened instead of discarding it; the
  desktop wizard finally reads the keyVerified result it has always written.
- Deterministic provider ordering (OM-10b).

Routines page crash (OM-14/19/32)
- Root cause: t.rich passed function handlers against messages declaring ICU
  arguments rather than tags. In a Server Component the raw function reaches
  the Flight serializer, which emits an error row carrying a digest inside a
  200 response. Present at 17 call sites; routines was the only server
  component among them, hence the only hard crash.
- Payload guards, digest shown on the error page, chat-specific recovery no
  longer offered on unrelated routes, and a CI guard for the whole class.

Plugin status (OM-16/24/36)
- Plugins reported ACTIVE with every credential removed. Readiness is now
  derived from declared required fields versus stored values, plus the
  existing circuit-breaker status that never reached the UI.

Setup-field safety (OM-17)
- A masked field under an email field, with no validation, invited a real
  account password. Adds a standing caution at every secret input, plus
  manifest-declared format validation enforced server-side in every write
  path, anchored to match HTML pattern semantics.
- Patterns come from untrusted manifests: an allowlist grammar rejects
  catastrophic shapes at load, and every match runs in a worker under a
  50 ms budget, so no pattern can stall the event loop.

Also: post-install next steps, skill scan verdict surfaced at import, raw
provider errors and request IDs replaced by classified codes, Enter sends in
chat (with IME handling), nav dropdown hover/click decoupled, dead tab link,
CLI install instructions no longer hidden when the CLI is missing, /help
page, onboarding rebuilt as visible steps with LLM access as step 1.
@Weegy

Weegy commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Follow-ups filed for everything this PR deliberately leaves open, so none of it gets rediscovered later:

Issue Covers Why not here
#600 OM-20/40 — nav dropdowns never open The provable hover/click race is fixed and tested, but it does not explain "never opens by any input". Needs Interceptor repro on the packaged desktop app; the issue lists exactly what to collect and what is already ruled out
#601 OM-28/29 — German localization ~418 measured items in 6 categories. i18n:check now warns on de === en, so the debt is CI-visible and cannot grow silently. I2 needs a glossary decision first
#602 OM-17/15 — Google Workspace manifest Cross-repo: that manifest is in neither this repo nor omadia-byte5-plugins, it ships via hub.omadia.ai. OM-17 is not closed for the customer until it declares pattern + pattern_hint — the standing caution added here protects users meanwhile
#603 OM-17 — json_file upload field The structural fix the tester asked for: uploading the key file removes the opportunity to make the mistake, rather than detecting it
#604 OM-09 — help assistant Blocked on an infra/commercial decision: it must run over a byte5-operated LLM path, because it has to work when the customer's key is broken
#605 Hygiene + 2 open decisions Undeclared @formatjs/icu-messageformat-parser, web-ui/CLAUDE.md drift about what i18n:check actually validates, middleware suite flakiness at default concurrency, the store-vs-admin provider contradiction (OM-06), and sign-off on the /setup 403 relaxation in this PR

@Weegy
Weegy merged commit 30ba163 into main Aug 3, 2026
9 checks passed
Weegy added a commit that referenced this pull request Aug 3, 2026
Two defects in the setup-field validation shipped in #599, both found by
writing the first real plugin manifest against it. Neither was visible from
inside this repo.

1. The pattern allowlist refused `{n,}` while allowing `+`, which is the
   same thing. The first realistic pattern anyone wrote against the feature
   — `[A-Za-z]{2,}` for an email TLD — was rejected for no safety reason and
   had to be respelled. Shape is now judged purely by the group-content
   rules that already govern `+` and `*`, so every spelling of a quantifier
   is treated alike. Each hostile pattern in the existing table is now also
   asserted in its `{n,}` spelling, so the counted form cannot open a door
   the `+` form keeps shut.

   The size cap had to be extended to `{n,}`'s minimum, which nothing needed
   to bound while the construct was illegal — otherwise `a{100000,}` would
   have become legal the moment the shape check stopped rejecting it.

   The client carried a hand-mirrored copy of the same rule, and there the
   bug was worse: a rejected pattern means no native `pattern=` attribute
   and a check that fails open, so the operator would have typed a bad
   value, seen nothing, and hit a 400 from a validation the client had
   silently opted out of. Both halves are fixed together.

2. Every `400 runtime.setup_field_invalid` returned the English hint
   whatever the UI language, because the locale parameter was never passed.
   English help text in a German UI is one of the named contributing factors
   of the finding this feature exists to prevent.

   The client now resolves the hint from the field it already holds, keyed
   on the violation's field, and falls back to the server string when the
   field is unknown. No API change: `hint` stays the English fallback for
   clients that have no manifest. The unused locale parameter is gone rather
   than left implying a threading that does not exist.

Middleware has no locale plumbing at all — no Accept-Language read anywhere,
and the locale cookie never leaves the Next.js layer. Threading one would
mean the server picking a language for a client it cannot see, which is the
same untranslatable-string-through-the-API mistake in a different costume.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant