OM-09: in-product help assistant — blocked on a byte5-operated LLM access path

[Weegy]

Follow-up from the first customer test report (PR #599). Finding OM-09, second half.

Why this matters more than a normal feature request

The tester wrote this while stuck, after the second failed chat attempt:

"Klingt blöd, aber ein Hilfebot wäre jetzt echt super."

They then spent the rest of the session looking for the mistake in themselves. Their framing:

"Ein Produkt, das sich 'An Agentic OS' nennt und dessen Kernversprechen ein Chat-Agent ist, bietet keinen Assistenten für die eigenen Bedienung."

It is also the best possible demo of our own technology, aimed at exactly the moment a prospective customer is deciding whether to keep going.

Already shipped in PR #599

A static /help page plus a nav entry: FAQ covering the four failure states this report exposed (invalid key, plugin needs configuration, CLI not installed, feature needs the Postgres backend), docs and discussion links, support contact, and version/build info. Plus a contextual help link at the LLM-access error.

What is left, and the constraint that defines it

An in-product help assistant that answers product questions and explains error states ("Warum bekomme ich invalid x-api-key?").

The hard constraint — the tester identified it themselves: it must run over a byte5-operated LLM access path. A help bot that uses the customer's own key fails in precisely the situation that generates most help requests, which is a broken or missing key. That makes this an infrastructure and commercial decision before it is a UI one:

• Which model, at whose cost, with what per-instance rate limit and abuse protection?
• Does the customer's prompt text leave their instance? For German mid-market customers this touches the same GDPR/processor question the product already surfaces elsewhere — it needs an answer before, not after.
• Does it work for self-hosted installs, and what happens when it cannot reach the byte5 endpoint?
• Grounding: product docs, the FAQ, and the instance's own live state (which providers are verified, which plugins need configuration) — the last one is what makes it genuinely better than a docs search.

Blocked on that decision. Filing it so the decision is tracked rather than rediscovered.

[Weegy]

Closing as not planned — product decision, an in-product help assistant is not on the roadmap.

The underlying OM-09 finding stands (a user with a broken LLM key has no in-product guidance), but it will be addressed through better setup copy and error hints rather than a second, byte5-operated LLM path. #602 and #607 cover the parts of that surface we are keeping.

[Weegy]

Reopening with a corrected scope. My previous close was too broad: the rejected part is the byte5-operated LLM path, not in-product help as such.

Out of scope — will not be built:

• A byte5-hosted LLM key/endpoint that omadia falls back to when the customer's own provider key is missing or broken. We do not want to operate an LLM path on behalf of customers, and the original framing of this issue made that a hard prerequisite.

In scope — worth designing:

1. Deterministic help that needs no model at all. Error-specific guidance rendered from the error itself: what failed, which setup field it came from, what a correct value looks like. This is the tier that has to work when nothing else does, and it is exactly the situation OM-09 was reported in. It overlaps with the pattern_hint work in OM-17/15: Google Workspace plugin manifest needs format validation, German labels, and prerequisites on the card #602/Setup-field pattern budget measures worker boot + IPC, so a valid value can be rejected and a healthy pattern mislabelled #607 and should reuse it rather than duplicate it.
2. Shipped, searchable troubleshooting content in the product — no network, no model, versioned with the release.
3. LLM-backed help on the customer's own key, available only once that key verifies. Strictly an enhancement on top of tiers 1-2, never the only path.

The design constraint to carry forward: the help surface must degrade to tier 1 without any behavioural cliff. A help feature that only exists when a working key exists cannot answer "my key does not work", which was the original complaint.

Next step is a scoped design proposal for tiers 1 and 2, not implementation.

[Weegy]

Design proposal — tiers 1 and 2

Grounded in what the codebase already has, so this is mostly wiring rather than new machinery. No LLM is involved in either tier.

The premise these two tiers have to satisfy

The report's situation was a customer whose provider key did not work. So the only help that counts is help that is already on the machine and needs no model, no key and no network. Everything below holds under those constraints.

Tier 1 — the error explains itself

What exists today. The server already returns a machine-readable code on failures — 238 distinct ones across middleware/src/routes/*.ts, e.g. runtime.setup_field_invalid. The setup-field path already carries a field key and an optional manifest-authored pattern_hint, and the web-ui already resolves that map by locale in web-ui/app/_lib/setupFieldPattern.ts (resolveSetupFieldHint). So the field-level case is solved; nothing else is.

What is missing. Every other one of those 238 codes reaches the user as generic copy or as nothing at all. There is no place a code maps to "what happened / what to do next".

Proposal. A single client-side catalogue, keyed by error code, living next to the messages files:

• web-ui/messages/{en,de}.json gains an errors.<code> namespace with two strings: what (one sentence) and next (the concrete action, imperative).
• A resolveErrorHelp(code, context) helper mirroring resolveSetupFieldHint: manifest-authored hint wins where one exists, catalogue entry second, existing generic copy last. Never blank.
• A vitest guard in the same family as the i18n parity guard from fix: honest status reporting, routines crash, and setup-field safety #599: every code: literal in middleware/src/routes/ must have a catalogue entry in every locale. That guard is what stops this decaying — without it the catalogue is stale within two sprints.

Scope control: 238 codes is too many to write well in one pass and most are unreachable by a normal operator. Start with the codes on the paths the report actually walked — plugin install, setup-field write, provider key save, provider verify — and let the guard's allowlist shrink over time rather than blocking on completeness.

Cost of being wrong here is low: a missing entry falls back to today's behaviour.

Tier 2 — shipped troubleshooting content

Proposal. Markdown pages built into the web-ui bundle, not fetched:

• web-ui/content/help/<topic>.{en,de}.md, rendered by an existing markdown path, versioned with the release. No network, no CMS, works offline in the desktop build.
• Client-side substring search over a build-time index. No search service. The corpus is small enough that this is honest, not a compromise.
• Tier 1's next string may carry a helpTopic so the error links straight into the relevant page. That link is the only coupling between the tiers, and it degrades to plain text if the topic is absent.

Initial topics, taken from the report rather than invented: "my LLM key is rejected", "Google Workspace: which value goes in which field", "what the plugin card's states mean", "where credentials are stored".

Sequencing

1. Tier 1 catalogue + resolver + guard, on the four paths above.
2. Tier 2 pages for the four topics, linked from tier 1.
3. Only then reconsider tier 3 (LLM help on the customer's own verified key) — and it stays an enhancement, never the only path.

Open decisions for the record

• German glossary. Tier 1 copy is user-facing German and collides directly with OM-28/29: complete the German localization — ~418 measured items across 6 categories #601's ~418-item localization backlog. Writing it before the glossary decision means writing it twice.
• Guard strictness at merge time. Enforcing all 238 codes on day one would block the PR; enforcing a named subset risks the allowlist becoming permanent. Suggest: allowlist with the count asserted, so growth is visible in a diff.
• Per the repo rule, all user-facing copy goes through the byte5 writing protocol before it lands, not after.

No implementation started — this is the design step, per the scope correction above.