Skip to content

feat(core,prompt): Simplified property value transformers for AI Prompt generation#250

Open
mattbrailsford wants to merge 3 commits into
v18/devfrom
v18/feature/prompt-block-schema-guard
Open

feat(core,prompt): Simplified property value transformers for AI Prompt generation#250
mattbrailsford wants to merge 3 commits into
v18/devfrom
v18/feature/prompt-block-schema-guard

Conversation

@mattbrailsford

@mattbrailsford mattbrailsford commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #249 — executing an AI Prompt against a rich-text (Umbraco.RichText / TipTap) property failed with an opaque HTTP 400 (invalid_json_schema) from strict providers like OpenAI (Responses API). (The issue was reported as "Block List/Grid", but the reporter's real case was RichText — whose write schema embeds the same block structure.)

The schema-driven wand constrains the LLM's strict structured output to the target property's CMS write schema. RichText's schema is { markup, blocks: {…} }, and blocks embeds an intentionally unconstrained values[].value node ({}"Any type - depends on property editor", from CMS BlockJsonSchemaHelper). That {} has no faithful equivalent in OpenAI's strict subset, so the request is rejected before generation.

Note: the original commit on this branch took a fail-fast approach; after design review it evolved into the general transformer below. The AIPromptSchemaCompatibility guard from that commit is retained as a fallback.

Approach — simplified value transformers

New general server-side abstraction IAISimplifiedPropertyValueTransformer (Umbraco.AI.Core, PropertyValueOperations/, discovered like IAIPropertyValueHandler): a per-editor simplified, strict-representable LLM schema plus a total/idempotent transform back to the editor's real write value.

RichTextSimplifiedPropertyValueTransformer offers the LLM a plain markup string ({ "type": "string" }) and wraps the result into { markup, blocks }, preserving any existing inline blocks from the property's current value.

Consumer chooses the representation by capability (no regression):

Consumer Schema presented
Prompt (one-shot, strict) the simplified schema when a transformer is registered; else the write schema; else the string model
Agent (loose, iterative, add_item handlers) unchanged — keeps the full write schema so block authoring is preserved

What AIPromptService now does

  • Uses the simplified schema when a transformer exists for the target editor, and runs the transform server-side so ValueChange.Value is the editor's write shape, while DisplayValue stays the simplified (markup) value for the preview.
  • Reads the current write-shape value from the serialized entity/element already in the runtime context (the same source as the currentValue template variable) to preserve existing blocks — no new request field / client plumbing.
  • Runs the IsStrictRepresentable guard on whichever schema is used (incl. a simplified one), so a mis-behaving transformer fails fast cleanly and block editors (no transformer) still fail fast rather than 400.
  • Runs in plain-text output mode for DisplayMode == TipTapTool prompts (decided server-side from the persisted entity), so the in-editor toolbar keeps receiving a string.
  • Surfaces transform failures as a clean 400 and isolates per-option failures.
  • Resolver: additive ResolvePropertyValueSchemaAsync; old ResolveValueSchemaAsync kept and marked [Obsolete("… Will be removed in v20")].

Frontend

applyValueChange (document/block/media adapters) resolves the editor alias from the data-type item repository when a field is empty, so value preparers still run into empty fields.

Validation

  • Builds green; unit tests pass (Umbraco.AI 936, Umbraco.AI.Prompt 84), incl. the RichText transformer suite (with a OpenAI structured output rejects Umbraco block editor prompt schemas #249 regression replaying a live request/response byte-for-byte) and resolver tests.
  • Manually validated end-to-end against a real OpenAI key: RichText summarize prompt → no 400, multiple options generated, editor updated; server response carries the { markup, blocks } write shape with the property's existing blocks preserved.

Not in scope (follow-ups)

  • Agent adoption of the transform at the dispatcher SetValue leaf (additive, non-breaking).
  • Moving DateTime normalisation server-side + preparer cleanup.
  • "Proper" recursive block-content generation for the Prompt path (intractable strict schema — value is doubly-conditional on contentTypeKey+alias; strict mode bans if/then/allOf).

🤖 Generated with Claude Code

…structured output

Executing a prompt against a Block List or Block Grid property failed with an
opaque HTTP 400 (invalid_json_schema) from strict providers like OpenAI. The
resolved CMS property value schema embeds an intentionally unconstrained `{}`
node (BlockJsonSchemaHelper's "any type - depends on property editor"), which
has no faithful equivalent in the strict structured-output subset.

Add AIPromptSchemaCompatibility to detect schemas that can't be represented in
the strict subset (nodes with no resolvable type, arrays without items) and
fail fast in AIPromptService with a clear, user-facing message instead of
letting the provider reject the request. Editors that expose no schema still
fall back to the string-based response models; well-typed editor schemas (e.g.
ColorPicker) remain schema-constrained.

Fixes #249

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mattbrailsford and others added 2 commits July 17, 2026 08:46
…pt generation

Executing an AI Prompt against a rich-text (Umbraco.RichText / TipTap) property
returned an opaque HTTP 400 (invalid_json_schema) from strict providers: the
schema-driven wand constrained structured output to the editor's write schema,
whose nested block value is an unconstrained `{}` node OpenAI strict mode rejects.

Introduce IAISimplifiedPropertyValueTransformer (Umbraco.AI.Core): a per-editor
simplified, strict-representable LLM schema plus a total/idempotent transform back
to the editor's write value. A RichText transformer offers the LLM a plain markup
string and wraps it into { markup, blocks }, preserving existing inline blocks.

AIPromptService now:
- uses the simplified schema when a transformer is registered for the target editor
  (else the write schema, else the string model);
- runs the transform server-side so ValueChange.Value is the editor's write shape,
  while DisplayValue stays the simplified (markup) value for the preview;
- reads the current write-shape value from the serialized entity/element already in
  the runtime context (block preservation on regeneration) — no new request field;
- runs the strict-representability guard on whichever schema is used (incl. a
  simplified one) so a mis-behaving transformer fails fast cleanly, and blocks
  (no transformer) still fail fast rather than 400;
- runs in plain-text output mode for TipTapTool prompts (decided server-side from
  prompt.DisplayMode), so the in-editor toolbar keeps receiving a string;
- surfaces transform failures as a clean 400 and isolates per-option failures.

The resolver gains ResolvePropertyValueSchemaAsync (the old ResolveValueSchemaAsync
is kept and marked [Obsolete], delegating to it). Client applyValueChange now
resolves the editor alias from the data type when a field is empty, so value
preparers still run into empty fields.

Fixes #249

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…quest/response

Pins the exact live execute payload from the manual #249 validation: given the LLM's
markup string plus the property's current value (with an empty blocks envelope,
layout: {}), the RichText transformer reproduces the server response value
byte-for-byte and preserves the existing empty blocks rather than substituting the
Empty("Umbraco.RichText") default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mattbrailsford mattbrailsford changed the title fix(prompt): Fail fast when property schema can't be constrained for structured output feat(core,prompt): Simplified property value transformers for AI Prompt generation Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenAI structured output rejects Umbraco block editor prompt schemas

1 participant