Skip to content

fix(render,verify): resolve nested @objectRef by FQN across ports (ADR-0041)#182

Merged
dmealing merged 3 commits into
mainfrom
fix/render-verify-fqn-nested-objectref
Jul 7, 2026
Merged

fix(render,verify): resolve nested @objectRef by FQN across ports (ADR-0041)#182
dmealing merged 3 commits into
mainfrom
fix/render-verify-fqn-nested-objectref

Conversation

@dmealing

@dmealing dmealing commented Jul 7, 2026

Copy link
Copy Markdown
Member

Intent

Port the recent Java-only render/verify fix (FQN-exact nested @objectref resolution, ADR-0041) to the other language ports and add cross-port conformance coverage. Investigation found the Java commit bundled TWO behaviors: (1) FQN-exact nested @objectref resolution — a real cross-port bug ALSO latent in Python, C#, and TS's verify + docs-annotator resolvers (Java and Kotlin were already correct); and (2) auto-derived has-accessor recognition in verify — a Java/Spring-local idiom no other port emits and no shared template uses. User-approved decision: port ONLY Behavior 2 (FQN-exact) to Python/C#/TS via fixture-first TDD (RED->GREEN), leave Behavior 1 Java-local. Deliberate choices a reviewer won't infer from the diff: (a) a NEW multi-package xpkg-collision/ sub-corpus under fixtures/template-output-render-conformance/ is the shared oracle, wired into the render-helper conformance runners of TS/Python/C#/Kotlin; Java is already covered by TemplateVerifyTest's own ADR-0041 collision test, so its heavy javac runner was intentionally left unchanged. (b) In C# the verify field-tree resolver was deliberately DECOUPLED into a new ResolveObjectRef, kept SEPARATE from the bare FindObject used by payload-record emission, so record identifiers stay bare and record emission is provably unchanged — a code-review pass caught that making the shared FindObject FQN-aware would make EmitRecord emit an invalid 'record acme::ai::X' for an FQN @payloadRef. (c) TS routes its two buggy resolvers (cli payload-field-tree, codegen-ts template-payload-tree) through the existing shared refMatchesObject SSOT instead of duplicating logic; the render-helper resolver was already FQN-safe. (d) Test payload records are hand-authored (not generated) to sidestep an orthogonal record-name collision from the two same-short-name Note VOs. Two pre-existing, out-of-scope C# parity gaps were noted but intentionally NOT fixed (verify FindObject lacks the SUBTYPE_VALUE filter Java has; payload-record emitter emits nothing for an FQN @payloadRef) — both mirror Java's untouched record-emitter scope. Per-port suites already run green locally: TS codegen-ts 911 + cli 369 (typecheck clean), Python render/codegen (only 2 pre-existing unrelated staleness-nudge failures), C# Codegen 267 + CLI-verify 23, Kotlin render-helper conformance 6/6 including the new collision case.

What Changed

  • Ported the FQN-exact nested @objectRef resolution fix (ADR-0041) from the JVM ports to the non-JVM render/verify codegen tier: the render-helper generators (C# RenderHelperGenerator, Python render_helper_generator), the verify payload-field-tree (TS cli payload-field-tree, C# PayloadCodegen.BuildTree), and the TS docs-annotator (codegen-ts template-payload-tree) now resolve a fully-qualified nested @objectRef exactly instead of bare-tail-collapsing it, so a cross-package short-name collision (e.g. acme::alpha::Note vs acme::beta::Note) binds to the correct value object.
  • TS routes its two buggy resolvers through the shared refMatchesObject SSOT (its render helper was already FQN-safe); C# keeps a verify-only ResolveObjectRef separate from the bare FindObject used for payload-record emission so generated record identifiers stay bare; the render-helper cycle guard is now keyed by FQN rather than bare name.
  • Added a new multi-package xpkg-collision/ sub-corpus under fixtures/template-output-render-conformance/, wired into the TS/Python/C#/Kotlin render-helper conformance runners (Java already covered by TemplateVerifyTest), and synced ADR-0041's consequences with the non-JVM follow-on.

Risk Assessment

✅ Low: A well-bounded, well-tested cross-port port of an existing fix; the six payload-tree resolvers are consistent and the primary collision behavior is covered by per-port fixtures, with only a minor regression-coverage gap on the already-correct cycle-guard fix.

Testing

Baseline: installed the Bun workspace and ran each port's toolchain. Automated: ran the new/updated ADR-0041 tests in all four ports (TS render-helper-conformance + template-payload-tree + cli payload-field-tree; Python render_helper_conformance; C# PayloadCodegenTests + RenderHelperConformanceTests; Kotlin KotlinRenderHelperConformanceTest) — all green — plus the full TS codegen-ts (992) and cli (369) suites to rule out regression from routing the resolvers through the shared refMatchesObject SSOT. Evidence: for the three resolver ports I demonstrated RED→GREEN by reverting only the source resolvers to the base commit and reproducing the exact described bug (bare-tail collapse → ERR_VAR_NOT_ON_PAYLOAD / empty nested subtree / wrong-package field), then restoring. As the end-user product surface I captured the actual generated render-helper source and its rendered document output "Alpha=AA Beta=BB" for TS and Python (C# compiles+executes the helper in-test asserting the same string). There is no UI/visual surface — this is a codegen/library change whose product is generated code + rendered text, captured here as CLI transcripts. All temporary source reverts and uv.lock were restored and the worktree is clean; overall result: the fix is correct and complete across all ports with no regressions.

Evidence: TS: generated DigestDoc render helper + rendered output

GENERATED DigestDoc.render.ts baked-in verify tree: verify: [{"name":"fromAlpha","fields":[{"name":"alphaText"}]},{"name":"fromBeta","fields":[{"name":"betaText"}]}] RENDERED: renderDigestDoc({fromAlpha:{alphaText:'AA'}, fromBeta:{betaText:'BB'}}) === "Alpha=AA Beta=BB" (PASS — each FQN @objectRef bound its own package's Note)

========== GENERATED: DigestDoc.render.ts ==========
import { render } from "@metaobjectsdev/render";
import type { Provider } from "@metaobjectsdev/render";
import type { Digest } from "./Digest.js";

/**
 * Render the DigestDoc document from a typed Digest payload. Wraps the
 * render() engine; the payload field tree is baked in so render()'s runtime drift
 * check matches the build-time gate enforced when this file was generated.
 */
export function renderDigestDoc(payload: Digest, provider: Provider): string {
  return render({ ref: "xpkg/digest", payload, format: "html", provider, verify: [{"name":"fromAlpha","fields":[{"name":"alphaText"}]},{"name":"fromBeta","fields":[{"name":"betaText"}]}] });
}

========== RENDERED OUTPUT ==========
template  : templates/xpkg/digest.mustache = 'Alpha={{fromAlpha.alphaText}} Beta={{fromBeta.betaText}}'
payload   : { fromAlpha:{alphaText:'AA'}, fromBeta:{betaText:'BB'} }
rendered  : "Alpha=AA Beta=BB"
expected  : "Alpha=AA Beta=BB"
RESULT    : PASS - each FQN @objectRef bound its own package's Note
Evidence: Python: generated digest_doc_render_helper.py + rendered output

GENERATED digest_doc_render_helper.py -> render_digest_doc(...) via RenderRequest(ref='xpkg/digest', format='html') RENDERED: 'Alpha=AA Beta=BB' (expected 'Alpha=AA Beta=BB' — PASS; build-time drift gate did NOT throw ERR_VAR_NOT_ON_PAYLOAD)

========== GENERATED: digest_doc_render_helper.py ==========
# @generated by metaobjects — DO NOT EDIT.
# Source metadata: DigestDoc (DigestDoc)
# Customize via DigestDoc_extra.py in this directory.

from __future__ import annotations

from metaobjects.render.renderer import RenderRequest, render


def render_digest_doc(payload, provider) -> str:
    """Render the ``DigestDoc`` document from a typed ``Digest`` payload.

    Wraps the render engine; the mustache↔payload-VO drift check ran at
    BUILD time (codegen fails on drift)."""
    return render(
        RenderRequest(
            payload=payload,
            provider=provider,
            ref="xpkg/digest",
            format="html",
        )
    )


__all__ = ["render_digest_doc"]

========== RENDERED OUTPUT ==========
template  : templates/xpkg/digest.mustache = 'Alpha={{fromAlpha.alphaText}} Beta={{fromBeta.betaText}}'
payload   : {'fromAlpha':{'alphaText':'AA'}, 'fromBeta':{'betaText':'BB'}}
rendered  : 'Alpha=AA Beta=BB'
expected  : 'Alpha=AA Beta=BB'
RESULT    : PASS - each FQN @objectRef bound its own package's Note
Evidence: Cross-port RED→GREEN summary (TS/Python/C#/Kotlin)
ADR-0041 — FQN-exact nested @objectRef resolution ported to Python / C# / TS
============================================================================
Shared oracle: fixtures/template-output-render-conformance/xpkg-collision/
  acme::alpha::Note { alphaText }   acme::beta::Note { betaText }
  acme::app::Digest { fromAlpha -> @objectRef acme::alpha::Note,
                      fromBeta  -> @objectRef acme::beta::Note }
  template xpkg/digest.mustache = "Alpha={{fromAlpha.alphaText}} Beta={{fromBeta.betaText}}"
  payload  { fromAlpha:{alphaText:"AA"}, fromBeta:{betaText:"BB"} }  => expect "Alpha=AA Beta=BB"

The bug: pre-fix resolvers matched @objectRef by BARE short-name, so BOTH FQN refs
collapse onto whichever "Note" loads first. {{fromBeta.betaText}} then lands on the
wrong element type and the build-time drift gate throws ERR_VAR_NOT_ON_PAYLOAD.

PER-PORT RED (source reverted to base 9b8955a5) -> GREEN (target 4e0e63a5):

TypeScript (cli payload-field-tree + codegen-ts template-payload-tree resolvers)
  RED : derivePayloadFieldTree(Digest) => fromAlpha.fields=[], fromBeta.fields=[]
        (FQN ref matched NOTHING under bare-name findObject)   -> 2 fail
  GREEN: fromAlpha->[alphaText], fromBeta->[betaText]           -> 4 pass
  render helper output (see ts-digest-render.txt): "Alpha=AA Beta=BB"
  full suites: codegen-ts 992 pass / 0 fail ; cli 369 pass / 2 skip / 0 fail

Python (render_helper_generator resolver)
  RED : codegen raises ValueError render-helper drift: template "DigestDoc"
        ref "xpkg/digest" — ERR_VAR_NOT_ON_PAYLOAD: {{fromBeta.betaText}} not on payload VO
  GREEN: render_render_helper generated, renders "Alpha=AA Beta=BB"  -> 6 pass
  (see python-digest-render.txt)

C# (PayloadCodegen.ResolveObjectRef [verify field-tree] + RenderHelperGenerator.ResolveNestedObjectRef)
  RED : BuildPayloadFieldTree(Digest) fromBeta => alphaText (Expected betaText)
        RenderHelper generate() throws ERR_VAR_NOT_ON_PAYLOAD: {{fromBeta.betaText}}   -> 2 fail
  GREEN: both bind own package; compiled DigestDocRenderHelper.Render() => "Alpha=AA Beta=BB"
  full PayloadCodegenTests + RenderHelperConformanceTests: 10 pass / 0 fail
        (incl. record-emission-stays-bare guard: Assert.DoesNotContain("acme::ai::", src))

Kotlin (already FQN-safe — regression/conformance guard only)
  KotlinRenderHelperConformanceTest: 6 pass / 0 fail incl.
        `xpkg collision resolves FQN nested objectRef` (compiles+asserts DigestDocRenderHelper.kt emitted)

Java: intentionally unchanged (already correct + covered by TemplateVerifyTest's own ADR-0041 test).

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 info
  • ⚠️ server/python/src/metaobjects/codegen/generators/render_helper_generator.py:118 - The render-helper field-tree walks key their cycle guard by the VO's BARE name — Python _derive_payload_field_tree (vo.name, render_helper_generator.py:118/120) and C# DerivePayloadFieldTree (vo.Name, RenderHelperGenerator.cs:244) — whereas the other three fixed walks key by the full (now-FQN) ref: TS render-helper derivePayloadFieldTree, cli payload-field-tree.ts, and C# verify BuildTree (visiting.Add(voName) with voName == the FQN ref). Now that nested @objectRef resolves FQN-exact, a NESTED cross-package short-name collision (e.g. acme::alpha::Note has a field.object @objectRef="acme::beta::Note", where both VOs share bare name Note) makes the bare-name guard treat the distinct-package beta::Note as an already-seen cycle and return an empty subtree — falsely truncating it. This diverges from the FQN-keyed walks (which render it correctly) and, within C#, makes the baked render-helper field tree disagree with the dotnet meta verify field tree for that shape — undermining the exact cross-package guarantee ADR-0041 targets and the byte-identical-across-ports conformance goal. The shipped xpkg-collision/ fixture only covers the SIBLING arrangement (each sibling recursion gets a fresh copy/immutable seen), so it does not catch this. Fix: key the cycle guard by resolution_key() / the full ref (as the other three walks already do) so all five walks agree; optionally add a nested-collision fixture case.
  • ℹ️ server/python/src/metaobjects/codegen/generators/render_helper_generator.py:164 - The fix makes NESTED @objectRef FQN-exact across ports, but the TOP-LEVEL @payloadRef resolution is not uniformly FQN-exact: Python _resolve_payload_vo matches child.name in (payload_ref, ref_short) (bare-tail, first-wins) and the TS docs annotator entry pre-strips the ref (template-doc-builder.ts:119 stripPackage(payloadRefRaw)) before calling the now-FQN-exact buildEnrichedPayloadTree, while TS render-helper and C# verify pass the full payloadRef and resolve FQN-exact. A template.output whose OWN @payloadRef is itself a cross-package short-name collision would still mis-bind the root payload VO in those two paths. This is pre-existing (these lines are outside the diff) and not exercised by the new fixture (payloadRef Digest is unique), so it is not a regression — noting it because the change's stated goal is uniform FQN-exact resolution across ports, and the guarantee is incomplete at the payload root.

🔧 Fix: key render-helper cycle guard by FQN not bare name
1 info still open:

  • ℹ️ server/python/src/metaobjects/codegen/generators/render_helper_generator.py:118 - The round-1 cycle-guard fix — keying the payload field-tree seen set by resolution_key()/ResolutionKey() instead of the bare name (this line, and C# RenderHelperGenerator.cs:244) — is correct but has no regression test. The shipped xpkg-collision/ fixture places the two same-short-name distinct-package Note VOs as SIBLING children of Digest, and each sibling recursion receives a fresh copy of seen (Python seen | {...}, C# new HashSet<string>(seen)), so the fixture yields the correct tree with EITHER the bare-name key or the FQN key — it does not exercise the fix. The fix only changes behavior when the two colliding VOs lie on ONE nested chain (e.g. Outeracme::alpha::Noteacme::beta::Note, where both Notes accumulate into the same seen). A future revert to the bare-name key would therefore silently reintroduce the false-truncation this fix closes. A nested-chain collision fixture case would lock it in.
✅ **Test** - passed

✅ No issues found.

  • TS: bun test packages/codegen-ts/test/render-helper-conformance.test.ts packages/codegen-ts/test/template-payload-tree.test.ts packages/cli/test/unit/payload-field-tree.test.ts (10 pass)
  • TS RED check: reverted payload-field-tree.ts + template-payload-tree.ts to base 9b8955a5 → 2 fail (empty nested subtrees), restored → 4 pass
  • TS full suites: bun test packages/codegen-ts (992 pass) and bun test packages/cli (369 pass, 0 fail)
  • TS artifact capture: generated DigestDoc.render.ts + rendered renderDigestDoc(...) === "Alpha=AA Beta=BB"
  • Python: uv run --extra dev pytest tests/codegen/test_render_helper_conformance.py (6 pass incl. test_document_digest_doc_resolves_fqn_nested_object_ref_across_collision)
  • Python RED check: reverted render_helper_generator.py to base → ERR_VAR_NOT_ON_PAYLOAD on {{fromBeta.betaText}}, restored → pass; captured generated digest_doc_render_helper.py rendering "Alpha=AA Beta=BB"
  • C#: dotnet test MetaObjects.Codegen.Tests --filter PayloadCodegenTests|RenderHelperConformanceTests (10 pass incl. the 2 new collision tests + record-stays-bare guard)
  • C# RED check: reverted PayloadCodegen.cs + RenderHelperGenerator.cs to base → 2 fail (ERR_VAR_NOT_ON_PAYLOAD; fromBeta bound to alphaText), restored → 2 pass; conformance test compiles+runs DigestDocRenderHelper returning "Alpha=AA Beta=BB"
  • Kotlin: mvn -pl codegen-kotlin test -Dtest=KotlinRenderHelperConformanceTest (6 pass incl. xpkg collision resolves FQN nested objectRef)
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

dmealing and others added 3 commits July 6, 2026 20:58
…ts (ADR-0041)

The verify --templates prompt-drift gate and render-helper payload field-tree
derivation resolved a nested field.object @objectref by BARE short-name, so a
fully-qualified ref (pkg::Name) bound the WRONG same-named object.value on a
cross-package short-name collision — emptying the element subtree and raising a
spurious ERR_VAR_NOT_ON_PAYLOAD on its inner {{fields}}. This ports the Java fix
into the three remaining buggy ports and gates it cross-port:

- Python: _resolve_nested_object_ref resolves FQN-exact when the ref contains "::".
- C#: RenderHelperGenerator.ResolveNestedObjectRef is FQN-exact; the verify
  field-tree gets a dedicated ResolveObjectRef, kept SEPARATE from the bare
  FindObject used by record emission so record identifiers stay bare.
- TS: the CLI verify (payload-field-tree) and docs-annotator (template-payload-tree)
  resolvers route through the shared refMatchesObject SSOT; the render-helper
  resolver was already FQN-safe.

Shared gate: a new xpkg-collision/ sub-corpus under
fixtures/template-output-render-conformance/ (two packages each declaring an
object.value Note; a Digest payload referencing both by FQN @objectref) drives
the render-helper conformance runners in TS/Python/C#/Kotlin and fails on revert.
Java is already covered by TemplateVerifyTest's ADR-0041 collision case.

Auto-derived has-accessor recognition in verify stays a per-port concern by
design — no other port emits has<Field> accessors and no shared template uses them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VXidfNN755CyTZTg1w8UV
@dmealing
dmealing merged commit 382df95 into main Jul 7, 2026
1 check passed
@dmealing
dmealing deleted the fix/render-verify-fqn-nested-objectref branch July 7, 2026 02:20
dmealing added a commit that referenced this pull request Jul 7, 2026
FQN-exact nested @objectref resolution across ports (ADR-0041, #182) — the CLI
verify + docs-annotator payload-tree walks now route through the shared
refMatchesObject resolver. Java/Kotlin (Maven) unaffected; PyPI 0.15.9 + NuGet
0.15.7 ship the same fix on their own lines.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VXidfNN755CyTZTg1w8UV
dmealing added a commit that referenced this pull request Jul 7, 2026
dmealing added a commit that referenced this pull request Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant