From 210bf3bf60d610075bf158c9f09c708e711e2537 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Thu, 9 Jul 2026 11:25:30 -0400 Subject: [PATCH 1/3] fix(loader): order-independent super-resolution for dotted extends to inherited members (#188) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Metadata super-resolution was LOAD-ORDER-SENSITIVE: a dotted `extends: Owner.member` reference (FR-024 child-targeting ref) targeting an INHERITED member failed with ERR_UNRESOLVED_SUPER whenever the referring file was resolved before the owner entity's own `extends` chain was wired. The deferred-super pass walked nodes in physical-declaration (= load) order and read the owner's EFFECTIVE children (inherited members appear only once the owner's OWN super is resolved), so a dotted ref to an inherited member resolved only if the owner's chain happened to be wired first — green under Node's readdir order, a hard build failure under Bun's (the `meta` bin runs under Bun on a base image with no real node, e.g. oven/bun:1). Resolving a corpus must be a pure function of the source SET, not its enumeration order. Durable fix (all ports — TS reference + Python / C# / Java; Kotlin shares Java's loader): replace the single pre-order walk with ON-DEMAND MEMOIZED resolution with cycle detection. When resolving a dotted ref, resolve the OWNER node's own extends-chain FIRST (recursively, memoized) so its effective children include the inherited member before the ref reads it; after wiring a node, resolve the TARGET's own chain too (multi-level inheritance, e.g. Base extends AbstractRoot). An `attempted` guard makes each node resolve — and, on failure, report — EXACTLY ONCE, preserving the old single-visit walk's no-duplicate-failures guarantee now that a node is reachable from both the pending loop and owner/target recursion. Tier-1 invariants unchanged: ERR_UNRESOLVED_SUPER / ERR_EXTENDS_TARGET_MISMATCH, the failure envelope, and the target-mismatch contract are byte-identical. Floor (TS SDK only): `listMetadataFiles` now sorts files to match its docstring and the metadata package's own `DirectorySource` — deterministic enumeration across runtimes/filesystems. The other ports' DirectorySource already sorts. Gated by a new cross-port conformance fixture (`extends-dotted-inherited-member-load-order`: an abstract base with `id`+`pk`, an entity that inherits them, a projection whose `pk`/`id` carry dotted extends to those inherited members — filenames ordered so the referrer loads FIRST, reproducing the bug) plus a TS shuffle-invariance test (all 6 source-order permutations → identical resolved model) and a duplicate-failure regression test. All ports green: TS metadata 2131/0 + shuffle/dedup, Python conformance 336/0, C# conformance 720/0, Java metadata 1125/0, Kotlin 272/0. Co-Authored-By: Claude --- .../expected-effective.json | 110 +++++++++ .../expected.json | 95 ++++++++ .../input/a-view.json | 39 ++++ .../input/b-customer.json | 17 ++ .../input/c-base.json | 17 ++ .../providers.json | 1 + server/csharp/MetaObjects/SuperResolve.cs | 82 ++++++- .../metaobjects/loader/MetaDataLoader.java | 212 +++++++++++++----- .../python/src/metaobjects/super_resolve.py | 142 +++++++++--- .../packages/metadata/src/super-resolve.ts | 56 ++++- .../metadata/test/super-resolve.test.ts | 161 +++++++++++++ server/typescript/packages/sdk/src/memory.ts | 8 +- 12 files changed, 852 insertions(+), 88 deletions(-) create mode 100644 fixtures/conformance/extends-dotted-inherited-member-load-order/expected-effective.json create mode 100644 fixtures/conformance/extends-dotted-inherited-member-load-order/expected.json create mode 100644 fixtures/conformance/extends-dotted-inherited-member-load-order/input/a-view.json create mode 100644 fixtures/conformance/extends-dotted-inherited-member-load-order/input/b-customer.json create mode 100644 fixtures/conformance/extends-dotted-inherited-member-load-order/input/c-base.json create mode 100644 fixtures/conformance/extends-dotted-inherited-member-load-order/providers.json diff --git a/fixtures/conformance/extends-dotted-inherited-member-load-order/expected-effective.json b/fixtures/conformance/extends-dotted-inherited-member-load-order/expected-effective.json new file mode 100644 index 000000000..0e2702cfc --- /dev/null +++ b/fixtures/conformance/extends-dotted-inherited-member-load-order/expected-effective.json @@ -0,0 +1,110 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.projection": { + "name": "CustomerView", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_customers" + } + }, + { + "field.uuid": { + "name": "id", + "extends": "acme::shop::Customer.id", + "@required": true, + "children": [ + { + "origin.passthrough": { + "@from": "acme::shop::Customer.id" + } + } + ] + } + }, + { + "field.string": { + "name": "name", + "children": [ + { + "origin.passthrough": { + "@from": "acme::shop::Customer.name" + } + } + ] + } + }, + { + "identity.primary": { + "name": "pk", + "extends": "acme::shop::Customer.pk", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Customer", + "extends": "acme::common::BaseEntity", + "children": [ + { + "field.uuid": { + "name": "id", + "@required": true + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": [ + "id" + ] + } + }, + { + "source.rdb": { + "@table": "customers" + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + } + ] + } + }, + { + "object.entity": { + "name": "BaseEntity", + "abstract": true, + "children": [ + { + "field.uuid": { + "name": "id", + "@required": true + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": [ + "id" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/extends-dotted-inherited-member-load-order/expected.json b/fixtures/conformance/extends-dotted-inherited-member-load-order/expected.json new file mode 100644 index 000000000..f0bffd871 --- /dev/null +++ b/fixtures/conformance/extends-dotted-inherited-member-load-order/expected.json @@ -0,0 +1,95 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.projection": { + "name": "CustomerView", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_customers" + } + }, + { + "field.uuid": { + "name": "id", + "extends": "acme::shop::Customer.id", + "children": [ + { + "origin.passthrough": { + "@from": "acme::shop::Customer.id" + } + } + ] + } + }, + { + "field.string": { + "name": "name", + "children": [ + { + "origin.passthrough": { + "@from": "acme::shop::Customer.name" + } + } + ] + } + }, + { + "identity.primary": { + "name": "pk", + "extends": "acme::shop::Customer.pk", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Customer", + "extends": "acme::common::BaseEntity", + "children": [ + { + "source.rdb": { + "@table": "customers" + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + } + ] + } + }, + { + "object.entity": { + "name": "BaseEntity", + "abstract": true, + "children": [ + { + "field.uuid": { + "name": "id", + "@required": true + } + }, + { + "identity.primary": { + "name": "pk", + "@fields": [ + "id" + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/extends-dotted-inherited-member-load-order/input/a-view.json b/fixtures/conformance/extends-dotted-inherited-member-load-order/input/a-view.json new file mode 100644 index 000000000..a9cd1620d --- /dev/null +++ b/fixtures/conformance/extends-dotted-inherited-member-load-order/input/a-view.json @@ -0,0 +1,39 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.projection": { + "name": "CustomerView", + "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_customers" } }, + { + "field.uuid": { + "name": "id", + "extends": "acme::shop::Customer.id", + "children": [ + { "origin.passthrough": { "@from": "acme::shop::Customer.id" } } + ] + } + }, + { + "field.string": { + "name": "name", + "children": [ + { "origin.passthrough": { "@from": "acme::shop::Customer.name" } } + ] + } + }, + { + "identity.primary": { + "name": "pk", + "extends": "acme::shop::Customer.pk", + "@fields": ["id"] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/extends-dotted-inherited-member-load-order/input/b-customer.json b/fixtures/conformance/extends-dotted-inherited-member-load-order/input/b-customer.json new file mode 100644 index 000000000..ac507a20d --- /dev/null +++ b/fixtures/conformance/extends-dotted-inherited-member-load-order/input/b-customer.json @@ -0,0 +1,17 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Customer", + "extends": "acme::common::BaseEntity", + "children": [ + { "source.rdb": { "@table": "customers" } }, + { "field.string": { "name": "name", "@required": true } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/extends-dotted-inherited-member-load-order/input/c-base.json b/fixtures/conformance/extends-dotted-inherited-member-load-order/input/c-base.json new file mode 100644 index 000000000..0edd36e13 --- /dev/null +++ b/fixtures/conformance/extends-dotted-inherited-member-load-order/input/c-base.json @@ -0,0 +1,17 @@ +{ + "metadata.root": { + "package": "acme::common", + "children": [ + { + "object.entity": { + "name": "BaseEntity", + "abstract": true, + "children": [ + { "field.uuid": { "name": "id", "@required": true } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/extends-dotted-inherited-member-load-order/providers.json b/fixtures/conformance/extends-dotted-inherited-member-load-order/providers.json new file mode 100644 index 000000000..aefa027b6 --- /dev/null +++ b/fixtures/conformance/extends-dotted-inherited-member-load-order/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types", "metaobjects-db"] diff --git a/server/csharp/MetaObjects/SuperResolve.cs b/server/csharp/MetaObjects/SuperResolve.cs index 5c4ddb13e..98ee0b3fb 100644 --- a/server/csharp/MetaObjects/SuperResolve.cs +++ b/server/csharp/MetaObjects/SuperResolve.cs @@ -249,22 +249,84 @@ public sealed record DeferredSuperFailure( TypeIdentity? Referrer = null); /// - /// Walk the tree, resolve every node's SuperRef against the full root, - /// collect unresolved refs. Idempotent: nodes that already have SuperData set are skipped. - /// Tracks an inherited context-package while walking so children whose own Package is - /// unset still resolve correctly (mirroring the parser's eager-resolution semantics). + /// Resolve every node's SuperRef against the full root, collecting unresolved + /// refs. Idempotent: nodes that already have SuperData set are skipped. + /// + /// + /// #188: super-resolution is ORDER-INDEPENDENT. The prior implementation + /// resolved each SuperRef in a single pre-order walk — physical (= load) + /// order. A dotted ref to an INHERITED member (extends: Owner.member + /// where Owner inherits member via its OWN extends) reads the + /// owner's EFFECTIVE — inherited members appear + /// only once the owner's extends chain is resolved. So it succeeded only when + /// the owner happened to resolve first: green under one directory-scan order, + /// ERR_UNRESOLVED_SUPER under another. Instead resolve ON DEMAND with + /// memoization + cycle detection (topological): before a dotted ref reads the + /// owner's children, resolve the owner's whole chain first, and after resolving + /// a node also resolve its target's chain (multi-level inheritance). The result + /// is a pure function of the source SET, independent of enumeration order. + /// + /// + /// + /// An inherited context-package is threaded while collecting the pending set so + /// children whose own Package is unset still resolve correctly (mirroring the + /// parser's eager-resolution semantics); it is captured per node so a node + /// resolved out of collection order still uses the correct package. + /// /// public static IReadOnlyList ResolveDeferredSupers(MetaData root) { var failures = new List(); + + // Every SuperRef-bearing node over the PHYSICAL declaration tree, plus each + // node's effective context-package (node.Package ?? nearest-ancestor + // package) — the same value the pre-order walk threaded. + var pending = new List(); + var effectivePkgOf = new Dictionary(ReferenceEqualityComparer.Instance); Walk(root, "", (node, ctxPkg) => { if (node.SuperRef is null) return; - if (node.SuperData is not null) return; - string effectivePkg = node.Package ?? ctxPkg; + effectivePkgOf[node] = node.Package ?? ctxPkg; + pending.Add(node); + }); + + // Cycle guard: a genuine super cycle is unresolvable → the node is left + // unresolved and reported as a failure below. + var inProgress = new HashSet(ReferenceEqualityComparer.Instance); + // Nodes already attempted this pass. On-demand resolution can reach a node + // via the `pending` loop AND via owner/target recursion — `attempted` makes + // each node resolve (and, on failure, report) EXACTLY ONCE, restoring the + // single-visit walk's no-duplicate-failures guarantee (a successful node is + // deduped by SuperData; this also covers the FAILURE path, which sets no + // marker). Mirrors the TS reference (#188). + var attempted = new HashSet(ReferenceEqualityComparer.Instance); + + void ResolveNode(MetaData node) + { + if (node.SuperRef is null || node.SuperData is not null) return; + if (!attempted.Add(node)) return; // already resolved-or-failed this pass — never re-report + if (!inProgress.Add(node)) return; // cycle — leave unresolved; reported below + string effectivePkg = effectivePkgOf.TryGetValue(node, out var pkg) ? pkg : (node.Package ?? ""); + + // For a dotted child-targeting ref, ResolveSuperRef reads the OWNER's + // effective Children() — which includes inherited members only once the + // owner's own extends chain is resolved. Resolve the owner node's chain + // FIRST (memoized), regardless of collection order. + if (IsChildTargetingRef(node.SuperRef)) + { + (string OwnerRef, string[] Path)? parsed = ParseChildTargetingRef(node.SuperRef); + if (parsed is not null) + { + MetaData? ownerNode = ResolveSuperRef(parsed.Value.OwnerRef, effectivePkg, root); + if (ownerNode is not null) ResolveNode(ownerNode); + } + } + // FR-024: thread the referrer's type so dotted `Entity.child` refs resolve // type-scoped (a field ref selects fields; an identity ref identities). MetaData? target = ResolveSuperRef(node.SuperRef, effectivePkg, root, new ReferrerScope(node.Type)); + inProgress.Remove(node); + if (target is not null) { // FR-024: a dotted ref must target a node of the SAME type and subtype @@ -289,12 +351,18 @@ public static IReadOnlyList ResolveDeferredSupers(MetaData { // Frozen — ignore; the loader should resolve before freeze. } + // Ensure the target's OWN chain is resolved too, so a later Children() + // read on `node` (codegen/validation) sees the full multi-level + // inherited set (e.g. Base extends AbstractRoot). Memoized — no re-work. + ResolveNode(target); } else { failures.Add(new DeferredSuperFailure(node.Fqn(), node.SuperRef, node)); } - }); + } + + foreach (var node in pending) ResolveNode(node); return failures.AsReadOnly(); } diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java b/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java index ef7ba8c28..1949f0e58 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java @@ -234,51 +234,132 @@ public void drainPendingExtends() { private void resolvePendingExtends() { if (pendingExtends.isEmpty()) return; + // #188 — deferred super-resolution must be ORDER-INDEPENDENT. The former + // single pre-order pass resolved each queued node's `extends` in physical- + // declaration (= load) order, so a dotted `extends: Owner.member` ref to an + // INHERITED member (Owner gets `member` via its OWN `extends`) only + // succeeded when Owner's own chain happened to be wired first — green under + // one directory-scan order, ERR_UNRESOLVED_SUPER under another (Node vs Bun + // readdir). Instead resolve ON DEMAND with memoization + cycle detection + // (topological): before a dotted ref reads the owner's EFFECTIVE children, + // resolve the owner node's whole chain first. The result is a pure function + // of the source SET, independent of enumeration order. Mirrors the TS + // reference (super-resolve.ts, #188). + // + // Identity-keyed maps/sets: MetaData nodes are compared by reference here + // (a queued entry's `child` is the SAME tree-node instance that + // resolveOwnerObject / getChildOfType returns), never by value-equality. + java.util.IdentityHashMap byChild = + new java.util.IdentityHashMap<>(); for (PendingExtends p : pendingExtends) { - MetaData superData = null; - // FR-024 (ADR-0029): dotted child-targeting ref `.` — - // the final ::-segment containing '.' is unambiguous (names cannot contain - // '.'). Resolve the OWNER object with the existing strategies, then select - // the child among the owner's EFFECTIVE children (includeParentData) by - // name + the REFERRER'S type (type-scoped: a field ref resolves fields, - // an identity ref identities). Dotted refs never fall through to the bare - // top-level lookup; the multi-dot form (X.y.z) is reserved → unresolved. - // A resolved dotted target whose type/subtype differs from the referrer's - // is ERR_EXTENDS_TARGET_MISMATCH (dotted-only — top-level extends behavior - // is unchanged). Mirrors the TS reference (super-resolve.ts, commit - // 809712f8) and C# SuperResolve.cs. The parser's getSuperMetaData never - // resolves dotted refs (no top-level node has a dotted name), so every - // dotted ref arrives here via the pending queue — single resolution site. - if (isChildTargetingRef(p.superName)) { - superData = resolveChildTargetingRef(p); - if (superData == null) { - throwUnresolvedSuper(p); + byChild.put(p.child, p); + } + java.util.Set inProgress = + java.util.Collections.newSetFromMap(new java.util.IdentityHashMap<>()); + java.util.Set resolved = + java.util.Collections.newSetFromMap(new java.util.IdentityHashMap<>()); + for (PendingExtends p : pendingExtends) { + resolvePendingNode(p, byChild, inProgress, resolved); + } + pendingExtends.clear(); + } + + /** + * Resolve ONE deferred {@code extends} entry, ON DEMAND and memoized (#188). + * + *

For a dotted child-targeting ref, the OWNER object's own {@code extends} + * chain is resolved FIRST (recursively, memoized) so the owner's EFFECTIVE + * children include inherited members before {@link #resolveChildTargetingRef} + * reads them — regardless of the order the sources were enumerated. After the + * node itself is wired, the resolved TARGET's own chain is resolved too, so a + * later effective-children read on this node sees the full multi-level + * inherited set (e.g. {@code Base extends AbstractRoot}). Memoization means no + * node is resolved twice; the {@code inProgress} set breaks a genuine super + * cycle (the node is left for its originating frame, which reports the failure + * as {@code ERR_UNRESOLVED_SUPER}).

+ * + *

The Tier-1 contract is unchanged: an unresolved ref throws + * {@code ERR_UNRESOLVED_SUPER} and a type/subtype-mismatched dotted target + * throws {@code ERR_EXTENDS_TARGET_MISMATCH}, both via the existing helpers.

+ */ + private void resolvePendingNode(PendingExtends p, + java.util.IdentityHashMap byChild, + java.util.Set inProgress, + java.util.Set resolved) { + if (resolved.contains(p.child)) return; // already wired (memoized) + if (inProgress.contains(p.child)) return; // cycle — leave to the outer frame + inProgress.add(p.child); + + MetaData superData; + // FR-024 (ADR-0029): dotted child-targeting ref `.` — + // the final ::-segment containing '.' is unambiguous (names cannot contain + // '.'). Resolve the OWNER object with the existing strategies, then select + // the child among the owner's EFFECTIVE children (includeParentData) by + // name + the REFERRER'S type (type-scoped: a field ref resolves fields, + // an identity ref identities). Dotted refs never fall through to the bare + // top-level lookup; the multi-dot form (X.y.z) is reserved → unresolved. + // A resolved dotted target whose type/subtype differs from the referrer's + // is ERR_EXTENDS_TARGET_MISMATCH (dotted-only — top-level extends behavior + // is unchanged). Mirrors the TS reference (super-resolve.ts) and C# + // SuperResolve.cs. The parser's getSuperMetaData never resolves dotted refs + // (no top-level node has a dotted name), so every dotted ref arrives here + // via the pending queue — single resolution site. + if (isChildTargetingRef(p.superName)) { + // #188: wire the owner's OWN chain first so its effective children + // include inherited members before resolveChildTargetingRef reads them. + MetaData owner = resolveOwnerObject(p); + if (owner != null) { + PendingExtends ownerPending = byChild.get(owner); + if (ownerPending != null) { + resolvePendingNode(ownerPending, byChild, inProgress, resolved); } - p.child.setSuperData(superData); - continue; } + superData = resolveChildTargetingRef(p); + } else { + superData = resolveTopLevelSuper(p); + } + if (superData == null) { + throwUnresolvedSuper(p); + } + + p.child.setSuperData(superData); + inProgress.remove(p.child); + resolved.add(p.child); + + // #188: resolve the TARGET's own chain too (multi-level inheritance), + // memoized — so a subsequent effective read on this node sees every + // inherited member. No-op when the target had no deferred extends. + PendingExtends targetPending = byChild.get(superData); + if (targetPending != null) { + resolvePendingNode(targetPending, byChild, inProgress, resolved); + } + } + + /** + * Resolve a non-dotted top-level {@code extends} ref: try the package-prepended + * name first, then the raw/FQN name. Returns {@code null} when neither resolves + * (the caller throws {@code ERR_UNRESOLVED_SUPER}). Extracted verbatim from the + * former inline {@link #resolvePendingExtends} loop body. + */ + private MetaData resolveTopLevelSuper(PendingExtends p) { + MetaData superData = null; + try { + String sn = p.superName; + String pkg = p.packageName == null ? "" : p.packageName; + if (sn.indexOf(PKG_SEPARATOR) < 0 && !pkg.isEmpty()) { + superData = getChildOfType(p.typeName, pkg + PKG_SEPARATOR + sn); + } + } catch (com.metaobjects.MetaDataNotFoundException ignore) { + // fall through to FQN lookup + } + if (superData == null) { try { - String sn = p.superName; - String pkg = p.packageName == null ? "" : p.packageName; - if (sn.indexOf(PKG_SEPARATOR) < 0 && !pkg.isEmpty()) { - superData = getChildOfType(p.typeName, pkg + PKG_SEPARATOR + sn); - } + superData = getChildOfType(p.typeName, p.superName); } catch (com.metaobjects.MetaDataNotFoundException ignore) { - // fall through to FQN lookup - } - if (superData == null) { - try { - superData = getChildOfType(p.typeName, p.superName); - } catch (com.metaobjects.MetaDataNotFoundException ignore) { - // still unresolved - } + // still unresolved } - if (superData == null) { - throwUnresolvedSuper(p); - } - p.child.setSuperData(superData); } - pendingExtends.clear(); + return superData; } /** @@ -316,20 +397,15 @@ private static boolean isChildTargetingRef(String ref) { } /** - * FR-024 (ADR-0029): resolve a dotted child-targeting {@code extends} ref. - * Splits {@code ....} (any depth), resolves - * the owner OBJECT via the existing pkg-prepend-then-FQN strategies, then - * selects the owner's EFFECTIVE child (includeParentData) by name + the - * referrer's type. A resolved target whose type/subtype differs from the - * referrer's throws {@code ERR_EXTENDS_TARGET_MISMATCH} (dotted-only check). + * FR-024 (ADR-0029): resolve just the OWNER object of a dotted child-targeting + * ref — the {@code } before the first {@code .} of the final + * {@code ::}-segment. Uses the same pkg-prepend → referrer-ancestry-package → + * FQN strategies as {@link #resolveChildTargetingRef}; returns {@code null} + * when the owner object is not found (or the ref is not a well-formed dotted + * ref). Extracted so #188's order-independent resolver can wire the owner's + * OWN {@code extends} chain before the effective-children read. */ - private MetaData resolveChildTargetingRef(PendingExtends p) { - // Addressing model (ADR-0029): the package qualifies the ROOT-level node - // only; each subsequent segment traverses CHILD NAMES to any depth - // (object → field → view: "Customer.priceCents.display"). INTERMEDIATE - // segments select by UNIQUE name among the current node's effective - // children (a cross-type name collision is ambiguous → unresolved); the - // FINAL segment is type-scoped to the referrer. Mirrors the TS reference. + private MetaData resolveOwnerObject(PendingExtends p) { int lastSep = p.superName.lastIndexOf(PKG_SEPARATOR); int segStart = lastSep < 0 ? 0 : lastSep + PKG_SEPARATOR.length(); String lastSegment = p.superName.substring(segStart); @@ -380,6 +456,40 @@ private MetaData resolveChildTargetingRef(PendingExtends p) { return null; } } + return owner; + } + + /** + * FR-024 (ADR-0029): resolve a dotted child-targeting {@code extends} ref. + * Splits {@code ....} (any depth), resolves + * the owner OBJECT via {@link #resolveOwnerObject}, then selects the owner's + * EFFECTIVE child (includeParentData) by name + the referrer's type. A + * resolved target whose type/subtype differs from the referrer's throws + * {@code ERR_EXTENDS_TARGET_MISMATCH} (dotted-only check). + */ + private MetaData resolveChildTargetingRef(PendingExtends p) { + // Addressing model (ADR-0029): the package qualifies the ROOT-level node + // only; each subsequent segment traverses CHILD NAMES to any depth + // (object → field → view: "Customer.priceCents.display"). INTERMEDIATE + // segments select by UNIQUE name among the current node's effective + // children (a cross-type name collision is ambiguous → unresolved); the + // FINAL segment is type-scoped to the referrer. Mirrors the TS reference. + int lastSep = p.superName.lastIndexOf(PKG_SEPARATOR); + int segStart = lastSep < 0 ? 0 : lastSep + PKG_SEPARATOR.length(); + String lastSegment = p.superName.substring(segStart); + String[] parts = lastSegment.split("\\.", -1); + if (parts.length < 2) { + return null; + } + for (String part : parts) { + if (part.isEmpty()) { + return null; // degenerate (empty segment) + } + } + MetaData owner = resolveOwnerObject(p); + if (owner == null) { + return null; + } // Traverse INTERMEDIATE segments by unique child name (effective view); // a missing name or a cross-type collision (e.g. a field AND an identity diff --git a/server/python/src/metaobjects/super_resolve.py b/server/python/src/metaobjects/super_resolve.py index 69c7e392a..47c116ab2 100644 --- a/server/python/src/metaobjects/super_resolve.py +++ b/server/python/src/metaobjects/super_resolve.py @@ -1,10 +1,11 @@ """Deferred super/extends resolution over the merged tree (2nd pass, pre-freeze). Mirrors TS ``resolveDeferredSupers`` in super-resolve.ts: -- Walk the tree over own_children(), tracking an inherited context package. -- Build the FQN index keyed by node.fqn() (own). -- For each node with an unresolved super_ref, resolve using - ``effective_pkg = node.package or inherited_context_pkg``. +- Build the FQN index keyed by node.fqn() (own), over the whole merged tree. +- Collect every node carrying an unresolved super_ref, tracking each node's + inherited context package. +- Resolve supers ON DEMAND with memoization + cycle detection (#188), NOT in a + single physical-declaration-order pass — so resolution is order-independent. """ from __future__ import annotations @@ -15,34 +16,81 @@ def resolve_supers(root: MetaData, errors: list[MetaError]) -> None: - """Walk every node in the merged tree; for each unresolved super_ref, resolve it. + """Resolve every unresolved super_ref in the merged tree, order-independently. Resolved → sets node.super_data. Unresolved → appends ERR_UNRESOLVED_SUPER to errors. + Dotted target type/subtype mismatch → appends ERR_EXTENDS_TARGET_MISMATCH. Already-resolved nodes (super_data is not None) are skipped (idempotent). + + #188: super-resolution must be ORDER-INDEPENDENT. A pre-order walk resolved + each node's super_ref in physical-declaration (= load) order, so a dotted ref + to an INHERITED member (``extends: Owner.member`` where Owner inherits + ``member`` via its OWN ``extends``) only succeeded when the owner's extends + chain happened to be wired first — green under one readdir order, throwing + ERR_UNRESOLVED_SUPER under another. Instead resolve ON DEMAND with + memoization + cycle detection: before a dotted ref reads ``owner.children()`` + (EFFECTIVE children — inherited members appear only once the owner's OWN + super is resolved), resolve the owner's whole chain first. The result is a + pure function of the source SET, independent of enumeration order. Mirrors + the TS reference (#188). """ index = _build_index(root) - _walk(root, "", index, errors) + # Every node carrying a super_ref, over the PHYSICAL declaration tree, with + # the inherited context package captured at that node's walk position (the + # effective_pkg the original pre-order walk computed — kept byte-identical so + # every existing fixture resolves the same way). + pending: list[MetaData] = [] + pkg_of: dict[MetaData, str | None] = {} + _collect(root, "", pending, pkg_of) -def _walk( - node: MetaData, - ctx_pkg: str, - index: dict[str, MetaData], - errors: list[MetaError], -) -> None: - """Visit *node* then recurse over own_children(), carrying an inherited context package.""" - if node.super_ref and node.super_data is None: + # Cycle guard (a genuine super cycle is unresolvable → reported as unresolved). + in_progress: set[MetaData] = set() + # Nodes already attempted this pass. On-demand resolution can reach a node + # via the ``pending`` loop AND via owner/target recursion — ``attempted`` + # makes each node resolve (and, on failure, report) EXACTLY ONCE, restoring + # the single-visit walk's no-duplicate-failures guarantee (a successful node + # is deduped by ``super_data``; this also covers the FAILURE path, which + # sets no marker). Mirrors the TS reference (#188). + attempted: set[MetaData] = set() + + def resolve_node(node: MetaData) -> None: + if not node.super_ref or node.super_data is not None: + return + if node in attempted: + return # already resolved-or-failed this pass — never re-report + if node in in_progress: + return # cycle — leave unresolved; the failure is reported below + attempted.add(node) + in_progress.add(node) # Referrer context package: own ``package`` if declared, else the - # file-default package captured at parse time, else the inherited - # walk context. Using file_default_package (not just the threaded - # ctx_pkg) is what lets a bare / same-package / cross-package - # ``extends`` resolve over the MERGED tree, where object nodes carry - # no own package and the parent chain no longer reaches the per-file - # root. Mirrors TS ``resolveDeferredSupers`` - # (``node.package ?? node.fileDefaultPackage``). - effective_pkg = node.package or node.file_default_package or ctx_pkg or None + # file-default package captured at parse time, else the inherited walk + # context. Captured during _collect (mirrors the original ``_walk`` + # effective_pkg). Using file_default_package (not just the threaded + # ctx_pkg) is what lets a bare / same-package / cross-package ``extends`` + # resolve over the MERGED tree. + effective_pkg = pkg_of[node] if node in pkg_of else ( + node.package or node.file_default_package or None + ) + + # For a dotted child-targeting ref, ``_resolve`` reads the OWNER's + # effective ``children()`` — which only includes inherited members once + # the owner's own extends chain is resolved. So resolve the owner node's + # chain FIRST (memoized), regardless of load order (#188). + if _is_child_targeting_ref(node.super_ref): + owner_ref = _owner_ref(node.super_ref) + if owner_ref is not None: + owner = _resolve(owner_ref, effective_pkg, index) + if owner is not None: + resolve_node(owner) + + # FR-024: thread the referrer's type so dotted ``Entity.child`` refs + # resolve type-scoped (a field ref selects fields; an identity ref + # identities). target = _resolve(node.super_ref, effective_pkg, index, referrer_type=node.type) + in_progress.discard(node) + if target is None: # FR5d / ADR-0009: emit a ResolvedSource envelope carrying the # referrer's files / json_path plus the referrer FQN + unresolved @@ -71,13 +119,43 @@ def _walk( )) else: node.super_data = target + # Ensure the target's OWN chain is resolved too, so a later + # ``children()`` read on *node* (codegen/validation) sees the full + # multi-level inherited set (e.g. Base extends AbstractRoot). + # Memoized — no re-work. + resolve_node(target) + + for node in pending: + resolve_node(node) + + +def _collect( + node: MetaData, + ctx_pkg: str, + pending: list[MetaData], + pkg_of: dict[MetaData, str | None], +) -> None: + """Pre-order walk: record every node carrying a super_ref plus the effective + context package at its walk position, then recurse over own_children(). + + Mirrors the referrer-context computation of the original ``_walk`` so the + on-demand resolver uses exactly the same effective_pkg the pre-order walk + did — resolution stays byte-identical for every existing fixture; only the + ORDER in which supers are wired changes (now order-independent, #188). + """ + if node.super_ref: + # own ``package`` if declared, else the file-default package captured at + # parse time, else the inherited walk context. + pkg_of[node] = node.package or node.file_default_package or ctx_pkg or None + pending.append(node) # The context package for children is this node's own package, if set, else inherit. next_ctx = node.package or ctx_pkg - # ADR-0039 sanctioned own: this IS the super-resolution walk — it runs BEFORE - # extends is resolved (it sets super_data), so it must iterate own children. + # ADR-0039 sanctioned own: this IS the super-resolution collection walk — it + # runs BEFORE extends is resolved (resolution sets super_data), so it must + # iterate own children. for child in node.own_children(): - _walk(child, next_ctx, index, errors) + _collect(child, next_ctx, pending, pkg_of) def _build_index(root: MetaData) -> dict[str, MetaData]: @@ -118,6 +196,20 @@ def _is_child_targeting_ref(ref: str) -> bool: return "." in last_segment +def _owner_ref(ref: str) -> str | None: + """The OWNER portion of a dotted child-targeting ref (``Owner.child`` → + ``Owner``, ``pkg::Owner.child.grandchild`` → ``pkg::Owner``): the package + prefix plus the first dotted segment. ``None`` for a degenerate ref (fewer + than two segments or any empty segment). Mirrors the TS + ``parseChildTargetingRef().ownerRef``.""" + last_sep = ref.rfind(PACKAGE_SEP) + seg_start = 0 if last_sep < 0 else last_sep + len(PACKAGE_SEP) + parts = ref[seg_start:].split(".") + if len(parts) < 2 or any(not p for p in parts): + return None + return ref[:seg_start] + parts[0] + + def _resolve( ref: str, context_pkg: str | None, diff --git a/server/typescript/packages/metadata/src/super-resolve.ts b/server/typescript/packages/metadata/src/super-resolve.ts index d395af90c..f3b5c9296 100644 --- a/server/typescript/packages/metadata/src/super-resolve.ts +++ b/server/typescript/packages/metadata/src/super-resolve.ts @@ -225,13 +225,55 @@ export interface DeferredSuperFailure { */ export function resolveDeferredSupers(root: MetaData): DeferredSuperFailure[] { const failures: DeferredSuperFailure[] = []; - walk(root, (node) => { - if (node.superRef === undefined) return; - if (node.superResolved !== undefined) return; + // #188: super-resolution must be ORDER-INDEPENDENT. A pre-order walk resolved + // each node's superRef in physical-declaration (= load) order, so a dotted + // ref to an INHERITED member (`extends: Owner.member` where Owner inherits + // `member` via its own `extends`) only succeeded when the owner's extends + // chain happened to be wired first — green under Node's readdir order, + // ERR_UNRESOLVED_SUPER under Bun's. Instead resolve ON DEMAND with + // memoization + cycle detection: before a dotted ref reads `owner.children()` + // (effective children — inherited members appear only once the owner's OWN + // super is resolved), resolve the owner's whole chain first. The result is a + // pure function of the source SET, independent of enumeration order. + const inProgress = new Set(); // cycle guard (a genuine super cycle is unresolvable → reported) + // Nodes already attempted this pass. Because resolution is now on-demand, a + // node can be reached both by the `pending` loop AND by owner/target + // recursion from another node — `attempted` makes each node resolve (and, on + // failure, report) EXACTLY ONCE, restoring the old single-visit walk's + // no-duplicate-failures guarantee (a successful node is already deduped by + // `superResolved`; this also covers the FAILURE path, which sets no marker). + const attempted = new Set(); + // Every node carrying a superRef, over the PHYSICAL declaration tree. + const pending: MetaData[] = []; + walk(root, (n) => { + if (n.superRef !== undefined) pending.push(n); + }); + + const resolveNode = (node: MetaData): void => { + if (node.superRef === undefined || node.superResolved !== undefined) return; + if (attempted.has(node)) return; // already resolved-or-failed this pass — never re-report + if (inProgress.has(node)) return; // cycle — leave unresolved; the failure is reported below + attempted.add(node); + inProgress.add(node); const effectivePkg = node.package ?? node.fileDefaultPackage ?? ""; + + // For a dotted child-targeting ref, `resolveSuperRef` reads the OWNER's + // effective `children()` — which only includes inherited members once the + // owner's own extends chain is resolved. So resolve the owner node's chain + // FIRST (memoized), regardless of load order. + if (isChildTargetingRef(node.superRef)) { + const parsed = parseChildTargetingRef(node.superRef); + if (parsed !== undefined) { + const ownerNode = resolveSuperRef(parsed.ownerRef, effectivePkg, root); + if (ownerNode !== undefined) resolveNode(ownerNode); + } + } + // FR-024: thread the referrer's type so dotted `Entity.child` refs resolve // type-scoped (a field ref selects fields; an identity ref identities). const target = resolveSuperRef(node.superRef, effectivePkg, root, { type: node.type }); + inProgress.delete(node); + if (target !== undefined) { // FR-024: a dotted ref must target a node of the SAME type and subtype // as the extending node. Dotted-only — top-level extends is unchanged. @@ -254,6 +296,10 @@ export function resolveDeferredSupers(root: MetaData): DeferredSuperFailure[] { } catch { // Frozen — ignore; the loader should resolve before freeze. } + // Ensure the target's OWN chain is resolved too, so a later `children()` + // read on `node` (codegen/validation) sees the full multi-level inherited + // set (e.g. Base extends AbstractRoot). Memoized — no re-work. + resolveNode(target); } else { failures.push({ nodeFqn: node.fqn(), @@ -262,7 +308,9 @@ export function resolveDeferredSupers(root: MetaData): DeferredSuperFailure[] { kind: "unresolved", }); } - }); + }; + + for (const node of pending) resolveNode(node); return failures; } diff --git a/server/typescript/packages/metadata/test/super-resolve.test.ts b/server/typescript/packages/metadata/test/super-resolve.test.ts index 1b9f33815..28d3f5da1 100644 --- a/server/typescript/packages/metadata/test/super-resolve.test.ts +++ b/server/typescript/packages/metadata/test/super-resolve.test.ts @@ -13,6 +13,12 @@ import type { MetaData } from "../src/shared/meta-data.js"; import { TypeId } from "../src/registry.js"; import { resolveSuperRef } from "../src/super-resolve.js"; import { expandRef } from "../src/naming-refs.js"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; +import { InMemoryStringSource } from "../src/loader/meta-data-source.js"; +import { composeRegistry } from "../src/provider.js"; +import { coreTypesProvider } from "../src/core-types.js"; +import { dbProvider } from "../src/persistence/db/db-provider.js"; +import { canonicalSerialize } from "../src/serializer-json.js"; import { TYPE_METADATA, TYPE_OBJECT, @@ -293,3 +299,158 @@ describe("resolveSuperRef — dotted Entity.child refs (FR-024)", () => { expect(resolveSuperRef("mypkg::Target", "other", root, { type: TYPE_OBJECT })).toBe(target); }); }); + +describe("resolveDeferredSupers — load-order independence (#188)", () => { + // Three docs: an abstract base declaring `id` + `pk`, an entity that INHERITS + // them via `extends`, and a projection whose `pk`/`id` carry a dotted + // `extends: Customer.pk` / `Customer.id` to those INHERITED members. Before + // #188 this resolved only when the owner's file was parsed before the + // referrer's — green under Node's readdir order, ERR_UNRESOLVED_SUPER under + // Bun's. Resolving a corpus must be a pure function of the source SET, so + // EVERY permutation must yield the same clean, byte-identical resolved model. + const base = JSON.stringify({ + "metadata.root": { + package: "acme::common", + children: [ + { + "object.entity": { + name: "BaseEntity", + abstract: true, + children: [ + { "field.uuid": { name: "id", "@required": true } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); + const customer = JSON.stringify({ + "metadata.root": { + package: "acme::shop", + children: [ + { + "object.entity": { + name: "Customer", + extends: "acme::common::BaseEntity", + children: [ + { "source.rdb": { "@table": "customers" } }, + { "field.string": { name: "name", "@required": true } }, + ], + }, + }, + ], + }, + }); + const view = JSON.stringify({ + "metadata.root": { + package: "acme::shop", + children: [ + { + "object.projection": { + name: "CustomerView", + children: [ + { "source.rdb": { "@kind": "view", "@view": "v_customers" } }, + { + "field.uuid": { + name: "id", + extends: "acme::shop::Customer.id", + children: [{ "origin.passthrough": { "@from": "acme::shop::Customer.id" } }], + }, + }, + { + "field.string": { + name: "name", + children: [{ "origin.passthrough": { "@from": "acme::shop::Customer.name" } }], + }, + }, + { + "identity.primary": { + name: "pk", + extends: "acme::shop::Customer.pk", + "@fields": ["id"], + }, + }, + ], + }, + }, + ], + }, + }); + const docs: Record = { base, customer, view }; + const names = Object.keys(docs); + const permutations = (xs: string[]): string[][] => + xs.length <= 1 ? [xs] : xs.flatMap((x, i) => permutations([...xs.slice(0, i), ...xs.slice(i + 1)]).map((p) => [x, ...p])); + + async function loadInOrder(order: string[]): Promise<{ ok: boolean; model: string }> { + const loader = new MetaDataLoader({ registry: composeRegistry([coreTypesProvider, dbProvider]), strict: true }); + const result = await loader.load(order.map((n) => new InMemoryStringSource(docs[n]!, { id: `${n}.json` }))); + if (result.errors.length) return { ok: false, model: "" }; + // The RESOLVED SEMANTIC model must be order-independent. Whole-tree + // canonical serialization is NOT: it preserves the top-level node SEQUENCE, + // which legitimately follows load order (base-first lists BaseEntity first; + // view-first lists CustomerView first). So compare the order-insensitive + // SET of each root child's canonically-serialized subtree — that captures + // every resolved relationship (who extends what, which inherited members + // are reachable) without pinning the top-level ordering. + const subtrees = result.root + .children() + .map((c) => canonicalSerialize(c).trim()) + .sort(); + return { ok: true, model: JSON.stringify(subtrees) }; + } + + it("every permutation of the source order resolves cleanly AND produces the same resolved model", async () => { + const results = await Promise.all(permutations(names).map(loadInOrder)); + // No permutation errors — the dotted ref to an inherited member resolves in every order. + for (const r of results) expect(r.ok).toBe(true); + // Every permutation yields the identical order-insensitive resolved model. + const first = results[0]!.model; + expect(first.length).toBeGreaterThan(2); + for (const r of results) expect(r.model).toBe(first); + }); + + it("reports an unresolvable owner EXACTLY ONCE (no duplicate failures from on-demand recursion)", async () => { + // An owner with an unresolvable top-level super, plus a projection whose + // dotted `extends: Owner.pk` reaches that owner via recursion. Under the + // on-demand resolver the owner is reachable both from the `pending` loop and + // from the referrer's owner-recursion — the `attempted` guard must ensure it + // is reported only ONCE (the pre-#188 single-visit walk's guarantee). + const owner = JSON.stringify({ + "metadata.root": { + package: "p", + children: [ + { "object.entity": { name: "Owner", extends: "p::DoesNotExist", children: [{ "field.uuid": { name: "id" } }] } }, + ], + }, + }); + const view = JSON.stringify({ + "metadata.root": { + package: "p", + children: [ + { + "object.projection": { + name: "V", + children: [ + { "source.rdb": { "@kind": "view", "@view": "v" } }, + { "identity.primary": { name: "pk", extends: "p::Owner.pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, + }); + const loader = new MetaDataLoader({ registry: composeRegistry([coreTypesProvider, dbProvider]), strict: true }); + const result = await loader.load([ + new InMemoryStringSource(view, { id: "a.json" }), + new InMemoryStringSource(owner, { id: "b.json" }), + ]); + const seen = new Map(); + for (const e of result.errors) { + const err = e as { code?: string; source?: { jsonPath?: string } }; + const key = `${err.code}@${err.source?.jsonPath ?? "?"}`; + seen.set(key, (seen.get(key) ?? 0) + 1); + } + for (const [key, count] of seen) expect(`${key} x${count}`).toBe(`${key} x1`); + }); +}); diff --git a/server/typescript/packages/sdk/src/memory.ts b/server/typescript/packages/sdk/src/memory.ts index 0f0e455e5..860c4d5b4 100644 --- a/server/typescript/packages/sdk/src/memory.ts +++ b/server/typescript/packages/sdk/src/memory.ts @@ -161,7 +161,13 @@ async function listMetadataFiles(dir: string): Promise { } const paths: string[] = []; const subdirs: string[] = []; - for (const entry of entries) { + // #188: sort the raw `readdir` entries so file order is deterministic across + // runtimes/filesystems (Node vs Bun return different `readdir` orders), matching + // this function's docstring and the metadata package's own `DirectorySource`. + // (Resolution is now order-INDEPENDENT — super-resolve.ts #188 — so this is the + // deterministic-enumeration FLOOR, not the fix; it keeps every derived artifact + // that preserves declaration order, e.g. serialization, stable across runtimes.) + for (const entry of [...entries].sort()) { if (entry === "_pending") continue; const full = join(dir, entry); const s = await stat(full); From 8630a254ad277c7a2f0efa4069a6c0138a38844b Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Thu, 9 Jul 2026 12:05:26 -0400 Subject: [PATCH 2/3] no-mistakes(review): Remove dead super-cycle guards, thread Java owner resolve --- server/csharp/MetaObjects/SuperResolve.cs | 10 ++++------ .../metaobjects/loader/MetaDataLoader.java | 19 ++++++++++--------- .../python/src/metaobjects/super_resolve.py | 11 ++++------- .../packages/metadata/src/super-resolve.ts | 7 +++---- 4 files changed, 21 insertions(+), 26 deletions(-) diff --git a/server/csharp/MetaObjects/SuperResolve.cs b/server/csharp/MetaObjects/SuperResolve.cs index 98ee0b3fb..6391d12ce 100644 --- a/server/csharp/MetaObjects/SuperResolve.cs +++ b/server/csharp/MetaObjects/SuperResolve.cs @@ -290,22 +290,21 @@ public static IReadOnlyList ResolveDeferredSupers(MetaData pending.Add(node); }); - // Cycle guard: a genuine super cycle is unresolvable → the node is left - // unresolved and reported as a failure below. - var inProgress = new HashSet(ReferenceEqualityComparer.Instance); // Nodes already attempted this pass. On-demand resolution can reach a node // via the `pending` loop AND via owner/target recursion — `attempted` makes // each node resolve (and, on failure, report) EXACTLY ONCE, restoring the // single-visit walk's no-duplicate-failures guarantee (a successful node is // deduped by SuperData; this also covers the FAILURE path, which sets no - // marker). Mirrors the TS reference (#188). + // marker). It is also the cycle guard: `attempted` is never removed, so a + // genuine super cycle (A→B→A) terminates on re-entry to the + // already-attempted node rather than recursing forever. Mirrors the TS + // reference (#188). var attempted = new HashSet(ReferenceEqualityComparer.Instance); void ResolveNode(MetaData node) { if (node.SuperRef is null || node.SuperData is not null) return; if (!attempted.Add(node)) return; // already resolved-or-failed this pass — never re-report - if (!inProgress.Add(node)) return; // cycle — leave unresolved; reported below string effectivePkg = effectivePkgOf.TryGetValue(node, out var pkg) ? pkg : (node.Package ?? ""); // For a dotted child-targeting ref, ResolveSuperRef reads the OWNER's @@ -325,7 +324,6 @@ void ResolveNode(MetaData node) // FR-024: thread the referrer's type so dotted `Entity.child` refs resolve // type-scoped (a field ref selects fields; an identity ref identities). MetaData? target = ResolveSuperRef(node.SuperRef, effectivePkg, root, new ReferrerScope(node.Type)); - inProgress.Remove(node); if (target is not null) { diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java b/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java index 1949f0e58..72b8cf2fd 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/MetaDataLoader.java @@ -314,7 +314,7 @@ private void resolvePendingNode(PendingExtends p, resolvePendingNode(ownerPending, byChild, inProgress, resolved); } } - superData = resolveChildTargetingRef(p); + superData = resolveChildTargetingRef(p, owner); } else { superData = resolveTopLevelSuper(p); } @@ -461,13 +461,15 @@ private MetaData resolveOwnerObject(PendingExtends p) { /** * FR-024 (ADR-0029): resolve a dotted child-targeting {@code extends} ref. - * Splits {@code ....} (any depth), resolves - * the owner OBJECT via {@link #resolveOwnerObject}, then selects the owner's - * EFFECTIVE child (includeParentData) by name + the referrer's type. A - * resolved target whose type/subtype differs from the referrer's throws - * {@code ERR_EXTENDS_TARGET_MISMATCH} (dotted-only check). - */ - private MetaData resolveChildTargetingRef(PendingExtends p) { + * Splits {@code ....} (any depth), then selects the + * owner's EFFECTIVE child (includeParentData) by name + the referrer's type. + * The owner object is supplied by the caller ({@link #resolvePendingNode}, + * which resolves it first to wire its own {@code extends} chain per #188) — + * resolved once, not re-resolved here. A resolved target whose type/subtype + * differs from the referrer's throws {@code ERR_EXTENDS_TARGET_MISMATCH} + * (dotted-only check). + */ + private MetaData resolveChildTargetingRef(PendingExtends p, MetaData owner) { // Addressing model (ADR-0029): the package qualifies the ROOT-level node // only; each subsequent segment traverses CHILD NAMES to any depth // (object → field → view: "Customer.priceCents.display"). INTERMEDIATE @@ -486,7 +488,6 @@ private MetaData resolveChildTargetingRef(PendingExtends p) { return null; // degenerate (empty segment) } } - MetaData owner = resolveOwnerObject(p); if (owner == null) { return null; } diff --git a/server/python/src/metaobjects/super_resolve.py b/server/python/src/metaobjects/super_resolve.py index 47c116ab2..ab1949a9c 100644 --- a/server/python/src/metaobjects/super_resolve.py +++ b/server/python/src/metaobjects/super_resolve.py @@ -45,14 +45,15 @@ def resolve_supers(root: MetaData, errors: list[MetaError]) -> None: pkg_of: dict[MetaData, str | None] = {} _collect(root, "", pending, pkg_of) - # Cycle guard (a genuine super cycle is unresolvable → reported as unresolved). - in_progress: set[MetaData] = set() # Nodes already attempted this pass. On-demand resolution can reach a node # via the ``pending`` loop AND via owner/target recursion — ``attempted`` # makes each node resolve (and, on failure, report) EXACTLY ONCE, restoring # the single-visit walk's no-duplicate-failures guarantee (a successful node # is deduped by ``super_data``; this also covers the FAILURE path, which - # sets no marker). Mirrors the TS reference (#188). + # sets no marker). It is also the cycle guard: ``attempted`` is never + # removed, so a genuine super cycle (A→B→A) terminates on re-entry to the + # already-attempted node rather than recursing forever. Mirrors the TS + # reference (#188). attempted: set[MetaData] = set() def resolve_node(node: MetaData) -> None: @@ -60,10 +61,7 @@ def resolve_node(node: MetaData) -> None: return if node in attempted: return # already resolved-or-failed this pass — never re-report - if node in in_progress: - return # cycle — leave unresolved; the failure is reported below attempted.add(node) - in_progress.add(node) # Referrer context package: own ``package`` if declared, else the # file-default package captured at parse time, else the inherited walk # context. Captured during _collect (mirrors the original ``_walk`` @@ -89,7 +87,6 @@ def resolve_node(node: MetaData) -> None: # resolve type-scoped (a field ref selects fields; an identity ref # identities). target = _resolve(node.super_ref, effective_pkg, index, referrer_type=node.type) - in_progress.discard(node) if target is None: # FR5d / ADR-0009: emit a ResolvedSource envelope carrying the diff --git a/server/typescript/packages/metadata/src/super-resolve.ts b/server/typescript/packages/metadata/src/super-resolve.ts index f3b5c9296..50588bf9f 100644 --- a/server/typescript/packages/metadata/src/super-resolve.ts +++ b/server/typescript/packages/metadata/src/super-resolve.ts @@ -235,13 +235,15 @@ export function resolveDeferredSupers(root: MetaData): DeferredSuperFailure[] { // (effective children — inherited members appear only once the owner's OWN // super is resolved), resolve the owner's whole chain first. The result is a // pure function of the source SET, independent of enumeration order. - const inProgress = new Set(); // cycle guard (a genuine super cycle is unresolvable → reported) // Nodes already attempted this pass. Because resolution is now on-demand, a // node can be reached both by the `pending` loop AND by owner/target // recursion from another node — `attempted` makes each node resolve (and, on // failure, report) EXACTLY ONCE, restoring the old single-visit walk's // no-duplicate-failures guarantee (a successful node is already deduped by // `superResolved`; this also covers the FAILURE path, which sets no marker). + // It is also the cycle guard: `attempted` is never removed, so a genuine + // super cycle (A→B→A) terminates on re-entry to the already-attempted node + // rather than recursing forever. const attempted = new Set(); // Every node carrying a superRef, over the PHYSICAL declaration tree. const pending: MetaData[] = []; @@ -252,9 +254,7 @@ export function resolveDeferredSupers(root: MetaData): DeferredSuperFailure[] { const resolveNode = (node: MetaData): void => { if (node.superRef === undefined || node.superResolved !== undefined) return; if (attempted.has(node)) return; // already resolved-or-failed this pass — never re-report - if (inProgress.has(node)) return; // cycle — leave unresolved; the failure is reported below attempted.add(node); - inProgress.add(node); const effectivePkg = node.package ?? node.fileDefaultPackage ?? ""; // For a dotted child-targeting ref, `resolveSuperRef` reads the OWNER's @@ -272,7 +272,6 @@ export function resolveDeferredSupers(root: MetaData): DeferredSuperFailure[] { // FR-024: thread the referrer's type so dotted `Entity.child` refs resolve // type-scoped (a field ref selects fields; an identity ref identities). const target = resolveSuperRef(node.superRef, effectivePkg, root, { type: node.type }); - inProgress.delete(node); if (target !== undefined) { // FR-024: a dotted ref must target a node of the SAME type and subtype From 5117d0d90c542529e83f02a033c69bf2a736c1a1 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Thu, 9 Jul 2026 12:35:14 -0400 Subject: [PATCH 3/3] no-mistakes(document): Document #188 load-order-independent super-resolution --- CHANGELOG.md | 1 + docs/features/abstracts-and-inheritance.md | 9 +++++++++ docs/features/loaders.md | 7 ++++++- 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7a565b40c..6079a4357 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,7 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm - **Typed value-object jsonb columns work end-to-end across all persistence ports (single + array-of-VO).** A `field.object @storage:jsonb` column — a single value-object OR an `@isArray` array-of-VO — now round-trips through every port's runtime/ORM write+read codec (TS Kysely, C# EF Core, Java OMDB/Gson, Kotlin Exposed, Python pg8000). Gated by a new array-of-VO dimension in the persistence-conformance `AllTypes` `op: roundtrip` scenario: a `labels` column (`field.object @isArray @storage:jsonb`) written as a 2-element, empty-`[]` (≠ `null`), and single-element array across three rows — the gap that had let a non-compiling / wrong array serializer ship in three ports. The single-VO jsonb path was already cross-port green; this closes the array-of-VO half. ### Fixed +- **Load-order-independent super-resolution for dotted `extends` to an inherited member, all five ports (#188).** Deferred super-resolution ran a single pre-order walk over the physical declaration tree, so a dotted ref to an INHERITED member (`extends: Owner.member` where `Owner` inherits `member` via its OWN `extends`) only resolved when the owner's extends chain happened to be wired first — green under one directory-scan order, `ERR_UNRESOLVED_SUPER` under another (Node vs Bun `readdir`). Resolution is now ON DEMAND with memoization + cycle detection: before a dotted ref reads the owner's effective `children()`, the owner's whole extends chain is resolved first, and the resolved target's chain is resolved too (multi-level inheritance). The result is a pure function of the source SET, independent of enumeration order. The TS SDK's `listMetadataFiles` also sorts its raw `readdir` entries so every declaration-order-preserving artifact (serialization) is stable across runtimes — a deterministic-enumeration floor (the other ports' directory sources already sort). Tier-1 invariants are unchanged (`ERR_UNRESOLVED_SUPER` / `ERR_EXTENDS_TARGET_MISMATCH`, failure envelope, target-mismatch contract byte-identical). Gated by a new `extends-dotted-inherited-member-load-order` conformance fixture (RED→GREEN in every port) + a TS shuffle-invariance test (six permutations → identical model) + a duplicate-failure regression test. - **C# / Java / Python array-of-VO jsonb write+read codecs.** C# EF Core model finalization threw `'ICollection must be a non-interface reference type'` on an `@isArray` `field.object @storage:jsonb` column — `DbContextGenerator` now emits `.OwnsMany(...).ToJson(...)` when `field.ResolvedIsArray()` (the `EntityGenerator` emits `ICollection`), with a coupled empty-`[]`-vs-`null` nullability fix. Java OMDB threw `Expected BEGIN_OBJECT but was BEGIN_ARRAY` — `GenericSQLDriver.deserializeJsonb` now branches on `isArray` (target `TypeToken.getParameterized(List.class, VO)` via `jsonbTargetType`) and the read path stores the resulting `List` through `setObjectArray` (not the scalar `setObject`). Python's `_coerce_write_value` now `json.dumps`s both dict and list jsonb-storage `field.object`/`field.map` values to a JSON text string — pg8000 binds a native `dict` to jsonb fine, but adapts a native `list` as a Postgres ARRAY literal (`{...,...}`) which the JSONB column rejects with `22P02`. ### Changed diff --git a/docs/features/abstracts-and-inheritance.md b/docs/features/abstracts-and-inheritance.md index f73ad6c3d..a4b9483eb 100644 --- a/docs/features/abstracts-and-inheritance.md +++ b/docs/features/abstracts-and-inheritance.md @@ -139,6 +139,11 @@ This means: - **Forward references are fine.** A field that `extends: shortSlug` can appear in a file the loader processes before the file that declares `shortSlug`. +- **Dotted refs to inherited members resolve in any order.** A field + `extends: Owner.member` works even when `member` reaches `Owner` only + through `Owner`'s own `extends:`. Super-resolution wires the owner's chain on + demand before reading its effective members, so file/enumeration order never + changes the result (#188). - **Multi-level chains work.** `Author extends BaseEntity extends Auditable` flattens to one effective shape. Each level can add children or override attrs. @@ -375,6 +380,10 @@ inheritance chain. extends B extends C, full chain flattening - [`extends-cross-file`](../../fixtures/conformance/extends-cross-file/) — forward reference across files +- [`extends-dotted-inherited-member-load-order`](../../fixtures/conformance/extends-dotted-inherited-member-load-order/) + — a dotted `extends:` to a member the owner reaches only through its own + `extends:` resolves regardless of file order (load-order-independent + super-resolution, #188) - [`extends-abstract-base`](../../fixtures/conformance/extends-abstract-base/) — abstract entities aren't instantiable; codegen skips them - [`error-extends-nonexistent`](../../fixtures/conformance/error-extends-nonexistent/) diff --git a/docs/features/loaders.md b/docs/features/loaders.md index e57cafa4d..8652d818e 100644 --- a/docs/features/loaders.md +++ b/docs/features/loaders.md @@ -142,7 +142,12 @@ name = author.field("name") 4. **Overlay merge.** Files sharing the same `package` + object `name` are merged. Last-writer-wins on attribute conflicts; structural children accumulate. 5. **Super-resolve.** `extends:` is resolved after all files load (deferred, not - eager) so a child can extend a base declared in any other file. + eager) so a child can extend a base declared in any other file. Resolution is + **order-independent** — a pure function of the source set — so a dotted + `extends:` to a member the owner reaches through its OWN `extends:` (e.g. + `extends: Owner.member` where `Owner` inherits `member`) resolves regardless of + file enumeration order; the owner's chain is wired on demand before its members + are read (#188). 6. **Validate.** Per-node validation runs; reserved-attr collisions, missing required attrs, bad enum members all surface here with `ERR_*` codes (see [`fixtures/conformance/ERROR-CODES.json`](../../fixtures/conformance/ERROR-CODES.json)).