Skip to content

meta migrate: aggregate/monitoring read-model views can't be modeled as projections — 4 missing origin/field capabilities #195

Description

@dmealing

Summary

object.projection + origin.* currently synthesizes view DDL for passthrough and count/sum/avg/min/max aggregate shapes. But a common class of admin/monitoring read-model views can't be expressed, so an adopter is forced to keep them as hand-authored, unmodeled views — which meta migrate can't own and meta verify --db can't drift-check (they're invisible/unmanaged).

Because meta migrate processes the whole metadata tree, a single un-expressible view also can't just be "skipped": the projection either derives a base entity + synthesizable body, or the migrate/baseline run aborts with cannot derive the base entity — declare an extends-bound identity (FR-024) and blocks migration of every other entity in the tree. So an adopter with even a few aggregate views can't adopt meta migrate at all until each is either expressible or explicitly excludable.

Concretely, four capabilities are missing. Each is a real column in a shipping adopter's monitoring views:

1. Boolean-any / boolean-all aggregate (bool_or / bool_and)

origin.aggregate @agg supports count | sum | avg | min | max. There's no boolean rollup. The canonical case is a per-group "did any related row fail?" flag:

bool_or(NOT success) AS has_error   -- one row per session/turn

Proposal: add @agg: any / @agg: all (→ bool_or / bool_and) with an optional @filter, or allow count(@filter) > 0 to lower to a boolean.

(generic column names below; the shapes are from real monitoring views but the names are anonymized.)

2. Computed-expression origin (per-row derived column)

No origin expresses a row-level computed column — only passthrough / aggregate / collection. The recurring case is substituting a heavy column with a cheap derived boolean:

(payload_json IS NOT NULL) AS has_payload   -- avoid shipping the heavy JSON text in the list view

Proposal: an origin.computed (or origin.expression) taking a portable, whitelisted expression over the base entity's own fields (IS NULL / IS NOT NULL, comparisons, COALESCE), emitted into the synthesized SELECT. Kept deliberately narrow (not a raw-SQL escape hatch).

3. Correlated latest-row origin (ORDER BY … LIMIT 1 per group)

No origin expresses "the latest related row's column X" — a correlated ORDER BY <ts> DESC LIMIT 1 / lateral. This is extremely common in session/entity summaries:

(SELECT label FROM child_a c WHERE c.parent_id = p.id AND c.is_active ORDER BY c.created_at DESC LIMIT 1) AS current_label
(SELECT score FROM child_b b WHERE b.parent_id = p.id                 ORDER BY b.created_at DESC LIMIT 1) AS latest_score

Proposal: an origin.latest (sibling of origin.aggregate) taking @via (relationship path) + @of (projected column) + @orderBy + optional @filter, lowering to a correlated subquery or DISTINCT ON.

4. Array-column typing for array_agg output

There's no field subtype/attr for a Postgres array column produced by array_agg (e.g. text[]), so an aggregate view that rolls values into an array column can't be projected and the column is dropped from the read model:

array_agg(DISTINCT category) AS categories   -- text[]

(isArray: true models a stored array base-table column, but there's no way to declare that an aggregate origin produces an array column.) Proposal: allow origin.aggregate @agg: arrayAgg (with isArray: true on the projection field) to emit array_agg(...).

Why it matters

Together these are the difference between "the adopter's monitoring/admin read-models are metadata-managed + drift-gated" and "they stay hand-written SQL that verify --db is blind to." The passthrough + basic-aggregate support already shipped covers the simple views; these four cover the aggregate/correlated ones that are otherwise the long tail blocking full meta migrate adoption.

Happy to provide anonymized DDL for each shape and to help with the migrate-ts + per-port work if useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions