feat!: dbColumnType slim-and-derive — Phase 1 (derive arrays + text default, drop uuid_array/text_array + @kind:text)

docs/superpowers/specs/2026-06-29-dbcolumntype-slim-and-derive-design.md

@@ -0,0 +1,92 @@
+# Slim the `@dbColumnType` vocabulary — derive what's derivable (cross-port)
+
+**Goal.** Apply the project's own principle ("pattern-derivable from metadata = codegen,
+never specify; never invent an attr for what's already there", ADR-0023) to the
+`@dbColumnType` escape hatch. Most of its values encode information already present in
+`field.subType` + `isArray` + `maxLength` and should be **derived, not declared**. This
+is a breaking metamodel-vocabulary change, scoped to the **pre-1.0 consolidation window**
+([ADR-0035](../../../spec/decisions/ADR-0035-one-zero-stability-commitment-and-version-unification.md)).
+
+**The verdicts being implemented** (from the design conversation):
+1. `text_array` → **remove**; derive native `text[]` from `field.string` + `isArray`.
+2. `uuid_array` → **remove**; derive native `uuid[]` from `field.uuid` + `isArray`.
+3. `uuid` (scalar) → **narrow**: `field.uuid` is the path; keep `dbColumnType:uuid` only as a documented "uuid column + string native" exception.
+4. `text` → **not an option; fix the default**: `field.string` no-`maxLength` → `text`; with `maxLength` → `varchar(n)`. Remove the JVM `dbColumnType:text` value and the Kotlin `@kind:text` hack.
+5. `jsonb` → **keep** (open-bag; not derivable).
+6. `timestamp_with_tz` → **flip the default**: `field.timestamp` defaults to `timestamptz`; the rare without-tz case uses `dbColumnType:timestamp`. (**Phase 2** — native-type-changing.)
+
+---
+
+## Architectural keys (from the cross-port grounding)
+
+- **The canonical Postgres DDL is TS-owned** — `migrate-ts` emits `canonical/schema.postgres.sql`, and every port provisions its test DB by executing it (ADR-0015). So the column-type *decision* lives in `migrate-ts`; each port's ORM/runtime mapper (TS codegen, C# EF, Kotlin Exposed, Java OMDB JDBC, Python pg8000) must **agree** with the canonical, and the persistence-conformance corpus is the cross-port gate.
+- **The `@dbColumnType` value-set is NOT cross-port-identical today** and `registry-conformance` does not gate attribute *value-sets* (only types/subtypes/attrs). The JVM ports carry an extra `text` value; only Kotlin implements `uuid_array`/`text_array`. **Add a cross-port value-set gate** so this can't drift again.
+
+### Current divergence (Postgres)
+| | TS (canonical) | C# (EF) | Kotlin (Exposed) | Java (OMDB/DTO) | Python (Pydantic/runtime) |
+|---|---|---|---|---|---|
+| `string` no-maxLen | `text` | `text` | `varchar(255)` | `varchar(50)` | `str` (no DDL) |
+| `string isArray` | `text[]` | `text[]` | not wired | not wired / throws | `list[str]` |
+| `field.uuid` scalar | exists | `Guid`+`uuid` | `UUID`+`uuid` | `UUID`(DTO)+`uuid` | `uuid.UUID` |
+| `field.uuid isArray` | derive | `Guid[]`/`uuid[]` | falls to scalar | **throws** | `list[uuid.UUID]` |
+| `uuid_array`/`text_array` override | vestigial | vestigial | implemented | vestigial | vestigial |
+| `dbColumnType: text` value | absent | absent | present+impl | present+vestigial | absent |
+| `@kind: text` field hack | — | — | yes (unregistered attr) | — | — |
+| `timestamp` default | no-tz | no-tz (forced) | no-tz | no-tz | no-tz (naive) |
+
+---
+
+## Target behavior (identical across all 5 ports)
+
+### Derived — NO attribute (the metadata already says it)
+- `field.string`, no `maxLength` → **`text`** (Postgres). With `maxLength: N` → **`varchar(N)`**.
+- `field.string`, `isArray: true` → native **`text[]`** + the port's `List<String>`/`string[]`/`list[str]` native type.
+- `field.uuid` scalar → **`uuid`** column + native UUID type (`java.util.UUID` / `Guid` / `uuid.UUID`; TS `string`).
+- `field.uuid`, `isArray: true` → native **`uuid[]`** + `List<UUID>`/`Guid[]`/`list[uuid.UUID]`.
+- `field.timestamp` → **`timestamptz`** (Phase 2; native `Instant`/`DateTimeOffset`/aware `datetime`).
+
+### Declared — the only surviving `@dbColumnType` values
+The closed set becomes **`{ uuid, jsonb, timestamp }`** (down from 6, identical in every port):
+- **`uuid`** (on `field.string`) — "uuid *column*, `string` native type." The narrow escape hatch for when an app wants a string-typed id over a uuid column. `field.uuid` is preferred and documented as such.
+- **`jsonb`** (on `field.string`) — the open JSON bag (parsed-value contract, #98). Not derivable.
+- **`timestamp`** (on `field.timestamp`) — the rare opt-out from the new `timestamptz` default → emits `TIMESTAMP WITHOUT TIME ZONE` + the port's naive/local temporal type. (Phase 2.)
+
+**Removed values:** `uuid_array`, `text_array`, `timestamp_with_tz` (now the default), and the JVM-only `text`. The Kotlin `@kind:text` field hack is removed (it's an unregistered attr that already fails strict verify).
+
+---
+
+## Phasing
+
+**Phase 1 (vocab slim + derive arrays + text default; no native-type change):**
+- Vocab: remove `uuid_array`, `text_array`, JVM `text`. Keep `uuid`, `jsonb`, `timestamp_with_tz` (still valid in Phase 1).
+- Derive `text[]` (field.string+isArray) and `uuid[]` (field.uuid+isArray) — *build* Java OMDB native-array binding (new) + Kotlin native-array default + ensure migrate-ts canonical emits them + C#/TS already do; Python emits `list[...]` (no DDL).
+- Fix JVM `field.string` default → `text` (Java OMDB stop hardcoding VARCHAR(50/255); Kotlin Exposed `text()` for no-maxLength). Remove the `@kind:text` hack.
+- Add the cross-port `@dbColumnType` value-set conformance gate.
+- **No native-type changes** → smaller blast radius; ships green first.
+
+**Phase 2 (timestamp default flip — native-type-changing):**
+- `field.timestamp` default → `timestamptz` (canonical migrate-ts + every port's temporal native type: `Instant`/`DateTimeOffset`/aware `datetime`). Remove `timestamp_with_tz`; add `dbColumnType: timestamp` (without-tz override). Update the wire/normalization contract + ADR-0019 temporal note as needed.
+- Migration: adopters drop `dbColumnType:timestamp_with_tz` (now the default) and add `dbColumnType:timestamp` only where they genuinely want a naive column.
+
+---
+
+## Per-port delta summary (Phase 1)
+
+- **migrate-ts (TS, canonical DDL):** confirm `field.string`+isArray → `text[]`, `field.uuid`+isArray → `uuid[]`; remove `uuid_array`/`text_array` recognition; field.string no-maxLength → `text` (already). The canonical `schema.postgres.sql` is the source of truth.
+- **TS codegen (codegen-ts):** remove `uuid_array`/`text_array` from `db-constants`; field.uuid+isArray → `uuid[]` (.array() with uuid element); field.string+isArray already `text[]`. (TS already correct on text default + scalar arrays.)
+- **C# (EF):** remove the two array values; ensure `field.uuid`+isArray → `Guid[]`/`uuid[]` (PrimitiveCollection); string default already `text`.
+- **Kotlin (Exposed):** remove the two array values + the `text` value + the `@kind:text` hack; make `isArray` (string→`text[]`, uuid→`uuid[]`) the **default** (currently only via the now-removed overrides); `field.string` no-maxLength → `text()` not `varchar(255)`.
+- **Java (OMDB + SpringTypeMapper):** remove the two array values + the vestigial `text` value; **build native `text[]`/`uuid[]` JDBC binding in OMDB** (new — currently throws/varchar(50)); `field.string` no-maxLength → align with canonical `text` (stop hardcoding VARCHAR(50)); `field.uuid`+isArray DTO → `List<UUID>` (currently throws).
+- **Python (Pydantic + runtime):** remove the two array values; `field.uuid`+isArray already `list[uuid.UUID]`; runtime write-coercion for native arrays if needed (no DDL).
+
+## Conformance
+- **Value-set gate (new):** a cross-port test asserting each port's `@dbColumnType` legal-value-set == `{uuid, jsonb, timestamp_with_tz}` (Phase 1) / `{uuid, jsonb, timestamp}` (Phase 2), byte-identical.
+- **persistence-conformance:** add/extend `op: roundtrip` scenarios for `field.string isArray` (→ `text[]`) and `field.uuid isArray` (→ `uuid[]`) so every port's runtime write+read is gated against the canonical native-array schema. Update `asset-native-column-types.yaml` (which uses the removed values) to the derived forms.
+- **Migrate-ts expected-schema:** the canonical `schema.postgres.sql` is regenerated and is the cross-port source of truth; assert it emits `text`/`text[]`/`uuid[]` for the derived cases.
+
+## Migration (adopter impact)
+- `dbColumnType:uuid_array` / `text_array` → drop the attr; the array-ness is derived from `isArray` (model the field `isArray: true`).
+- `dbColumnType:text` (JVM) / `@kind:text` → drop it; `text` is the default for a no-`maxLength` string.
+- `dbColumnType:uuid` (262× in one large audited adopter) → keep working (still valid), but prefer migrating to `field.uuid` where a native UUID type is wanted (this also fixes the `String`↔`UUID` divergence the audit flagged).
+- **Phase 2:** drop `dbColumnType:timestamp_with_tz` (now default); add `dbColumnType:timestamp` only for genuine without-tz columns.
+- The 0.x→1.0 migration guide ([docs/1.0-readiness.md](../../1.0-readiness.md) B2) carries this.

server/csharp/MetaObjects.Codegen.Tests/ScalarArrayCodegenTests.cs

@@ -48,6 +48,19 @@ public class ScalarArrayCodegenTests
     ]}}
     """;
 
+    // An entity with a field.uuid isArray:true — array-ness is derived, not via
+    // the removed dbColumnType:uuid_array value (Phase 1 slim-and-derive).
+    private const string UuidArrayModel = """
+    { "metadata.root": { "package": "acme", "children": [
+      { "object.entity": { "name": "Widget", "children": [
+        { "source.rdb": { "@table": "widgets" } },
+        { "field.long": { "name": "id" } },
+        { "field.uuid": { "name": "refs", "isArray": true } },
+        { "identity.primary": { "@fields": "id" } }
+      ]}}
+    ]}}
+    """;
+
     // An entity with both a scalar enum (no isArray) and an enum-array, so we can
     // assert the scalar path is unchanged (regression guard).
     private const string MixedEnumModel = """
@@ -188,6 +201,38 @@ public void Enum_array_field_does_not_emit_has_conversion_without_element_type_i
         Assert.DoesNotContain("Statuses).HasConversion<string>()", dbCtx);
     }
 
+    // -------------------------------------------------------------------------
+    // field.uuid isArray — Phase 1 slim-and-derive: array-ness via isArray:true,
+    // not the removed dbColumnType:uuid_array value.
+    // ScalarFor("uuid") == "Guid", so the existing scalar-array path handles this.
+    // -------------------------------------------------------------------------
+
+    [Fact]
+    public void Uuid_array_field_emits_ICollection_Guid_property()
+    {
+        var ctx = Ctx(Load(UuidArrayModel));
+        var src = Assert.Single(new EntityGenerator().Generate(ctx)).Content;
+
+        // field.uuid isArray:true → ICollection<Guid> with List<Guid> initializer.
+        Assert.Contains("public ICollection<Guid> Refs { get; set; } = new List<Guid>();", src);
+        // Guard: the scalar form must NOT appear.
+        Assert.DoesNotContain("public Guid? Refs", src);
+        Assert.DoesNotContain("public Guid Refs", src);
+    }
+
+    [Fact]
+    public void Uuid_array_field_emits_PrimitiveCollection_in_dbcontext()
+    {
+        var ctx = Ctx(Load(UuidArrayModel));
+        var dbCtx = Assert.Single(new DbContextGenerator().Generate(ctx)).Content;
+
+        // EF Core 8 primitive collection API for the uuid array (derived, not via
+        // the removed dbColumnType:uuid_array value).
+        Assert.Contains(
+            "modelBuilder.Entity<Widget>().PrimitiveCollection(x => x.Refs);",
+            dbCtx);
+    }
+
     // -------------------------------------------------------------------------
     // Compile check — generated scalar-array entity must be valid C#
     // -------------------------------------------------------------------------
@@ -208,6 +253,14 @@ public void Generated_enum_array_entity_compiles()
         AssertCompiles(src, "enumarray");
     }
 
+    [Fact]
+    public void Generated_uuid_array_entity_compiles()
+    {
+        var ctx = Ctx(Load(UuidArrayModel));
+        var src = Assert.Single(new EntityGenerator().Generate(ctx)).Content;
+        AssertCompiles(src, "uuidarray");
+    }
+
     // -------------------------------------------------------------------------
     // Helpers
     // -------------------------------------------------------------------------

server/csharp/MetaObjects.Conformance.Tests/R6Plan2LoaderTests.cs

@@ -134,6 +134,26 @@ public void DbColumnType_unknown_value_is_ERR_BAD_ATTR_VALUE()
         Assert.Contains("allowed", err.Message);
     }
 
+    // ------------------------------------------------------------------------
+    // Phase 1 slim-and-derive: uuid_array + text_array are removed values.
+    // They now fall through Rule 1 (unknown value) → ERR_BAD_ATTR_VALUE.
+    // Array-ness is derived from field.uuid/field.string isArray:true instead.
+    // ------------------------------------------------------------------------
+
+    [Theory]
+    [InlineData("uuid_array")]
+    [InlineData("text_array")]
+    public void DbColumnType_removed_array_values_are_ERR_BAD_ATTR_VALUE(string removedValue)
+    {
+        var json = Pairing(FIELD_SUBTYPE_STRING, removedValue);
+        var res = LoadJson(json);
+        var err = res.Errors.FirstOrDefault(e =>
+            e.Code == ErrorCode.ERR_BAD_ATTR_VALUE && e.Message.Contains("dbColumnType"));
+        Assert.NotNull(err);
+        Assert.Contains(removedValue, err!.Message);
+        Assert.Contains("allowed", err.Message);
+    }
+
     // ------------------------------------------------------------------------
     // Plan 2b — own-only: an INHERITED @dbColumnType is not re-validated against
     // the inheriting field's subtype (mirrors the field.enum own-only policy).

server/csharp/MetaObjects/Loader/ValidationPasses.cs

@@ -1854,8 +1854,7 @@ private static void WalkDbColumnType(MetaData node, List<MetaError> errors)
                 // Rule 2: legal (subtype × value) pairing.
                 var requiredSubType = value switch
                 {
-                    DB_COLUMN_TYPE_UUID or DB_COLUMN_TYPE_JSONB
-                        or DB_COLUMN_TYPE_UUID_ARRAY or DB_COLUMN_TYPE_TEXT_ARRAY => FIELD_SUBTYPE_STRING,
+                    DB_COLUMN_TYPE_UUID or DB_COLUMN_TYPE_JSONB => FIELD_SUBTYPE_STRING,
                     DB_COLUMN_TYPE_TIMESTAMP_TZ => FIELD_SUBTYPE_TIMESTAMP,
                     _ => null, // unreachable (Rule 1)
                 };

server/csharp/MetaObjects/Persistence/Db/DbConstants.cs

@@ -51,6 +51,8 @@ public static class DbConstants
     /// Physical DB column-type override on a field (<c>@dbColumnType</c>). Closed value
     /// set: <see cref="DB_COLUMN_TYPE_UUID"/> / <see cref="DB_COLUMN_TYPE_JSONB"/> /
     /// <see cref="DB_COLUMN_TYPE_TIMESTAMP_TZ"/>.
+    /// Array-ness is derived from <c>isArray: true</c> on the field, not from a separate
+    /// <c>uuid_array</c>/<c>text_array</c> value (removed in Phase 1 of the slim-and-derive pass).
     /// </summary>
     public const string FIELD_ATTR_DB_COLUMN_TYPE = "dbColumnType";
 
@@ -60,18 +62,12 @@ public static class DbConstants
     public const string DB_COLUMN_TYPE_JSONB        = "jsonb";
     /// <summary><c>@dbColumnType: timestamp_with_tz</c> — <c>timestamp with time zone</c> column (legal on field.timestamp).</summary>
     public const string DB_COLUMN_TYPE_TIMESTAMP_TZ = "timestamp_with_tz";
-    /// <summary><c>@dbColumnType: uuid_array</c> — native Postgres <c>uuid[]</c> array column (legal on field.string).</summary>
-    public const string DB_COLUMN_TYPE_UUID_ARRAY   = "uuid_array";
-    /// <summary><c>@dbColumnType: text_array</c> — native Postgres <c>text[]</c> array column (legal on field.string).</summary>
-    public const string DB_COLUMN_TYPE_TEXT_ARRAY   = "text_array";
 
     /// <summary>The closed set of legal <c>@dbColumnType</c> values.</summary>
     public static readonly IReadOnlyList<string> VALID_DB_COLUMN_TYPES = new[]
     {
         DB_COLUMN_TYPE_UUID,
         DB_COLUMN_TYPE_JSONB,
         DB_COLUMN_TYPE_TIMESTAMP_TZ,
-        DB_COLUMN_TYPE_UUID_ARRAY,
-        DB_COLUMN_TYPE_TEXT_ARRAY,
     };
 }