feat: typed value-object jsonb columns work end-to-end across all ports

[dmealing]

Intent

Typed value-object jsonb columns (field.object @storage:jsonb, single AND @isarray array-of-VO) work end-to-end across all persistence ports, gated by a new cross-port array-of-VO roundtrip conformance scenario. (Re-run: the prior run's review PASSED and fixed a consumer-doc gap in docs/ports/kotlin.md; its test step's actual work completed green but the flaky zai provider dropped the stream at the very end — pinned to OpenRouter now. Local branch synced to the review-fix commit 883969a.)

Three runtime-PROVEN bugs + one architectural refactor:

1. KOTLIN (codegen-kotlin) — Fable-researched, Doug-approved REFACTOR to Jackson, dropping decorative kotlinx @serializable from entity/value/projection classes (no consumer used kotlinx — all Jackson; the design spec calls @serializable decorative). The typed jsonb column previously emitted VO.serializer() which needs the kotlinx compiler plugin, and once that plugin is on EVERY class with a UUID/Instant/BigDecimal/URI/Inet/temporal field fails to compile. Now: plain data classes + a generated per-package MetaJsonbMapper.kt (Jackson) via Exposed's lambda jsonb overload — no plugin, matching Java(Gson)/C#(System.Text.Json)/TS/Python. @serializable KEPT only on prompt payloads + enums (genuinely kotlinx-decoded by the FR-006 output parser). New gate: compile-WITHOUT-the-plugin + Testcontainers-PG roundtrip of a GENERATED typed-jsonb table. Supersedes the unmerged fix/kotlin-exposed-jsonb-object-column.
2. C# (MetaObjects.Codegen) — array-of-VO jsonb was BROKEN (EF model finalization threw 'ICollection must be a non-interface reference type'). DbContextGenerator emits .OwnsMany when field.ResolvedIsArray(); a coupled empty-[]-vs-null nullability defect fixed.
3. JAVA (OMDB) — array-of-VO jsonb was a GAP ('Expected BEGIN_OBJECT but was BEGIN_ARRAY'). GenericSQLDriver deserializeJsonb branches on isArray (TypeToken<List>) + setObjectArray; RoundtripWriter authors a List insert.

DELIBERATE (treat as intended): the persistence-conformance oracle's hand-written Kotlin AllTypesTable.kt keeps a raw-JSON-String identity codec (the runner is port-agnostic String-based); the generated typed Jackson form is proven by GeneratedTypedJsonbRoundTripTest. All ports use the RESOLVING array accessor (ADR-0039). A 4-angle simplifier pass ran (2nd commit: Kotlin support-file-emit + codec-FQN dedup).

All ports green: TS 27/0, C# 1402/0, Java OMDB 48/0 + persistence 24/0, Kotlin 272/0 + integration 97/0 (Testcontainers PG). Maven + NuGet line change; batches with merged-unpublished #185 for release.

What Changed

• Kotlin codegen-kotlin switched typed jsonb VO columns off kotlinx @Serializable + VO.serializer() (which required the compiler plugin and failed to compile on any class with UUID/Instant/BigDecimal/URI/Inet/temporal fields) onto plain data classes plus a generated per-package MetaJsonbMapper.kt using Jackson through Exposed's lambda jsonb overload; @Serializable is retained only on prompt payloads + enums (genuinely kotlinx-decoded by the FR-006 output parser). A new GeneratedTypedJsonbRoundTripTest proves the generated typed-jsonb table round-trips against Testcontainers PG with no kotlinx plugin on the compile path.
• Fixed array-of-value-object jsonb (field.object @storage:jsonb @isArray) round-tripping in three ports: C# DbContextGenerator/EntityGenerator emit .OwnsMany(...).ToJson (with a coupled empty-[]-vs-null nullability fix) and WriteCoercion coerces the array; Java OMDB GenericSQLDriver.deserializeJsonb branches on isArray (TypeToken<List<VO>>) plus setObjectArray, and RoundtripWriter authors a List<ValueObject> insert; Python _coerce_write_value json.dumpss the list before binding (pg8000 binds a native list as a Postgres ARRAY literal the JSONB column rejects).
• Added a cross-port roundtrip-all-types persistence-conformance scenario: a new AllTypes entity carries one field of every persistable field.* subtype including the array-of-VO labels jsonb column (written as 2-element / empty-[] / single-element arrays across three rows), gating writes through each port's runtime/ORM write codec — not raw SQL — against Testcontainers Postgres.

Risk Assessment

⚠️ Medium: No material bugs found — the cross-port jsonb work is well-gated (compile-without-plugin + Testcontainers round-trip + conformance corpus) and the simplifier is behavior-preserving — but it carries a consumer-facing breaking change (dropping kotlinx @serializable from generated entity/value/projection classes) plus a new required consumer runtime dependency (Jackson databind/module-kotlin/jsr310), so it is more than cosmetic and warrants the human ack the intent already records.

Testing

The Python jsonb array-of-VO write-codec fix is verified working end-to-end: the previously-failing roundtrip-all-types and update-delete-all-types persistence-conformance scenarios now pass against Testcontainers Postgres, the full 24-scenario persistence suite is green, and a runtime evidence script confirms the labels array-of-VO is written through the ObjectManager write codec and stored as a JSON array on a jsonb column (read back identically); a before/after demo shows the raw-list bind failing with 22P02 pre-fix vs json.dumps binding correctly post-fix. The only Python suite failures (2 staleness-nudge tests) are pre-existing and unrelated — only object_manager.py changed between base and target. Working tree left clean.

Evidence: Python array-of-VO jsonb runtime roundtrip (evidence script output)

== Field metadata (resolved, ADR-0039) == labels: subtype=object isArray=True storage='jsonb' settings: subtype=object isArray=False storage='jsonb' == Write+read through ObjectManager runtime == written labels = [{'key': 'alpha', 'weight': 10}, {'key': 'beta', 'weight': 20}] read-back labels= [{'key': 'alpha', 'weight': 10}, {'key': 'beta', 'weight': 20}] == Raw JSONB column value in Postgres == pg_typeof(labels) = jsonb raw labels column = [{'key': 'alpha', 'weight': 10}, {'key': 'beta', 'weight': 20}] PASS: array-of-VO jsonb (labels) + single-VO jsonb (settings) round-tripped; labels persisted as a JSON array on a jsonb column.

== Field metadata (resolved, ADR-0039) ==
  labels:   subtype=object isArray=True storage='jsonb'
  settings: subtype=object isArray=False storage='jsonb'

== Write+read through ObjectManager runtime ==
  written labels  = [{'key': 'alpha', 'weight': 10}, {'key': 'beta', 'weight': 20}]
  read-back labels= [{'key': 'alpha', 'weight': 10}, {'key': 'beta', 'weight': 20}]
  written settings= {'theme': 'dark', 'notify': True, 'retries': 3}
  read-back settings= {'theme': 'dark', 'notify': True, 'retries': 3}

== Raw JSONB column value in Postgres ==
  pg_typeof(labels) = jsonb
  raw labels column = [{'key': 'alpha', 'weight': 10}, {'key': 'beta', 'weight': 20}] (parsed: [{'key': 'alpha', 'weight': 10}, {'key': 'beta', 'weight': 20}])
  raw settings column = {'theme': 'dark', 'notify': True, 'retries': 3}

PASS: array-of-VO jsonb (labels) + single-VO jsonb (settings) round-tripped; labels persisted as a JSON array on a jsonb column.

Evidence: Before/after jsonb array-of-VO bind demo

=== BEFORE (pre-fix: native list passed through to jsonb column) === RESULT: raw-list bind FAILED with SQLSTATE 22P02 === AFTER (post-fix: json.dumps(list) -> JSON text bound to jsonb column) === AFTER-RESULT: json.dumps(list) bind OK; stored labels = [{'key': 'alpha', 'weight': 10}, {'key': 'beta', 'weight': 20}] | pg_typeof = jsonb

Demonstrates the array-of-VO jsonb write codec fix in server/python object_manager.py
(_coerce_write_value). Same Testcontainers PG, same committed schema, same labels
value (a Python list of Label value-objects). Only difference: how the list is
handed to pg8000 for the jsonb column.

=== BEFORE (pre-fix behavior: native Python list passed through to the jsonb column,
            the old "dict/list passes through natively" comment) ===
RESULT: raw-list bind FAILED with SQLSTATE 22P02

=== AFTER (post-fix behavior: json.dumps(list) -> JSON text bound to jsonb column) ===
AFTER-RESULT: json.dumps(list) bind OK; stored labels = [{'key': 'alpha', 'weight': 10}, {'key': 'beta', 'weight': 20}] | pg_typeof = jsonb

Evidence: Persistence conformance (all-types scenarios) result

============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-9.0.3, pluggy-1.6.0 -- /home/doug/.no-mistakes/worktrees/4a36a911fd68/01KX3EGT9AAZKHDCG03HD6ACXX/server/python/.venv/bin/python
cachedir: /dev/shm/pytest-cache
rootdir: /home/doug/.no-mistakes/worktrees/4a36a911fd68/01KX3EGT9AAZKHDCG03HD6ACXX/server/python
configfile: pyproject.toml
plugins: anyio-4.13.0
collecting ... collected 24 items / 22 deselected / 2 selected

tests/integration/test_query_scenarios.py::test_query_scenario[roundtrip-all-types] PASSED [ 50%]
tests/integration/test_query_scenarios.py::test_query_scenario[update-delete-all-types] PASSED [100%]

======================= 2 passed, 22 deselected in 7.00s =======================

Evidence: Evidence script (reproducible)

"""Evidence: array-of-VO jsonb column write+read through the Python ObjectManager runtime.

Boots a fresh Testcontainers Postgres, applies the committed TS-produced schema
verbatim, loads the canonical metadata, then writes an AllTypes row whose
`labels` field is a field.object @storage:jsonb isArray:true (array of the Label
value object) and `settings` is a single-VO jsonb field.object. Reads the row
back through the runtime AND inspects the raw JSONB column value stored in the DB
to prove the array form (a JSON array `[ {...}, {...} ]`) was persisted — the gap
that previously failed with PG 22P02 when pg8000 bound a Python list as a
Postgres ARRAY literal.

Run: cd server/python && uv run python _evidence_array_of_vo_jsonb.py
"""
from __future__ import annotations

import json
import sys
from contextlib import closing
from pathlib import Path

import pg8000.dbapi as pg8000

from metaobjects import load_directory
from metaobjects.runtime import ObjectManager, PostgresDriver
from metaobjects.runtime.object_manager import _column_of, _q

# Reuse the suite's container + schema helpers so this exercises the SAME
# infrastructure the conformance gate does.
sys.path.insert(0, str(Path("tests").resolve()))
from integration.postgres_container import PostgresContainer  # noqa: E402
from integration.query_runner import _apply_schema_artifact  # noqa: E402

CORPUS = Path(__file__).resolve().parents[2] / "fixtures" / "persistence-conformance"
SCHEMA = CORPUS / "canonical" / "schema.postgres.sql"


def main() -> None:
    result = load_directory(CORPUS / "canonical")
    if result.errors:
        raise RuntimeError("metadata did not load cleanly: " + "; ".join(e.message for e in result.errors))
    root = result.root
    # Find the AllTypes table + its labels/settings columns from metadata
    # (top-level entity — metadata.root is never extended, own == effective).
    alltypes = next(c for c in root.own_children() if getattr(c, "name", None) == "AllTypes")
    labels_f = alltypes.find_field("labels")
    settings_f = alltypes.find_field("settings")
    print("== Field metadata (resolved, ADR-0039) ==")
    print(f"  labels:   subtype={labels_f.sub_type} isArray={labels_f.is_array} "
          f"storage={labels_f.get_meta_attr('storage')!r}")
    print(f"  settings: subtype={settings_f.sub_type} isArray={settings_f.is_array} "
          f"storage={settings_f.get_meta_attr('storage')!r}")

    row = {
        "id": "00000000-0000-0000-0000-000000000001",
        "sVal": "hello",
        "iVal": 42,
        "lVal": 9007199254740991,
        "dVal": 0.125,
        "fVal": 1.5,
        "decVal": "123.456",
        "bVal": True,
        "dateVal": "2026-06-03",
        "timeVal": "14:30:00.123",
        "tsVal": "2026-06-03T14:30:00.123",
        "tsTzVal": "2026-06-03T14:30:00.123Z",
        "moneyVal": 199900,
        "enumVal": "MEDIUM",
        "uuidVal": "AAAAAAAA-AAAA-4AAA-8AAA-AAAAAAAAAAAA",
        "uriVal": "https://example.com/a/b?q=1",
        "inetVal": "192.168.1.1",
        "inet6Val": "2001:db8::1",
        # single-VO jsonb (dict)
        "settings": {"theme": "dark", "notify": True, "retries": 3},
        # array-of-VO jsonb (list) — the case that used to fail with 22P02
        "labels": [
            {"key": "alpha", "weight": 10},
            {"key": "beta", "weight": 20},
        ],
    }

    with PostgresContainer() as pg:
        info = pg.info()
        _apply_schema_artifact(info, SCHEMA)
        with closing(pg8000.connect(
            host=info.host, port=info.port, user=info.user,
            password=info.password, database=info.database,
        )) as conn:
            driver = PostgresDriver(conn)
            om = ObjectManager(root, driver)
            created = om.create("AllTypes", row)
            pk = om.primary_key_field("AllTypes")
            read_back = om.find_by_id("AllTypes", created[pk])

            # Inspect the RAW jsonb column value straight from the DB, using the
            # runtime's own table/column-name derivation (single source of truth).
            table = om._table_name(alltypes)
            labels_col = _column_of(alltypes.find_field("labels"))
            settings_col = _column_of(alltypes.find_field("settings"))
            pk_col = _column_of(alltypes.find_field(pk))
            cur = conn.cursor()
            cur.execute(
                f"SELECT {_q(labels_col)}, {_q(settings_col)}, pg_typeof({_q(labels_col)}) "
                f"FROM {_q(table)} WHERE {_q(pk_col)} = %s",
                [created[pk]],
            )
            raw_labels, raw_settings, labels_type = cur.fetchone()

    print("\n== Write+read through ObjectManager runtime ==")
    print(f"  written labels  = {row['labels']}")
    print(f"  read-back labels= {read_back['labels']}")
    print(f"  written settings= {row['settings']}")
    print(f"  read-back settings= {read_back['settings']}")

    print("\n== Raw JSONB column value in Postgres ==")
    print(f"  pg_typeof(labels) = {labels_type}")
    print(f"  raw labels column = {raw_labels!r} (parsed: {json.loads(raw_labels) if isinstance(raw_labels, str) else raw_labels})")
    print(f"  raw settings column = {raw_settings!r}")

    # Assertions: the array-of-VO survived a real write+read and was stored as a JSON array.
    assert read_back["labels"] == row["labels"], "array-of-VO labels did not round-trip!"
    assert read_back["settings"] == row["settings"], "single-VO settings did not round-trip!"
    parsed = json.loads(raw_labels) if isinstance(raw_labels, str) else raw_labels
    assert isinstance(parsed, list) and len(parsed) == 2, "labels column not a 2-element JSON array!"
    assert parsed[0] == {"key": "alpha", "weight": 10}, "first label element mismatch"
    assert str(labels_type) == "jsonb", f"labels column type is {labels_type}, expected jsonb"
    print("\nPASS: array-of-VO jsonb (labels) + single-VO jsonb (settings) round-tripped; "
          "labels persisted as a JSON array on a jsonb column.")


if __name__ == "__main__":
    main()

- Outcome: ⚠️ 1 info across 2 runs (50m2s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - medium risk

✅ No issues found.

⚠️ **Test** - 1 info

• 🚨 server/python/src/metaobjects/runtime/object_manager.py:696 - Python port fails the new cross-port array-of-VO jsonb roundtrip gate (roundtrip-all-types). The labels field (field.object @storage:jsonb isArray:true, JSONB column) was added to the shared scenario in this change; Python's _coerce_write_value passes a field.object list through expecting pg8000 to 'bind a dict/list to the jsonb column natively.' That holds for a dict (single-VO settings works) but NOT for a list: pg8000 adapts a Python list as a Postgres ARRAY literal {...,...} (each element json-stringified), which the JSONB column rejects with 22P02 'invalid input syntax for type json' on parameter $19. There is no isArray branch for jsonb objects (only for native uuid[] columns). The Java OMDB and C# ports received the analogous array-of-VO jsonb fix in THIS change (GenericSQLDriver.deserializeJsonb isArray->List<VO> + setObjectArray; DbContextGenerator .OwnsMany(...).ToJson); Python got no such fix and the scenario is not deferred (EXPECTED_FAILURES = {}), so Python's persistence-conformance is red on roundtrip-all-types — contradicting the change's 'across all persistence ports' headline. Likely fix: for an isArray field.object @storage:jsonb, json.dumps(value) to a JSON array string and bind that.
• mvn -pl codegen-kotlin -am install -DskipTests (install Kotlin codegen + upstream deps to .m2)
• cd server/java/integration-tests-kotlin && mvn test -Dtest='GeneratedTypedJsonbRoundTripTest' (headline Kotlin gate: generated typed jsonb, no kotlinx plugin, Testcontainers PG) — 1/0
• cd server/java/integration-tests-kotlin && mvn test (full Kotlin integration suite) — 97/0
• cd server/java && mvn -pl codegen-kotlin test (Kotlin codegen unit/snapshot tests) — 272/0
• cd server/java && mvn -pl omdb test (Java OMDB suite incl testTypedJsonbArrayRoundTrip + testJsonbEmptyAndSingleArrayRoundTrip) — 48/0
• cd server/java/integration-tests && mvn test -Dtest='QueryScenarioTests' (Java persistence-conformance incl roundtrip-all-types, Testcontainers PG) — 24/0
• cd server/csharp && dotnet build MetaObjects.sln
• cd server/csharp && dotnet test MetaObjects.Codegen.Tests --filter FullyQualifiedName~ObjectFieldCodegenTests (C# OwnsMany codegen regression) — 5/0
• cd server/csharp && dotnet test MetaObjects.IntegrationTests --filter FullyQualifiedName~QueryScenarioTests (C# conformance incl roundtrip-all-types, Testcontainers PG) — 24/0
• cd repo-root && bun install; cd server/typescript/packages/integration-tests && bun test --test-name-pattern roundtrip-all-types test/query.test.ts (TS conformance, Testcontainers PG) — 1/0 (23 filtered)
• cd server/python && uv sync --group dev --extra integration; uv run pytest tests/integration/test_query_scenarios.py -k roundtrip-all-types (Python conformance, Testcontainers PG) — 1 FAILED

🔧 Fix: fix Python jsonb write codec for array-of-VO roundtrip
1 info still open:

• ℹ️ server/python/tests/codegen/test_cli_staleness_nudge.py:68 - Two Python tests (test_gen_nudges_on_stale_manifest, test_verify_nudges_on_stale_manifest) fail, but this is pre-existing and unrelated to the jsonb change under test. The CLI source emits npx meta agent-docs --server python while the tests assert the string metaobjects agent-docs. Between base commit 644d27f and target 1de6b67, the ONLY file changed under server/python is src/metaobjects/runtime/object_manager.py (the jsonb codec); cli.py, agent_context/, and test_cli_staleness_nudge.py are byte-identical across base→target, so these failures exist identically on the base commit and are not introduced or affected by this change.
• git show 1de6b676 — reviewed the fix: adds import json and the jsonb-storage field.object/field.map serialization branch in server/python/src/metaobjects/runtime/object_manager.py _coerce_write_value (~line 711)
• uv run pytest tests/integration/test_query_scenarios.py -k 'roundtrip-all-types or update-delete-all-types' -v — both array-of-VO/all-types scenarios PASS
• uv run pytest tests/integration/test_query_scenarios.py -v — full 24-scenario persistence-conformance suite, all PASS (119s)
• uv run pytest tests/ — full Python suite: 1405 passed, 2 failed (pre-existing staleness-nudge CLI message mismatch, unrelated to the jsonb fix)
• git diff 644d27f0 1de6b676 --name-only -- server/python/ — confirmed ONLY object_manager.py changed; cli.py/agent_context/test_cli_staleness_nudge.py byte-identical base→target, so the 2 failures are pre-existing
• Evidence script (python_array_of_vo_jsonb_evidence.py): boots Testcontainers PG, applies committed schema, writes an AllTypes row with labels (field.object @storage:jsonb isArray:true array-of-VO) + settings (single-VO jsonb) through ObjectManager.create(), reads back via find_by_id, and inspects the raw JSONB column — labels stored as a 2-element JSON array on a jsonb column, round-trips exactly
• Before/after raw-bind demo: native Python list -> jsonb column fails with SQLSTATE 22P02 'invalid input syntax for type json' (pre-fix); json.dumps(list) -> jsonb column stores [{key:alpha,weight:10},{key:beta,weight:20}] as jsonb (post-fix)

🔧 **Document** - 1 issue found → auto-fixed ✅

• ℹ️ server/java/codegen-kotlin/README.md:199 - README states "75 tests in this module (mvn -pl codegen-kotlin test)"; this branch added at least one new test (emits plain data class without Serializable annotation in KotlinEntityGeneratorTest), so the count is now stale. Could not verify the exact current total without running the Maven suite — a human should run mvn -pl codegen-kotlin test and update the number.

🔧 Fix: docs(kotlin): fix stale test count and @serializable example
✅ Re-checked - no issues remain.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.