Skip to content

feat(codegen-ts): fix projection/driver-type codegen and add advisory verify pass#80

Merged
dmealing merged 3 commits into
mainfrom
feat/codegen-friction-model-first
Jun 27, 2026
Merged

feat(codegen-ts): fix projection/driver-type codegen and add advisory verify pass#80
dmealing merged 3 commits into
mainfrom
feat/codegen-friction-model-first

Conversation

@dmealing

Copy link
Copy Markdown
Member

Intent

Make MetaObjects usable end-to-end by an AI coding agent. Live agent builds showed the agent WANTS to use MetaObjects but abandons it and hand-rolls because of real codegen friction; this bundle fixes that friction and strengthens guidance.

CODEGEN FIXES (codegen-ts):

  • queries-file: a projection (object.projection, read-only/view-backed) now emits READ-ONLY queries (findById + list from the view), NOT table-style create/update importing a nonexistent InsertSchema. Fixes a real TS2724 that made a declared projection fail to compile, forcing a revert to a hand-rolled aggregate. Mirrors the existing isProjection guard in routes-file.
  • queries-file: generated SQLite Db type is now BaseSQLiteDatabase<'sync' | 'async', unknown> so it accepts better-sqlite3 (sync) AND libsql/Turso/D1 (async). The old <'async'> pin rejected better-sqlite3, forcing 'db: any' casts.
  • queries-file + callable-file: generated Postgres Db type is the base PgDatabase<PgQueryResultHKT, ...> every PG driver extends (node-postgres, postgres.js, Neon, Vercel, pglite), not just NodePgDatabase.
    DELIBERATE: all three driver-type changes were VERIFIED by compiling generated query shapes against the REAL drivers, not just regenerated. Golden + conformance snapshots regenerated to match; diffs confined to those type lines.

VERIFY-AS-TEACHER (cli): meta verify AND meta gen run an ADVISORY pass flagging hand-rolled aggregates / money-as-float / CHECK-IN enums, naming the construct that models them. DELIBERATE: warnings only, NEVER changes exit code (bias to under-flag; >15% false-positive rate is a project kill criterion). Opt out via --no-antipatterns / META_NO_ANTIPATTERNS=1.

AGENT-CONTEXT SKILLS (intentional prompting changes the user explicitly requested):

  • authoring: a model-first / generate-first operating principle at the top, meta types vocabulary search folded in. Grounded in EF Code-First / Terraform / model-driven framing; an A/B probe showed no regression.
  • codegen: 'write your own generators' is now a first-class section (real apps rarely use OOTB codegen as-is) using the accurate Generator / perEntity / EmittedFile API. Preserved docs(agent-context): teach granular codegen control and the runtime→Fastify mount API #78's @kind:view note during rebase.
  • verify: run meta verify before declaring a build done.

The one pre-existing failing cli test (detectPackageManager) is unrelated/environmental (pm-detect.ts / migrate-ux.test.ts, never touched). No deps changed.

What Changed

  • codegen-ts queries-file: a read-only object.projection now emits read-only queries (findById + list from the view) instead of table-style create/update that imported a nonexistent <Name>InsertSchema (a real TS2724 break), mirroring the existing isProjection guard in routes-file. The generated SQLite Db type is widened to BaseSQLiteDatabase<'sync' | 'async', unknown> (accepts better-sqlite3 plus libsql/Turso/D1) and the Postgres Db type to the base PgDatabase<PgQueryResultHKT, ...> (every PG driver, not just node-postgres) across queries-file and callable-file; golden + conformance snapshots regenerated to match.
  • cli verify-as-teacher: meta verify and meta gen run an advisory anti-pattern scan (anti-patterns.ts) flagging hand-rolled aggregates, money-as-float, and CHECK-IN enums and naming the metadata construct that models them. Warnings only — never alters exit code — and both commands honor the --no-antipatterns flag and META_NO_ANTIPATTERNS=1 opt-outs.
  • agent-context skills & docs: added a model-first/generate-first operating principle and meta types vocabulary search to the authoring skill, a first-class "write your own generators" section (using the real Generator/perEntity/EmittedFile API) to the codegen skill, and a "run verify before declaring done" note to the verify skill, with corresponding cli/codegen-ts README and docs/features/cli.md updates.

Risk Assessment

✅ Low: Well-bounded codegen type-widening and an advisory-only CLI pass that never affects exit codes; the one prior actionable finding (opt-out asymmetry) is now correctly fixed symmetrically across both commands, the projection queries path faithfully mirrors the existing routes-file guard, and the driver-type changes satisfy Drizzle's real type-param constraints.

Testing

Baseline targeted tests (projection queries, anti-patterns, verify integration, arg parsing) all pass, and the full codegen-ts suite (904/0) confirms the regenerated golden + conformance snapshots match the new driver-type and projection codegen. Beyond unit tests I produced product-level evidence: the actual generated projection query file for both dialects (proving the TS2724-causing InsertSchema import is gone and queries are read-only over the view), grep of the committed golden snapshots showing the new BaseSQLiteDatabase&lt;\&#34;sync\&#34; | \&#34;async\&#34;, unknown&gt; and PgDatabase&lt;PgQueryResultHKT, ...&gt; Db types, and live meta CLI transcripts of the verify-as-teacher advisory firing (all three rules, exit code unchanged, both opt-outs working) on both meta verify and meta gen. No rendered-UI surface is involved (CLI + codegen output), so CLI transcripts and generated-code artifacts are the appropriate end-user evidence. The only failing test is the pre-existing, environmental detectPackageManager case the author called out — its source is byte-identical at base and target and untouched here (root cause: a bun.lock exists up-tree). Transient generated/sample artifacts were removed and the worktree is clean.

Evidence: Generated projection queries (read-only, both dialects, no InsertSchema)

ProgramSummary.queries.ts (sqlite): type Db = BaseSQLiteDatabase<"sync" | "async", unknown>; import { type ProgramSummary, programSummaryView } ...; findProgramSummaryById + listProgramSummaries select from(programSummaryView). NO InsertSchema, NO create/update/delete. Postgres variant: type Db = PgDatabase<PgQueryResultHKT, Record<string, never>>.


========== ProgramSummary.queries.ts (dialect=sqlite, projection/read-only) ==========
// @generated by @metaobjectsdev/codegen-ts — DO NOT EDIT.
// Source metadata: ProgramSummary (ProgramSummary) — projection (read-only)
// Customize via ProgramSummary.extra.ts in this directory (additional queries, custom logic).
import { eq } from "drizzle-orm";

import type { BaseSQLiteDatabase } from "drizzle-orm/sqlite-core";
type Db = BaseSQLiteDatabase<"sync" | "async", unknown>;

import { type ProgramSummary, programSummaryView } from "./ProgramSummary";
export async function findProgramSummaryById(db: Db, id: number): Promise<ProgramSummary | null> {
  const [row] = await db.select().from(programSummaryView).where(eq(programSummaryView.id, id)).limit(1);
  return row ?? null;
}

export async function listProgramSummaries(
  db: Db,
  opts?: { limit?: number; offset?: number },
): Promise<ProgramSummary[]> {
  let q = db.select().from(programSummaryView).$dynamic();
  if (opts?.limit !== undefined) {
    q = q.limit(opts.limit);
  }
  if (opts?.offset !== undefined) {
    q = q.offset(opts.offset);
  }
  return q;
}


========== ProgramSummary.queries.ts (dialect=postgres, projection/read-only) ==========
// @generated by @metaobjectsdev/codegen-ts — DO NOT EDIT.
// Source metadata: ProgramSummary (ProgramSummary) — projection (read-only)
// Customize via ProgramSummary.extra.ts in this directory (additional queries, custom logic).
import { eq } from "drizzle-orm";

import type { PgDatabase, PgQueryResultHKT } from "drizzle-orm/pg-core";
type Db = PgDatabase<PgQueryResultHKT, Record<string, never>>;

import { type ProgramSummary, programSummaryView } from "./ProgramSummary";
export async function findProgramSummaryById(db: Db, id: number): Promise<ProgramSummary | null> {
  const [row] = await db.select().from(programSummaryView).where(eq(programSummaryView.id, id)).limit(1);
  return row ?? null;
}

export async function listProgramSummaries(
  db: Db,
  opts?: { limit?: number; offset?: number },
): Promise<ProgramSummary[]> {
  let q = db.select().from(programSummaryView).$dynamic();
  if (opts?.limit !== undefined) {
    q = q.limit(opts.limit);
  }
  if (opts?.offset !== undefined) {
    q = q.offset(opts.offset);
  }
  return q;
}
Evidence: verify-as-teacher CLI transcript (3 rules, exit 0, both opt-outs)

meta verify — 5 place(s) hand-roll what MetaObjects can model (advisory): db/schema.sql:3 → field.enum; src/pricing.ts:2,5 → field.currency; src/stats.ts:2,5 → origin.aggregate. [exit 0]. --no-antipatterns and META_NO_ANTIPATTERNS=1 both produce no advisory lines, exit 0.

################################################################
# $ meta verify   (advisory verify-as-teacher pass)
################################################################
meta verify — running --templates (default). Explicit subverbs: --templates (prompt drift), --db (schema drift), --codegen (codegen drift).
meta verify — no template.* nodes found; nothing to check.
meta: meta verify — 5 place(s) hand-roll what MetaObjects can model (advisory — does not fail the build):
meta:   db/schema.sql:3 — a fixed value set enforced by a CHECK — MetaObjects has field.enum (generates the type union + validation + the CHECK for you). Run `meta types field.enum`.
meta:   src/pricing.ts:2 — money handled as a float / hand-rolled minor units — MetaObjects has field.currency (integer minor units stored, formatted client-side). Run `meta types field.currency`.
meta:   src/pricing.ts:5 — money handled as a float / hand-rolled minor units — MetaObjects has field.currency (integer minor units stored, formatted client-side). Run `meta types field.currency`.
meta:   src/stats.ts:2 — you're computing an aggregate by hand — MetaObjects derives it: declare an object.projection with an origin.aggregate child and call its generated query. Run `meta types origin.aggregate`.
meta:   src/stats.ts:5 — you're computing an aggregate by hand — MetaObjects derives it: declare an object.projection with an origin.aggregate child and call its generated query. Run `meta types origin.aggregate`.
[exit code: 0]

################################################################
# $ meta verify --no-antipatterns   (opt-out flag)
################################################################
meta verify — running --templates (default). Explicit subverbs: --templates (prompt drift), --db (schema drift), --codegen (codegen drift).
meta verify — no template.* nodes found; nothing to check.
[exit code: 0]

################################################################
# $ META_NO_ANTIPATTERNS=1 meta verify   (env opt-out)
################################################################
meta verify — running --templates (default). Explicit subverbs: --templates (prompt drift), --db (schema drift), --codegen (codegen drift).
meta verify — no template.* nodes found; nothing to check.
[exit code: 0]
Evidence: meta gen advisory CLI transcript (advisory on the write run + opt-out)

meta gen — 1 place(s) hand-roll what MetaObjects can model (advisory): src/reports.ts:2 → origin.aggregate, Run meta types origin.aggregate. [exit 0]. META_NO_ANTIPATTERNS=1 meta gen: no advisory line.

################################################################
# $ meta gen   (real write run — advisory reaches the command agents run)
################################################################
gen[4]{file,status}:
  ../../home/doug/.no-mistakes/worktrees/4a36a911fd68/01KW1YVYM475JKHQ0YNCJ2AG03/server/typescript/src/generated/Product.ts,new
  ../../home/doug/.no-mistakes/worktrees/4a36a911fd68/01KW1YVYM475JKHQ0YNCJ2AG03/server/typescript/src/generated/Product.queries.ts,new
  ../../home/doug/.no-mistakes/worktrees/4a36a911fd68/01KW1YVYM475JKHQ0YNCJ2AG03/server/typescript/src/generated/Product.routes.ts,new
  ../../home/doug/.no-mistakes/worktrees/4a36a911fd68/01KW1YVYM475JKHQ0YNCJ2AG03/server/typescript/src/generated/index.ts,new
summary: 4 written
help[2]: typecheck the generated code with `npx tsc`,run schema with `meta migrate --db <url> --slug <name>`
meta: 
meta gen — 1 place(s) hand-roll what MetaObjects can model (advisory — declaring the construct lets codegen own it):
meta:   src/reports.ts:2 — you're computing an aggregate by hand — MetaObjects derives it: declare an object.projection with an origin.aggregate child and call its generated query. Run `meta types origin.aggregate`.
[exit code: 0]

################################################################
# $ META_NO_ANTIPATTERNS=1 meta gen   (opt-out honored on gen too)
################################################################
gen[4]{file,status}:
  ../../home/doug/.no-mistakes/worktrees/4a36a911fd68/01KW1YVYM475JKHQ0YNCJ2AG03/server/typescript/src/generated/Product.ts,unchanged
  ../../home/doug/.no-mistakes/worktrees/4a36a911fd68/01KW1YVYM475JKHQ0YNCJ2AG03/server/typescript/src/generated/Product.queries.ts,unchanged
  ../../home/doug/.no-mistakes/worktrees/4a36a911fd68/01KW1YVYM475JKHQ0YNCJ2AG03/server/typescript/src/generated/Product.routes.ts,unchanged
  ../../home/doug/.no-mistakes/worktrees/4a36a911fd68/01KW1YVYM475JKHQ0YNCJ2AG03/server/typescript/src/generated/index.ts,unchanged
summary: 4 unchanged
help[2]: typecheck the generated code with `npx tsc`,run schema with `meta migrate --db <url> --slug <name>`
[exit code: 0]
Evidence: Advisory teaching pointer resolves (meta types origin.aggregate)

origin.aggregate — A count/sum/avg/min/max (@agg) computed over a column (@of) reached along a relationship (@agg, @of, @via). 4 matches.

=== the advisory points the agent at: meta types origin.aggregate ===
origin.aggregate             A count/sum/avg/min/max (@agg) computed over a column (@of) reached along a relationship…  (@agg, @of, @via)
origin.aggregate @agg        Aggregate function applied over the relationship path: count, sum, avg, min, or max.
origin.aggregate @of         Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durati…
origin.aggregate @via        Dotted relationship path from the base entity to the aggregated rows (e.g. 'Program.week…

4 matches.
- Outcome: ⚠️ 1 info across 1 run (4m58s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 3 issues found → auto-fixed ✅
  • ⚠️ server/typescript/packages/cli/src/commands/verify.ts:115 - The two advisory opt-outs are split asymmetrically across the two commands, contradicting the documented contract ('Opt out via --no-antipatterns / META_NO_ANTIPATTERNS=1'). meta verify honors only the --no-antipatterns flag and ignores the META_NO_ANTIPATTERNS=1 env var (verify.ts:115 checks flags.noAntipatterns only). Conversely meta gen honors only the env var (gen.ts:134), and because parseGenArgs uses strict:true without registering --no-antipatterns, running meta gen --no-antipatterns hard-errors the entire gen command rather than silencing the advisory. A user who sets META_NO_ANTIPATTERNS=1 globally still gets nagged on verify, and a user who learned --no-antipatterns from meta verify --help crashes meta gen. Make both commands honor both opt-outs (verify should also check the env var; gen should accept the flag), and document the gen advisory + its opt-out in meta gen help (currently undocumented).
  • ℹ️ server/typescript/packages/cli/src/lib/anti-patterns.ts:86 - scanSourceForAntiPatterns does a synchronous recursive walk reading every .ts/.tsx/.js/.sql file under cwd on every meta gen (non-dry-run) and meta verify. It is bounded (skips node_modules/build/dot-dirs, files >512KB, wrapped in try/catch) so impact is small for normal projects, but on a large repo it adds noticeable latency to commands that previously did no source scan. Acceptable as-is; noted as a tradeoff.
  • ℹ️ server/typescript/packages/cli/src/lib/anti-patterns.ts:48 - The hand-rolled-aggregate reduce rule (/=&gt;[^;]*\+/ or /,\s*0\s*\)/) will flag non-numeric reduces such as string concatenation arr.reduce((a,b)=&gt;a+b) or any .reduce(fn, 0)-shaped accumulator that isn't a money/metric sum. This is a known false-positive surface against the project's >15% false-positive kill criterion, but the pass is advisory-only and deliberately biased to under-flag, so it is the author's intentional tradeoff — flagged for awareness only.

🔧 Fix: honor both anti-pattern opt-outs on meta gen and verify
✅ Re-checked - no issues remain.

⚠️ **Test** - 1 info
  • ℹ️ server/typescript/packages/cli/test/migrate-ux.test.ts - Pre-existing, unrelated test failure: detectPackageManager &gt; returns bun as default when no lockfile is found (migrate-ux.test.ts) fails because the worktree has a bun.lock at the repo root, so detection walks up and finds it instead of returning the no-lockfile default. The source (pm-detect.ts) and test are byte-identical at the base and target commits and untouched by this change — exactly the environmental failure the author flagged. Not fixed (out of scope / environmental).
  • bun test packages/codegen-ts/test/projection/queries-file.test.ts packages/codegen-ts/test/templates/queries-file.test.ts packages/cli/test/anti-patterns.test.ts packages/cli/test/integration/verify.test.ts packages/cli/test/unit/args-gen.test.ts packages/cli/test/unit/args-verify.test.ts — 49 pass
  • bun test packages/codegen-ts — 904 pass / 0 fail (validates regenerated golden + conformance snapshots incl. driver-type lines)
  • bun test packages/cli — 342 pass, 1 fail (pre-existing environmental detectPackageManager failure, untouched by this change)
  • Generated ProgramSummary.queries.ts for sqlite + postgres via renderQueriesFile and confirmed read-only shape, no InsertSchema, select from programSummaryView, and the new Db types
  • Ran meta verify --cwd &lt;sample&gt; against a project with AVG/reduce-sum, money-float, and CHECK-IN: all 3 rules warned, exit 0; --no-antipatterns and META_NO_ANTIPATTERNS=1 both suppressed
  • Ran meta gen --cwd &lt;sample&gt; on a real scaffolded project: advisory fired on the write run, exit 0, opt-out honored
  • meta types origin.aggregate — confirmed the advisory's teaching pointer resolves to real vocabulary
  • Confirmed meta init scaffolds the updated .claude/skills/ docs into a consumer project
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

dmealing and others added 3 commits June 26, 2026 08:32
…s-teacher, model-first guidance

Codegen fixes (the friction that forced agents to abandon MetaObjects and hand-roll):
- queries-file: projections now emit READ-ONLY queries (findById + list from the
  view) — no `<Name>InsertSchema` import / create/update. Fixes TS2724 that made a
  declared projection fail to compile, forcing a revert to hand-rolled aggregates.
- queries-file: SQLite `Db` accepts BOTH sync (better-sqlite3) and async
  (libsql/Turso/D1) drivers (`BaseSQLiteDatabase<"sync" | "async", unknown>`) — the
  pinned `<"async">` rejected better-sqlite3, the most common driver.
- queries-file + callable-file: Postgres `Db` is the base `PgDatabase<PgQueryResultHKT,
  …>` every PG driver extends (node-postgres, postgres.js, Neon, Vercel, pglite),
  not just `NodePgDatabase`.
  All three verified by compiling generated query shapes against the real drivers.

verify-as-teacher (cli): `meta verify` + `meta gen` run an advisory pass that flags
hand-rolled aggregates / money-as-float / CHECK-IN enums and names the construct that
models them. Warnings only — never fails the build. `--no-antipatterns` opts out.

Agent-context skills:
- authoring: a model-first / generate-first operating principle (declare metadata,
  generate persistence/DAO/API/UI; hand-write only what metadata can't express),
  with `meta types` vocabulary search folded in.
- codegen: write-your-own-generators is now a first-class section (real apps rarely
  use OOTB codegen as-is) with the accurate `Generator` / `perEntity` API.
- verify: run `meta verify` before calling a build done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuZWKnWzYGVnESijL7uuky
@dmealing
dmealing merged commit 835d6be into main Jun 27, 2026
29 checks passed
@dmealing
dmealing deleted the feat/codegen-friction-model-first branch June 27, 2026 01:54
dmealing added a commit that referenced this pull request Jun 27, 2026
…#81)

* fix(codegen-ts): projection read-type mirrors view-column nullability + cross-port compile guards

A non-required (nullable) projection field generated a nullable Drizzle view column
but a NON-null Zod read type, so the generated projection query returned `T | null`
into a non-null `<Name>` field and failed to compile under strict TS (the likely
source of the "null is not assignable to number" friction seen building real apps).
Fix: the projection Zod read schema now appends `.nullable()` whenever the view
column is not `.notNull()`, so the read type matches what `db.select().from(view)`
actually yields (and is more correct — a nullable derived field is typed nullable).

Closes the cross-port test gap that let the original PR #80 projection bug ship:
only C# compiled generated projection code. Adds a real compile/structure guard to
every port that lacked one — TS (tsc), Python (exec), Java (in-process javac of the
DTO), Kotlin (kotlin-compile-testing of the data class) — each exercising a
projection with a required id + a non-required derived field. Test-only for
Java/Kotlin/Python (those ports were already correct); the TS change is the fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LuZWKnWzYGVnESijL7uuky

* no-mistakes(document): docs: projection read-type nullability mirrors view column

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
dmealing added a commit that referenced this pull request Jul 16, 2026
… parity)

Kotlin has no metamodel registry of its own (it rides the Java metadata module), so its
#195 work is codegen typing only — mirrors the committed native-typing reference (aff49ce).

- KotlinGenUtil.kt: originGuaranteedNonNull — true iff the field carries origin.aggregate
  @agg:any|all|collect (COALESCE-guaranteed non-null: any→false, all→true, collect→[]);
  first/computed excluded (stay nullable). Reads own children (ADR-0039/0029).
- KotlinEntityGenerator.kt + KotlinExposedTableGenerator.kt: joined into the projection
  data-class + Exposed view-column nullable decision (nullable = !required && !originGuaranteedNonNull),
  keeping the read type and the view column consistent (the PR-#80 invariant).
- KotlinPayloadGenerator.kt (the cross-port payload-typing reference): extended the origin
  dispatch — @agg:any|all → Boolean, @agg:collect → List<T> (T = @Of element type), new
  origin.computed → declared subType (nullable), new origin.first → @Of source type (nullable).
- KotlinPayloadGeneratorTest.kt + KotlinProjectionCompileTest.kt: new tests (payload dispatch
  types; projection DTO + Exposed column nullability precision + a real Kotlin compile).

Verified: mvn -pl codegen-kotlin test 282 run / 0 fail, BUILD SUCCESS; compiles under
allWarningsAsErrors; the in-test generated projection compiles. Rides on the committed Java
metadata validation (547c698).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NGQ7oSuNcjhsMHWwZzhBwr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant