Skip to content

feat(docs-site): HTML documentation-site generator wired as meta docs --site#171

Merged
dmealing merged 22 commits into
mainfrom
feat/docs-site
Jul 4, 2026
Merged

feat(docs-site): HTML documentation-site generator wired as meta docs --site#171
dmealing merged 22 commits into
mainfrom
feat/docs-site

Conversation

@dmealing

@dmealing dmealing commented Jul 4, 2026

Copy link
Copy Markdown
Member

Intent

Bring a proven HTML documentation-site generator into the metaobjects monorepo as @metaobjectsdev/docs-site, wired as a 'meta docs --site' surface, in three phases on this branch.

Phase 1 (port): new server/typescript/packages/docs-site package (loader, link-graph, page builders, mermaid emitters, site orchestrator, link-check) + templates + assets + a generic 'acme' fixture + golden test. CLI: a --site surface on 'meta docs' that is ADDITIVE to the markdown --model/--api surfaces (when --site is the only surface flag, markdown is deliberately suppressed via an explicit empty surface list; combined with --model/--api both emit). The site has its OWN model loader, so a site-only run returns before the sdk loadMemory path. Strict-mode type fixes were needed because the monorepo tsconfig is stricter than the source project (exactOptionalPropertyTypes/noUncheckedIndexedAccess) — these are type-only, golden byte-identical.

Phase 2 (shared relationship IR): re-source the site's relationship diagram edges from the metadata relationship PRIMITIVES (deriveM2MFields, findReferenceBetween, MetaRelationship getters), keeping LinkGraph as the graph shell. Deliberate design decisions (approved by the user): the M:N-through junction is KEPT as its own node AND a distinct logical M:N edge is added (faithfulness over decluttering); relationship-vs-bare-FK double edges are de-duplicated (richer relationship edge wins); M:N renders as mermaid }o--o{ (ER) / dashed (flowchart); onDelete/junction fold into the edge label. This surfaced and fixed a REAL bug in the shared SSOT: deriveM2MFields resolved the @through junction by bare name only while @through is written fully-qualified — now FQN-aware (mirrors the existing meta-identity idiom) + a regression test. Verified this was a TS-ONLY deviation: the Python and Java/Kotlin ports already handle FQN @through, so the fix brings TS into sync (ports-sync doctrine satisfied). The acme fixture was deliberately extended with M:N + directed/symmetric self-joins + onDelete, so its golden regenerates.

Phase 3 (scaffold-and-own, ADR-0034): 'meta docs --scaffold-site' copies the site templates+assets into codegen/docs-site/ write-only-if-absent (never clobbers edits); generateSite gains an assetsDir override so owned CSS/JS win; 'meta docs --site' auto-detects codegen/docs-site/ and uses the owned copies (bundled fallback). Deliberately genericized the mermaid CURATED palette: dropped the hardcoded domain-name->color map (a vocabulary carryover from the source project) for a neutral hashed palette — golden stayed byte-identical.

Constraints held throughout: metaobjects is PUBLIC (no private/downstream names, no absolute local paths in anything committed); determinism (byte-identical golden across two regenerations) + link-check (throws on dangling) gates; no new package dependency (primitives come from @metaobjectsdev/metadata, already a dep). Validated end-to-end via the real meta CLI (scaffold->edit->owned-copy-wins) + a scale smoke on an in-repo conformance model (20 pages, 0 dangling). Suites green: docs-site 40/0, cli 366/0, metadata 2097/0, all typecheck clean.

What Changed

  • Added a new @metaobjectsdev/docs-site package (loader, link-graph, page builders, mermaid emitters, site orchestrator, link-check, templates, assets, and an acme golden fixture) that emits a static HTML documentation site from metadata.
  • Wired a --site surface onto meta docs that is additive to the markdown --model/--api surfaces (site-only runs suppress markdown), plus a --scaffold-site command that copies templates/assets into codegen/docs-site/ write-only-if-absent and a generateSite assetsDir override so owned CSS/JS win over the bundled defaults (ADR-0034 scaffold-and-own); --site auto-detects the owned copies.
  • Re-sourced the relationship diagram edges from the metadata relationship primitives (deriveM2MFields/findReferenceBetween), rendering M:N as ER }o--o{/dashed flowchart edges with onDelete/junction labels, and fixed deriveM2MFields to resolve a fully-qualified @through junction by FQN as well as bare name (with a regression test) — bringing TS in line with the Python/Java ports. A Node crash from Bun-only import.meta.dir in site.ts/scaffold.ts was corrected via the fileURLToPath idiom during review.

Risk Assessment

✅ Low: A large but well-bounded new package addition guarded by byte-identical golden tests and a link-check gate, with the only prior error (Node import.meta.dir crash) already fixed and the shared metadata M2M change correctly scoped, tested, and consistent with the other language ports.

Testing

Ran the branch's own gate suites (docs-site 40/0, cli 367/0, metadata M:N regression 15/0), then drove the real meta docs CLI end-to-end against the generic acme fixture to produce a 22-page HTML site. Captured reviewer-visible screenshots via headless Chrome of the home page (Core ERD with the neutral hashed palette), the Order entity page (Fields/Indexes/Validators/Relationships/Inheritance plus the Neighborhood ER diagram rendering the Phase-2 M:N-via-junction edge and onDelete label), the shop package index, and the coverage page. Additionally verified determinism (two regenerations byte-identical), the additive-surface semantics (site-only suppresses markdown; --site --model emits both), and the Phase-3 scaffold-and-own loop (scaffold → owned-CSS-wins → re-scaffold preserves the edit). All checks passed; the worktree stayed clean (all generated output written under /tmp). One background test task showed exit 143, but that was an initial run terminated by a working-directory issue and was superseded by a successful foreground re-run of the same suite.

  • Evidence: Site home page — Core ERD + stats + package cards (meta docs --site) (local file: /tmp/no-mistakes-evidence/01KWQK9VTJYWAPC4PTNESX4SCA/home.png)
  • Evidence: Order entity page — Neighborhood ER diagram showing Phase-2 M:N-via-OrderProduct edge + onDelete label (local file: /tmp/no-mistakes-evidence/01KWQK9VTJYWAPC4PTNESX4SCA/order-entity.png)
  • Evidence: Shop package index page (local file: /tmp/no-mistakes-evidence/01KWQK9VTJYWAPC4PTNESX4SCA/shop-package.png)
  • Evidence: Coverage page — rendered node kinds & attributes (local file: /tmp/no-mistakes-evidence/01KWQK9VTJYWAPC4PTNESX4SCA/coverage.png)
Evidence: Scaffold-and-own + determinism CLI transcript
meta docs --scaffold-site — 11 created, 0 preserved
(edit owned assets/site.css)
meta docs --site — wrote 22 page(s)
generated site.css carries OWNED-THEME-MARKER-EVIDENCE → OWNED CSS WON
meta docs --scaffold-site — 0 created, 11 preserved → PRESERVED hand-edit
Determinism: diff -rq of two regenerations → IDENTICAL (byte-for-byte)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 2 issues found → auto-fixed ✅
  • 🚨 server/typescript/packages/docs-site/src/site.ts:63 - docs-site resolves its bundled templates/assets with import.meta.dir, a Bun-only global that is undefined under Node. resolve(import.meta.dir, "../templates") therefore becomes resolve(undefined, ...) and throws TypeError: The "path" argument must be of type string. Received undefined when the CLI runs from its published Node build (dist/bin/meta.js, produced by tsc, which imports docs-site's dist/index.js default export). Both meta docs --site and meta docs --scaffold-site crash for Node consumers. Tests don't catch it because they run under bun test, which resolves the package's bun export condition to src where import.meta.dir works. The repo's established Node-safe idiom is dirname(fileURLToPath(import.meta.url)) (used in codegen-ts for exactly this Node/compiled-binary compatibility). Affects site.ts:63, site.ts:73, and scaffold.ts:22.
  • ℹ️ server/typescript/packages/docs-site/src/builders/index-data.ts:11 - SiteOptions.core / CoreConfig (pin/exclude/n) is accepted, typed, and threaded through to buildIndexPage(..., { core: opts.core, ... }), but buildIndexPage never reads opts.core — the core-map size (CORE_MAX=28) and seed counts (8/5/3) are hardcoded. The CLI's core: { n: 15 } in emitSite is therefore dead config. Either wire it into buildIndexPage or drop the option to avoid implying it has effect.

🔧 Fix: fix Node import.meta.dir crash and wire docs-site core config
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • bun test packages/docs-site/test → 40 pass / 0 fail (loader, link-graph, mermaid, golden byte-identical, scaffold, link-check)
  • bun test packages/cli/test → 367 pass / 2 skip / 0 fail (docs command surface)
  • bun test packages/metadata/test/relationship-m2m.test.ts → 15 pass / 0 fail (FQN @through junction regression)
  • meta docs <acme> --site via real CLI run() → wrote 22 pages, exit 0; captured screenshots of home/index, Order entity (M:N ER diagram), shop package, coverage
  • Determinism: regenerated the site twice and diff -rq reported byte-identical output
  • Phase 3: meta docs --scaffold-site (11 created) → edited owned assets/site.css → re-ran --site and grepped the generated site.css for the owned marker (owned CSS won) → re-scaffolded and confirmed the hand-edit preserved (0 created / 11 preserved)
  • Additive surfaces: site-only run emits no markdown at the out root; --site --model emits 16 markdown pages AND the site/index.html
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

dmealing and others added 22 commits July 4, 2026 18:17
… surface (3-phase design)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eta docs --site (3 tasks)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tsdev/docs-site

Verbatim port of the model-agnostic generator (loader, link-graph, page builders,
mermaid emitters, site orchestrator, link-check) + templates + assets + generic acme
fixture + tests, wired as a workspace package. 29 tests pass; golden byte-identical.
Phase 1 of docs-site (meta docs --site); CLI wiring in a follow-up task.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e resolution

The monorepo's tsconfig.base.json is stricter than the source bun project
(noUncheckedIndexedAccess + exactOptionalPropertyTypes). Applied minimal
type-only fixes so `tsc -p .` (build) and typecheck pass: non-null assertions
on in-bounds array/match accesses, default-destructure on guaranteed splits,
and `| undefined` on the optional interface props that receive explicit
undefined. No generator behavior change — golden byte-identical, 29 tests pass.

Verified the BUILT dist package resolves templates/assets via
resolve(import.meta.dir, "../templates") from dist/ (18 pages, 0 dangling on
the acme fixture). bun.lock registers the workspace package.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a `--site` surface to `meta docs` that renders the browsable HTML
documentation site via @metaobjectsdev/docs-site, under `<out>/site`.

- `--site` is additive: combined with --model/--api it emits both the markdown
  surfaces AND the site; when it is the ONLY surface flag, markdown emission is
  suppressed (parseDocsArgs passes an explicit empty markdown surface list,
  which resolveDocsConfig honors via `??`).
- The site has its OWN model loader (docs-site loadModel), so a site-only run
  returns before the sdk loadMemory/GenContext path — decoupled, one fewer
  failure surface, and no gen config required.
- `--templates <dir>` is threaded as the docs-site templatesDir override (the
  scaffold-and-own seam; a consumer template of the same basename wins).
- Help text (top-level + `docs --help` FLAGS) documents --site/--model/--api/
  --metamodel; cli.test.ts help snapshot updated for the one changed line.
- docs-command.test.ts covers site-only (markdown suppressed), --site --model
  additive, and site determinism.

CLI typecheck green; 361 pass / 0 fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… relationship IR

Design for re-sourcing the site's relationship edges from the metadata
relationship PRIMITIVES (deriveM2MFields / findReferenceBetween /
MetaRelationship) rather than the lossy buildApiModel output or the
Drizzle-shaped buildRelationMap. Keeps LinkGraph as the graph shell; enriches
relationship edges (M:N-through-junction with the junction kept as a node,
belongs-to direction, directed + symmetric self-joins, onDelete); makes
structural edges inheritance-aware via resolved accessors; dedupes
relationship-vs-bare-FK. Extends the acme fixture (golden regenerates);
additive mermaid connector/dashed-M:N rendering; presentation + gates preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Task-by-task plan: extend the acme fixture (M:N + directed/symmetric self-joins
+ onDelete); extend Ref + belongs-to enrichment + relationship-vs-FK dedupe;
M:N-through-junction edges via deriveM2MFields; }o--o{ / dashed-flowchart
rendering with junction kept as a node; genericize CURATED palette keys; full
verification. Golden regenerated per output-affecting task; determinism +
link-check + typecheck gates at each step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lf-joins + onDelete

Adds Product, OrderProduct, CustomerReferral, CustomerFriend to the acme::shop
fixture: Order.products (M:N via OrderProduct), Order.customer gains
@onDelete: cascade, Customer.referrals (directed self-join via
CustomerReferral, @sourceRefField: referrerId), Customer.friends (symmetric
self-join via CustomerFriend). Golden regenerated (4 new pages), determinism
verified via two independent regenerations (byte-identical).

Also updates load.test.ts and package-index-data.test.ts hardcoded
object name/count lists — pure structural fallout of the 4 new fixture
entities, unrelated to relationship-edge derivation.

link-graph.test.ts's pre-existing backlink/degree assertions on Customer
(refsTo length/degree == 3) are left failing on purpose: they assume the
FK-vs-relationship dedupe that Task 2 of this plan implements.
…ame (deriveM2MFields)

deriveM2MFields resolved the @through junction via root.findObject, which
matches by bare object name only. Since @through is conventionally written
fully-qualified (package::Name) and the loader's own FQN-aware validation
(refMatchesObject) already accepts that form, every FQN @through passed
loading but then threw M2MDerivationError at derivation time, silently
dropping the M:N edge for any caller that swallows the error.

Fall back to the bare suffix after the last "::" when the direct lookup
misses, mirroring the existing resolvedTargetPkField idiom in
meta-identity.ts. Adds a regression test exercising a fully-qualified
@through.
…ymmetric) via deriveM2MFields

Replaces the Task 2 temporary addM2mEdge stub with the real derivation:
resolves the junction FQN via resolveRef, then calls deriveM2MFields to
compute sourceJoinField/targetJoinField (hetero match-by-target-entity,
directed self-join via @sourceRefField, symmetric self-join in declaration
order). A derivation failure (ambiguous self-join) skips the logical edge
rather than throwing out of generation — the junction's two FK edges still
render.

Unblocked by the upstream deriveM2MFields FQN-junction-lookup fix in the
metadata package (prior commit) — @through is conventionally written fully
qualified, which the bare-name-only lookup could not resolve.
…kage into a neutral palette

The CURATED map keyed diagram colors by domain-specific package names carried
over from the source project — a soft vocabulary leak in a public repo. Replace
it with a plain fixed PALETTE array and hash every package's leaf name into a
slot (the same deterministic slot the previously-unmapped packages already
used). No hardcoded domain vocabulary remains, and the fixture golden is
byte-identical (packages that matched a curated key now hash to an equivalent
slot).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ship derivation

Addresses the Task 2 review's Minor: the double-cast (as unknown as MetaObject)
now carries a comment noting the dn.kind === "object" runtime guarantee and why
the unknown bridge is required for tsc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sub-project A (this spec): meta docs --scaffold-site copies templates+assets into
codegen/docs-site/ (ADR-0034 write-only-if-absent); assetsDir override so owned
CSS/JS are used; auto-detect on meta docs --site; docs; validation via the real
meta CLI (scaffold→edit→regen→owned-copy-wins) + a scale/parity smoke against a
substantial model. Documents the sequenced rollout: cross-port deriveM2MFields
FQN sync (Python/Java in-repo), prerelease, and two downstream adopter conversions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…sks, TDD)

Task-by-task: assetsDir override in generateSite; export
SITE_TEMPLATE_NAMES/SITE_ASSET_NAMES/readSiteFile; meta docs --scaffold-site
(copy to codegen/docs-site, write-only-if-absent) + auto-detect owned copies on
--site; docs + real-CLI scaffold→edit→owned-wins validation + a scale smoke
against an in-repo conformance model. Determinism/link-check/typecheck gates per
task.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…te param + test import

Adds the docs-site README documenting `meta docs --scaffold-site` (copy templates+
assets into codegen/docs-site/, edit, regenerate — owned copies win, engine stays a
dep). Folds in two review Minors: emitSite's now-unused `flags` param dropped (+ both
call sites), and an unused mkdirSync import removed from the assetsDir test.

Validated end-to-end via the real meta CLI: --scaffold-site writes 11 files; an edited
owned template AND asset both win in the output; re-scaffold reports 0 created / 11
preserved (never clobbers); and generateSite renders a real in-repo conformance model
to 20 pages with 0 dangling links.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dmealing
dmealing merged commit 6d60744 into main Jul 4, 2026
1 check passed
@dmealing
dmealing deleted the feat/docs-site branch July 4, 2026 22:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant