perf(prefetch): stop calling our own API over the wire during server render

.agents/skills/tool-registry-boundary/SKILL.md

@@ -68,6 +68,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the
 
 Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph.
 
+The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. This catches bloat the registry rule misses — a prefetch importing `listTables` cost the Tables page 444 modules without ever touching `@/tools/registry`.
+
+Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again.
+
 ## How to verify an edge actually got cut
 
 Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph:

.claude/commands/tool-registry-boundary.md

@@ -67,6 +67,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the
 
 Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph.
 
+The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. This catches bloat the registry rule misses — a prefetch importing `listTables` cost the Tables page 444 modules without ever touching `@/tools/registry`.
+
+Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again.
+
 ## How to verify an edge actually got cut
 
 Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph:

.claude/rules/sim-queries.md

@@ -143,6 +143,22 @@ const handler = useCallback(() => {
 }, [data])
 ```
 
+## Server prefetching
+
+A server prefetch fills the *same* cache key a client hook fills, so it must be indistinguishable from a client fetch. Five rules:
+
+1. **Read the data layer, never our own API over HTTP.** A server-to-server call to `/api/...` costs a round trip and a second authentication for data the process can already read. Where the route runs an application use case, call that same use case with a principal from the same auth policy the route declares — not a manager underneath it.
+2. **Match the wire shape the hook caches.** The hook's data is whatever `requestJson(contract, …)` produced, so the seed must equal it. Two traps: a contract field declared `z.coerce.date()` means the hook holds a `Date` where raw route JSON holds a string; a passthrough response schema (`z.custom`) means the hook caches route JSON *verbatim*, so seeding raw rows leaks `Date`s and server-only fields. When the route projects before responding, share that projection — have the route and the prefetch call one function.
+3. **Prove the viewer.** Data-layer reads carry no authorization; the route used to provide it. Resolve the viewer (`getWorkspaceHostContextForViewer`, already `cache`d by the layout so it costs nothing) and return early on failure, caching nothing — the client fetch then reaches the route for the real 403. Never widen what a viewer can see.
+4. **Always `await`.** Only a settled query is dehydrated, so an unawaited prefetch is silently dropped from the payload and the pane waterfalls anyway.
+5. **Don't repeat what the layout already seeded.** `getQueryClient()` builds a new client per server call, so a page re-seeding a layout key is a genuine second read — and `HydrationBoundary` defers an already-seen query to an effect, which SSR never runs, so it never reaches the server render either.
+
+Reuse the hook's exported `staleTime` constant and its key factory; `dehydrate` carries neither options nor `staleTime`, and freshness is per-observer.
+
+Seed with `setQueryData` only when the prefetch must be able to *decline* to create an entry (an empty list that has to fall through to a route's creation path). `prefetchQuery` and `ensureQueryData` always create one.
+
+Keep prefetch imports light. A page prefetch's imports land in that route's server graph, so pulling a barrel to reach one function can drag thousands of modules behind it — `bun run check:tool-registry-boundary` gates this per page.
+
 ## Boundary Types
 
 - Hooks import named type aliases from `@/lib/api/contracts/**` (e.g., `import { listEntitiesContract, type EntityList } from '@/lib/api/contracts/entities'`). Never write `z.input<...>` / `z.output<...>` in hooks, and never `import { z } from 'zod'` in client code.

.cursor/commands/tool-registry-boundary.md

@@ -63,6 +63,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the
 
 Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph.
 
+The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. This catches bloat the registry rule misses — a prefetch importing `listTables` cost the Tables page 444 modules without ever touching `@/tools/registry`.
+
+Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again.
+
 ## How to verify an edge actually got cut
 
 Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph:

apps/sim/app/_shell/providers/get-query-client.ts

@@ -1,12 +1,11 @@
-import { defaultShouldDehydrateQuery, isServer, QueryClient } from '@tanstack/react-query'
+import { isServer, QueryClient } from '@tanstack/react-query'
 import { isDesktopApp } from '@/lib/desktop'
 
 export function makeQueryClient() {
   return new QueryClient({
     defaultOptions: {
       queries: {
         staleTime: 30 * 1000,
-        gcTime: 5 * 60 * 1000,
         // The desktop app window lives for days, so cross-session changes —
         // an admin upgrading your org/workspace role, a workspace you were
         // auto-added to, seat/entitlement changes — would otherwise stay
@@ -18,16 +17,19 @@ export function makeQueryClient() {
         // frequent and noisy. Per-query overrides (e.g. useWorkspaceSchedules
         // pins this off) always win over this default.
         refetchOnWindowFocus: isDesktopApp(),
-        retry: 1,
+        /**
+         * Query core already defaults retries to 0 on the server and 3 in the browser;
+         * only the browser number is ours to change. Stating one value for both would
+         * silently opt server prefetches into a retry, and because the layout awaits
+         * them that spends a retry backoff of document latency on a read whose failure
+         * the client recovers from on its own.
+         */
+        retry: isServer ? 0 : 1,
         retryOnMount: false,
       },
       mutations: {
         retry: false,
       },
-      dehydrate: {
-        shouldDehydrateQuery: (query) =>
-          defaultShouldDehydrateQuery(query) || query.state.status === 'pending',
-      },
     },
   })
 }

apps/sim/app/api/pinned-items/route.ts

@@ -2,40 +2,21 @@ import { db, pinnedItem } from '@sim/db'
 import { createLogger } from '@sim/logger'
 import { getPostgresErrorCode } from '@sim/utils/errors'
 import { generateId } from '@sim/utils/id'
-import { and, eq, ne } from 'drizzle-orm'
 import { type NextRequest, NextResponse } from 'next/server'
 import {
   createPinnedItemContract,
   listPinnedItemsContract,
   type PinnedItemApi,
-  pinnedResourceTypeSchema,
 } from '@/lib/api/contracts'
 import { parseRequest } from '@/lib/api/server'
 import { getSession } from '@/lib/auth'
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
-import { filterToActiveResources, pinnableResourceExists } from '@/lib/pinned-items/resources'
+import { listPinnedItemsForUser } from '@/lib/pinned-items/queries'
+import { pinnableResourceExists } from '@/lib/pinned-items/resources'
 import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils'
 
 const logger = createLogger('PinnedItemsAPI')
 
-/**
- * Narrows a stored row to the wire shape, dropping any row whose `resourceType` this build does
- * not recognise.
- *
- * `pinned_item.resource_type` is plain `text` — deliberately, so the set of pinnable kinds can
- * grow — while the contract is a closed enum. During a rolling deploy an older pod can therefore
- * read a pin a newer one wrote. Returning it would fail response validation and take the WHOLE
- * list down rather than the single row, so the unknown kind is skipped instead.
- *
- * `filterToActiveResources` already drops these as a side effect of not having a table to look
- * them up in; this makes the guarantee explicit and compiler-checked at the wire boundary.
- */
-function toPinnedItemApi(row: typeof pinnedItem.$inferSelect): PinnedItemApi | null {
-  const resourceType = pinnedResourceTypeSchema.safeParse(row.resourceType)
-  if (!resourceType.success) return null
-  return { ...row, resourceType: resourceType.data, pinnedAt: row.pinnedAt.toISOString() }
-}
-
 /** Lists the session user's pinned items in a workspace, optionally filtered to one `resourceType`. */
 export const GET = withRouteHandler(async (request: NextRequest) => {
   const session = await getSession()
@@ -52,30 +33,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
     return NextResponse.json({ error: 'Access denied to this workspace' }, { status: 403 })
   }
 
-  const rows = await db
-    .select()
-    .from(pinnedItem)
-    .where(
-      and(
-        eq(pinnedItem.userId, session.user.id),
-        eq(pinnedItem.workspaceId, workspaceId),
-        /**
-         * A `workspace` pin stores `workspaceId === resourceId`, so it would otherwise
-         * appear in this workspace's unscoped listing as a resource *inside* itself.
-         * It is read from the workspace-list payload instead, so it is excluded here
-         * rather than left for a future unscoped caller to mistake for a real resource.
-         */
-        resourceType
-          ? eq(pinnedItem.resourceType, resourceType)
-          : ne(pinnedItem.resourceType, 'workspace')
-      )
-    )
-
-  const activeRows = await filterToActiveResources(rows, workspaceId)
-
-  const pinnedItems = activeRows
-    .map(toPinnedItemApi)
-    .filter((item): item is PinnedItemApi => item !== null)
+  const pinnedItems = await listPinnedItemsForUser(session.user.id, workspaceId, resourceType)
 
   return NextResponse.json({ pinnedItems })
 })

apps/sim/app/api/table/[tableId]/columns/route.test.ts

@@ -49,10 +49,12 @@ vi.mock('@/lib/table/columns/service', () => ({
   updateColumnOptions: mockUpdateColumnOptions,
   updateColumnType: mockUpdateColumnType,
 }))
+vi.mock('@/lib/table/wire', () => ({
+  normalizeColumn: (c: unknown) => c,
+}))
 vi.mock('@/app/api/table/utils', () => ({
   accessError: () => new Response('denied', { status: 403 }),
   checkAccess: mockCheckAccess,
-  normalizeColumn: (c: unknown) => c,
   orchestrationOutcomeErrorResponse: (
     outcome: { error?: string; errorCode?: OrchestrationErrorCode },
     fallback: string

apps/sim/app/api/table/[tableId]/columns/route.ts

@@ -13,10 +13,10 @@ import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
 import { addTableColumn, deleteColumn } from '@/lib/table'
 import { signalTableSchemaChanged } from '@/lib/table/events'
 import { performUpdateTableColumn } from '@/lib/table/orchestration'
+import { normalizeColumn } from '@/lib/table/wire'
 import {
   accessError,
   checkAccess,
-  normalizeColumn,
   orchestrationOutcomeErrorResponse,
   rootErrorMessage,
   tableLockErrorResponse,

apps/sim/app/api/table/[tableId]/groups/route.test.ts

@@ -54,7 +54,7 @@ vi.mock('@/lib/table/application/groups', () => ({
   updateTableGroupUseCase: mocks.useCases.update,
 }))
 
-vi.mock('@/app/api/table/utils', () => ({
+vi.mock('@/lib/table/wire', () => ({
   normalizeColumn: vi.fn(),
 }))
 

apps/sim/app/api/table/[tableId]/groups/route.ts

@@ -12,7 +12,7 @@ import {
 } from '@/lib/table/application/groups'
 import { tableOperations } from '@/lib/table/application/operations'
 import type { TableDefinition } from '@/lib/table/types'
-import { normalizeColumn } from '@/app/api/table/utils'
+import { normalizeColumn } from '@/lib/table/wire'
 
 const rateLimit = internalRateLimits.none({
   reason: 'Existing authenticated table group mutations have no request-rate policy',

apps/sim/app/api/table/[tableId]/route.test.ts

@@ -51,9 +51,11 @@ vi.mock('@/lib/workspaces/permissions/utils', () => ({
 vi.mock('@/app/api/table/utils', () => ({
   accessError: () => new Response('denied', { status: 403 }),
   checkAccess: mockCheckAccess,
-  normalizeColumn: (column: unknown) => column,
   tableLockErrorResponse: () => null,
 }))
+vi.mock('@/lib/table/wire', () => ({
+  normalizeColumn: (column: unknown) => column,
+}))
 
 import { GET, PATCH } from '@/app/api/table/[tableId]/route'
 

apps/sim/app/api/table/[tableId]/route.ts

@@ -18,11 +18,11 @@ import {
   performUpdateTableLocks,
 } from '@/lib/table/orchestration'
 import { TABLE_LOCK_FLAGS, TABLE_LOCK_KINDS } from '@/lib/table/types'
+import { normalizeColumn } from '@/lib/table/wire'
 import { getWorkspaceWithOwner } from '@/lib/workspaces/permissions/utils'
 import {
   accessError,
   checkAccess,
-  normalizeColumn,
   orchestrationOutcomeErrorResponse,
   tableLockErrorResponse,
 } from '@/app/api/table/utils'

apps/sim/app/api/table/import-csv/route.test.ts

@@ -34,7 +34,6 @@ vi.mock('@/app/api/table/utils', async () => {
   const { asOrchestrationError, messageForOrchestrationError, statusForOrchestrationError } =
     await import('@/lib/core/orchestration/types')
   return {
-    normalizeColumn: (column: unknown) => column,
     csvProxyBodyCapResponse: () => null,
     multipartErrorResponse: (error: { code: string; message: string }) =>
       NextResponse.json(

apps/sim/app/api/table/route.ts

@@ -15,8 +15,9 @@ import {
   type TableSchema,
   type TableScope,
 } from '@/lib/table'
+import { normalizeColumn, toTableListItem, toWireTimestamp } from '@/lib/table/wire'
 import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils'
-import { normalizeColumn, orchestrationErrorResponse } from '@/app/api/table/utils'
+import { orchestrationErrorResponse } from '@/app/api/table/utils'
 
 const logger = createLogger('TableAPI')
 
@@ -140,14 +141,8 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
           maxRows: table.maxRows,
           folderId: table.folderId ?? null,
           locks: table.locks,
-          createdAt:
-            table.createdAt instanceof Date
-              ? table.createdAt.toISOString()
-              : String(table.createdAt),
-          updatedAt:
-            table.updatedAt instanceof Date
-              ? table.updatedAt.toISOString()
-              : String(table.updatedAt),
+          createdAt: toWireTimestamp(table.createdAt),
+          updatedAt: toWireTimestamp(table.updatedAt),
         },
         message: 'Table created successfully',
       },
@@ -198,41 +193,10 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
 
     logger.info(`[${requestId}] Listed ${tables.length} tables in workspace ${params.workspaceId}`)
 
-    const responseTables = tables.map((t) => {
-      const schemaData = t.schema as TableSchema
-      return {
-        id: t.id,
-        name: t.name,
-        description: t.description,
-        schema: {
-          columns: schemaData.columns.map(normalizeColumn),
-        },
-        rowCount: t.rowCount,
-        maxRows: t.maxRows,
-        locks: t.locks,
-        workspaceId: t.workspaceId,
-        folderId: t.folderId ?? null,
-        createdBy: t.createdBy,
-        createdAt: t.createdAt instanceof Date ? t.createdAt.toISOString() : String(t.createdAt),
-        updatedAt: t.updatedAt instanceof Date ? t.updatedAt.toISOString() : String(t.updatedAt),
-        archivedAt:
-          t.archivedAt instanceof Date
-            ? t.archivedAt.toISOString()
-            : t.archivedAt
-              ? String(t.archivedAt)
-              : null,
-        jobStatus: t.jobStatus ?? null,
-        jobId: t.jobId ?? null,
-        jobType: t.jobType ?? null,
-        jobError: t.jobError ?? null,
-        jobRowsProcessed: t.jobRowsProcessed ?? 0,
-      }
-    })
-
     return NextResponse.json({
       success: true,
       data: {
-        tables: responseTables,
+        tables: tables.map(toTableListItem),
         totalCount: tables.length,
       },
     })

apps/sim/app/api/table/utils.ts

@@ -17,7 +17,6 @@ import {
 import type { MultipartError } from '@/lib/core/utils/multipart'
 import type { ColumnDefinition, Filter, TableDefinition, TablePredicate } from '@/lib/table'
 import { buildFilterClause, getTableById, TableQueryValidationError } from '@/lib/table'
-import { typeMetadataOf } from '@/lib/table/column-types'
 import { USER_TABLE_ROWS_SQL_NAME } from '@/lib/table/constants'
 import { TableLockedError } from '@/lib/table/mutation-locks'
 import { isTablePredicate } from '@/lib/table/query-builder/converters'
@@ -358,21 +357,3 @@ export function serverErrorResponse(message = 'Internal server error') {
 export const CreateColumnSchema = createTableColumnBodySchema
 export const UpdateColumnSchema = updateTableColumnBodySchema
 export const DeleteColumnSchema = deleteTableColumnBodySchema
-
-export function normalizeColumn(
-  col: ColumnDefinition
-): ColumnDefinition & { required: boolean; unique: boolean } {
-  return {
-    // Preserve the stable column id — it's the row-data storage key, so dropping
-    // it makes clients fall back to `name` and miss id-keyed cell values.
-    ...(col.id ? { id: col.id } : {}),
-    name: col.name,
-    type: col.type,
-    required: col.required ?? false,
-    unique: col.unique ?? false,
-    ...(col.workflowGroupId ? { workflowGroupId: col.workflowGroupId } : {}),
-    // Type-specific metadata is forwarded generically: naming keys here meant a
-    // new type's metadata was stored server-side but silently never returned.
-    ...typeMetadataOf(col),
-  }
-}

apps/sim/app/api/v1/admin/responses.ts

@@ -51,9 +51,7 @@ export function errorResponse(
   return NextResponse.json(body, { status })
 }
 
-// =============================================================================
 // Common Error Responses
-// =============================================================================
 
 export function unauthorizedResponse(message = 'Authentication required'): NextResponse {
   return errorResponse('UNAUTHORIZED', message, 401)