diff --git a/.agents/skills/tool-registry-boundary/SKILL.md b/.agents/skills/tool-registry-boundary/SKILL.md index a1463f1c8fa..6e1caaf0a64 100644 --- a/.agents/skills/tool-registry-boundary/SKILL.md +++ b/.agents/skills/tool-registry-boundary/SKILL.md @@ -68,6 +68,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph. +The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. This catches bloat the registry rule misses — a prefetch importing `listTables` cost the Tables page 444 modules without ever touching `@/tools/registry`. + +Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again. + ## How to verify an edge actually got cut Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph: diff --git a/.claude/commands/tool-registry-boundary.md b/.claude/commands/tool-registry-boundary.md index 676fa256cbc..da6758efe3e 100644 --- a/.claude/commands/tool-registry-boundary.md +++ b/.claude/commands/tool-registry-boundary.md @@ -67,6 +67,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph. +The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. This catches bloat the registry rule misses — a prefetch importing `listTables` cost the Tables page 444 modules without ever touching `@/tools/registry`. + +Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again. + ## How to verify an edge actually got cut Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph: diff --git a/.claude/rules/sim-queries.md b/.claude/rules/sim-queries.md index 14acca8f2cb..a71ee448dde 100644 --- a/.claude/rules/sim-queries.md +++ b/.claude/rules/sim-queries.md @@ -143,6 +143,22 @@ const handler = useCallback(() => { }, [data]) ``` +## Server prefetching + +A server prefetch fills the *same* cache key a client hook fills, so it must be indistinguishable from a client fetch. Five rules: + +1. **Read the data layer, never our own API over HTTP.** A server-to-server call to `/api/...` costs a round trip and a second authentication for data the process can already read. Where the route runs an application use case, call that same use case with a principal from the same auth policy the route declares — not a manager underneath it. +2. **Match the wire shape the hook caches.** The hook's data is whatever `requestJson(contract, …)` produced, so the seed must equal it. Two traps: a contract field declared `z.coerce.date()` means the hook holds a `Date` where raw route JSON holds a string; a passthrough response schema (`z.custom`) means the hook caches route JSON *verbatim*, so seeding raw rows leaks `Date`s and server-only fields. When the route projects before responding, share that projection — have the route and the prefetch call one function. +3. **Prove the viewer.** Data-layer reads carry no authorization; the route used to provide it. Resolve the viewer (`getWorkspaceHostContextForViewer`, already `cache`d by the layout so it costs nothing) and return early on failure, caching nothing — the client fetch then reaches the route for the real 403. Never widen what a viewer can see. +4. **Always `await`.** Only a settled query is dehydrated, so an unawaited prefetch is silently dropped from the payload and the pane waterfalls anyway. +5. **Don't repeat what the layout already seeded.** `getQueryClient()` builds a new client per server call, so a page re-seeding a layout key is a genuine second read — and `HydrationBoundary` defers an already-seen query to an effect, which SSR never runs, so it never reaches the server render either. + +Reuse the hook's exported `staleTime` constant and its key factory; `dehydrate` carries neither options nor `staleTime`, and freshness is per-observer. + +Seed with `setQueryData` only when the prefetch must be able to *decline* to create an entry (an empty list that has to fall through to a route's creation path). `prefetchQuery` and `ensureQueryData` always create one. + +Keep prefetch imports light. A page prefetch's imports land in that route's server graph, so pulling a barrel to reach one function can drag thousands of modules behind it — `bun run check:tool-registry-boundary` gates this per page. + ## Boundary Types - Hooks import named type aliases from `@/lib/api/contracts/**` (e.g., `import { listEntitiesContract, type EntityList } from '@/lib/api/contracts/entities'`). Never write `z.input<...>` / `z.output<...>` in hooks, and never `import { z } from 'zod'` in client code. diff --git a/.cursor/commands/tool-registry-boundary.md b/.cursor/commands/tool-registry-boundary.md index d560d4f40e5..62fb47f53fd 100644 --- a/.cursor/commands/tool-registry-boundary.md +++ b/.cursor/commands/tool-registry-boundary.md @@ -63,6 +63,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph. +The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. This catches bloat the registry rule misses — a prefetch importing `listTables` cost the Tables page 444 modules without ever touching `@/tools/registry`. + +Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again. + ## How to verify an edge actually got cut Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph: diff --git a/apps/sim/app/_shell/providers/get-query-client.ts b/apps/sim/app/_shell/providers/get-query-client.ts index 681fd4ca84f..7fe869b9212 100644 --- a/apps/sim/app/_shell/providers/get-query-client.ts +++ b/apps/sim/app/_shell/providers/get-query-client.ts @@ -1,4 +1,4 @@ -import { defaultShouldDehydrateQuery, isServer, QueryClient } from '@tanstack/react-query' +import { isServer, QueryClient } from '@tanstack/react-query' import { isDesktopApp } from '@/lib/desktop' export function makeQueryClient() { @@ -6,7 +6,6 @@ export function makeQueryClient() { defaultOptions: { queries: { staleTime: 30 * 1000, - gcTime: 5 * 60 * 1000, // The desktop app window lives for days, so cross-session changes — // an admin upgrading your org/workspace role, a workspace you were // auto-added to, seat/entitlement changes — would otherwise stay @@ -18,16 +17,19 @@ export function makeQueryClient() { // frequent and noisy. Per-query overrides (e.g. useWorkspaceSchedules // pins this off) always win over this default. refetchOnWindowFocus: isDesktopApp(), - retry: 1, + /** + * Query core already defaults retries to 0 on the server and 3 in the browser; + * only the browser number is ours to change. Stating one value for both would + * silently opt server prefetches into a retry, and because the layout awaits + * them that spends a retry backoff of document latency on a read whose failure + * the client recovers from on its own. + */ + retry: isServer ? 0 : 1, retryOnMount: false, }, mutations: { retry: false, }, - dehydrate: { - shouldDehydrateQuery: (query) => - defaultShouldDehydrateQuery(query) || query.state.status === 'pending', - }, }, }) } diff --git a/apps/sim/app/api/pinned-items/route.ts b/apps/sim/app/api/pinned-items/route.ts index bf31285fb61..376bf510f60 100644 --- a/apps/sim/app/api/pinned-items/route.ts +++ b/apps/sim/app/api/pinned-items/route.ts @@ -2,40 +2,21 @@ import { db, pinnedItem } from '@sim/db' import { createLogger } from '@sim/logger' import { getPostgresErrorCode } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' -import { and, eq, ne } from 'drizzle-orm' import { type NextRequest, NextResponse } from 'next/server' import { createPinnedItemContract, listPinnedItemsContract, type PinnedItemApi, - pinnedResourceTypeSchema, } from '@/lib/api/contracts' import { parseRequest } from '@/lib/api/server' import { getSession } from '@/lib/auth' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' -import { filterToActiveResources, pinnableResourceExists } from '@/lib/pinned-items/resources' +import { listPinnedItemsForUser } from '@/lib/pinned-items/queries' +import { pinnableResourceExists } from '@/lib/pinned-items/resources' import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils' const logger = createLogger('PinnedItemsAPI') -/** - * Narrows a stored row to the wire shape, dropping any row whose `resourceType` this build does - * not recognise. - * - * `pinned_item.resource_type` is plain `text` — deliberately, so the set of pinnable kinds can - * grow — while the contract is a closed enum. During a rolling deploy an older pod can therefore - * read a pin a newer one wrote. Returning it would fail response validation and take the WHOLE - * list down rather than the single row, so the unknown kind is skipped instead. - * - * `filterToActiveResources` already drops these as a side effect of not having a table to look - * them up in; this makes the guarantee explicit and compiler-checked at the wire boundary. - */ -function toPinnedItemApi(row: typeof pinnedItem.$inferSelect): PinnedItemApi | null { - const resourceType = pinnedResourceTypeSchema.safeParse(row.resourceType) - if (!resourceType.success) return null - return { ...row, resourceType: resourceType.data, pinnedAt: row.pinnedAt.toISOString() } -} - /** Lists the session user's pinned items in a workspace, optionally filtered to one `resourceType`. */ export const GET = withRouteHandler(async (request: NextRequest) => { const session = await getSession() @@ -52,30 +33,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => { return NextResponse.json({ error: 'Access denied to this workspace' }, { status: 403 }) } - const rows = await db - .select() - .from(pinnedItem) - .where( - and( - eq(pinnedItem.userId, session.user.id), - eq(pinnedItem.workspaceId, workspaceId), - /** - * A `workspace` pin stores `workspaceId === resourceId`, so it would otherwise - * appear in this workspace's unscoped listing as a resource *inside* itself. - * It is read from the workspace-list payload instead, so it is excluded here - * rather than left for a future unscoped caller to mistake for a real resource. - */ - resourceType - ? eq(pinnedItem.resourceType, resourceType) - : ne(pinnedItem.resourceType, 'workspace') - ) - ) - - const activeRows = await filterToActiveResources(rows, workspaceId) - - const pinnedItems = activeRows - .map(toPinnedItemApi) - .filter((item): item is PinnedItemApi => item !== null) + const pinnedItems = await listPinnedItemsForUser(session.user.id, workspaceId, resourceType) return NextResponse.json({ pinnedItems }) }) diff --git a/apps/sim/app/api/table/[tableId]/columns/route.test.ts b/apps/sim/app/api/table/[tableId]/columns/route.test.ts index 6223c12bff6..24830309efc 100644 --- a/apps/sim/app/api/table/[tableId]/columns/route.test.ts +++ b/apps/sim/app/api/table/[tableId]/columns/route.test.ts @@ -49,10 +49,12 @@ vi.mock('@/lib/table/columns/service', () => ({ updateColumnOptions: mockUpdateColumnOptions, updateColumnType: mockUpdateColumnType, })) +vi.mock('@/lib/table/wire', () => ({ + normalizeColumn: (c: unknown) => c, +})) vi.mock('@/app/api/table/utils', () => ({ accessError: () => new Response('denied', { status: 403 }), checkAccess: mockCheckAccess, - normalizeColumn: (c: unknown) => c, orchestrationOutcomeErrorResponse: ( outcome: { error?: string; errorCode?: OrchestrationErrorCode }, fallback: string diff --git a/apps/sim/app/api/table/[tableId]/columns/route.ts b/apps/sim/app/api/table/[tableId]/columns/route.ts index 54ca6de54e8..2b2aa60c131 100644 --- a/apps/sim/app/api/table/[tableId]/columns/route.ts +++ b/apps/sim/app/api/table/[tableId]/columns/route.ts @@ -13,10 +13,10 @@ import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { addTableColumn, deleteColumn } from '@/lib/table' import { signalTableSchemaChanged } from '@/lib/table/events' import { performUpdateTableColumn } from '@/lib/table/orchestration' +import { normalizeColumn } from '@/lib/table/wire' import { accessError, checkAccess, - normalizeColumn, orchestrationOutcomeErrorResponse, rootErrorMessage, tableLockErrorResponse, diff --git a/apps/sim/app/api/table/[tableId]/groups/route.test.ts b/apps/sim/app/api/table/[tableId]/groups/route.test.ts index cad09be8b65..7d628cfdeb3 100644 --- a/apps/sim/app/api/table/[tableId]/groups/route.test.ts +++ b/apps/sim/app/api/table/[tableId]/groups/route.test.ts @@ -54,7 +54,7 @@ vi.mock('@/lib/table/application/groups', () => ({ updateTableGroupUseCase: mocks.useCases.update, })) -vi.mock('@/app/api/table/utils', () => ({ +vi.mock('@/lib/table/wire', () => ({ normalizeColumn: vi.fn(), })) diff --git a/apps/sim/app/api/table/[tableId]/groups/route.ts b/apps/sim/app/api/table/[tableId]/groups/route.ts index b1f9a1c4749..f8f14909ac4 100644 --- a/apps/sim/app/api/table/[tableId]/groups/route.ts +++ b/apps/sim/app/api/table/[tableId]/groups/route.ts @@ -12,7 +12,7 @@ import { } from '@/lib/table/application/groups' import { tableOperations } from '@/lib/table/application/operations' import type { TableDefinition } from '@/lib/table/types' -import { normalizeColumn } from '@/app/api/table/utils' +import { normalizeColumn } from '@/lib/table/wire' const rateLimit = internalRateLimits.none({ reason: 'Existing authenticated table group mutations have no request-rate policy', diff --git a/apps/sim/app/api/table/[tableId]/route.test.ts b/apps/sim/app/api/table/[tableId]/route.test.ts index 43cbf68ae83..6e1b9a957c9 100644 --- a/apps/sim/app/api/table/[tableId]/route.test.ts +++ b/apps/sim/app/api/table/[tableId]/route.test.ts @@ -51,9 +51,11 @@ vi.mock('@/lib/workspaces/permissions/utils', () => ({ vi.mock('@/app/api/table/utils', () => ({ accessError: () => new Response('denied', { status: 403 }), checkAccess: mockCheckAccess, - normalizeColumn: (column: unknown) => column, tableLockErrorResponse: () => null, })) +vi.mock('@/lib/table/wire', () => ({ + normalizeColumn: (column: unknown) => column, +})) import { GET, PATCH } from '@/app/api/table/[tableId]/route' diff --git a/apps/sim/app/api/table/[tableId]/route.ts b/apps/sim/app/api/table/[tableId]/route.ts index e14a3bdf775..4f61ce4ea12 100644 --- a/apps/sim/app/api/table/[tableId]/route.ts +++ b/apps/sim/app/api/table/[tableId]/route.ts @@ -18,11 +18,11 @@ import { performUpdateTableLocks, } from '@/lib/table/orchestration' import { TABLE_LOCK_FLAGS, TABLE_LOCK_KINDS } from '@/lib/table/types' +import { normalizeColumn } from '@/lib/table/wire' import { getWorkspaceWithOwner } from '@/lib/workspaces/permissions/utils' import { accessError, checkAccess, - normalizeColumn, orchestrationOutcomeErrorResponse, tableLockErrorResponse, } from '@/app/api/table/utils' diff --git a/apps/sim/app/api/table/import-csv/route.test.ts b/apps/sim/app/api/table/import-csv/route.test.ts index dae8f0c3d63..dea46a06a3a 100644 --- a/apps/sim/app/api/table/import-csv/route.test.ts +++ b/apps/sim/app/api/table/import-csv/route.test.ts @@ -34,7 +34,6 @@ vi.mock('@/app/api/table/utils', async () => { const { asOrchestrationError, messageForOrchestrationError, statusForOrchestrationError } = await import('@/lib/core/orchestration/types') return { - normalizeColumn: (column: unknown) => column, csvProxyBodyCapResponse: () => null, multipartErrorResponse: (error: { code: string; message: string }) => NextResponse.json( diff --git a/apps/sim/app/api/table/route.ts b/apps/sim/app/api/table/route.ts index 28714885cb5..be28a064fd1 100644 --- a/apps/sim/app/api/table/route.ts +++ b/apps/sim/app/api/table/route.ts @@ -15,8 +15,9 @@ import { type TableSchema, type TableScope, } from '@/lib/table' +import { normalizeColumn, toTableListItem, toWireTimestamp } from '@/lib/table/wire' import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils' -import { normalizeColumn, orchestrationErrorResponse } from '@/app/api/table/utils' +import { orchestrationErrorResponse } from '@/app/api/table/utils' const logger = createLogger('TableAPI') @@ -140,14 +141,8 @@ export const POST = withRouteHandler(async (request: NextRequest) => { maxRows: table.maxRows, folderId: table.folderId ?? null, locks: table.locks, - createdAt: - table.createdAt instanceof Date - ? table.createdAt.toISOString() - : String(table.createdAt), - updatedAt: - table.updatedAt instanceof Date - ? table.updatedAt.toISOString() - : String(table.updatedAt), + createdAt: toWireTimestamp(table.createdAt), + updatedAt: toWireTimestamp(table.updatedAt), }, message: 'Table created successfully', }, @@ -198,41 +193,10 @@ export const GET = withRouteHandler(async (request: NextRequest) => { logger.info(`[${requestId}] Listed ${tables.length} tables in workspace ${params.workspaceId}`) - const responseTables = tables.map((t) => { - const schemaData = t.schema as TableSchema - return { - id: t.id, - name: t.name, - description: t.description, - schema: { - columns: schemaData.columns.map(normalizeColumn), - }, - rowCount: t.rowCount, - maxRows: t.maxRows, - locks: t.locks, - workspaceId: t.workspaceId, - folderId: t.folderId ?? null, - createdBy: t.createdBy, - createdAt: t.createdAt instanceof Date ? t.createdAt.toISOString() : String(t.createdAt), - updatedAt: t.updatedAt instanceof Date ? t.updatedAt.toISOString() : String(t.updatedAt), - archivedAt: - t.archivedAt instanceof Date - ? t.archivedAt.toISOString() - : t.archivedAt - ? String(t.archivedAt) - : null, - jobStatus: t.jobStatus ?? null, - jobId: t.jobId ?? null, - jobType: t.jobType ?? null, - jobError: t.jobError ?? null, - jobRowsProcessed: t.jobRowsProcessed ?? 0, - } - }) - return NextResponse.json({ success: true, data: { - tables: responseTables, + tables: tables.map(toTableListItem), totalCount: tables.length, }, }) diff --git a/apps/sim/app/api/table/utils.ts b/apps/sim/app/api/table/utils.ts index e049b1a3f68..037c6c83cc8 100644 --- a/apps/sim/app/api/table/utils.ts +++ b/apps/sim/app/api/table/utils.ts @@ -17,7 +17,6 @@ import { import type { MultipartError } from '@/lib/core/utils/multipart' import type { ColumnDefinition, Filter, TableDefinition, TablePredicate } from '@/lib/table' import { buildFilterClause, getTableById, TableQueryValidationError } from '@/lib/table' -import { typeMetadataOf } from '@/lib/table/column-types' import { USER_TABLE_ROWS_SQL_NAME } from '@/lib/table/constants' import { TableLockedError } from '@/lib/table/mutation-locks' import { isTablePredicate } from '@/lib/table/query-builder/converters' @@ -358,21 +357,3 @@ export function serverErrorResponse(message = 'Internal server error') { export const CreateColumnSchema = createTableColumnBodySchema export const UpdateColumnSchema = updateTableColumnBodySchema export const DeleteColumnSchema = deleteTableColumnBodySchema - -export function normalizeColumn( - col: ColumnDefinition -): ColumnDefinition & { required: boolean; unique: boolean } { - return { - // Preserve the stable column id — it's the row-data storage key, so dropping - // it makes clients fall back to `name` and miss id-keyed cell values. - ...(col.id ? { id: col.id } : {}), - name: col.name, - type: col.type, - required: col.required ?? false, - unique: col.unique ?? false, - ...(col.workflowGroupId ? { workflowGroupId: col.workflowGroupId } : {}), - // Type-specific metadata is forwarded generically: naming keys here meant a - // new type's metadata was stored server-side but silently never returned. - ...typeMetadataOf(col), - } -} diff --git a/apps/sim/app/api/v1/admin/responses.ts b/apps/sim/app/api/v1/admin/responses.ts index 9308df895dc..3ecc5353b29 100644 --- a/apps/sim/app/api/v1/admin/responses.ts +++ b/apps/sim/app/api/v1/admin/responses.ts @@ -51,9 +51,7 @@ export function errorResponse( return NextResponse.json(body, { status }) } -// ============================================================================= // Common Error Responses -// ============================================================================= export function unauthorizedResponse(message = 'Authentication required'): NextResponse { return errorResponse('UNAUTHORIZED', message, 401) diff --git a/apps/sim/app/api/v1/admin/types.ts b/apps/sim/app/api/v1/admin/types.ts index 4256076d457..a6062ca8eee 100644 --- a/apps/sim/app/api/v1/admin/types.ts +++ b/apps/sim/app/api/v1/admin/types.ts @@ -20,9 +20,7 @@ import type { InferSelectModel } from 'drizzle-orm' import type { Edge } from 'reactflow' import type { BlockState, Loop, Parallel } from '@/stores/workflows/workflow/types' -// ============================================================================= // Database Model Types (inferred from schema) -// ============================================================================= export type DbUser = InferSelectModel export type DbWorkspace = InferSelectModel @@ -33,9 +31,7 @@ export type DbSubscription = InferSelectModel export type DbMember = InferSelectModel export type DbUserStats = InferSelectModel -// ============================================================================= // Pagination -// ============================================================================= export interface PaginationParams { limit: number @@ -74,9 +70,7 @@ export function createPaginationMeta(total: number, limit: number, offset: numbe } } -// ============================================================================= // API Response Types -// ============================================================================= export interface AdminListResponse { data: T[] @@ -95,9 +89,7 @@ export interface AdminErrorResponse { } } -// ============================================================================= // User Types -// ============================================================================= export interface AdminUser { id: string @@ -121,9 +113,7 @@ export function toAdminUser(dbUser: DbUser): AdminUser { } } -// ============================================================================= // Workspace Types -// ============================================================================= export interface AdminWorkspace { id: string @@ -148,9 +138,7 @@ export function toAdminWorkspace(dbWorkspace: DbWorkspace): AdminWorkspace { } } -// ============================================================================= // Folder Types -// ============================================================================= export interface AdminFolder { id: string @@ -179,9 +167,7 @@ export function toAdminFolder(dbFolder: DbWorkflowFolder): AdminFolder { } } -// ============================================================================= // Workflow Types -// ============================================================================= export interface AdminWorkflow { id: string @@ -233,9 +219,7 @@ export function toAdminWorkflow(dbWorkflow: AdminWorkflowSource): AdminWorkflow } } -// ============================================================================= // Workflow Variable Types -// ============================================================================= export type VariableType = 'string' | 'number' | 'boolean' | 'object' | 'array' | 'plain' @@ -246,9 +230,7 @@ export interface WorkflowVariable { value: unknown } -// ============================================================================= // Export/Import Types -// ============================================================================= export interface WorkflowExportState { blocks: Record @@ -296,9 +278,7 @@ export interface WorkspaceExportPayload { folders: FolderExportPayload[] } -// ============================================================================= // Import Types -// ============================================================================= export interface WorkflowImportRequest { workspaceId: string @@ -328,9 +308,7 @@ export interface WorkspaceImportResponse { results: ImportResult[] } -// ============================================================================= // Utility Functions -// ============================================================================= /** * Extract workflow metadata from various export formats. @@ -384,9 +362,7 @@ function getNestedString(obj: Record, path: string): string | u return typeof current === 'string' ? current : undefined } -// ============================================================================= // Organization Types -// ============================================================================= export interface AdminOrganization { id: string @@ -432,9 +408,7 @@ export function toAdminOrganization(dbOrg: AdminOrganizationSource): AdminOrgani } } -// ============================================================================= // Subscription Types -// ============================================================================= export interface AdminSubscription { id: string @@ -470,9 +444,7 @@ export function toAdminSubscription(dbSub: DbSubscription): AdminSubscription { } } -// ============================================================================= // Member Types -// ============================================================================= export interface AdminMember { id: string @@ -492,9 +464,7 @@ export interface AdminMemberDetail extends AdminMember { billingBlocked: boolean } -// ============================================================================= // Workspace Member Types -// ============================================================================= export interface AdminWorkspaceMember { id: string @@ -508,9 +478,7 @@ export interface AdminWorkspaceMember { userImage: string | null } -// ============================================================================= // User Billing Types -// ============================================================================= interface AdminUserBilling { userId: string @@ -539,9 +507,7 @@ export interface AdminUserBillingWithSubscription extends AdminUserBilling { }> } -// ============================================================================= // Organization Billing Summary Types -// ============================================================================= export interface AdminOrganizationBillingSummary { organizationId: string @@ -587,9 +553,7 @@ export interface AdminDeploymentVersion { deployedByName: string | null } -// ============================================================================= // Audit Log Types -// ============================================================================= export type DbAuditLog = InferSelectModel diff --git a/apps/sim/app/api/v1/tables/[tableId]/columns/route.ts b/apps/sim/app/api/v1/tables/[tableId]/columns/route.ts index aa3f74d8157..ae8c00affab 100644 --- a/apps/sim/app/api/v1/tables/[tableId]/columns/route.ts +++ b/apps/sim/app/api/v1/tables/[tableId]/columns/route.ts @@ -12,10 +12,10 @@ import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { addTableColumn, deleteColumn } from '@/lib/table' import { signalTableSchemaChanged } from '@/lib/table/events' import { performUpdateTableColumn } from '@/lib/table/orchestration' +import { normalizeColumn } from '@/lib/table/wire' import { accessError, checkAccess, - normalizeColumn, orchestrationErrorResponse, orchestrationOutcomeErrorResponse, tableLockErrorResponse, diff --git a/apps/sim/app/api/v1/tables/[tableId]/route.ts b/apps/sim/app/api/v1/tables/[tableId]/route.ts index caaf87d8be7..5d46bdf619b 100644 --- a/apps/sim/app/api/v1/tables/[tableId]/route.ts +++ b/apps/sim/app/api/v1/tables/[tableId]/route.ts @@ -6,10 +6,10 @@ import { generateRequestId } from '@/lib/core/utils/request' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import type { TableSchema } from '@/lib/table' import { performDeleteTable } from '@/lib/table/orchestration' +import { normalizeColumn } from '@/lib/table/wire' import { accessError, checkAccess, - normalizeColumn, orchestrationOutcomeErrorResponse, tableLockErrorResponse, } from '@/app/api/table/utils' diff --git a/apps/sim/app/api/v1/tables/route.test.ts b/apps/sim/app/api/v1/tables/route.test.ts index ded5f484e8d..f12bceb2334 100644 --- a/apps/sim/app/api/v1/tables/route.test.ts +++ b/apps/sim/app/api/v1/tables/route.test.ts @@ -35,9 +35,11 @@ vi.mock('@/app/api/v1/middleware', () => ({ })) vi.mock('@/app/api/table/utils', () => ({ - normalizeColumn: (column: unknown) => column, orchestrationErrorResponse: mocks.orchestrationErrorResponse, })) +vi.mock('@/lib/table/wire', () => ({ + normalizeColumn: (column: unknown) => column, +})) vi.mock('@/lib/table', () => ({ createTable: mocks.createTable, diff --git a/apps/sim/app/api/v1/tables/route.ts b/apps/sim/app/api/v1/tables/route.ts index e8a13eb9090..ecd742efb29 100644 --- a/apps/sim/app/api/v1/tables/route.ts +++ b/apps/sim/app/api/v1/tables/route.ts @@ -12,7 +12,8 @@ import { TableConflictError, type TableSchema, } from '@/lib/table' -import { normalizeColumn, orchestrationErrorResponse } from '@/app/api/table/utils' +import { normalizeColumn } from '@/lib/table/wire' +import { orchestrationErrorResponse } from '@/app/api/table/utils' import { checkRateLimit, createRateLimitResponse, diff --git a/apps/sim/app/api/v2/tables/[tableId]/columns/route.ts b/apps/sim/app/api/v2/tables/[tableId]/columns/route.ts index 03030a64729..4440d408b74 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/columns/route.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/columns/route.ts @@ -12,7 +12,7 @@ import { updateTableColumnUseCase, } from '@/lib/table/application/columns' import { tableOperations } from '@/lib/table/application/operations' -import { normalizeColumn } from '@/app/api/table/utils' +import { normalizeColumn } from '@/lib/table/wire' export const dynamic = 'force-dynamic' export const revalidate = 0 diff --git a/apps/sim/app/api/v2/tables/[tableId]/groups/route.ts b/apps/sim/app/api/v2/tables/[tableId]/groups/route.ts index 8910958f55a..b92dfb69d5f 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/groups/route.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/groups/route.ts @@ -13,7 +13,7 @@ import { updateTableGroupUseCase, } from '@/lib/table/application/groups' import { tableOperations } from '@/lib/table/application/operations' -import { normalizeColumn } from '@/app/api/table/utils' +import { normalizeColumn } from '@/lib/table/wire' export const dynamic = 'force-dynamic' export const revalidate = 0 diff --git a/apps/sim/app/api/v2/tables/utils.ts b/apps/sim/app/api/v2/tables/utils.ts index 04d02c3264e..d262e015001 100644 --- a/apps/sim/app/api/v2/tables/utils.ts +++ b/apps/sim/app/api/v2/tables/utils.ts @@ -14,8 +14,9 @@ import { import { predicateToStorage } from '@/lib/table/select-values' import type { Filter, TableLockKind } from '@/lib/table/types' import type { TableView } from '@/lib/table/views/service' +import { normalizeColumn } from '@/lib/table/wire' import { getUserEmailsByIds, requireResolvedUserEmail } from '@/lib/users/queries' -import { CSV_IMPORT_PROXY_BODY_CAP_BYTES, normalizeColumn } from '@/app/api/table/utils' +import { CSV_IMPORT_PROXY_BODY_CAP_BYTES } from '@/app/api/table/utils' import { v2Error, v2ErrorForOrchestration } from '@/app/api/v2/lib/response' /** diff --git a/apps/sim/app/workspace/[workspaceId]/files/page.tsx b/apps/sim/app/workspace/[workspaceId]/files/page.tsx index 389b4d17ded..11dc9fd450e 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/files/page.tsx @@ -23,9 +23,7 @@ export default async function FilesPage({ params }: { params: Promise<{ workspac const [{ workspaceId }, session] = await Promise.all([params, getSession()]) const queryClient = getQueryClient() - if (session?.user?.id) { - await prefetchFilesBrowser(queryClient, workspaceId, session.user.id) - } + await prefetchFilesBrowser(queryClient, workspaceId, session?.user?.id) return ( diff --git a/apps/sim/app/workspace/[workspaceId]/files/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/files/prefetch.ts index 90e653dbd8c..6ed88de422d 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/prefetch.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/prefetch.ts @@ -20,9 +20,8 @@ import { * and still not reach the server render (`HydrationBoundary` defers an already-seen query to an * effect, which SSR never runs). See the note on that entry. * - * Folders read the data layer; the payload is shaped to its route contract so a hydrated entry - * matches a client fetch. Everything else still goes through its route — see - * {@link prefetchInternalJson}. + * Folders and the chrome reads all go through the data layer, shaped to their route contracts so a + * hydrated entry matches a client fetch. * * That read carries no authorization of its own, so the viewer is proved first. This reuses the * layout's `cache`d host-context lookup rather than re-deriving the permission, so it costs no @@ -32,8 +31,9 @@ import { export async function prefetchFilesBrowser( queryClient: QueryClient, workspaceId: string, - userId: string + userId: string | undefined ): Promise { + if (!userId) return const hostContext = await getWorkspaceHostContextForViewer(workspaceId, userId) if (!hostContext) return @@ -43,6 +43,6 @@ export async function prefetchFilesBrowser( queryFn: () => listWorkspaceFileFolders(workspaceId, { scope: 'active' }), staleTime: WORKSPACE_FILE_FOLDERS_STALE_TIME, }), - prefetchResourceListChrome(queryClient, workspaceId, 'file'), + prefetchResourceListChrome(queryClient, workspaceId, 'file', userId), ]) } diff --git a/apps/sim/app/workspace/[workspaceId]/home/page.tsx b/apps/sim/app/workspace/[workspaceId]/home/page.tsx index b7a6cc4ea95..cfb87f8ce04 100644 --- a/apps/sim/app/workspace/[workspaceId]/home/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/home/page.tsx @@ -1,11 +1,8 @@ import { Suspense } from 'react' -import { dehydrate, HydrationBoundary } from '@tanstack/react-query' import type { Metadata } from 'next' import { redirect } from 'next/navigation' import { getSession } from '@/lib/auth' import { isChatEnabled } from '@/lib/core/config/env-flags' -import { getQueryClient } from '@/app/_shell/providers/get-query-client' -import { prefetchHomeLists } from '@/app/workspace/[workspaceId]/home/prefetch' import { resolveTableViewsEnabled } from '@/app/workspace/[workspaceId]/home/resolve-table-views-flag' import { Home } from './home' import { HomeFallback } from './home-fallback' @@ -23,23 +20,19 @@ export default async function HomePage({ params }: { params: Promise<{ workspace redirect(`/workspace/${workspaceId}`) } - const queryClient = getQueryClient() - const listsPrefetch = prefetchHomeLists(queryClient, workspaceId) - + /** + * Home prefetches nothing of its own. Both lists it reads — workflow folders and + * the workspace file list — are hydrated by `prefetchWorkspaceSidebar` in the + * layout under the same keys, and re-reading them here would cost a second query + * per request without reaching the server render. + */ const session = await getSession() const userId = session?.user?.id const tableViewsEnabled = await resolveTableViewsEnabled(workspaceId, userId) - await listsPrefetch return ( - - }> - - - + }> + + ) } diff --git a/apps/sim/app/workspace/[workspaceId]/home/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/home/prefetch.ts deleted file mode 100644 index f08791c0bbd..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/home/prefetch.ts +++ /dev/null @@ -1,50 +0,0 @@ -import type { QueryClient } from '@tanstack/react-query' -import type { FolderApi } from '@/lib/api/contracts' -import type { ListWorkspaceFilesResponse } from '@/lib/api/contracts/workspace-files' -import { prefetchInternalJson } from '@/app/workspace/[workspaceId]/lib/prefetch-internal-fetch' -import { FOLDER_LIST_STALE_TIME, folderKeys, mapFolder } from '@/hooks/queries/utils/folder-keys' -import { - WORKSPACE_FILES_LIST_STALE_TIME, - workspaceFilesKeys, -} from '@/hooks/queries/workspace-files' - -/** - * Prefetches the home page's secondary lists — folders and workspace files — - * under the same query keys their client hooks (`useFolders`, - * `useWorkspaceFiles`) use, so the home view paints populated on first render. - * - * The workflow list (`workflowKeys.list(ws, 'active')`) is already hydrated by - * the workspace sidebar prefetch and is intentionally not repeated here. - * - * Folders are fetched through the route and mapped with the same `mapFolder` - * the hook applies, matching its cached shape (string dates → `Date`). Files - * carry `Date` fields, so they go through the route and cache the serialized - * wire shape — see {@link prefetchInternalJson}. - */ -export async function prefetchHomeLists( - queryClient: QueryClient, - workspaceId: string -): Promise { - await Promise.all([ - queryClient.prefetchQuery({ - queryKey: folderKeys.list(workspaceId, 'active', 'workflow'), - queryFn: async () => { - const { folders } = await prefetchInternalJson<{ folders?: FolderApi[] }>( - `/api/folders?workspaceId=${workspaceId}&scope=active&resourceType=workflow` - ) - return (folders ?? []).map(mapFolder) - }, - staleTime: FOLDER_LIST_STALE_TIME, - }), - queryClient.prefetchQuery({ - queryKey: workspaceFilesKeys.list(workspaceId, 'active'), - queryFn: async () => { - const data = await prefetchInternalJson( - `/api/workspaces/${workspaceId}/files?scope=active` - ) - return data.success ? data.files : [] - }, - staleTime: WORKSPACE_FILES_LIST_STALE_TIME, - }), - ]) -} diff --git a/apps/sim/app/workspace/[workspaceId]/knowledge/page.tsx b/apps/sim/app/workspace/[workspaceId]/knowledge/page.tsx index 48b7934bb13..8bef3b5b32f 100644 --- a/apps/sim/app/workspace/[workspaceId]/knowledge/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/knowledge/page.tsx @@ -1,6 +1,7 @@ import { Suspense } from 'react' import { dehydrate, HydrationBoundary } from '@tanstack/react-query' import type { Metadata } from 'next' +import { getSession } from '@/lib/auth' import { getQueryClient } from '@/app/_shell/providers/get-query-client' import { FOLDERED_RESOURCE_HEADERS } from '@/app/workspace/[workspaceId]/components/folders/foldered-resources' import { Knowledge } from '@/app/workspace/[workspaceId]/knowledge/knowledge' @@ -24,10 +25,9 @@ export default async function KnowledgePage({ }: { params: Promise<{ workspaceId: string }> }) { - const { workspaceId } = await params - + const [{ workspaceId }, session] = await Promise.all([params, getSession()]) const queryClient = getQueryClient() - await prefetchKnowledgeBases(queryClient, workspaceId) + await prefetchKnowledgeBases(queryClient, workspaceId, session?.user?.id) return ( diff --git a/apps/sim/app/workspace/[workspaceId]/knowledge/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/knowledge/prefetch.ts index 7c9d45cb668..1bfaf144acf 100644 --- a/apps/sim/app/workspace/[workspaceId]/knowledge/prefetch.ts +++ b/apps/sim/app/workspace/[workspaceId]/knowledge/prefetch.ts @@ -1,9 +1,10 @@ import type { QueryClient } from '@tanstack/react-query' -import type { FolderApi } from '@/lib/api/contracts/folders' -import type { KnowledgeBaseData } from '@/lib/api/contracts/knowledge' -import { prefetchInternalJson } from '@/app/workspace/[workspaceId]/lib/prefetch-internal-fetch' +import { listKnowledgeBasesContract } from '@/lib/api/contracts/knowledge' +import { internalSessionAuth } from '@/lib/api/server/routes' +import { internalKnowledgePresenters } from '@/lib/knowledge/api/internal-route' +import { listInternalKnowledgeBases } from '@/lib/knowledge/application/knowledge-bases' +import { prefetchResourceFolders } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-folders' import { prefetchResourceListChrome } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome' -import { FOLDER_LIST_STALE_TIME, folderKeys, mapFolder } from '@/hooks/queries/utils/folder-keys' import { KNOWLEDGE_BASE_LIST_STALE_TIME, knowledgeKeys } from '@/hooks/queries/utils/knowledge-keys' /** @@ -16,35 +17,41 @@ import { KNOWLEDGE_BASE_LIST_STALE_TIME, knowledgeKeys } from '@/hooks/queries/u * beside, so prefetching one without the other still flashes an ungrouped list — and a * `?folderId=` deep link renders an empty breadcrumb until the folders arrive. * - * The list carries `Date` fields, so it goes through the `/api/knowledge` route and caches the - * serialized wire shape — see {@link prefetchInternalJson}. Folders are mapped with the same - * `mapFolder` the hook applies, so the hydrated entry matches a client fetch exactly. + * The bases list runs the same `listInternalKnowledgeBases` application use case + * `GET /api/knowledge` runs, authenticated with the same `internalSessionAuth` policy, and is + * projected through the same `internalKnowledgePresenters.list` presenter and the contract's + * response schema. Nothing about authorization moves here: the use case still loads the + * canonical workspace context and authorizes the session principal against + * `knowledgeOperations.list`. An unauthenticated or unauthorized viewer throws inside the + * query function, which caches nothing and leaves the client fetch to reach the route for the + * real 401/403. + * + * Folders read the data layer and are mapped with the same `mapFolder` the hook applies, + * matching the workspace sidebar prefetch. That read carries no authorization of its own, so + * the viewer is proved first; `getWorkspaceHostContextForViewer` is `cache`d and the layout has + * already resolved it for this request, so it costs no additional queries. */ export async function prefetchKnowledgeBases( queryClient: QueryClient, - workspaceId: string + workspaceId: string, + userId: string | undefined ): Promise { await Promise.all([ queryClient.prefetchQuery({ queryKey: knowledgeKeys.list(workspaceId, 'active'), queryFn: async () => { - const result = await prefetchInternalJson<{ data: KnowledgeBaseData[] }>( - `/api/knowledge?workspaceId=${workspaceId}&scope=active` - ) - return result.data + const principal = await internalSessionAuth.authenticate() + const result = await listInternalKnowledgeBases.execute({ + principal, + input: { workspaceId, scope: 'active' }, + }) + return listKnowledgeBasesContract.response.schema.parse( + internalKnowledgePresenters.list(result) + ).data }, staleTime: KNOWLEDGE_BASE_LIST_STALE_TIME, }), - queryClient.prefetchQuery({ - queryKey: folderKeys.list(workspaceId, 'active', 'knowledge_base'), - queryFn: async () => { - const { folders } = await prefetchInternalJson<{ folders?: FolderApi[] }>( - `/api/folders?workspaceId=${workspaceId}&scope=active&resourceType=knowledge_base` - ) - return (folders ?? []).map(mapFolder) - }, - staleTime: FOLDER_LIST_STALE_TIME, - }), - prefetchResourceListChrome(queryClient, workspaceId, 'knowledge_base'), + prefetchResourceFolders(queryClient, workspaceId, 'knowledge_base', userId), + prefetchResourceListChrome(queryClient, workspaceId, 'knowledge_base', userId), ]) } diff --git a/apps/sim/app/workspace/[workspaceId]/lib/prefetch-internal-fetch.ts b/apps/sim/app/workspace/[workspaceId]/lib/prefetch-internal-fetch.ts deleted file mode 100644 index 4ba194395e6..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/lib/prefetch-internal-fetch.ts +++ /dev/null @@ -1,25 +0,0 @@ -import { headers } from 'next/headers' -import { getInternalApiBaseUrl } from '@/lib/core/utils/urls' - -/** - * Server-side GET against an internal `/api` route, forwarding the incoming - * request's cookie so the route authenticates as the current user. - * - * The legacy path. Reading the data layer and shaping the result through the - * route's response contract — as `files/prefetch.ts` does — is canonical: it - * drops a server-to-server request and its duplicate auth, and the contract - * parse is what guarantees the hydrated entry matches a client fetch. Prefetches - * still on this helper have not been converted; a converted one must prove the - * viewer itself, since the route's own authorization no longer runs. - */ -export async function prefetchInternalJson(path: string): Promise { - const cookie = (await headers()).get('cookie') - // boundary-raw-fetch: server-side RSC prefetch forwarding the session cookie to an internal API route; requestJson is client-only and cannot run here - const response = await fetch(`${getInternalApiBaseUrl()}${path}`, { - headers: cookie ? { cookie } : {}, - }) - if (!response.ok) { - throw new Error(`Prefetch failed for ${path}: ${response.status}`) - } - return response.json() as Promise -} diff --git a/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-folders.ts b/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-folders.ts new file mode 100644 index 00000000000..ecd08257221 --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-folders.ts @@ -0,0 +1,37 @@ +import type { QueryClient } from '@tanstack/react-query' +import type { FolderResourceType } from '@/lib/api/contracts/folders' +import { listFoldersForWorkspace } from '@/lib/folders/queries' +import { getWorkspaceHostContextForViewer } from '@/lib/workspaces/host-context' +import { FOLDER_LIST_STALE_TIME, folderKeys, mapFolder } from '@/hooks/queries/utils/folder-keys' + +/** + * Prefetches one resource family's folder tree under the same key its client + * `useFolders` hook reads, mapped with the same `mapFolder` the hook applies so a + * hydrated entry matches a client fetch. + * + * Shared by the resource list pages so the key, stale time, and mapper cannot + * drift apart across them. Self-guarding like {@link prefetchResourceListChrome}: + * the read carries no authorization of its own, so an unproven viewer caches + * nothing and their client fetch reaches the route for the real 403. + * `getWorkspaceHostContextForViewer` is `cache`d and the layout has already + * resolved it for this request, so the proof costs no additional queries. + */ +export async function prefetchResourceFolders( + queryClient: QueryClient, + workspaceId: string, + resourceType: FolderResourceType, + userId: string | undefined +): Promise { + if (!userId) return + const hostContext = await getWorkspaceHostContextForViewer(workspaceId, userId) + if (!hostContext) return + + await queryClient.prefetchQuery({ + queryKey: folderKeys.list(workspaceId, 'active', resourceType), + queryFn: async () => { + const rows = await listFoldersForWorkspace(workspaceId, 'active', resourceType) + return rows.map(mapFolder) + }, + staleTime: FOLDER_LIST_STALE_TIME, + }) +} diff --git a/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome.ts b/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome.ts index 5d9241aa23f..96b324b9d86 100644 --- a/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome.ts +++ b/apps/sim/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome.ts @@ -1,12 +1,10 @@ import type { QueryClient } from '@tanstack/react-query' -import type { PinnedItemApi, PinnedResourceType } from '@/lib/api/contracts/pinned-items' -import { prefetchInternalJson } from '@/app/workspace/[workspaceId]/lib/prefetch-internal-fetch' +import type { PinnedResourceType } from '@/lib/api/contracts/pinned-items' +import { listPinnedItemsForUser } from '@/lib/pinned-items/queries' +import { getWorkspaceHostContextForViewer } from '@/lib/workspaces/host-context' +import { getWorkspaceMemberProfiles } from '@/lib/workspaces/permissions/utils' import { PINNED_ITEMS_STALE_TIME, pinnedItemKeys } from '@/hooks/queries/utils/pinned-item-keys' -import { - WORKSPACE_MEMBERS_STALE_TIME, - type WorkspaceMember, - workspaceKeys, -} from '@/hooks/queries/workspace' +import { WORKSPACE_MEMBERS_STALE_TIME, workspaceKeys } from '@/hooks/queries/workspace' /** * Prefetches the two lists every foldered resource page needs to paint a row completely, @@ -19,21 +17,28 @@ import { * * Members back the Owner column; without them every owner cell paints empty and fills in * after. Both are cheap and shared with the page's own list prefetch in one `Promise.all`. + * + * Both read the data layer through the same functions their routes call, so a hydrated entry + * matches what a client fetch would parse out of the response. Neither read carries + * authorization of its own, so the viewer is proved first — `getWorkspaceHostContextForViewer` + * is `cache`d and the layout has already resolved it for this request, so it costs no + * additional queries. A viewer without access caches nothing and the client fetch reaches the + * route for the real 403. */ export async function prefetchResourceListChrome( queryClient: QueryClient, workspaceId: string, - resourceType: PinnedResourceType + resourceType: PinnedResourceType, + userId: string | undefined ): Promise { + if (!userId) return + const hostContext = await getWorkspaceHostContextForViewer(workspaceId, userId) + if (!hostContext) return + const prefetchPinned = (type: PinnedResourceType) => queryClient.prefetchQuery({ queryKey: pinnedItemKeys.list(workspaceId, type), - queryFn: async () => { - const { pinnedItems } = await prefetchInternalJson<{ pinnedItems: PinnedItemApi[] }>( - `/api/pinned-items?workspaceId=${workspaceId}&resourceType=${type}` - ) - return pinnedItems - }, + queryFn: () => listPinnedItemsForUser(userId, workspaceId, type), staleTime: PINNED_ITEMS_STALE_TIME, }) @@ -42,12 +47,7 @@ export async function prefetchResourceListChrome( prefetchPinned('folder'), queryClient.prefetchQuery({ queryKey: workspaceKeys.members(workspaceId), - queryFn: async () => { - const { members } = await prefetchInternalJson<{ members: WorkspaceMember[] }>( - `/api/workspaces/${workspaceId}/members` - ) - return members - }, + queryFn: () => getWorkspaceMemberProfiles(workspaceId), staleTime: WORKSPACE_MEMBERS_STALE_TIME, }), ]) diff --git a/apps/sim/app/workspace/[workspaceId]/lib/prefetch.test.ts b/apps/sim/app/workspace/[workspaceId]/lib/prefetch.test.ts index 4bb0510df1d..32309a08be9 100644 --- a/apps/sim/app/workspace/[workspaceId]/lib/prefetch.test.ts +++ b/apps/sim/app/workspace/[workspaceId]/lib/prefetch.test.ts @@ -5,29 +5,67 @@ import { QueryClient } from '@tanstack/react-query' import { beforeEach, describe, expect, it, vi } from 'vitest' const { + mockAuthenticate, mockGetWorkspaceHostContextForViewer, + mockGetWorkspaceMemberProfiles, + mockKnowledgePresenterList, + mockListFoldersForWorkspace, + mockListInternalKnowledgeBases, + mockListPinnedItemsForUser, + mockListTables, mockListWorkspaceFileFolders, mockListWorkspaceFilesWithShares, - mockPrefetchInternalJson, } = vi.hoisted(() => ({ + mockAuthenticate: vi.fn(), mockGetWorkspaceHostContextForViewer: vi.fn(), + mockGetWorkspaceMemberProfiles: vi.fn(), + mockKnowledgePresenterList: vi.fn(), + mockListFoldersForWorkspace: vi.fn(), + mockListInternalKnowledgeBases: vi.fn(), + mockListPinnedItemsForUser: vi.fn(), + mockListTables: vi.fn(), mockListWorkspaceFileFolders: vi.fn(), mockListWorkspaceFilesWithShares: vi.fn(), - mockPrefetchInternalJson: vi.fn(), })) vi.mock('@/lib/workspaces/host-context', () => ({ getWorkspaceHostContextForViewer: mockGetWorkspaceHostContextForViewer, })) +vi.mock('@/lib/folders/queries', () => ({ + listFoldersForWorkspace: mockListFoldersForWorkspace, +})) vi.mock('@/lib/workspace-files/queries', () => ({ listWorkspaceFilesWithShares: mockListWorkspaceFilesWithShares, })) vi.mock('@/lib/uploads/contexts/workspace/workspace-file-folder-manager', () => ({ listWorkspaceFileFolders: mockListWorkspaceFileFolders, })) - -vi.mock('@/app/workspace/[workspaceId]/lib/prefetch-internal-fetch', () => ({ - prefetchInternalJson: mockPrefetchInternalJson, +vi.mock('@/lib/pinned-items/queries', () => ({ + listPinnedItemsForUser: mockListPinnedItemsForUser, +})) +vi.mock('@/lib/workspaces/permissions/utils', () => ({ + getWorkspaceMemberProfiles: mockGetWorkspaceMemberProfiles, +})) +vi.mock('@/lib/table/service', () => ({ + listTables: mockListTables, +})) +/** + * `typeMetadataOf` is the one leaf of the real wire projection that reaches the + * column-type registry, and through it every type module's icon and editor. Stub + * that leaf only, so `toTableListItem`'s timestamp, `metadata`, and job + * normalization stay under test rather than being mocked away wholesale. + */ +vi.mock('@/lib/table/column-types', () => ({ + typeMetadataOf: () => ({}), +})) +vi.mock('@/lib/api/server/routes', () => ({ + internalSessionAuth: { authenticate: mockAuthenticate }, +})) +vi.mock('@/lib/knowledge/application/knowledge-bases', () => ({ + listInternalKnowledgeBases: { execute: mockListInternalKnowledgeBases }, +})) +vi.mock('@/lib/knowledge/api/internal-route', () => ({ + internalKnowledgePresenters: { list: mockKnowledgePresenterList }, })) vi.mock('@sim/emcn', () => ({ @@ -35,7 +73,6 @@ vi.mock('@sim/emcn', () => ({ })) import { prefetchFilesBrowser } from '@/app/workspace/[workspaceId]/files/prefetch' -import { prefetchHomeLists } from '@/app/workspace/[workspaceId]/home/prefetch' import { prefetchKnowledgeBases } from '@/app/workspace/[workspaceId]/knowledge/prefetch' import { prefetchTables } from '@/app/workspace/[workspaceId]/tables/prefetch' import { folderKeys } from '@/hooks/queries/utils/folder-keys' @@ -57,40 +94,167 @@ describe('workspace list prefetches', () => { beforeEach(() => { vi.clearAllMocks() mockGetWorkspaceHostContextForViewer.mockResolvedValue({ viewer: { permission: 'admin' } }) + mockListFoldersForWorkspace.mockResolvedValue([]) mockListWorkspaceFilesWithShares.mockResolvedValue([]) mockListWorkspaceFileFolders.mockResolvedValue([]) + mockListPinnedItemsForUser.mockResolvedValue([]) + mockGetWorkspaceMemberProfiles.mockResolvedValue([]) + mockListTables.mockResolvedValue([]) + mockAuthenticate.mockResolvedValue({ kind: 'session', userId: USER_ID, sessionId: 'sess-1' }) + mockListInternalKnowledgeBases.mockResolvedValue({ knowledgeBases: [] }) + mockKnowledgePresenterList.mockReturnValue({ success: true, data: [] }) + }) + + describe.each([ + { + name: 'prefetchKnowledgeBases', + run: (client: QueryClient) => prefetchKnowledgeBases(client, WORKSPACE_ID, USER_ID), + resourceType: 'knowledge_base' as const, + }, + { + name: 'prefetchTables', + run: (client: QueryClient) => prefetchTables(client, WORKSPACE_ID, USER_ID), + resourceType: 'table' as const, + }, + ])('$name folder reads', ({ run, resourceType }) => { + it('reads folders from the data layer rather than over the wire', async () => { + const folderRow = { + id: 'fld-1', + name: 'Folder', + userId: 'u-1', + workspaceId: WORKSPACE_ID, + parentId: null, + resourceType, + locked: false, + sortOrder: 0, + createdAt: '2026-01-01T00:00:00.000Z', + updatedAt: '2026-01-02T00:00:00.000Z', + deletedAt: null, + } + mockListFoldersForWorkspace.mockResolvedValue([folderRow]) + const client = makeClient() + + await run(client) + + expect(mockListFoldersForWorkspace).toHaveBeenCalledWith(WORKSPACE_ID, 'active', resourceType) + const cached = client.getQueryData( + folderKeys.list(WORKSPACE_ID, 'active', resourceType) + ) as Array<{ + resourceType: string + createdAt: Date + }> + expect(cached).toHaveLength(1) + expect(cached[0].resourceType).toBe(resourceType) + expect(cached[0].createdAt).toBeInstanceOf(Date) + }) + + it('skips the folder read when the viewer cannot be proved', async () => { + mockGetWorkspaceHostContextForViewer.mockResolvedValue(null) + const client = makeClient() + + await run(client) + + expect(mockListFoldersForWorkspace).not.toHaveBeenCalled() + expect( + client.getQueryData(folderKeys.list(WORKSPACE_ID, 'active', resourceType)) + ).toBeUndefined() + }) }) describe('prefetchKnowledgeBases', () => { - it('primes the exact key useKnowledgeBasesQuery reads and unwraps data', async () => { - const bases = [{ id: 'kb-1' }] - mockPrefetchInternalJson.mockResolvedValue({ data: bases }) + /** + * The bases list is a protected read behind an application operation, so the prefetch runs + * the same use case the route declares, with a principal from the same auth policy — + * rather than reaching past it to a manager. + */ + it('runs the route’s own use case with a session principal', async () => { const client = makeClient() - await prefetchKnowledgeBases(client, WORKSPACE_ID) + await prefetchKnowledgeBases(client, WORKSPACE_ID, USER_ID) - expect(mockPrefetchInternalJson).toHaveBeenCalledWith( - `/api/knowledge?workspaceId=${WORKSPACE_ID}&scope=active` - ) - expect(client.getQueryData(knowledgeKeys.list(WORKSPACE_ID, 'active'))).toEqual(bases) + expect(mockAuthenticate).toHaveBeenCalled() + expect(mockListInternalKnowledgeBases).toHaveBeenCalledWith({ + principal: { kind: 'session', userId: USER_ID, sessionId: 'sess-1' }, + input: { workspaceId: WORKSPACE_ID, scope: 'active' }, + }) + expect(client.getQueryData(knowledgeKeys.list(WORKSPACE_ID, 'active'))).toEqual([]) + }) + + it('caches nothing when the session principal cannot be built', async () => { + mockAuthenticate.mockRejectedValue(new Error('Unauthorized')) + const client = makeClient() + + await prefetchKnowledgeBases(client, WORKSPACE_ID, USER_ID) + + expect(mockListInternalKnowledgeBases).not.toHaveBeenCalled() + expect(client.getQueryData(knowledgeKeys.list(WORKSPACE_ID, 'active'))).toBeUndefined() }) }) describe('prefetchTables', () => { - it('primes the exact key useTablesList reads and unwraps data.tables', async () => { - const tables = [{ id: 't-1' }] - mockPrefetchInternalJson.mockResolvedValue({ data: { tables } }) + const TABLE_ROW = { + id: 't-1', + name: 'people', + description: null, + schema: { columns: [{ id: 'c1', name: 'name', type: 'string' }] }, + metadata: { columnWidths: { c1: 120 } }, + rowCount: 3, + maxRows: 10_000, + workspaceId: WORKSPACE_ID, + folderId: null, + createdBy: 'u-1', + locks: { + schemaLocked: false, + insertLocked: false, + updateLocked: false, + deleteLocked: false, + }, + archivedAt: null, + createdAt: new Date('2026-01-01T00:00:00.000Z'), + updatedAt: new Date('2026-01-02T00:00:00.000Z'), + } + + it('reads tables from the data layer', async () => { + mockListTables.mockResolvedValue([TABLE_ROW]) const client = makeClient() - await prefetchTables(client, WORKSPACE_ID) + await prefetchTables(client, WORKSPACE_ID, USER_ID) - expect(mockPrefetchInternalJson).toHaveBeenCalledWith( - `/api/table?workspaceId=${WORKSPACE_ID}&scope=active` - ) - expect(client.getQueryData(tableKeys.list(WORKSPACE_ID, 'active'))).toEqual(tables) + expect(mockListTables).toHaveBeenCalledWith(WORKSPACE_ID, { scope: 'active' }) }) - }) + /** + * `listTablesContract`'s response schema is a passthrough, so a client fetch caches the + * route's JSON verbatim. Seeding the raw data-layer row would put `Date`s and the + * server-only `metadata` field under a key the hook never sees them on. + */ + it('seeds the wire shape a client fetch caches, not the raw data-layer row', async () => { + mockListTables.mockResolvedValue([TABLE_ROW]) + const client = makeClient() + + await prefetchTables(client, WORKSPACE_ID, USER_ID) + + const [cached] = client.getQueryData(tableKeys.list(WORKSPACE_ID, 'active')) as Array< + Record + > + expect(cached.createdAt).toBe('2026-01-01T00:00:00.000Z') + expect(cached.updatedAt).toBe('2026-01-02T00:00:00.000Z') + expect(cached.archivedAt).toBeNull() + expect(cached).not.toHaveProperty('metadata') + expect(cached.jobStatus).toBeNull() + expect(cached.jobRowsProcessed).toBe(0) + }) + + it('caches no tables when the viewer cannot be proved', async () => { + mockGetWorkspaceHostContextForViewer.mockResolvedValue(null) + const client = makeClient() + + await prefetchTables(client, WORKSPACE_ID, USER_ID) + + expect(mockListTables).not.toHaveBeenCalled() + expect(client.getQueryData(tableKeys.list(WORKSPACE_ID, 'active'))).toBeUndefined() + }) + }) describe('prefetchFilesBrowser', () => { it('primes the folder key the client hook reads', async () => { const folders = [{ id: 'folder-1' }] @@ -150,106 +314,70 @@ describe('workspace list prefetches', () => { }, { name: 'tables', - run: (client: QueryClient) => prefetchTables(client, WORKSPACE_ID), + run: (client: QueryClient) => prefetchTables(client, WORKSPACE_ID, USER_ID), resourceType: 'table' as const, }, { name: 'knowledge', - run: (client: QueryClient) => prefetchKnowledgeBases(client, WORKSPACE_ID), + run: (client: QueryClient) => prefetchKnowledgeBases(client, WORKSPACE_ID, USER_ID), resourceType: 'knowledge_base' as const, }, ] for (const { name, run, resourceType } of chromeCases) { it(`primes pinned ids (${resourceType} + folder) and members for ${name}`, async () => { - const pinnedItems = [{ id: 'p-1', resourceId: 'r-1' }] - const members = [{ userId: 'u-1', name: 'Ada' }] - mockPrefetchInternalJson.mockImplementation(async (path: string) => { - if (path.startsWith('/api/pinned-items')) return { pinnedItems } - if (path.endsWith('/members')) return { members } - if (path.includes('/folders')) return { folders: [] } - return { success: true, files: [], data: { tables: [] } } - }) + /** + * Distinct fixtures per key: identical ones would still pass if the two pin + * namespaces were crossed. + */ + const resourcePins = [{ id: 'p-1', resourceType, resourceId: 'r-1' }] + const folderPins = [{ id: 'p-2', resourceType: 'folder' as const, resourceId: 'fld-1' }] + const members = [{ userId: 'u-1', name: 'Ada', image: null }] + mockListPinnedItemsForUser.mockImplementation( + async (_userId: string, _workspaceId: string, type: string) => + type === 'folder' ? folderPins : resourcePins + ) + mockGetWorkspaceMemberProfiles.mockResolvedValue(members) const client = makeClient() await run(client) - expect(mockPrefetchInternalJson).toHaveBeenCalledWith( - `/api/pinned-items?workspaceId=${WORKSPACE_ID}&resourceType=${resourceType}` - ) - expect(mockPrefetchInternalJson).toHaveBeenCalledWith( - `/api/pinned-items?workspaceId=${WORKSPACE_ID}&resourceType=folder` - ) - expect(mockPrefetchInternalJson).toHaveBeenCalledWith( - `/api/workspaces/${WORKSPACE_ID}/members` - ) + expect(mockListPinnedItemsForUser).toHaveBeenCalledWith(USER_ID, WORKSPACE_ID, resourceType) + expect(mockListPinnedItemsForUser).toHaveBeenCalledWith(USER_ID, WORKSPACE_ID, 'folder') + expect(mockGetWorkspaceMemberProfiles).toHaveBeenCalledWith(WORKSPACE_ID) expect(client.getQueryData(pinnedItemKeys.list(WORKSPACE_ID, resourceType))).toEqual( - pinnedItems - ) - expect(client.getQueryData(pinnedItemKeys.list(WORKSPACE_ID, 'folder'))).toEqual( - pinnedItems + resourcePins ) + expect(client.getQueryData(pinnedItemKeys.list(WORKSPACE_ID, 'folder'))).toEqual(folderPins) expect(client.getQueryData(workspaceKeys.members(WORKSPACE_ID))).toEqual(members) }) - } - }) - describe('prefetchHomeLists', () => { - it('primes folder + file keys, mapping folder rows to the client shape', async () => { - const folderRow = { - id: 'folder-1', - name: 'Docs', - userId: 'u-1', - workspaceId: WORKSPACE_ID, - parentId: null, - resourceType: 'workflow', - locked: false, - sortOrder: 0, - createdAt: '2026-01-01T00:00:00.000Z', - updatedAt: '2026-01-02T00:00:00.000Z', - deletedAt: null, - } - const files = [{ id: 'f-1' }] - mockPrefetchInternalJson.mockImplementation(async (path: string) => - path.startsWith('/api/folders') ? { folders: [folderRow] } : { success: true, files } - ) - const client = makeClient() + it(`caches no chrome for ${name} when the viewer cannot be proved`, async () => { + mockGetWorkspaceHostContextForViewer.mockResolvedValue(null) + const client = makeClient() - await prefetchHomeLists(client, WORKSPACE_ID) + await run(client) - expect(mockPrefetchInternalJson).toHaveBeenCalledWith( - `/api/folders?workspaceId=${WORKSPACE_ID}&scope=active&resourceType=workflow` - ) - const cachedFolders = client.getQueryData(folderKeys.list(WORKSPACE_ID, 'active')) as Array<{ - id: string - resourceType: string - createdAt: Date - }> - expect(cachedFolders).toHaveLength(1) - expect(cachedFolders[0].resourceType).toBe('workflow') - // The wire shape carries ISO strings; the client shape carries Dates. - expect(cachedFolders[0].createdAt).toBeInstanceOf(Date) - expect(client.getQueryData(workspaceFilesKeys.list(WORKSPACE_ID, 'active'))).toEqual(files) - }) + expect(mockListPinnedItemsForUser).not.toHaveBeenCalled() + expect(mockGetWorkspaceMemberProfiles).not.toHaveBeenCalled() + expect(client.getQueryData(pinnedItemKeys.list(WORKSPACE_ID, resourceType))).toBeUndefined() + expect(client.getQueryData(workspaceKeys.members(WORKSPACE_ID))).toBeUndefined() + }) + } }) describe('graceful failure', () => { it.each([ [ 'prefetchKnowledgeBases', - (client: QueryClient) => prefetchKnowledgeBases(client, WORKSPACE_ID), + (client: QueryClient) => prefetchKnowledgeBases(client, WORKSPACE_ID, USER_ID), knowledgeKeys.list(WORKSPACE_ID, 'active'), ], [ 'prefetchTables', - (client: QueryClient) => prefetchTables(client, WORKSPACE_ID), + (client: QueryClient) => prefetchTables(client, WORKSPACE_ID, USER_ID), tableKeys.list(WORKSPACE_ID, 'active'), ], - [ - 'prefetchHomeLists', - (client: QueryClient) => prefetchHomeLists(client, WORKSPACE_ID), - folderKeys.list(WORKSPACE_ID, 'active'), - ], [ 'prefetchFilesBrowser', (client: QueryClient) => prefetchFilesBrowser(client, WORKSPACE_ID, USER_ID), @@ -258,8 +386,13 @@ describe('workspace list prefetches', () => { ] as const)( '%s does not throw when the fetcher rejects (page still renders, client refetches)', async (_name, prefetch, queryKey) => { - mockPrefetchInternalJson.mockRejectedValue(new Error('500')) - mockListWorkspaceFilesWithShares.mockRejectedValue(new Error('500')) + const boom = new Error('500') + mockListWorkspaceFilesWithShares.mockRejectedValue(boom) + mockListFoldersForWorkspace.mockRejectedValue(boom) + mockListTables.mockRejectedValue(boom) + mockListInternalKnowledgeBases.mockRejectedValue(boom) + mockListPinnedItemsForUser.mockRejectedValue(boom) + mockGetWorkspaceMemberProfiles.mockRejectedValue(boom) const client = makeClient() await expect(prefetch(client)).resolves.toBeUndefined() diff --git a/apps/sim/app/workspace/[workspaceId]/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/prefetch.ts index a356c96ab5b..0023b34d851 100644 --- a/apps/sim/app/workspace/[workspaceId]/prefetch.ts +++ b/apps/sim/app/workspace/[workspaceId]/prefetch.ts @@ -120,12 +120,11 @@ async function seedWorkspaceList( * seeds nothing, leaving the client fetch to reach `GET /api/workspaces`' * default-workspace creation path — the same outcome a rejecting `queryFn` used * to produce, without routing a normal state through the error channel. That - * matters because `makeQueryClient` dehydrates pending queries and sets - * `retryOnMount: false`: were this read ever deferred, its rejection would - * hydrate the client query into an error state nothing retries, permanently - * locking a brand-new viewer out of workspace creation. Seeding also skips the - * `retry: 1` default, which previously ran the whole read a second time, a - * retry delay later, purely to re-derive an outcome already known. + * matters because only a settled query is dehydrated: an unawaited read would be + * dropped from the payload entirely, so the switcher would waterfall on every + * cold load rather than paint populated. Seeding also skips the `retry` default, + * which previously ran the whole read a second time, a retry delay later, purely + * to re-derive an outcome already known. */ export async function prefetchWorkspaceSidebar( queryClient: QueryClient, diff --git a/apps/sim/app/workspace/[workspaceId]/settings/[section]/page.tsx b/apps/sim/app/workspace/[workspaceId]/settings/[section]/page.tsx index 2f84f225401..22d0fbc4f9a 100644 --- a/apps/sim/app/workspace/[workspaceId]/settings/[section]/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/settings/[section]/page.tsx @@ -25,7 +25,7 @@ import { } from '@/app/workspace/[workspaceId]/settings/navigation' import { resolveWorkspaceGroup } from '@/ee/access-control/utils/permission-check' import { isForkingAvailableForWorkspace } from '@/ee/workspace-forking/lib/lineage/authz' -import { prefetchGeneralSettings, prefetchUserProfile } from './prefetch' +import { prefetchGeneralSettings } from './prefetch' import { SettingsPage } from './settings' interface WorkspaceSettingsSectionPageProps { @@ -170,8 +170,13 @@ export default async function WorkspaceSettingsSectionPage({ } const queryClient = getQueryClient() - void prefetchGeneralSettings(queryClient) - void prefetchUserProfile(queryClient) + /** + * Awaited, not fired and forgotten: only a settled query is dehydrated, so an unawaited + * prefetch is dropped from the payload and the panel waterfalls anyway. The viewer's + * profile is already seeded by the workspace layout under the same key, so it is not + * repeated here. + */ + await prefetchGeneralSettings(queryClient) return ( diff --git a/apps/sim/app/workspace/[workspaceId]/settings/[section]/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/settings/[section]/prefetch.ts index 059690a037b..9cbcf3d5f61 100644 --- a/apps/sim/app/workspace/[workspaceId]/settings/[section]/prefetch.ts +++ b/apps/sim/app/workspace/[workspaceId]/settings/[section]/prefetch.ts @@ -1,24 +1,21 @@ import type { QueryClient } from '@tanstack/react-query' -import { headers } from 'next/headers' import { getSession } from '@/lib/auth' -import { getInternalApiBaseUrl } from '@/lib/core/utils/urls' -import { getUserProfile, getUserSettings } from '@/lib/users/queries' +import { getUserSettings } from '@/lib/users/queries' import { GENERAL_SETTINGS_STALE_TIME, generalSettingsKeys, mapGeneralSettingsResponse, } from '@/hooks/queries/general-settings' -import { SUBSCRIPTION_DATA_STALE_TIME, subscriptionKeys } from '@/hooks/queries/subscription' -import { - mapUserProfileResponse, - USER_PROFILE_STALE_TIME, - userProfileKeys, -} from '@/hooks/queries/user-profile' /** * Prefetch general settings server-side via the shared data layer. - * Uses the same query keys as the client `useGeneralSettings` hook - * so data is shared via HydrationBoundary. + * + * Uses the same query key and mapper as the client `useGeneralSettings` hook, so the + * hydrated entry is indistinguishable from one a client fetch produced. + * + * Callers must `await` this. Only a settled query is dehydrated, so an unawaited prefetch + * is dropped from the payload entirely and the panel waterfalls on every load as if it had + * never been prefetched. */ export function prefetchGeneralSettings(queryClient: QueryClient) { return queryClient.prefetchQuery({ @@ -31,48 +28,3 @@ export function prefetchGeneralSettings(queryClient: QueryClient) { staleTime: GENERAL_SETTINGS_STALE_TIME, }) } - -/** - * Prefetch subscription data server-side. Unlike the other prefetches this goes - * through the internal billing API rather than calling the data layer directly: - * the billing summary contains `Date` fields (and an untyped `metadata` blob) that - * `NextResponse.json` serializes to the string wire shape the client caches. Going - * through the route yields that exact shape, avoiding a Date-vs-string mismatch - * between server-hydrated and client-fetched data. Uses the same query key as the - * client `useSubscriptionData` hook (with includeOrg=false) so data is shared via - * HydrationBoundary. - */ -export function prefetchSubscriptionData(queryClient: QueryClient) { - return queryClient.prefetchQuery({ - queryKey: subscriptionKeys.user(false), - queryFn: async () => { - const h = await headers() - const cookie = h.get('cookie') - const response = await fetch(`${getInternalApiBaseUrl()}/api/billing?context=user`, { - headers: cookie ? { cookie } : {}, - }) - if (!response.ok) throw new Error(`Subscription prefetch failed: ${response.status}`) - return response.json() - }, - staleTime: SUBSCRIPTION_DATA_STALE_TIME, - }) -} - -/** - * Prefetch user profile server-side via the shared data layer. - * Uses the same query keys as the client `useUserProfile` hook - * so data is shared via HydrationBoundary. - */ -export function prefetchUserProfile(queryClient: QueryClient) { - return queryClient.prefetchQuery({ - queryKey: userProfileKeys.profile(), - queryFn: async () => { - const session = await getSession() - if (!session?.user?.id) throw new Error('Unauthorized') - const user = await getUserProfile(session.user.id) - if (!user) throw new Error('User not found') - return mapUserProfileResponse(user) - }, - staleTime: USER_PROFILE_STALE_TIME, - }) -} diff --git a/apps/sim/app/workspace/[workspaceId]/tables/page.tsx b/apps/sim/app/workspace/[workspaceId]/tables/page.tsx index 0e9390a5d95..1e9cb1f0592 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/page.tsx @@ -1,6 +1,7 @@ import { Suspense } from 'react' import { dehydrate, HydrationBoundary } from '@tanstack/react-query' import type { Metadata } from 'next' +import { getSession } from '@/lib/auth' import { getQueryClient } from '@/app/_shell/providers/get-query-client' import TablesLoading from '@/app/workspace/[workspaceId]/tables/loading' import { prefetchTables } from '@/app/workspace/[workspaceId]/tables/prefetch' @@ -17,10 +18,9 @@ export const metadata: Metadata = { * route-level `loading.tsx` covers the navigation/chunk-load transition. */ export default async function TablesPage({ params }: { params: Promise<{ workspaceId: string }> }) { - const { workspaceId } = await params - + const [{ workspaceId }, session] = await Promise.all([params, getSession()]) const queryClient = getQueryClient() - await prefetchTables(queryClient, workspaceId) + await prefetchTables(queryClient, workspaceId, session?.user?.id) return ( diff --git a/apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts index 5a548885511..a937a26e753 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts +++ b/apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts @@ -1,9 +1,9 @@ import type { QueryClient } from '@tanstack/react-query' -import type { FolderApi } from '@/lib/api/contracts/folders' -import type { TableDefinition } from '@/lib/table' -import { prefetchInternalJson } from '@/app/workspace/[workspaceId]/lib/prefetch-internal-fetch' +import { listTables } from '@/lib/table/service' +import { toTableListItem } from '@/lib/table/wire' +import { getWorkspaceHostContextForViewer } from '@/lib/workspaces/host-context' +import { prefetchResourceFolders } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-folders' import { prefetchResourceListChrome } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome' -import { FOLDER_LIST_STALE_TIME, folderKeys, mapFolder } from '@/hooks/queries/utils/folder-keys' import { TABLE_LIST_STALE_TIME, tableKeys } from '@/hooks/queries/utils/table-keys' /** @@ -14,33 +14,36 @@ import { TABLE_LIST_STALE_TIME, tableKeys } from '@/hooks/queries/utils/table-ke * only placed correctly relative to the folder rows it sits beside, so * prefetching one without the other still flashes an ungrouped list. * - * Table definitions carry `Date` fields, so the list goes through the - * `/api/table` route and caches the serialized wire shape — see - * {@link prefetchInternalJson}. Folders are mapped with the same `mapFolder` the - * hook applies so the hydrated entry matches a client fetch exactly. + * The list goes through {@link toTableListItem}, the projection `GET /api/table` itself + * returns, because `listTablesContract`'s response schema is a passthrough that neither + * coerces nor strips — the client caches the route's JSON verbatim, so seeding raw rows + * would put `Date` objects and the server-only `metadata` field under that key. + * + * The read carries no authorization of its own, so the viewer is proved first. + * `getWorkspaceHostContextForViewer` resolves the same effective workspace permission the + * route's own check does (both bottom out in `checkWorkspaceAccess`), and it is `cache`d and + * already resolved by the layout for this request, so it costs no additional queries. A viewer + * without access caches nothing and the client fetch reaches the route for the real 403. */ -export async function prefetchTables(queryClient: QueryClient, workspaceId: string): Promise { +export async function prefetchTables( + queryClient: QueryClient, + workspaceId: string, + userId: string | undefined +): Promise { + if (!userId) return + const hostContext = await getWorkspaceHostContextForViewer(workspaceId, userId) + if (!hostContext) return + await Promise.all([ queryClient.prefetchQuery({ queryKey: tableKeys.list(workspaceId, 'active'), queryFn: async () => { - const response = await prefetchInternalJson<{ data: { tables: TableDefinition[] } }>( - `/api/table?workspaceId=${workspaceId}&scope=active` - ) - return response.data.tables + const tables = await listTables(workspaceId, { scope: 'active' }) + return tables.map(toTableListItem) }, staleTime: TABLE_LIST_STALE_TIME, }), - queryClient.prefetchQuery({ - queryKey: folderKeys.list(workspaceId, 'active', 'table'), - queryFn: async () => { - const { folders } = await prefetchInternalJson<{ folders?: FolderApi[] }>( - `/api/folders?workspaceId=${workspaceId}&scope=active&resourceType=table` - ) - return (folders ?? []).map(mapFolder) - }, - staleTime: FOLDER_LIST_STALE_TIME, - }), - prefetchResourceListChrome(queryClient, workspaceId, 'table'), + prefetchResourceFolders(queryClient, workspaceId, 'table', userId), + prefetchResourceListChrome(queryClient, workspaceId, 'table', userId), ]) } diff --git a/apps/sim/blocks/blocks/datadog.ts b/apps/sim/blocks/blocks/datadog.ts index 776edecd1ca..96e090aa262 100644 --- a/apps/sim/blocks/blocks/datadog.ts +++ b/apps/sim/blocks/blocks/datadog.ts @@ -81,9 +81,7 @@ export const DatadogBlock: BlockConfig = { value: () => 'datadog_submit_metrics', }, - // ======================== // Submit Metrics inputs - // ======================== { id: 'series', title: 'Metrics Data (JSON)', @@ -113,9 +111,7 @@ Return ONLY valid JSON - no explanations, no markdown code blocks.`, }, }, - // ======================== // Query Timeseries inputs - // ======================== { id: 'query', title: 'Query', @@ -179,9 +175,7 @@ Return ONLY the numeric timestamp - no explanations, no quotes, no extra text.`, }, }, - // ======================== // Create Event inputs - // ======================== { id: 'title', title: 'Event Title', @@ -251,9 +245,7 @@ Return the event description text directly - no extra formatting needed.`, mode: 'advanced', }, - // ======================== // Create Monitor inputs - // ======================== { id: 'name', title: 'Monitor Name', @@ -366,9 +358,7 @@ Return ONLY valid JSON - no explanations, no markdown code blocks.`, }, }, - // ======================== // Get Monitor inputs - // ======================== { id: 'monitorId', title: 'Monitor ID', @@ -378,9 +368,7 @@ Return ONLY valid JSON - no explanations, no markdown code blocks.`, required: true, }, - // ======================== // List Monitors inputs - // ======================== { id: 'listMonitorName', title: 'Filter by Name', @@ -398,9 +386,7 @@ Return ONLY valid JSON - no explanations, no markdown code blocks.`, mode: 'advanced', }, - // ======================== // Mute Monitor inputs - // ======================== { id: 'muteMonitorId', title: 'Monitor ID', @@ -439,9 +425,7 @@ Return ONLY the numeric timestamp - no explanations, no quotes, no extra text.`, }, }, - // ======================== // Query Logs inputs - // ======================== { id: 'logQuery', title: 'Search Query', @@ -514,9 +498,7 @@ Return ONLY the relative time string - no explanations, no quotes, no extra text mode: 'advanced', }, - // ======================== // Send Logs inputs - // ======================== { id: 'logs', title: 'Logs (JSON)', @@ -547,9 +529,7 @@ Return ONLY valid JSON - no explanations, no markdown code blocks.`, }, }, - // ======================== // Create Downtime inputs - // ======================== { id: 'downtimeScope', title: 'Scope', @@ -626,9 +606,7 @@ Return ONLY the numeric timestamp - no explanations, no quotes, no extra text.`, mode: 'advanced', }, - // ======================== // List Downtimes inputs - // ======================== { id: 'currentOnly', title: 'Current Only', @@ -637,9 +615,7 @@ Return ONLY the numeric timestamp - no explanations, no quotes, no extra text.`, mode: 'advanced', }, - // ======================== // Cancel Downtime inputs - // ======================== { id: 'downtimeId', title: 'Downtime ID', @@ -649,9 +625,7 @@ Return ONLY the numeric timestamp - no explanations, no quotes, no extra text.`, required: true, }, - // ======================== // Authentication (common) - // ======================== { id: 'apiKey', title: 'API Key', diff --git a/apps/sim/blocks/blocks/datagma.ts b/apps/sim/blocks/blocks/datagma.ts index fe29ea7eb8f..a58afaa8c05 100644 --- a/apps/sim/blocks/blocks/datagma.ts +++ b/apps/sim/blocks/blocks/datagma.ts @@ -52,9 +52,7 @@ export const DatagmaBlock: BlockConfig = { value: () => 'datagma_find_email', }, - // ------------------------------------------------------------------------- // Find Email - // ------------------------------------------------------------------------- { id: 'fe_fullName', title: 'Full Name', @@ -80,9 +78,7 @@ export const DatagmaBlock: BlockConfig = { mode: 'advanced', }, - // ------------------------------------------------------------------------- // Enrich Person - // ------------------------------------------------------------------------- { id: 'ep_data', title: 'Email, LinkedIn URL, or Full Name', @@ -130,9 +126,7 @@ export const DatagmaBlock: BlockConfig = { mode: 'advanced', }, - // ------------------------------------------------------------------------- // Enrich Company - // ------------------------------------------------------------------------- { id: 'ec_data', title: 'Company Domain, Name, or SIREN', @@ -166,9 +160,7 @@ export const DatagmaBlock: BlockConfig = { mode: 'advanced', }, - // ------------------------------------------------------------------------- // Find Phone - // ------------------------------------------------------------------------- { id: 'fp_username', title: 'LinkedIn URL', @@ -185,9 +177,7 @@ export const DatagmaBlock: BlockConfig = { condition: { field: 'operation', value: 'datagma_find_phone' }, }, - // ------------------------------------------------------------------------- // API Key — hidden on hosted Sim for operations with hosted-key support - // ------------------------------------------------------------------------- { id: 'apiKey', title: 'API Key', diff --git a/apps/sim/blocks/blocks/google_maps.ts b/apps/sim/blocks/blocks/google_maps.ts index 82ab10f35fc..45930ae51e6 100644 --- a/apps/sim/blocks/blocks/google_maps.ts +++ b/apps/sim/blocks/blocks/google_maps.ts @@ -124,7 +124,6 @@ export const GoogleMapsBlock: BlockConfig = { condition: { field: 'operation', value: 'speed_limits' }, }, - // ========== Geocode ========== { id: 'address', title: 'Address', diff --git a/apps/sim/blocks/blocks/google_slides.ts b/apps/sim/blocks/blocks/google_slides.ts index 3288312f37c..32b8299893c 100644 --- a/apps/sim/blocks/blocks/google_slides.ts +++ b/apps/sim/blocks/blocks/google_slides.ts @@ -480,7 +480,6 @@ export const GoogleSlidesBlock: BlockConfig = { }, }, - // ========== Write Operation Fields ========== { id: 'slideIndex', title: 'Slide Index', @@ -508,7 +507,6 @@ Return ONLY the slide content - no explanations, no markdown formatting markers, }, }, - // ========== Create Operation Fields ========== { id: 'title', title: 'Presentation Title', @@ -578,7 +576,6 @@ Return ONLY the slide content - no explanations, no markdown formatting markers, }, }, - // ========== Replace All Text Operation Fields ========== { id: 'findText', title: 'Find Text', @@ -618,7 +615,6 @@ Return ONLY the replacement text - no explanations, no quotes, no extra text.`, mode: 'advanced', }, - // ========== Add Slide Operation Fields ========== { id: 'layout', title: 'Slide Layout', @@ -675,7 +671,6 @@ Return ONLY the JSON array - no explanations, no markdown, no extra text.`, }, }, - // ========== Add Image Operation Fields ========== { id: 'pageObjectId', title: 'Slide ID', @@ -735,7 +730,6 @@ Return ONLY the JSON array - no explanations, no markdown, no extra text.`, condition: { field: 'operation', value: 'add_image' }, }, - // ========== Get Thumbnail Operation Fields ========== { id: 'thumbnailPageId', title: 'Slide ID', @@ -765,7 +759,6 @@ Return ONLY the JSON array - no explanations, no markdown, no extra text.`, value: () => 'PNG', }, - // ========== Get Page Operation Fields ========== { id: 'getPageObjectId', title: 'Page/Slide ID', @@ -775,7 +768,6 @@ Return ONLY the JSON array - no explanations, no markdown, no extra text.`, required: true, }, - // ========== Delete Object Operation Fields ========== { id: 'deleteObjectId', title: 'Object ID', @@ -785,7 +777,6 @@ Return ONLY the JSON array - no explanations, no markdown, no extra text.`, required: true, }, - // ========== Duplicate Object Operation Fields ========== { id: 'duplicateObjectId', title: 'Object ID', @@ -803,7 +794,6 @@ Return ONLY the JSON array - no explanations, no markdown, no extra text.`, mode: 'advanced', }, - // ========== Reorder Slides Operation Fields ========== { id: 'reorderSlideIds', title: 'Slide IDs', @@ -821,7 +811,6 @@ Return ONLY the JSON array - no explanations, no markdown, no extra text.`, required: true, }, - // ========== Create Table Operation Fields ========== { id: 'tablePageObjectId', title: 'Slide ID', @@ -875,7 +864,6 @@ Return ONLY the JSON array - no explanations, no markdown, no extra text.`, condition: { field: 'operation', value: 'create_table' }, }, - // ========== Create Shape Operation Fields ========== { id: 'shapePageObjectId', title: 'Slide ID', @@ -936,7 +924,6 @@ Return ONLY the JSON array - no explanations, no markdown, no extra text.`, condition: { field: 'operation', value: 'create_shape' }, }, - // ========== Insert Text Operation Fields ========== { id: 'insertTextObjectId', title: 'Object ID', @@ -972,7 +959,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, condition: { field: 'operation', value: 'insert_text' }, }, - // ========== Copy Presentation Operation Fields ========== { id: 'sourcePresentationSelector', title: 'Source Presentation', @@ -1031,7 +1017,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, condition: { field: 'operation', value: 'copy_presentation' }, }, - // ========== Export Presentation Operation Fields ========== { id: 'exportFormat', title: 'Export Format', @@ -1049,7 +1034,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, condition: { field: 'operation', value: 'export_presentation' }, }, - // ========== Batch Update (Raw) Operation Fields ========== { id: 'requestsJson', title: 'Requests (JSON Array)', @@ -1074,7 +1058,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, mode: 'advanced', }, - // ========== Replace All Shapes With Image Fields ========== { id: 'replaceShapesImageUrl', title: 'Image URL', @@ -1117,7 +1100,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, mode: 'advanced', }, - // ========== Replace Image Fields ========== { id: 'replaceImageObjectId', title: 'Image Object ID', @@ -1146,7 +1128,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, condition: { field: 'operation', value: 'replace_image' }, }, - // ========== Update Image Properties Fields ========== { id: 'imagePropsObjectId', title: 'Image Object ID', @@ -1223,7 +1204,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, mode: 'advanced', }, - // ========== Text Style Fields ========== { id: 'textObjectId', title: 'Object ID', @@ -1551,7 +1531,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, condition: { field: 'operation', value: 'create_paragraph_bullets' }, }, - // ========== Update Shape Properties Fields ========== { id: 'shapePropsObjectId', title: 'Shape Object ID', @@ -1656,7 +1635,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, mode: 'advanced', }, - // ========== Update Page Properties Fields ========== { id: 'pagePropsObjectId', title: 'Slide ID', @@ -1710,7 +1688,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, mode: 'advanced', }, - // ========== Update Slide Properties Fields ========== { id: 'slidePropsObjectId', title: 'Slide ID', @@ -1741,7 +1718,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, mode: 'advanced', }, - // ========== Update Alt Text Fields ========== { id: 'altTextObjectId', title: 'Element Object ID', @@ -1762,7 +1738,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, condition: { field: 'operation', value: 'update_page_element_alt_text' }, }, - // ========== Update Element Transform Fields ========== { id: 'transformObjectId', title: 'Element Object ID', @@ -1822,7 +1797,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, condition: { field: 'operation', value: 'update_page_element_transform' }, }, - // ========== Z-Order Fields ========== { id: 'zOrderObjectIds', title: 'Object IDs', @@ -1845,7 +1819,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, required: true, }, - // ========== Group / Ungroup Fields ========== { id: 'groupChildrenObjectIds', title: 'Children Object IDs', @@ -1871,7 +1844,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, required: true, }, - // ========== Create Line Fields ========== { id: 'linePageObjectId', title: 'Slide ID', @@ -1919,7 +1891,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, condition: { field: 'operation', value: 'create_line' }, }, - // ========== Update Line Properties Fields ========== { id: 'linePropsObjectId', title: 'Line Object ID', @@ -1987,7 +1958,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, mode: 'advanced', }, - // ========== Update Line Category Fields ========== { id: 'lineCategoryObjectId', title: 'Line Object ID', @@ -2008,7 +1978,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, required: true, }, - // ========== Reroute Line Fields ========== { id: 'rerouteLineObjectId', title: 'Line Object ID', @@ -2017,7 +1986,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, required: true, }, - // ========== Table Row/Column Insert/Delete Fields ========== { id: 'tableTargetObjectId', title: 'Table Object ID', @@ -2089,7 +2057,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, condition: { field: 'operation', value: 'insert_table_columns' }, }, - // ========== Merge / Unmerge / Cell / Border Table Range Fields ========== { id: 'tableRangeObjectId', title: 'Table Object ID', @@ -2170,7 +2137,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, required: true, }, - // ========== Update Table Cell Properties Fields ========== { id: 'tableCellBackgroundColor', title: 'Cell Background Color', @@ -2213,7 +2179,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, mode: 'advanced', }, - // ========== Update Table Border Properties Fields ========== { id: 'tableBorderPosition', title: 'Border Position', @@ -2269,7 +2234,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, mode: 'advanced', }, - // ========== Update Table Column Properties Fields ========== { id: 'tableColumnPropsObjectId', title: 'Table Object ID', @@ -2307,7 +2271,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, mode: 'advanced', }, - // ========== Update Table Row Properties Fields ========== { id: 'tableRowPropsObjectId', title: 'Table Object ID', @@ -2345,7 +2308,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, mode: 'advanced', }, - // ========== Sheets Chart Embed Fields ========== { id: 'chartPageObjectId', title: 'Slide ID', @@ -2416,7 +2378,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, condition: { field: 'operation', value: 'create_sheets_chart' }, }, - // ========== Refresh Sheets Chart Fields ========== { id: 'refreshChartObjectId', title: 'Chart Object ID', @@ -2425,7 +2386,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, required: true, }, - // ========== Replace All Shapes With Sheets Chart Fields ========== { id: 'replaceShapesChartFindText', title: 'Find Text (Token)', @@ -2449,7 +2409,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, mode: 'advanced', }, - // ========== Create Video Fields ========== { id: 'videoPageObjectId', title: 'Slide ID', @@ -2505,7 +2464,6 @@ Return ONLY the text content - no explanations, no markdown formatting markers, condition: { field: 'operation', value: 'create_video' }, }, - // ========== Update Video Properties Fields ========== { id: 'videoPropsObjectId', title: 'Video Object ID', diff --git a/apps/sim/blocks/blocks/icypeas.ts b/apps/sim/blocks/blocks/icypeas.ts index 176a2963ab3..061d02e9a80 100644 --- a/apps/sim/blocks/blocks/icypeas.ts +++ b/apps/sim/blocks/blocks/icypeas.ts @@ -45,9 +45,7 @@ export const IcypeasBlock: BlockConfig = { value: () => 'icypeas_find_email', }, - // ----------------------------------------------------------------------- // Find Email - // ----------------------------------------------------------------------- { id: 'fe_firstname', title: 'First Name', @@ -71,9 +69,7 @@ export const IcypeasBlock: BlockConfig = { condition: { field: 'operation', value: 'icypeas_find_email' }, }, - // ----------------------------------------------------------------------- // Verify Email - // ----------------------------------------------------------------------- { id: 've_email', title: 'Email Address', @@ -83,9 +79,7 @@ export const IcypeasBlock: BlockConfig = { condition: { field: 'operation', value: 'icypeas_verify_email' }, }, - // ----------------------------------------------------------------------- // API Key — hidden on hosted Sim for all operations (hosted-key supported) - // ----------------------------------------------------------------------- { id: 'apiKey', title: 'API Key', diff --git a/apps/sim/blocks/blocks/s3.ts b/apps/sim/blocks/blocks/s3.ts index 1c9953d349d..9a341a5a055 100644 --- a/apps/sim/blocks/blocks/s3.ts +++ b/apps/sim/blocks/blocks/s3.ts @@ -186,7 +186,6 @@ export const S3Block: BlockConfig = { required: true, }, - // ===== UPLOAD (PUT OBJECT) FIELDS ===== { id: 'objectKey', title: 'Object Key/Path', @@ -245,7 +244,6 @@ export const S3Block: BlockConfig = { mode: 'advanced', }, - // ===== DOWNLOAD (GET OBJECT) FIELDS ===== { id: 's3Uri', title: 'S3 Object URL', @@ -255,7 +253,6 @@ export const S3Block: BlockConfig = { required: true, }, - // ===== LIST OBJECTS FIELDS ===== { id: 'prefix', title: 'Prefix/Folder', @@ -292,7 +289,6 @@ export const S3Block: BlockConfig = { required: true, }, - // ===== COPY OBJECT FIELDS ===== { id: 'sourceBucket', title: 'Source Bucket', diff --git a/apps/sim/lib/api/contracts/tables.ts b/apps/sim/lib/api/contracts/tables.ts index e668a3a1d5a..2c5ed7ad829 100644 --- a/apps/sim/lib/api/contracts/tables.ts +++ b/apps/sim/lib/api/contracts/tables.ts @@ -1058,13 +1058,11 @@ export type InsertTableRowBodyInput = z.input export type BatchInsertTableRowsBodyInput = z.input export type BatchUpdateTableRowsBodyInput = z.input export type UpdateTableRowBodyInput = z.input -// ============================================================================ // CSV import form schemas // // Both `/api/table/import-csv` and `/api/table/[tableId]/import-csv` parse a // `multipart/form-data` body, so these schemas are validated *form-field by // form-field* in the routes (not as a single contract body). -// ============================================================================ export const csvFileSchema = z .unknown() @@ -1409,10 +1407,8 @@ export const deleteTableRowsAsyncContract = defineRouteContract({ }, }) -// ============================================================================ // Workflow group contracts (`/api/table/[tableId]/groups`, `/cancel-runs`, // `/columns/run`, `/rows/run`, `/rows/[rowId]/cells/[groupId]/run`) -// ============================================================================ const workflowGroupOutputSchema = z.object({ // Workflow outputs carry blockId/path; enrichment outputs carry outputId and diff --git a/apps/sim/lib/copilot/chat/persisted-message.ts b/apps/sim/lib/copilot/chat/persisted-message.ts index 76372f25d9e..54b66e90c95 100644 --- a/apps/sim/lib/copilot/chat/persisted-message.ts +++ b/apps/sim/lib/copilot/chat/persisted-message.ts @@ -173,9 +173,7 @@ export function stripToolResultOutput(message: PersistedMessage): PersistedMessa return changed ? { ...message, contentBlocks } : message } -// --------------------------------------------------------------------------- // Write: OrchestratorResult → PersistedMessage -// --------------------------------------------------------------------------- function resolveToolState(block: ContentBlock): PersistedToolState { const tc = block.toolCall @@ -424,11 +422,9 @@ export function buildPersistedUserMessage(params: UserMessageParams): PersistedM return message } -// --------------------------------------------------------------------------- // Read: raw JSONB → PersistedMessage // Handles both canonical (type: 'tool', 'text', 'span', 'complete') and // legacy (type: 'tool_call', 'thinking', 'subagent', 'stopped') blocks. -// --------------------------------------------------------------------------- const CANONICAL_BLOCK_TYPES: Set = new Set(Object.values(MothershipStreamV1EventType)) diff --git a/apps/sim/lib/copilot/chat/process-contents.ts b/apps/sim/lib/copilot/chat/process-contents.ts index 689eaed8a02..c21214d8367 100644 --- a/apps/sim/lib/copilot/chat/process-contents.ts +++ b/apps/sim/lib/copilot/chat/process-contents.ts @@ -811,9 +811,7 @@ async function processExecutionLogFromDb( } } -// --------------------------------------------------------------------------- // Active resource context resolution (direct DB lookups, workspace-scoped) -// --------------------------------------------------------------------------- /** * Resolves the content of the currently active resource tab via direct DB diff --git a/apps/sim/lib/copilot/request/lifecycle/run.ts b/apps/sim/lib/copilot/request/lifecycle/run.ts index 389f31564a2..1471eddfba9 100644 --- a/apps/sim/lib/copilot/request/lifecycle/run.ts +++ b/apps/sim/lib/copilot/request/lifecycle/run.ts @@ -434,9 +434,7 @@ export async function runCopilotLifecycle( } } -// --------------------------------------------------------------------------- // Per-subagent checkpoint resume (concurrent fan-out) -// --------------------------------------------------------------------------- // // Under the per-subagent checkpoint model each paused subagent is its OWN // checkpoint chain (frame.checkpointId) joined at the orchestrator. Instead of @@ -754,9 +752,7 @@ async function driveSubagentChains( } } -// --------------------------------------------------------------------------- // Checkpoint loop – the core state machine -// --------------------------------------------------------------------------- async function runCheckpointLoop( initialPayload: Record, @@ -1119,9 +1115,7 @@ async function runCheckpointLoop( } } -// --------------------------------------------------------------------------- // Execution context builder -// --------------------------------------------------------------------------- async function buildExecutionContext( requestPayload: Record, @@ -1245,9 +1239,7 @@ async function ensureHeadlessRunIdentity(input: { } } -// --------------------------------------------------------------------------- // Helpers -// --------------------------------------------------------------------------- /** * Adds `enterpriseByokEligible: true` to the initial mothership payload when the diff --git a/apps/sim/lib/copilot/request/lifecycle/start.ts b/apps/sim/lib/copilot/request/lifecycle/start.ts index 3067e971335..5a6ed9f0bb2 100644 --- a/apps/sim/lib/copilot/request/lifecycle/start.ts +++ b/apps/sim/lib/copilot/request/lifecycle/start.ts @@ -420,9 +420,7 @@ export function createSSEStream(params: StreamingOrchestrationParams): ReadableS }) } -// --------------------------------------------------------------------------- // Title generation (fire-and-forget side effect) -// --------------------------------------------------------------------------- function fireTitleGeneration(params: { chatId?: string @@ -483,9 +481,7 @@ function fireTitleGeneration(params: { }) } -// --------------------------------------------------------------------------- // Chat title helper -// --------------------------------------------------------------------------- export async function requestChatTitle(params: { message: string diff --git a/apps/sim/lib/copilot/request/session/contract.ts b/apps/sim/lib/copilot/request/session/contract.ts index dde683b966c..65b9de6cc99 100644 --- a/apps/sim/lib/copilot/request/session/contract.ts +++ b/apps/sim/lib/copilot/request/session/contract.ts @@ -148,9 +148,7 @@ export type ParseStreamEventEnvelopeResult = | ParseStreamEventEnvelopeSuccess | ParseStreamEventEnvelopeFailure -// --------------------------------------------------------------------------- // Structural helpers (CSP-safe – no codegen / eval / new Function) -// --------------------------------------------------------------------------- function isOptionalString(value: unknown): value is string | undefined { return value === undefined || typeof value === 'string' @@ -188,7 +186,6 @@ function isStreamScope(value: unknown): value is MothershipStreamV1StreamScope { ) } -// --------------------------------------------------------------------------- // Contract envelope validator (replaces Ajv runtime compilation) // // Validates the envelope shell (v, seq, ts, stream, trace?, scope?) and that @@ -196,7 +193,6 @@ function isStreamScope(value: unknown): value is MothershipStreamV1StreamScope { // Per-payload-variant validation is intentionally lightweight: the server // already performs strict schema validation; the client only needs enough // structural checking to safely dispatch inside the switch statement. -// --------------------------------------------------------------------------- const KNOWN_EVENT_TYPES: ReadonlySet = new Set(Object.values(MothershipStreamV1EventType)) @@ -323,9 +319,7 @@ function isContractEnvelope(value: unknown): value is MothershipStreamV1EventEnv } } -// --------------------------------------------------------------------------- // Synthetic file-preview envelope validators -// --------------------------------------------------------------------------- function isSyntheticEnvelopeBase(value: unknown): value is Omit< SyntheticFilePreviewEventEnvelope, @@ -400,9 +394,7 @@ export function isSyntheticFilePreviewEventEnvelope( return isSyntheticEnvelopeBase(value) && isSyntheticFilePreviewPayload(value.payload) } -// --------------------------------------------------------------------------- // Stream event type guards -// --------------------------------------------------------------------------- export function isToolCallStreamEvent(event: SessionStreamEvent): event is ToolCallStreamEvent { return event.type === 'tool' && isRecordLike(event.payload) && event.payload.phase === 'call' @@ -426,9 +418,7 @@ export function isSubagentSpanStreamEvent( return event.type === 'span' && isRecordLike(event.payload) && event.payload.kind === 'subagent' } -// --------------------------------------------------------------------------- // Public contract validators & parsers -// --------------------------------------------------------------------------- export function isContractStreamEventEnvelope( value: unknown diff --git a/apps/sim/lib/core/telemetry.ts b/apps/sim/lib/core/telemetry.ts index a36c7ff28b2..3d1ca1b39a7 100644 --- a/apps/sim/lib/core/telemetry.ts +++ b/apps/sim/lib/core/telemetry.ts @@ -405,9 +405,7 @@ export function trackPlatformEvent( } } -// ============================================================================ // PLATFORM TELEMETRY EVENTS -// ============================================================================ // // Naming Convention: // Event: platform.{resource}.{past_tense_action} @@ -428,7 +426,6 @@ export function trackPlatformEvent( // - Webhook: platform.webhook.* // - Billing: platform.billing.* // - Template: platform.template.* -// ============================================================================ /** * Platform Events - Typed event tracking helpers diff --git a/apps/sim/lib/logs/log-views.ts b/apps/sim/lib/logs/log-views.ts index 1f679ce53b0..db8cfaa4045 100644 --- a/apps/sim/lib/logs/log-views.ts +++ b/apps/sim/lib/logs/log-views.ts @@ -45,9 +45,7 @@ const DEFAULT_MATCH_TIME_BUDGET_MS = 5_000 */ const DEFAULT_MAX_SCANNED_CHARS = 64 * 1024 * 1024 -// --------------------------------------------------------------------------- // Overview (Level 2): block tree with timing + cost, NO input/output. -// --------------------------------------------------------------------------- export interface OverviewSpan { id: string @@ -77,9 +75,7 @@ export function toOverview(spans: TraceSpan[]): OverviewSpan[] { }) } -// --------------------------------------------------------------------------- // Full (Level 3): block tree WITH materialized input/output. -// --------------------------------------------------------------------------- export interface FullSpan extends OverviewSpan { startTime?: string @@ -179,9 +175,7 @@ async function materializeField(value: unknown, ctx: LogViewContext): Promise { + const rows = await db + .select() + .from(pinnedItem) + .where( + and( + eq(pinnedItem.userId, userId), + eq(pinnedItem.workspaceId, workspaceId), + /** + * A `workspace` pin stores `workspaceId === resourceId`, so it would otherwise + * appear in this workspace's unscoped listing as a resource *inside* itself. + * It is read from the workspace-list payload instead, so it is excluded here + * rather than left for a future unscoped caller to mistake for a real resource. + */ + resourceType + ? eq(pinnedItem.resourceType, resourceType) + : ne(pinnedItem.resourceType, 'workspace') + ) + ) + + const activeRows = await filterToActiveResources(rows, workspaceId) + + return activeRows.map(toPinnedItemApi).filter((item): item is PinnedItemApi => item !== null) +} diff --git a/apps/sim/lib/pptx-renderer/core/viewer.ts b/apps/sim/lib/pptx-renderer/core/viewer.ts index 3434a8ba225..bd24037a059 100644 --- a/apps/sim/lib/pptx-renderer/core/viewer.ts +++ b/apps/sim/lib/pptx-renderer/core/viewer.ts @@ -129,9 +129,7 @@ export class PptxViewer extends EventTarget { } } - // ----------------------------------------------------------------------- // Event dispatch helpers - // ----------------------------------------------------------------------- private emitRenderStart(): void { this._isRendering = true @@ -163,9 +161,7 @@ export class PptxViewer extends EventTarget { this.dispatchEvent(new CustomEvent('nodeerror', { detail: { nodeId, error } })) } - // ----------------------------------------------------------------------- // Public: load / render modes - // ----------------------------------------------------------------------- /** * Load a parsed presentation model. Does NOT render — call `renderList()` or @@ -202,9 +198,7 @@ export class PptxViewer extends EventTarget { await this.queueRender() } - // ----------------------------------------------------------------------- // Instance open - // ----------------------------------------------------------------------- async open( input: PreviewInput, @@ -247,9 +241,7 @@ export class PptxViewer extends EventTarget { checkAborted() } - // ----------------------------------------------------------------------- // Static factory - // ----------------------------------------------------------------------- static async open( input: PreviewInput, @@ -269,9 +261,7 @@ export class PptxViewer extends EventTarget { return viewer } - // ----------------------------------------------------------------------- // Navigation - // ----------------------------------------------------------------------- async goToSlide(index: number, scrollOptions?: ScrollIntoViewOptions): Promise { if (!this.presentation) return @@ -318,9 +308,7 @@ export class PptxViewer extends EventTarget { await this.queueRender() } - // ----------------------------------------------------------------------- // Getters - // ----------------------------------------------------------------------- get presentationData(): PresentationData | null { return this.presentation @@ -354,9 +342,7 @@ export class PptxViewer extends EventTarget { return this._fitMode } - // ----------------------------------------------------------------------- // Typed event helpers - // ----------------------------------------------------------------------- on( type: K, @@ -382,9 +368,7 @@ export class PptxViewer extends EventTarget { return [...this.mountedSlides].sort((a, b) => a - b) } - // ----------------------------------------------------------------------- // External slide rendering - // ----------------------------------------------------------------------- /** * Render a single slide into an external container element. @@ -429,9 +413,7 @@ export class PptxViewer extends EventTarget { // No-op in base class } - // ----------------------------------------------------------------------- // Cleanup - // ----------------------------------------------------------------------- destroy(): void { this.teardownAdaptiveResize() @@ -459,9 +441,7 @@ export class PptxViewer extends EventTarget { this.destroy() } - // ----------------------------------------------------------------------- // Internal: rendering pipeline - // ----------------------------------------------------------------------- private normalizeZoomPercent(percent: number): number { if (!Number.isFinite(percent)) return 100 @@ -960,9 +940,7 @@ export class PptxViewer extends EventTarget { } } -// ----------------------------------------------------------------------- // Standalone helper (shared with Renderer.ts) -// ----------------------------------------------------------------------- async function normalizePreviewInput(input: PreviewInput): Promise { if (input instanceof ArrayBuffer) return input diff --git a/apps/sim/lib/pptx-renderer/model/presentation.ts b/apps/sim/lib/pptx-renderer/model/presentation.ts index bdb3564d1d4..2c7dcd8e150 100644 --- a/apps/sim/lib/pptx-renderer/model/presentation.ts +++ b/apps/sim/lib/pptx-renderer/model/presentation.ts @@ -268,9 +268,7 @@ export function buildPresentation(files: PptxFiles): PresentationData { return result } -// --------------------------------------------------------------------------- // Placeholder Position Inheritance -// --------------------------------------------------------------------------- /** * Extract placeholder info (type, idx) from a raw placeholder XML node diff --git a/apps/sim/lib/pptx-renderer/renderer/chart-renderer.ts b/apps/sim/lib/pptx-renderer/renderer/chart-renderer.ts index db7cd8e03b6..a3a3abf3003 100644 --- a/apps/sim/lib/pptx-renderer/renderer/chart-renderer.ts +++ b/apps/sim/lib/pptx-renderer/renderer/chart-renderer.ts @@ -12,9 +12,7 @@ import type { SafeXmlNode } from '../parser/xml-parser' import type { RenderContext } from './render-context' import { resolveColor } from './style-resolver' -// --------------------------------------------------------------------------- // Types -// --------------------------------------------------------------------------- interface SeriesData { name: string @@ -73,9 +71,7 @@ type OoxmlChartType = | 'stockChart' | 'surface3DChart' -// --------------------------------------------------------------------------- // Chart Type Mapping -// --------------------------------------------------------------------------- const CHART_TYPE_ELEMENTS: OoxmlChartType[] = [ 'barChart', @@ -94,9 +90,7 @@ const CHART_TYPE_ELEMENTS: OoxmlChartType[] = [ 'surface3DChart', ] -// --------------------------------------------------------------------------- // Data Extraction Helpers -// --------------------------------------------------------------------------- /** * Extract text values from a strRef or strCache structure. @@ -643,9 +637,7 @@ function parseSeries(chartTypeNode: SafeXmlNode, ctx: RenderContext): SeriesData return seriesArr } -// --------------------------------------------------------------------------- // Chart Title -// --------------------------------------------------------------------------- /** * Extract chart title from chartSpace > chart > title. @@ -769,9 +761,7 @@ function getChartThemeFontFamily(ctx: RenderContext): string | undefined { ) } -// --------------------------------------------------------------------------- // Legend -// --------------------------------------------------------------------------- /** Parsed legend info including overlay flag. */ interface LegendInfo { @@ -1106,9 +1096,7 @@ function hasManualGrid( ) } -// --------------------------------------------------------------------------- // Axis Parsing -// --------------------------------------------------------------------------- const DEFAULT_AXIS_INFO: AxisInfo = { deleted: false, @@ -1296,9 +1284,7 @@ function applyAxisInfo( } } -// --------------------------------------------------------------------------- // ECharts Option Builders -// --------------------------------------------------------------------------- /** * Convert OOXML data label position to ECharts bar label position. @@ -2001,9 +1987,7 @@ function buildScatterChartOption( } } -// --------------------------------------------------------------------------- // Bubble Chart -// --------------------------------------------------------------------------- function buildBubbleChartOption( chartTypeNode: SafeXmlNode, @@ -2103,9 +2087,7 @@ function buildBubbleChartOption( } } -// --------------------------------------------------------------------------- // Stock Chart (Candlestick) -// --------------------------------------------------------------------------- function buildStockChartOption( _chartTypeNode: SafeXmlNode, @@ -2310,9 +2292,7 @@ function buildStockChartOption( } } -// --------------------------------------------------------------------------- // Data Table (c:dTable) -// --------------------------------------------------------------------------- /** Parsed c:dTable info for building the chart data table. */ interface DataTableInfo { @@ -2404,9 +2384,7 @@ function buildDataTableElement(info: DataTableInfo, seriesColors?: string[]): HT return table } -// --------------------------------------------------------------------------- // Main Chart XML Parser -// --------------------------------------------------------------------------- /** * Extract background colors from chartSpace and plotArea. @@ -2556,9 +2534,7 @@ function buildChartPalette(chartXml: SafeXmlNode, ctx: RenderContext): string[] return accents } -// --------------------------------------------------------------------------- // Chart-Space Default Font Size + Legend Grid Adjustment -// --------------------------------------------------------------------------- /** * Apply chart-space default font size to all text elements in the ECharts option @@ -3295,9 +3271,7 @@ export function parseChartXml(chartXml: SafeXmlNode, ctx: RenderContext): ParseC } } -// --------------------------------------------------------------------------- // Public Render Function -// --------------------------------------------------------------------------- /** * Render a chart node into an HTML element with an ECharts instance. diff --git a/apps/sim/lib/pptx-renderer/renderer/group-renderer.ts b/apps/sim/lib/pptx-renderer/renderer/group-renderer.ts index c9352ee7aab..ceae50be271 100644 --- a/apps/sim/lib/pptx-renderer/renderer/group-renderer.ts +++ b/apps/sim/lib/pptx-renderer/renderer/group-renderer.ts @@ -7,9 +7,7 @@ import type { GroupNodeData } from '../model/nodes/group-node' import type { ShapeNodeData } from '../model/nodes/shape-node' import type { RenderContext } from './render-context' -// --------------------------------------------------------------------------- // Group Rendering -// --------------------------------------------------------------------------- /** * Render a group node into an absolutely-positioned HTML element. @@ -171,9 +169,7 @@ export function renderGroup( return wrapper } -// --------------------------------------------------------------------------- // Child Node Parsing -// --------------------------------------------------------------------------- import { parseChartNode } from '../model/nodes/chart-node' import { parseGroupNode } from '../model/nodes/group-node' diff --git a/apps/sim/lib/pptx-renderer/renderer/image-renderer.ts b/apps/sim/lib/pptx-renderer/renderer/image-renderer.ts index 49dbb66f6ac..b77ea8d4aae 100644 --- a/apps/sim/lib/pptx-renderer/renderer/image-renderer.ts +++ b/apps/sim/lib/pptx-renderer/renderer/image-renderer.ts @@ -26,9 +26,7 @@ function isEmfFormat(path: string): boolean { return ext === 'emf' } -// --------------------------------------------------------------------------- // Image Rendering -// --------------------------------------------------------------------------- /** * Render a picture node into an absolutely-positioned HTML element. @@ -373,9 +371,7 @@ function renderUnsupportedPlaceholder(wrapper: HTMLElement, path: string): void wrapper.appendChild(placeholder) } -// --------------------------------------------------------------------------- // EMF Rendering -// --------------------------------------------------------------------------- /** * Render EMF content by extracting embedded PDF or bitmap data. @@ -480,9 +476,7 @@ function createFillImage(url: string): HTMLImageElement { return img } -// --------------------------------------------------------------------------- // Duotone Effect -// --------------------------------------------------------------------------- import type { SafeXmlNode } from '../parser/xml-parser' @@ -548,9 +542,7 @@ function applyDuotoneFilter( } } -// --------------------------------------------------------------------------- // Luminance Effect -// --------------------------------------------------------------------------- /** * Apply a luminance (brightness/contrast) effect to an image. @@ -606,9 +598,7 @@ function applyLumEffect(lum: SafeXmlNode, img: HTMLImageElement): void { } } -// --------------------------------------------------------------------------- // BiLevel Effect -// --------------------------------------------------------------------------- /** * Apply a bi-level (threshold) effect to an image. diff --git a/apps/sim/lib/pptx-renderer/renderer/predefined-table-styles.ts b/apps/sim/lib/pptx-renderer/renderer/predefined-table-styles.ts index d02eeef85eb..818b9088549 100644 --- a/apps/sim/lib/pptx-renderer/renderer/predefined-table-styles.ts +++ b/apps/sim/lib/pptx-renderer/renderer/predefined-table-styles.ts @@ -12,9 +12,7 @@ import { parseXml, type SafeXmlNode } from '../parser/xml-parser' -// --------------------------------------------------------------------------- // UUID → (styleName, accent) map — 74 entries across 11 style groups -// --------------------------------------------------------------------------- const styleIdMap = new Map([ // Themed-Style-1 @@ -114,9 +112,7 @@ const styleIdMap = new Map([ ['{46F890A9-2807-4EBB-B81D-B2AA78EC7F39}', ['Dark-Style-2', 'accent5']], ]) -// --------------------------------------------------------------------------- // XML helpers — reduce boilerplate in style generators -// --------------------------------------------------------------------------- const NS = 'xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main"' @@ -166,9 +162,7 @@ function stylePart( return parts.join('') } -// --------------------------------------------------------------------------- // Style group XML generators -// --------------------------------------------------------------------------- function themedStyle1(accent: string, styleId: string): string { const hasAccent = accent !== '' @@ -740,17 +734,13 @@ function darkStyle2(accent: string, styleId: string): string { return wrapTblStyle(styleId, 'Dark-Style-2', parts.join('')) } -// --------------------------------------------------------------------------- // XML wrapper -// --------------------------------------------------------------------------- function wrapTblStyle(styleId: string, styleName: string, innerXml: string): string { return `${innerXml}` } -// --------------------------------------------------------------------------- // Style generator dispatch -// --------------------------------------------------------------------------- const styleGenerators: Record string> = { 'Themed-Style-1': themedStyle1, @@ -766,9 +756,7 @@ const styleGenerators: Record strin 'Dark-Style-2': darkStyle2, } -// --------------------------------------------------------------------------- // Module-level cache & public API -// --------------------------------------------------------------------------- const cache = new Map() diff --git a/apps/sim/lib/pptx-renderer/renderer/shape-renderer.ts b/apps/sim/lib/pptx-renderer/renderer/shape-renderer.ts index db73643b419..4219971011f 100644 --- a/apps/sim/lib/pptx-renderer/renderer/shape-renderer.ts +++ b/apps/sim/lib/pptx-renderer/renderer/shape-renderer.ts @@ -38,9 +38,7 @@ import { } from './style-resolver' import { renderTextBody } from './text-renderer' -// --------------------------------------------------------------------------- // Shape blipFill (image fill) — resolve to blob URL for reuse (e.g. SVG/PNG in process diagrams) -// --------------------------------------------------------------------------- /** Resolve shape blipFill to a blob URL so we can render it (e.g. slide 23 process graphic). */ function resolveShapeBlipUrl(blipFill: SafeXmlNode, ctx: RenderContext): string | null { @@ -55,9 +53,7 @@ function resolveShapeBlipUrl(blipFill: SafeXmlNode, ctx: RenderContext): string return getOrCreateBlobUrl(mediaPath, data, ctx.mediaUrlCache) } -// --------------------------------------------------------------------------- // Line End Marker (Arrowhead) Helpers -// --------------------------------------------------------------------------- let markerIdCounter = 0 let gradientIdCounter = 0 @@ -278,9 +274,7 @@ function getLineEndsFromLn(ln: SafeXmlNode): { headEnd?: LineEndInfo; tailEnd?: return out } -// --------------------------------------------------------------------------- // Shape Rendering -// --------------------------------------------------------------------------- /** * Render a shape node into an absolutely-positioned HTML element with SVG geometry. diff --git a/apps/sim/lib/pptx-renderer/renderer/slide-renderer.ts b/apps/sim/lib/pptx-renderer/renderer/slide-renderer.ts index 66aa0a3f21b..ba3e021e584 100644 --- a/apps/sim/lib/pptx-renderer/renderer/slide-renderer.ts +++ b/apps/sim/lib/pptx-renderer/renderer/slide-renderer.ts @@ -20,9 +20,7 @@ import { createRenderContext, type RenderContext } from './render-context' import { renderShape } from './shape-renderer' import { renderTable } from './table-renderer' -// --------------------------------------------------------------------------- // Types -// --------------------------------------------------------------------------- export interface SlideRendererOptions { /** Called when a single node fails to render. */ @@ -53,9 +51,7 @@ export interface SlideHandle { [Symbol.dispose](): void } -// --------------------------------------------------------------------------- // Node Dispatch -// --------------------------------------------------------------------------- /** * Dispatch a typed node to its appropriate renderer. @@ -86,9 +82,7 @@ function renderNode(node: BaseNodeData, ctx: RenderContext): HTMLElement { } } -// --------------------------------------------------------------------------- // Error Placeholder -// --------------------------------------------------------------------------- /** * Create a visual error placeholder at the node's position. @@ -116,9 +110,7 @@ function createErrorPlaceholder(node: BaseNodeData): HTMLElement { return el } -// --------------------------------------------------------------------------- // Master/Layout Shape Parsing -// --------------------------------------------------------------------------- /** * Check whether a shape node is a placeholder (has p:ph in nvPr). @@ -183,9 +175,7 @@ function parseTemplateShapes(spTree: SafeXmlNode, _slideNodes: BaseNodeData[]): return nodes } -// --------------------------------------------------------------------------- // Main Slide Render Function -// --------------------------------------------------------------------------- /** * Render a complete slide into an HTML element. diff --git a/apps/sim/lib/pptx-renderer/renderer/style-resolver.ts b/apps/sim/lib/pptx-renderer/renderer/style-resolver.ts index a9eb8295655..11e78995b98 100644 --- a/apps/sim/lib/pptx-renderer/renderer/style-resolver.ts +++ b/apps/sim/lib/pptx-renderer/renderer/style-resolver.ts @@ -9,9 +9,7 @@ import type { ColorModifier } from '../utils/color' import { applyColorModifiers, hslToRgb, presetColorToHex, rgbToHex } from '../utils/color' import type { RenderContext } from './render-context' -// --------------------------------------------------------------------------- // Color Resolution -// --------------------------------------------------------------------------- /** * Build a cache key for a color node based on its tag, value, and modifiers. @@ -205,9 +203,7 @@ function resolveColorWithPlaceholder( return resolveColorUncached(colorNode, ctx, placeholderColorNode) } -// --------------------------------------------------------------------------- // Fill Resolution -// --------------------------------------------------------------------------- /** * Resolve a fill from shape properties (spPr) into a CSS background value. @@ -263,9 +259,7 @@ export function resolveFill(spPr: SafeXmlNode, ctx: RenderContext): string { return '' } -// --------------------------------------------------------------------------- // Pattern Fill Resolution -// --------------------------------------------------------------------------- /** * Resolve `` into a CSS background value using repeating gradients. @@ -500,9 +494,7 @@ function resolveGradient( return `linear-gradient(180deg, ${stopsStr})` } -// --------------------------------------------------------------------------- // Line Style Resolution -// --------------------------------------------------------------------------- /** * Resolve a line (outline) node into CSS-compatible properties. @@ -623,9 +615,7 @@ function ooxmlDashToCss(val: string): string { } } -// --------------------------------------------------------------------------- // Gradient Fill Resolution (structured data for SVG use) -// --------------------------------------------------------------------------- export interface GradientFillData { type: 'linear' | 'radial' @@ -753,9 +743,7 @@ export function resolveThemeFillReference( return { fillCss: resolveColorToCss(fillRef, ctx), gradientFillData: null } } -// --------------------------------------------------------------------------- // Gradient Stroke Resolution -// --------------------------------------------------------------------------- export interface GradientStrokeData { stops: Array<{ position: number; color: string }> diff --git a/apps/sim/lib/pptx-renderer/renderer/table-renderer.ts b/apps/sim/lib/pptx-renderer/renderer/table-renderer.ts index f457d03d713..de514e85a98 100644 --- a/apps/sim/lib/pptx-renderer/renderer/table-renderer.ts +++ b/apps/sim/lib/pptx-renderer/renderer/table-renderer.ts @@ -18,9 +18,7 @@ import type { RenderContext } from './render-context' import { resolveColor, resolveLineStyle } from './style-resolver' import { renderTextBody } from './text-renderer' -// --------------------------------------------------------------------------- // Table Style Lookup -// --------------------------------------------------------------------------- /** * Find a table style node by its ID from presentation.tableStyles. @@ -387,9 +385,7 @@ function applyTableBackground(table: HTMLElement, tblStyle: SafeXmlNode, ctx: Re } } -// --------------------------------------------------------------------------- // Table Rendering -// --------------------------------------------------------------------------- /** * Render a table node into an absolutely-positioned HTML element. @@ -528,9 +524,7 @@ export function renderTable(node: TableNodeData, ctx: RenderContext): HTMLElemen return wrapper } -// --------------------------------------------------------------------------- // Cell Property Application -// --------------------------------------------------------------------------- /** * Apply table cell properties (tcPr) to a element. diff --git a/apps/sim/lib/pptx-renderer/renderer/text-renderer.ts b/apps/sim/lib/pptx-renderer/renderer/text-renderer.ts index 42679dcf9e3..28c73270d11 100644 --- a/apps/sim/lib/pptx-renderer/renderer/text-renderer.ts +++ b/apps/sim/lib/pptx-renderer/renderer/text-renderer.ts @@ -12,9 +12,7 @@ import { SafeXmlNode } from '../parser/xml-parser' import type { RenderContext } from './render-context' import { resolveColor, resolveColorToCss } from './style-resolver' -// --------------------------------------------------------------------------- // Style Inheritance Helpers -// --------------------------------------------------------------------------- /** * Find paragraph properties at a specific indent level from a list style node. @@ -227,9 +225,7 @@ function mergeParagraphProps(target: MergedParagraphStyle, pPr: SafeXmlNode): vo } } -// --------------------------------------------------------------------------- // Run Style Resolution -// --------------------------------------------------------------------------- interface MergedRunStyle { fontSize?: number @@ -420,9 +416,7 @@ function resolveGradientForText(gradFill: SafeXmlNode, ctx: RenderContext): stri return `linear-gradient(180deg, ${stopsStr})` } -// --------------------------------------------------------------------------- // Bullet Generation -// --------------------------------------------------------------------------- function generateAutoNumber(type: string, index: number): string { const num = index + 1 @@ -466,9 +460,7 @@ function toRoman(num: number): string { return result } -// --------------------------------------------------------------------------- // Main Render Function -// --------------------------------------------------------------------------- /** * Render a text body into the provided container element. diff --git a/apps/sim/lib/pptx-renderer/shapes/presets.ts b/apps/sim/lib/pptx-renderer/shapes/presets.ts index 572b0c4f9e8..00f969a87ce 100644 --- a/apps/sim/lib/pptx-renderer/shapes/presets.ts +++ b/apps/sim/lib/pptx-renderer/shapes/presets.ts @@ -146,14 +146,10 @@ function mirrorAbsolutePathVertically(path: string, height: number): string { return out.join(' ') } -// --------------------------------------------------------------------------- // Preset shape registry -// --------------------------------------------------------------------------- export const presetShapes: Map = new Map() -// ===== Basic Shapes ===== - presetShapes.set('rect', (w, h) => `M0,0 L${w},0 L${w},${h} L0,${h} Z`) presetShapes.set('roundRect', (w, h, adjustments) => { @@ -451,8 +447,6 @@ presetShapes.set('diagStripe', (w, h, adjustments) => { return [`M0,${y2}`, `L${x2},0`, `L${w},0`, `L0,${h}`, 'Z'].join(' ') }) -// ===== Star Shapes ===== - presetShapes.set('star4', (w, h, adjustments) => { // OOXML default adj=12500 → innerRatio = 12500/50000 = 0.25 const a = adj(adjustments, 'adj', 12500) * 2 @@ -597,8 +591,6 @@ presetShapes.set('star32', (w, h, adjustments) => { return starShape(w, h, 32, Math.min(Math.max(a, 0), 1)) }) -// ===== Lines & Connectors ===== - // OOXML line: diagonal (0,0→w,h) when both extents are non-zero. // Keep explicit horizontal/vertical handling for zero-extent cases so 1px SVGs remain visible. presetShapes.set('line', (w, h) => { @@ -713,8 +705,6 @@ presetShapes.set('bentConnector5', (w, h, adjustments) => { return `M0,0 L${x1},0 L${x1},${y1} L${x2},${y1} L${x2},${h} L${w},${h}` }) -// ===== Arrow Shapes ===== - presetShapes.set('rightArrow', (w, h, adjustments) => { const a1 = adj(adjustments, 'adj1', 50000) // shaft width ratio const a2 = adj(adjustments, 'adj2', 50000) // head length ratio @@ -1262,8 +1252,6 @@ presetShapes.set('stripedRightArrow', (w, h, adjustments) => { ].join(' ') }) -// ===== Bent / Curved / Special Arrows ===== - presetShapes.set('bentArrow', (w, h, adjustments) => { // OOXML bentArrow: L-shaped arrow with rounded bend, arrowhead pointing right. // Uses 4 adjustments per ECMA-376 spec. @@ -2293,8 +2281,6 @@ presetShapes.set('swooshArrow', (w, h, adjustments) => { ].join(' ') }) -// ===== Flowchart Shapes ===== - presetShapes.set('flowChartProcess', (w, h) => `M0,0 L${w},0 L${w},${h} L0,${h} Z`) presetShapes.set('flowChartDecision', (w, h) => { @@ -2677,8 +2663,6 @@ presetShapes.set('flowChartMultidocument', (w, h) => { ].join(' ') }) -// ===== Callout Shapes ===== - presetShapes.set('wedgeRectCallout', (w, h, adjustments) => { // OOXML spec: adaptive callout pointer on the edge closest to the tip const hc = w / 2 @@ -2842,8 +2826,6 @@ presetShapes.set('borderCallout1', (w, h, adjustments) => { return `M0,0 L${w},0 L${w},${h} L0,${h} Z M${x1},${y1} L${x2},${y2}` }) -// ===== Block / 3D Shapes ===== - presetShapes.set('cube', (w, h, adjustments) => { const a = adj(adjustments, 'adj', 25000) const depth = Math.min(w, h) * a @@ -2963,8 +2945,6 @@ presetShapes.set('cloud', (w, h) => { return parts.join(' ') }) -// ===== Frame, Donut, Misc ===== - presetShapes.set('frame', (w, h, adjustments) => { const a = adj(adjustments, 'adj1', 12500) const t = Math.min(w, h) * a @@ -3109,8 +3089,6 @@ presetShapes.set('blockArc', (w, h, adjustments) => { ].join(' ') }) -// ===== Gear Shapes ===== - presetShapes.set('gear6', (w, h, adjustments) => { const a1 = adjustments?.get('adj1') ?? 15000 const a2 = adjustments?.get('adj2') ?? 3526 @@ -3218,8 +3196,6 @@ function gearShape(w: number, h: number, teeth: number, adj1Raw: number, adj2Raw return parts.join(' ') } -// ===== Misc Shapes ===== - presetShapes.set('mathPlus', (w, h, adjustments) => { // OOXML: adj1=23520 (max 73490). dx1 = w*73490/200000, dx2 = ss*a/200000 const ss = Math.min(w, h) @@ -3916,7 +3892,6 @@ presetShapes.set('rightBrace', (w, h, adjustments) => { ) }) -// ===== Action Buttons ===== // Action buttons are multi-path shapes: background rect + icon with darken fill + icon outline + rect outline. // OOXML spec uses ss*3/8 as the icon half-size (dx2), with the icon centred at (hc, vc). // Shapes with multiPathPresets entries below get proper 3D treatment. Remaining shapes @@ -3930,9 +3905,7 @@ presetShapes.set('actionButtonBlank', (w, h) => `M0,0 L${w},0 L${w},${h} L0,${h} // Multi-path action button presets are registered after the multiPathPresets Map // declaration (see below in the multiPathPresets section). -// --------------------------------------------------------------------------- // Action button icon paths (rendered as a second with contrasting fill) -// --------------------------------------------------------------------------- const actionButtonIcons = new Map string>() // actionButtonHome icon removed — uses multiPathPresets entry below @@ -4060,8 +4033,6 @@ export function getActionButtonIconPath( return generator?.(w, h) } -// ===== Aliases and common alternative names ===== - // Some shapes are known by multiple names in different OOXML versions // flowChartOfflineStorage: registered as multiPathPreset (see below) @@ -4415,15 +4386,11 @@ presetShapes.set('funnel', (w, h) => { return `${body} ${inset}` }) -// ===== Fallback ===== - /** * Get the SVG path for a preset shape, falling back to a simple rectangle * if the shape type is not implemented. */ -// --------------------------------------------------------------------------- // Preset shape overlays — additional paths for 3D-like shapes (lighter top face, etc.) -// --------------------------------------------------------------------------- interface PresetOverlay { /** SVG path d-attribute for the overlay */ @@ -4468,10 +4435,8 @@ export function getPresetOverlays( return gen ? gen(w, h, adjustments) : [] } -// --------------------------------------------------------------------------- // Multi-path preset shapes — complex shapes with multiple SVG paths // Each path has its own fill modifier and stroke behavior, matching OOXML spec. -// --------------------------------------------------------------------------- /** A single sub-path within a multi-path preset shape. */ export interface PresetSubPath { @@ -4507,7 +4472,6 @@ type MultiPathPresetGenerator = ( const multiPathPresets: Map = new Map() -// ===== Action Button multi-path presets (OOXML spec-accurate) ===== // Common helper: OOXML action button guide values function _abGuides(w: number, h: number) { const ss = Math.min(w, h) diff --git a/apps/sim/lib/pptx-renderer/utils/color.ts b/apps/sim/lib/pptx-renderer/utils/color.ts index cb7be995976..a71a646f3a2 100644 --- a/apps/sim/lib/pptx-renderer/utils/color.ts +++ b/apps/sim/lib/pptx-renderer/utils/color.ts @@ -1,16 +1,12 @@ -// ============================================================================ // OOXML Color Utilities // Full color manipulation for PowerPoint XML color processing -// ============================================================================ export { hexToRgb, hslToRgb, rgbToHex, rgbToHsl, toCssColor } from '@/lib/colors' import { hexToRgb, hslToRgb, rgbToHex, rgbToHsl } from '@/lib/colors' -// --------------------------------------------------------------------------- // sRGB ↔ Linear RGB conversion (IEC 61966-2-1) // PowerPoint applies tint/shade in linear (scene-referred) space. -// --------------------------------------------------------------------------- function srgbToLinear(c: number): number { const s = c / 255 @@ -22,9 +18,7 @@ function linearToSrgb(c: number): number { return Math.max(0, Math.min(255, Math.round(s * 255))) } -// --------------------------------------------------------------------------- // OOXML Color Modifiers -// --------------------------------------------------------------------------- /** * Apply tint modifier (mix toward white in linear RGB space). @@ -145,9 +139,7 @@ export function applyAlpha(alpha: number): number { return Math.max(0, Math.min(1, alpha / 100000)) } -// --------------------------------------------------------------------------- // Composite Modifier Application -// --------------------------------------------------------------------------- export interface ColorModifier { name: string @@ -217,9 +209,7 @@ export function applyColorModifiers( return { color, alpha } } -// --------------------------------------------------------------------------- // OOXML Preset Color Table -// --------------------------------------------------------------------------- const PRESET_COLORS: Record = { // Basic colors diff --git a/apps/sim/lib/pptx-renderer/utils/pdf-renderer.ts b/apps/sim/lib/pptx-renderer/utils/pdf-renderer.ts index 7c3fee93463..c51bfcaeab7 100644 --- a/apps/sim/lib/pptx-renderer/utils/pdf-renderer.ts +++ b/apps/sim/lib/pptx-renderer/utils/pdf-renderer.ts @@ -15,9 +15,7 @@ * fallback, no global state pollution. */ -// --------------------------------------------------------------------------- // Resolved pdfjs URL — computed once from main thread's module resolution -// --------------------------------------------------------------------------- let _pdfjsUrl: string | null = null @@ -32,9 +30,7 @@ function getPdfjsUrl(): string | null { return _pdfjsUrl || null } -// --------------------------------------------------------------------------- // Worker-based renderer (fully isolated from main thread pdfjs) -// --------------------------------------------------------------------------- /** * Inline source for the PDF render worker. @@ -163,9 +159,7 @@ function renderInWorker( }) } -// --------------------------------------------------------------------------- // Public API -// --------------------------------------------------------------------------- /** * Render page 1 of a PDF to a blob URL image. diff --git a/apps/sim/lib/table/__tests__/find-row-matches.test.ts b/apps/sim/lib/table/__tests__/find-row-matches.test.ts index 5ff29ce21fd..2d91742e2d7 100644 --- a/apps/sim/lib/table/__tests__/find-row-matches.test.ts +++ b/apps/sim/lib/table/__tests__/find-row-matches.test.ts @@ -18,6 +18,9 @@ vi.mock('@/lib/table/sql', () => ({ })) vi.mock('@/lib/table/trigger', () => ({ fireTableTrigger: vi.fn() })) +vi.mock('@/lib/table/workflow-group-deps', () => ({ + stripGroupDeps: vi.fn(), +})) vi.mock('@/lib/table/workflow-columns', () => ({ assertValidSchema: vi.fn(), scheduleRunsForRows: vi.fn(), diff --git a/apps/sim/lib/table/__tests__/service-filter-threading.test.ts b/apps/sim/lib/table/__tests__/service-filter-threading.test.ts index f49fb3763e1..17cee4f63a1 100644 --- a/apps/sim/lib/table/__tests__/service-filter-threading.test.ts +++ b/apps/sim/lib/table/__tests__/service-filter-threading.test.ts @@ -26,6 +26,9 @@ vi.mock('@/lib/table/trigger', () => ({ fireTableTrigger: vi.fn(), })) +vi.mock('@/lib/table/workflow-group-deps', () => ({ + stripGroupDeps: vi.fn(), +})) vi.mock('@/lib/table/workflow-columns', () => ({ assertValidSchema: vi.fn(), scheduleRunsForRows: vi.fn(), diff --git a/apps/sim/lib/table/columns/service.ts b/apps/sim/lib/table/columns/service.ts index f7cc0b6803d..b0a67b8284d 100644 --- a/apps/sim/lib/table/columns/service.ts +++ b/apps/sim/lib/table/columns/service.ts @@ -57,7 +57,8 @@ import type { UpdateColumnTypeData, } from '@/lib/table/types' import { validateColumnDefinition } from '@/lib/table/validation' -import { assertValidSchema, stripGroupDeps } from '@/lib/table/workflow-columns' +import { assertValidSchema } from '@/lib/table/workflow-columns' +import { stripGroupDeps } from '@/lib/table/workflow-group-deps' const logger = createLogger('TableColumnService') const COLUMN_RETYPE_SCAN_MAX_BYTES = 32 * 1024 * 1024 diff --git a/apps/sim/lib/table/import-runner.test.ts b/apps/sim/lib/table/import-runner.test.ts index 168bfba8d82..fb5b2794df5 100644 --- a/apps/sim/lib/table/import-runner.test.ts +++ b/apps/sim/lib/table/import-runner.test.ts @@ -55,7 +55,7 @@ vi.mock('@/lib/uploads/core/storage-service', () => ({ downloadFileStream: mockDownloadFileStream, headObject: mockHeadObject, })) -vi.mock('@/app/api/table/utils', () => ({ +vi.mock('@/lib/table/wire', () => ({ normalizeColumn: (col: unknown) => col, })) diff --git a/apps/sim/lib/table/import-runner.ts b/apps/sim/lib/table/import-runner.ts index 9afb692daf3..2ddf3ffe220 100644 --- a/apps/sim/lib/table/import-runner.ts +++ b/apps/sim/lib/table/import-runner.ts @@ -38,8 +38,8 @@ import { assertRowDelete, assertRowInsert, assertSchemaMutable } from '@/lib/tab import type { DbTransaction } from '@/lib/table/planner' import { nextImportStartOrderKey, nextImportStartPosition } from '@/lib/table/rows/ordering' import { getTableById } from '@/lib/table/service' +import { normalizeColumn } from '@/lib/table/wire' import { deleteFile, downloadFileStream, headObject } from '@/lib/uploads/core/storage-service' -import { normalizeColumn } from '@/app/api/table/utils' const logger = createLogger('TableImportRunner') diff --git a/apps/sim/lib/table/jobs/service.ts b/apps/sim/lib/table/jobs/service.ts index 3dbc48eadcd..24f1ec5cf39 100644 --- a/apps/sim/lib/table/jobs/service.ts +++ b/apps/sim/lib/table/jobs/service.ts @@ -15,7 +15,7 @@ import { db } from '@sim/db' import { tableJobs, userTableDefinitions, userTableRows } from '@sim/db/schema' import { and, asc, desc, eq, gt, inArray, ne, or, sql } from 'drizzle-orm' import type { DbOrTx } from '@/lib/db/types' -import { pendingDeleteMask } from '@/lib/table/rows/service' +import { pendingDeleteMask } from '@/lib/table/rows/pending-delete-mask' import type { RowData, TableDefinition, diff --git a/apps/sim/lib/table/rows/pending-delete-mask.ts b/apps/sim/lib/table/rows/pending-delete-mask.ts new file mode 100644 index 00000000000..5036657e307 --- /dev/null +++ b/apps/sim/lib/table/rows/pending-delete-mask.ts @@ -0,0 +1,63 @@ +import { db } from '@sim/db' +import { tableJobs, userTableRows } from '@sim/db/schema' +import { createLogger } from '@sim/logger' +import { toError } from '@sim/utils/errors' +import { and, eq, lte, notInArray, type SQL, sql } from 'drizzle-orm' +import { USER_TABLE_ROWS_SQL_NAME } from '@/lib/table/constants' +import { buildFilterClause } from '@/lib/table/sql' +import type { TableDefinition, TableDeleteJobPayload } from '@/lib/table/types' + +const logger = createLogger('TablePendingDeleteMask') + +/** + * Visibility mask for a running delete job: returns a clause keeping only rows the job will NOT + * delete, or `undefined` when no delete job is running. The job's persisted scope + * ({@link TableDeleteJobPayload}) defines the doomed set — `matches(filter) AND created_at <= + * cutoff AND id NOT IN excludeRowIds` — exactly what the worker's `selectRowIdPage` selects, so + * mid-job reads (refresh, other clients, exports) are consistent with the eventual result. The + * mask lifts automatically when the job leaves `running` (done, failed, or canceled). + * + * `(doomed) IS NOT TRUE` rather than `NOT (doomed)`: JSONB predicates evaluate to NULL on missing + * cells, and those rows are NOT selected for deletion (NULL ≠ TRUE) — they must stay visible. + */ +export async function pendingDeleteMask(table: TableDefinition): Promise { + const [job] = await db + .select({ payload: tableJobs.payload }) + .from(tableJobs) + .where( + and( + eq(tableJobs.tableId, table.id), + eq(tableJobs.status, 'running'), + eq(tableJobs.type, 'delete') + ) + ) + .limit(1) + if (!job?.payload) return undefined + const scope = job.payload as TableDeleteJobPayload + + // A bounded delete (explicit limit) deletes only the first `maxRows` matches, so the filter-based + // mask — which hides every match — would over-hide the rows beyond the cap this job never touches. + // Leave those reads unmasked; the bounded delete is eventually consistent like a bounded update. + if (scope.maxRows !== undefined) return undefined + + const doomedParts: SQL[] = [] + if (scope.filter && Object.keys(scope.filter).length > 0) { + try { + const clause = buildFilterClause(scope.filter, USER_TABLE_ROWS_SQL_NAME, table.schema.columns) + if (clause) doomedParts.push(clause) + } catch (error) { + // Schema drifted mid-job (column renamed/deleted). Showing doomed rows briefly beats + // failing every read; the worker resolves the same way on its next page. + logger.warn(`Skipping delete-job mask for table ${table.id}: stale filter`, { + error: toError(error).message, + }) + return undefined + } + } + if (scope.cutoff) doomedParts.push(lte(userTableRows.createdAt, new Date(scope.cutoff))) + if (scope.excludeRowIds && scope.excludeRowIds.length > 0) { + doomedParts.push(notInArray(userTableRows.id, scope.excludeRowIds)) + } + if (doomedParts.length === 0) return undefined + return sql`(${and(...doomedParts)}) IS NOT TRUE` +} diff --git a/apps/sim/lib/table/rows/service.ts b/apps/sim/lib/table/rows/service.ts index 7a65cfd10b3..ab9fbf6ea68 100644 --- a/apps/sim/lib/table/rows/service.ts +++ b/apps/sim/lib/table/rows/service.ts @@ -11,11 +11,10 @@ */ import { db } from '@sim/db' -import { tableJobs, userTableRows } from '@sim/db/schema' +import { userTableRows } from '@sim/db/schema' import { createLogger } from '@sim/logger' -import { toError } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' -import { and, asc, count, eq, inArray, lte, notInArray, type SQL, sql } from 'drizzle-orm' +import { and, asc, count, eq, inArray, type SQL, sql } from 'drizzle-orm' import { OrchestrationError } from '@/lib/core/orchestration/types' import { assertRowCapacity, @@ -61,6 +60,7 @@ import { selectRowDataPage, selectRowIdPage, } from '@/lib/table/rows/ordering' +import { pendingDeleteMask } from '@/lib/table/rows/pending-delete-mask' import { mutateTableRowsWithSecretProvenance } from '@/lib/table/rows/secret-provenance' import { buildFilterClause, @@ -91,7 +91,6 @@ import type { RowExecutions, Sort, TableDefinition, - TableDeleteJobPayload, TableRow, TableRowsCursor, UpdateRowData, @@ -1056,58 +1055,6 @@ export async function findRowMatches( * @param requestId - Request ID for logging * @returns Query result with rows and pagination info */ -/** - * Visibility mask for a running delete job: returns a clause keeping only rows the job will NOT - * delete, or `undefined` when no delete job is running. The job's persisted scope - * ({@link TableDeleteJobPayload}) defines the doomed set — `matches(filter) AND created_at <= - * cutoff AND id NOT IN excludeRowIds` — exactly what the worker's `selectRowIdPage` selects, so - * mid-job reads (refresh, other clients, exports) are consistent with the eventual result. The - * mask lifts automatically when the job leaves `running` (done, failed, or canceled). - * - * `(doomed) IS NOT TRUE` rather than `NOT (doomed)`: JSONB predicates evaluate to NULL on missing - * cells, and those rows are NOT selected for deletion (NULL ≠ TRUE) — they must stay visible. - */ -export async function pendingDeleteMask(table: TableDefinition): Promise { - const [job] = await db - .select({ payload: tableJobs.payload }) - .from(tableJobs) - .where( - and( - eq(tableJobs.tableId, table.id), - eq(tableJobs.status, 'running'), - eq(tableJobs.type, 'delete') - ) - ) - .limit(1) - if (!job?.payload) return undefined - const scope = job.payload as TableDeleteJobPayload - - // A bounded delete (explicit limit) deletes only the first `maxRows` matches, so the filter-based - // mask — which hides every match — would over-hide the rows beyond the cap this job never touches. - // Leave those reads unmasked; the bounded delete is eventually consistent like a bounded update. - if (scope.maxRows !== undefined) return undefined - - const doomedParts: SQL[] = [] - if (scope.filter && Object.keys(scope.filter).length > 0) { - try { - const clause = buildFilterClause(scope.filter, USER_TABLE_ROWS_SQL_NAME, table.schema.columns) - if (clause) doomedParts.push(clause) - } catch (error) { - // Schema drifted mid-job (column renamed/deleted). Showing doomed rows briefly beats - // failing every read; the worker resolves the same way on its next page. - logger.warn(`Skipping delete-job mask for table ${table.id}: stale filter`, { - error: toError(error).message, - }) - return undefined - } - } - if (scope.cutoff) doomedParts.push(lte(userTableRows.createdAt, new Date(scope.cutoff))) - if (scope.excludeRowIds && scope.excludeRowIds.length > 0) { - doomedParts.push(notInArray(userTableRows.id, scope.excludeRowIds)) - } - if (doomedParts.length === 0) return undefined - return sql`(${and(...doomedParts)}) IS NOT TRUE` -} /** * `COUNT(*)` for a filtered view, kept inside the tenant's rows: measured diff --git a/apps/sim/lib/table/service.ts b/apps/sim/lib/table/service.ts index 7cb7e35c760..7da186caa17 100644 --- a/apps/sim/lib/table/service.ts +++ b/apps/sim/lib/table/service.ts @@ -55,7 +55,7 @@ import { UNLOCKED_TABLE_LOCKS, } from '@/lib/table/types' import { validateTableName, validateTableSchema } from '@/lib/table/validation' -import { stripGroupDeps } from '@/lib/table/workflow-columns' +import { stripGroupDeps } from '@/lib/table/workflow-group-deps' const logger = createLogger('TableService') diff --git a/apps/sim/lib/table/wire.ts b/apps/sim/lib/table/wire.ts new file mode 100644 index 00000000000..514389596a2 --- /dev/null +++ b/apps/sim/lib/table/wire.ts @@ -0,0 +1,69 @@ +import { typeMetadataOf } from '@/lib/table/column-types' +import type { ColumnDefinition, TableDefinition } from '@/lib/table/types' + +/** + * Projects a stored column onto its wire form: optional flags defaulted and + * type-specific metadata forwarded generically. + * + * Deliberately not re-exported from the `@/lib/table` barrel — routes that mock + * the barrel wholesale would otherwise lose this projection and silently emit + * unnormalized columns. + */ +export function normalizeColumn( + col: ColumnDefinition +): ColumnDefinition & { required: boolean; unique: boolean } { + return { + // Preserve the stable column id — it's the row-data storage key, so dropping + // it makes clients fall back to `name` and miss id-keyed cell values. + ...(col.id ? { id: col.id } : {}), + name: col.name, + type: col.type, + required: col.required ?? false, + unique: col.unique ?? false, + ...(col.workflowGroupId ? { workflowGroupId: col.workflowGroupId } : {}), + // Type-specific metadata is forwarded generically: naming keys here meant a + // new type's metadata was stored server-side but silently never returned. + ...typeMetadataOf(col), + } +} + +/** Serializes a stored timestamp to the ISO string a table wire shape carries. */ +export function toWireTimestamp(value: Date | string): string { + return value instanceof Date ? value.toISOString() : String(value) +} + +/** + * Projects a stored table definition onto the exact shape + * `GET /api/table` returns and `useTablesList` caches. + * + * The single source of truth for that shape: the route and the tables page + * prefetch both call it, so a server-hydrated cache entry is indistinguishable + * from one a client fetch produced. Timestamps become ISO strings (the client + * never sees `Date` — the list contract's response schema is a passthrough + * `z.custom`, so it neither coerces nor strips), columns are normalized, job + * fields are defaulted, and `metadata` is withheld as server-only. + */ +export function toTableListItem(table: TableDefinition): TableDefinition { + return { + id: table.id, + name: table.name, + description: table.description, + schema: { + columns: table.schema.columns.map(normalizeColumn), + }, + rowCount: table.rowCount, + maxRows: table.maxRows, + locks: table.locks, + workspaceId: table.workspaceId, + folderId: table.folderId ?? null, + createdBy: table.createdBy, + createdAt: toWireTimestamp(table.createdAt), + updatedAt: toWireTimestamp(table.updatedAt), + archivedAt: table.archivedAt ? toWireTimestamp(table.archivedAt) : null, + jobStatus: table.jobStatus ?? null, + jobId: table.jobId ?? null, + jobType: table.jobType ?? null, + jobError: table.jobError ?? null, + jobRowsProcessed: table.jobRowsProcessed ?? 0, + } +} diff --git a/apps/sim/lib/table/workflow-columns.ts b/apps/sim/lib/table/workflow-columns.ts index ecfceb9be0e..34211a12266 100644 --- a/apps/sim/lib/table/workflow-columns.ts +++ b/apps/sim/lib/table/workflow-columns.ts @@ -1072,34 +1072,6 @@ export async function runWorkflowColumn(opts: { return { dispatchId, shouldSignalRowsChanged: true } } -// ───────────────────────────── Validation ───────────────────────────── - -/** -/** - * Removes the given column names from a group's `dependencies.columns` and from - * its `inputMappings` (any mapping whose source `columnName` was removed). When - * either list ends up empty, drops the field entirely so schema validation - * doesn't see an empty object. Returns the same group reference when nothing - * changed. - */ -export function stripGroupDeps(group: WorkflowGroup, removed: ReadonlySet): WorkflowGroup { - const cols = group.dependencies?.columns ?? [] - const mappings = group.inputMappings ?? [] - const filteredDeps = cols.filter((d) => !removed.has(d)) - const filteredMappings = mappings.filter((m) => !removed.has(m.columnName)) - const depsChanged = filteredDeps.length !== cols.length - const mappingsChanged = filteredMappings.length !== mappings.length - if (!depsChanged && !mappingsChanged) return group - const next: WorkflowGroup = { ...group } - if (depsChanged) { - next.dependencies = filteredDeps.length > 0 ? { columns: filteredDeps } : undefined - } - if (mappingsChanged) { - next.inputMappings = filteredMappings.length > 0 ? filteredMappings : undefined - } - return next -} - /** * Validates schema-level invariants. Run on every `addTableColumn`, * `addWorkflowGroup`, `updateWorkflowGroup`, `renameColumn`, `reorderColumns`, diff --git a/apps/sim/lib/table/workflow-group-deps.ts b/apps/sim/lib/table/workflow-group-deps.ts new file mode 100644 index 00000000000..1200a815146 --- /dev/null +++ b/apps/sim/lib/table/workflow-group-deps.ts @@ -0,0 +1,29 @@ +import type { WorkflowGroup } from '@/lib/table/types' + +/** + * Drops the given column ids from a workflow group's dependencies and input + * mappings, returning the group unchanged when neither referenced them. + * + * A pure projection over the group, deliberately kept in its own leaf module + * rather than alongside the group runtime in `workflow-columns`: that module + * reaches the executor and, through it, the executable tool registry, so any + * server graph importing this helper from there pays ~4,700 modules for a + * function that only reshapes an object. + */ +export function stripGroupDeps(group: WorkflowGroup, removed: ReadonlySet): WorkflowGroup { + const cols = group.dependencies?.columns ?? [] + const mappings = group.inputMappings ?? [] + const filteredDeps = cols.filter((d) => !removed.has(d)) + const filteredMappings = mappings.filter((m) => !removed.has(m.columnName)) + const depsChanged = filteredDeps.length !== cols.length + const mappingsChanged = filteredMappings.length !== mappings.length + if (!depsChanged && !mappingsChanged) return group + const next: WorkflowGroup = { ...group } + if (depsChanged) { + next.dependencies = filteredDeps.length > 0 ? { columns: filteredDeps } : undefined + } + if (mappingsChanged) { + next.inputMappings = filteredMappings.length > 0 ? filteredMappings : undefined + } + return next +} diff --git a/apps/sim/lib/table/workflow-groups/service.ts b/apps/sim/lib/table/workflow-groups/service.ts index 35378a997b9..137372f1e68 100644 --- a/apps/sim/lib/table/workflow-groups/service.ts +++ b/apps/sim/lib/table/workflow-groups/service.ts @@ -37,7 +37,8 @@ import type { WorkflowGroup, WorkflowGroupOutput, } from '@/lib/table/types' -import { assertValidSchema, runWorkflowColumn, stripGroupDeps } from '@/lib/table/workflow-columns' +import { assertValidSchema, runWorkflowColumn } from '@/lib/table/workflow-columns' +import { stripGroupDeps } from '@/lib/table/workflow-group-deps' const logger = createLogger('TableWorkflowGroupsService') /** diff --git a/apps/sim/scripts/export-workflow.ts b/apps/sim/scripts/export-workflow.ts index 289efff5ab6..8cdcb4416d9 100755 --- a/apps/sim/scripts/export-workflow.ts +++ b/apps/sim/scripts/export-workflow.ts @@ -30,7 +30,6 @@ import { workflow } from '../../../packages/db/schema.js' import { loadWorkflowFromNormalizedTables } from '../lib/workflows/persistence/utils.js' import { sanitizeForExport } from '../lib/workflows/sanitization/json-sanitizer.js' -// ---------- CLI argument parsing ---------- const args = process.argv.slice(2) const workflowId = args[0] const outputFile = args[1] // Optional output filename @@ -48,7 +47,6 @@ if (!workflowId) { process.exit(1) } -// ---------- Main export function ---------- async function exportWorkflow(workflowId: string, outputFile?: string): Promise { try { // Fetch workflow metadata @@ -109,7 +107,6 @@ async function exportWorkflow(workflowId: string, outputFile?: string): Promise< } } -// ---------- Execute ---------- exportWorkflow(workflowId, outputFile) .then(() => { process.exit(0) diff --git a/apps/sim/tools/datadog/types.ts b/apps/sim/tools/datadog/types.ts index a356913c9bb..fc6e0f62eec 100644 --- a/apps/sim/tools/datadog/types.ts +++ b/apps/sim/tools/datadog/types.ts @@ -21,9 +21,7 @@ interface DatadogBaseParams extends DatadogWriteOnlyParams { applicationKey: string } -// ======================== // METRICS TYPES -// ======================== export type MetricType = 'gauge' | 'rate' | 'count' | 'distribution' @@ -115,9 +113,7 @@ interface GetMetricMetadataResponse extends ToolResponse { output: GetMetricMetadataOutput } -// ======================== // EVENTS TYPES -// ======================== export type EventAlertType = | 'error' @@ -192,9 +188,7 @@ interface QueryEventsResponse extends ToolResponse { output: QueryEventsOutput } -// ======================== // MONITORS TYPES -// ======================== export type MonitorType = | 'metric alert' @@ -356,9 +350,7 @@ interface UnmuteMonitorResponse extends ToolResponse { output: UnmuteMonitorOutput } -// ======================== // LOGS TYPES -// ======================== interface LogEntry { ddsource?: string @@ -411,9 +403,7 @@ export interface QueryLogsResponse extends ToolResponse { output: QueryLogsOutput } -// ======================== // DOWNTIME TYPES -// ======================== export interface CreateDowntimeParams extends DatadogBaseParams { scope: string // Scope to apply downtime (e.g., "host:myhost" or "*") @@ -480,9 +470,7 @@ export interface CancelDowntimeResponse extends ToolResponse { output: CancelDowntimeOutput } -// ======================== // SLO TYPES -// ======================== export type SloType = 'metric' | 'monitor' | 'time_slice' @@ -560,9 +548,7 @@ interface GetSloHistoryResponse extends ToolResponse { output: GetSloHistoryOutput } -// ======================== // DASHBOARD TYPES -// ======================== export type DashboardLayoutType = 'ordered' | 'free' @@ -639,9 +625,7 @@ interface ListDashboardsResponse extends ToolResponse { output: ListDashboardsOutput } -// ======================== // HOSTS TYPES -// ======================== interface ListHostsParams extends DatadogBaseParams { filter?: string // Filter hosts by name, alias, or tag @@ -691,9 +675,7 @@ interface ListHostsResponse extends ToolResponse { output: ListHostsOutput } -// ======================== // INCIDENTS TYPES -// ======================== export type IncidentSeverity = 'SEV-1' | 'SEV-2' | 'SEV-3' | 'SEV-4' | 'SEV-5' | 'UNKNOWN' export type IncidentState = 'active' | 'stable' | 'resolved' diff --git a/apps/sim/tools/datagma/types.ts b/apps/sim/tools/datagma/types.ts index c018eb3e4e4..33af052c1d8 100644 --- a/apps/sim/tools/datagma/types.ts +++ b/apps/sim/tools/datagma/types.ts @@ -4,12 +4,10 @@ interface DatagmaBaseParams { apiKey: string } -// --------------------------------------------------------------------------- // Find Email (findEmail) // Endpoint: GET https://gateway.datagma.net/api/ingress/v6/findEmail // Auth: apiId query param // Docs: https://datagmaapi.readme.io/reference/find-work-email-address -// --------------------------------------------------------------------------- export interface DatagmaFindEmailParams extends DatagmaBaseParams { fullName: string @@ -30,13 +28,11 @@ export interface DatagmaFindEmailResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Enrich Person // Endpoint: GET https://gateway.datagma.net/api/ingress/v2/full // Auth: apiId query param // Docs: https://datagmaapi.readme.io/reference/ingressservice_fullapiv2 // Pricing: 2 credits per successful response -// --------------------------------------------------------------------------- export interface DatagmaEnrichPersonParams extends DatagmaBaseParams { /** Email address, LinkedIn URL, or full name (use with companyKeyword) */ @@ -69,13 +65,11 @@ export interface DatagmaEnrichPersonResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Enrich Company (via full endpoint with company domain/name) // Endpoint: GET https://gateway.datagma.net/api/ingress/v2/full // Auth: apiId query param // Docs: https://datagmaapi.readme.io/reference/ingressservice_fullapiv2 // Pricing: 2 credits per successful response -// --------------------------------------------------------------------------- export interface DatagmaEnrichCompanyParams extends DatagmaBaseParams { /** Company domain, name, or SIREN number */ @@ -98,13 +92,11 @@ export interface DatagmaEnrichCompanyResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Find Phone (via search endpoint or enrich with phoneFull) // Endpoint: GET https://gateway.datagma.net/api/ingress/v1/search // Auth: apiId query param // Docs: https://datagmaapi.readme.io/reference/find-a-phone-number // Pricing: 30 credits per phone number found (1 credit = 1 email) -// --------------------------------------------------------------------------- export interface DatagmaFindPhoneParams extends DatagmaBaseParams { /** LinkedIn URL of the person */ @@ -123,13 +115,11 @@ export interface DatagmaFindPhoneResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Get Credits // Endpoint: GET https://gateway.datagma.net/api/ingress/v1/mine // Auth: apiId query param // Docs: https://datagmaapi.readme.io/reference/ingressservice_getcredit // Pricing: free (no credit consumed) -// --------------------------------------------------------------------------- export interface DatagmaGetCreditsParams extends DatagmaBaseParams {} @@ -139,9 +129,7 @@ export interface DatagmaGetCreditsResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Union of all response types -// --------------------------------------------------------------------------- export type DatagmaResponse = | DatagmaFindEmailResponse diff --git a/apps/sim/tools/dropbox/types.ts b/apps/sim/tools/dropbox/types.ts index 2b9440e6359..bfb5cf1c8ae 100644 --- a/apps/sim/tools/dropbox/types.ts +++ b/apps/sim/tools/dropbox/types.ts @@ -1,8 +1,6 @@ import type { UserFileLike } from '@/lib/core/utils/user-file' import type { ToolFileData, ToolResponse } from '@/tools/types' -// ===== Core Types ===== - interface DropboxFileMetadata { '.tag': 'file' id: string @@ -61,14 +59,10 @@ interface DropboxSearchMatch { } } -// ===== Base Params ===== - interface DropboxBaseParams { accessToken?: string } -// ===== Upload Params ===== - export interface DropboxUploadParams extends DropboxBaseParams { path: string file?: UserFileLike @@ -86,8 +80,6 @@ export interface DropboxUploadResponse extends ToolResponse { } } -// ===== Download Params ===== - export interface DropboxDownloadParams extends DropboxBaseParams { path: string } @@ -101,8 +93,6 @@ export interface DropboxDownloadResponse extends ToolResponse { } } -// ===== List Folder Params ===== - export interface DropboxListFolderParams extends DropboxBaseParams { path: string recursive?: boolean @@ -119,8 +109,6 @@ export interface DropboxListFolderResponse extends ToolResponse { } } -// ===== Create Folder Params ===== - export interface DropboxCreateFolderParams extends DropboxBaseParams { path: string autorename?: boolean @@ -132,8 +120,6 @@ export interface DropboxCreateFolderResponse extends ToolResponse { } } -// ===== Delete Params ===== - export interface DropboxDeleteParams extends DropboxBaseParams { path: string } @@ -145,8 +131,6 @@ export interface DropboxDeleteResponse extends ToolResponse { } } -// ===== Copy Params ===== - export interface DropboxCopyParams extends DropboxBaseParams { fromPath: string toPath: string @@ -159,8 +143,6 @@ export interface DropboxCopyResponse extends ToolResponse { } } -// ===== Move Params ===== - export interface DropboxMoveParams extends DropboxBaseParams { fromPath: string toPath: string @@ -173,8 +155,6 @@ export interface DropboxMoveResponse extends ToolResponse { } } -// ===== Get Metadata Params ===== - export interface DropboxGetMetadataParams extends DropboxBaseParams { path: string includeMediaInfo?: boolean @@ -187,8 +167,6 @@ export interface DropboxGetMetadataResponse extends ToolResponse { } } -// ===== Create Shared Link Params ===== - export interface DropboxCreateSharedLinkParams extends DropboxBaseParams { path: string requestedVisibility?: 'public' | 'team_only' | 'password' @@ -202,8 +180,6 @@ export interface DropboxCreateSharedLinkResponse extends ToolResponse { } } -// ===== Search Params ===== - export interface DropboxSearchParams extends DropboxBaseParams { query: string path?: string @@ -219,8 +195,6 @@ export interface DropboxSearchResponse extends ToolResponse { } } -// ===== Get Temporary Link Params ===== - interface DropboxGetTemporaryLinkParams extends DropboxBaseParams { path: string } @@ -232,8 +206,6 @@ interface DropboxGetTemporaryLinkResponse extends ToolResponse { } } -// ===== List Shared Links Params ===== - export interface DropboxListSharedLinksParams extends DropboxBaseParams { path?: string directOnly?: boolean @@ -248,8 +220,6 @@ export interface DropboxListSharedLinksResponse extends ToolResponse { } } -// ===== List Revisions Params ===== - interface DropboxFileRevision { '.tag': 'file' id: string @@ -275,8 +245,6 @@ export interface DropboxListRevisionsResponse extends ToolResponse { } } -// ===== Restore Params ===== - export interface DropboxRestoreParams extends DropboxBaseParams { path: string rev: string @@ -288,8 +256,6 @@ export interface DropboxRestoreResponse extends ToolResponse { } } -// ===== Combined Response Type ===== - export type DropboxResponse = | DropboxUploadResponse | DropboxDownloadResponse diff --git a/apps/sim/tools/dropcontact/types.ts b/apps/sim/tools/dropcontact/types.ts index 34d2fec5f07..fb64e9a6a14 100644 --- a/apps/sim/tools/dropcontact/types.ts +++ b/apps/sim/tools/dropcontact/types.ts @@ -4,9 +4,7 @@ export interface DropcontactBaseParams { apiKey: string } -// --------------------------------------------------------------------------- // Shared output property constants -// --------------------------------------------------------------------------- export const DROPCONTACT_EMAIL_ITEM_OUTPUT_PROPERTIES = { email: { type: 'string', description: 'Email address' }, @@ -26,9 +24,7 @@ export const DROPCONTACT_EMAILS_OUTPUT: OutputProperty = { }, } -// --------------------------------------------------------------------------- // Enrich Contact (single-contact async enrichment) -// --------------------------------------------------------------------------- export interface DropcontactEnrichContactParams extends DropcontactBaseParams { /** Email address of the contact to enrich */ diff --git a/apps/sim/tools/enrow/types.ts b/apps/sim/tools/enrow/types.ts index 1638a9ad81b..22827c4fbcd 100644 --- a/apps/sim/tools/enrow/types.ts +++ b/apps/sim/tools/enrow/types.ts @@ -5,9 +5,7 @@ export interface EnrowBaseParams { apiKey: string } -// --------------------------------------------------------------------------- // Email Finder — single -// --------------------------------------------------------------------------- export interface EnrowFindEmailParams extends EnrowBaseParams { fullname: string @@ -31,9 +29,7 @@ export interface EnrowFindEmailResponse extends ToolResponse { output: EnrowFindEmailResult } -// --------------------------------------------------------------------------- // Email Verifier — single -// --------------------------------------------------------------------------- export interface EnrowVerifyEmailParams extends EnrowBaseParams { email: string @@ -51,15 +47,11 @@ export interface EnrowVerifyEmailResponse extends ToolResponse { output: EnrowVerifyEmailResult } -// --------------------------------------------------------------------------- // Union response type (used in BlockConfig generic) -// --------------------------------------------------------------------------- export type EnrowResponse = EnrowFindEmailResponse | EnrowVerifyEmailResponse -// --------------------------------------------------------------------------- // Shared output property constants -// --------------------------------------------------------------------------- /** Reusable output-property definition for the Enrow job ID. */ export const ENROW_ID_OUTPUT: OutputProperty = { diff --git a/apps/sim/tools/google_forms/types.ts b/apps/sim/tools/google_forms/types.ts index 9a191d3f549..2d143adc937 100644 --- a/apps/sim/tools/google_forms/types.ts +++ b/apps/sim/tools/google_forms/types.ts @@ -1,8 +1,6 @@ import type { ToolResponse } from '@/tools/types' -// ============================================ // Common Types -// ============================================ export interface GoogleFormsResponse { responseId?: string @@ -83,9 +81,7 @@ export interface GoogleFormsWatch { errorType?: string } -// ============================================ // Get Responses Params -// ============================================ export interface GoogleFormsGetResponsesParams { accessToken: string @@ -96,9 +92,7 @@ export interface GoogleFormsGetResponsesParams { filter?: string } -// ============================================ // Get Form Params & Response -// ============================================ export interface GoogleFormsGetFormParams { accessToken: string @@ -120,9 +114,7 @@ export interface GoogleFormsGetFormResponse extends ToolResponse { } } -// ============================================ // Create Form Params & Response -// ============================================ export interface GoogleFormsCreateFormParams { accessToken: string @@ -141,9 +133,7 @@ export interface GoogleFormsCreateFormResponse extends ToolResponse { } } -// ============================================ // Batch Update Params & Response -// ============================================ interface GoogleFormsBatchUpdateRequest { updateFormInfo?: { @@ -190,9 +180,7 @@ export interface GoogleFormsBatchUpdateResponse extends ToolResponse { } } -// ============================================ // Set Publish Settings Params & Response -// ============================================ export interface GoogleFormsSetPublishSettingsParams { accessToken: string @@ -208,9 +196,7 @@ export interface GoogleFormsSetPublishSettingsResponse extends ToolResponse { } } -// ============================================ // Watch Params & Responses -// ============================================ export interface GoogleFormsCreateWatchParams { accessToken: string diff --git a/apps/sim/tools/google_maps/types.ts b/apps/sim/tools/google_maps/types.ts index 03d3bf3d226..b43bb6ceaff 100644 --- a/apps/sim/tools/google_maps/types.ts +++ b/apps/sim/tools/google_maps/types.ts @@ -94,7 +94,6 @@ interface Pollutant { } // Geocode -// ============================================================================ export interface GoogleMapsGeocodeParams { apiKey: string @@ -115,9 +114,7 @@ export interface GoogleMapsGeocodeResponse extends ToolResponse { } } -// ============================================================================ // Reverse Geocode -// ============================================================================ export interface GoogleMapsReverseGeocodeParams { apiKey: string @@ -135,9 +132,7 @@ export interface GoogleMapsReverseGeocodeResponse extends ToolResponse { } } -// ============================================================================ // Directions -// ============================================================================ interface DirectionsStep { instruction: string @@ -196,9 +191,7 @@ export interface GoogleMapsDirectionsResponse extends ToolResponse { } } -// ============================================================================ // Distance Matrix -// ============================================================================ interface DistanceMatrixElement { distanceText: string @@ -232,9 +225,7 @@ export interface GoogleMapsDistanceMatrixResponse extends ToolResponse { } } -// ============================================================================ // Places Search -// ============================================================================ interface PlaceResult { placeId: string @@ -269,9 +260,7 @@ export interface GoogleMapsPlacesSearchResponse extends ToolResponse { } } -// ============================================================================ // Places Nearby Search -// ============================================================================ interface NearbyPlaceResult { placeId: string @@ -305,9 +294,7 @@ export interface GoogleMapsPlacesNearbyResponse extends ToolResponse { } } -// ============================================================================ // Place Details -// ============================================================================ interface PlaceReview { authorName: string @@ -358,9 +345,7 @@ export interface GoogleMapsPlaceDetailsResponse extends ToolResponse { } } -// ============================================================================ // Elevation -// ============================================================================ export interface GoogleMapsElevationParams { apiKey: string @@ -377,9 +362,7 @@ export interface GoogleMapsElevationResponse extends ToolResponse { } } -// ============================================================================ // Timezone -// ============================================================================ export interface GoogleMapsTimezoneParams { apiKey: string @@ -400,9 +383,7 @@ export interface GoogleMapsTimezoneResponse extends ToolResponse { } } -// ============================================================================ // Snap to Roads -// ============================================================================ export interface GoogleMapsSnapToRoadsParams { apiKey: string @@ -417,9 +398,7 @@ export interface GoogleMapsSnapToRoadsResponse extends ToolResponse { } } -// ============================================================================ // Speed Limits -// ============================================================================ export interface GoogleMapsSpeedLimitsParams { apiKey: string @@ -435,9 +414,7 @@ export interface GoogleMapsSpeedLimitsResponse extends ToolResponse { } } -// ============================================================================ // Validate Address -// ============================================================================ export interface GoogleMapsValidateAddressParams { apiKey: string @@ -466,9 +443,7 @@ export interface GoogleMapsValidateAddressResponse extends ToolResponse { } } -// ============================================================================ // Geolocate -// ============================================================================ export interface GoogleMapsGeolocateParams { apiKey: string @@ -489,9 +464,7 @@ export interface GoogleMapsGeolocateResponse extends ToolResponse { } } -// ============================================================================ // Air Quality -// ============================================================================ export interface GoogleMapsAirQualityParams { apiKey: string @@ -518,9 +491,7 @@ export interface GoogleMapsAirQualityResponse extends ToolResponse { } } -// ============================================================================ // Pollen -// ============================================================================ /** * Calendar date returned by the Pollen and Solar APIs @@ -600,9 +571,7 @@ export interface GoogleMapsPollenResponse extends ToolResponse { } } -// ============================================================================ // Solar -// ============================================================================ interface SolarPanelConfig { panelsCount: number diff --git a/apps/sim/tools/icypeas/types.ts b/apps/sim/tools/icypeas/types.ts index 44f0fafe628..6c89d43daf6 100644 --- a/apps/sim/tools/icypeas/types.ts +++ b/apps/sim/tools/icypeas/types.ts @@ -5,9 +5,7 @@ export interface IcypeasBaseParams { apiKey: string } -// --------------------------------------------------------------------------- // Email Finder (single email discovery) -// --------------------------------------------------------------------------- export interface IcypeasFindEmailParams extends IcypeasBaseParams { firstname?: string @@ -30,9 +28,7 @@ export interface IcypeasFindEmailResponse extends ToolResponse { output: IcypeasFindEmailOutput } -// --------------------------------------------------------------------------- // Email Verification -// --------------------------------------------------------------------------- export interface IcypeasVerifyEmailParams extends IcypeasBaseParams { email: string @@ -53,15 +49,11 @@ export interface IcypeasVerifyEmailResponse extends ToolResponse { output: IcypeasVerifyEmailOutput } -// --------------------------------------------------------------------------- // Union response type used by the block -// --------------------------------------------------------------------------- export type IcypeasResponse = IcypeasFindEmailResponse | IcypeasVerifyEmailResponse -// --------------------------------------------------------------------------- // Shared output property constants -// --------------------------------------------------------------------------- export const ICYPEAS_SEARCH_ID_OUTPUT: OutputProperty = { type: 'string', diff --git a/apps/sim/tools/intercom/types.ts b/apps/sim/tools/intercom/types.ts index aa7f4072164..3bca7f82b3e 100644 --- a/apps/sim/tools/intercom/types.ts +++ b/apps/sim/tools/intercom/types.ts @@ -14,9 +14,7 @@ const logger = createLogger('Intercom') * - https://developers.intercom.com/docs/references/rest-api/api.intercom.io/tickets/ticket */ -// ============================================================================ // Location Output Properties -// ============================================================================ /** * Output definition for location object (nested in contact) @@ -41,9 +39,7 @@ export const INTERCOM_LOCATION_OUTPUT: OutputProperty = { properties: INTERCOM_LOCATION_OUTPUT_PROPERTIES, } -// ============================================================================ // Social Profiles Output Properties -// ============================================================================ /** * Output definition for social profile object @@ -76,9 +72,7 @@ export const INTERCOM_SOCIAL_PROFILES_OUTPUT: OutputProperty = { }, } -// ============================================================================ // List Reference Output Properties (tags, notes, companies on contact) -// ============================================================================ /** * Output definition for list reference objects (used for tags, notes, companies on contacts) @@ -100,9 +94,7 @@ export const INTERCOM_LIST_REFERENCE_OUTPUT: OutputProperty = { properties: INTERCOM_LIST_REFERENCE_OUTPUT_PROPERTIES, } -// ============================================================================ // Tag Output Properties -// ============================================================================ /** * Output definition for tag objects @@ -134,9 +126,7 @@ export const INTERCOM_TAGS_ARRAY_OUTPUT: OutputProperty = { }, } -// ============================================================================ // Admin Output Properties -// ============================================================================ /** * Output definition for admin avatar object @@ -206,9 +196,7 @@ export const INTERCOM_ADMINS_ARRAY_OUTPUT: OutputProperty = { }, } -// ============================================================================ // Contact Output Properties -// ============================================================================ /** * Core contact properties (common fields) @@ -350,9 +338,7 @@ export const INTERCOM_CONTACTS_ARRAY_OUTPUT: OutputProperty = { }, } -// ============================================================================ // Company Output Properties -// ============================================================================ /** * Output definition for company plan object @@ -495,9 +481,7 @@ export const INTERCOM_COMPANIES_ARRAY_OUTPUT: OutputProperty = { }, } -// ============================================================================ // Conversation Output Properties -// ============================================================================ /** * Output definition for conversation source object @@ -872,9 +856,7 @@ export const INTERCOM_CONVERSATIONS_ARRAY_OUTPUT: OutputProperty = { }, } -// ============================================================================ // Ticket Output Properties -// ============================================================================ /** * Output definition for ticket type object @@ -977,9 +959,7 @@ export const INTERCOM_TICKET_OUTPUT: OutputProperty = { properties: INTERCOM_TICKET_OUTPUT_PROPERTIES, } -// ============================================================================ // Pagination Output Properties -// ============================================================================ /** * Output definition for pagination cursor diff --git a/apps/sim/tools/leadmagic/types.ts b/apps/sim/tools/leadmagic/types.ts index 394106f38a5..d1836b804bf 100644 --- a/apps/sim/tools/leadmagic/types.ts +++ b/apps/sim/tools/leadmagic/types.ts @@ -4,9 +4,7 @@ interface LeadMagicBaseParams { apiKey: string } -// --------------------------------------------------------------------------- // Shared output property constants -// --------------------------------------------------------------------------- export const LEADMAGIC_PROFILE_OUTPUT_PROPERTIES = { profile_url: { type: 'string', description: 'LinkedIn profile URL' }, @@ -22,9 +20,7 @@ export const LEADMAGIC_PROFILE_OUTPUT_PROPERTIES = { company_website: { type: 'string', description: 'Company website', optional: true }, } as const satisfies Record -// --------------------------------------------------------------------------- // Email Validation -// --------------------------------------------------------------------------- export interface LeadMagicValidateEmailParams extends LeadMagicBaseParams { email: string @@ -47,9 +43,7 @@ export interface LeadMagicValidateEmailResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Email Finder -// --------------------------------------------------------------------------- export interface LeadMagicFindEmailParams extends LeadMagicBaseParams { first_name?: string @@ -76,9 +70,7 @@ export interface LeadMagicFindEmailResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Mobile Finder -// --------------------------------------------------------------------------- export interface LeadMagicFindMobileParams extends LeadMagicBaseParams { profile_url?: string @@ -96,9 +88,7 @@ export interface LeadMagicFindMobileResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Profile Search (LinkedIn enrichment by profile URL) -// --------------------------------------------------------------------------- export interface LeadMagicProfileSearchParams extends LeadMagicBaseParams { profile_url: string @@ -129,9 +119,7 @@ export interface LeadMagicProfileSearchResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Profile to Email (LinkedIn URL → work email) -// --------------------------------------------------------------------------- export interface LeadMagicProfileToEmailParams extends LeadMagicBaseParams { profile_url: string @@ -146,9 +134,7 @@ export interface LeadMagicProfileToEmailResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Email to Profile (work/personal email → LinkedIn profile URL) -// --------------------------------------------------------------------------- export interface LeadMagicEmailToProfileParams extends LeadMagicBaseParams { work_email?: string @@ -163,9 +149,7 @@ export interface LeadMagicEmailToProfileResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Company Search -// --------------------------------------------------------------------------- export interface LeadMagicCompanySearchParams extends LeadMagicBaseParams { company_domain?: string @@ -197,9 +181,7 @@ export interface LeadMagicCompanySearchResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Role Finder -// --------------------------------------------------------------------------- export interface LeadMagicRoleFinderParams extends LeadMagicBaseParams { job_title: string @@ -221,9 +203,7 @@ export interface LeadMagicRoleFinderResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Get Credits (balance check — free, no hosting) -// --------------------------------------------------------------------------- export interface LeadMagicGetCreditsParams extends LeadMagicBaseParams {} @@ -233,9 +213,7 @@ export interface LeadMagicGetCreditsResponse extends ToolResponse { } } -// --------------------------------------------------------------------------- // Union response type -// --------------------------------------------------------------------------- export type LeadMagicResponse = | LeadMagicValidateEmailResponse diff --git a/apps/sim/tools/linear/types.ts b/apps/sim/tools/linear/types.ts index dbd9992cd3a..b8611210bfc 100644 --- a/apps/sim/tools/linear/types.ts +++ b/apps/sim/tools/linear/types.ts @@ -520,8 +520,6 @@ export const ISSUE_LIST_OUTPUT_PROPERTIES = { labels: LABELS_OUTPUT, } as const satisfies Record -// ===== Core Types ===== - interface LinearIssue { id: string title: string @@ -654,8 +652,6 @@ interface LinearCycle { } } -// ===== Request Params ===== - export interface LinearReadIssuesParams { teamId?: string projectId?: string @@ -997,8 +993,6 @@ export interface LinearUpdateNotificationParams { accessToken?: string } -// ===== Response Types ===== - export interface LinearReadIssuesResponse extends ToolResponse { output: { issues?: LinearIssue[] @@ -1394,8 +1388,6 @@ export interface LinearUpdateNotificationResponse extends ToolResponse { } } -// ===== Customer Types ===== - interface LinearCustomer { id: string name: string @@ -1447,8 +1439,6 @@ export interface LinearListCustomersResponse extends ToolResponse { } } -// ===== Customer Need (Request) Types ===== - interface LinearCustomerNeed { id: string body?: string @@ -1577,8 +1567,6 @@ export interface LinearMergeCustomersResponse extends ToolResponse { } } -// ===== Customer Status Types ===== - interface LinearCustomerStatus { id: string name: string @@ -1649,8 +1637,6 @@ export interface LinearListCustomerStatusesResponse extends ToolResponse { } } -// ===== Customer Tier Types ===== - interface LinearCustomerTier { id: string name: string @@ -1720,8 +1706,6 @@ export interface LinearListCustomerTiersResponse extends ToolResponse { } } -// ===== Project Label Types ===== - interface LinearProjectLabel { id: string name: string @@ -1816,8 +1800,6 @@ export interface LinearRemoveLabelFromProjectResponse extends ToolResponse { } } -// ===== Project Milestone Types ===== - interface LinearProjectMilestone { id: string name: string @@ -1887,8 +1869,6 @@ export interface LinearListProjectMilestonesResponse extends ToolResponse { } } -// ===== Project Status Types ===== - interface LinearProjectStatus { id: string name: string @@ -1961,8 +1941,6 @@ export interface LinearListProjectStatusesResponse extends ToolResponse { } } -// ===== Project Delete Types ===== - export interface LinearDeleteProjectParams { projectId: string accessToken?: string diff --git a/apps/sim/tools/params.ts b/apps/sim/tools/params.ts index efbca622015..08b3a8b31a6 100644 --- a/apps/sim/tools/params.ts +++ b/apps/sim/tools/params.ts @@ -31,9 +31,7 @@ import type { const logger = createLogger('ToolsParams') type ToolParamDefinition = ToolConfig['params'][string] -// ============================================================================ // Tag/Value Parsing Utilities -// ============================================================================ interface Option { label: string diff --git a/apps/sim/tools/stripe/types.ts b/apps/sim/tools/stripe/types.ts index 5b1055d9bf0..88725dfa7c5 100644 --- a/apps/sim/tools/stripe/types.ts +++ b/apps/sim/tools/stripe/types.ts @@ -1295,9 +1295,7 @@ interface StripeMetadata { [key: string]: string } -// ============================================================================ // Payment Intent Types -// ============================================================================ interface PaymentIntentObject { id: string @@ -1394,9 +1392,7 @@ export interface PaymentIntentListResponse extends ToolResponse { } } -// ============================================================================ // Customer Types -// ============================================================================ interface CustomerObject { id: string @@ -1484,9 +1480,7 @@ export interface CustomerDeleteResponse extends ToolResponse { } } -// ============================================================================ // Subscription Types -// ============================================================================ interface SubscriptionObject { id: string @@ -1581,9 +1575,7 @@ export interface SubscriptionListResponse extends ToolResponse { } } -// ============================================================================ // Invoice Types -// ============================================================================ interface InvoiceObject { id: string @@ -1691,9 +1683,7 @@ export interface InvoiceDeleteResponse extends ToolResponse { } } -// ============================================================================ // Charge Types -// ============================================================================ interface ChargeObject { id: string @@ -1775,9 +1765,7 @@ export interface ChargeListResponse extends ToolResponse { } } -// ============================================================================ // Product Types -// ============================================================================ interface ProductObject { id: string @@ -1860,9 +1848,7 @@ export interface ProductDeleteResponse extends ToolResponse { } } -// ============================================================================ // Price Types -// ============================================================================ interface PriceObject { id: string @@ -1940,9 +1926,7 @@ export interface PriceListResponse extends ToolResponse { } } -// ============================================================================ // Event Types -// ============================================================================ interface EventObject { id: string diff --git a/apps/sim/tools/wordpress/types.ts b/apps/sim/tools/wordpress/types.ts index 146b72905c1..9936ab70980 100644 --- a/apps/sim/tools/wordpress/types.ts +++ b/apps/sim/tools/wordpress/types.ts @@ -18,9 +18,7 @@ export type PostStatus = 'publish' | 'draft' | 'pending' | 'private' | 'future' // Comment status types export type CommentStatus = 'approved' | 'hold' | 'spam' | 'trash' -// ============================================ // POST OPERATIONS -// ============================================ // Create Post export interface WordPressCreatePostParams extends WordPressBaseParams { @@ -142,9 +140,7 @@ interface WordPressSearchPostsResponse extends ToolResponse { } } -// ============================================ // PAGE OPERATIONS -// ============================================ // Create Page export interface WordPressCreatePageParams extends WordPressBaseParams { @@ -249,9 +245,7 @@ export interface WordPressListPagesResponse extends ToolResponse { } } -// ============================================ // MEDIA OPERATIONS -// ============================================ // Upload Media export interface WordPressUploadMediaParams extends WordPressBaseParams { @@ -334,9 +328,7 @@ export interface WordPressDeleteMediaResponse extends ToolResponse { } } -// ============================================ // COMMENT OPERATIONS -// ============================================ // Create Comment export interface WordPressCreateCommentParams extends WordPressBaseParams { @@ -426,9 +418,7 @@ export interface WordPressDeleteCommentResponse extends ToolResponse { } } -// ============================================ // TAXONOMY OPERATIONS (Categories & Tags) -// ============================================ // Create Category export interface WordPressCreateCategoryParams extends WordPressBaseParams { @@ -585,9 +575,7 @@ export interface WordPressDeleteTagResponse extends ToolResponse { } } -// ============================================ // USER OPERATIONS -// ============================================ // Get Current User export interface WordPressGetCurrentUserParams extends WordPressBaseParams {} @@ -641,9 +629,7 @@ export interface WordPressGetUserResponse extends ToolResponse { } } -// ============================================ // SEARCH OPERATIONS -// ============================================ // Search Content export interface WordPressSearchContentParams extends WordPressBaseParams { diff --git a/package.json b/package.json index 81f5d23783b..7d4605fe321 100644 --- a/package.json +++ b/package.json @@ -34,7 +34,7 @@ "check:api-validation:strict": "bun run scripts/check-api-validation-contracts.ts --check --enforce-boundary-baseline", "check:realtime-prune": "bun run scripts/check-realtime-prune-graph.ts", "check:tool-request-boundary": "bun run scripts/check-tool-request-boundary.ts", - "check:tool-registry-boundary": "bun run scripts/check-tool-registry-boundary.ts", + "check:tool-registry-boundary": "bun run scripts/check-tool-registry-boundary.ts --check", "check:trigger-block-cycle": "bun run scripts/check-trigger-block-cycle.ts", "check:import-specifiers": "bun run scripts/check-import-specifiers.ts", "check:sql-date-binding": "bun run scripts/check-sql-date-binding.ts", diff --git a/packages/db/scripts/migrate-block-api-keys-to-byok.ts b/packages/db/scripts/migrate-block-api-keys-to-byok.ts index 009e3e9b35e..ad6ef39c5be 100644 --- a/packages/db/scripts/migrate-block-api-keys-to-byok.ts +++ b/packages/db/scripts/migrate-block-api-keys-to-byok.ts @@ -52,7 +52,6 @@ import { index, json, jsonb, pgTable, text, timestamp } from 'drizzle-orm/pg-cor import { drizzle } from 'drizzle-orm/postgres-js' import postgres from 'postgres' -// ---------- CLI ---------- const DRY_RUN = process.argv.includes('--dry-run') function parseMapArgs(): Record { @@ -225,7 +224,6 @@ if (DRY_RUN && FROM_FILE) { process.exit(1) } -// ---------- Env ---------- function getEnv(name: string): string | undefined { if (typeof process !== 'undefined' && process.env && name in process.env) { return process.env[name] @@ -245,7 +243,6 @@ if (!ENCRYPTION_KEY || ENCRYPTION_KEY.length !== 64) { process.exit(1) } -// ---------- Encryption (mirrors apps/sim/lib/core/security/encryption.ts) ---------- function getEncryptionKeyBuffer(): Buffer { return Buffer.from(ENCRYPTION_KEY!, 'hex') } @@ -282,7 +279,6 @@ async function decryptSecret(encryptedValue: string): Promise { return decrypted } -// ---------- Schema ---------- const workspaceTable = pgTable('workspace', { id: text('id').primaryKey(), ownerId: text('owner_id').notNull(), @@ -347,7 +343,6 @@ const WORKSPACE_CONCURRENCY = 100 const WORKSPACE_BATCH_SIZE = 1000 const SLEEP_MS = 30_000 -// ---------- DB ---------- const postgresClient = postgres(CONNECTION_STRING, { prepare: false, idle_timeout: 20, @@ -357,7 +352,6 @@ const postgresClient = postgres(CONNECTION_STRING, { }) const db = drizzle(postgresClient) -// ---------- Helpers ---------- const TOOL_INPUT_SUBBLOCK_IDS: Record = { agent: 'tools', human_in_the_loop: 'notification', @@ -754,7 +748,6 @@ async function processWorkspace( } } -// ---------- Main ---------- async function run() { console.log(`Mode: ${DRY_RUN ? 'DRY RUN (audit + preview)' : 'LIVE'}`) console.log( diff --git a/packages/db/scripts/migrate-deployment-versions.ts b/packages/db/scripts/migrate-deployment-versions.ts index 98685135e06..98841d98508 100644 --- a/packages/db/scripts/migrate-deployment-versions.ts +++ b/packages/db/scripts/migrate-deployment-versions.ts @@ -9,7 +9,6 @@ import { sql } from 'drizzle-orm' import { drizzle } from 'drizzle-orm/postgres-js' import postgres from 'postgres' -// ---------- Minimal env helpers ---------- function getEnv(name: string): string | undefined { if (typeof process !== 'undefined' && process.env && name in process.env) { return process.env[name] @@ -23,7 +22,6 @@ if (!CONNECTION_STRING) { process.exit(1) } -// ---------- Minimal schema (only what we need) ---------- import { boolean, index, integer, json, jsonb, pgTable, text, timestamp } from 'drizzle-orm/pg-core' // Tables referenced by the script @@ -117,7 +115,6 @@ const workflowDeploymentVersion = pgTable( }) ) -// ---------- DB client ---------- const postgresClient = postgres(CONNECTION_STRING, { prepare: false, idle_timeout: 20, @@ -127,7 +124,6 @@ const postgresClient = postgres(CONNECTION_STRING, { }) const db = drizzle(postgresClient) -// ---------- Minimal types ---------- type WorkflowState = { blocks: Record edges: Array<{ @@ -141,7 +137,6 @@ type WorkflowState = { parallels: Record } -// ---------- Normalized loader (inline of loadWorkflowFromNormalizedTables) ---------- async function loadWorkflowFromNormalizedTables(workflowId: string) { const [blocks, edges, subflows] = await Promise.all([ db.select().from(workflowBlocks).where(sql`${workflowBlocks.workflowId} = ${workflowId}`), @@ -210,7 +205,6 @@ async function loadWorkflowFromNormalizedTables(workflowId: string) { } } -// ---------- Migration ---------- const DRY_RUN = process.argv.includes('--dry-run') const BATCH_SIZE = 50 diff --git a/packages/utils/src/fractional-indexing.ts b/packages/utils/src/fractional-indexing.ts index 7eab3d5912d..9e5a54a8e54 100644 --- a/packages/utils/src/fractional-indexing.ts +++ b/packages/utils/src/fractional-indexing.ts @@ -14,16 +14,12 @@ * with no trailing zero. */ -// --------------------------------------------------------------------------- // Digits -// --------------------------------------------------------------------------- /** Default digit alphabet. Must be in ascending character-code order. */ export const BASE_62_DIGITS = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz' -// --------------------------------------------------------------------------- // Integer-part helpers -// --------------------------------------------------------------------------- /** Length the integer part must have, derived from its first character. */ function getIntegerLength(head: string): number { @@ -126,9 +122,7 @@ function decrementInteger(x: string, digits: string): string | null { return head + digs.join('') } -// --------------------------------------------------------------------------- // Midpoint -// --------------------------------------------------------------------------- /** * Fraction strictly between `a` and `b` (both without integer parts). `a` may be @@ -168,9 +162,7 @@ function midpoint(a: string, b: string | null | undefined, digits: string): stri return digits[digitA] + midpoint(a.slice(1), null, digits) } -// --------------------------------------------------------------------------- // Public API -// --------------------------------------------------------------------------- /** * Returns a key that sorts strictly between `a` and `b`. Either may be null for diff --git a/scripts/check-tool-registry-boundary.baseline.json b/scripts/check-tool-registry-boundary.baseline.json new file mode 100644 index 00000000000..5e8ebeb9d3b --- /dev/null +++ b/scripts/check-tool-registry-boundary.baseline.json @@ -0,0 +1,382 @@ +{ + "generatedFrom": "app/workspace page/layout module graphs", + "tolerance": { + "modules": 25, + "percent": 2 + }, + "entries": { + "app/workspace/[workspaceId]/chat/[chatId]/layout.tsx": { + "modules": 5, + "gateways": {} + }, + "app/workspace/[workspaceId]/chat/[chatId]/page.tsx": { + "modules": 2890, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/home/home.tsx": 1328, + "apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/mothership-view.tsx": 971, + "apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/index.ts": 836, + "apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/resource-content/index.ts": 833, + "apps/sim/triggers/registry.ts": 446, + "apps/sim/blocks/registry.ts": 301, + "apps/sim/lib/auth/index.ts": 297, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx": 295 + } + }, + "app/workspace/[workspaceId]/files/[fileId]/page.tsx": { + "modules": 1909, + "gateways": { + "apps/sim/triggers/registry.ts": 446, + "apps/sim/blocks/registry.ts": 328, + "apps/sim/lib/auth/index.ts": 300, + "apps/sim/app/workspace/[workspaceId]/files/files.tsx": 275, + "apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/index.ts": 143, + "apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/file-viewer.tsx": 129, + "apps/sim/lib/api/contracts/index.ts": 107, + "apps/sim/lib/webhooks/providers/index.ts": 99 + } + }, + "app/workspace/[workspaceId]/files/[fileId]/view/page.tsx": { + "modules": 57, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/files/[fileId]/view/file-viewer.tsx": 56, + "apps/sim/hooks/queries/workspace-files.ts": 53 + } + }, + "app/workspace/[workspaceId]/files/page.tsx": { + "modules": 1909, + "gateways": { + "apps/sim/triggers/registry.ts": 446, + "apps/sim/blocks/registry.ts": 328, + "apps/sim/lib/auth/index.ts": 300, + "apps/sim/app/workspace/[workspaceId]/files/files.tsx": 277, + "apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/index.ts": 143, + "apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/file-viewer.tsx": 129, + "apps/sim/lib/api/contracts/index.ts": 107, + "apps/sim/lib/webhooks/providers/index.ts": 99 + } + }, + "app/workspace/[workspaceId]/home/layout.tsx": { + "modules": 6, + "gateways": {} + }, + "app/workspace/[workspaceId]/home/page.tsx": { + "modules": 2890, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/home/home.tsx": 1328, + "apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/mothership-view.tsx": 971, + "apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/index.ts": 836, + "apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/resource-content/index.ts": 833, + "apps/sim/triggers/registry.ts": 446, + "apps/sim/blocks/registry.ts": 301, + "apps/sim/lib/auth/index.ts": 297, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx": 295 + } + }, + "app/workspace/[workspaceId]/integrations/[block]/page.tsx": { + "modules": 1268, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/integrations/[block]/integration-block-detail.tsx": 1243, + "apps/sim/blocks/registry.ts": 925, + "apps/sim/triggers/index.ts": 482, + "apps/sim/lib/api/contracts/index.ts": 128, + "apps/sim/stores/workflows/registry/store.ts": 82, + "apps/sim/lib/api/contracts/tools/index.ts": 60, + "apps/sim/hooks/queries/deployments.ts": 59, + "apps/sim/lib/workflows/comparison/compare.ts": 56 + } + }, + "app/workspace/[workspaceId]/integrations/connected/[credentialId]/page.tsx": { + "modules": 1254, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/integrations/connected/[credentialId]/connected-credential-detail.tsx": 1253, + "apps/sim/triggers/registry.ts": 481, + "apps/sim/blocks/registry.ts": 331, + "apps/sim/lib/api/contracts/index.ts": 134, + "apps/sim/stores/workflows/registry/store.ts": 62, + "apps/sim/lib/api/contracts/tools/index.ts": 60, + "apps/sim/hooks/queries/deployments.ts": 59, + "apps/sim/lib/workflows/comparison/compare.ts": 56 + } + }, + "app/workspace/[workspaceId]/integrations/page.tsx": { + "modules": 1253, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/integrations/integrations.tsx": 979, + "apps/sim/blocks/registry.ts": 926, + "apps/sim/triggers/index.ts": 482, + "apps/sim/lib/api/contracts/index.ts": 130, + "apps/sim/stores/workflows/registry/store.ts": 83, + "apps/sim/lib/api/contracts/tools/index.ts": 60, + "apps/sim/hooks/queries/deployments.ts": 59, + "apps/sim/lib/workflows/comparison/compare.ts": 56 + } + }, + "app/workspace/[workspaceId]/knowledge/[id]/[documentId]/page.tsx": { + "modules": 1460, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/knowledge/[id]/[documentId]/document.tsx": 1183, + "apps/sim/triggers/registry.ts": 481, + "apps/sim/blocks/registry.ts": 322, + "apps/sim/blocks/registry-maps.ts": 319, + "apps/sim/lib/api/contracts/index.ts": 119, + "apps/sim/lib/api/contracts/tools/index.ts": 60, + "apps/sim/connectors/registry.ts": 53, + "apps/sim/app/workspace/[workspaceId]/components/index.ts": 52 + } + }, + "app/workspace/[workspaceId]/knowledge/[id]/page.tsx": { + "modules": 1461, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/knowledge/[id]/base.tsx": 1184, + "apps/sim/triggers/registry.ts": 481, + "apps/sim/blocks/registry.ts": 322, + "apps/sim/blocks/registry-maps.ts": 319, + "apps/sim/lib/api/contracts/index.ts": 119, + "apps/sim/lib/api/contracts/tools/index.ts": 60, + "apps/sim/connectors/registry.ts": 53, + "apps/sim/app/workspace/[workspaceId]/components/index.ts": 52 + } + }, + "app/workspace/[workspaceId]/knowledge/page.tsx": { + "modules": 2091, + "gateways": { + "apps/sim/triggers/registry.ts": 446, + "apps/sim/blocks/registry.ts": 317, + "apps/sim/app/workspace/[workspaceId]/knowledge/prefetch.ts": 249, + "apps/sim/lib/knowledge/application/knowledge-bases.ts": 198, + "apps/sim/app/workspace/[workspaceId]/knowledge/knowledge.tsx": 168, + "apps/sim/lib/auth/index.ts": 158, + "apps/sim/lib/knowledge/orchestration/index.ts": 121, + "apps/sim/lib/knowledge/orchestration/connectors.ts": 116 + } + }, + "app/workspace/[workspaceId]/layout.tsx": { + "modules": 1957, + "gateways": { + "apps/sim/triggers/registry.ts": 446, + "apps/sim/blocks/registry.ts": 316, + "apps/sim/app/workspace/[workspaceId]/components/workspace-chrome/index.ts": 255, + "apps/sim/app/workspace/[workspaceId]/w/components/sidebar/sidebar.tsx": 247, + "apps/sim/lib/auth/index.ts": 180, + "apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/index.ts": 151, + "apps/sim/lib/api/contracts/index.ts": 109, + "apps/sim/lib/webhooks/providers/index.ts": 99 + } + }, + "app/workspace/[workspaceId]/logs/page.tsx": { + "modules": 1696, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/logs/logs.tsx": 1421, + "apps/sim/triggers/registry.ts": 481, + "apps/sim/app/workspace/[workspaceId]/logs/components/index.ts": 418, + "apps/sim/app/workspace/[workspaceId]/logs/components/log-details/components/execution-snapshot/index.ts": 366, + "apps/sim/app/workspace/[workspaceId]/w/components/preview/components/preview-editor/index.ts": 322, + "apps/sim/blocks/registry.ts": 318, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/index.ts": 286, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/sub-block.tsx": 256 + } + }, + "app/workspace/[workspaceId]/page.tsx": { + "modules": 5, + "gateways": {} + }, + "app/workspace/[workspaceId]/settings/[section]/layout.tsx": { + "modules": 4, + "gateways": {} + }, + "app/workspace/[workspaceId]/settings/[section]/page.tsx": { + "modules": 1977, + "gateways": { + "apps/sim/triggers/registry.ts": 446, + "apps/sim/app/workspace/[workspaceId]/settings/[section]/settings.tsx": 409, + "apps/sim/blocks/registry.ts": 320, + "apps/sim/lib/auth/index.ts": 282, + "apps/sim/lib/api/contracts/index.ts": 106, + "apps/sim/lib/webhooks/providers/index.ts": 99, + "apps/sim/lib/api/contracts/tools/index.ts": 59, + "apps/sim/lib/workflows/lifecycle.ts": 48 + } + }, + "app/workspace/[workspaceId]/settings/billing/credit-usage/layout.tsx": { + "modules": 4, + "gateways": {} + }, + "app/workspace/[workspaceId]/settings/billing/credit-usage/page.tsx": { + "modules": 1573, + "gateways": { + "apps/sim/lib/auth/index.ts": 1445, + "apps/sim/triggers/index.ts": 447, + "apps/sim/blocks/registry.ts": 330, + "apps/sim/blocks/registry-maps.ts": 327, + "apps/sim/lib/api/contracts/index.ts": 122, + "apps/sim/lib/webhooks/providers/index.ts": 99, + "apps/sim/stores/workflows/registry/store.ts": 71, + "apps/sim/lib/api/contracts/tools/index.ts": 60 + } + }, + "app/workspace/[workspaceId]/settings/layout.tsx": { + "modules": 3, + "gateways": {} + }, + "app/workspace/[workspaceId]/settings/page.tsx": { + "modules": 1, + "gateways": {} + }, + "app/workspace/[workspaceId]/settings/secrets/[credentialId]/page.tsx": { + "modules": 1283, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/settings/secrets/[credentialId]/secret-detail.tsx": 1282, + "apps/sim/app/workspace/[workspaceId]/components/credential-detail/index.ts": 993, + "apps/sim/components/permissions/index.ts": 980, + "apps/sim/components/permissions/add-people-modal.tsx": 971, + "apps/sim/app/workspace/[workspaceId]/providers/workspace-permissions-provider.tsx": 969, + "apps/sim/triggers/registry.ts": 481, + "apps/sim/blocks/registry.ts": 333, + "apps/sim/blocks/registry-maps.ts": 330 + } + }, + "app/workspace/[workspaceId]/skills/[skillId]/page.tsx": { + "modules": 1374, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/skills/[skillId]/skill-detail.tsx": 1373, + "apps/sim/triggers/registry.ts": 481, + "apps/sim/blocks/registry.ts": 332, + "apps/sim/blocks/registry-maps.ts": 330, + "apps/sim/lib/api/contracts/index.ts": 126, + "apps/sim/app/workspace/[workspaceId]/skills/components/skill-fields/index.ts": 89, + "apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/rich-markdown-field.tsx": 86, + "apps/sim/lib/api/contracts/tools/index.ts": 60 + } + }, + "app/workspace/[workspaceId]/skills/new/page.tsx": { + "modules": 1372, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/skills/new/skill-create.tsx": 1371, + "apps/sim/triggers/registry.ts": 481, + "apps/sim/blocks/registry.ts": 332, + "apps/sim/blocks/registry-maps.ts": 330, + "apps/sim/lib/api/contracts/index.ts": 126, + "apps/sim/app/workspace/[workspaceId]/skills/components/skill-fields/index.ts": 89, + "apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/rich-markdown-field.tsx": 86, + "apps/sim/lib/api/contracts/tools/index.ts": 60 + } + }, + "app/workspace/[workspaceId]/skills/page.tsx": { + "modules": 1236, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/skills/skills.tsx": 962, + "apps/sim/app/workspace/[workspaceId]/integrations/components/showcase-with-explore/index.ts": 950, + "apps/sim/blocks/registry.ts": 938, + "apps/sim/blocks/registry-maps.ts": 936, + "apps/sim/triggers/index.ts": 482, + "apps/sim/lib/api/contracts/index.ts": 135, + "apps/sim/stores/workflows/registry/store.ts": 84, + "apps/sim/hooks/queries/deployments.ts": 60 + } + }, + "app/workspace/[workspaceId]/tables/[tableId]/page.tsx": { + "modules": 2186, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/tables/[tableId]/table.tsx": 574, + "apps/sim/triggers/registry.ts": 446, + "apps/sim/app/workspace/[workspaceId]/w/components/preview/index.ts": 328, + "apps/sim/lib/auth/index.ts": 302, + "apps/sim/blocks/registry.ts": 301, + "apps/sim/app/workspace/[workspaceId]/w/components/preview/components/preview-editor/index.ts": 286, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/index.ts": 259, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/sub-block.tsx": 230 + } + }, + "app/workspace/[workspaceId]/tables/page.tsx": { + "modules": 1767, + "gateways": { + "apps/sim/triggers/registry.ts": 446, + "apps/sim/blocks/registry.ts": 327, + "apps/sim/lib/auth/index.ts": 296, + "apps/sim/app/workspace/[workspaceId]/tables/tables.tsx": 121, + "apps/sim/lib/api/contracts/index.ts": 111, + "apps/sim/lib/webhooks/providers/index.ts": 99, + "apps/sim/lib/api/contracts/tools/index.ts": 60, + "apps/sim/app/workspace/[workspaceId]/components/index.ts": 50 + } + }, + "app/workspace/[workspaceId]/upgrade/page.tsx": { + "modules": 263, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/upgrade/upgrade.tsx": 256, + "apps/sim/app/workspace/[workspaceId]/upgrade/hooks/index.ts": 210, + "apps/sim/lib/billing/client/upgrade.ts": 205, + "apps/sim/hooks/queries/organization.ts": 201, + "apps/sim/hooks/queries/workspace.ts": 192, + "apps/sim/lib/api/contracts/index.ts": 190, + "apps/sim/lib/api/contracts/tools/index.ts": 61, + "apps/sim/lib/api/contracts/v1/index.ts": 38 + } + }, + "app/workspace/[workspaceId]/w/[workflowId]/layout.tsx": { + "modules": 2145, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/error/index.tsx": 2144, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/index.ts": 540, + "apps/sim/triggers/registry.ts": 481, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/index.ts": 458, + "apps/sim/blocks/registry.ts": 318, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/index.ts": 285, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/index.ts": 141, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/sub-block.tsx": 134 + } + }, + "app/workspace/[workspaceId]/w/[workflowId]/page.tsx": { + "modules": 2172, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx": 2171, + "apps/sim/triggers/registry.ts": 481, + "apps/sim/blocks/registry.ts": 318, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/index.ts": 305, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/index.ts": 267, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/index.ts": 224, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/index.ts": 140, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/sub-block.tsx": 133 + } + }, + "app/workspace/[workspaceId]/w/page.tsx": { + "modules": 2145, + "gateways": { + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/index.ts": 904, + "apps/sim/triggers/registry.ts": 481, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/index.ts": 458, + "apps/sim/blocks/registry.ts": 318, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/index.ts": 285, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/index.ts": 141, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/sub-block.tsx": 134, + "apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/error/index.tsx": 133 + } + }, + "app/workspace/layout.tsx": { + "modules": 1194, + "gateways": { + "apps/sim/app/workspace/providers/socket-provider.tsx": 1184, + "apps/sim/triggers/registry.ts": 481, + "apps/sim/blocks/registry.ts": 333, + "apps/sim/blocks/registry-maps.ts": 330, + "apps/sim/lib/api/contracts/index.ts": 139, + "apps/sim/stores/workflows/registry/store.ts": 63, + "apps/sim/hooks/queries/deployments.ts": 60, + "apps/sim/lib/api/contracts/tools/index.ts": 60 + } + }, + "app/workspace/page.tsx": { + "modules": 1188, + "gateways": { + "apps/sim/lib/auth/stale-session-recovery.ts": 959, + "apps/sim/triggers/index.ts": 482, + "apps/sim/blocks/registry.ts": 333, + "apps/sim/blocks/registry-maps.ts": 330, + "apps/sim/lib/api/contracts/index.ts": 138, + "apps/sim/stores/workflows/registry/store.ts": 62, + "apps/sim/lib/api/contracts/tools/index.ts": 60, + "apps/sim/hooks/queries/deployments.ts": 56 + } + } + } +} diff --git a/scripts/check-tool-registry-boundary.ts b/scripts/check-tool-registry-boundary.ts index e2d33bf862c..a47b09ebfe2 100644 --- a/scripts/check-tool-registry-boundary.ts +++ b/scripts/check-tool-registry-boundary.ts @@ -1,6 +1,7 @@ #!/usr/bin/env bun /** - * Fails if a workspace route can reach the executable tool registry. + * Fails if a workspace route can reach the executable tool registry, or if any + * route's module graph grows past its recorded baseline. * * `@/tools/registry` is a barrel over 4,300+ tools whose `ToolConfig`s hold * closures (`request.headers`, `transformResponse`, `directExecution`). Those @@ -19,11 +20,21 @@ * `formatParameterLabel`. Neither import looks remotely suspicious at the call * site, which is why this is a lint and not a convention. * + * The registry is only the loudest instance of the problem. The same walk yields + * each entry's exact module count, so `--check` additionally ratchets those + * counts against `check-tool-registry-boundary.baseline.json` (mirroring + * `check-react-query-patterns.ts`): an entry may not exceed its recorded size by + * more than `max(25 modules, 2%)`. A regression is reported with the dominator + * chain that grew — the import edge every one of the new modules must pass + * through — because a bare number is not actionable. + * * Usage: - * bun run scripts/check-tool-registry-boundary.ts - * bun run scripts/check-tool-registry-boundary.ts --verbose # print counts + * bun run scripts/check-tool-registry-boundary.ts # registry gate only + * bun run scripts/check-tool-registry-boundary.ts --check # + graph-weight ratchet + * bun run scripts/check-tool-registry-boundary.ts --verbose # print counts + * bun run scripts/check-tool-registry-boundary.ts --update-baseline */ -import { existsSync, readdirSync, readFileSync, statSync } from 'node:fs' +import { existsSync, readdirSync, readFileSync, statSync, writeFileSync } from 'node:fs' import { dirname, join, relative, resolve } from 'node:path' import { fileURLToPath } from 'node:url' @@ -74,8 +85,14 @@ const EXTENSIONS = ['.ts', '.tsx', '.js', '.jsx', '.mjs'] * `REQUIRE_RE` matters for the same reason: this codebase uses lazy * `require('@/…')` to break import cycles (`tools/params.ts` reaches `@/blocks` * that way), and those edges are as real as static ones. + * + * In `IMPORT_RE` the `from` clause is optional AND lazily optional (`??`). A + * greedy `?` tries to match the clause before trying to skip it, so a bare + * side-effect import (`import '@/executor'`) was swallowed as the prefix of the + * *next* statement's `from`: that edge was dropped and the next one attributed to + * the wrong importer. */ -const IMPORT_RE = /(?:^|\n)\s*import\s+(?!type\b)(?:[\s\S]*?from\s*)?['"]([^'"]+)['"]/g +const IMPORT_RE = /(?:^|\n)\s*import\s+(?!type\b)(?:[\s\S]*?from\s*)??['"]([^'"]+)['"]/g const REEXPORT_RE = /(?:^|\n)\s*export\s+(?!type\b)(?:\*(?:\s+as\s+[\w$]+)?|\{[\s\S]*?\})\s*from\s*['"]([^'"]+)['"]/g const DYNAMIC_IMPORT_RE = /\bimport\s*\(\s*['"]([^'"]+)['"]\s*\)/g @@ -105,41 +122,75 @@ function resolveSpecifier(specifier: string, importer: string): string | null { return null } +const NO_DEPS: readonly string[] = [] + +/** + * Resolved value-import edges out of one file. + * + * Memoized across entries: the 34 entries overlap heavily (every route drags in + * the same shell), so without this each file is re-read and re-scanned once per + * entry that reaches it. The cache is what pays for the dominator analysis added + * below — the whole check got faster, not slower. + */ +const depsCache = new Map() + +function depsOf(file: string): readonly string[] { + const cached = depsCache.get(file) + if (cached) return cached + + let source: string + try { + source = readFileSync(file, 'utf8') + } catch { + depsCache.set(file, NO_DEPS) + return NO_DEPS + } + + const deps: string[] = [] + const seen = new Set() + for (const pattern of [IMPORT_RE, REEXPORT_RE, DYNAMIC_IMPORT_RE, REQUIRE_RE]) { + pattern.lastIndex = 0 + let match = pattern.exec(source) + while (match !== null) { + const resolved = resolveSpecifier(match[1], file) + if (resolved && !seen.has(resolved)) { + seen.add(resolved) + deps.push(resolved) + } + match = pattern.exec(source) + } + } + depsCache.set(file, deps) + return deps +} + interface Walk { + entry: string reachable: Set importedBy: Map } +/** + * Breadth-first so `importedBy` records the *shortest* path to each module — + * a depth-first parent chain reports whatever winding route the stack happened + * to take, which reads as noise in a failure message. + */ function walk(entry: string): Walk { - const reachable = new Set() + const reachable = new Set([entry]) const importedBy = new Map() const queue = [entry] - reachable.add(entry) - - while (queue.length > 0) { - const file = queue.pop() as string - let source: string - try { - source = readFileSync(file, 'utf8') - } catch { - continue - } - for (const pattern of [IMPORT_RE, REEXPORT_RE, DYNAMIC_IMPORT_RE, REQUIRE_RE]) { - pattern.lastIndex = 0 - let match = pattern.exec(source) - while (match !== null) { - const resolved = resolveSpecifier(match[1], file) - if (resolved && !reachable.has(resolved)) { - reachable.add(resolved) - importedBy.set(resolved, file) - queue.push(resolved) - } - match = pattern.exec(source) - } + + for (let head = 0; head < queue.length; head++) { + const file = queue[head] + for (const resolved of depsOf(file)) { + if (reachable.has(resolved)) continue + reachable.add(resolved) + importedBy.set(resolved, file) + queue.push(resolved) } } - return { reachable, importedBy } + return { entry, reachable, importedBy } } /** Walks parent links back to the entry so the offending edge is obvious. */ @@ -153,9 +204,234 @@ function explainChain({ importedBy }: Walk, target: string): string[] { return chain.reverse() } +const BASELINE_PATH = join(SCRIPT_DIR, 'check-tool-registry-boundary.baseline.json') + +/** + * A graph may grow by `max(TOLERANCE_MODULES, TOLERANCE_PERCENT%)` before failing. + * + * Both halves are needed. A pure percentage lets the 2,186-module workflow route + * absorb 40 modules while pinning the 263-module upgrade route to 5, which would + * fail on an ordinary component addition. A pure absolute number is the same + * trade in reverse. The floor is sized so adding a feature's worth of components + * and hooks is free, while every regression this guard has actually seen — the + * `listTables` prefetch at +444, the registry at +4,700 — is far outside it. + */ +const TOLERANCE_MODULES = 25 +const TOLERANCE_PERCENT = 2 + +/** Smallest dominated subtree worth naming as a gateway in the baseline. */ +const GATEWAY_MIN_MODULES = 30 +/** Gateways recorded per entry, largest first. */ +const GATEWAY_LIMIT = 8 + +interface BaselineEntry { + modules: number + /** Module → number of modules reachable *only* through it. See `gatewaysFor`. */ + gateways: Record +} + +interface Baseline { + generatedFrom: string + tolerance: { modules: number; percent: number } + entries: Record +} + +function allowanceFor(baselineModules: number): number { + return Math.max(TOLERANCE_MODULES, Math.ceil((baselineModules * TOLERANCE_PERCENT) / 100)) +} + +interface Dominators { + /** Immediate dominator of each module; the entry maps to itself. */ + idom: Map + /** Size of each module's dominator subtree — its exclusive cost to this entry. */ + weight: Map +} + +/** + * Dominator tree of the entry's import graph (Cooper–Harvey–Kennedy). + * + * The point is the weight: a module's dominator-subtree size is exactly how many + * modules would leave the graph if its incoming edge were cut. That turns "this + * page gained 444 modules" into "this page gained 444 modules through + * `lib/table/index.ts`", which names the import to delete. + * + * Computed lazily — only for entries that regress, plus every entry during + * `--update-baseline`. + */ +function dominators({ entry, reachable }: Walk): Dominators { + const succ = new Map() + for (const file of reachable) { + succ.set( + file, + depsOf(file).filter((dep) => reachable.has(dep)) + ) + } + + // Iterative DFS postorder, then reverse it for the RPO numbering the + // algorithm's `intersect` walks against. + const postorder: string[] = [] + const visited = new Set([entry]) + const stack: Array<{ node: string; next: number }> = [{ node: entry, next: 0 }] + while (stack.length > 0) { + const frame = stack[stack.length - 1] + const children = succ.get(frame.node) as string[] + if (frame.next < children.length) { + const child = children[frame.next++] + if (!visited.has(child)) { + visited.add(child) + stack.push({ node: child, next: 0 }) + } + } else { + postorder.push(frame.node) + stack.pop() + } + } + + const rpo = [...postorder].reverse() + const rpoIndex = new Map() + rpo.forEach((node, index) => rpoIndex.set(node, index)) + + const preds = new Map() + for (const [file, children] of succ) { + if (!rpoIndex.has(file)) continue + for (const child of children) { + if (!rpoIndex.has(child)) continue + const list = preds.get(child) + if (list) list.push(file) + else preds.set(child, [file]) + } + } + + const idom = new Map([[entry, entry]]) + const intersect = (a: string, b: string): string => { + let left = a + let right = b + while (left !== right) { + while ((rpoIndex.get(left) as number) > (rpoIndex.get(right) as number)) + left = idom.get(left) as string + while ((rpoIndex.get(right) as number) > (rpoIndex.get(left) as number)) + right = idom.get(right) as string + } + return left + } + + let changed = true + while (changed) { + changed = false + for (let i = 1; i < rpo.length; i++) { + const node = rpo[i] + let candidate: string | null = null + for (const pred of preds.get(node) ?? []) { + if (!idom.has(pred)) continue + candidate = candidate === null ? pred : intersect(candidate, pred) + } + if (candidate !== null && idom.get(node) !== candidate) { + idom.set(node, candidate) + changed = true + } + } + } + + // A node's dominator parent always has a smaller RPO index, so folding sizes + // from the deepest index upward completes every subtree in one pass. + const weight = new Map() + for (const node of rpo) weight.set(node, 1) + for (let i = rpo.length - 1; i >= 1; i--) { + const node = rpo[i] + const parent = idom.get(node) + if (!parent || parent === node) continue + weight.set(parent, (weight.get(parent) as number) + (weight.get(node) as number)) + } + + return { idom, weight } +} + +/** + * The heaviest gateway modules of an entry, collapsed to one per chain. + * + * In a pass-through chain `a → b → c` every link dominates the same subtree, so + * reporting all three says the same thing three times. `weight[idom] > weight + 1` + * keeps only the topmost link of each chain — the branch point nearest the entry, + * which is the edge a developer can actually delete. + */ +function gatewaysFor(walkResult: Walk, doms: Dominators): Array<[string, number]> { + const gateways: Array<[string, number]> = [] + for (const [node, size] of doms.weight) { + if (node === walkResult.entry || size < GATEWAY_MIN_MODULES) continue + const parent = doms.idom.get(node) + if (!parent || parent === node) continue + if (parent !== walkResult.entry && (doms.weight.get(parent) as number) <= size + 1) continue + gateways.push([relative(ROOT, node), size]) + } + gateways.sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])) + return gateways.slice(0, GATEWAY_LIMIT) +} + +/** Dominator-tree ancestry of a module: the edges every path from the entry crosses. */ +function dominatorChain(walkResult: Walk, doms: Dominators, target: string): string[] { + const chain: string[] = [] + let current = target + for (let guard = 0; guard < 10_000; guard++) { + chain.push(relative(ROOT, current)) + if (current === walkResult.entry) break + const parent = doms.idom.get(current) + if (!parent || parent === current) break + current = parent + } + return chain.reverse() +} + +function loadBaseline(): Baseline | null { + try { + return JSON.parse(readFileSync(BASELINE_PATH, 'utf8')) as Baseline + } catch { + return null + } +} + +const RERECORD_COMMAND = 'bun run scripts/check-tool-registry-boundary.ts --update-baseline' +const REBASELINE_HINT = `If the growth is intentional, re-record it: ${RERECORD_COMMAND}` + +/** + * Reports a regressed entry by naming the gateways that grew, not just the delta. + * + * Baseline gateways are matched by path, so an edge that is new (absent from the + * baseline) and an edge that got heavier are both surfaced, largest growth first. + */ +function reportRegression(entry: string, walkResult: Walk, before: BaselineEntry, after: number) { + const allowance = allowanceFor(before.modules) + console.error( + `\n❌ ${entry} grew to ${after} modules (baseline ${before.modules}, +${after - before.modules}, allowed +${allowance})` + ) + + const doms = dominators(walkResult) + const growth = gatewaysFor(walkResult, doms) + .map(([module, size]) => ({ module, size, delta: size - (before.gateways[module] ?? 0) })) + .filter((row) => row.delta > 0) + .sort((a, b) => b.delta - a.delta) + + if (growth.length === 0) { + console.error( + ' No single import edge accounts for it — the growth is spread across many small modules.' + ) + console.error(` Compare with --verbose to see which entries moved. ${REBASELINE_HINT}`) + return + } + + for (const row of growth.slice(0, 3)) { + console.error(` +${row.delta} modules reach it only via ${row.module} (${row.size} total):`) + const chain = dominatorChain(walkResult, doms, join(ROOT, row.module)) + for (const step of chain) console.error(` ${step}`) + } + console.error(` ${REBASELINE_HINT}`) +} + function main() { const verbose = process.argv.includes('--verbose') + const check = process.argv.includes('--check') + const update = process.argv.includes('--update-baseline') const failures: string[] = [] + let ratchetFailures = 0 const entryRoot = join(APP, ENTRY_ROOT) if (!existsSync(entryRoot)) { @@ -170,9 +446,10 @@ function main() { process.exit(1) } + const walked = new Map() for (const entry of entries) { - const entryPath = join(APP, entry) - const result = walk(entryPath) + const result = walk(join(APP, entry)) + walked.set(entry, result) if (result.reachable.has(FORBIDDEN)) { failures.push(entry) console.error(`\n❌ ${entry} can reach @/tools/registry via:`) @@ -184,6 +461,83 @@ function main() { } } + if (update) { + const baseline: Baseline = { + generatedFrom: `${ENTRY_ROOT} page/layout module graphs`, + tolerance: { modules: TOLERANCE_MODULES, percent: TOLERANCE_PERCENT }, + entries: {}, + } + for (const entry of entries) { + const result = walked.get(entry) as Walk + baseline.entries[entry] = { + modules: result.reachable.size, + gateways: Object.fromEntries(gatewaysFor(result, dominators(result))), + } + } + writeFileSync(BASELINE_PATH, `${JSON.stringify(baseline, null, 2)}\n`) + console.log( + `✓ Baseline written for ${entries.length} entries: ${relative(ROOT, BASELINE_PATH)}` + ) + process.exit(failures.length > 0 ? 1 : 0) + } + + if (check) { + const baseline = loadBaseline() + if (!baseline) { + console.error( + `\n❌ Missing ${relative(ROOT, BASELINE_PATH)}. Refusing to pass without a ratchet — ` + + 'generate it with --update-baseline.' + ) + process.exit(1) + } + + const shrunk: string[] = [] + const unbaselined: string[] = [] + let regressed = 0 + + for (const entry of entries) { + const result = walked.get(entry) as Walk + const before = baseline.entries[entry] + if (!before) { + unbaselined.push(`${entry} (${result.reachable.size} modules)`) + continue + } + const after = result.reachable.size + if (after > before.modules + allowanceFor(before.modules)) { + regressed++ + reportRegression(entry, result, before, after) + } else if (after < before.modules - allowanceFor(before.modules)) { + shrunk.push(`${entry}: ${before.modules} → ${after} (−${before.modules - after})`) + } + } + + const removed = Object.keys(baseline.entries).filter((entry) => !entries.includes(entry)) + + if (shrunk.length > 0) { + console.log(`\nℹ ${shrunk.length} entr(ies) shrank below baseline:`) + for (const line of shrunk) console.log(` ${line}`) + console.log(` Lock the win in, or it can be spent again: ${RERECORD_COMMAND}`) + } + if (unbaselined.length > 0) { + console.log(`\nℹ ${unbaselined.length} new entr(ies) not yet in the baseline (unratcheted):`) + for (const line of unbaselined) console.log(` ${line}`) + console.log(` ${REBASELINE_HINT}`) + } + if (removed.length > 0) { + console.log(`\nℹ ${removed.length} baseline entr(ies) no longer exist: ${removed.join(', ')}`) + } + + if (regressed > 0) { + ratchetFailures = regressed + console.error( + `\n${regressed} route(s) exceeded their module-graph baseline by more than max(${TOLERANCE_MODULES}, ${TOLERANCE_PERCENT}%).` + ) + console.error( + 'Every module in a page graph is parsed and shipped, so this is a real page-weight cost.' + ) + } + } + if (failures.length > 0) { console.error( `\n${failures.length} route(s) reach the executable tool registry, which adds ~4,700 modules to each.` @@ -195,10 +549,13 @@ function main() { '(outputs), or `@/tools/tool-ids` (existence/resolution). Only code that executes a tool' ) console.error('may import `getTool`. See .agents/skills/tool-registry-boundary/SKILL.md.') - process.exit(1) } + if (failures.length > 0 || ratchetFailures > 0) process.exit(1) + console.log(`✓ tool registry stays out of ${entries.length} workspace page/layout graphs`) + if (check) + console.log(`✓ ${entries.length} page/layout graphs within their module-count baseline`) } main()