perf(prefetch): stop calling our own API over the wire during server render by waleedlatif1 · Pull Request #6657 · simstudioai/sim

waleedlatif1 · 2026-08-13T04:28:01Z

Summary

Eight server-render prefetches called our own API routes over HTTP. When INTERNAL_API_BASE_URL is unset, getInternalApiBaseUrl() falls back to the public base URL, so each became RSC → public HTTPS → whatever fronts the app → back in, awaited inside the render. Even with an internal base URL set it is still a server-to-server request plus a duplicate authentication for data the process can read directly.

All eight are gone and prefetch-internal-fetch.ts is deleted.

Read	How
`/home` workflow folders + file list	Deleted — the layout already seeds both keys
tables / knowledge folders	Shared `prefetchResourceFolders`
workspace files	`listWorkspaceFilesWithShares`
knowledge bases	The route's own use case, principal, and presenter
tables list	Shared `toTableListItem` projection
pinned items	New `listPinnedItemsForUser`
workspace members	`getWorkspaceMemberProfiles`

Bugs fixed along the way

/home fetched the workflow folder list twice per request. getQueryClient() builds a new client per server call, so the layout's copy and the page's never deduped.
/home seeded the wrong type. It cached raw route JSON under workspaceFilesKeys.list while files/prefetch.ts seeded the same key from a direct read; the contract declares those dates z.coerce.date(), so a file record's type depended on which page you landed on.
/settings shipped two prefetches without awaiting them. Only a settled query is dehydrated, so they were dropped from the payload and the panel waterfalled anyway.
The viewer profile was prefetched twice — once by the layout, once by settings.
prefetchSubscriptionData was dead code carrying an unannotated raw fetch.

Why these aren't boundary bypasses

Knowledge runs listInternalKnowledgeBases — the route's own use case — with a principal from the same internalSessionAuth policy the route declares, then the same presenter and contract parse. Tables and pinned items extract the route's own logic into lib/table/wire.ts and lib/pinned-items/queries.ts, which the routes now call too, so there is one source of truth rather than a copy that can drift. Every converted read proves the viewer first and caches nothing on failure, so an unauthorized viewer's client fetch still reaches the route for the real 403 — verified by unfolding both paths to checkWorkspaceAccess.

React Query mechanism

retry is now scoped to the browser: query-core defaults it to 0 on the server, and stating one value for both opted awaited prefetches into a retry backoff of document latency. The gcTime default is dropped — 5 minutes is already the browser default, and setting it explicitly overrode the server's Infinity, leaving a live timer and payload per request. The pending-query dehydration opt-in is removed now that nothing streams.

Module-graph guard

check:tool-registry-boundary now also ratchets per-page module counts against a committed baseline, attributing a regression to the import that caused it via a dominator tree. This branch hit a +444 module regression when a prefetch imported listTables from a barrel that reaches the executor; the guard caught it. Its import regex also missed bare side-effect imports, so import '@/tools/registry' could have slipped past entirely — fixed.

Cutting that edge properly (stripGroupDeps and pendingDeleteMask moved to leaf modules) took the Tables page from 2,186 → 1,767 modules while keeping the direct read.

Prefetch guidance added to .claude/rules/sim-queries.md.

Also in this PR

Removes 546 section separator comments (// ═════ Foo ═════) across 48 files. CLAUDE.md already rules them out; they decorate rather than explain, and they drift — one in workflow-columns.ts still labelled a section whose function had moved away. Pure deletion, no source line touched, and lines inside template literals were skipped so no generated string changed.

Type of Change

Bug fix

Testing

prefetch.test.ts 21/21 (from 12), covering every converted read plus a viewer-cannot-be-proved case for each. type-check, biome, lint:check, and check:audits all pass.

Not verified locally: the lib/table suites cannot load in a git worktree — a pre-existing postcss/tailwind resolution failure that also hits files this PR never touches. They exercise pendingDeleteMask, which this PR moves, so please confirm Lint and Test is green before merging. Nothing here is browser-verified.

Checklist

Code follows project style guidelines
Self-reviewed my changes
Tests added/updated and passing
No new warnings introduced
I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

vercel · 2026-08-13T04:28:07Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview	Aug 13, 2026 7:14am

cursor · 2026-08-13T04:28:13Z

PR Summary

Medium Risk
Touches server render data paths and React Query hydration for multiple workspace routes; incorrect wire shapes or auth gaps could cause cache mismatches or over-fetching, but tests and viewer guards were added and routes share the same projections.

Overview
Server prefetches no longer call internal /api routes over HTTP. Workspace list pages (tables, knowledge, files, chrome) now read through the same data-layer functions and wire projections their routes use—listTables + toTableListItem, listPinnedItemsForUser, listInternalKnowledgeBases with session auth, shared folder/member helpers—so hydrated React Query keys match client fetches without a round trip or duplicate auth. prefetch-internal-fetch.ts is removed.

Routing and hydration fixes: Home drops duplicate folder/file prefetches (layout already seeds those keys) and no longer wraps HydrationBoundary for lists only the sidebar hydrates. Settings awaits general-settings prefetch and stops re-seeding profile/subscription. Pages pass session?.user?.id into prefetches; unauthenticated viewers cache nothing.

Shared modules: normalizeColumn moves to lib/table/wire.ts (routes updated); pinned-items listing is centralized in lib/pinned-items/queries.ts for GET and prefetch.

Query client defaults: Server retry stays 0; browser keeps retry: 1. Default gcTime and pending-query dehydration are removed so per-request server clients don’t inherit browser GC or retry behavior.

Guardrails: check:tool-registry-boundary ratchets per-route module counts against a baseline; docs add server-prefetch rules in sim-queries.md. Tests in prefetch.test.ts expand to cover wire shapes and viewer checks.

Cleanup: Hundreds of decorative // ═══ section comments removed across unrelated files (blocks, pptx-renderer, copilot, etc.)—no logic change there.

^{Reviewed by Cursor Bugbot for commit d258fc3. Configure here.}

greptile-apps · 2026-08-13T04:30:40Z

Greptile Summary

The PR replaces server-render self-HTTP prefetches with authorized in-process reads while preserving React Query keys and wire projections. It also extracts lightweight table and pinned-item helpers, adjusts server query defaults, and adds a module-count regression guard.

Removes internal API round trips from workspace prefetch paths.
Shares route projections and direct-read helpers with server prefetches.
Avoids duplicate or unawaited prefetches and restores server-specific React Query defaults.
Extracts table leaf modules to reduce route module graphs.
Extends the tool-registry boundary audit with module-count baselines.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
apps/sim/app/workspace/[workspaceId]/prefetch.ts	Reworks workspace sidebar prefetching around an already-authorized host context and direct contract-shaped reads.
apps/sim/app/workspace/[workspaceId]/knowledge/prefetch.ts	Uses the route’s authentication policy, application use case, presenter, and response contract for in-process knowledge-base prefetching.
apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts	Prefetches active tables directly after proving workspace access and applies the shared route projection.
apps/sim/app/workspace/[workspaceId]/files/prefetch.ts	Seeds the workspace-files cache from a contract-parsed direct read after viewer authorization.
apps/sim/lib/table/wire.ts	Centralizes lightweight table wire normalization and list projection without importing the heavier table barrel.
apps/sim/lib/pinned-items/queries.ts	Extracts the pinned-item list query so the API route and server prefetch share filtering and serialization behavior.
apps/sim/app/_shell/providers/get-query-client.ts	Restores environment-specific React Query retry and garbage-collection behavior and removes pending-query dehydration.
scripts/check-tool-registry-boundary.ts	Adds route module-count regression checks and improves import-edge coverage for the boundary audit.

Sequence Diagram

sequenceDiagram
  participant RSC as Server-rendered page
  participant Auth as Viewer authorization
  participant Read as Use case / data read
  participant QC as React Query cache
  participant UI as Hydrated client
  RSC->>Auth: Prove workspace viewer
  alt Viewer authorized
    Auth-->>RSC: Authorized context
    RSC->>Read: Read data in process
    Read-->>RSC: Contract-compatible result
    RSC->>QC: Seed client query key
    QC-->>UI: Dehydrate and hydrate
  else Viewer not proven
    Auth-->>RSC: No context
    RSC-->>QC: Do not seed query
    UI->>UI: Client query reaches API route
  end

_{Reviews (5): Last reviewed commit: "chore: remove section separator comments" | Re-trigger Greptile}

waleedlatif1 · 2026-08-13T04:40:04Z

@greptile

waleedlatif1 · 2026-08-13T04:40:09Z

@cursor review

waleedlatif1 · 2026-08-13T06:10:53Z

@greptile

waleedlatif1 · 2026-08-13T06:10:55Z

@cursor review

…er the wire Four server-render prefetches went out over HTTP to our own routes. With INTERNAL_API_BASE_URL unset in prod, getInternalApiBaseUrl() falls back to the public base URL, so each was RSC -> public HTTPS -> load balancer -> back into the app, awaited inside the render with a second round of auth. - /home fetched the workflow folder list that the workspace layout had already fetched, under the identical query key. Since getQueryClient() builds a new client per call on the server, the two never deduped: same data, twice a request, once directly and once over the wire. Dropped; the layout's entry already hydrates it. - /home cached raw route JSON under workspaceFilesKeys.list, while files/prefetch.ts seeds that same key from listWorkspaceFilesWithShares. The contract declares the date fields z.coerce.date(), so consumers hold Dates — a file record's type depended on which page the viewer landed on. Now reads the same function files/prefetch.ts does. - tables and knowledge folder reads now call listFoldersForWorkspace, matching the sidebar prefetch. These reads carry no authorization of their own, so each surface proves the viewer through getWorkspaceHostContextForViewer first and caches nothing when it fails, leaving the client fetch to reach the route for the real 403. Both it and getSession are cache()d and already resolved by the layout, so the proof costs no extra queries. Left on the wire, deliberately: the tables and knowledge lists, whose cached shape is the serialized wire shape, and pinned items and members, which have no exported data-layer function.

Passing an empty-string userId ran a real permission query that could only return null. Take an optional userId instead and skip straight to the unauthorized path, matching how the home prefetch is called.

…egacy helper Converts the last four server-render prefetches that called our own API over HTTP, and deletes prefetch-internal-fetch.ts now that nothing imports it. - knowledge bases: runs the route's own listInternalKnowledgeBases use case with a principal from the same internalSessionAuth policy the route declares, then projects through the same presenter and contract. Not a bypass of the application boundary — the same path, called in-process. - tables: extracts the route's list projection into lib/table/wire.ts as toTableListItem, which the route and the prefetch now both call. This matters because listTablesContract's response schema is a passthrough z.custom, so a client fetch caches the route's JSON verbatim. Seeding listTables() directly would have put Date objects and the server-only metadata field under a key the hook never sees them on. - pinned items: extracts the route's inline query into lib/pinned-items/queries.ts as listPinnedItemsForUser, which the route now calls too. - workspace members: getWorkspaceMemberProfiles already existed; the prefetch calls it directly. normalizeColumn moves from app/api/table/utils.ts to lib/table/wire.ts with ten importers repointed. That also removes a pre-existing lib/* -> app/api/* boundary violation in lib/table/import-runner.ts. No response shape changes: the v1/v2 edits are import-path moves only. Every converted read proves the viewer first and caches nothing when that fails, so an unauthorized viewer's client fetch still reaches the route for the real 403. Authorization equivalence was checked by unfolding both paths to checkWorkspaceAccess rather than assumed.

cursor

✅ Bugbot reviewed your changes and found no new issues!

1 issue from previous review remains unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 9596043. Configure here.}

…ify the call shape - Extract prefetchResourceFolders. The same eight-line folder prefetch was written three times, varying only by resourceType, with the key, stale time and mapper kept in sync by hand. - Adopting it removes the conditional spread from the tables and knowledge prefetches. Tables can now early-return, matching prefetchFilesBrowser: prefetchResourceListChrome already self-guards on the same cached host context, so a null context meant the function did nothing either way. - Take userId as string | undefined everywhere and guard inside, so every prefetch module has one calling convention rather than two. - Export toWireTimestamp and use it for the create-table response's own copy of the same idiom, and drop a cast that the extraction made dead: the parameter is already TableDefinition, whose schema is TableSchema. - Read params and the session concurrently on the tables and knowledge pages, matching the files page, and drop TSDoc that restated each prefetch's own.

… edge Reading listTables from a page prefetch put the executable tool registry into the Tables page server graph — ~4,700 modules, which check:tool-registry-boundary rejects. lib/table/service reaches workflow-columns by several independent paths (directly, and through jobs/service and rows/service), so severing one edge is not enough; untangling that belongs in its own change. - The tables list goes back through GET /api/table, with the reason recorded so the next person does not repeat the attempt. Folders and chrome on that page stay on the data layer. - stripGroupDeps moves to its own leaf module. It is a pure projection over a WorkflowGroup, but living beside the group runtime meant every importer of lib/table/service paid for the executor to get it. Net effect on the Tables page graph: 2,186 modules to 1,742.

…et page graphs Answers the question the previous commit left open: the tables list did not have to stay on HTTP. lib/table/service reached the executor through jobs/service -> rows/service -> workflow-columns, for one symbol. pendingDeleteMask is a delete-visibility SQL clause with no executor involvement, so it moves to its own leaf and that chain is cut. The tables prefetch now reads the data layer like every other one, and prefetch-internal-fetch.ts is deleted: nothing in the app calls its own API over HTTP during a server render any more. stripGroupDeps likewise moves to a leaf rather than being re-exported through workflow-columns, so its importers no longer pull the executor to get a pure projection. React Query mechanism fixes, all found by audit: - settings/[section] fired two prefetches without awaiting them. Only a settled query is dehydrated, so those were shipped mid-flight; a rejection hydrated into an error state retryOnMount: false never retries, leaving the panel broken for the session. Awaited now, and the pending-dehydration opt-in is removed since nothing streams. - The viewer profile was prefetched by both the layout and the settings page. Separate server QueryClients mean that was a real second read per request. - prefetchSubscriptionData was dead, and hand-rolled an unannotated raw fetch. - retry is scoped to the browser. Query core defaults it to 0 on the server; stating one value for both opted awaited prefetches into a retry backoff. The gcTime default is dropped entirely — 5 minutes is already the browser default, and setting it explicitly overrode the server's Infinity, leaving a live timer and payload per request. check:tool-registry-boundary now also ratchets per-page module counts against a committed baseline, attributing a regression to the import that caused it via a dominator tree. It caught a +444 regression in this branch by hand; it would have caught it in CI. Its import regex also missed bare side-effect imports, so `import '@/tools/registry'` could have slipped past it entirely. Prefetch guidance added to .claude/rules/sim-queries.md.

…tionale Audit findings from the migration. - pending-delete-mask imported its schema tables from @sim/db rather than @sim/db/schema, which the module it came from was careful to split. The global test mocks are bound per-entrypoint and only the schema mock exports tables, so every suite that reaches pendingDeleteMask would have failed on a missing mock export. Restored to the original convention, and the same split applied to the new pinned-items queries module before it grows a test. - The settings prefetch and page justified awaiting with a mechanism this branch removed — pending queries being shipped with their promise. Only a settled query is dehydrated now, so an unawaited prefetch is dropped from the payload entirely. Same conclusion, correct reason, and no longer contradicting the rule this branch added. - Removed the doc block left orphaned above validateSchema when stripGroupDeps moved out of workflow-columns. Skill projections regenerated after trimming the boundary skill.

waleedlatif1 · 2026-08-13T07:01:35Z

@greptile

waleedlatif1 · 2026-08-13T07:01:36Z

@cursor review

Separators like these are non-TSDoc decoration that CLAUDE.md already rules out. This is the only one in a file this branch touches; the rest of the repo is swept separately.

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 7199af8. Configure here.}

CLAUDE.md already rules these out ("No ==== separators. No non-TSDoc comments"), but 546 of them had accumulated across 48 files. They decorate rather than explain, and they drift: a separator says "Validation" while the code beneath it moved elsewhere, as one in workflow-columns already had. Pure deletion — no source line was touched, and lines inside template literals were skipped so nothing in a generated string changed.

waleedlatif1 · 2026-08-13T07:07:59Z

@greptile

waleedlatif1 · 2026-08-13T07:08:01Z

@cursor review

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d258fc3. Configure here.}

vercel Bot temporarily deployed to Preview August 13, 2026 04:30 Inactive

greptile-apps Bot reviewed Aug 13, 2026

View reviewed changes

Comment thread apps/sim/app/workspace/[workspaceId]/home/prefetch.ts Outdated

cursor Bot reviewed Aug 13, 2026

View reviewed changes

Comment thread apps/sim/app/workspace/[workspaceId]/lib/prefetch.test.ts Outdated

vercel Bot temporarily deployed to Preview August 13, 2026 06:10 Inactive

waleedlatif1 changed the title ~~perf(prefetch): read the data layer instead of calling our own API over the wire~~ perf(prefetch): stop calling our own API over the wire during server render Aug 13, 2026

waleedlatif1 added 3 commits August 12, 2026 23:13

improvement(prefetch): skip the viewer proof when there is no session

2cde1b2

Passing an empty-string userId ran a real permission query that could only return null. Take an optional userId instead and skip straight to the unauthorized path, matching how the home prefetch is called.

cursor Bot reviewed Aug 13, 2026

View reviewed changes

waleedlatif1 added 3 commits August 12, 2026 23:22

waleedlatif1 force-pushed the perf/prefetch-direct-reads branch from 9596043 to ea4577e Compare August 13, 2026 06:48

waleedlatif1 requested a review from a team as a code owner August 13, 2026 06:48

vercel Bot deployed to Preview August 13, 2026 06:54 View deployment

chore(table): drop a section separator comment

5b25865

Separators like these are non-TSDoc decoration that CLAUDE.md already rules out. This is the only one in a file this branch touches; the rest of the repo is swept separately.

vercel Bot temporarily deployed to Preview August 13, 2026 07:04 Inactive

cursor Bot reviewed Aug 13, 2026

View reviewed changes

waleedlatif1 merged commit c49751b into staging Aug 13, 2026
29 of 31 checks passed

waleedlatif1 deleted the perf/prefetch-direct-reads branch August 13, 2026 07:12

vercel Bot deployed to Preview August 13, 2026 07:14 View deployment

This was referenced Aug 13, 2026

v0.8.1: grok 4.6, v2 endpoints extension, workflow UI improvements #6646

Open

fix(queries): close correctness and stale-data defects found auditing the query layer #6659

Merged

Uh oh!

Conversation

waleedlatif1 commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Bugs fixed along the way

Why these aren't boundary bypasses

React Query mechanism

Module-graph guard

Also in this PR

Type of Change

Testing

Checklist

Uh oh!

vercel Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

greptile-apps Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

Uh oh!

waleedlatif1 commented Aug 13, 2026

Uh oh!

waleedlatif1 commented Aug 13, 2026

Uh oh!

Uh oh!

waleedlatif1 commented Aug 13, 2026

Uh oh!

waleedlatif1 commented Aug 13, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

waleedlatif1 commented Aug 13, 2026

Uh oh!

waleedlatif1 commented Aug 13, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

waleedlatif1 commented Aug 13, 2026

Uh oh!

waleedlatif1 commented Aug 13, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

waleedlatif1 commented Aug 13, 2026 •

edited

Loading

vercel Bot commented Aug 13, 2026 •

edited

Loading

cursor Bot commented Aug 13, 2026 •

edited

Loading

greptile-apps Bot commented Aug 13, 2026 •

edited

Loading