perf(server): stop calling our own API over HTTP during render, execution, and tool runs (#6660)

* fix(prefetch): parse the file-folder seed through its contract

The audit found this key was the workspaceFilesKeys bug waiting to recur. The
manager's record type and workspaceFileFolderSchema are two independent
declarations that agree today by coincidence; the seed had no parse, so adding a
column to one would have silently cached a shape a client fetch strips — and
three of its fields are z.coerce.date(), the exact divergence that put ISO
strings under the file-list key.

Its sibling is immune because listWorkspaceFilesWithShares parses at the data
layer. This does the same at the seed, and adds the shape-parity assertion the
key never had. Verified falsifiable: removing the parse turns it red. Doing so
also exposed the existing folder test as fixture-thin — a folder with only an
id, which the contract rightly rejects — so it now uses a real row.

Also points the credential block's fetchQuery at the exported staleTime
constant instead of restating 60 * 1000; it was a fifth producer on that key
free to drift from the four that share it.

* fix(queries): stop a table cell edit throwing, and an upload gate failing shut

Two functional bugs found auditing the query layer.

patchCachedRows walked tableKeys.rowsRoot non-exact, but rowsRoot is a prefix:
the find (search results) and write (pending writes) subtrees hang off it with
non-paged shapes, and the updater's old.pages.map threw on them. It runs inside
onMutate, so the whole cell edit rejected before reaching the server — reachable
as soon as a find entry exists, i.e. after the user searches the table once. The
sibling isDefaultOrderRowsQuery already excluded those subtrees and its docstring
claimed they "never match"; that was only true of the sibling. Both now share one
isRowListQueryKey helper so they cannot drift apart again.

useCloudStorageConfigured combined staleTime: Infinity, retry: false, and the
global retryOnMount: false on a workspace-independent key, so one transient
failure left it errored for the tab's lifetime with no way back — navigating or
switching workspace cannot change the key, and the upload path fails closed, so
cloud-backed uploads stayed disabled until a full reload. useVoiceSettings
carries the same three options and already escapes this with retryOnMount: true;
this one now matches.

Note: hooks/queries/workspace-files.test.tsx cannot load in a git worktree
(pre-existing postcss resolution failure), so CI is the first place that file
runs against this change.

* perf(server): memoize the request-scoped workspace and entitlement reads

The workspace row was read ~3x per workspace route and ~5x on settings, and the
same Max-tier entitlement was resolved twice on one render.

Memoization is deliberately partial. getWorkspaceWithOwner accepts a
transaction and forUpdate, and live callers use both, so only the plain
no-options read routes through the memo; a row read inside one caller's
transaction or under a lock it alone holds can never be served to a later
caller. includeArchived is part of the key so the two variants cannot alias.

Three substitutions were considered and rejected as behavior changes, not
optimizations: hostContext.ownerBilling resolves subscriptions differently from
hasWorkspaceTierAccess and exposes no Max tier, so it cannot answer the
Inbox/Sandbox gates; isOrganizationOnEnterprisePlan carries self-host
short-circuits ownerBilling has no equivalent for; and widening
WorkspaceHostContext to carry the full row would push owner and org ids onto
the wire for every viewer to save a server-side read, since that type is a
response contract rather than an internal struct.

* fix(selectors): key CloudWatch lists by search, and stop a caller erasing the credential gate

The CloudWatch log-group and log-stream selectors forwarded `search` into the
request as `prefix` but left it out of the query key, so every keystroke
resolved to the same fresh entry and no refetch fired. Server-side filtering
was dead: a log group outside the first page could not be reached. An audit of
all 69 selector definitions found these two and no others.

useSelectorOptions resolved `args.enabled ?? definition.enabled(...)`, so a
caller supplying its own gate replaced the definition's precondition rather
than narrowing it. useSelectorDisplayName knows nothing about credentials, so a
card holding a saved value with no credential context ran a query that could
only reject. The two are now conjoined. The detail hooks keep the override
deliberately — resolving one known id needs less context than listing, which
their TSDoc already documents.

The list-key fix has a test, proven to fail without it. The `enabled` change
has none: loading use-selector-query pulls the selector registry and emcn CSS,
which cannot resolve in a git worktree.

* fix(queries): give optimistic rows collision-free ids

generateTempId used Date.now(), so two rows created in the same millisecond
shared an id and the first server response overwrote both — leaving one row
duplicated and the other's real id lost until a refetch. Now uses generateId(),
matching what the workflow mutations already do. Reachable by double-clicking
create, or by any scripted or bulk create.

Also documents the contract of fetchOAuthConnections, which reports an unknown
connection state as disconnected. No consumer reads that field today — both
read names and icons, and connection state comes from useWorkspaceCredentials —
so letting the query reject would blank the suggested-action rows and drop the
credential page to raw provider ids. The note is what stops a future consumer
branching on it silently.

* fix(queries): close six stale-data gaps found auditing the query layer

Each was verified against the mutation that changes the data and the keys that
expose it, not taken on report.

- Workspace usage/credits were invalidated nowhere in the app. Six sites already
  refreshed subscriptionKeys after credits moved — post-run, post-wand, limit
  edits, upgrades, top-ups — and none touched workspace usage, so the credits
  chip and the run gate held their page-load values until a reload. Adds one
  shared invalidateWorkspaceUsage and calls it from all six.
- Knowledge-base list doc counts went stale: document upload, delete, and bulk
  delete invalidated only the detail key, though the list carries docCount.
- Plan switches that do not redirect refreshed only the host context, leaving
  subscription and credit state showing the previous plan.
- The copilot tool-event handler invalidated a raw workflowKeys.list, which
  covers only the active scope and skips the selector prefix; it now uses the
  shared invalidateWorkflowLists like the other thirteen call sites.
- scheduleKeys.byId was a strict prefix of scheduleKeys.schedule, so the two
  addressings aliased, and nothing invalidated byId. De-aliased and invalidated.

Not changed: the CSV preview key already folds in the file version and storage
key, so a content update addresses a different cache entry — version-in-key is
the mechanism there, not a missing invalidation.

Tests added for the usage and knowledge fixes, both proven to fail without them.
The other four live in files that cannot load in a git worktree (pre-existing
postcss resolution failure), so CI is where they first run.

* improvement(queries): make the row-list prefix non-collidable, and share the usage refresh

Two corrections from reviewing the previous commits.

patchCachedRows was fixed with a predicate naming the sibling subtrees to skip —
a denylist that rots the moment a fifth subtree is added under rowsRoot. The key
factory already separated row lists under an 'infinite' segment; it just had no
prefix accessor, so every caller reached for the parent and subtracted. Adding
infiniteRowsRoot lets the walk be an allowlist by construction and deletes the
predicate, the helper, and both docblocks explaining the subtraction.

The searched-rows view is consequently no longer patched by a cell edit and is
left to its own refetch — it holds a flat result, not pages. That is recorded on
the function rather than left to be rediscovered.

The delayed usage refresh was written out three times across two files, a
duplication the previous commit enlarged rather than introduced. It is now one
scheduleUsageRefresh beside the keys it invalidates, which also gives the bare
1000ms a name and one place to change it.

* improvement(prefetch): budget the workspace file seed and narrow its columns

* improvement(api): resolve credentials and checkpoint reverts in-process

* improvement(executor): run router and evaluator provider calls in-process

* improvement(queries): fix a second row-cache collision, and make the memoized workspace read actually dedupe

* chore(test): type the evaluator provider-request helper instead of using any

* improvement(perf): restore the parallel file read, and close the gaps a diff audit surfaced

* improvement(perf): drop a duplicate authorization, parallelize the credential reads, and trim the comments

Author: waleedlatif1

apps/sim/app/api/auth/oauth/token/route.test.ts

@@ -24,6 +24,7 @@ vi.mock('@/lib/oauth/credential-service', () => ({
 
 vi.mock('@/lib/auth/credential-access', () => ({
   authorizeCredentialUse: mockAuthorizeCredentialUse,
+  authorizeCredentialUseForAuth: mockAuthorizeCredentialUse,
 }))
 
 import { TokenServiceAccountValidationError } from '@/lib/credentials/token-service-accounts/errors'

apps/sim/app/api/auth/oauth/token/route.ts

@@ -11,17 +11,9 @@ import { authorizeCredentialUse } from '@/lib/auth/credential-access'
 import { AuthType, checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
 import { generateRequestId } from '@/lib/core/utils/request'
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
-import { TokenServiceAccountValidationError } from '@/lib/credentials/token-service-accounts/errors'
-import {
-  getCredential,
-  getOAuthToken,
-  refreshTokenIfNeeded,
-  resolveOAuthAccountId,
-  resolveServiceAccountToken,
-} from '@/lib/oauth/credential-service'
-import { extractSalesforceInstanceUrl, isSalesforceOAuthProviderId } from '@/lib/oauth/salesforce'
+import { getCredential, getOAuthToken } from '@/lib/oauth/credential-service'
+import { completeOAuthCredentialToken, resolveCredentialToken } from '@/lib/oauth/token-resolution'
 import { captureServerEvent } from '@/lib/posthog/server'
-import { extractZohoDeskBaseFromScope } from '@/tools/zoho_desk/host-allowlist'
 
 export const dynamic = 'force-dynamic'
 
@@ -123,194 +115,25 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
       }
     }
 
-    if (!credentialId) {
-      return NextResponse.json({ error: 'Credential ID is required' }, { status: 400 })
-    }
-
-    const resolved = await resolveOAuthAccountId(credentialId)
-    if (resolved?.credentialType === 'service_account' && resolved.credentialId) {
-      const authz = await authorizeCredentialUse(request, {
-        credentialId,
-        workflowId: workflowId ?? undefined,
-        requireWorkflowIdForInternal: false,
-        callerUserId,
-      })
-      if (!authz.ok) {
-        return NextResponse.json({ error: authz.error || 'Unauthorized' }, { status: 403 })
-      }
-
-      const saActorId = authz.requesterUserId
-      const saWorkspaceId = resolved.workspaceId ?? authz.workspaceId ?? null
-      const emitServiceAccountAccess = () => {
-        if (!saActorId) return
-        recordAudit({
-          workspaceId: saWorkspaceId,
-          actorId: saActorId,
-          action: AuditAction.CREDENTIAL_ACCESSED,
-          resourceType: AuditResourceType.CREDENTIAL,
-          resourceId: resolved.credentialId ?? credentialId,
-          description: `Accessed service account credential for provider ${resolved.providerId ?? 'unknown'}`,
-          metadata: {
-            provider: resolved.providerId,
-            credentialType: 'service_account',
-          },
-          request,
-        })
-        captureServerEvent(
-          saActorId,
-          'credential_used',
-          {
-            credential_type: 'service_account',
-            provider_id: resolved.providerId ?? 'unknown',
-            ...(saWorkspaceId ? { workspace_id: saWorkspaceId } : {}),
-          },
-          saWorkspaceId ? { groups: { workspace: saWorkspaceId } } : undefined
-        )
-      }
-
-      try {
-        const result = await resolveServiceAccountToken(
-          resolved.credentialId,
-          resolved.providerId,
-          scopes ?? [],
-          impersonateEmail
-        )
-        emitServiceAccountAccess()
-        return NextResponse.json(
-          {
-            accessToken: result.accessToken,
-            cloudId: result.cloudId,
-            domain: result.domain,
-            instanceUrl: result.instanceUrl,
-            apiDomain: result.apiDomain,
-            authStyle: result.authStyle,
-          },
-          { status: 200 }
-        )
-      } catch (error) {
-        logger.error(`[${requestId}] Service account token error:`, error)
-        if (error instanceof TokenServiceAccountValidationError) {
-          // Classified provider outages are infra failures, not bad credentials.
-          if (error.code === 'provider_unavailable') {
-            return NextResponse.json(
-              { error: 'Credential provider is temporarily unavailable' },
-              { status: 502 }
-            )
-          }
-          // A stored host that no longer resolves is a configuration failure —
-          // surface the code so runtime consumers can say "check the host"
-          // instead of a generic auth error.
-          if (error.code === 'site_not_found') {
-            return NextResponse.json(
-              {
-                code: error.code,
-                error: 'Credential host not found — reconnect the credential with a valid host',
-              },
-              { status: 400 }
-            )
-          }
-          // A revoked/rotated-away or misconfigured stored secret — surface the
-          // code so runtime consumers can prompt to reconnect the credential
-          // rather than showing a generic auth failure.
-          if (error.code === 'invalid_credentials') {
-            return NextResponse.json(
-              {
-                code: error.code,
-                error: 'Credential rejected by the provider — reconnect the credential',
-              },
-              { status: 401 }
-            )
-          }
-        }
-        return NextResponse.json({ error: 'Failed to get service account token' }, { status: 401 })
-      }
-    }
-
-    const authz = await authorizeCredentialUse(request, {
+    const auth = await checkSessionOrInternalAuth(request, { requireWorkflowId: false })
+    const result = await resolveCredentialToken(auth, {
+      requestId,
       credentialId,
       workflowId: workflowId ?? undefined,
-      requireWorkflowIdForInternal: false,
+      scopes,
+      impersonateEmail,
       callerUserId,
+      auditRequest: request,
     })
-    if (!authz.ok || !authz.credentialOwnerUserId) {
-      return NextResponse.json({ error: authz.error || 'Unauthorized' }, { status: 403 })
-    }
-
-    const resolvedCredentialId = authz.resolvedCredentialId || credentialId
-    const credential = await getCredential(
-      requestId,
-      resolvedCredentialId,
-      authz.credentialOwnerUserId
-    )
-
-    if (!credential) {
-      return NextResponse.json({ error: 'Credential not found' }, { status: 404 })
-    }
-
-    const oauthActorId = authz.requesterUserId
-    const oauthWorkspaceId = authz.workspaceId ?? null
-
-    try {
-      const { accessToken } = await refreshTokenIfNeeded(
-        requestId,
-        credential,
-        resolvedCredentialId
-      )
-
-      if (oauthActorId) {
-        recordAudit({
-          workspaceId: oauthWorkspaceId,
-          actorId: oauthActorId,
-          action: AuditAction.CREDENTIAL_ACCESSED,
-          resourceType: AuditResourceType.CREDENTIAL,
-          resourceId: resolvedCredentialId,
-          description: `Accessed OAuth credential for provider ${credential.providerId}`,
-          metadata: {
-            provider: credential.providerId,
-            credentialType: 'oauth',
-          },
-          request,
-        })
-        captureServerEvent(
-          oauthActorId,
-          'credential_used',
-          {
-            credential_type: 'oauth',
-            provider_id: credential.providerId,
-            ...(oauthWorkspaceId ? { workspace_id: oauthWorkspaceId } : {}),
-          },
-          oauthWorkspaceId ? { groups: { workspace: oauthWorkspaceId } } : undefined
-        )
-      }
-
-      const instanceUrl = isSalesforceOAuthProviderId(credential.providerId)
-        ? extractSalesforceInstanceUrl(credential.scope)
-        : undefined
-
-      // Zoho Desk persists its data-center-specific REST base URL in the scope
-      // string (derived from the token response api_domain) so callers never
-      // assume a host. Surface it as apiDomain for tool param injection.
-      let apiDomain: string | undefined
-      if (credential.providerId === 'zoho-desk' && credential.scope) {
-        // Use the shared extractor, not a local regex: it also enforces https +
-        // the Zoho apex allowlist. This value is injected into EVERY tool call,
-        // so an unvalidated host here would receive the OAuth token.
-        apiDomain = extractZohoDeskBaseFromScope(credential.scope)
-      }
 
+    if (!result.ok) {
       return NextResponse.json(
-        {
-          accessToken,
-          idToken: credential.idToken || undefined,
-          ...(instanceUrl && { instanceUrl }),
-          ...(apiDomain && { apiDomain }),
-        },
-        { status: 200 }
+        { ...(result.code ? { code: result.code } : {}), error: result.error },
+        { status: result.status }
       )
-    } catch (error) {
-      logger.error(`[${requestId}] Failed to refresh access token:`, error)
-      return NextResponse.json({ error: 'Failed to refresh access token' }, { status: 401 })
     }
+
+    return NextResponse.json(result.token, { status: 200 })
   } catch (error) {
     logger.error(`[${requestId}] Error getting access token`, error)
     return NextResponse.json({ error: 'Internal server error' }, { status: 500 })
@@ -366,70 +189,20 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
       return NextResponse.json({ error: 'No access token available' }, { status: 400 })
     }
 
-    const actorId = authz.requesterUserId
-    const workspaceId = authz.workspaceId ?? null
-
-    try {
-      const { accessToken } = await refreshTokenIfNeeded(
-        requestId,
-        credential,
-        resolvedCredentialId
-      )
-
-      if (actorId) {
-        recordAudit({
-          workspaceId,
-          actorId,
-          action: AuditAction.CREDENTIAL_ACCESSED,
-          resourceType: AuditResourceType.CREDENTIAL,
-          resourceId: resolvedCredentialId,
-          description: `Accessed OAuth credential for provider ${credential.providerId}`,
-          metadata: {
-            provider: credential.providerId,
-            credentialType: 'oauth',
-          },
-          request,
-        })
-        captureServerEvent(
-          actorId,
-          'credential_used',
-          {
-            credential_type: 'oauth',
-            provider_id: credential.providerId,
-            ...(workspaceId ? { workspace_id: workspaceId } : {}),
-          },
-          workspaceId ? { groups: { workspace: workspaceId } } : undefined
-        )
-      }
-
-      const instanceUrl = isSalesforceOAuthProviderId(credential.providerId)
-        ? extractSalesforceInstanceUrl(credential.scope)
-        : undefined
-
-      // Zoho Desk persists its data-center-specific REST base URL in the scope
-      // string (derived from the token response api_domain) so callers never
-      // assume a host. Surface it as apiDomain for tool param injection.
-      let apiDomain: string | undefined
-      if (credential.providerId === 'zoho-desk' && credential.scope) {
-        // Use the shared extractor, not a local regex: it also enforces https +
-        // the Zoho apex allowlist. This value is injected into EVERY tool call,
-        // so an unvalidated host here would receive the OAuth token.
-        apiDomain = extractZohoDeskBaseFromScope(credential.scope)
-      }
+    const result = await completeOAuthCredentialToken({
+      requestId,
+      credential,
+      resolvedCredentialId,
+      actorId: authz.requesterUserId,
+      workspaceId: authz.workspaceId ?? null,
+      auditRequest: request,
+    })
 
-      return NextResponse.json(
-        {
-          accessToken,
-          idToken: credential.idToken || undefined,
-          ...(instanceUrl && { instanceUrl }),
-          ...(apiDomain && { apiDomain }),
-        },
-        { status: 200 }
-      )
-    } catch (error) {
-      logger.error(`[${requestId}] Failed to refresh access token:`, error)
-      return NextResponse.json({ error: 'Failed to refresh access token' }, { status: 401 })
+    if (!result.ok) {
+      return NextResponse.json({ error: result.error }, { status: result.status })
     }
+
+    return NextResponse.json(result.token, { status: 200 })
   } catch (error) {
     logger.error(`[${requestId}] Error fetching access token`, error)
     return NextResponse.json({ error: 'Internal server error' }, { status: 500 })