fix: treat persistent external-ingest search 404 as empty result (fixes #330265)

[vs-code-engineering]

Summary

Copilot Chat's workspace chunk search (external ingest / embeddings code search) throws an unhandled POST external-embeddings-code-search failed with status 404 when the server-side fileset index is not ready. ExternalIngestIndex.search() retries a 404 once after a 2s delay, but if the index is still not ready the retry rethrows, and the error propagates uncaught out of searchLocalDiff → telemetry. This is a transient, expected state (large workspaces / first index) but currently surfaces as a hard error, polluting error telemetry (win32, extension 0.60.0).

Fixes #330265
Recommended reviewer: @mjbvz

Culprit Commit

Field	Value
Commit	5e1ee93d
Author	@mjbvz
PR	#(not identified)
Message	Added the single-retry 404 workaround in ExternalIngestIndex.search()
Why	The retry block handles a 404 by waiting 2s and calling searchFilesets again, but on a persistent 404 (index still not ready) the second call rethrows, leaving no handler for the expected transient state.

Code Flow

sequenceDiagram
    participant Search as CodeSearchChunkSearch.searchLocalDiff
    participant Index as ExternalIngestIndex.search
    participant Client as ExternalIngestClient.searchFilesets
    participant API as GitHub external embeddings API

    Search->>Index: search(sizing, query, token)
    Index->>Client: searchFilesets(...)
    Client->>API: POST /external/embeddings/code/search
    API-->>Client: 404 (index not ready)
    Note over Index: ⚠️ Root cause:<br/>retry once after 2s
    Index->>Client: searchFilesets(...) retry
    Client->>API: POST /external/embeddings/code/search
    API-->>Client: 404 (still not ready)
    Note over Client: 💥 throws ExternalIngestRequestError<br/>404 -> unhandled error telemetry

Loading

Affected Files

File	Role	Evidence
extensions/copilot/src/platform/workspaceChunkSearch/node/codeSearch/externalIngestClient.ts	crash site	L173: throw new ExternalIngestRequestError(...failed with status ${response.status})
extensions/copilot/src/platform/workspaceChunkSearch/node/codeSearch/externalIngestIndex.ts	root cause (fix site)	L485-L489: single 404 retry rethrows on persistent 404
extensions/copilot/src/platform/workspaceChunkSearch/node/codeSearch/codeSearchChunkSearch.ts	propagation	L783: searchLocalDiff awaits search() and lets the error escape

Repro Steps

1. Open a large workspace with Copilot Chat where remote workspace indexing (external ingest) has just been triggered but the server-side fileset is not yet ready.
2. Issue a workspace search / #codebase query that runs searchLocalDiff.
3. The embeddings search returns 404; after the single 2s retry the fileset is still not ready and returns 404 again.
4. The ExternalIngestRequestError is rethrown and reported as an unhandled error.

How the Fix Works

Chosen approach (externalIngestIndex.ts → search()): wrap the second (retry) searchFilesets call so that a persistent 404 is recognized as the same "index not ready" transient condition the surrounding code already documents, and return undefined instead of rethrowing. search() already maps an empty/undefined result to [] (L496), so callers get a valid empty result while code-search results from the parallel remote path are unaffected. A logService.warn is retained so the condition remains observable — nothing is silently swallowed and no logService.error/telemetry-feeding throw is removed. Any non-404 error still rethrows unchanged.

This fixes the condition at the layer that owns the 404 retry semantics (the data consumer of an external GitHub API), rather than guarding at the crash site in externalIngestClient.ts. The producer of the 404 is the remote GitHub embeddings service — a cross-process boundary the extension cannot fix — so handling the transient status where the retry policy already lives is the correct location.

Alternatives considered: removing the throw in makeRequest (rejected — it would suppress 404s for every caller including deletes/index updates and remove the telemetry signal); widening only at the top-level search() catch (rejected — it would also swallow genuine non-transient 404s and lose the "after retry" specificity).

Recommended Owner

@mjbvz — author of the external ingest 404 retry logic and owner of the workspace chunk search / external ingest area (externalIngestIndex.ts, codeSearchChunkSearch.ts telemetry owner).

Generated by errors-fix · opus48 · 441.2 AIC · ⌖ 11.3 AIC · ⊞ 18.6K · ◷

---

Note

This was originally intended as a pull request, but PR creation failed. The changes have been pushed to the branch fix/external-ingest-404-index-not-ready-5e85eda2ea998478.

Original error: ERR_API: [2026-08-11T15:15:42.673Z] create pull request in microsoft/vscode failed (attempt 1)

Original error: Validation Failed: {"resource":"PullRequest","code":"custom","field":"fork_collab","message":"fork_collab Fork collab can't be granted by someone without permission"} - https://docs.github.com/rest/pulls/pulls#create-a-pull-request
Retryable: false
Suggestion: This error cannot be resolved by retrying. Please check the error details and fix the underlying issue.

To create the pull request manually:

gh pr create --title "fix: treat persistent external-ingest search 404 as empty result (fixes #330265)" --base main --head vscodebot-pr:fix/external-ingest-404-index-not-ready-5e85eda2ea998478 --repo microsoft/vscode

Show patch preview (38 of 38 lines)

From 9947005e44091901fa56efa6b401426c283de07a Mon Sep 17 00:00:00 2001
From: "github-actions[bot]" <github-actions[bot]@users.noreply.github.com>
Date: Tue, 11 Aug 2026 15:07:51 +0000
Subject: [PATCH] fix: treat persistent external-ingest search 404 as empty
 result

---
 .../node/codeSearch/externalIngestIndex.ts          | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/extensions/copilot/src/platform/workspaceChunkSearch/node/codeSearch/externalIngestIndex.ts b/extensions/copilot/src/platform/workspaceChunkSearch/node/codeSearch/externalIngestIndex.ts
index acc320a46f4..e18a6c61a6d 100644
--- a/extensions/copilot/src/platform/workspaceChunkSearch/node/codeSearch/externalIngestIndex.ts
+++ b/extensions/copilot/src/platform/workspaceChunkSearch/node/codeSearch/externalIngestIndex.ts
@@ -486,7 +486,18 @@ export class ExternalIngestIndex extends Disposable {
 							// On the first index or a large workspace, there might be a slight delay on the service
 							// before the index is actually ready. Workaround by retrying just once after a short delay.
 							await raceCancellationError(timeout(2000), token);
-							return await this._client.searchFilesets(filesetName, resolvedQuery, sizing.maxResultCountHint, callTracker, token);
+							try {
+								return await this._client.searchFilesets(filesetName, resolvedQuery, sizing.maxResultCountHint, callTracker, token);
+							} catch (retryErr) {
+								if (retryErr instanceof ExternalIngestRequestError && retryErr.response.status === 404) {
+									// The index is still not ready server side. This is an expected transient state
+									// (not an error we can act on), so treat it as an empty search result instead of
+									// surfacing an unhandled error.
+									this._logService.warn(`ExternalIngestIndex: Fileset '${filesetName}' not ready (404) after retry, returning no results`);
+									return undefined;
+								}
+								throw retryErr;
+							}
 						}
 						
... (truncated)

[vs-code-engineering]

Recovered as #330346 by errors-fix-followup from workflow run 31504665622.