Skip to content

feat(routeFromHar): add interceptAPIRequests option#41294

Open
stkevintan wants to merge 4 commits into
microsoft:mainfrom
stkevintan:fix-22869
Open

feat(routeFromHar): add interceptAPIRequests option#41294
stkevintan wants to merge 4 commits into
microsoft:mainfrom
stkevintan:fix-22869

Conversation

@stkevintan

@stkevintan stkevintan commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Fixes #22869

Summary

  • Adds opt-in interceptAPIRequests option to BrowserContext.routeFromHAR so page.request.* / context.request.* calls are also served from the HAR file.
  • Defaults to false — fully backward compatible.
  • API-request lookup is restricted to entries marked _apiRequest: true (already written by the HAR recorder), so browser-side recordings are never served to API requests for the same URL.

Make `BrowserContext.routeFromHAR` also intercept requests issued via
APIRequestContext (`page.request.*` / `context.request.*`) when the new
`interceptAPIRequests: true` option is set. Defaults to `false` so existing
behavior is unchanged.

Under the hood, `HarBackend.lookup` gains an `apiRequestOnly` flag that
filters matches to entries with `_apiRequest: true` (already written by the
HAR recorder), so API-side replay never picks up a browser-side recording
for the same URL.

Fixes microsoft#22869
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dgozman dgozman requested a review from dcrousso June 15, 2026 15:45
@dgozman

dgozman commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

@dcrousso Could you please take a look?

@stkevintan

Copy link
Copy Markdown
Contributor Author

Could you please help review? @dcrousso @yury-s

@dcrousso dcrousso left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks so much for taking the time to submit a PR!

this is a really great start, but i think it's missing a few things in order to fully work

please let me know if any of my comments are unclear

Comment on lines +724 to +733
return {
body: lookupResult.body ?? Buffer.from(''),
log,
response: {
url: urlString,
status: lookupResult.status ?? 0,
statusText: '',
headers: lookupResult.headers ?? [],
},
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i dont think this handles other side-effects of API requests (e.g. setting cookies)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — the HAR path now parses set-cookie from the matched response and runs the same addCookies (with the per-cookie retry fallback) as the live path. Covered by should apply set-cookie side-effects from intercepted APIRequestContext requests.

if (postData)
setHeader(headers, 'content-length', String(postData.byteLength));
const { body, log, response } = await this._sendRequestWithRetries(progress, requestUrl, options, postData, params.maxRetries);
const harResponse = await this._lookupInHar(progress, requestUrl, method, headers, postData);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this will prevent the dispatch of APIRequestContext.Events.Request and APIRequestContext.Events.RequestFinished meaning that if a new recording is captured while replaying from the HAR then it wont include any previously captured API requests

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. The short-circuit bypassed _sendRequest, which is the only emitter of Request/RequestFinished. The HAR path now emits both events (mirroring _sendRequest), so capturing a new recording while replaying still includes the API requests. Added a test (should re-record intercepted APIRequestContext requests into a new HAR).

response: {
url: urlString,
status: lookupResult.status ?? 0,
statusText: '',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this empty?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was empty because the first cut only carried status. Now populated from entry.response.statusText (plus securityDetails/serverAddr per your other comment).

body: lookupResult.body ?? Buffer.from(''),
log,
response: {
url: urlString,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the case of a redirect, the value of url holds final endpoint URL instead of original requested URL, so i think we need to modify HarBackend.prototype.lookup to also include the url of the entry

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated HarBackend.lookup to also return the matched entry.request.url, and the response url now uses it, so it reflects the final endpoint after a redirect chain rather than the originally-requested URL.

notFound: options.notFound,
baseURL: options.baseURL,
};
this._harForAPIRequests.push(registration);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Page.prototype.route/BrowserContext.prototype.route give priority to the newest route (i.e. unshift) instead of the oldest

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to .unshift() so the newest registration wins, matching Page.route/BrowserContext.route.

Comment on lines +232 to +239
const harResponse = await this._lookupInHar(progress, requestUrl, method, headers, postData);
let body: Buffer;
let log: string[];
let response: Omit<channels.APIResponse, 'fetchUid'>;
if (harResponse)
({ body, log, response } = harResponse);
else
({ body, log, response } = await this._sendRequestWithRetries(progress, requestUrl, options, postData, params.maxRetries));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we can simplify this

const { body, log response } = (await this._lookupInHar(progress, requestUrl, method, headers, postData)) || (await this._sendRequestWithRetries(progress, requestUrl, options, postData, params.maxRetries)));

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied your suggestion: const { body, log, response } = (await this._lookupInHar(...)) || (await this._sendRequestWithRetries(...));. Both paths return the same SendRequestResult shape.

log.push(`HAR: ${lookupResult.message ?? 'lookup failed'}`);
continue;
}
if (lookupResult.action === 'noentry') {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: 'missing' would be a better name here

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kept 'noentry' here — the action is part of the LocalUtilsHarLookupResult wire protocol (localUtils.yml), so renaming it would be a breaking change for cross-version client/server compatibility. Happy to rename if you'd still prefer it and are OK treating it as a protocol change.

return {
body: lookupResult.body ?? Buffer.from(''),
log,
response: {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about securityDetails and serverAddr?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added — both are now returned from lookup (entry._securityDetails, entry.serverIPAddress/entry._serverPort) and mapped onto the response.

Comment on lines +122 to +130
const urlMatch = this._options.urlMatch;
const { registrationId } = await context._channel.harForAPIRequestsStart({
har,
urlGlob: isString(urlMatch) ? urlMatch : undefined,
urlRegexSource: isRegExp(urlMatch) ? urlMatch.source : undefined,
urlRegexFlags: isRegExp(urlMatch) ? urlMatch.flags : undefined,
notFound: this._notFoundAction,
});
this._apiRequestRegistrations.push({ context, registrationId });

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a way to use a RouteHandler like addContextRoute/addPageRoute since it seems like _handle might already be able to do a lot of what you're adding in the backend (i.e. loading and looking up in a .har)?

@stkevintan stkevintan Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RouteHandler/_handle fulfills a browser Route, which only exists for traffic that reaches the browser/CDP. APIRequestContext issues requests directly over Node http/https (_sendRequest) and never creates a Route, so _handle can't intercept it — this why #11502 added har recording/tracing ability to APIRequestContext separately.

for (const registration of registrations) {
if (!urlMatches(registration.baseURL, urlString, registration.urlMatch))
continue;
const lookupResult = await progress.race(registration.harBackend.lookup(urlString, method, headersArray, postData, false, { apiRequestOnly: true }));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we may need to pass maxRedirects into this so that it stops if the limit is reached and throws an error if its exceeded instead of giving undefined

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maxRedirects is now threaded into lookup_harFindResponse. It mirrors the live path: a negative limit means "don't follow" (returns the 3xx entry as-is), and exhausting the budget throws Max redirect count exceeded instead of returning undefined. Added should throw when intercepted APIRequestContext request exceeds maxRedirects.

Make the HAR-replay path for APIRequestContext behave like the live
network path and tighten the server-side plumbing:

- Emit Request/RequestFinished events so a recording captured while
  replaying still includes the API requests.
- Apply set-cookie side-effects via addCookies, like the live path.
- Honor maxRedirects (throw when exceeded) and report the final entry
  URL on redirects.
- Populate statusText, securityDetails and serverAddr on the response;
  log via progress.log alongside the in-memory log.
- Reuse the already-open HarBackend (looked up by harId) instead of
  opening a second backend for the same HAR file.
- Rename to routeAPIRequestsFromHar/unrouteAPIRequestsFromHar, give the
  newest registration priority (unshift), and fold the registration
  bookkeeping into the dispatcher's _disposables.

Fixes: microsoft#22869
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@stkevintan stkevintan requested a review from dcrousso June 23, 2026 15:43
# Conflicts:
#	packages/playwright-core/src/client/harRouter.ts
@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

7380 passed, 1122 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

2 failed
❌ [chromium-library] › library/browsercontext-add-init-script.spec.ts:28 › should work without navigation, after all bindings @chromium-ubuntu-22.04-arm-node20
❌ [chromium-library] › library/browsercontext-expose-function.spec.ts:77 › should be callable from-inside addInitScript @chromium-ubuntu-22.04-arm-node20

3 flaky ⚠️ [chromium-library] › library/chromium/chromium.spec.ts:211 › should intercept service worker requests (main and within) `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/popup.spec.ts:260 › should not throw when click closes popup `@chromium-ubuntu-22.04-arm-node20`
⚠️ [playwright-test] › ui-mode-trace.spec.ts:388 › should reveal errors in the sourcetab `@windows-latest-node22`

49151 passed, 1142 skipped


Merge workflow run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] routeFromHar should also apply to apiRequestContext

3 participants