Upgrade Playwright to v1.60

[Mamaduka]

What?

Upgrade Playwright to v1.60.

What's new?

https://playwright.dev/docs/release-notes#version-160

I think new page.locator('#dropzone').drop could be useful for our tests. I'll start looking into migrations as a follow-up. The new description option could also be handy.

Testing Instructions

• Run any e2e test locally; the command should download new browsers and run the test successfully.
• CI checks are green.

[jsnajdr]

Good idea, should cause no trouble. The other day I did an upgrade from 1.58 to 1.59 locally, in an attempt to find a bug, but never committed it as a PR.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: Mamaduka <mamaduka@git.wordpress.org>
Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org>
Co-authored-by: jsnajdr <jsnajdr@git.wordpress.org>
Co-authored-by: manzoorwanijk <manzoorwanijk@git.wordpress.org>
Co-authored-by: aduth <aduth@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[github-actions]

Size Change: +118 B (0%)

Total Size: 7.51 MB

📦 View Changed

Filename	Size	Change
build/scripts/editor/index.min.js	477 kB	+118 B (+0.02%)

_{compressed-size-action}

[Mamaduka]

It looks like previews end up with stale windows in the latest Chrome and DIP enabled. Pushed some test code for confirmations.

Here's the finding from Claude's debug sessions:

Summary

Playwright 1.60 ships Chrome for Testing 148 (1.58 used 145). The failing e2e tests aren't a test-infra issue — they're surfacing a Chromium 148 regression in Document-Isolation-Policy: isolate-and-credentialless (DIP), which Gutenberg sends on editor screens for Chromium ≥137 to get cross-origin isolation.

Proven by toggling the gutenberg_use_document_isolation_policy filter on the same Chrome 148:

Cluster	DIP on (default)	DIP off
Preview / block-context (window.open('', 'wp-preview-X') reuse)	❌ stale tab / waitForNavigation hangs	✅ pass
Client-side media processing	❌ silent server fallback	⏭️ skip (CSM needs isolation)

• Preview: under DIP, Chrome 148 no longer re-navigates a reused named popup → preview shows stale content. DIP's spec says popups should keep working, so this is a genuine browser regression (closest public match: crbug 336222177). Likely affects real Chrome 148 users, not just CI.
• CSM: same header is what enables crossOriginIsolated; a second 148 regression in the isolated/credentialless runtime makes the transcode silently fall back.
• The Chrome 146/147/148 release notes document none of this.

**Playwright 1.60 ships Chrome for Testing 148** (1.58 used 145). The failing e2e tests aren't a test-infra issue — they're surfacing a **Chromium 148 regression in `Document-Isolation-Policy: isolate-and-credentialless`** (DIP), which Gutenberg sends on editor screens for Chromium ≥137 to get cross-origin isolation.

Proven by toggling the gutenberg_use_document_isolation_policy filter on the same Chrome 148:

Cluster	DIP on (default)	DIP off
Preview / block-context (window.open('', 'wp-preview-X') reuse)	❌ stale tab / waitForNavigation hangs	✅ pass
Client-side media processing	❌ silent server fallback	⏭️ skip (CSM needs isolation)

• Preview: under DIP, Chrome 148 no longer re-navigates a reused named popup → preview shows stale content. DIP's spec says popups should keep working, so this is a genuine browser regression (closest public match: [crbug 336222177](https://issues.chromium.org/issues/336222177)). Likely affects real Chrome 148 users, not just CI.
• CSM: same header is what enables crossOriginIsolated; a second 148 regression in the isolated/credentialless runtime makes the transcode silently fall back.
• The Chrome 146/147/148 release notes document none of this.

[Mamaduka]

I guess we can wait until the next release and see if the issue is fixed in the next Chrome upgrade

cc @adamsilverstein, just in case you've seen any similar reports.

[manzoorwanijk]

Peer dependency should usually be the lowest supported version.

Also, IMHO, this peer dep here should be made optional, otherwise it breaks in strict peer deps mode for consumers.

[Mamaduka]

What's your recommendation?

[manzoorwanijk]

Leave this unchanged and add the updated version as a dev dependency, as the package uses it but doesn't declare it and it receives the dependency from hoisting.

gutenberg/packages/scripts/config/playwright.config.js

Line 5 in 229c4d2

const { defineConfig, devices } = require( '@playwright/test' );

[manzoorwanijk]

Add this to peerDependenciesMeta below

"@playwright/test": {
	"optional": true
},

[Mamaduka]

Looking at the Git history, we've been bumping these versions alongside others. I don't remember exactly, but I think there were some version mismatches in the past on wordpress-develop.

P.S. I'm not against making those changes, mostly was double-checking previous updates.

[manzoorwanijk]

Yes, but it's better to go with best practices.

[manzoorwanijk]

CCing @jsnajdr and @aduth for their opinion.

[aduth]

From my perspective, the peer dependency should be as wide as we can possibly allow based on what we actually need to use from this package or are fearful could break in how we use in future versions, even * or >=1. As best I can tell, that's whatever version provides a /cli export that accepts as 'test' argument and config flags and returns a 0 or non-0 status code, which is probably... all of them, forever? (i.e. *) We could add an upper bound like <= 2 or ^1 if we worry about those expectations breaking in a future major version.

The advantage being that this is a lot less annoying for downstream consumers. Why should someone have to bump their @playwright/test when upgrading @wordpress/scripts, if we're not actually doing anything in @wordpress/scripts that needs that newer version?

That's not to say we don't want to be using the same, consistent version across Gutenberg and/or wordpress-develop, but that's a project-level concern. We are our own consumer. So defining devDependencies with the higher version number makes sense so that we are using the one, newer version that has the new feature we want to use.

[manzoorwanijk]

Heads up: Following #79221, we will need to update the dependency in root package.json as well.

[Mamaduka]

It looks like Chrome and DIP issues still exist even after the recent Chromium version bump in Playwright - #78632 (comment).

That's a blocker for this update, until those issues are tested and resolved manually :(

[jsnajdr]

I'm afraid that the Document-Isolation-Policy is a real Gutenberg bug that we'll need to address. The Playwright update merely surfaces it, by virtue of installing a newer Chrome.

When opening a preview (in openPreviewWindow) we do three steps:

• window.open a new named tab, potentially reusing the already opened one.
• writeInterstitialMessage( previewWindow.document ) uses the ancient document.write to render an animated WP logo while the preview is loading.
• previewWindow.location = link forces a reload of the preview, with latest changes.

The code that crashes is the previewWindow.document access. If you comment out the writeInterstitialMessage call, the e2e test will be green again. Document-Isolation-Policy isolates the two windows (editor and preview) from each other, even if they are same-origin. And window.document is one of the things you can't touch any more. We'll have to write the interstitial in some less direct way, e.g., the preview window hosting the interstitial-writing script and receiving a message from the opener.

The https://issues.chromium.org/issues/336222177 bug is not really that related. It's about window.open not reusing the same tab, or window.location setter failing to navigate. None of that happens for us, our issue is the window.document access. Frankly this is fairly typical LLM debugging output, it's good at pointing you in the right direction, but there is often some subtle error, and a human judgment and review is needed to walk the last mile.

[jsnajdr]

Locally I'm able to fix the preview window error by adding these two lines before writeInterstitialMessage:

previewWindow.location = 'about:blank';
await new Promise( ( r ) => setTimeout( r, 100 ) );
writeInterstitialMessage( previewWindow.document );

The point is to reset the preview window location to about:blank because on this location the synchronous document access still works. That's why the first attempt to open the preview succeeds. Only the second one fails.

The 100ms wait is there to wait until the window really loads the new location. We can't add event listeners (load) to the window, so we need to wait. setTimeout( 0 ) is not long enough, but 100ms works.

If only we could tidy up this patch and make it production-ready, it would also fix the issue.

[adamsilverstein]

I'm afraid that the Document-Isolation-Policy is a real Gutenberg bug that we'll need to address.

Catching up here... DIP shouldn't be active on previews, are you seeing it active there? If so, maybe we missed something or something changed? Are there steps to reproduce this manually? is this for previews in a new tab pr inline?

[jsnajdr]

DIP shouldn't be active on previews, are you seeing it active there?

DIP is active only on the main editor tab, on the preview tabs it's not active. But that's enough to put both tabs into different "agent clusters", and they become isolated from each other. Like if they were cross-origin.

Are there steps to reproduce this manually?

Yes, it's very straighforward:

1. Edit a post
2. Open a preview by clicking "Preview in new tab" in the "View" menu
3. Make edits to the post
4. Click "Preview in new tab" again.

It's supposed to save the latest edits and update the existing preview tab with new content.

But the actual result is that the tab still shows the old content, and in console there is a security error that the preview tab's previewWindow.document can't be accessed.

[adamsilverstein]

DIP shouldn't be active on previews, are you seeing it active there?

DIP is active only on the main editor tab, on the preview tabs it's not active. But that's enough to put both tabs into different "agent clusters", and they become isolated from each other. Like if they were cross-origin.

Are there steps to reproduce this manually?

Yes, it's very straighforward:

Thanks for clarifying, this helps me understand the issue more clearly.

But the actual result is that the tab still shows the old content, and in console there is a security error that the preview tab's previewWindow.document can't be accessed.

Ah - I didn't realize we were doing this kind of inter-tab communication for the preview window.

Locally I'm able to fix the preview window error by adding these two lines before writeInterstitialMessage:
The 100ms wait is there to wait until the window really loads the new location.

That fix makes sense, but feels a bit fragile. I wonder if we could catch the error if it occurs and retry, in case the tab takes longer to reset to about:blank?

I don't quite understand how upgrading Playwright exposed this bug; regardless it feels like something we legitimately need to fix! Thanks for the ping.

[adamsilverstein]

@jsnajdr - I pushed a proposed fix in #79342. Rather than a fixed timeout, the PR sets the initial load directly, then uses the about:blank/reset approach, trying at 50ms intervals up to 1 second - using try/catch along the way to avoid the error.

[adamsilverstein]

don't quite understand how upgrading Playwright exposed this bug; regardless it feels like something we legitimately need to fix! Thanks for the ping.

oh, I see - its an issue with the new Chrome version.

[adamsilverstein]

I spent more time on this and discovered the underlying cause of the client side media failures issue wasn't quite what we expected. I opened a follow up issue just to fix this in #79377. The short version is:

• Previous Playwright versions relied on a Chrome instance that lacked Document Isolation Policy, so the client side media tests were actually being skipped.
• When they finally started running in the newer Playwright, a test environment only bug surfaced - the tests ran the client side media flows before wasm-vips was able to initialize which takes some small amount of time because the engine is lazy loaded and needs to be decoded as its embedded inline to avoid issues we our inability to set headers
• Users never experience these issues because they aren't that fast, the problem only surfaced in the artificially fast test environment

The solution I came up with in #79378 should resolve the issue. I'm going to open a draft PR combining that with the Playwright update to verify CI goes green.

[adamsilverstein]

The solution I came up with in #79378 should resolve the issue. I'm going to open a draft PR combining that with the Playwright update to verify CI goes green.

I opened a draft combined PR in #79381 that include the Playwright upgrade, the preview timing fix and the wasm loading fix. If CI goes green it will validate the fixes resolve the issues.

[adamsilverstein]

@jsnajdr - Merging in #79495 into this PR will bring CI here green. It includes the preview fix as well as a bunch of follow up client-side media test fixes, see the ticket description for details.

[adamsilverstein]

Tests should go green after aade9a4