Skip to content

Commit 45eb9ea

Browse files
DavertMikclaude
andcommitted
feat(CDPBrowser): CDP screencast recording with APNG assembly
Adds startScreencast/stopScreencast to CDPBrowser via raw CDP Page.startScreencast, buffering acknowledged Page.screencastFrame events and muxing them into an APNG on stop. Verified format:'png' is honored by both Chrome and Obscura v0.2.0 (magic bytes checked directly, not assumed) before building the assembler; frames with a differing size than the first are dropped with a debug note rather than guessed at, and a non-PNG frame aborts assembly with a debug note instead of producing a broken file. New lib/helper/extras/apngAssembler.js does pure container-level PNG chunk muxing (IHDR/acTL/fcTL/IDAT/fdAT/IEND, CRC-32) with no new dependencies and no pixel encoding. The screencast plugin now supports this as a second path alongside its existing Playwright page.screencast (webm) flow, chosen automatically per active helper; captions/showActions stay Playwright-only, .srt subtitles are unchanged since they're driven by step events either way. Also fixes the plugin being unreachable for CDP-family helpers at all: getBrowserHelper only recognizes the 4 standard acting helpers, so a local duck-typed fallback (any helper exposing startScreencast) is used instead of widening that shared list for every plugin that consumes it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 2db3021 commit 45eb9ea

10 files changed

Lines changed: 734 additions & 37 deletions

File tree

docs/alternative-browsers.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,7 @@ process — there is nothing to start by hand in the common case:
143143
| Layout / getBoundingClientRect | yes | yes (v0.2.0+ default builds); synthetic on `-no-render`/v0.1.x | yes |
144144
| Screenshots | yes | yes (v0.2.0+ default builds); no on `-no-render`/v0.1.x | yes |
145145
| Visibility assertions | yes | yes (v0.2.0+ default builds); no, DOM-presence only, on `-no-render`/v0.1.x | yes |
146+
| Screencast / video (`screencast` plugin) | yes — WebM via `page.screencast`, with caption burn-in | yes — APNG via CDP `Page.startScreencast`, assembled in-process (v0.2.0+ default builds; verified PNG frames on the live server); no caption burn-in | untested |
146147
| Startup cost | seconds + ~300 MB install | instant, 70 MB binary | ~1 s, zero local |
147148
| Parallel scale | machine-bound | machine-bound (light) | near-unlimited (cloud) |
148149
| Where it runs | local/grid | local | Cloudflare only |

docs/helpers/CDPBrowser.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -183,6 +183,18 @@ silently returning every candidate node instead of none or one. The result is ca
183183

184184
Returns **[Promise][4]<[boolean][5]>** `true` if the polyfill should be injected.
185185

186+
### _onScreencastFrame
187+
188+
`Page.screencastFrame` handler: ignores frames from a session other than the currently active
189+
one (stale frames from a previous test, since the listener is never removed), acknowledges the
190+
frame so the browser keeps sending more, and buffers `{data, timestamp}` for `stopScreencast`
191+
to assemble.
192+
193+
#### Parameters
194+
195+
* `params` &#x20;
196+
* `sessionId` &#x20;
197+
186198
### _onTrafficLoadingFailed
187199

188200
`Network.loadingFailed` handler, resolving a still-pending `response` promise to `null`
@@ -1591,6 +1603,26 @@ I.startRecordingTraffic();
15911603

15921604
Returns **[Promise][4]<void>**&#x20;
15931605

1606+
### startScreencast
1607+
1608+
Starts recording a CDP `Page.startScreencast` session for the current test's target: frames
1609+
arrive as `Page.screencastFrame` events, are acknowledged immediately (`Page.screencastFrameAck`,
1610+
required or the browser stops sending more), and buffered in `this._screencastFrames`. The
1611+
underlying `Page.screencastFrame` listener is installed once (lazily) and left in place, like
1612+
`startRecordingTraffic`'s listeners, since `CDPConnection` has no listener-removal API; it
1613+
filters by `this.sessionId` so only the currently active test's frames are buffered. Call
1614+
`stopScreencast` to end the capture and assemble the buffered frames into an APNG.
1615+
1616+
```js
1617+
I.startScreencast();
1618+
```
1619+
1620+
#### Parameters
1621+
1622+
* `options` **[object][3]?** {maxWidth: number, maxHeight: number, quality: number, everyNthFrame: number} — CDP `Page.startScreencast` pass-throughs. `format` is always `'png'`.
1623+
1624+
Returns **[Promise][4]<void>**&#x20;
1625+
15941626
### stopRecordingTraffic
15951627

15961628
Stops recording network traffic started by `startRecordingTraffic`. Already-recorded requests
@@ -1602,6 +1634,29 @@ I.stopRecordingTraffic();
16021634

16031635
Returns **void**&#x20;
16041636

1637+
### stopScreencast
1638+
1639+
Stops the screencast started by `startScreencast` and assembles the buffered frames into a
1640+
single APNG (Animated PNG) file, returned as a Buffer. Frame delays are derived from the CDP
1641+
frame metadata's `timestamp` deltas (frame arrival is activity-driven — Obscura and Chrome both
1642+
only emit a frame on damage — so this reproduces the actual pacing of what happened, not a
1643+
fixed frame rate); the last frame is held for `options.lastFrameDelayMs` (default 1000ms) since
1644+
it has no "next" frame to derive a delay from. Every frame is checked for the PNG signature
1645+
before assembly — CDP's `format: 'png'` is honored by both Chrome and Obscura (verified
1646+
directly), but if some other engine ever sends a different format regardless, this reports it
1647+
via `debugSection` and returns `null` instead of muxing a broken file. Returns `null` if no
1648+
frames were captured (screencast never started, or stopped immediately after starting).
1649+
1650+
```js
1651+
const apngBuffer = await I.stopScreencast();
1652+
```
1653+
1654+
#### Parameters
1655+
1656+
* `options` **[object][3]?** {lastFrameDelayMs: number} — hold time in milliseconds for the final frame (default 1000).
1657+
1658+
Returns **[Promise][4]<[object][3]>** a Buffer with the assembled APNG, or null if there was nothing to assemble.
1659+
16051660
### type
16061661

16071662
Types characters into the currently focused element (as set by `click`, `focus`, etc), one at a

docs/plugins.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ Retries each failed step in a test.
7171

7272
## [screencast](/plugins/screencast)
7373

74-
Records WebM video of tests using Playwright's screencast API.
74+
Records a video of tests. Uses Playwright's `page.screencast` API (WebM) when the active helper is Playwright, or raw CDP `Page.startScreencast` (APNG, assembled in-process) when the active helper is `CDPBrowser` or a subclass (`Obscura`, `Kitesurf`, ...). Which path is used is detected automatically per test run; nothing in the config changes between them.
7575

7676
## [screenshot](/plugins/screenshot)
7777

docs/plugins/screencast.md

Lines changed: 18 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -9,16 +9,21 @@ title: screencast
99

1010
## screencast
1111

12-
Records WebM video of tests using Playwright's screencast API.
13-
14-
When `captions` is enabled, action annotations are burned into the video;
15-
when `subtitles` is enabled, a standalone `.srt` is also produced. Default
16-
`on=fail` keeps videos for failed tests only; `on=test` keeps every test's
17-
video.
18-
19-
Note: enabling Playwright's helper-level `video: true` together with this
20-
plugin produces two independent recordings (`output/videos/*.webm` from the
21-
helper, `output/screencast/*.webm` from this plugin).
12+
Records a video of tests. Uses Playwright's `page.screencast` API (WebM) when the active
13+
helper is Playwright, or raw CDP `Page.startScreencast` (APNG, assembled in-process) when the
14+
active helper is `CDPBrowser` or a subclass (`Obscura`, `Kitesurf`, ...). Which path is used is
15+
detected automatically per test run; nothing in the config changes between them.
16+
17+
When `captions` is enabled, action annotations are burned into the video — Playwright only,
18+
via `page.screencast.showActions()`/`showChapter()`; silently absent on the CDP path, since CDP
19+
screencast frames are raw, uncomposited page captures with no overlay mechanism. `subtitles`
20+
(a standalone `.srt`) works identically on both paths, since it's driven by step events, not by
21+
the video API. Default `on=fail` keeps videos for failed tests only; `on=test` keeps every
22+
test's video.
23+
24+
Note: enabling Playwright's helper-level `video: true` together with this plugin produces two
25+
independent recordings (`output/videos/*.webm` from the helper, `output/screencast/*.webm` from
26+
this plugin).
2227

2328
#### Configuration
2429

@@ -38,11 +43,11 @@ plugins: {
3843

3944
Other config options:
4045

41-
* `captions`: burn-in action overlays via `page.screencast.showActions()`. Default: true.
46+
* `captions`: burn-in action overlays via `page.screencast.showActions()`. Playwright only. Default: true.
4247
* `subtitles`: also write a standalone `.srt` file alongside the video. Default: false.
4348
* `video`: record a video. With `video=false, subtitles=true`, only the `.srt` is produced. Default: true.
44-
* `size`: pass-through `{ width, height }` for `screencast.start`.
45-
* `quality`: pass-through 0–100 for `screencast.start`.
49+
* `size`: pass-through `{ width, height }` `screencast.start`'s `size` on Playwright, `maxWidth`/`maxHeight` on the CDP path.
50+
* `quality`: pass-through 0–100 for `screencast.start` (Playwright) or CDP `Page.startScreencast` (CDPBrowser family).
4651

4752
CLI examples:
4853

lib/helper/CDPBrowser.js

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ import WebElement from '../element/WebElement.js'
1919
import CDPElementHandle from './extras/CDPElementHandle.js'
2020
import { checkFocusBeforeType, checkFocusBeforePressKey } from './extras/focusCheck.js'
2121
import { dontSeeTraffic, seeTraffic, grabRecordedNetworkTraffics, flushNetworkTraffics } from './network/actions.js'
22+
import { assembleApng, isPng } from './extras/apngAssembler.js'
2223

2324
/**
2425
* ## Configuration
@@ -95,6 +96,9 @@ class CDPBrowser extends Helper {
9596
this.recordedAtLeastOnce = false
9697
this._pendingTrafficResponses = new Map()
9798
this._trafficListenersInstalled = false
99+
this._screencastFrames = []
100+
this._screencastActive = false
101+
this._screencastListenerInstalled = false
98102
}
99103

100104
/**
@@ -2436,6 +2440,97 @@ class CDPBrowser extends Helper {
24362440
flushNetworkTraffics() {
24372441
return flushNetworkTraffics.call(this)
24382442
}
2443+
2444+
/**
2445+
* Starts recording a CDP `Page.startScreencast` session for the current test's target: frames
2446+
* arrive as `Page.screencastFrame` events, are acknowledged immediately (`Page.screencastFrameAck`,
2447+
* required or the browser stops sending more), and buffered in `this._screencastFrames`. The
2448+
* underlying `Page.screencastFrame` listener is installed once (lazily) and left in place, like
2449+
* `startRecordingTraffic`'s listeners, since `CDPConnection` has no listener-removal API; it
2450+
* filters by `this.sessionId` so only the currently active test's frames are buffered. Call
2451+
* `stopScreencast` to end the capture and assemble the buffered frames into an APNG.
2452+
*
2453+
* ```js
2454+
* I.startScreencast();
2455+
* ```
2456+
*
2457+
* @param {object} [options] {maxWidth: number, maxHeight: number, quality: number, everyNthFrame: number} — CDP `Page.startScreencast` pass-throughs. `format` is always `'png'`.
2458+
* @returns {Promise<void>}
2459+
*/
2460+
async startScreencast(options = {}) {
2461+
if (!this._screencastListenerInstalled) {
2462+
this._screencastListenerInstalled = true
2463+
this.cdp.on('Page.screencastFrame', (params, sessionId) => this._onScreencastFrame(params, sessionId))
2464+
}
2465+
this._screencastFrames = []
2466+
this._screencastActive = true
2467+
const params = { format: 'png' }
2468+
if (options.maxWidth) params.maxWidth = options.maxWidth
2469+
if (options.maxHeight) params.maxHeight = options.maxHeight
2470+
if (options.quality != null) params.quality = options.quality
2471+
if (options.everyNthFrame) params.everyNthFrame = options.everyNthFrame
2472+
await this.cdp.send('Page.startScreencast', params, this.sessionId)
2473+
}
2474+
2475+
/**
2476+
* `Page.screencastFrame` handler: ignores frames from a session other than the currently active
2477+
* one (stale frames from a previous test, since the listener is never removed), acknowledges the
2478+
* frame so the browser keeps sending more, and buffers `{data, timestamp}` for `stopScreencast`
2479+
* to assemble.
2480+
*
2481+
* @protected
2482+
*/
2483+
_onScreencastFrame(params, sessionId) {
2484+
if (!this._screencastActive || sessionId !== this.sessionId) return
2485+
this._screencastFrames.push({ data: params.data, timestamp: params.metadata && params.metadata.timestamp })
2486+
this.cdp.send('Page.screencastFrameAck', { sessionId: params.sessionId }, this.sessionId).catch(() => null)
2487+
}
2488+
2489+
/**
2490+
* Stops the screencast started by `startScreencast` and assembles the buffered frames into a
2491+
* single APNG (Animated PNG) file, returned as a Buffer. Frame delays are derived from the CDP
2492+
* frame metadata's `timestamp` deltas (frame arrival is activity-driven — Obscura and Chrome both
2493+
* only emit a frame on damage — so this reproduces the actual pacing of what happened, not a
2494+
* fixed frame rate); the last frame is held for `options.lastFrameDelayMs` (default 1000ms) since
2495+
* it has no "next" frame to derive a delay from. Every frame is checked for the PNG signature
2496+
* before assembly — CDP's `format: 'png'` is honored by both Chrome and Obscura (verified
2497+
* directly), but if some other engine ever sends a different format regardless, this reports it
2498+
* via `debugSection` and returns `null` instead of muxing a broken file. Returns `null` if no
2499+
* frames were captured (screencast never started, or stopped immediately after starting).
2500+
*
2501+
* ```js
2502+
* const apngBuffer = await I.stopScreencast();
2503+
* ```
2504+
*
2505+
* @param {object} [options] {lastFrameDelayMs: number} — hold time in milliseconds for the final frame (default 1000).
2506+
* @returns {Promise<object>} a Buffer with the assembled APNG, or null if there was nothing to assemble.
2507+
*/
2508+
async stopScreencast(options = {}) {
2509+
this._screencastActive = false
2510+
if (this.cdp && this.cdp.isConnected && this.sessionId) {
2511+
await this.cdp.send('Page.stopScreencast', {}, this.sessionId).catch(() => null)
2512+
}
2513+
const frames = this._screencastFrames
2514+
this._screencastFrames = []
2515+
if (!frames.length) return null
2516+
2517+
const buffers = frames.map(f => Buffer.from(f.data, 'base64'))
2518+
const nonPngIndex = buffers.findIndex(b => !isPng(b))
2519+
if (nonPngIndex !== -1) {
2520+
this.debugSection('Screencast', `frame ${nonPngIndex} is not a PNG (first bytes: ${buffers[nonPngIndex].subarray(0, 8).toString('hex')}) — this engine isn't honoring format: 'png'; skipping APNG assembly`)
2521+
return null
2522+
}
2523+
2524+
const apngFrames = buffers.map((buffer, i) => {
2525+
const delayMs = i < frames.length - 1 && frames[i + 1].timestamp && frames[i].timestamp ? Math.max(1, Math.round((frames[i + 1].timestamp - frames[i].timestamp) * 1000)) : options.lastFrameDelayMs ?? 1000
2526+
return { buffer, delayMs }
2527+
})
2528+
2529+
return assembleApng(apngFrames, {
2530+
lastFrameDelayMs: options.lastFrameDelayMs ?? 1000,
2531+
onDropFrame: info => this.debugSection('Screencast', `dropped a frame with size ${info.width}x${info.height}, expected ${info.expectedWidth}x${info.expectedHeight}`),
2532+
})
2533+
}
24392534
}
24402535

24412536
export default CDPBrowser

0 commit comments

Comments
 (0)