Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/reference/smoke-perf-validation-surface.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,13 @@ overlay runtime integration tests, and scroll-capture session semantics tests.
| `scripts/smoke/replay-scroll-capture.sh` | Script entrypoint | deterministic replay | Runs the latest recorded trace through worker-pairwise replay. |
| `scripts/smoke/replay-scroll-capture-self-check.sh` | Script entrypoint | deterministic replay | Runs the worker-pairwise replay self-check without a user trace. |
| `scripts/smoke/analyze-scroll-capture-trace.sh` | Script entrypoint | deterministic replay | Emits summary-only replay analysis for semantic drift triage. |
| `scripts/smoke/native-hud-follow-macos.sh` | Script entrypoint | live macOS perf smoke | Dedicated HUD/loupe follow-cadence smoke for performance work, not part of the default macOS smoke aggregation. |
| `scripts/smoke/native-hud-follow-macos.sh` | Script entrypoint | live macOS perf smoke | HUD/loupe follow-cadence smoke for performance work, including delivered mouse-event count, sample refresh cadence, active-layer chrome cadence, and frame-tick cadence. |
| `scripts/smoke/native-visual-contract-macos.sh` | Script entrypoint | live macOS smoke | Core native-host behavior contract: repeated real click freezes, repeated held drag freezes, in-drag and frozen screenshots, click/drag editability, border-leak, scrim, and handoff telemetry gates. |
| `scripts/smoke/self-check-macos.sh` | Script entrypoint | smoke readiness | Verifies macOS smoke tooling and replay self-check without the real GUI run. |
| `scripts/smoke/macos.sh` | Script entrypoint | smoke aggregation | Runs the core native visual contract, then recorded-trace replay when a local trace exists or replay self-check when it does not. |
| `scripts/smoke/self-check-macos.sh` | Script entrypoint | smoke readiness | Verifies native HUD-follow smoke tooling readiness without the real GUI run. |
| `scripts/smoke/macos.sh` | Script entrypoint | smoke aggregation | Runs the core native visual contract and HUD-follow responsiveness smoke. |
| `scripts/perf/local.sh` | Script entrypoint | deterministic benches | Runs the committed Criterion smoke-sized benchmark sweep. |
| `scripts/perf/self-check-macos.sh` | Script entrypoint | perf aggregation | Runs local deterministic benches plus macOS smoke readiness. |
| `scripts/perf/macos.sh` | Script entrypoint | perf aggregation | Runs local deterministic benches, the dedicated HUD-follow perf smoke, the core native visual contract, and recorded-trace replay. |
| `scripts/perf/macos.sh` | Script entrypoint | perf aggregation | Runs local deterministic benches, the HUD-follow perf smoke, and the core native visual contract. |
| `packages/rsnap-overlay/src/overlay/replay_support.rs` tests | Deterministic replay / bench | replay harness | Trace round-trip, replay mode selection, and summary classification. |
| `packages/rsnap-overlay/benches/scroll_capture.rs` | Deterministic replay / bench | hot-path perf | Stable fingerprint, overlap-match, and one-step session commit baselines. |
| `packages/rsnap-overlay/src/overlay/tests/worker_tick_runtime.rs` | Overlay runtime integration | overlay runtime | Request issuance, retry timing, backoff, and fresh-input worker scheduling. |
Expand Down
22 changes: 10 additions & 12 deletions docs/runbook/performance-validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,20 +56,18 @@ worker-pairwise self-check path when no recorded user trace is available.
- Use this for routine local comparisons and for regressions that do not require a real desktop
session.
- `scripts/perf/self-check-macos.sh`
- Runs `scripts/perf/local.sh`, then runs the native HUD-follow self-check plus
recorded-live-trace scroll-capture replay self-check.
- Runs `scripts/perf/local.sh`, then runs the native HUD-follow self-check through
`scripts/smoke/self-check-macos.sh`.
- Use this to validate that the dedicated macOS environment, permissions, and smoke harness are
ready without treating it as an end-to-end performance assertion.
- `scripts/perf/macos.sh`
- Runs `scripts/perf/local.sh`, the dedicated native-host HUD-follow perf smoke, the core native
visual contract smoke, plus recorded-live-trace scroll-capture replay.
visual contract smoke.
- Use this only on a dedicated logged-in macOS desktop session with the expected Screen
Recording and automation permissions.
- `scripts/smoke/macos.sh`
- Runs the core native visual contract smoke.
- Runs recorded-live-trace replay when a local `manifest.json` exists under the scroll-capture
trace directory; otherwise it runs `scripts/smoke/replay-scroll-capture-self-check.sh` so a
missing optional trace does not fail unrelated native-host validation.
- Runs the native HUD-follow responsiveness smoke.

For the downward scroll-capture rebuild, the expected verification sequence is:

Expand Down Expand Up @@ -109,8 +107,12 @@ Dedicated macOS smoke:
- Requires a logged-in macOS desktop session.
- Requires the expected Screen Recording and automation permissions for the smoke scripts.
- Covers the native-host HUD-follow desktop path. The hard follow gate uses active pointer-movement
cadence (`live_chrome.active_layer_chrome_render_gap`) rather than startup, Tab-expand, or close
transition gaps.
cadence (`live_chrome.active_layer_chrome_render_gap`) and frame-tick cadence
(`live_chrome.frame_tick_gap`) rather than startup, Tab-expand, or close transition gaps.
- Requires the smoke harness to deliver enough mouse-movement input. For the default smooth event
path, `native-hud-follow-macos.sh` expects at least half of the requested
`(PATH_DURATION_MS / 1000) * PATH_RATE_HZ` event count; override with `MIN_MOUSE_EVENTS` only
when validating a different input driver or intentionally degraded environment.
- Interpret cadence metrics by class:
- display-bound visual presentation metrics are gated against
`min(active display maximum refresh rate, 120 Hz)`, so a `60 Hz` monitor has a `16.67 ms`
Expand All @@ -133,10 +135,6 @@ Dedicated macOS smoke:
worker-pairwise overlay or session logic before attempting more desktop-session repro. If the
command reports that no trace manifests were found, that is an operator/setup failure: record a
fresh live trace first or rerun the example with `--trace <manifest-path>`.
- `scripts/smoke/macos.sh` choosing replay self-check:
treat it as expected when the machine has no recorded scroll-capture trace. It is not evidence
for or against the latest user-recorded live trace; run `scripts/smoke/replay-scroll-capture.sh`
with a real trace when scroll-capture replay evidence is required.
- `scripts/smoke/replay-scroll-capture-self-check.sh` failures:
treat them as deterministic regressions in the replay harness itself, not as evidence about the
latest user-recorded live trace.
Expand Down
6 changes: 3 additions & 3 deletions native/macos-host/Sources/RsnapHostBridgeProbe/main.swift
Original file line number Diff line number Diff line change
Expand Up @@ -104,15 +104,15 @@ enum RsnapHostBridgeProbe {
scene.statusMessage == nil,
scene.toolbarItems.contains(where: { $0.kind == .pointer && $0.selected }),
scene.toolbarItems.contains(where: { $0.kind == .ocr && $0.enabled }),
scene.toolbarItems.contains(where: { $0.kind == .scroll && $0.enabled }),
!scene.toolbarItems.contains(where: { $0.kind == .scroll }),
scene.toolbarItems.contains(where: { $0.kind == .copy && $0.enabled }),
scene.toolbarItems.contains(where: { $0.kind == .save && $0.enabled })
else {
fatalError("unexpected frozen scene: \(scene)")
}
try session.send(event: .toolbarItemInvoked(.scroll))
guard try session.takeNextRequest() == .startScrollCapture else {
fatalError("expected a start-scroll-capture host request")
guard try session.takeNextRequest() == nil else {
fatalError("scroll toolbar invocation should stay disabled")
}

try session.send(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -140,11 +140,27 @@ final class FrozenFrameAuthority: @unchecked Sendable {
lock.unlock()
}

func fresh(maxAge: TimeInterval) -> SCShareableContent? {
func fresh(
maxAge: TimeInterval,
covering displayIDs: Set<CGDirectDisplayID>? = nil
) -> SCShareableContent? {
let now = ProcessInfo.processInfo.systemUptime
lock.lock()
let content = now - cachedAtUptime <= maxAge ? self.content : nil
lock.unlock()
guard let content else {
return nil
}
guard !content.displays.isEmpty else {
return nil
}
guard let displayIDs else {
return content
}
let availableDisplayIDs = Set(content.displays.map(\.displayID))
guard displayIDs.isSubset(of: availableDisplayIDs) else {
return nil
}
return content
}
}
Expand Down Expand Up @@ -184,6 +200,27 @@ final class FrozenFrameAuthority: @unchecked Sendable {
)
return
}
guard Self.shareableContentHasDisplays(content) else {
NativeHostTelemetry.frozenAuthorityWarning(
"frozen_authority.content_cache_refresh_invalid",
captureID: captureID,
source: source,
displayID: 0,
error: Self.shareableContentDisplayDetail(
content,
requiredDisplayIDs: []
)
)
NativeHostTelemetry.frozenAuthorityContentLookupTiming(
captureID: captureID,
source: source,
totalMilliseconds: NativeHostTelemetry.milliseconds(since: startedAtUptime),
success: false,
displayCount: content.displays.count,
windowCount: content.windows.count
)
return
}
Self.shareableContentCache.store(content)
NativeHostTelemetry.frozenAuthorityContentLookupTiming(
captureID: captureID,
Expand All @@ -196,8 +233,13 @@ final class FrozenFrameAuthority: @unchecked Sendable {
}
}

private static func cachedShareableContent() -> SCShareableContent? {
shareableContentCache.fresh(maxAge: shareableContentCacheMaxAge)
private static func cachedShareableContent(
covering displayIDs: Set<CGDirectDisplayID>? = nil
) -> SCShareableContent? {
shareableContentCache.fresh(
maxAge: shareableContentCacheMaxAge,
covering: displayIDs
)
}

func hasFreshShareableContentCache() -> Bool {
Expand Down Expand Up @@ -314,7 +356,7 @@ final class FrozenFrameAuthority: @unchecked Sendable {
retryUntilUptime: TimeInterval,
requestID: UInt64
) {
if let content = Self.cachedShareableContent() {
if let content = Self.cachedShareableContent(covering: targetIDs) {
let preparedFilters = Self.contentFilters(
for: targets,
in: content,
Expand Down Expand Up @@ -388,17 +430,51 @@ final class FrozenFrameAuthority: @unchecked Sendable {
return
}

let contentCoversTargets = Self.shareableContent(content, covers: targetIDs)
NativeHostTelemetry.frozenAuthorityContentLookupTiming(
captureID: captureID,
source: source,
totalMilliseconds: NativeHostTelemetry.milliseconds(since: startedAtUptime),
success: true,
success: contentCoversTargets,
displayCount: content.displays.count,
windowCount: content.windows.count
)
guard self.isCurrentSetupRequest(requestID, targetIDs: targetIDs) else {
return
}
guard contentCoversTargets else {
NativeHostTelemetry.frozenAuthorityWarning(
"frozen_authority.content_lookup_invalid",
captureID: captureID,
source: source,
displayID: 0,
error: Self.shareableContentDisplayDetail(
content,
requiredDisplayIDs: targetIDs
)
)
if ProcessInfo.processInfo.systemUptime < retryUntilUptime {
DispatchQueue.global(qos: .userInteractive).asyncAfter(
deadline: .now() + Self.selfCaptureFilterRetryInterval
) { [weak self] in
self?.rebuildStreamsFromShareableContent(
targets: targets,
targetIDs: targetIDs,
selfCaptureExceptionWindowIDs: selfCaptureExceptionWindowIDs,
includedCurrentProcessWindowIDs: includedCurrentProcessWindowIDs,
captureID: captureID,
source: source,
startedAtUptime: startedAtUptime,
retryUntilUptime: retryUntilUptime,
requestID: requestID
)
}
return
}
self.finishSetup(targetIDs: targetIDs)
return
}
Self.shareableContentCache.store(content)
let preparedFilters = Self.contentFilters(
for: targets,
in: content,
Expand Down Expand Up @@ -476,7 +552,8 @@ final class FrozenFrameAuthority: @unchecked Sendable {
source: String,
startedAtUptime: TimeInterval
) {
if let content = Self.cachedShareableContent() {
let targetIDs = Set(targets.map(\.displayID))
if let content = Self.cachedShareableContent(covering: targetIDs) {
NativeHostTelemetry.frozenAuthorityContentLookupTiming(
captureID: captureID,
source: source,
Expand All @@ -500,6 +577,7 @@ final class FrozenFrameAuthority: @unchecked Sendable {
)
return
}
let retryUntilUptime = startedAtUptime + Self.selfCaptureFilterRetryWindow
SCShareableContent.getExcludingDesktopWindows(false, onScreenWindowsOnly: false) {
[weak self] content, error in
guard let self else {
Expand Down Expand Up @@ -527,14 +605,46 @@ final class FrozenFrameAuthority: @unchecked Sendable {
self.finishSetup(generation: requestGeneration)
return
}
let contentCoversTargets = Self.shareableContent(content, covers: targetIDs)
NativeHostTelemetry.frozenAuthorityContentLookupTiming(
captureID: captureID,
source: source,
totalMilliseconds: NativeHostTelemetry.milliseconds(since: startedAtUptime),
success: true,
success: contentCoversTargets,
displayCount: content.displays.count,
windowCount: content.windows.count
)
guard contentCoversTargets else {
NativeHostTelemetry.frozenAuthorityWarning(
"frozen_authority.content_lookup_invalid",
captureID: captureID,
source: source,
displayID: 0,
error: Self.shareableContentDisplayDetail(
content,
requiredDisplayIDs: targetIDs
)
)
if ProcessInfo.processInfo.systemUptime < retryUntilUptime {
DispatchQueue.global(qos: .userInteractive).asyncAfter(
deadline: .now() + Self.selfCaptureFilterRetryInterval
) { [weak self] in
self?.configureStreamsFromShareableContent(
targets: targets,
selfCaptureExceptionWindowIDs: selfCaptureExceptionWindowIDs,
includedCurrentProcessWindowIDs: includedCurrentProcessWindowIDs,
generation: requestGeneration,
captureID: captureID,
source: source,
startedAtUptime: startedAtUptime
)
}
return
}
self.finishSetup(generation: requestGeneration)
return
}
Self.shareableContentCache.store(content)
let preparedFilters = Self.contentFilters(
for: targets,
in: content,
Expand Down Expand Up @@ -1130,6 +1240,31 @@ final class FrozenFrameAuthority: @unchecked Sendable {
return snapshot
}

private static func shareableContentHasDisplays(_ content: SCShareableContent) -> Bool {
!content.displays.isEmpty
}

private static func shareableContent(
_ content: SCShareableContent,
covers displayIDs: Set<CGDirectDisplayID>
) -> Bool {
guard shareableContentHasDisplays(content) else {
return false
}
let availableDisplayIDs = Set(content.displays.map(\.displayID))
return displayIDs.isSubset(of: availableDisplayIDs)
}

private static func shareableContentDisplayDetail(
_ content: SCShareableContent,
requiredDisplayIDs: Set<CGDirectDisplayID>
) -> String {
let required = requiredDisplayIDs.sorted().map { String($0) }.joined(separator: ",")
let available = content.displays.map(\.displayID).sorted().map { String($0) }.joined(
separator: ",")
return "requiredDisplayIDs=\(required) availableDisplayIDs=\(available)"
}

private static func streamConfiguration(for target: DisplayTarget) -> SCStreamConfiguration {
let configuration = SCStreamConfiguration()
configuration.width = target.widthPixels
Expand Down
Loading