diff --git a/docs/decisions/annotation-pen-style.md b/docs/decisions/annotation-pen-style.md new file mode 100644 index 00000000..3b8670d6 --- /dev/null +++ b/docs/decisions/annotation-pen-style.md @@ -0,0 +1,69 @@ +# Annotation Pen Style + +Status: accepted +Date: 2026-04-07 +Context: + +- `rsnap` uses the pen tool for screenshot annotation, not for professional drawing. +- Users prefer a polished handwritten annotation look over faithful reproduction of every mouse + wobble. +- The governing behavior contract for this decision is `docs/spec/annotation-pen.md`. +- Earlier iterations exposed the wrong tradeoff: + - raw-point fidelity preserved tiny dents and uneven curvature that made rough circles and arcs + look amateurish + - segmented capsule or dense-dab rendering produced visible scalloping, gaps, or jagged preview + - auto-closing loops and shape guessing changed user intent too aggressively for general + annotation strokes +- The product goal is therefore "make the mark look better than the hand drew it" while keeping + the stroke recognizably human and fast enough for live annotation. + +Decision: + +- Treat the pen as an annotation stylizer, not a faithful brush. +- Optimize the final stroke for visual quality even when that means discarding small input + deviations. +- Keep the implementation layered: + - use a light online stroke model during drag so preview remains smooth and reasonably consistent + with the final stroke + - apply a stronger finalize pass on release to suppress micro-wobbles and improve curvature + continuity +- Default the finalize pass to preserve the large-scale path while aggressively removing + high-frequency local dents, bumps, and shallow reversals. +- Prefer an open freehand annotation model over generic shape inference or auto-closing behavior. +- Keep the live interaction and the finalized result within the same visual family so the product + feels like a polished annotation pen instead of a post-hoc shape replacement tool. + +Alternatives considered: + +- Preserve raw input as the main source of truth. + - Rejected because it retains hand jitter and produces visibly uneven annotation curves. +- Smooth only after mouse release with no online modeling. + - Rejected because preview quality remains poor and the final stroke changes too abruptly on + release. +- Auto-close near-loops or infer shapes such as circles and checks. + - Rejected because annotation strokes are varied and the correction often overshoots user intent. +- Render the stroke as segmented capsules or dense visible dabs. + - Rejected because segment boundaries and scalloping are noticeable in `egui` preview. +- Keep increasing generic smoothing passes without explicitly targeting micro-wobbles. + - Rejected because it softens the whole stroke without reliably eliminating the small dents users + actually notice. + +Consequences: + +- Future tuning should bias toward stronger beautification, not higher input fidelity. +- "Small" defects should be interpreted relative to annotation scale, especially brush width and + short local span, rather than as fixed absolute pixels. +- Changes that preserve tiny dents, shallow notches, or uneven arc curvature are regressions even + if they are more faithful to the pointer path. +- If a future implementation needs to become more precise, precision should be an explicit mode, + not the default annotation behavior. +- New work on the pen should prefer: + - micro-wobble suppression + - curvature-continuity improvements + - preview/final consistency + - subtle online prediction or stabilization +- New work on the pen should avoid by default: + - generic shape inference + - auto-closing loops + - segmented visible stroke primitives + - raw-path fidelity as a success metric diff --git a/docs/decisions/index.md b/docs/decisions/index.md index f0457b86..eca21ad4 100644 --- a/docs/decisions/index.md +++ b/docs/decisions/index.md @@ -43,5 +43,5 @@ Then keep the body decision-oriented: ## Current decision records -No durable decision records have been written yet. Add one here when a tradeoff needs to remain -discoverable beyond commit history and code comments. +- `docs/decisions/annotation-pen-style.md` for the pen-tool tradeoff that prioritizes polished + screenshot annotation over faithful pointer-path reproduction diff --git a/docs/spec/annotation-pen.md b/docs/spec/annotation-pen.md new file mode 100644 index 00000000..aeb5d7ad --- /dev/null +++ b/docs/spec/annotation-pen.md @@ -0,0 +1,126 @@ +# rsnap Annotation Pen Contract + +Purpose: Define the normative behavior contract for the Frozen-mode pen tool used for screenshot +annotation. + +Status: normative + +Read this when: You are implementing, reviewing, tuning, or validating pen-tool behavior in +Frozen mode, including preview, commit, undo/redo, and export. + +Not this document: Design rationale for why annotation beautification is preferred over pointer +fidelity, or current implementation notes. Use +`docs/decisions/annotation-pen-style.md` for rationale and `docs/reference/` for implementation +context. + +Defines: +- the product goal and required behavior for Frozen-mode pen strokes +- preview, commit, export, and undo/redo invariants for pen annotations +- prohibited default behaviors for generic shape inference and automatic loop closure + +## Scope + +This contract applies to the `FrozenToolbarTool::Pen` path after a capture has entered Frozen +mode. + +This contract governs: + +- stroke preview while the pointer is down +- stroke finalization when the pointer is released +- committed annotation rendering in the frozen surface +- annotation export behavior for copy and save flows +- undo and redo behavior for committed pen strokes + +This contract does not require: + +- professional illustration or precision drawing behavior +- automatic conversion into canonical shapes such as circles, rectangles, arrows, or checks +- exact reproduction of the pointer path + +## Product objective + +The pen tool is an annotation stylizer, not a faithful freehand brush. + +Required product objective: + +- The pen tool MUST prioritize producing visually polished screenshot annotations over preserving + every small deviation in the input path. +- The resulting mark SHOULD look recognizably hand-drawn, but better than the raw pointer motion. +- Small high-frequency wobble, dents, shallow reversals, and uneven curvature SHOULD be treated as + disposable noise when removing them improves the overall mark. +- Large-scale stroke direction, openness, and endpoint intent MUST remain recognizable. + +## Required behavior + +### Availability and mode boundary + +- The pen tool applies only in Frozen mode. +- The pen tool annotates the current frozen capture and does not modify live-mode selection flow. +- Pen annotations are part of the frozen capture state and therefore participate in preview, copy, + save, and undo/redo. + +### Preview and commit consistency + +- Drag-time preview MUST already use a beautified stroke path family rather than raw pointer + segments. +- Pointer-release finalization MAY apply stronger beautification than drag-time preview. +- Pointer-release finalization MUST remain in the same geometric family as preview and MUST NOT + replace the stroke with a materially different shape character. +- The user MUST NOT see a low-quality jagged preview that becomes the first acceptable result only + after release. + +### Beautification contract + +- Pen beautification MUST prefer final visual quality over raw pointer fidelity. +- The beautification pass MUST suppress small local defects when they are inconsistent with the + surrounding stroke trend. +- The beautification pass MUST improve curvature continuity for arcs and rounded marks rather than + only smoothing positions point-by-point. +- "Small" defects MUST be interpreted relative to annotation scale, especially stroke width and + short local span, rather than as a fixed absolute pixel threshold. +- The default tuning SHOULD be aggressive enough that rough circles, arcs, and check-like marks + look intentionally smooth without requiring the user to draw like a professional illustrator. + +### Open-stroke semantics + +- Pen strokes MUST remain open by default. +- The implementation MUST NOT auto-close near-loops by default. +- The implementation MUST NOT infer or replace generic shapes by default. +- If future precision or shape-assisted behavior is added, it MUST be an explicit mode or user + action rather than the default pen behavior. + +### Endpoint and large-scale path intent + +- Finalized strokes MUST preserve endpoint intent. +- Finalized strokes MAY move endpoints slightly only when required to preserve visible continuity, + but they MUST NOT materially relocate the apparent start or end of the mark. +- Finalized strokes MUST preserve the large-scale path trend even when local jitter is discarded. + +### Rendering and export + +- On-screen pen rendering MUST appear continuous and anti-aliased. +- Exported pen rendering MUST appear continuous and anti-aliased. +- The committed export path used by clipboard copy and save MUST include the same committed pen + annotations visible in Frozen mode. +- Export MUST NOT omit committed pen strokes and MUST NOT render them with a different stroke + family from the committed on-screen result. + +### Undo and redo + +- Undo and redo MUST operate on committed pen strokes. +- Undo and redo MUST round-trip the committed beautified stroke, not the raw pointer samples. +- Re-export after undo or redo MUST reflect the current committed stroke set exactly. + +## Explicit non-goals for the default pen behavior + +- exact pointer-path fidelity +- generic shape recognition +- auto-closing loops +- exposing every tiny dent or wobble in the raw hand motion +- professional drawing precision as the default interaction goal + +## Governing rationale + +The accepted rationale for this contract lives in: + +- `docs/decisions/annotation-pen-style.md` diff --git a/docs/spec/capture-session.md b/docs/spec/capture-session.md index e0b04ba3..3588e532 100644 --- a/docs/spec/capture-session.md +++ b/docs/spec/capture-session.md @@ -16,6 +16,7 @@ Defines: - capture-session entry, live-mode, frozen-mode, and export invariants - hovered-window, region-selection, and fullscreen fallback behavior - the current macOS scroll-capture contract +- the presence of Frozen-mode annotation state in session output This repository contains a pure-Rust screenshot prototype targeting macOS first, with a cross-platform architecture. @@ -61,6 +62,8 @@ cross-platform architecture. - `Space` -> copy the frozen cropped PNG (region/window/fullscreen) to the system clipboard, then exit - On macOS, the frozen toolbar may expose `Recognize Text`, which runs Apple Vision OCR on the current frozen capture, copies the recognized text to the clipboard, and exits - Cmd+S (macOS) / Ctrl+S -> save the frozen cropped PNG to disk, then exit + - In Frozen mode, toolbar-driven annotations are part of the frozen capture state; the pen-tool + contract lives in `docs/spec/annotation-pen.md` - Esc -> cancel and exit without copying - After a dragged-region freeze enters Frozen mode, dragging inside the bright region repositions the frozen capture rect without resizing it and keeps it on the same monitor @@ -195,5 +198,6 @@ Research and cross-platform notes live in: ## Current non-goals -- Annotation/editor UI, pinning, and advanced editing tools. +- Rich annotation/editor tooling beyond the current frozen toolbar tools, pinning, and advanced + editing workflows. - Cross-monitor selection and cross-monitor window capture behavior. diff --git a/docs/spec/index.md b/docs/spec/index.md index 2709e2e6..947a79b1 100644 --- a/docs/spec/index.md +++ b/docs/spec/index.md @@ -54,6 +54,8 @@ Then keep the body explicit: ## Current specs +- `docs/spec/annotation-pen.md` for the Frozen-mode pen-tool behavior contract and beautification + invariants - `docs/spec/capture-session.md` for the capture-flow and scroll-capture behavior contract - `docs/spec/performance.md` for render cadence, performance scenarios, metrics, thresholds, and known performance-contract gaps diff --git a/packages/rsnap-overlay/src/overlay.rs b/packages/rsnap-overlay/src/overlay.rs index 688b5590..c15e0ba1 100644 --- a/packages/rsnap-overlay/src/overlay.rs +++ b/packages/rsnap-overlay/src/overlay.rs @@ -157,7 +157,8 @@ use self::rendering::{ #[cfg(all(target_os = "macos", test))] use self::session_state::InflightScrollCaptureObservation; use self::session_state::{ - CursorMoveTrace, FrozenMosaicDragState, FrozenSelectionDragCursorMoveTiming, + ActiveFrozenBrushStroke, CursorMoveTrace, FrozenBrushModelState, FrozenBrushState, + FrozenBrushStroke, FrozenMosaicDragState, FrozenSelectionDragCursorMoveTiming, FrozenSelectionDragState, FrozenToolbarPointerState, FrozenToolbarState, HudDrawConfig, LiveSampleApplyResult, ScrollCaptureState, SlowOperationLogger, WindowFreezeCaptureTarget, }; @@ -349,6 +350,33 @@ const FROZEN_SELECTION_RESIZE_HANDLE_CENTER_DOT_RADIUS_POINTS: f32 = 1.15; const FROZEN_SELECTION_RESIZE_HANDLE_STROKE_WIDTH_POINTS: f32 = 1.3; const FROZEN_MOSAIC_BLOCK_SIZE_PX: u32 = 12; const FROZEN_EDIT_HISTORY_LIMIT: usize = 24; +const FROZEN_BRUSH_STROKE_WIDTH_POINTS: f32 = 3.5; +const FROZEN_BRUSH_POINT_SPACING_MIN_POINTS: f32 = 0.25; +const FROZEN_BRUSH_PREVIEW_POINT_SPACING_MIN_POINTS: f32 = 0.1; +const FROZEN_BRUSH_MODELED_POINT_SPACING_MIN_POINTS: f32 = 0.25; +const FROZEN_BRUSH_MODEL_INPUT_RESPONSE_MIN: f32 = 0.12; +const FROZEN_BRUSH_MODEL_INPUT_RESPONSE_MAX: f32 = 0.96; +const FROZEN_BRUSH_MODEL_SPEED_FLOOR_POINTS_PER_SECOND: f32 = 12.0; +const FROZEN_BRUSH_MODEL_SPEED_CEILING_POINTS_PER_SECOND: f32 = 1_200.0; +const FROZEN_BRUSH_MODEL_OUTPUT_RATE_HZ: f32 = 180.0; +const FROZEN_BRUSH_MODEL_TIMESTEP_SECONDS: f32 = 1.0 / FROZEN_BRUSH_MODEL_OUTPUT_RATE_HZ; +const FROZEN_BRUSH_MODEL_SPRING_CONSTANT: f32 = 540.0; +const FROZEN_BRUSH_MODEL_DRAG_CONSTANT: f32 = 42.0; +#[cfg(test)] +const FROZEN_BRUSH_MODEL_SYNTHETIC_SAMPLE_INTERVAL_SECONDS: f32 = 1.0 / 120.0; +const FROZEN_BRUSH_MODEL_CURVE_TURN_RADIANS: f32 = 0.2; +const FROZEN_BRUSH_MODEL_CURVE_AMPLITUDE_POINTS: f32 = FROZEN_BRUSH_STROKE_WIDTH_POINTS * 0.08; +const FROZEN_BRUSH_MODEL_CURVE_RESPONSE_BOOST: f32 = 0.34; +const FROZEN_BRUSH_MODEL_FEATURE_TURN_RADIANS: f32 = 0.78; +const FROZEN_BRUSH_MODEL_SHARP_TURN_RADIANS: f32 = 1.45; +const FROZEN_BRUSH_MODEL_FEATURE_AMPLITUDE_POINTS: f32 = FROZEN_BRUSH_STROKE_WIDTH_POINTS * 0.22; +const FROZEN_BRUSH_STREAMLINE_RESPONSE_MIN: f32 = 0.18; +const FROZEN_BRUSH_STREAMLINE_RESPONSE_MAX: f32 = 0.78; +const FROZEN_BRUSH_STREAMLINE_DISTANCE_CEILING_POINTS: f32 = 6.0; +const FROZEN_BRUSH_PREVIEW_ROUNDING_PASSES: usize = 1; +const FROZEN_BRUSH_COMMIT_ROUNDING_PASSES: usize = 2; +const FROZEN_BRUSH_RENDER_SAMPLE_STEP_POINTS: f32 = 0.25; +const FROZEN_BRUSH_COLOR_RGBA: [u8; 4] = [255, 69, 58, 255]; const WINDOW_CAPTURE_MATTE_LIGHT_RGBA: image::Rgba = image::Rgba([246, 246, 246, 255]); const WINDOW_CAPTURE_MATTE_DARK_RGBA: image::Rgba = image::Rgba([24, 24, 24, 255]); const SCROLL_PREVIEW_WINDOW_WIDTH_POINTS: f64 = 260.0; @@ -891,6 +919,7 @@ pub struct OverlaySession { left_mouse_button_down: bool, left_mouse_button_down_monitor: Option, left_mouse_button_down_global: Option, + frozen_brush: FrozenBrushState, frozen_selection_drag: FrozenSelectionDragState, frozen_mosaic_drag: FrozenMosaicDragState, frozen_mosaic_undo_stack: Vec, @@ -1096,6 +1125,7 @@ impl OverlaySession { toolbar_left_button_down: false, toolbar_left_button_went_down: false, toolbar_left_button_went_up: false, toolbar_pointer_local: None, left_mouse_button_down: false, left_mouse_button_down_monitor: None, left_mouse_button_down_global: None, + frozen_brush: FrozenBrushState::default(), frozen_selection_drag: FrozenSelectionDragState::default(), frozen_mosaic_drag: FrozenMosaicDragState::default(), frozen_mosaic_undo_stack: Vec::new(), @@ -1587,6 +1617,7 @@ impl OverlaySession { self.state.frozen_mosaic_preview_rect = None; self.state.drag_rect = None; self.state.hovered_window_rect = None; + self.frozen_brush = FrozenBrushState::default(); self.frozen_selection_drag = FrozenSelectionDragState::default(); self.frozen_mosaic_drag = FrozenMosaicDragState::default(); @@ -1740,6 +1771,27 @@ impl OverlaySession { .flatten() } + fn frozen_capture_rect_for_monitor(&self, monitor: MonitorRect) -> Option { + self.state + .frozen_capture_rect + .or_else(|| Some(RectPoints::new(0, 0, monitor.width, monitor.height))) + .filter(|capture_rect| !capture_rect.is_empty()) + } + + fn frozen_brush_capture_target(&self) -> Option<(MonitorRect, RectPoints)> { + if !matches!(self.state.mode, OverlayMode::Frozen) + || self.scroll_capture.active + || self.state.frozen_image.is_none() + { + return None; + } + + let monitor = self.state.monitor?; + let capture_rect = self.frozen_capture_rect_for_monitor(monitor)?; + + Some((monitor, capture_rect)) + } + fn frozen_auto_center_available(&self) -> bool { self.frozen_capture_rect_drag_target().is_some() } @@ -1861,6 +1913,30 @@ impl OverlaySession { }; } + if self.toolbar_state.selected_tool == FrozenToolbarTool::Pen + && let Some((target_monitor, capture_rect)) = self.frozen_brush_capture_target() + { + if target_monitor != monitor { + return CursorIcon::Default; + } + if self.frozen_brush.active_stroke.is_some() { + return CursorIcon::Crosshair; + } + + let Some(cursor) = self.state.cursor else { + return CursorIcon::Default; + }; + let Some((cursor_x, cursor_y)) = monitor.local_u32(cursor) else { + return CursorIcon::Default; + }; + + return if capture_rect.contains((cursor_x, cursor_y)) { + CursorIcon::Crosshair + } else { + CursorIcon::Default + }; + } + let Some((target_monitor, capture_rect)) = self.frozen_selection_drag_target() else { return CursorIcon::Default; }; @@ -2137,6 +2213,765 @@ impl OverlaySession { self.state.frozen_mosaic_preview_rect = None; } + fn begin_frozen_brush_stroke(&mut self, global: GlobalPoint) -> bool { + if self.toolbar_state.selected_tool != FrozenToolbarTool::Pen { + return false; + } + if self.frozen_brush.active_stroke.is_some() { + return false; + } + + let Some((monitor, capture_rect)) = self.frozen_brush_capture_target() else { + return false; + }; + let Some((cursor_x, cursor_y)) = monitor.local_u32(global) else { + return false; + }; + + if !capture_rect.contains((cursor_x, cursor_y)) { + return false; + } + + let point = Pos2::new(cursor_x as f32, cursor_y as f32); + let sampled_at = Instant::now(); + + self.frozen_brush.redo_strokes.clear(); + self.sync_frozen_toolbar_state(); + + self.frozen_brush.active_stroke = + Some(Self::new_active_frozen_brush_stroke(point, sampled_at)); + + self.request_redraw_for_monitor(monitor); + + true + } + + fn update_frozen_brush_stroke(&mut self, global: GlobalPoint) -> bool { + let Some((monitor, capture_rect)) = self.frozen_brush_capture_target() else { + return false; + }; + let Some((cursor_x, cursor_y)) = monitor.local_u32(global) else { + return false; + }; + let Some(active_stroke) = self.frozen_brush.active_stroke.as_mut() else { + return false; + }; + let point = Self::clamped_point_in_capture_rect(capture_rect, cursor_x, cursor_y); + let sampled_at = Instant::now(); + + if let Some(previous) = active_stroke.raw_points.last().copied() + && previous.distance(point) < FROZEN_BRUSH_POINT_SPACING_MIN_POINTS + { + return false; + } + + Self::append_frozen_brush_raw_sample(active_stroke, point, sampled_at); + + true + } + + fn finish_frozen_brush_stroke(&mut self) -> bool { + let Some(stroke) = self.frozen_brush.active_stroke.take() else { + return false; + }; + + if stroke.points.is_empty() { + return false; + } + + self.frozen_brush + .committed_strokes + .push(FrozenBrushStroke { points: Self::finished_frozen_brush_points(&stroke) }); + self.sync_frozen_toolbar_state(); + + if let Some(monitor) = self.state.monitor { + self.request_redraw_for_monitor(monitor); + } + + true + } + + fn undo_frozen_brush_stroke(&mut self) -> bool { + let Some(stroke) = self.frozen_brush.committed_strokes.pop() else { + return false; + }; + + self.frozen_brush.redo_strokes.push(stroke); + self.sync_frozen_toolbar_state(); + + self.toolbar_state.needs_redraw = true; + + self.request_redraw_toolbar_window(); + + if let Some(monitor) = self.state.monitor { + self.request_redraw_for_monitor(monitor); + } + + true + } + + fn redo_frozen_brush_stroke(&mut self) -> bool { + let Some(stroke) = self.frozen_brush.redo_strokes.pop() else { + return false; + }; + + self.frozen_brush.committed_strokes.push(stroke); + self.sync_frozen_toolbar_state(); + + self.toolbar_state.needs_redraw = true; + + self.request_redraw_toolbar_window(); + + if let Some(monitor) = self.state.monitor { + self.request_redraw_for_monitor(monitor); + } + + true + } + + fn clamped_point_in_capture_rect( + capture_rect: RectPoints, + cursor_x: u32, + cursor_y: u32, + ) -> Pos2 { + let max_x = capture_rect.x.saturating_add(capture_rect.width.saturating_sub(1)); + let max_y = capture_rect.y.saturating_add(capture_rect.height.saturating_sub(1)); + + Pos2::new( + cursor_x.clamp(capture_rect.x, max_x) as f32, + cursor_y.clamp(capture_rect.y, max_y) as f32, + ) + } + + fn new_active_frozen_brush_stroke(point: Pos2, sampled_at: Instant) -> ActiveFrozenBrushStroke { + ActiveFrozenBrushStroke { + raw_points: vec![point], + points: vec![point], + model_state: FrozenBrushModelState { + filtered_input_point: point, + modeled_point: point, + modeled_velocity: Vec2::ZERO, + modeled_elapsed_seconds: 0.0, + }, + started_at: sampled_at, + last_sample_at: sampled_at, + } + } + + fn append_frozen_brush_raw_sample( + active_stroke: &mut ActiveFrozenBrushStroke, + point: Pos2, + sampled_at: Instant, + ) { + let delta_seconds = sampled_at + .saturating_duration_since(active_stroke.last_sample_at) + .as_secs_f32() + .max(1.0 / 240.0); + + active_stroke.raw_points.push(point); + + let snap_boost = Self::frozen_brush_feature_snap_boost(&active_stroke.raw_points); + let response = Self::frozen_brush_input_response_with_feature_boost( + &active_stroke.raw_points, + delta_seconds, + Self::frozen_brush_response_floor_boost(&active_stroke.raw_points, snap_boost), + ); + + active_stroke.model_state.filtered_input_point = Pos2::new( + active_stroke.model_state.filtered_input_point.x + + ((point.x - active_stroke.model_state.filtered_input_point.x) * response), + active_stroke.model_state.filtered_input_point.y + + ((point.y - active_stroke.model_state.filtered_input_point.y) * response), + ); + + let elapsed_seconds = + sampled_at.saturating_duration_since(active_stroke.started_at).as_secs_f32(); + + Self::advance_frozen_brush_model( + &mut active_stroke.points, + &mut active_stroke.model_state, + elapsed_seconds, + ); + + if snap_boost >= 1.0 { + active_stroke.model_state.modeled_point = Pos2::new( + active_stroke.model_state.modeled_point.x + + (point.x - active_stroke.model_state.modeled_point.x), + active_stroke.model_state.modeled_point.y + + (point.y - active_stroke.model_state.modeled_point.y), + ); + active_stroke.model_state.modeled_velocity *= 0.35; + + Self::push_modeled_frozen_brush_point( + &mut active_stroke.points, + active_stroke.model_state.modeled_point, + ); + } else if snap_boost > 0.0 { + active_stroke.model_state.modeled_point = Pos2::new( + active_stroke.model_state.modeled_point.x + + ((point.x - active_stroke.model_state.modeled_point.x) * 0.24), + active_stroke.model_state.modeled_point.y + + ((point.y - active_stroke.model_state.modeled_point.y) * 0.24), + ); + active_stroke.model_state.modeled_velocity *= 0.82; + + Self::push_modeled_frozen_brush_point( + &mut active_stroke.points, + active_stroke.model_state.modeled_point, + ); + } + + active_stroke.last_sample_at = sampled_at; + } + + #[cfg(test)] + fn frozen_brush_input_response(points: &[Pos2], delta_seconds: f32) -> f32 { + let snap_boost = Self::frozen_brush_feature_snap_boost(points); + + Self::frozen_brush_input_response_with_feature_boost( + points, + delta_seconds, + Self::frozen_brush_response_floor_boost(points, snap_boost), + ) + } + + fn frozen_brush_input_response_with_feature_boost( + points: &[Pos2], + delta_seconds: f32, + feature_boost: f32, + ) -> f32 { + let speed_points_per_second = points + .windows(2) + .last() + .map_or(0.0, |window| window[0].distance(window[1]) / delta_seconds); + let normalized_speed = ((speed_points_per_second + - FROZEN_BRUSH_MODEL_SPEED_FLOOR_POINTS_PER_SECOND) + / (FROZEN_BRUSH_MODEL_SPEED_CEILING_POINTS_PER_SECOND + - FROZEN_BRUSH_MODEL_SPEED_FLOOR_POINTS_PER_SECOND)) + .clamp(0.0, 1.0); + let base_response = FROZEN_BRUSH_MODEL_INPUT_RESPONSE_MIN + + ((FROZEN_BRUSH_MODEL_INPUT_RESPONSE_MAX - FROZEN_BRUSH_MODEL_INPUT_RESPONSE_MIN) + * normalized_speed); + + base_response.max(feature_boost) + } + + fn frozen_brush_response_floor_boost(points: &[Pos2], snap_boost: f32) -> f32 { + Self::frozen_brush_curve_response_boost(points).max(snap_boost) + } + + fn frozen_brush_preview_rounding_passes(points: &[Pos2]) -> usize { + if Self::frozen_brush_has_sustained_curve_context(points) { + FROZEN_BRUSH_PREVIEW_ROUNDING_PASSES + 1 + } else { + FROZEN_BRUSH_PREVIEW_ROUNDING_PASSES + } + } + + fn frozen_brush_curve_response_boost(points: &[Pos2]) -> f32 { + if Self::frozen_brush_has_sustained_curve_context(points) { + FROZEN_BRUSH_MODEL_CURVE_RESPONSE_BOOST + } else { + 0.0 + } + } + + fn frozen_brush_feature_snap_boost(points: &[Pos2]) -> f32 { + let len = points.len(); + + if len < 3 { + return 0.0; + } + + let previous = points[len - 2]; + let current = points[len - 1]; + let anchor = points[len - 3]; + let current_turn_angle = Self::frozen_brush_turn_angle(anchor, previous, current); + let current_amplitude = + Self::frozen_brush_point_to_segment_distance(previous, anchor, current); + + if current_turn_angle < FROZEN_BRUSH_MODEL_FEATURE_TURN_RADIANS + || current_amplitude < FROZEN_BRUSH_MODEL_FEATURE_AMPLITUDE_POINTS + { + return 0.0; + } + if Self::frozen_brush_has_sustained_curve_context(points) { + return 0.0; + } + if current_turn_angle >= FROZEN_BRUSH_MODEL_SHARP_TURN_RADIANS { + return 1.0; + } + if len < 4 { + return 0.0; + } + + let support = points[len - 4]; + let previous_turn = Self::frozen_brush_signed_turn(support, anchor, previous); + let current_turn = Self::frozen_brush_signed_turn(anchor, previous, current); + let previous_turn_angle = Self::frozen_brush_turn_angle(support, anchor, previous); + let previous_amplitude = + Self::frozen_brush_point_to_segment_distance(anchor, support, previous); + + if previous_turn * current_turn < 0.0 + && previous_turn_angle >= FROZEN_BRUSH_MODEL_FEATURE_TURN_RADIANS + && previous_amplitude >= FROZEN_BRUSH_MODEL_FEATURE_AMPLITUDE_POINTS + { + return 0.94; + } + + 0.0 + } + + fn frozen_brush_has_sustained_curve_context(points: &[Pos2]) -> bool { + let len = points.len(); + + if len < 5 { + return false; + } + + let recent_points = &points[len - 5..]; + let mut turn_sign: f32 = 0.0; + + for window in recent_points.windows(3) { + let signed_turn = Self::frozen_brush_signed_turn(window[0], window[1], window[2]); + let turn_angle = Self::frozen_brush_turn_angle(window[0], window[1], window[2]); + let amplitude = + Self::frozen_brush_point_to_segment_distance(window[1], window[0], window[2]); + + if turn_angle < FROZEN_BRUSH_MODEL_CURVE_TURN_RADIANS + || amplitude < FROZEN_BRUSH_MODEL_CURVE_AMPLITUDE_POINTS + { + return false; + } + if turn_sign.abs() <= f32::EPSILON { + turn_sign = signed_turn.signum(); + } else if signed_turn.signum() != turn_sign { + return false; + } + } + + true + } + + fn advance_frozen_brush_model( + points: &mut Vec, + model_state: &mut FrozenBrushModelState, + target_elapsed_seconds: f32, + ) { + if target_elapsed_seconds <= model_state.modeled_elapsed_seconds { + return; + } + + while model_state.modeled_elapsed_seconds + FROZEN_BRUSH_MODEL_TIMESTEP_SECONDS + < target_elapsed_seconds + { + Self::step_frozen_brush_model(points, model_state, FROZEN_BRUSH_MODEL_TIMESTEP_SECONDS); + } + + let remainder = target_elapsed_seconds - model_state.modeled_elapsed_seconds; + + if remainder > f32::EPSILON { + Self::step_frozen_brush_model(points, model_state, remainder); + } + } + + fn step_frozen_brush_model( + points: &mut Vec, + model_state: &mut FrozenBrushModelState, + delta_seconds: f32, + ) { + let displacement = model_state.filtered_input_point - model_state.modeled_point; + let acceleration = (displacement * FROZEN_BRUSH_MODEL_SPRING_CONSTANT) + - (model_state.modeled_velocity * FROZEN_BRUSH_MODEL_DRAG_CONSTANT); + + model_state.modeled_velocity += acceleration * delta_seconds; + model_state.modeled_point += model_state.modeled_velocity * delta_seconds; + model_state.modeled_elapsed_seconds += delta_seconds; + + Self::push_modeled_frozen_brush_point(points, model_state.modeled_point); + } + + fn finished_frozen_brush_points(stroke: &ActiveFrozenBrushStroke) -> Vec { + let source_points = + if stroke.points.len() >= 2 { &stroke.points } else { &stroke.raw_points }; + + Self::processed_frozen_brush_points( + source_points, + stroke.raw_points[0], + stroke.raw_points[stroke.raw_points.len().saturating_sub(1)], + FROZEN_BRUSH_COMMIT_ROUNDING_PASSES, + true, + ) + } + + fn rendered_frozen_brush_points(points: &[Pos2], sample_step: f32) -> Vec { + match points { + [] => Vec::new(), + [first] => vec![*first], + [first, second] => { + let sample_step = sample_step.max(0.1); + let mut rendered = vec![*first]; + + Self::append_frozen_brush_linear_segment( + &mut rendered, + *first, + *second, + sample_step, + ); + + rendered + }, + _ => { + let sample_step = sample_step.max(0.1); + let mut rendered = Vec::with_capacity(points.len() * 6); + + rendered.push(points[0]); + + for index in 0..points.len().saturating_sub(1) { + let previous = if index == 0 { points[0] } else { points[index - 1] }; + let start = points[index]; + let end = points[index + 1]; + let next = points + .get(index + 2) + .copied() + .unwrap_or(points[points.len().saturating_sub(1)]); + + Self::append_frozen_brush_curve_segment( + &mut rendered, + previous, + start, + end, + next, + sample_step, + ); + } + + rendered + }, + } + } + + fn active_frozen_brush_display_points(active_stroke: &ActiveFrozenBrushStroke) -> Vec { + let source_points = if active_stroke.points.len() >= 2 { + &active_stroke.points + } else { + &active_stroke.raw_points + }; + let rounding_passes = Self::frozen_brush_preview_rounding_passes(&active_stroke.raw_points); + + Self::processed_frozen_brush_points( + source_points, + active_stroke.raw_points[0], + active_stroke.raw_points[active_stroke.raw_points.len().saturating_sub(1)], + rounding_passes, + false, + ) + } + + fn preview_frozen_brush_points(active_stroke: &ActiveFrozenBrushStroke) -> Vec { + Self::active_frozen_brush_display_points(active_stroke) + } + + fn processed_frozen_brush_points( + source_points: &[Pos2], + start_point: Pos2, + end_point: Pos2, + rounding_passes: usize, + streamline: bool, + ) -> Vec { + match source_points { + [] => Vec::new(), + [_] | [_, _] => vec![start_point, end_point], + _ => { + let streamlined = if streamline { + Self::streamlined_frozen_brush_points(source_points) + } else { + source_points.to_vec() + }; + let mut rounded = + Self::rounded_open_frozen_brush_points(&streamlined, rounding_passes); + + if rounded.len() < 2 { + return vec![start_point, end_point]; + } + + rounded[0] = start_point; + + if let Some(last) = rounded.last_mut() { + *last = end_point; + } + + rounded + }, + } + } + + fn streamlined_frozen_brush_points(raw_points: &[Pos2]) -> Vec { + let Some(first) = raw_points.first().copied() else { + return Vec::new(); + }; + let mut streamlined = vec![first]; + let mut filtered = first; + + for window in raw_points.windows(2) { + let previous_raw = window[0]; + let current_raw = window[1]; + let normalized_distance = ((previous_raw.distance(current_raw) + - FROZEN_BRUSH_POINT_SPACING_MIN_POINTS) + / (FROZEN_BRUSH_STREAMLINE_DISTANCE_CEILING_POINTS + - FROZEN_BRUSH_POINT_SPACING_MIN_POINTS)) + .clamp(0.0, 1.0); + let response = FROZEN_BRUSH_STREAMLINE_RESPONSE_MIN + + ((FROZEN_BRUSH_STREAMLINE_RESPONSE_MAX - FROZEN_BRUSH_STREAMLINE_RESPONSE_MIN) + * normalized_distance); + + filtered = Pos2::new( + filtered.x + ((current_raw.x - filtered.x) * response), + filtered.y + ((current_raw.y - filtered.y) * response), + ); + + Self::push_processed_frozen_brush_point(&mut streamlined, filtered); + } + + streamlined[0] = raw_points[0]; + + let last_raw = raw_points[raw_points.len().saturating_sub(1)]; + + if let Some(last) = streamlined.last_mut() { + *last = last_raw; + } + + streamlined + } + + fn rounded_open_frozen_brush_points(points: &[Pos2], passes: usize) -> Vec { + if points.len() <= 2 || passes == 0 { + return points.to_vec(); + } + + let mut rounded = points.to_vec(); + + for _ in 0..passes { + if rounded.len() <= 2 { + break; + } + + let mut next = Vec::with_capacity((rounded.len() * 2).saturating_sub(2)); + + next.push(rounded[0]); + + for window in rounded.windows(2) { + let start = window[0]; + let end = window[1]; + let quarter = + Pos2::new((start.x * 0.75) + (end.x * 0.25), (start.y * 0.75) + (end.y * 0.25)); + let three_quarters = + Pos2::new((start.x * 0.25) + (end.x * 0.75), (start.y * 0.25) + (end.y * 0.75)); + + Self::push_processed_frozen_brush_point(&mut next, quarter); + Self::push_processed_frozen_brush_point(&mut next, three_quarters); + } + + let last = rounded[rounded.len().saturating_sub(1)]; + + if next.last().is_none_or(|point| { + point.distance(last) > FROZEN_BRUSH_PREVIEW_POINT_SPACING_MIN_POINTS + }) { + next.push(last); + } else if let Some(last_point) = next.last_mut() { + *last_point = last; + } + + rounded = next; + } + + rounded + } + + fn append_frozen_brush_curve_segment( + points: &mut Vec, + previous: Pos2, + start: Pos2, + end: Pos2, + next: Pos2, + sample_step: f32, + ) { + let approximate_length = start.distance(end); + let steps = ((approximate_length / sample_step).ceil().max(1.0)) as usize; + + for step in 1..=steps { + let t = step as f32 / steps as f32; + let point = Self::catmull_rom_frozen_brush_point(previous, start, end, next, t); + + Self::push_frozen_brush_sample_point(points, point); + } + } + + fn append_frozen_brush_linear_segment( + points: &mut Vec, + start: Pos2, + end: Pos2, + sample_step: f32, + ) { + let approximate_length = start.distance(end); + let steps = ((approximate_length / sample_step).ceil().max(1.0)) as usize; + + for step in 1..=steps { + let t = step as f32 / steps as f32; + let point = Pos2::new(start.x + (end.x - start.x) * t, start.y + (end.y - start.y) * t); + + Self::push_frozen_brush_sample_point(points, point); + } + } + + fn catmull_rom_frozen_brush_point( + previous: Pos2, + start: Pos2, + end: Pos2, + next: Pos2, + t: f32, + ) -> Pos2 { + const CENTRIPETAL_ALPHA: f32 = 0.5; + const MIN_PARAMETER_STEP: f32 = 1.0e-3; + + let t0 = 0.0; + let t1 = t0 + previous.distance(start).powf(CENTRIPETAL_ALPHA).max(MIN_PARAMETER_STEP); + let t2 = t1 + start.distance(end).powf(CENTRIPETAL_ALPHA).max(MIN_PARAMETER_STEP); + let t3 = t2 + end.distance(next).powf(CENTRIPETAL_ALPHA).max(MIN_PARAMETER_STEP); + let sample_t = t1 + ((t2 - t1) * t.clamp(0.0, 1.0)); + let a1 = Self::centripetal_catmull_rom_lerp(previous, start, t0, t1, sample_t); + let a2 = Self::centripetal_catmull_rom_lerp(start, end, t1, t2, sample_t); + let a3 = Self::centripetal_catmull_rom_lerp(end, next, t2, t3, sample_t); + let b1 = Self::centripetal_catmull_rom_lerp(a1, a2, t0, t2, sample_t); + let b2 = Self::centripetal_catmull_rom_lerp(a2, a3, t1, t3, sample_t); + + Self::centripetal_catmull_rom_lerp(b1, b2, t1, t2, sample_t) + } + + fn centripetal_catmull_rom_lerp( + start: Pos2, + end: Pos2, + start_t: f32, + end_t: f32, + sample_t: f32, + ) -> Pos2 { + let span = (end_t - start_t).max(f32::EPSILON); + let start_weight = (end_t - sample_t) / span; + let end_weight = (sample_t - start_t) / span; + + Pos2::new( + (start.x * start_weight) + (end.x * end_weight), + (start.y * start_weight) + (end.y * end_weight), + ) + } + + fn push_frozen_brush_sample_point(points: &mut Vec, point: Pos2) { + if points.last().is_none_or(|previous| previous.distance(point) > f32::EPSILON) { + points.push(point); + } + } + + fn push_modeled_frozen_brush_point(points: &mut Vec, point: Pos2) { + let Some(previous) = points.last().copied() else { + points.push(point); + + return; + }; + + if previous.distance(point) < f32::EPSILON { + return; + } + if previous.distance(point) < FROZEN_BRUSH_MODELED_POINT_SPACING_MIN_POINTS + && points.len() > 1 + { + if let Some(last) = points.last_mut() { + *last = point; + } + + return; + } + + points.push(point); + } + + fn push_processed_frozen_brush_point(points: &mut Vec, point: Pos2) { + let Some(previous) = points.last().copied() else { + points.push(point); + + return; + }; + + if previous.distance(point) < FROZEN_BRUSH_PREVIEW_POINT_SPACING_MIN_POINTS { + if let Some(last) = points.last_mut() { + *last = point; + } + + return; + } + + points.push(point); + } + + #[cfg(test)] + fn corrected_frozen_brush_points(points: &[Pos2]) -> Vec { + if points.len() <= 2 { + return points.to_vec(); + } + + let started_at = Instant::now(); + let mut stroke = Self::new_active_frozen_brush_stroke(points[0], started_at); + + for (index, point) in points.iter().copied().enumerate().skip(1) { + let sampled_at = started_at + + Duration::from_secs_f32( + index as f32 * FROZEN_BRUSH_MODEL_SYNTHETIC_SAMPLE_INTERVAL_SECONDS, + ); + + Self::append_frozen_brush_raw_sample(&mut stroke, point, sampled_at); + } + + Self::finished_frozen_brush_points(&stroke) + } + + fn frozen_brush_point_to_segment_distance(point: Pos2, start: Pos2, end: Pos2) -> f32 { + let segment_x = end.x - start.x; + let segment_y = end.y - start.y; + let segment_length_sq = (segment_x * segment_x) + (segment_y * segment_y); + + if segment_length_sq <= f32::EPSILON { + return point.distance(start); + } + + let t = (((point.x - start.x) * segment_x) + ((point.y - start.y) * segment_y)) + / segment_length_sq; + let t = t.clamp(0.0, 1.0); + let projection = Pos2::new(start.x + (segment_x * t), start.y + (segment_y * t)); + + point.distance(projection) + } + + fn frozen_brush_turn_angle(previous: Pos2, current: Pos2, next: Pos2) -> f32 { + let first = current - previous; + let second = next - current; + let first_length = first.length(); + let second_length = second.length(); + + if first_length <= f32::EPSILON || second_length <= f32::EPSILON { + return 0.0; + } + + let cosine = (first.dot(second) / (first_length * second_length)).clamp(-1.0, 1.0); + + cosine.acos() + } + + fn frozen_brush_signed_turn(previous: Pos2, current: Pos2, next: Pos2) -> f32 { + let first = current - previous; + let second = next - current; + + (first.x * second.y) - (first.y * second.x) + } + fn update_frozen_selection_drag_rect(&mut self, global: GlobalPoint) -> bool { if !self.frozen_selection_drag.active { return false; @@ -2667,6 +3502,290 @@ impl OverlaySession { } } + fn annotated_frozen_export_image(&self, mut export_image: RgbaImage) -> RgbaImage { + if self.frozen_brush.committed_strokes.is_empty() + && self.frozen_brush.active_stroke.is_none() + { + return export_image; + } + + let Some(monitor) = self.state.monitor else { + return export_image; + }; + let Some(capture_rect) = self.frozen_capture_rect_for_monitor(monitor) else { + return export_image; + }; + + Self::rasterize_frozen_brush_strokes( + &mut export_image, + monitor, + capture_rect, + &self.frozen_brush, + ); + + export_image + } + + fn rasterize_frozen_brush_strokes( + export_image: &mut RgbaImage, + monitor: MonitorRect, + capture_rect: RectPoints, + frozen_brush: &FrozenBrushState, + ) { + if export_image.width() == 0 || export_image.height() == 0 { + return; + } + + let radius = (FROZEN_BRUSH_STROKE_WIDTH_POINTS * monitor.scale_factor() * 0.5).max(1.0); + let color = image::Rgba(FROZEN_BRUSH_COLOR_RGBA); + let mut coverage_mask = + vec![0_u8; export_image.width() as usize * export_image.height() as usize]; + + for stroke in &frozen_brush.committed_strokes { + Self::rasterize_frozen_brush_stroke( + &mut coverage_mask, + export_image.width(), + export_image.height(), + stroke, + capture_rect, + monitor, + radius, + ); + } + + if let Some(active_stroke) = &frozen_brush.active_stroke { + let display_points = Self::active_frozen_brush_display_points(active_stroke); + + Self::rasterize_frozen_brush_points( + &mut coverage_mask, + export_image.width(), + export_image.height(), + &display_points, + capture_rect, + monitor, + radius, + ); + } + + Self::blend_frozen_brush_coverage_mask(export_image, &coverage_mask, color); + } + + fn rasterize_frozen_brush_stroke( + coverage_mask: &mut [u8], + export_width: u32, + export_height: u32, + stroke: &FrozenBrushStroke, + capture_rect: RectPoints, + monitor: MonitorRect, + radius: f32, + ) { + Self::rasterize_frozen_brush_points( + coverage_mask, + export_width, + export_height, + &stroke.points, + capture_rect, + monitor, + radius, + ); + } + + fn rasterize_frozen_brush_points( + coverage_mask: &mut [u8], + export_width: u32, + export_height: u32, + points: &[Pos2], + capture_rect: RectPoints, + monitor: MonitorRect, + radius: f32, + ) { + let rendered_points = + Self::rendered_frozen_brush_points(points, FROZEN_BRUSH_RENDER_SAMPLE_STEP_POINTS); + let Some(first) = rendered_points.first().copied() else { + return; + }; + let scale_factor = monitor.scale_factor(); + let mut previous = + Self::frozen_brush_point_to_export_pixels(first, capture_rect, scale_factor); + + Self::rasterize_frozen_brush_circle( + coverage_mask, + export_width, + export_height, + previous, + radius, + ); + + for point in rendered_points.into_iter().skip(1) { + let current = + Self::frozen_brush_point_to_export_pixels(point, capture_rect, scale_factor); + + Self::rasterize_frozen_brush_segment( + coverage_mask, + export_width, + export_height, + previous, + current, + radius, + ); + + previous = current; + } + } + + fn frozen_brush_point_to_export_pixels( + point: Pos2, + capture_rect: RectPoints, + scale_factor: f32, + ) -> Pos2 { + Pos2::new( + (point.x - capture_rect.x as f32) * scale_factor, + (point.y - capture_rect.y as f32) * scale_factor, + ) + } + + fn rasterize_frozen_brush_segment( + coverage_mask: &mut [u8], + export_width: u32, + export_height: u32, + start: Pos2, + end: Pos2, + radius: f32, + ) { + let delta = end - start; + let delta_len_sq = delta.length_sq(); + + if delta_len_sq <= f32::EPSILON { + Self::rasterize_frozen_brush_circle( + coverage_mask, + export_width, + export_height, + start, + radius, + ); + + return; + } + + let min_x = ((start.x.min(end.x) - radius - 0.5).floor().max(0.0)) as u32; + let min_y = ((start.y.min(end.y) - radius - 0.5).floor().max(0.0)) as u32; + let max_x = ((start.x.max(end.x) + radius + 0.5) + .ceil() + .min((export_width.saturating_sub(1)) as f32)) as u32; + let max_y = ((start.y.max(end.y) + radius + 0.5) + .ceil() + .min((export_height.saturating_sub(1)) as f32)) as u32; + + for y in min_y..=max_y { + for x in min_x..=max_x { + let sample = Pos2::new(x as f32 + 0.5, y as f32 + 0.5); + let projection = ((sample - start).dot(delta) / delta_len_sq).clamp(0.0, 1.0); + let nearest = start + delta * projection; + let coverage = Self::frozen_brush_coverage(sample.distance(nearest), radius); + + Self::update_frozen_brush_coverage_mask( + coverage_mask, + export_width, + x, + y, + coverage, + ); + } + } + } + + fn rasterize_frozen_brush_circle( + coverage_mask: &mut [u8], + export_width: u32, + export_height: u32, + center: Pos2, + radius: f32, + ) { + if export_width == 0 || export_height == 0 { + return; + } + + let min_x = ((center.x - radius - 0.5).floor().max(0.0)) as u32; + let min_y = ((center.y - radius - 0.5).floor().max(0.0)) as u32; + let max_x = + (center.x + radius + 0.5).ceil().min((export_width.saturating_sub(1)) as f32) as u32; + let max_y = + (center.y + radius + 0.5).ceil().min((export_height.saturating_sub(1)) as f32) as u32; + + if min_x > max_x || min_y > max_y { + return; + } + + for y in min_y..=max_y { + for x in min_x..=max_x { + let sample = Pos2::new(x as f32 + 0.5, y as f32 + 0.5); + let coverage = Self::frozen_brush_coverage(sample.distance(center), radius); + + Self::update_frozen_brush_coverage_mask( + coverage_mask, + export_width, + x, + y, + coverage, + ); + } + } + } + + fn frozen_brush_coverage(distance: f32, radius: f32) -> u8 { + ((radius + 0.5 - distance).clamp(0.0, 1.0) * 255.0).round() as u8 + } + + fn update_frozen_brush_coverage_mask( + coverage_mask: &mut [u8], + export_width: u32, + x: u32, + y: u32, + coverage: u8, + ) { + if coverage == 0 { + return; + } + + let index = y as usize * export_width as usize + x as usize; + + coverage_mask[index] = coverage_mask[index].max(coverage); + } + + fn blend_frozen_brush_coverage_mask( + export_image: &mut RgbaImage, + coverage_mask: &[u8], + color: image::Rgba, + ) { + let source_alpha = color[3] as f32 / 255.0; + + for (index, pixel) in export_image.pixels_mut().enumerate() { + let mask_alpha = coverage_mask[index]; + + if mask_alpha == 0 { + continue; + } + + let src_a = (mask_alpha as f32 / 255.0) * source_alpha; + let dst_a = pixel[3] as f32 / 255.0; + let out_a = src_a + dst_a * (1.0 - src_a); + + if out_a <= f32::EPSILON { + continue; + } + + for channel in 0..3 { + let src = color[channel] as f32 / 255.0; + let dst = pixel[channel] as f32 / 255.0; + let out = (src * src_a + dst * dst_a * (1.0 - src_a)) / out_a; + + pixel[channel] = (out * 255.0).round().clamp(0.0, 255.0) as u8; + } + + pixel[3] = (out_a * 255.0).round().clamp(0.0, 255.0) as u8; + } + } + #[cfg(target_os = "macos")] fn cropped_monitor_frozen_region_image( &self, @@ -2966,6 +4085,44 @@ impl OverlaySession { self.apply_frozen_mosaic_edit(preview_rect) } + fn frozen_undo_available(&self) -> bool { + match self.toolbar_state.selected_tool { + FrozenToolbarTool::Pen => !self.frozen_brush.committed_strokes.is_empty(), + FrozenToolbarTool::Mosaic => !self.frozen_mosaic_undo_stack.is_empty(), + _ => { + !self.frozen_brush.committed_strokes.is_empty() + || !self.frozen_mosaic_undo_stack.is_empty() + }, + } + } + + fn frozen_redo_available(&self) -> bool { + match self.toolbar_state.selected_tool { + FrozenToolbarTool::Pen => !self.frozen_brush.redo_strokes.is_empty(), + FrozenToolbarTool::Mosaic => !self.frozen_mosaic_redo_stack.is_empty(), + _ => { + !self.frozen_brush.redo_strokes.is_empty() + || !self.frozen_mosaic_redo_stack.is_empty() + }, + } + } + + fn perform_frozen_undo(&mut self) -> bool { + match self.toolbar_state.selected_tool { + FrozenToolbarTool::Pen => self.undo_frozen_brush_stroke(), + FrozenToolbarTool::Mosaic => self.undo_frozen_mosaic_edit(), + _ => self.undo_frozen_brush_stroke() || self.undo_frozen_mosaic_edit(), + } + } + + fn perform_frozen_redo(&mut self) -> bool { + match self.toolbar_state.selected_tool { + FrozenToolbarTool::Pen => self.redo_frozen_brush_stroke(), + FrozenToolbarTool::Mosaic => self.redo_frozen_mosaic_edit(), + _ => self.redo_frozen_brush_stroke() || self.redo_frozen_mosaic_edit(), + } + } + fn flatten_window_image_with_matte(image: &RgbaImage, matte: image::Rgba) -> RgbaImage { let mut out = image.clone(); @@ -3193,6 +4350,9 @@ impl OverlaySession { ) { if state == ElementState::Released && button == MouseButton::Left { self.commit_frozen_mosaic_drag(); + + let _ = self.finish_frozen_brush_stroke(); + self.stop_frozen_selection_drag(); self.sync_overlay_cursor_icons(); } @@ -3825,6 +4985,9 @@ impl OverlaySession { if should_trace_frozen_selection_drag_timing { self.trace_frozen_selection_drag_cursor_move(monitor, old_monitor, old_cursor, timing); } + if self.update_frozen_brush_stroke(global) { + self.request_redraw_for_monitor(monitor); + } OverlayControl::Continue } @@ -3876,6 +5039,9 @@ impl OverlaySession { if should_trace_frozen_selection_drag_timing { self.trace_frozen_selection_drag_cursor_move(monitor, old_monitor, old_cursor, timing); } + if self.update_frozen_brush_stroke(global) { + self.request_redraw_for_monitor(monitor); + } OverlayControl::Continue } @@ -4250,13 +5416,17 @@ impl OverlaySession { ElementState::Pressed => { let cursor = self.current_frozen_interaction_cursor(); - if !self.begin_frozen_selection_drag(cursor) { + if self.toolbar_state.selected_tool == FrozenToolbarTool::Pen { + let _ = self.begin_frozen_brush_stroke(cursor); + } else if !self.begin_frozen_selection_drag(cursor) { let _ = self.begin_frozen_mosaic_drag(cursor); } self.sync_overlay_cursor_icons(); }, ElementState::Released => { + let _ = self.finish_frozen_brush_stroke(); + self.stop_frozen_selection_drag(); self.stop_frozen_mosaic_drag(); self.sync_overlay_cursor_icons(); @@ -4992,7 +6162,9 @@ impl OverlaySession { .map(|session| session.export_image().clone()); } - self.cropped_frozen_capture_image().or_else(|| self.state.frozen_image.clone()) + self.cropped_frozen_capture_image() + .or_else(|| self.state.frozen_image.clone()) + .map(|export_image| self.annotated_frozen_export_image(export_image)) } #[cfg(target_os = "macos")] @@ -5278,8 +6450,8 @@ impl OverlaySession { fn sync_frozen_toolbar_state(&mut self) { self.toolbar_state.auto_center_available = self.frozen_auto_center_available(); - self.toolbar_state.undo_available = !self.frozen_mosaic_undo_stack.is_empty(); - self.toolbar_state.redo_available = !self.frozen_mosaic_redo_stack.is_empty(); + self.toolbar_state.undo_available = self.frozen_undo_available(); + self.toolbar_state.redo_available = self.frozen_redo_available(); self.toolbar_state.scroll_capture_active = self.scroll_capture.active; // Keep drag-region toolbar geometry stable across the authoritative frozen-capture handoff: // show the Scroll slot immediately, but keep it disabled until final_capture_ready flips. @@ -5946,6 +7118,7 @@ impl OverlaySession { self.frozen_capture_source, self.frozen_capture_source == FrozenCaptureSource::FullscreenFallback, frozen_toolbar_reserved_rect, + (!self.scroll_capture.active).then_some(&self.frozen_brush), toolbar_state, toolbar_input, ) { @@ -6087,18 +7260,18 @@ impl OverlaySession { fn handle_toolbar_action(&mut self, action: FrozenToolbarTool) -> OverlayControl { match action { - FrozenToolbarTool::AutoCenter => { - self.auto_center_frozen_capture_rect(); + FrozenToolbarTool::Undo => { + let _ = self.perform_frozen_undo(); OverlayControl::Continue }, - FrozenToolbarTool::Undo => { - self.undo_frozen_mosaic_edit(); + FrozenToolbarTool::Redo => { + let _ = self.perform_frozen_redo(); OverlayControl::Continue }, - FrozenToolbarTool::Redo => { - self.redo_frozen_mosaic_edit(); + FrozenToolbarTool::AutoCenter => { + self.auto_center_frozen_capture_rect(); OverlayControl::Continue }, diff --git a/packages/rsnap-overlay/src/overlay/cursor_runtime.rs b/packages/rsnap-overlay/src/overlay/cursor_runtime.rs index 59dece3e..ab2e0a94 100644 --- a/packages/rsnap-overlay/src/overlay/cursor_runtime.rs +++ b/packages/rsnap-overlay/src/overlay/cursor_runtime.rs @@ -1,7 +1,7 @@ #[allow(unused_imports)] use crate::overlay::{ - CURSOR_POLL_INTERVAL_MIN, DeviceCursorPointSource, Duration, Instant, - LIVE_HOVER_HIT_TEST_INTERVAL, OverlayMode, OverlaySession, + CURSOR_POLL_INTERVAL_MIN, DeviceCursorPointSource, Duration, GlobalPoint, Instant, + LIVE_HOVER_HIT_TEST_INTERVAL, MonitorRect, OverlayMode, OverlaySession, }; impl OverlaySession { @@ -13,10 +13,14 @@ impl OverlaySession { let interval = self.frozen_cursor_tracking_interval(self.state.monitor).max(CURSOR_POLL_INTERVAL_MIN); let now = Instant::now(); + let brush_sampling_active = self.frozen_brush.active_stroke.is_some(); + let poll_due = now.duration_since(self.last_frozen_cursor_poll_at) >= interval; self.schedule_egui_repaint_after(interval); - if let Some((monitor, global)) = self.last_fresh_event_cursor() { + if (!brush_sampling_active || !poll_due) + && let Some((monitor, global)) = self.last_fresh_event_cursor() + { let old_monitor = self.active_cursor_monitor(); if tracing::enabled!(tracing::Level::TRACE) { @@ -31,40 +35,11 @@ impl OverlaySession { return; } - let previous_drag_rect = self.state.drag_rect; - - self.update_cursor_state(monitor, global); - self.update_hud_window_position(monitor, global); - self.update_live_drag_rect(monitor, global); - self.update_frozen_selection_drag_rect(global); - self.update_frozen_mosaic_drag_rect(global); - self.sync_overlay_cursor_icons(); - self.force_apply_pending_hud_and_loupe_moves(); - self.request_redraw_hud_window(); - - if self.state.alt_held || self.loupe_window_visible { - self.request_redraw_loupe_window(); - } - - if let Some(old_monitor) = old_monitor - && old_monitor != monitor - { - self.request_redraw_for_monitor(old_monitor); - } - - if Self::live_overlay_redraw_needed_for_cursor_update( - old_monitor, - monitor, - previous_drag_rect, - self.state.drag_rect, - ) { - self.request_redraw_for_monitor(monitor); - } + self.apply_frozen_cursor_tracking_update(old_monitor, monitor, global); return; } - - if now.duration_since(self.last_frozen_cursor_poll_at) < interval { + if !poll_due { return; } @@ -88,6 +63,15 @@ impl OverlaySession { return; } + self.apply_frozen_cursor_tracking_update(old_monitor, monitor, global); + } + + fn apply_frozen_cursor_tracking_update( + &mut self, + old_monitor: Option, + monitor: MonitorRect, + global: GlobalPoint, + ) { let previous_drag_rect = self.state.drag_rect; self.update_cursor_state(monitor, global); @@ -95,6 +79,9 @@ impl OverlaySession { self.update_live_drag_rect(monitor, global); self.update_frozen_selection_drag_rect(global); self.update_frozen_mosaic_drag_rect(global); + + let brush_changed = self.update_frozen_brush_stroke(global); + self.sync_overlay_cursor_icons(); self.force_apply_pending_hud_and_loupe_moves(); self.request_redraw_hud_window(); @@ -109,12 +96,13 @@ impl OverlaySession { self.request_redraw_for_monitor(old_monitor); } - if Self::live_overlay_redraw_needed_for_cursor_update( - old_monitor, - monitor, - previous_drag_rect, - self.state.drag_rect, - ) { + if brush_changed + || Self::live_overlay_redraw_needed_for_cursor_update( + old_monitor, + monitor, + previous_drag_rect, + self.state.drag_rect, + ) { self.request_redraw_for_monitor(monitor); } } diff --git a/packages/rsnap-overlay/src/overlay/hud_runtime.rs b/packages/rsnap-overlay/src/overlay/hud_runtime.rs index 7795ea17..a892b8ca 100644 --- a/packages/rsnap-overlay/src/overlay/hud_runtime.rs +++ b/packages/rsnap-overlay/src/overlay/hud_runtime.rs @@ -127,6 +127,7 @@ impl OverlaySession { None, None, None, + None, )?; summary.renderer_draw_elapsed = Some(draw_started_at.elapsed()); diff --git a/packages/rsnap-overlay/src/overlay/rendering.rs b/packages/rsnap-overlay/src/overlay/rendering.rs index a4419473..76275457 100644 --- a/packages/rsnap-overlay/src/overlay/rendering.rs +++ b/packages/rsnap-overlay/src/overlay/rendering.rs @@ -18,16 +18,17 @@ use crate::overlay::{ BindingType, BlendState, Buffer, BufferBindingType, BufferSize, BufferUsages, ClippedPrimitive, Color32, ColorWrites, CompositeAlphaMode, Cow, CurrentSurfaceTexture, Device, Duration, Event, ExperimentalFeatures, Features, FilterMode, FontDefinitions, FontFamily, FrontFace, - FrozenCaptureSource, FrozenSelectionCorner, FrozenToolbarPointerState, FrozenToolbarState, - FullOutput, HudAnchor, HudTheme, Id, Instant, LayerId, LoadOp, MemoryHints, MipmapFilterMode, - MonitorRect, MultisampleState, Mutex, Order, OverlayMode, OverlaySession, OverlayState, - PhysicalSize, PipelineCompilationOptions, PointerButton, PolygonMode, Pos2, PowerPreference, - PresentMode, PrimitiveTopology, Queue, Rect, RectPoints, RenderPipeline, Renderer, Result, - SLOW_OP_WARN_RENDER, Sampler, SamplerBindingType, ScreenDescriptor, ShaderSource, ShaderStages, - SlowOperationLogger, StoreOp, Surface, SurfaceCapabilities, SurfaceFrameSkipReason, - SurfaceTexture, Texture, TextureAspect, TextureSampleType, TextureUsages, TextureView, - TextureViewDescriptor, TextureViewDimension, ThemeMode, ToolbarPlacement, Trace, Variant, Vec2, - ViewportId, Visuals, WindowId, WindowRendererPath, WrapErr, eyre, hud_helpers, mem, + FrozenBrushState, FrozenCaptureSource, FrozenSelectionCorner, FrozenToolbarPointerState, + FrozenToolbarState, FullOutput, HudAnchor, HudTheme, Id, Instant, LayerId, LoadOp, MemoryHints, + MipmapFilterMode, MonitorRect, MultisampleState, Mutex, Order, OverlayMode, OverlaySession, + OverlayState, PhysicalSize, PipelineCompilationOptions, PointerButton, PolygonMode, Pos2, + PowerPreference, PresentMode, PrimitiveTopology, Queue, Rect, RectPoints, RenderPipeline, + Renderer, Result, SLOW_OP_WARN_RENDER, Sampler, SamplerBindingType, ScreenDescriptor, + ShaderSource, ShaderStages, SlowOperationLogger, StoreOp, Surface, SurfaceCapabilities, + SurfaceFrameSkipReason, SurfaceTexture, Texture, TextureAspect, TextureSampleType, + TextureUsages, TextureView, TextureViewDescriptor, TextureViewDimension, ThemeMode, + ToolbarPlacement, Trace, Variant, Vec2, ViewportId, Visuals, WindowId, WindowRendererPath, + WrapErr, eyre, hud_helpers, mem, }; #[derive(Clone, Copy, Debug, Eq, PartialEq)] @@ -691,6 +692,7 @@ impl WindowRenderer { frozen_capture_source: FrozenCaptureSource, frozen_capture_is_fullscreen_fallback: bool, frozen_toolbar_reserved_rect: Option, + frozen_brush_state: Option<&FrozenBrushState>, selection_flow_geometry_cache: &mut SelectionFlowGeometryCache, selection_dashed_border_cache: &mut SelectionDashedBorderCache, mut toolbar_state: Option<&mut FrozenToolbarState>, @@ -787,6 +789,7 @@ impl WindowRenderer { frozen_selection_resize_handles_enabled, frozen_capture_source, frozen_toolbar_reserved_rect, + frozen_brush_state, frozen_capture_is_fullscreen_fallback, selection_flow_enabled, selection_flow_stroke_width_px, @@ -1319,6 +1322,7 @@ impl WindowRenderer { frozen_capture_source: FrozenCaptureSource, frozen_capture_is_fullscreen_fallback: bool, frozen_toolbar_reserved_rect: Option, + frozen_brush_state: Option<&FrozenBrushState>, toolbar_state: Option<&mut FrozenToolbarState>, toolbar_pointer: Option, ) -> Result<()> { @@ -1381,6 +1385,7 @@ impl WindowRenderer { frozen_capture_source, frozen_capture_is_fullscreen_fallback, frozen_toolbar_reserved_rect, + frozen_brush_state, &mut selection_flow_cache, &mut selection_dashed_border_cache, toolbar_state, diff --git a/packages/rsnap-overlay/src/overlay/rendering/affordances.rs b/packages/rsnap-overlay/src/overlay/rendering/affordances.rs index 4f9b4e2c..45383acf 100644 --- a/packages/rsnap-overlay/src/overlay/rendering/affordances.rs +++ b/packages/rsnap-overlay/src/overlay/rendering/affordances.rs @@ -9,7 +9,8 @@ use crate::overlay::rendering::{ }; #[allow(unused_imports)] use crate::overlay::{ - self, Align, Align2, Area, Color32, CornerRadius, FROZEN_SELECTION_DASHED_BORDER_WIDTH_PX, + self, Align, Align2, Area, Color32, CornerRadius, FROZEN_BRUSH_COLOR_RGBA, + FROZEN_BRUSH_STROKE_WIDTH_POINTS, FROZEN_SELECTION_DASHED_BORDER_WIDTH_PX, FROZEN_SELECTION_RESIZE_HANDLE_CENTER_DOT_RADIUS_POINTS, FROZEN_SELECTION_RESIZE_HANDLE_CORNER_KEEPOUT_POINTS, FROZEN_SELECTION_RESIZE_HANDLE_HIT_OFFSET_POINTS, @@ -18,13 +19,13 @@ use crate::overlay::{ FROZEN_SELECTION_RESIZE_HANDLE_OUTER_RADIUS_POINTS, FROZEN_SELECTION_RESIZE_HANDLE_STROKE_WIDTH_POINTS, FROZEN_SELECTION_SCRIM_ALPHA_DARK, FROZEN_SELECTION_SCRIM_ALPHA_LIGHT, FROZEN_TOOLBAR_BUTTON_SIZE_POINTS, - FROZEN_TOOLBAR_ITEM_SPACING_POINTS, FontFamily, FontId, FrozenCaptureSource, + FROZEN_TOOLBAR_ITEM_SPACING_POINTS, FontFamily, FontId, FrozenBrushState, FrozenCaptureSource, FrozenSelectionCorner, FrozenToolbarPointerState, FrozenToolbarState, FrozenToolbarTool, HUD_PILL_CORNER_RADIUS_POINTS, HUD_PILL_INNER_MARGIN_X_POINTS, HUD_PILL_INNER_MARGIN_Y_POINTS, HUD_PILL_STROKE_WIDTH_POINTS, HudPillGeometry, HudTheme, Id, LIVE_DRAG_SELECTION_SCRIM_ALPHA_DARK, LIVE_DRAG_SELECTION_SCRIM_ALPHA_LIGHT, LIVE_DRAG_START_THRESHOLD_PX, LayerId, Layout, Mesh, MonitorRect, Order, OverlayMode, - OverlayState, Painter, Pos2, Rect, RectPoints, SELECTION_DASHED_BORDER_ALPHA, + OverlaySession, OverlayState, Painter, Pos2, Rect, RectPoints, SELECTION_DASHED_BORDER_ALPHA, SELECTION_DASHED_BORDER_DASH_LENGTH_PX, SELECTION_DASHED_BORDER_GAP_LENGTH_PX, SELECTION_DASHED_BORDER_WIDTH_PX, SELECTION_FLOW_CORE_FLOW_WIDTH, SELECTION_FLOW_CORNER_RADIUS_PX, SELECTION_FLOW_FLOW_BOOST, SELECTION_FLOW_LIGHT_PALETTE, @@ -137,6 +138,7 @@ impl WindowRenderer { frozen_selection_resize_handles_enabled: bool, frozen_capture_source: FrozenCaptureSource, frozen_toolbar_reserved_rect: Option, + frozen_brush_state: Option<&FrozenBrushState>, _frozen_capture_is_fullscreen_fallback: bool, _selection_flow_enabled: bool, _selection_flow_stroke_width_px: f32, @@ -199,10 +201,99 @@ impl WindowRenderer { selection_dashed_border_cache, ); } + if let Some(frozen_brush_state) = frozen_brush_state { + let brush_painter = painter.with_clip_rect(rect); + + has_affordance |= Self::render_frozen_brush_strokes(&brush_painter, frozen_brush_state); + } has_affordance } + pub(in crate::overlay) fn render_frozen_brush_strokes( + painter: &Painter, + frozen_brush_state: &FrozenBrushState, + ) -> bool { + let color = Color32::from_rgba_unmultiplied( + FROZEN_BRUSH_COLOR_RGBA[0], + FROZEN_BRUSH_COLOR_RGBA[1], + FROZEN_BRUSH_COLOR_RGBA[2], + FROZEN_BRUSH_COLOR_RGBA[3], + ); + let radius = FROZEN_BRUSH_STROKE_WIDTH_POINTS * 0.5; + let mut drew = false; + + for brush in &frozen_brush_state.committed_strokes { + drew |= Self::paint_frozen_brush_stroke(painter, &brush.points, radius, color); + } + + if let Some(active_stroke) = &frozen_brush_state.active_stroke { + let preview_points = OverlaySession::preview_frozen_brush_points(active_stroke); + + drew |= Self::paint_frozen_brush_stroke(painter, &preview_points, radius, color); + } + + drew + } + + fn paint_frozen_brush_stroke( + painter: &Painter, + points: &[Pos2], + radius: f32, + color: Color32, + ) -> bool { + let rendered_points = OverlaySession::rendered_frozen_brush_points( + points, + overlay::FROZEN_BRUSH_RENDER_SAMPLE_STEP_POINTS, + ); + + match rendered_points.as_slice() { + [] => false, + [point] => { + painter.circle_filled(*point, radius, color); + + true + }, + _ => { + let first = rendered_points[0]; + let second = rendered_points[1]; + let penultimate = rendered_points[rendered_points.len().saturating_sub(2)]; + let last = rendered_points[rendered_points.len().saturating_sub(1)]; + let total_length = rendered_points + .windows(2) + .fold(0.0, |length, window| length + window[0].distance(window[1])); + let max_cap_inset = (total_length * 0.5 - 0.01).max(0.0); + let cap_inset = radius.min(max_cap_inset); + let mut body_points = rendered_points.clone(); + + if cap_inset > 0.0 { + let start_delta = second - first; + + if start_delta.length_sq() > f32::EPSILON { + let start_dir = start_delta.normalized(); + + body_points[0] = first + (start_dir * cap_inset); + } + + let end_delta = last - penultimate; + + if end_delta.length_sq() > f32::EPSILON { + let end_dir = end_delta.normalized(); + let last_index = body_points.len().saturating_sub(1); + + body_points[last_index] = last - (end_dir * cap_inset); + } + } + + painter.add(Shape::line(body_points, Stroke::new(radius * 2.0, color))); + painter.circle_filled(first, radius, color); + painter.circle_filled(last, radius, color); + + true + }, + } + } + pub(in crate::overlay) fn frozen_capture_focus_rect( state: &OverlayState, screen_rect: Rect, diff --git a/packages/rsnap-overlay/src/overlay/session_state.rs b/packages/rsnap-overlay/src/overlay/session_state.rs index c5dabe70..1ee09e24 100644 --- a/packages/rsnap-overlay/src/overlay/session_state.rs +++ b/packages/rsnap-overlay/src/overlay/session_state.rs @@ -183,6 +183,35 @@ impl Default for FrozenToolbarState { } } +#[derive(Clone, Debug, Default, PartialEq)] +pub(super) struct FrozenBrushStroke { + pub(super) points: Vec, +} + +#[derive(Clone, Copy, Debug)] +pub(super) struct FrozenBrushModelState { + pub(super) filtered_input_point: Pos2, + pub(super) modeled_point: Pos2, + pub(super) modeled_velocity: Vec2, + pub(super) modeled_elapsed_seconds: f32, +} + +#[derive(Clone, Debug)] +pub(super) struct ActiveFrozenBrushStroke { + pub(super) raw_points: Vec, + pub(super) points: Vec, + pub(super) model_state: FrozenBrushModelState, + pub(super) started_at: Instant, + pub(super) last_sample_at: Instant, +} + +#[derive(Debug, Default)] +pub(super) struct FrozenBrushState { + pub(super) committed_strokes: Vec, + pub(super) redo_strokes: Vec, + pub(super) active_stroke: Option, +} + #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub(super) struct FrozenSelectionDragState { pub(super) active: bool, diff --git a/packages/rsnap-overlay/src/overlay/tests.rs b/packages/rsnap-overlay/src/overlay/tests.rs index 0542b1b1..754df1e1 100644 --- a/packages/rsnap-overlay/src/overlay/tests.rs +++ b/packages/rsnap-overlay/src/overlay/tests.rs @@ -51,15 +51,16 @@ use crate::overlay::PngAction; #[cfg(target_os = "macos")] use crate::overlay::session_state::ScrollCaptureLiveFrame; use crate::overlay::{ - self, FrozenSelectionDragState, FrozenToolbarState, FrozenToolbarTool, - HUD_LOUPE_STRIP_GAP_POINTS, HudRedrawSummary, HudTheme, OCCLUDED_FRAME_REDRAW_RETRY_WINDOW, - OverlaySession, Pos2, Rect, SCROLL_CAPTURE_SAMPLE_INTERVAL, - SELECTION_DASHED_BORDER_DASH_LENGTH_PX, SELECTION_DASHED_BORDER_GAP_LENGTH_PX, - SELECTION_DASHED_BORDER_WIDTH_PX, SELECTION_SIZE_BADGE_GAP_PX, - SELECTION_SIZE_BADGE_INSIDE_MARGIN_PX, SELECTION_SIZE_BADGE_SCREEN_MARGIN_PX, - SelectionDashedBorderCache, SelectionDashedBorderMetrics, SelectionFlowGeometryCache, - SelectionSizeBadgeTarget, SurfaceFrameSkipReason, TOOLBAR_CAPTURE_GAP_PX, - TOOLBAR_SCREEN_MARGIN_PX, ToolbarPlacement, Vec2, WindowRenderer, hud_helpers, + self, ActiveFrozenBrushStroke, FROZEN_BRUSH_COLOR_RGBA, FrozenBrushModelState, + FrozenSelectionDragState, FrozenToolbarState, FrozenToolbarTool, HUD_LOUPE_STRIP_GAP_POINTS, + HudRedrawSummary, HudTheme, OCCLUDED_FRAME_REDRAW_RETRY_WINDOW, OverlaySession, Pos2, Rect, + SCROLL_CAPTURE_SAMPLE_INTERVAL, SELECTION_DASHED_BORDER_DASH_LENGTH_PX, + SELECTION_DASHED_BORDER_GAP_LENGTH_PX, SELECTION_DASHED_BORDER_WIDTH_PX, + SELECTION_SIZE_BADGE_GAP_PX, SELECTION_SIZE_BADGE_INSIDE_MARGIN_PX, + SELECTION_SIZE_BADGE_SCREEN_MARGIN_PX, SelectionDashedBorderCache, + SelectionDashedBorderMetrics, SelectionFlowGeometryCache, SelectionSizeBadgeTarget, + SurfaceFrameSkipReason, TOOLBAR_CAPTURE_GAP_PX, TOOLBAR_SCREEN_MARGIN_PX, ToolbarPlacement, + Vec2, WindowRenderer, hud_helpers, }; #[cfg(target_os = "macos")] use crate::overlay::{ @@ -381,6 +382,455 @@ fn begin_png_action_copies_preview_render_image_during_active_scroll_capture() { assert_eq!(session.state.error_message.as_deref(), Some("Copying...")); } +#[test] +fn current_export_image_includes_frozen_brush_strokes() { + let monitor = test_monitor(); + let mut session = OverlaySession::new(); + + session.state.begin_freeze(monitor); + session + .state + .finish_freeze(monitor, image::RgbaImage::from_pixel(8, 8, Rgba([12, 34, 56, 255]))); + + session.state.frozen_capture_rect = Some(RectPoints::new(0, 0, 8, 8)); + session.authoritative_frozen_capture_ready = true; + session.toolbar_state.selected_tool = FrozenToolbarTool::Pen; + + assert!(session.begin_frozen_brush_stroke(GlobalPoint::new(2, 2))); + assert!(session.update_frozen_brush_stroke(GlobalPoint::new(5, 2))); + assert!(session.finish_frozen_brush_stroke()); + + let export_image = session.current_export_image().expect("annotated export image"); + + assert_eq!(export_image.get_pixel(7, 7), &Rgba([12, 34, 56, 255])); + assert_eq!(export_image.get_pixel(2, 2), &Rgba(FROZEN_BRUSH_COLOR_RGBA)); +} + +#[test] +fn frozen_brush_undo_and_redo_update_export_image() { + let monitor = test_monitor(); + let mut session = OverlaySession::new(); + + session.state.begin_freeze(monitor); + session + .state + .finish_freeze(monitor, image::RgbaImage::from_pixel(8, 8, Rgba([12, 34, 56, 255]))); + + session.state.frozen_capture_rect = Some(RectPoints::new(0, 0, 8, 8)); + session.authoritative_frozen_capture_ready = true; + session.toolbar_state.selected_tool = FrozenToolbarTool::Pen; + + assert!(session.begin_frozen_brush_stroke(GlobalPoint::new(3, 3))); + assert!(session.finish_frozen_brush_stroke()); + assert!(session.undo_frozen_brush_stroke()); + + let undone = session.current_export_image().expect("undo export image"); + + assert_eq!(undone.get_pixel(3, 3), &Rgba([12, 34, 56, 255])); + assert!(session.redo_frozen_brush_stroke()); + + let redone = session.current_export_image().expect("redo export image"); + + assert_eq!(redone.get_pixel(3, 3), &Rgba(FROZEN_BRUSH_COLOR_RGBA)); +} + +#[test] +fn current_export_image_antialiases_frozen_brush_edges() { + let monitor = test_monitor(); + let background = Rgba([240, 240, 240, 255]); + let mut session = OverlaySession::new(); + + session.state.begin_freeze(monitor); + session.state.finish_freeze(monitor, image::RgbaImage::from_pixel(16, 16, background)); + + session.state.frozen_capture_rect = Some(RectPoints::new(0, 0, 16, 16)); + session.authoritative_frozen_capture_ready = true; + session.toolbar_state.selected_tool = FrozenToolbarTool::Pen; + + assert!(session.begin_frozen_brush_stroke(GlobalPoint::new(3, 3))); + assert!(session.update_frozen_brush_stroke(GlobalPoint::new(12, 12))); + assert!(session.finish_frozen_brush_stroke()); + + let export_image = session.current_export_image().expect("annotated export image"); + let has_antialiased_edge = export_image + .pixels() + .any(|pixel| pixel != &background && pixel != &Rgba(FROZEN_BRUSH_COLOR_RGBA)); + + assert!(has_antialiased_edge, "expected blended edge pixels around the exported brush"); +} + +fn significant_y_direction_reversals(points: &[Pos2], min_delta: f32) -> usize { + let mut last_direction = 0_i8; + let mut reversals = 0; + + for window in points.windows(2) { + let delta_y = window[1].y - window[0].y; + let direction = if delta_y > min_delta { + 1 + } else if delta_y < -min_delta { + -1 + } else { + 0 + }; + + if direction == 0 { + continue; + } + if last_direction != 0 && direction != last_direction { + reversals += 1; + } + + last_direction = direction; + } + + reversals +} + +#[test] +fn rendered_frozen_brush_points_round_corners_into_a_curve() { + let points = [Pos2::new(1.0, 1.0), Pos2::new(1.0, 5.0), Pos2::new(5.0, 5.0)]; + let rendered = OverlaySession::rendered_frozen_brush_points( + &points, + overlay::FROZEN_BRUSH_RENDER_SAMPLE_STEP_POINTS, + ); + + assert_eq!(rendered.first().copied(), Some(points[0])); + assert_eq!(rendered.last().copied(), Some(points[2])); + assert!(rendered.len() > points.len()); + assert!(rendered.iter().any(|point| { + (point.x - points[0].x).abs() > f32::EPSILON && (point.y - points[2].y).abs() > f32::EPSILON + })); +} + +#[test] +fn corrected_frozen_brush_points_preserve_open_stroke_endpoints() { + let points = [Pos2::new(1.0, 1.0), Pos2::new(4.0, 2.0), Pos2::new(7.0, 6.0)]; + let corrected = OverlaySession::corrected_frozen_brush_points(&points); + + assert_eq!(corrected.first().copied(), Some(points[0])); + assert_eq!(corrected.last().copied(), Some(points[2])); + assert!(corrected.len() >= 2); +} + +#[test] +fn corrected_frozen_brush_points_keep_annotation_loops_open() { + let points = [ + Pos2::new(2.0, 0.0), + Pos2::new(6.0, 1.0), + Pos2::new(8.0, 4.0), + Pos2::new(7.0, 8.0), + Pos2::new(3.0, 9.0), + Pos2::new(0.0, 6.0), + Pos2::new(1.5, 1.0), + ]; + let corrected = OverlaySession::corrected_frozen_brush_points(&points); + + assert!(corrected.len() >= 2); + assert_eq!(corrected.first().copied(), Some(points[0])); + assert_eq!(corrected.last().copied(), Some(points[6])); + assert_ne!(corrected.first().copied(), corrected.last().copied()); +} + +#[test] +fn corrected_frozen_brush_points_suppress_small_local_dent() { + let points = [ + Pos2::new(0.0, 0.0), + Pos2::new(5.0, 0.4), + Pos2::new(8.0, -1.6), + Pos2::new(11.0, 0.6), + Pos2::new(16.0, 0.3), + ]; + let corrected = OverlaySession::corrected_frozen_brush_points(&points); + let deepest_y = corrected.iter().fold(f32::INFINITY, |deepest, point| deepest.min(point.y)); + + assert_eq!(corrected.first().copied(), Some(points[0])); + assert_eq!(corrected.last().copied(), Some(points[4])); + assert!(deepest_y > -0.8, "expected final stroke to smooth away the local dent: {corrected:?}"); +} + +#[test] +fn corrected_frozen_brush_points_preserve_monotonic_arc_trend() { + let points = [ + Pos2::new(0.0, 0.0), + Pos2::new(3.0, 1.8), + Pos2::new(6.0, 3.8), + Pos2::new(9.0, 5.1), + Pos2::new(11.0, 4.6), + Pos2::new(14.0, 6.4), + Pos2::new(18.0, 8.8), + ]; + let corrected = OverlaySession::corrected_frozen_brush_points(&points); + + assert_eq!(corrected.first().copied(), Some(points[0])); + assert_eq!(corrected.last().copied(), Some(points[6])); + assert!(corrected.windows(2).all(|pair| pair[1].y + 0.05 >= pair[0].y)); +} + +#[test] +fn corrected_frozen_brush_points_preserve_small_wave_backbone() { + let points = [ + Pos2::new(0.0, 0.0), + Pos2::new(4.0, 1.8), + Pos2::new(8.0, -1.7), + Pos2::new(12.0, 1.9), + Pos2::new(16.0, -1.8), + Pos2::new(20.0, 1.7), + Pos2::new(24.0, 0.0), + ]; + let corrected = OverlaySession::corrected_frozen_brush_points(&points); + let reversals = significant_y_direction_reversals(&corrected, 0.12); + let (min_y, max_y) = corrected.iter().fold((f32::INFINITY, f32::NEG_INFINITY), |acc, point| { + (acc.0.min(point.y), acc.1.max(point.y)) + }); + + assert_eq!(corrected.first().copied(), Some(points[0])); + assert_eq!(corrected.last().copied(), Some(points[6])); + assert!( + reversals >= 3, + "expected the corrected stroke to keep the wave skeleton: {corrected:?}" + ); + assert!( + max_y - min_y >= 1.0, + "expected the corrected stroke to retain visible wave amplitude: {corrected:?}" + ); +} + +#[test] +fn preview_frozen_brush_points_keep_live_modeled_path_before_commit() { + let active_stroke = ActiveFrozenBrushStroke { + raw_points: vec![ + Pos2::new(0.0, 0.0), + Pos2::new(4.0, 6.0), + Pos2::new(8.0, -2.0), + Pos2::new(12.0, 4.0), + ], + points: vec![Pos2::new(0.0, 0.0), Pos2::new(6.0, 2.0), Pos2::new(12.0, 4.0)], + model_state: FrozenBrushModelState { + filtered_input_point: Pos2::new(12.0, 4.0), + modeled_point: Pos2::new(12.0, 4.0), + modeled_velocity: Vec2::ZERO, + modeled_elapsed_seconds: 0.03, + }, + started_at: Instant::now(), + last_sample_at: Instant::now(), + }; + let preview = OverlaySession::preview_frozen_brush_points(&active_stroke); + let committed = OverlaySession::corrected_frozen_brush_points(&active_stroke.raw_points); + + assert_eq!(preview.first().copied(), active_stroke.raw_points.first().copied()); + assert_eq!(preview.last().copied(), active_stroke.raw_points.last().copied()); + assert_ne!(preview, active_stroke.points); + assert_ne!(preview, committed); +} + +#[test] +fn preview_frozen_brush_points_follow_modeled_centerline_instead_of_raw_wobble() { + let active_stroke = ActiveFrozenBrushStroke { + raw_points: vec![ + Pos2::new(0.0, 0.0), + Pos2::new(1.0, 0.55), + Pos2::new(2.0, -0.50), + Pos2::new(3.0, 0.48), + Pos2::new(4.0, -0.42), + Pos2::new(5.0, 0.36), + Pos2::new(6.0, -0.28), + Pos2::new(7.0, 0.18), + Pos2::new(8.0, 0.0), + ], + points: vec![Pos2::new(0.0, 0.0), Pos2::new(4.0, 0.04), Pos2::new(8.0, 0.0)], + model_state: FrozenBrushModelState { + filtered_input_point: Pos2::new(8.0, 0.0), + modeled_point: Pos2::new(8.0, 0.0), + modeled_velocity: Vec2::ZERO, + modeled_elapsed_seconds: 0.05, + }, + started_at: Instant::now(), + last_sample_at: Instant::now(), + }; + let preview = OverlaySession::preview_frozen_brush_points(&active_stroke); + let (min_y, max_y) = preview.iter().fold((f32::INFINITY, f32::NEG_INFINITY), |acc, point| { + (acc.0.min(point.y), acc.1.max(point.y)) + }); + + assert_eq!(preview.first().copied(), active_stroke.raw_points.first().copied()); + assert_eq!(preview.last().copied(), active_stroke.raw_points.last().copied()); + assert!( + max_y - min_y <= 0.20, + "expected preview to stay close to the modeled centerline instead of exposing raw wobble: {preview:?}" + ); +} + +#[test] +fn rendered_live_frozen_brush_wave_preview_avoids_hard_inflection_kinks() { + let raw_points = [ + Pos2::new(0.0, 0.0), + Pos2::new(4.0, 1.8), + Pos2::new(8.0, -1.7), + Pos2::new(12.0, 1.9), + Pos2::new(16.0, -1.8), + Pos2::new(20.0, 1.7), + Pos2::new(24.0, 0.0), + ]; + let started_at = Instant::now(); + let mut stroke = OverlaySession::new_active_frozen_brush_stroke(raw_points[0], started_at); + + for (index, point) in raw_points.iter().copied().enumerate().skip(1) { + let sampled_at = started_at + + Duration::from_secs_f32( + index as f32 * overlay::FROZEN_BRUSH_MODEL_SYNTHETIC_SAMPLE_INTERVAL_SECONDS, + ); + + OverlaySession::append_frozen_brush_raw_sample(&mut stroke, point, sampled_at); + } + + let preview = OverlaySession::preview_frozen_brush_points(&stroke); + let rendered = OverlaySession::rendered_frozen_brush_points( + &preview, + overlay::FROZEN_BRUSH_RENDER_SAMPLE_STEP_POINTS, + ); + let max_turn_angle = rendered.windows(3).fold(0.0_f32, |max_turn, window| { + max_turn.max(OverlaySession::frozen_brush_turn_angle(window[0], window[1], window[2])) + }); + + assert!( + significant_y_direction_reversals(&rendered, 0.12) >= 2, + "expected live preview to keep visible oscillation: {rendered:?}" + ); + assert!( + max_turn_angle <= 0.48, + "expected live preview to round inflections instead of producing hard kinks: {rendered:?}" + ); +} + +#[test] +fn rendered_live_frozen_brush_arc_preview_avoids_corner_snap() { + let raw_points = [ + Pos2::new(0.0, 0.0), + Pos2::new(2.4, 0.2), + Pos2::new(4.7, 1.1), + Pos2::new(6.6, 2.8), + Pos2::new(7.7, 5.1), + Pos2::new(7.8, 7.8), + Pos2::new(6.8, 10.2), + Pos2::new(4.9, 12.0), + ]; + let started_at = Instant::now(); + let mut stroke = OverlaySession::new_active_frozen_brush_stroke(raw_points[0], started_at); + + for (index, point) in raw_points.iter().copied().enumerate().skip(1) { + let sampled_at = started_at + + Duration::from_secs_f32( + index as f32 * overlay::FROZEN_BRUSH_MODEL_SYNTHETIC_SAMPLE_INTERVAL_SECONDS, + ); + + OverlaySession::append_frozen_brush_raw_sample(&mut stroke, point, sampled_at); + } + + let preview = OverlaySession::preview_frozen_brush_points(&stroke); + let rendered = OverlaySession::rendered_frozen_brush_points( + &preview, + overlay::FROZEN_BRUSH_RENDER_SAMPLE_STEP_POINTS, + ); + let max_turn_angle = rendered.windows(3).fold(0.0_f32, |max_turn, window| { + max_turn.max(OverlaySession::frozen_brush_turn_angle(window[0], window[1], window[2])) + }); + + assert!( + max_turn_angle <= 0.42, + "expected sustained arc preview to stay rounded instead of preserving a corner: {rendered:?}" + ); +} + +#[test] +fn rendered_live_frozen_brush_suppresses_slow_straight_wobble() { + let raw_points = [ + Pos2::new(0.0, 0.0), + Pos2::new(0.45, 0.18), + Pos2::new(0.9, -0.15), + Pos2::new(1.35, 0.17), + Pos2::new(1.8, -0.13), + Pos2::new(2.25, 0.16), + Pos2::new(2.7, -0.12), + Pos2::new(3.15, 0.14), + Pos2::new(3.6, -0.10), + Pos2::new(4.05, 0.12), + Pos2::new(4.5, -0.09), + Pos2::new(4.95, 0.10), + Pos2::new(5.4, -0.08), + Pos2::new(5.85, 0.08), + Pos2::new(6.3, -0.07), + Pos2::new(6.75, 0.06), + Pos2::new(7.2, -0.05), + Pos2::new(7.6, 0.03), + Pos2::new(8.0, 0.0), + ]; + let started_at = Instant::now(); + let mut stroke = OverlaySession::new_active_frozen_brush_stroke(raw_points[0], started_at); + + for (index, point) in raw_points.iter().copied().enumerate().skip(1) { + let sampled_at = started_at + + Duration::from_secs_f32( + index as f32 * overlay::FROZEN_BRUSH_MODEL_SYNTHETIC_SAMPLE_INTERVAL_SECONDS, + ); + + OverlaySession::append_frozen_brush_raw_sample(&mut stroke, point, sampled_at); + } + + let preview = OverlaySession::preview_frozen_brush_points(&stroke); + let rendered = OverlaySession::rendered_frozen_brush_points( + &preview, + overlay::FROZEN_BRUSH_RENDER_SAMPLE_STEP_POINTS, + ); + let (min_y, max_y) = rendered.iter().fold((f32::INFINITY, f32::NEG_INFINITY), |acc, point| { + (acc.0.min(point.y), acc.1.max(point.y)) + }); + + assert!( + significant_y_direction_reversals(&rendered, 0.03) <= 1, + "expected slow straight preview to suppress visible wobble reversals: {rendered:?}" + ); + assert!( + max_y - min_y <= 0.26, + "expected slow straight preview to stay close to a single line: {rendered:?}" + ); +} + +#[test] +fn frozen_brush_model_response_follows_fast_strokes_more_closely() { + let points = [Pos2::new(0.0, 0.0), Pos2::new(16.0, 0.0)]; + let slow = OverlaySession::frozen_brush_input_response(&points, 16.0 / 120.0); + let fast = OverlaySession::frozen_brush_input_response(&points, 16.0 / 1_600.0); + + assert!(slow < fast); + assert!(slow >= overlay::FROZEN_BRUSH_MODEL_INPUT_RESPONSE_MIN); + assert!(fast <= overlay::FROZEN_BRUSH_MODEL_INPUT_RESPONSE_MAX); +} + +#[test] +fn frozen_brush_model_response_boosts_sustained_curve_motion() { + let straight_points = [ + Pos2::new(0.0, 0.0), + Pos2::new(2.0, 0.0), + Pos2::new(4.0, 0.0), + Pos2::new(6.0, 0.0), + Pos2::new(8.0, 0.0), + ]; + let curved_points = [ + Pos2::new(0.0, 0.0), + Pos2::new(2.0, 0.5), + Pos2::new(3.7, 1.6), + Pos2::new(5.0, 3.2), + Pos2::new(5.8, 5.1), + ]; + let straight = OverlaySession::frozen_brush_input_response(&straight_points, 2.0 / 120.0); + let curved = OverlaySession::frozen_brush_input_response(&curved_points, 2.0 / 120.0); + + assert!( + curved > straight, + "expected sustained curved motion to get a higher live response than straight motion: straight={straight}, curved={curved}" + ); +} + #[cfg(target_os = "macos")] #[test] fn begin_ocr_action_exits_with_deferred_request_and_clears_stale_png_output_intent() { diff --git a/packages/rsnap-overlay/src/overlay/tests/rendering_behaviors.rs b/packages/rsnap-overlay/src/overlay/tests/rendering_behaviors.rs index d3c14d4b..dfe7ca6f 100644 --- a/packages/rsnap-overlay/src/overlay/tests/rendering_behaviors.rs +++ b/packages/rsnap-overlay/src/overlay/tests/rendering_behaviors.rs @@ -1898,6 +1898,7 @@ fn render_frozen_capture_affordance_keeps_tiny_frozen_badge_path() { false, FrozenCaptureSource::None, None, + None, false, true, 1.0, diff --git a/packages/rsnap-overlay/src/overlay/toolbar_runtime.rs b/packages/rsnap-overlay/src/overlay/toolbar_runtime.rs index 09085158..1cffe63b 100644 --- a/packages/rsnap-overlay/src/overlay/toolbar_runtime.rs +++ b/packages/rsnap-overlay/src/overlay/toolbar_runtime.rs @@ -277,6 +277,7 @@ impl OverlaySession { self.frozen_capture_source, self.frozen_capture_source == FrozenCaptureSource::FullscreenFallback, None, + None, Some(&mut self.toolbar_state), toolbar_input, );