feat(vision-mixer): zone borders drawn live on PGM and the multiview#625
Open
srperens wants to merge 4 commits into
Open
feat(vision-mixer): zone borders drawn live on PGM and the multiview#625srperens wants to merge 4 commits into
srperens wants to merge 4 commits into
Conversation
Mixer-state-driven graphics belong in the mixer: borders frame boxes
whose geometry only the mixer knows in real time (morphs, takes and
punch-ins drive the compositor pads via control bindings), so no
external graphics source could stay in sync. External graphics (lower
thirds, bugs) remain on DSK.
- Zone gains border: Option<ZoneBorder> { color: #RRGGBB[AA], width }
(serde default — additive API). The border belongs to the BOX, not the
source: a capacity-1 swap zone keeps its frame whoever is pushed in.
Borders draw around the actual rendered source boxes (aspect-fitted
rects), hugging the picture rather than the layout rect.
- Width is resolution-normalized: expressed in PGM canvas pixels and
scaled by region_width / pgm_width at every render target, so 4 px on
air looks proportionally identical on the PVW display and PiP tiles.
- New PGM overlay (pgm_overlay.rs): cairo → appsrc → one pad on the
dist compositor below the DSK stack (zorder 90), mirroring the
multiview overlay pattern. Geometry is read LIVE from the compositor
sink pads each timer tick (property reads, never a buffer probe) —
borders track animations frame by frame and fade with their box's
alpha. Dirty-hash skips redraws when nothing moved.
- Multiview parity: the existing mv overlay draws the same borders on
PiP tiles and the PVW big display (PGM big shows the dist output and
carries them already), with a live-geometry hash in its dirty check
so borders follow morphs there too.
- Pad-loop audit: the classic-take reset and FTB restore leave the new
PGM overlay pad alone / restore it (it sits beyond the DSK range).
- Zone editor: color swatch + width field on the active-zone control
row (v2.21). Operator guide updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mpling time Borders trailed their boxes during morphs/takes: the overlay renderer sampled the pads' current property values, but an overlay frame pushed now is composited 1-3 output frames later — by which time the control bindings driving the animation have moved the box further. Evaluate the bindings at (compositor position + 66 ms lead) instead via GstControlBinding::get_value, so the border is drawn where the box WILL be when the frame composites. Properties without an active binding fall back to plain reads. Applied to both the PGM overlay and the multiview border pass (shared pad_geometry_at helper). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The trailing fix evaluated border geometry 2 frame periods ahead, which overshot — borders led their boxes during morphs. One period matches reality: query_position() reports the frame currently being produced and a buffer pushed now lands in the next one. Confirmed empirically (0 frames trails, 2 leads, 1 tracks). The lead derives from each output's actual framerate — PGM and multiview can differ and each renderer uses its own. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…PU format Self-review findings on the zone-borders branch: - Build the PGM overlay machinery only when num_pips > 0: zones (and thus borders) cannot exist without PiPs, and the default mixer config has none — skipping it saves a compositor pad, an appsrc chain and a render thread on every plain mixer. The FTB/take pad-index guards remain valid either way (the pad index simply never matches). - Timer restart race (also fixed in the pre-existing multiview timer): a fast flow restart can re-register a NEW renderer under the same block id before the old timer thread observes the unregistration — the presence check alone would keep the old thread (and its strong AppSrc ref into the dead pipeline) alive forever. Both timers now verify registry identity (Arc::ptr_eq) instead. - CPU backend: match the forced output format with a capsfilter after videoconvert_pgm_overlay, mirroring the DSK and multiview overlay chains. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Zone borders for PiP compositions — the CNN-style colored frames around interview boxes. The architectural principle: graphics that are a function of the mixer's own state are drawn by the mixer; externally-fed graphics stay on DSK/external HTML. Borders frame boxes whose geometry only the mixer knows in real time (morphs, takes, punch-ins drive the compositor pads via control bindings), so no external graphics source could stay in sync.
Data model (additive)
Zone.border: Option<ZoneBorder> { color: "#RRGGBB[AA]", width }with#[serde(default)]region_width / pgm_widthat every render target — 4 px on air looks proportionally identical on the PVW display and PiP tiles, at any resolutionPGM overlay (new)
pgm_overlay.rs: cairo → appsrc → one pad on the dist compositor at zorder 90 (above all video incl. lifted morph pads, below the DSK stack so lower thirds cover borders)Multiview parity
Robustness
Test plan
pipeline_lifecycle_testgreen — renderer registries hold no strong pipeline refs (WeakRefs throughout), unregistered on flow stop🤖 Generated with Claude Code