feat(core,engine): data-duck — compile-time audio ducking under voice tracks#1337
Open
mvanhorn wants to merge 2 commits into
Open
feat(core,engine): data-duck — compile-time audio ducking under voice tracks#1337mvanhorn wants to merge 2 commits into
mvanhorn wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Declarative audio ducking: a music track marked
data-duckautomatically lowers itself whenever anydata-role="voice"track is audible, with smooth fade ramps.data-duckaccepts dB ("-12dB") or a linear multiplier;data-duck-fadesets the ramp (default 0.3s). Rendered evidence (same composition with and withoutdata-duck):The difference row is the ducking, isolated: digital silence outside the voice windows (the two mixes are bit-identical there) and exactly the removed music energy inside them.
Why
Voiceover-over-music is the default shape of narrated video, and getting the balance right currently means hand-authoring volume keyframes against every voice clip's start and end, then re-timing them whenever narration shifts (#1176 was a user doing exactly this with GSAP volume tweens). Consumer editors auto-duck; neither HyperFrames nor Remotion offers it declaratively.
The timeline knows every overlap window at compile time, so ducking doesn't need sidechain analysis. It compiles to volume keyframes and rides the sample-accurate volume automation the engine already has (#1117) — deterministic, and identical in preview and render.
How
packages/core/src/compiler/timingCompiler.ts:compileAudioDuckingreadsdata-duck/data-duck-fade/data-role, computes voice-overlap intervals from clip timing (merging gaps shorter than 2x the fade so the music doesn't pump between close voice clips), and writes the resulting keyframes to an internaldata-hf-duck-keyframesattribute. Recompilation-safe: the attribute is regenerated, never accumulated.packages/engine/src/services/audioElementParser.ts(extracted from audioMixer): parses the duck keyframes into each track'svolumeKeyframes, multiplying with any authoreddata-volume-keyframesenvelope rather than replacing it.packages/core/src/runtime/mediaVolumeEnvelope.ts: preview applies the same multiplied envelope, so what you hear in preview is what renders.probeStage.ts: runtime-probed automation (GSAP volume tweens) multiplies with duck keyframes instead of overwriting them.data-duck/data-duck-fade/data-rolerows in the html-schema reference plus a concepts section.Out of scope by design: runtime-triggered audio (clips whose timing isn't declarative) — documented as a limitation.
Test plan
The waveform evidence above is real render output from this branch. Sample-subtracting the two renders: bit-identical outside voice windows (diff RMS −270dB), and inside them the measured delta is −37.7dB against a theoretical −37.6dB for a −12dB duck of this music bed — the envelope lands within 0.1dB of spec.
sayvoiceover over a music bed)