Skip to content

feat(spf): expose audio tracks & renditions on the SPF media adapter - #1811

Closed
spuppo-mux wants to merge 4 commits into
videojs:mainfrom
spuppo-mux:feat/spf-adapter-audio-tracks
Closed

feat(spf): expose audio tracks & renditions on the SPF media adapter#1811
spuppo-mux wants to merge 4 commits into
videojs:mainfrom
spuppo-mux:feat/spf-adapter-audio-tracks

Conversation

@spuppo-mux

@spuppo-mux spuppo-mux commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Part 2 of #1795 (video → audio → streamType → live → parity/autoplay).

Based on #1803 (not using that as a base because the branch is on my fork so correct this after 1803 is merged)

What changed

  • SPF: getAudioTracks(state) selector collapses the model's per-(language x quality-group) audio tracks into one AudioTrackInfo per (language, name), carrying trackIds. deriveAudioTracks behavior publishes state.audioTracks (same reactor as deriveVideoRenditions: derive on resolve, clear on unload, skip unchanged).
  • Core (media-tracks.ts): projects audioTracks -> AudioTrackList (kind = default ? 'main' : 'alternative', label = name, language); reflects enabled from selectedAudioTrackId; feeds a newly-enabled track back to userAudioTrackSelection.

Design decisions

  • The SPF model doesn't split the way the DOM does. SPF stores a flat (language x quality-group) list in one switching set; the DOM wants one AudioTrack per language. The dedupe is model logic, so it lives in SPF (getAudioTracks). trackIds is what lets the adapter map a resolved selectedAudioTrackId (a specific group) back to its language track.
    enabled reflects the resolved selectedAudioTrackId, not the track pinned by the user. Audio exposes a single enabled property (vs. video which exposes active + selected). It's restored on rebuild via the untracked reflect in #projectAudioTracks.
  • Write by { language, name }. name disambiguates same-language tracks (e.g. "English" vs "English commentary"); language alone would be ambiguous.

Deferred: audioRenditions

Left empty (as in HlsJsMedia, which never populates it).

Test plan

  • spf: getAudioTracks (language collapse, same-language-different-name kept distinct, empty cases); deriveAudioTracks behavior (publish/skip-unchanged/clear).
  • core: projects one track per language, reflects enabled from any quality group of the selected language, feeds selection back as { language, name }, no-write when reflection re-enables the playing track, removes on destroy.
  • Full core dom/media + spf engine/behaviors green; typecheck, lint, check:workspace clean.
    <simple-hls-video>.audioTracks flows through CustomMediaElement automatically (same path as video).

Note

Medium Risk
Touches playback track selection and DOM↔engine sync with feedback-loop guards; behavior is well covered by tests but mistakes could affect ABR pinning or audio switching at runtime.

Overview
Exposes video renditions and audio tracks on the Simple HLS DOM media element by deriving engine signals from the resolved presentation and projecting them onto videoRenditions / audioTracks, with two-way wiring for quality and language selection.

SPF layer: Adds getVideoRenditions and getAudioTracks (audio collapsed to one entry per language+name with trackIds for quality-group mapping). New deriveVideoRenditions and deriveAudioTracks behaviors publish state.videoRenditions and state.audioTracks on resolve, clear on unload, and skip redundant writes when the id-set is unchanged. Both are composed into the Simple HLS engine.

Core adapter: SimpleHlsMedia now stacks MediaTracksMixin and SimpleHlsMediaMediaTracksMixin, which sync engine signals to DOM track lists, reflect active/selected renditions and enabled audio from engine selection, forward user picks to userVideoTrackSelection / userAudioTrackSelection (including ABR when rendition index is -1), and re-subscribe on each src change because the SPF engine is recreated per assignment.

Reviewed by Cursor Bugbot for commit cf4dc03. Bugbot is set up for automated code reviews on this repo. Configure here.

@spuppo-mux
spuppo-mux requested a review from cjpillsbury July 7, 2026 21:29
@vercel

vercel Bot commented Jul 7, 2026

Copy link
Copy Markdown

@spuppo-mux is attempting to deploy a commit to the Mux Team on Vercel.

A member of the Team first needs to authorize it.

@netlify

netlify Bot commented Jul 7, 2026

Copy link
Copy Markdown

Deploy Preview for vjs10-site ready!

Name Link
🔨 Latest commit cf4dc03
🔍 Latest deploy log https://app.netlify.com/projects/vjs10-site/deploys/6a4d6fb776c95d0008c72ab7
😎 Deploy Preview https://deploy-preview-1811--vjs10-site.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit cf4dc03. Configure here.

}

const sameAudioTracks = (a: AudioTrackInfo[] | undefined, b: AudioTrackInfo[]): boolean =>
!!a && a.length === b.length && a.every((track, index) => track.id === b[index]?.id);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale audio trackIds skipped

Medium Severity

sameAudioTracks treats a reload as unchanged when collapsed track ids match, but getAudioTracks can still produce different trackIds for the same slots. The engine then keeps stale trackIds, so selectedAudioTrackId may not match and the DOM adapter can show the wrong language as enabled.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit cf4dc03. Configure here.

@spuppo-mux

Copy link
Copy Markdown
Collaborator Author

Re-implemented in #1826

@spuppo-mux spuppo-mux closed this Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant