Design generic models

[tcely]

As part of expanding site support; we should design an architecture that allows us to best describe streams / downloads, and their audio / video / thumbnail / subtitle assets.

Video and thumbnail assets are usually a single entry for each media key. We probably won't need to support multiple versions of these.

Audio is often presented as multiple formats and we can technically download as many as the file container supports. This lets us create a final file that has a translation audio track, and both original audio and default audio tracks available during playback.

Subtitles are likewise offered in multiple languages and it could be useful to download many of these. We also need to be concerned if they should be embedded or kept external alongside the final video file. Possibly in a dedicated subdirectory with names that partially match the final video file.

YouTube, as an example, has playlists and either streams or videos. The different types of playlists used have various URL templates. Once you have a key for the stream / video / playlist, these can be inserted into the proper template for the asset type that is desired.

Other sites may operate differently, but we expect most will have a unique identifier for each stream / video / playlist.

An additional complication will be format selection. I think a list of profiles that can be reused makes the most sense here.
We should compose these profiles from smaller pieces that can also be reused.

---

Asset types:

• video
• audio
• subtitle
• thumbnail

Codec:

• supported asset type
• codec name

Subtitle:

• language
• original language
• extension
• codec
• machine generated

Thumbnail:

• extension
• codec
• height
  • min
  • max
  • convert(ed) to
• width
  • min
  • max
  • convert(ed) to
• orientation
  • portrait
  • landscape
• resized

Container:

• extension
• supported asset type
• number of type supported
• supported codec

Audio:

• language
• default
• original
• bit rate
• frequency
• range compression
• extension
• codec
• machine generated

Video:

• Frames per second
  • min
  • max
• Codec preference
  • acceptable codecs
  • ordering priority
• Resolution
  • min
  • max
• Dynamic Range
  • min
  • max
• Upscaled
  • accept
  • deny
• Pre-combined video & audio
  • deny
  • accept & deconstruct
  • accept & preserve

[tcely]

Suggestions from Google's AI

URL: https://share.google/aimode/dxu8XXDMOsp7Utxnq

---

To expand TubeSync's site support, you need a decoupled architecture that separates Source Discovery (finding IDs) from Asset Mapping (matching IDs to streams).

Here is a proposed plan to structure these entities and profiles.

1. The "Media Manifest" Architecture
   Instead of downloading directly, introduce a Manifest layer. When a URL is processed, the system generates a manifest that maps unique identifiers to available streams.

   • Media Key: The unique ID (e.g., YouTube Video ID).
   • Asset Map: A registry of all available streams (Video, Audio, Subtitle, Thumb) with their metadata (Bitrate, Lang, Codec).
   • The Mixer: A logic layer that compares the Asset Map against your User Profiles to pick the winning combination.

2. Hierarchical Profile System
   To keep profiles reusable and modular, use a "Lego-block" approach:

   • Base Atoms: Smallest units (e.g., codec: h264, lang: en).
   • Asset Requirements: Groups of atoms (e.g., "High-Quality Audio" = bitrate > 192k + codec: opus).
   • Media Profiles: The final recipe.
   • Example "Global Archive" Profile:
     • Video: Max resolution, Prefer AV1 > VP9 > H264.
     • Audio: Include [Original, Translation, Default].
     • Subtitles: Include [All], External .srt in /subs folder.
     • Container: MKV (to support multiple audio/sub tracks).

3. Execution Strategy

   • Phase A: The Registry (Metadata)
     Implement the data models you listed as a Capability Matrix. When TubeSync hits a new site, it shouldn't just ask "Where is the file?", it should ask "What assets do you have?" and populate the Audio, Video, and Subtitle objects based on your schema.

   • Phase B: The Selection Engine
     Build a resolver that scores available assets.

     • Example: If a video is available in 4K/AV1 and 1080p/H264, and the profile says Upscaled: Deny and Max Resolution: 1080p, the engine automatically selects the H264 stream.

   • Phase C: The Post-Processor (Muxing & Storage)

     • Audio Muxing: Since you want multiple tracks (Original + Translation), the engine must support "Multi-Source Assembly." It fetches the video stream and N audio streams, then uses ffmpeg to map them into a single container.
     • Subtitle Sidecars: Add a "Filesystem Template" to the profile.
       • Option 1: Embedded (Mux into MKV).
       • Option 2: Sidecar (Path: ./{video_name}_subs/{lang}.{ext}).

4. Site-Specific Template Mapping
   Since different sites use different URL patterns, use a Provider Template system:

   • Provider: YouTube
     • video_url: https://www.youtube.com/watch?v={key}
     • thumbnail_template: https://i.ytimg.com/vi/{key}/maxresdefault.jpg
     • playlist_template: https://www.youtube.com/playlist?list=PL{playlist-id}

---

The corrections you made solidify the architecture by ensuring the URL templates and metadata schemas are concrete. Separating the "Discovery" (ID extraction) from the "Manifest" (stream mapping) is the right move for scaling beyond YouTube.

To advance this, we should look at how the "Mixer" logic handles conflicts or missing assets.

Refined Logic for the Selection Engine:

• Fallback Chains: If a profile requires 4K but the manifest only shows 1080p, the engine needs a "best-available" vs. "strict-match" flag.
• Language Priority: For the multi-track audio requirement, you’ll need a priority list (e.g., [Original, en, jp]) so the engine knows which streams to pull if the site offers 20+ languages.
• Container Constraints: If the user chooses MP4 but wants 3 audio tracks, the system must either auto-switch to MKV or strip the extra tracks based on a pre-defined rule.

Data Model Extension:

For the Site-Specific Templates, consider adding a "Key Extractor" regex per provider. This allows the system to turn a messy URL into that clean {key} used in your templates.

[meeb]

Sounds like a good plan to me. Regarding the {key} stuff and requiring templates per source type I was originally planning to expand source support by just entirely relying on yt-dlp. yt-dlp returns lists of media items for things like playlist pages for most sources so I was testing to see if we could entirely replace {key} extraction with UUID5'ing the entire URL and using that as a "unique-ness" check for media to download. This would probably need some URL formatting (e.g. parameter ordering [alphabetical?], forced lower-case and truncation of URI fragments etc.).

If that worked we wouldn't need to have any site-specific templates at all and it would support whatever yt-dlp supported.