Skip to content

fix: live audio/video format misalignment#458

Merged
wyf7107 merged 1 commit into
google:mainfrom
allen-stephen:fix/live-input
Jun 25, 2026
Merged

fix: live audio/video format misalignment#458
wyf7107 merged 1 commit into
google:mainfrom
allen-stephen:fix/live-input

Conversation

@allen-stephen

Copy link
Copy Markdown
Contributor

Live audio and video were sent and played in incorrect formats:

  • Capture mic audio at 16 kHz (was 22 kHz); the Live API expects 16 kHz PCM input.
  • Signal the input sample rate via mime_type 'audio/pcm;rate=16000' so the backend can interpret incoming audio correctly.
  • Play back audio at 24 kHz (was 22 kHz) to match the Live API's output rate.
  • Encode camera frames as JPEG to match the 'image/jpeg' mime_type
  • Resample mic audio with linear interpolation instead of nearest-neighbor to reduce aliasing artifacts.

@wyf7107 wyf7107 merged commit b751fc8 into google:main Jun 25, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants