Speech-to-text fails on subsequent recordings — MediaRecorder sends near-empty audio chunks

[orAAKLe]

Description

Speech-to-text (STT) transcription works correctly on the first recording attempt, but consistently fails on all subsequent attempts within the same session. This affects the Linux Electron desktop app (v0.15.0) but does not reproduce on mobile browsers.

Steps to reproduce

1. Open CodeNomad on Linux
2. Click the microphone button and speak → transcription succeeds
3. Click the microphone button a second time and speak → transcription fails with no visible error
4. Subsequent attempts also fail
5. Refresh the app (Ctrl+R) → first attempt works again → subsequent ones fail again

Observed behavior

Server debug logs (--log-level debug) show:

Attempt	Audio payload size	Result
1st	61,154 bytes	200 OK (965ms)
2nd	110 bytes	400 "The audio file could not be decoded or its format is not supported"
3rd	110 bytes	400 same error

The 110-byte payload is just the WebM container header with no actual audio data.

Relevant server logs:

[INFO] speech.transcribe mimeType=audio/webm;codecs=opus bytes=61154 → status=200
[INFO] speech.transcribe mimeType=audio/webm;codecs=opus bytes=110
[WARN] speech.transcribe verbose_json failed; retrying default format
  err=BadRequestError: 400 The audio file could not be decoded or its format is not supported.
→ status=502

Root cause

The MediaRecorder (or its underlying MediaStream) is reused between recordings without being re-created. On Chromium/Linux, stop() → start() on the same MediaRecorder instance does not properly reset the audio capture pipeline, resulting in near-empty audio chunks (~110 bytes) on all attempts after the first.

The same code path works correctly on mobile (Android/iOS) because their WebRTC/MediaRecorder implementations handle the lifecycle differently.

Expected behavior

Each microphone session should produce a valid audio recording, regardless of how many times the user starts/stops recording.

Suggested fix

Re-create a fresh MediaStream and MediaRecorder instance for each recording session rather than reusing the previous instance. Ensure proper cleanup (stream.getTracks().forEach(t => t.stop())) of the previous stream before creating a new one.

Environment

• CodeNomad version: 0.15.0 (Electron)
• OS: Arch Linux, PipeWire 1.6.6
• Browser/Engine: Electron/Chromium
• Microphone: Roland UA-22 (USB audio interface), also reproduced with USB webcam mic
• STT provider: OpenAI whisper-1 via openai-compatible adapter

[shantur]

Hi @orAAKLe ,

Thanks for using CodeNomad and creating the issue.
Although, it shouldn't be very different, can you give the latest version a try too?

This seems more like some issue with Electron and Linux.
Do you have any other Linux environments that you could try? Ubuntu 24.04 maybe?

Thanks

[shantur]

@CodeNomadBot

Can you check what could be the issue and maybe see if you can find any relevant bug reports on Electron for Linux?

[codenomadbot]

I checked the current code and did a targeted search for related Electron/Linux reports.

What I found:

• The failure is almost certainly client-side capture, not the STT server/provider path. The server logs the received byte size in packages/server/src/speech/providers/openai-compatible.ts before forwarding to OpenAI-compatible STT. A 110-byte audio/webm;codecs=opus payload means the renderer already produced a near-empty WebM container.
• In the current checkout (0.17.1, Electron 39.0.0), packages/ui/src/components/prompt-input/usePromptVoiceInput.ts already creates a fresh stream and recorder for each recording:
  • startRecording() calls navigator.mediaDevices.getUserMedia({ audio: true }).
  • It then calls createRecorder(mediaStream) and constructs a new MediaRecorder.
  • finalizeRecording() clears mediaRecorder / mediaStream and stops tracks via stream?.getTracks().forEach((track) => track.stop()).
• I also checked the file history back to the original voice-input commit, and that path also created a fresh MediaStream and MediaRecorder per startRecording(). So the code I can see does not appear to reuse a single recorder instance across attempts.

Relevant external search results:

• I did not find an exact Electron issue matching “Linux MediaRecorder first audio recording works, subsequent recordings are 110-byte WebM headers”.
• Electron’s tracker has adjacent media-capture issues, but the results I found were mostly desktop capture, old platform-specific capture bugs, or unrelated WebRTC/media output behavior, not this exact microphone-only repeat-recording symptom.
• I did find examples in other Chromium/Electron apps where Linux WebM/MediaRecorder output can be unreliable or requires defensive handling, and one recent Chromium/Electron-adjacent workaround pattern was to avoid trusting WebM container metadata or to add more defensive recording boundaries. That supports treating this as a Chromium/Linux capture lifecycle issue, but I don’t have a precise upstream bug link for this exact failure mode.

My current hypothesis:

• The issue is still consistent with Chromium/Linux MediaRecorder or WebRTC audio capture failing to deliver encoded audio frames after the first mic session.
• However, based on current source, the cause is probably not app-level reuse of the same MediaRecorder object. It may instead be lower-level device/session state in Chromium/PipeWire/Electron, or a timing issue around stopping the recorder and immediately starting a later capture.

Recommended next steps:

1. Ask the reporter to retest on the latest CodeNomad release, since the reported version is 0.15.0 and current workspace is 0.17.1 with Electron 39.0.0.
2. Add temporary renderer-side debug logs around dataavailable to record each chunk size, final blob size, recorder.state, recorder.mimeType, stream track readyState, and track.muted for every attempt. That will confirm whether Chromium stops producing audio frames or whether we are finalizing before the last chunk arrives.
3. If reproducible on latest, the safest app-side mitigation would be defensive rather than assuming provider failure:
   • Reject obviously invalid recordings client-side/server-side when the final blob is below a small threshold for WebM/Opus, and show a clear “no audio captured” error.
   • Consider calling mediaRecorder.requestData() before stop() when available, and/or using mediaRecorder.start(timeslice) (for example 250ms) so chunks are flushed periodically instead of relying only on the final stop flush.
   • Optionally add a short delay between stopping tracks/finalization and allowing the next recording start on Electron/Linux if logs show a device-release race.
4. If the latest release still reproduces, it would be worth filing upstream with a minimal Electron Fiddle that repeatedly does getUserMedia({ audio: true }) -> new MediaRecorder(stream) -> start -> stop -> stop tracks, and attaches the first and second WebM sizes. The key evidence is “first blob contains audio, second blob is header-only despite a fresh recorder and fresh stream”.

I didn’t make code changes from this comment because the request was to investigate and look for relevant Electron/Linux bug reports, and the current code does not directly match the proposed reuse bug.

--
Yours,
CodeNomadBot