feat: add RobustMicrophone to handle hardware disconnects (#6076)#6183
feat: add RobustMicrophone to handle hardware disconnects (#6076)#6183anushkagupta200615-jpg wants to merge 1 commit into
Conversation
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| except asyncio.TimeoutError: | ||
| # Stall detected! | ||
| logger.warning( | ||
| f"RobustMicrophone: No audio frames received for {self._stall_timeout}s. Restarting microphone..." | ||
| ) | ||
| await self._close_internal() | ||
| await asyncio.sleep(0.5) # Brief pause before reconnecting | ||
| self._start_internal() | ||
| except Exception as e: | ||
| logger.error(f"RobustMicrophone error in monitor loop: {e}", exc_info=e) | ||
| await asyncio.sleep(1.0) | ||
| await self._close_internal() | ||
| self._start_internal() |
There was a problem hiding this comment.
🔴 Unprotected _start_internal() in error handlers crashes the monitor loop permanently
In _monitor_loop, both exception handlers call self._start_internal() (lines 105 and 110) without any try/except protection. If _start_internal() raises an exception (e.g., device unavailable after disconnect, permission error, or any failure in open_input()/create_audio_track()/from_track()), the exception propagates out of the while self._running: loop, causing the asyncio task to silently terminate. The _running flag remains True, so the RobustMicrophone appears operational but is permanently dead with no recovery possible. This completely defeats the class's stated purpose of automatically restarting on failure.
Affected code paths
Line 105 (after timeout stall detection):
except asyncio.TimeoutError:
await self._close_internal()
await asyncio.sleep(0.5)
self._start_internal() # unprotected - crashes loop on failureLine 110 (after generic error):
except Exception as e:
await asyncio.sleep(1.0)
await self._close_internal()
self._start_internal() # unprotected - crashes loop on failureThe same issue exists at line 83 where the initial _start_internal() is called without protection.
Prompt for agents
The _start_internal() calls inside both exception handlers (lines 105 and 110) and at line 83 need to be wrapped in try/except. If _start_internal() fails, the monitor loop should continue running and retry after a delay, rather than letting the exception propagate out and kill the task.
A possible approach:
1. Wrap each _start_internal() call in a try/except that catches Exception
2. On failure, log the error and continue the while loop (which will try again on the next iteration)
3. Consider adding a retry delay and/or exponential backoff for repeated _start_internal() failures
4. Consider adding a max_retries parameter to eventually give up and signal failure
The initial _start_internal() at line 83 should also be inside the while loop's try/except, or have its own retry logic, since device initialization can fail at startup too.
Relevant file: livekit-agents/livekit/agents/utils/robust_microphone.py, function _monitor_loop (lines 82-110).
Was this helpful? React with 👍 or 👎 to provide feedback.
|
|
||
| self._mic_obj = self._devices.open_input(**kwargs) | ||
| self._mic_track = rtc.LocalAudioTrack.create_audio_track("robust-mic-internal", self._mic_obj.source) | ||
| self._mic_stream = rtc.AudioStream.from_track(self._mic_track) |
There was a problem hiding this comment.
🚩 AudioStream.from_track called without sample_rate/num_channels unlike codebase pattern
The existing usage of rtc.AudioStream.from_track in livekit-agents/livekit/agents/voice/room_io/_input.py:303-308 passes sample_rate, num_channels, and frame_size_ms explicitly. The new code at line 79 calls rtc.AudioStream.from_track(self._mic_track) without these parameters. If the from_track defaults differ from self._sample_rate/self._num_channels, the frames forwarded to self._source could have mismatched format. This might be benign if the local mic track already outputs at the configured rate, but it deviates from established patterns and could cause subtle audio issues.
Was this helpful? React with 👍 or 👎 to provide feedback.
2 participants
This PR implements a robust wrapper for rtc.MediaDevices.open_input() to address the issue where the audio stream silently stops producing frames if a physical hardware interruption occurs (e.g. microphone cable looseness).
Since the underlying disconnect bug resides in the core livekit-python C++ SDK and doesn't surface any errors to the agent, this PR introduces the RobustMicrophone utility into the Agent SDK. It acts as a drop-in proxy that actively monitors the audio stream and safely auto-recovers the connection upon detecting a hardware stall.
Changes Made:
Added RobustMicrophone: Implemented a new utility class in livekit.agents.utils.robust_microphone.
Watchdog Recovery: The class wraps the stream reading inside an asyncio.wait_for watchdog. If no frames are received within stall_timeout (default 2 seconds), it safely tears down the frozen open_input stream and automatically re-initializes it in the background.
AudioSource Proxy: The wrapper transparently pipes the frames into an exposed rtc.AudioSource, ensuring that the LocalAudioTrack published by the agent remains perfectly intact and alive during the silent hardware reset.
Testing: Added tests/test_robust_microphone.py with mocked timeouts to ensure the stall-detection and auto-recovery logic executes reliably.