feat(core): add user transcription timeout

[chenghao-mou]

• add a new event when vad detects any speech, but stt fails to produce any transcript during that timeout

from livekit.agents import AgentSession, UserTranscriptionTimeoutEvent

  session = AgentSession(
      stt=...,
      llm=...,
      tts=...,
      # VAD heard speech but no transcript landed within 5s of the user stopping.
      # 5.0 is the default; set to None to disable.
      transcription_timeout=5.0,
  )

  @session.on("user_transcription_timeout")
  def _on_transcription_timeout(ev: UserTranscriptionTimeoutEvent) -> None:
      # ev.speech_duration   -> total VAD speech (s) this turn that produced no transcript
      # ev.vad_speech_started_at -> when the user first started speaking (epoch seconds)

      # ignore very short blips (coughs, door slams) that VAD picks up as "speech"
      if ev.speech_duration < 0.7:
          return

      session.generate_reply(
          instructions="Tell the user you didn't catch that and ask them to repeat it.",
      )

[devin-ai-integration]

Devin Review found 2 potential issues.

[devin-ai-integration]

🚩 New event not forwarded in SessionHost remote transport

The SessionHost in remote_session.py registers handlers for specific events (lines 371-379) and forwards them over the transport. The new user_transcription_timeout event is not registered or forwarded. This means remote sessions won't receive this event. This is likely acceptable as a first iteration (not all events need remote transport support immediately), but it's an inconsistency with the event being in EventTypes and AgentEvent.

(Refers to lines 366-379)

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🚩 Timer not cancelled when agent starts speaking — relies on lazy guard

When _arm_transcription_timeout schedules the timer and the agent subsequently starts speaking (via on_start_of_agent_speech at audio_recognition.py:385), the timer handle is NOT cancelled. Instead, _on_transcription_timeout at line 1806 checks self._agent_speaking and bails out if True. This is a valid lazy-check pattern but has a subtle edge case: if the agent speaks briefly (e.g. a backchannel like "hmm") and finishes before the timer fires, the timeout event will still fire even though the agent responded. This may be intentional (the user's speech was still never transcribed), but could surprise users who expect any agent speech to suppress the event.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[chenghao-mou]

Since there is no user content, we should still emit the signal so the agent can check.

[longcw]

maybe skip this when stt is not set?

[davidzhao]

should we have a default handler with the above implementation? any downsides of having this work automatically?

[longcw]

should we have a default handler with the above implementation? any downsides of having this work automatically?

I am wondering how many false alarms it may have, it can by risky if there are some background noise.