feat: Implemented FallbackAdapter for LLM

[KrishnaShuk]

Description

#868
This PR implements the FallbackAdapter for LLMs, bringing feature parity with the Python SDK. This component allows agents to automatically switch to backup LLM providers if the primary provider fails, times out, or becomes unavailable, ensuring high availability for production agents.

Python(agents) SDK code for fallback adapter: https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/llm/fallback_adapter.py

Changes Made

• Created packages/agents/src/llm/FallbackAdapter.ts implementing the FallbackAdapter and FallbackLLMStream classes.
• Implemented Priority Failover: Accepts a list of LLMs and attempts them in order.
• Implemented Active Health Check: Background recovery logic that uses the actual ChatContext to verify when a failed provider has recovered (instead of a simple timer).
• Implemented State Tracking: Emits llm_availability_changed events when providers go down or recover.
• Implemented Glitch Protection: Respects retryOnChunkSent to prevent the agent from repeating partial sentences if a stream fails mid-generation.
• Exported the new adapter in packages/agents/src/llm/index.ts.

Pre-Review Checklist

• Build passes: All builds (lint, typecheck, tests) pass locally
• AI-generated code reviewed: Removed unnecessary comments and ensured code quality
• Changes explained: All changes are properly documented and justified above
• Scope appropriate: All changes relate to the PR title, or explanations provided for why they're included
• Video demo: N/A (Backend logic change, verified via Unit Tests)

Testing

• Automated tests added/updated
• All tests pass (8 tests covering initialization, failover, total failure, events, and stream delegation)
• Make sure both restaurant_agent.ts and realtime_agent.ts work properly (This change is additive/opt-in and does not affect existing agents unless configured to use the adapter)

[changeset-bot]

🦋 Changeset detected

Latest commit: fb99648

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 15 packages

Name	Type
@livekit/agents	Patch
@livekit/agents-plugin-anam	Patch
@livekit/agents-plugin-bey	Patch
@livekit/agents-plugin-cartesia	Patch
@livekit/agents-plugin-deepgram	Patch
@livekit/agents-plugin-elevenlabs	Patch
@livekit/agents-plugin-google	Patch
@livekit/agents-plugin-inworld	Patch
@livekit/agents-plugin-livekit	Patch
@livekit/agents-plugin-neuphonic	Patch
@livekit/agents-plugin-openai	Patch
@livekit/agents-plugin-resemble	Patch
@livekit/agents-plugin-rime	Patch
@livekit/agents-plugin-silero	Patch
@livekit/agents-plugins-test	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[toubatbrian]

Have you tested fallback yet? Could you add a simple example ts file under examples showcasing the basic usage of this class. Thanks!

[KrishnaShuk]

I won’t be able to do this today. I’ll test the fallback and add a simple example TS file under examples tomorrow.

[toubatbrian]

Thanks @KrishnaShuk!

[KrishnaShuk]

I need to test this further. I’ll provide an update shortly.

[KrishnaShuk]

@toubatbrian I have made changes in the fallback_adapter.ts file. Some behaviors were not in accordance with the Python implementation. Now the fallback adapter is in complete parity with the agents (Python) adapter.

After searching in the agents repo, I didn't find any tests for the LLM adapter, whereas tests for STT and TTS adapters were present. I have made a test file fallback_adapter.test.ts for it, covering the following behaviors as per the Python implementation:

• Sequential Fallback: Verifies that if the primary LLM fails immediately, it correctly falls back to the next available LLM.
• Safety Guard (Chunking): Ensures that if an LLM fails after it has already started sending chunks (text/tools), the adapter stops and throws an error instead of falling back. This prevents "stitched" responses (e.g., half a sentence from LLM A, followed by a full sentence from LLM B).
• Safety Override: Verifies that the retryOnChunkSent option correctly overrides the safety guard if the user explicitly allows it.
• Exhaustion: Confirms that if all configured LLMs fail, the adapter correctly throws an
  APIConnectionError
• Availability Events: Verifies that llm_availability_changed events are emitted when an LLM is marked as unavailable.
• Initialization & Validation: Checks that the adapter requires at least one LLM and initializes correctly.

[KrishnaShuk]

I changed .then() to .finally() in llm.ts to ensure the stream queue is properly closed even when the mainTask fails (eg. due to connection errors or timeouts).
Previously, queue.close() was only called in the .then() block. If mainTask threw an error, the queue would remain open indefinitely. This caused consumers iterating over the stream (like FallbackAdapter) to hang forever waiting for the next chunk or a close signal, preventing error handling or fallback logic from triggering.

Using .finally() guarantees that queue.close() is called regardless of whether the task succeeds or fails, ensuring proper stream termination and error propagation.

@yura2100 also raised this concern in this issue.

Reference commit: fb99648

[KrishnaShuk]

All the test cases are passing and i have also tested it with an example agent.
From my side, the adapter is working best.