Describe the bug
Versions: @livekit/agents 1.4.8, @livekit/agents-plugin-openai 1.4.8, node 22, LiveKit Cloud agents.
Repro: Start an AgentSession whose llm is openai.realtime.RealtimeModel and whose Agent is constructed with one or more tools (via llm.tool). On the first chat-context sync the session logs failed to sync chat context to remove never-played messages with Error: Unsupported item type: agent_config_update thrown by livekitItemToOpenAIItem (realtime_model.js) via createChatCtxUpdateEvents → updateChatCtx.
Cause: agent_activity.ts inserts an AgentConfigUpdate chat item on enter when the agent has tools/instructions; the realtime plugin's livekit→OpenAI item conversion has no branch for agent_config_update (the non-realtime path / chat_context supports excludeConfigUpdate, but the realtime sync path does not skip it).
Impact: Non-fatal (caught), but breaks the never-played-message sync — observed as degraded post-tool speech behavior and missing transcript items on those turns.
Suggested fix: Handle (or skip, like excludeConfigUpdate) agent_config_update items in the realtime plugin's chat-context conversion.
Relevant log output
level=error msg="failed to sync chat context to remove never-played messages"
error: Unsupported item type: agent_config_update
at livekitItemToOpenAIItem (@livekit/agents-plugin-openai/dist/realtime/realtime_model.js)
at RealtimeSession.createChatCtxUpdateEvents
at RealtimeSession.updateChatCtx
at AgentActivity._realtimeGenerationTaskImpl
Describe your environment
Node 22, LiveKit Cloud agents.
Minimal reproducible example
No response
Additional information
Root cause
-
@livekit/agents 1.4.8 inserts an AgentConfigUpdate chat item when an agent enters with tools or instructions (voice/agent_activity.js ~line 347: runOnEnter && (instructions || tools)), and again on updateTools/updateInstructions.
-
@livekit/agents-plugin-openai 1.4.8's livekitItemToOpenAIItem (realtime path) has no case for type: "agent_config_update" and throws. The framework already supports skipping these via an excludeConfigUpdate flag in llm/chat_context, but the realtime plugin's chat-ctx conversion path doesn't pass it.
Describe the bug
Versions: @livekit/agents 1.4.8, @livekit/agents-plugin-openai 1.4.8, node 22, LiveKit Cloud agents.
Repro: Start an AgentSession whose llm is openai.realtime.RealtimeModel and whose Agent is constructed with one or more tools (via llm.tool). On the first chat-context sync the session logs failed to sync chat context to remove never-played messages with Error: Unsupported item type: agent_config_update thrown by livekitItemToOpenAIItem (realtime_model.js) via createChatCtxUpdateEvents → updateChatCtx.
Cause: agent_activity.ts inserts an AgentConfigUpdate chat item on enter when the agent has tools/instructions; the realtime plugin's livekit→OpenAI item conversion has no branch for agent_config_update (the non-realtime path / chat_context supports excludeConfigUpdate, but the realtime sync path does not skip it).
Impact: Non-fatal (caught), but breaks the never-played-message sync — observed as degraded post-tool speech behavior and missing transcript items on those turns.
Suggested fix: Handle (or skip, like excludeConfigUpdate) agent_config_update items in the realtime plugin's chat-context conversion.
Relevant log output
Describe your environment
Node 22, LiveKit Cloud agents.
Minimal reproducible example
No response
Additional information
Root cause
@livekit/agents 1.4.8 inserts an AgentConfigUpdate chat item when an agent enters with tools or instructions (voice/agent_activity.js ~line 347: runOnEnter && (instructions || tools)), and again on updateTools/updateInstructions.
@livekit/agents-plugin-openai 1.4.8's livekitItemToOpenAIItem (realtime path) has no case for type: "agent_config_update" and throws. The framework already supports skipping these via an excludeConfigUpdate flag in llm/chat_context, but the realtime plugin's chat-ctx conversion path doesn't pass it.