-
Notifications
You must be signed in to change notification settings - Fork 142
Description
Select which package(s) are affected
@livekit/components-core
Describe the bug
Bug
Description
The useTranscriptions hook produces incorrect chronological ordering when transcription segments arrive with empty text initially and get populated later. This causes messages to appear out of order (e.g., AI responses before user questions).
Expected Behavior
Transcriptions should be ordered chronologically based on when speech occurred, using timing metadata like firstReceivedTime.
[Correct order]
User: "Hello" (spoken at T1, text arrives late)
AI: "Hi there!" (spoken at T2, text arrives immediately)
Actual Behavior
Transcriptions are ordered based on array insertion order - when text first appears in the stream, not when speech occurred.
[Incorrect order]
AI: "Hi there!" (inserted first - text arrived immediately)
User: "Hello" (inserted second - text arrived late)
Issues
❌ Array uses .push() - insertion order, not chronological order
❌ TextStreamInfo (from reader.info) lacks timing metadata (firstReceivedTime, startTime, etc.)
Reproduction
Steps to Reproduce
- Set up a LiveKit room with transcription enabled
- Use
useTranscriptions()hook in a React component - Have a user speak (segment arrives with empty/partial text initially)
- Have AI respond immediately (segment arrives with full text)
- User's transcription text gets populated later via updates
- Observe AI message appears before user message in the returned array
Root Cause Analysis
File: @livekit/components-core/src/components/textStream.ts (lines 80-105)
The setupTextStream function maintains transcription order based on insertion:
streamObservable.subscribe((accumulatedText) => {
const index = textStreams.findIndex(/* ... */);
if (index !== -1) {
// Update existing stream in place
textStreams[index] = { ...textStreams[index], text: accumulatedText };
textStreamsSubject.next([...textStreams]);
} else {
// Problem: New streams pushed to end based on insertion order
textStreams.push({
text: accumulatedText,
participantInfo,
streamInfo: reader.info,
});
textStreamsSubject.next([...textStreams]);
}
});
### Logs
```shellSystem Info
### System Info
Binaries:
Node: 20.19.5
npm: 10.8.2
pnpm: 10.15.1
Browsers:
Chrome: 144.0.7559.110
Firefox: 147.0
Safari: 26.2
npmPackages:
@livekit/components-react: 2.9.19
livekit-client: 2.17.0Severity
serious, but I can work around it
Additional Information
No response