Skip to content

useTranscriptions hook produces incorrect ordering when segments arrive with delayed text #1280

@itsmeprashant

Description

@itsmeprashant

Select which package(s) are affected

@livekit/components-core

Describe the bug

Bug

Description

The useTranscriptions hook produces incorrect chronological ordering when transcription segments arrive with empty text initially and get populated later. This causes messages to appear out of order (e.g., AI responses before user questions).

Expected Behavior

Transcriptions should be ordered chronologically based on when speech occurred, using timing metadata like firstReceivedTime.

[Correct order]

User: "Hello" (spoken at T1, text arrives late)
AI: "Hi there!" (spoken at T2, text arrives immediately)

Actual Behavior

Transcriptions are ordered based on array insertion order - when text first appears in the stream, not when speech occurred.

[Incorrect order]

AI: "Hi there!" (inserted first - text arrived immediately)
User: "Hello" (inserted second - text arrived late)

Issues

❌ Array uses .push() - insertion order, not chronological order
❌ TextStreamInfo (from reader.info) lacks timing metadata (firstReceivedTime, startTime, etc.)

Reproduction

Steps to Reproduce

  1. Set up a LiveKit room with transcription enabled
  2. Use useTranscriptions() hook in a React component
  3. Have a user speak (segment arrives with empty/partial text initially)
  4. Have AI respond immediately (segment arrives with full text)
  5. User's transcription text gets populated later via updates
  6. Observe AI message appears before user message in the returned array

Root Cause Analysis

File: @livekit/components-core/src/components/textStream.ts (lines 80-105)

The setupTextStream function maintains transcription order based on insertion:

streamObservable.subscribe((accumulatedText) => {
  const index = textStreams.findIndex(/* ... */);
  
  if (index !== -1) {
    // Update existing stream in place
    textStreams[index] = { ...textStreams[index], text: accumulatedText };
    textStreamsSubject.next([...textStreams]);
  } else {
    // Problem: New streams pushed to end based on insertion order
    textStreams.push({
      text: accumulatedText,
      participantInfo,
      streamInfo: reader.info,
    });
    textStreamsSubject.next([...textStreams]);
  }
});


### Logs

```shell

System Info

### System Info

Binaries:
Node: 20.19.5 
npm: 10.8.2
pnpm: 10.15.1

Browsers:
Chrome: 144.0.7559.110
Firefox: 147.0
Safari: 26.2

npmPackages:
@livekit/components-react: 2.9.19
livekit-client: 2.17.0

Severity

serious, but I can work around it

Additional Information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions