Skip to content

fix(voice): send the relay identity token where the backend can read it - #5428

Merged
M3gA-Mind merged 1 commit into
tinyhumansai:mainfrom
YellowSnnowmann:fix/voice-agent-relay-token-delivery
Aug 6, 2026
Merged

fix(voice): send the relay identity token where the backend can read it#5428
M3gA-Mind merged 1 commit into
tinyhumansai:mainfrom
YellowSnnowmann:fix/voice-agent-relay-token-delivery

Conversation

@YellowSnnowmann

@YellowSnnowmann YellowSnnowmann commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Send the signed voice-relay identity token via customLlmExtraBody so it
    actually reaches the backend relay; userId alone never did.
  • Unit test pinning that the token travels in customLlmExtraBody, not only in
    userId.

Problem

A realtime voice session passed the relay token only as startSession({ userId }).
The provider puts that on its own conversation-init event, but it does not place
it on the Custom-LLM request it makes to our relay. A capture of a live
POST /voice-agent/chat/completions carried only:

[messages, model, max_tokens, stream, stream_options, temperature, tools]

No user, no user_id. The relay could not identify the caller, rejected the
turn with 400 request is missing the user token, and the desktop agent was
never asked to answer — the session connected, listened, and stayed silent.

Solution

Pass the same token through customLlmExtraBody, which does reach the
Custom-LLM request. The provider forwards it nested under an
elevenlabs_extra_body key rather than merging it into the top level, and the
relay reads that shape (backend PR below). userId is left in place for
provider-side attribution, so nothing about the session binding changes.

Confirmed end to end against a live agent: with this change the relay logs
AUTH OK and dispatches to the desktop, where the orchestrator runs the turn.

Submission Checklist

  • Tests added or updated (happy path + at least one failure / edge case) — the existing session-start test now asserts customLlmExtraBody, plus a dedicated case covering the token-delivery contract on its own. Failure paths for this hook (signed-URL rejection, SDK onError, disconnect) are already covered in the same suite.
  • Diff coverage ≥ 80% — both changed lines are inside startSession, exercised by every passing case in useRealtimeVoiceSession.test.ts (9 passing).
  • Coverage matrix updated — N/A: behaviour-only fix inside an already-listed surface (5.1.x Voice); no feature row added, removed, or renamed.
  • No new external network dependencies introduced — the ElevenLabs SDK is already a dependency; this only adds a field to an existing call.
  • Manual smoke checklist updated — N/A: realtime voice ships dark behind VITE_VOICE_MODE and is not on a release-cut surface.
  • Linked issue closed via Closes #NNN — see ## Related.

Impact

Desktop only, and only the realtime voice mode, which is gated behind
VITE_VOICE_MODE plus the persisted mascot.voiceMode and ships off by
default. No change to the classic turn-based voice path.

Security: the value sent is the same short-lived, purpose-scoped relay token
already passed as userId — no new secret leaves the client, and the relay
still verifies the signature rather than trusting any id in the payload.

Ordering: the backend accepts the token from both the top level and
elevenlabs_extra_body, so the two changes are independent and this can merge
in either order.

Related

Depends on tinyhumansai/backend#1230 (relay reads the nested field and keeps the
stream alive while the desktop turn runs).

Follow-up to #5407 / #5399 (ElevenLabs Voice Agents).

A realtime voice session passed the signed relay token only as `userId`.
The provider never puts that value on the Custom-LLM request it makes to our
relay: a capture of a live `POST /voice-agent/chat/completions` carried only
[messages, model, max_tokens, stream, stream_options, temperature, tools], so
the relay had no identity to verify and rejected every turn with "request is
missing the user token". The desktop agent was never asked to answer.

`customLlmExtraBody` does reach that request, forwarded under an
`elevenlabs_extra_body` key, which is where the relay looks. Send the token
there as well; `userId` stays for provider-side attribution.
@YellowSnnowmann
YellowSnnowmann requested a review from a team August 6, 2026 14:32

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 42 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4eff01be-6156-455a-b6be-5679f3e56705

📥 Commits

Reviewing files that changed from the base of the PR and between d781949 and e634fce.

📒 Files selected for processing (2)
  • app/src/features/human/voice/useRealtimeVoiceSession.test.ts
  • app/src/features/human/voice/useRealtimeVoiceSession.ts

Comment @coderabbitai help to get the list of available commands.

@M3gA-Mind
M3gA-Mind merged commit bb4493c into tinyhumansai:main Aug 6, 2026
23 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants