Skip to content

Conversation

@markritterman
Copy link

Problem

When using OpenRouter with streaming and stream_options.include_usage=true, usage data (token counts) is not captured in the StreamEndEvent.

Root Cause

OpenRouter sends usage data in a separate final chunk after the finish_reason chunk:

Chunk N-1: {"choices": [{"finish_reason": "stop", ...}]}           <- has finish_reason, NO usage
Chunk N:   {"choices": [{"finish_reason": null, ...}], "usage": {...}}  <- has usage, finish_reason is null

The current code only extracts usage when finish_reason !== null:

if ($rawFinishReason !== null) {
    // ...
    $usage = $this->extractUsage($data);  // Only runs when finish_reason is set
}

Since the usage chunk has finish_reason: null, usage extraction is skipped.

Solution

Move usage extraction outside the finish_reason check so it runs on every chunk. The extractUsage() method already safely returns null when there's no usage data.

Testing

Tested with OpenRouter API (via vLLM endpoint):

Before fix:

StreamEndEvent: usage = null

After fix:

StreamEndEvent: usage = {promptTokens: 27, completionTokens: 2, ...}

Impact

  • No breaking changes
  • Fixes streaming usage tracking for OpenRouter and any OpenAI-compatible API that sends usage in a separate chunk
  • extractUsage() is idempotent and safe to call on every chunk

OpenRouter sends usage data in a separate final chunk after the
finish_reason chunk. The current code only extracts usage when
finish_reason is non-null, but the usage chunk has finish_reason: null.

This moves usage extraction outside the finish_reason check so it
runs on every chunk. extractUsage() already safely returns null
when there's no usage data, so this is safe.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant