Skip to content

fix: preserve interrupted reasoning and improve usage settlement#517

Merged
chenyme merged 1 commit into
devfrom
streaming
Jul 20, 2026
Merged

fix: preserve interrupted reasoning and improve usage settlement#517
chenyme merged 1 commit into
devfrom
streaming

Conversation

@chenyme

@chenyme chenyme commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix interrupted model generations losing streamed reasoning content and reporting incomplete token usage.

This change:

  • Flushes and persists buffered reasoning with an independent settlement context after cancellation.
  • Prevents canceled request contexts from blocking terminal trace persistence.
  • Preserves the richer live reasoning trace on the frontend for interrupted responses only.
  • Uses upstream usage when available and estimates missing output/reasoning usage from visible content.
  • Records usage provenance as observed, recovered, estimated, or mixed.
  • Retains estimated input usage from earlier tool-loop calls when the current Responses API usage is recovered.
  • Adds an SQLite-backed integration test covering cancellation, persistence, and trace reload.

Change type

  • Bug fix
  • Feature
  • Documentation
  • Refactor
  • Configuration / deployment
  • Security hardening
  • Other

Affected areas

  • Frontend / UI
  • Backend / API
  • Authentication / authorization
  • Conversations / streaming
  • Files / RAG / extraction
  • Model routing / providers
  • MCP / tools
  • Billing / payments
  • Admin console
  • Deployment / Docker / configuration
  • Documentation

Verification

  • pnpm --filter @deeix/web check
  • pnpm --filter @deeix/web build
  • cd backend && go test ./internal/application/conversation ./internal/application/billing -count=1
  • git diff --check HEAD
  • Verified cancellation against the locally configured PostgreSQL LongCat-2.1 model.

Screenshots, API examples, or logs

Local LongCat-2.1 cancellation verification:

  • Reasoning received before cancellation: 80 bytes
  • Persisted reasoning after reload: 80 bytes
  • Estimated reasoning usage: 18 tokens
  • Usage source: estimated

The upstream did not return final usage after cancellation, so visible-content estimation was used as the fallback.

Configuration, migration, and compatibility notes

  • No database migration is required.
  • No public API contract or Swagger changes are required.
  • Existing successful generation behavior is unchanged.
  • Recovered upstream output and reasoning usage remains authoritative.
  • Token estimates are used only when final upstream usage is unavailable or incomplete.
  • Usage provenance is stored in the existing pricing snapshot JSON.

Documentation

  • Documentation is not needed for this change.
  • Documentation was updated.
  • Documentation still needs to be updated.

Security and privacy

  • No secrets, tokens, credentials, local config, or personal data are included.
  • User data access remains scoped by authenticated user context unless an admin-only path explicitly requires broader access.
  • Security-sensitive behavior was reviewed, including authentication, authorization, provider routing, file processing, billing, and admin APIs where relevant.

Checklist

  • I searched existing issues and pull requests.
  • Changes are focused and do not include unrelated refactors.
  • Tests or static verification were run where practical.
  • User-facing behavior, deployment steps, API contracts, or configuration changes are documented.
  • Generated artifacts are included only when this project explicitly requires them.
  • Caches, build output, .pyc files, .env files, and local storage data are not committed.

@chenyme chenyme self-assigned this Jul 20, 2026
@chenyme chenyme linked an issue Jul 20, 2026 that may be closed by this pull request
3 tasks
@chenyme
chenyme merged commit afca792 into dev Jul 20, 2026
11 checks passed
@chenyme
chenyme deleted the streaming branch July 20, 2026 05:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: 模型输出时终止会丢失部分输出内容和token统计不准确

1 participant