Skip to content

fix(streaming): stop fallback no longer clobbers a slow engine finalize (tail-word loss) - #225

Open
initcore0 wants to merge 1 commit into
mainfrom
claude/amazing-fermat-ce47c4
Open

fix(streaming): stop fallback no longer clobbers a slow engine finalize (tail-word loss)#225
initcore0 wants to merge 1 commit into
mainfrom
claude/amazing-fermat-ce47c4

Conversation

@initcore0

Copy link
Copy Markdown
Owner

Problem

stopAppleSpeech() arms a one-shot 2.0s stuck-session fallback that completes the session with the current streamingText if no final arrived. For Parakeet, runStop drains the entire queued audio feed through the decoder and then awaits session.finish() — with any decode backlog this exceeds 2s, so the fallback fired first with the stale partial, set appleDidCompleteFinal, and the engine's genuine final (containing the tail words) was dropped by the guard in handleAppleSpeechFinal. The last words of a long dictation were silently lost even though the engine decoded them. WhisperKit's finalizeTail re-decode has the same window. (Second tail-word-loss path catalogued in the #224 follow-ups.)

Fix

Of the options considered — (a) let a genuine final supersede a fallback-completed session, (b) skip/scale the fallback while the engine is finalizing, (c) re-arm while the drain runs — this implements b+c, which keeps the completion guard's single-final invariant intact instead of retrofitting supersede/delta-paste logic onto a completed session:

  • StreamingTranscriptionEngine.isFinalizing (new, default false): true while a non-cancel stop is still actively producing the genuine final. Parakeet + WhisperKit implement it as a pending-stop counter (a bool would let session N's completion clear session N+1's already-enqueued finalize), set at stop enqueue (runStop can sit behind a slow predecessor on the lifecycle chain) and balanced by defer on every runStop exit path.
  • StreamingRoutePolicy.runStopFallback (pure, @MainActor, injectable sleep): re-arming poll replacing the one-shot timer.
    • Re-arms every 2s while the engine reports finalizing.
    • One grace poll after finalizing ends — the delivered final still rides a main-actor Task hop into handleAppleSpeechFinal; firing in that gap would still clobber it.
    • 30s hard cap: a hung finalize still ends in the stale-partial fallback, preserving the stuck-session guarantee the timer exists for.
    • Engines without the signal (Apple Speech, SpeechAnalyzer — both deliver finals synchronously/guaranteed — plus lean-build stubs and test fakes) keep the exact old behavior: fallback at the first 2s poll.

AppState.stopAppleSpeech now just wires the loop with its existing session fence; all new semantics live tested in core.

Tests

StreamingStopFallbackTests (7 cases) drives the loop with a scripted slow engine:

  • testRearmsThroughSlowEngineDrainAndNeverClobbersGenuineFinal — the bug case: 3 polls of drain, then the genuine final lands; fails against the old one-shot semantics (stale completion at poll 1).
  • Grace-poll covers the delivery hop; grace is one poll, not forever; 30s hard cap; fence short-circuit; stuck-session latency stays one interval; protocol default is false.

swift test: 1911 tests green. ./build.sh: clean.

🤖 Generated with Claude Code

The 2.0s stuck-session fallback in stopAppleSpeech was a one-shot deadline.
Parakeet's runStop drains the entire queued audio feed through the decoder
before session.finish() — on a long dictation that exceeds 2s, so the timer
fired first with the STALE streamingText, set appleDidCompleteFinal, and the
engine's genuine final (holding the tail words) was dropped by the completion
guard. Net effect: the last words of a long dictation silently lost even
though the engine decoded them. WhisperKit's finalizeTail re-decode has the
same window.

Fix: the fallback is now a re-arming poll (StreamingRoutePolicy.runStopFallback,
pure + tested) gated on a new StreamingTranscriptionEngine.isFinalizing signal:

- While the engine reports finalizing, the fallback re-arms instead of firing.
- One grace poll after finalizing ends covers the final's main-actor Task hop
  into handleAppleSpeechFinal (completing in that gap would still clobber it).
- A 30s hard cap fires the stale-partial fallback anyway if the finalize hangs,
  preserving the stuck-session guarantee the timer exists for.
- Engines without the signal (Apple Speech, SpeechAnalyzer, stubs, fakes) keep
  the default false — first-poll fallback, identical to the old 2s behavior.

Parakeet and WhisperKit report finalizing via a pending-non-cancel-stop counter
(not a bool: on a fast stop→start→stop, session N's completion must not clear
session N+1's already-counted finalize), incremented at stop ENQUEUE (runStop
can sit behind a slow predecessor in the lifecycle chain) and balanced on every
runStop exit path.

Tests: StreamingStopFallbackTests drives the loop with a scripted slow engine —
the drain/no-clobber case fails against the old one-shot semantics; grace,
hard-cap, fence, and default-signal cases pinned alongside.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant