Skip to content

Play dictation start cue after capture is ready #478

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
freshyjmp wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Play dictation start cue after capture is ready #478

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
4e9fa70
fix: play start cue after capture is ready
freshyjmp Jun 29, 2026
a8890dc
perf: reuse stopped audio engine briefly
freshyjmp Jun 30, 2026
33bae45
fix: address capture readiness review feedback
freshyjmp Jun 30, 2026
73cf1e4
fix: recover hotkey event tap asynchronously
freshyjmp Jun 30, 2026
0c81e63
fix: tie start cue to recording session
freshyjmp Jun 30, 2026
311a3cf
test: cover start cue readiness gating
freshyjmp Jun 30, 2026
53da738
fix: address audio readiness review follow-ups
freshyjmp Jun 30, 2026
21a82d1
fix: require fresh samples after route recovery
freshyjmp Jun 30, 2026
84115b7
fix: avoid stale start cue recovery races
freshyjmp Jun 30, 2026
fa95c81
fix: reuse cached startup recovery decision
freshyjmp Jun 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Fluid.xcodeproj/project.pbxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
7C91B0012F42AA0100C0DEF0 /* HotkeyShortcutTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7C91B0022F42AA0100C0DEF0 /* HotkeyShortcutTests.swift */; };
7CDB0A2D2F3C4D5600FB7CAD /* DictationE2ETests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7CDB0A292F3C4D5600FB7CAD /* DictationE2ETests.swift */; };
7CDB0A2E2F3C4D5600FB7CAD /* AudioFixtureLoader.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7CDB0A2A2F3C4D5600FB7CAD /* AudioFixtureLoader.swift */; };
7CFA10012F54000100C0DEF0 /* StartCueCaptureReadinessTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 7CFA10022F54000100C0DEF0 /* StartCueCaptureReadinessTests.swift */; };
86CAA2D4EF18433096185602 /* LLMClientRequestBodyTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 343B29013F4441D6A797D12D /* LLMClientRequestBodyTests.swift */; };
7CDB0A2F2F3C4D5600FB7CAD /* dictation_fixture.wav in Resources */ = {isa = PBXBuildFile; fileRef = 7CDB0A2B2F3C4D5600FB7CAD /* dictation_fixture.wav */; };
7CDB0A302F3C4D5600FB7CAD /* XCTest.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = 7CDB0A2C2F3C4D5600FB7CAD /* XCTest.framework */; };
Expand All @@ -36,6 +37,7 @@
7C91B0022F42AA0100C0DEF0 /* HotkeyShortcutTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HotkeyShortcutTests.swift; sourceTree = "<group>"; };
7CDB0A292F3C4D5600FB7CAD /* DictationE2ETests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = DictationE2ETests.swift; sourceTree = "<group>"; };
343B29013F4441D6A797D12D /* LLMClientRequestBodyTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LLMClientRequestBodyTests.swift; sourceTree = "<group>"; };
7CFA10022F54000100C0DEF0 /* StartCueCaptureReadinessTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = StartCueCaptureReadinessTests.swift; sourceTree = "<group>"; };
7CDB0A2A2F3C4D5600FB7CAD /* AudioFixtureLoader.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AudioFixtureLoader.swift; sourceTree = "<group>"; };
7CDB0A2B2F3C4D5600FB7CAD /* dictation_fixture.wav */ = {isa = PBXFileReference; lastKnownFileType = audio.wav; path = dictation_fixture.wav; sourceTree = "<group>"; };
7CDB0A2C2F3C4D5600FB7CAD /* XCTest.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = XCTest.framework; path = Platforms/MacOSX.platform/Developer/Library/Frameworks/XCTest.framework; sourceTree = DEVELOPER_DIR; };
Expand Down Expand Up @@ -107,6 +109,7 @@
7CDB0A292F3C4D5600FB7CAD /* DictationE2ETests.swift */,
7C91B0022F42AA0100C0DEF0 /* HotkeyShortcutTests.swift */,
343B29013F4441D6A797D12D /* LLMClientRequestBodyTests.swift */,
7CFA10022F54000100C0DEF0 /* StartCueCaptureReadinessTests.swift */,
);
path = FluidDictationIntegrationTests;
sourceTree = "<group>";
Expand Down Expand Up @@ -262,6 +265,7 @@
7CDB0A2D2F3C4D5600FB7CAD /* DictationE2ETests.swift in Sources */,
7C91B0012F42AA0100C0DEF0 /* HotkeyShortcutTests.swift in Sources */,
86CAA2D4EF18433096185602 /* LLMClientRequestBodyTests.swift in Sources */,
7CFA10012F54000100C0DEF0 /* StartCueCaptureReadinessTests.swift in Sources */,
);
runOnlyForDeploymentPostprocessing = 0;
};
Expand Down
48 changes: 39 additions & 9 deletions Sources/Fluid/ContentView.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2830,15 +2830,14 @@ struct ContentView: View {
self.menuBarManager.showRecordingOverlayImmediately()
}

if !self.isRecordingForCommand, !self.isRecordingForRewrite {
TranscriptionSoundPlayer.shared.playStartSound()
}

Task {
await self.asr.start()
if !self.asr.isRunning {
self.menuBarManager.hideRecordingOverlayImmediately(reason: "asr_start_failed")
return
}
guard let sessionID = self.asr.currentRecordingSessionID else { return }
await self.playStartCueWhenCaptureReady(sessionID: sessionID)
}

// Pre-load model in background while recording (avoids 10s freeze on stop)
Expand Down Expand Up @@ -3077,9 +3076,12 @@ struct ContentView: View {
"Starting voice recording for command",
source: "ContentView"
)
TranscriptionSoundPlayer.shared.playStartSound()
Task {
await self.asr.start()
guard self.asr.isRunning,
let sessionID = self.asr.currentRecordingSessionID
else { return }
await self.playStartCueWhenCaptureReady(sessionID: sessionID)
}
},
rewriteModeCallback: {
Expand Down Expand Up @@ -3112,9 +3114,12 @@ struct ContentView: View {

// Start recording immediately for the edit instruction
DebugLogger.shared.info("Starting voice recording for edit mode", source: "ContentView")
TranscriptionSoundPlayer.shared.playStartSound()
Task {
await self.asr.start()
guard self.asr.isRunning,
let sessionID = self.asr.currentRecordingSessionID
else { return }
await self.playStartCueWhenCaptureReady(sessionID: sessionID)
}
},
isDictateRecordingProvider: {
Expand Down Expand Up @@ -3430,24 +3435,49 @@ extension ContentView {
self.appBench("asr_start_skipped reason=already_running")
return
}
if SettingsStore.shared.enableTranscriptionSounds {
TranscriptionSoundPlayer.shared.playStartSound()
}
Task {
let asrStartStartedAt = ProcessInfo.processInfo.systemUptime
DebugLogger.shared.benchmark("APP_BENCH", message: "asr_start_call", source: "AppBenchmark")
await self.asr.start()
if !self.asr.isRunning {
self.menuBarManager.hideRecordingOverlayImmediately(reason: "asr_start_failed")
DebugLogger.shared.benchmark(
"APP_BENCH",
message: "asr_start_return elapsedMs=\(Int(((ProcessInfo.processInfo.systemUptime - asrStartStartedAt) * 1000).rounded()))",
source: "AppBenchmark"
)
return
}
DebugLogger.shared.benchmark(
"APP_BENCH",
message: "asr_start_return elapsedMs=\(Int(((ProcessInfo.processInfo.systemUptime - asrStartStartedAt) * 1000).rounded()))",
source: "AppBenchmark"
)
guard let sessionID = self.asr.currentRecordingSessionID else { return }
await self.playStartCueWhenCaptureReady(sessionID: sessionID)
}
}

private func playStartCueWhenCaptureReady(sessionID: UInt64) async {
let cueWaitStartedAt = ProcessInfo.processInfo.systemUptime
let ready = await self.asr.waitForCaptureReadyForStartCue(sessionID: sessionID)
DebugLogger.shared.benchmark(
"APP_BENCH",
message: "start_cue_ready ready=\(ready) elapsedMs=\(Int(((ProcessInfo.processInfo.systemUptime - cueWaitStartedAt) * 1000).rounded()))",
source: "AppBenchmark"
)

guard ready,
self.asr.isRunning,
self.asr.currentRecordingSessionID == sessionID
else {
DebugLogger.shared.debug("Start cue skipped because capture is no longer active", source: "ContentView")
return
}

TranscriptionSoundPlayer.shared.playStartSound()

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mute capture while playing the start cue

When transcription sounds are enabled and the output is audible to the microphone (built-in speakers, loopback/monitoring, or loud headphones), this plays the start cue while ASRService is still running and its tap is appending to the same buffer that stop() later transcribes and saves. That means the cue itself can be captured at the beginning of the dictation and can pollute the final/streaming transcription; consider temporarily gating or clearing capture around the cue playback after readiness has been established.

Useful? React with 👍 / 👎.

}

private func beginDictationRecording(for selection: SettingsStore.DictationPromptSelection, mode: ActiveRecordingMode) {
let settings = SettingsStore.shared
settings.setDictationPromptSelection(selection, for: .secondary)
Expand Down
Loading