feat(core): summarizer streaming/dynamic-context + whisper model fallback + pro bump [WIP/divergent]#532
feat(core): summarizer streaming/dynamic-context + whisper model fallback + pro bump [WIP/divergent]#532dishit-wednesday wants to merge 46 commits into
Conversation
…+ autoprune logs) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…insights
- transcriptSummarizer: thread optional GBNF grammar + repeat_penalty + onToken
through the map/combine passes (applied on the final pass); add abort() that
interrupts the in-flight completion via llmService.stopGeneration and clears
the summarizing flag, so a Stop/preempt frees the single model lock at once
- llm.generateWithMaxTokens: accept { onToken, grammar, repeatPenalty }; a bad
grammar retries ungrammared so it can never brick generation
Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Dev-only ( __DEV__ ) harness to paste a GBNF grammar / temp / prefill and see the next reply constrained, for validating the insights grammar recipe: - devInference / devInferenceStore: armable overrides applied per turn - DevGrammarModal + ChatHeader button (lights when armed) - llm.generateResponse + llmToolGeneration: applyDevGrammarOverrides, with an ungrammared retry so a bad grammar never bricks chat - litert + LiteRTModule.kt: setConstrainedDecoding (LLGuidance json_schema/lark/ regex) armed per-conversation Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Download + unzip ggerganov's per-model -encoder.mlmodelc next to the ggml .bin so whisper.cpp runs the encoder on the Apple Neural Engine (~2-3x faster encode, frees the CPU). ensureCoreMLEncoder backfills a missing encoder once per session; whisperModels gains coreMLUrl per model (absent for the tdrz checkpoint). Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
- Refresh the Hexagon HTP backend libs (v69/v73/v75/v79/v81) for LiteRT. - Add whisper-vad/ggml-silero-v5.1.2.bin (VAD model for speech-gated capture). Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Drop the guard that forced 0 GPU layers on iOS devices under 4GB, so Metal offload is allowed there too (Android caps unchanged). Test updated. Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Cover importance scoring/gating, extractive action-item detection + refine, and insights parse/merge. Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…ens + recorder infra) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Chunk budget scales to the active backend's real context (local/LiteRT/remote) instead of a fixed 2048; threads onToken through all three backends for live streaming; adds abort() to cancel/stop an in-flight summary. Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…is deleted Deleting the active Whisper model now adopts another present model instead of dropping to no-model; refreshPresentModels adopts a present model when the active is missing. Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…a, prompt fix, recorder baseline) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…nning + handoff notes Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…p (ec24cac) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
… default whisperStore deleteModel/deleteModelById now fall back to another on-disk model (mock listDownloadedModels); loadModel auto-enables CoreML when the encoder asset is present. Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThe change adds development-only grammar and constrained-decoding controls, iOS Whisper CoreML encoder support, recording insight tests, model fallback reconciliation, platform-specific GPU behavior, and extensive recorder, intelligence, and transcript-quality planning documents. ChangesOn-device generation and transcription
Estimated code review effort: 4 (Complex) | ~60 minutes 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
/gemini review |
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 9
Note
Due to the large number of review comments, Critical, Major severity comments were prioritized as inline comments.
🟡 Minor comments (24)
src/screens/ChatScreen/ChatScreenComponents.tsx-89-90 (1)
89-90: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
devActivedoesn't account for LiteRT constraints.The selector only checks
s.grammar.trim().length > 0(GBNF), but the store also supportslitertConstraintStringfor LiteRT. If a dev enables the override with only a LiteRT constraint set, the header icon won't light up, making it unclear the override is active.🛡️ Proposed fix
- const devActive = useDevInferenceStore((s) => s.enabled && s.grammar.trim().length > 0); + const devActive = useDevInferenceStore( + (s) => s.enabled && (s.grammar.trim().length > 0 || s.litertConstraintString.trim().length > 0), + );🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/screens/ChatScreen/ChatScreenComponents.tsx` around lines 89 - 90, Update the devActive selector in ChatScreenComponents to consider both configured constraint sources: keep requiring s.enabled, but treat a non-empty trimmed grammar or a non-empty trimmed litertConstraintString as active. Preserve the inactive state when neither constraint is configured.docs/plans/locket-continuous-smart-recorder-handoff.md-168-168 (1)
168-168: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winUse approved brand vocabulary.
“Robustness” is listed as a forbidden term in the repository’s documentation style. Replace it with “whole-day reliability” or another concrete description.
As per coding guidelines, documentation must follow the brand standard and avoid forbidden words.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/locket-continuous-smart-recorder-handoff.md` at line 168, Update the “Whole-day robustness” heading in the documented plan to use approved brand vocabulary, replacing “robustness” with “whole-day reliability” or another concrete description while preserving the heading’s meaning.Source: Coding guidelines
docs/plans/locket-continuous-smart-recorder-handoff.md-136-137 (1)
136-137: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winAdd blank lines around the constants table.
The table immediately follows the heading, triggering Markdownlint MD058.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/locket-continuous-smart-recorder-handoff.md` around lines 136 - 137, Add a blank line between the “Decided constants” heading and its table, and ensure the table has the required surrounding blank lines to satisfy Markdownlint MD058.Source: Linters/SAST tools
docs/plans/locket-insights-handoff.md-29-37 (1)
29-37: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winAdd languages to all fenced code blocks.
Use
textfor the pipeline/configuration blocks so Markdownlint MD040 passes and rendering remains explicit.Also applies to: 176-180
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/locket-insights-handoff.md` around lines 29 - 37, Update the fenced code blocks in the handoff document, including the blocks around the pipeline and the referenced later section, to specify text as the language identifier. Preserve each block’s contents and formatting while ensuring every fence has an explicit language.Source: Linters/SAST tools
docs/plans/locket-insights-screen-plan.md-47-53 (1)
47-53: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winAdd languages to the fenced documentation blocks.
Use
textfor the loose output format and router pseudocode so Markdownlint MD040 passes.Also applies to: 121-129
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/locket-insights-screen-plan.md` around lines 47 - 53, Add the text language identifier to the fenced documentation blocks showing the loose output format and router pseudocode, including the corresponding block at the other referenced section, so every fence has an explicit language for Markdownlint MD040.Source: Linters/SAST tools
docs/plans/locket-insights-handoff.md-1-1 (1)
1-1: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winRemove em dashes from the documentation copy.
Examples include the title, “TL;DR — current state,” “pure heuristics — no model,” and “— llama.rn only.” Replace them with colons, semicolons, or standard hyphens.
As per coding guidelines, documentation copy must not use em dashes.
Also applies to: 13-18, 66-75, 115-123
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/locket-insights-handoff.md` at line 1, Remove every em dash from the documentation copy in the handoff document, including the title and the referenced “TL;DR,” “pure heuristics,” and “llama.rn only” text. Replace each with an appropriate colon, semicolon, or standard hyphen while preserving the original meaning.Source: Coding guidelines
src/services/litert.ts-142-155 (1)
142-155: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winAdd tests for the new LiteRT paths
src/services/litert.ts:142-155,507-511
getContextTokens()has no direct coverage.- The
__DEV__-guardedsetConstrainedDecodingbranch inresetConversation()also needs a dev-mode test.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/services/litert.ts` around lines 142 - 155, The new LiteRT paths lack test coverage. Add tests for getContextTokens() and for resetConversation() with __DEV__ enabled, covering the setConstrainedDecoding branch and its expected invocation when the dev constraint is armed, while preserving existing behavior for disabled or empty constraints.Source: Coding guidelines
docs/plans/continuous-smart-recorder-plan.md-27-29 (1)
27-29: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winAdd a language to the fenced block.
Use
```tsfor the TypeScript snippet so the document passes MD040.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/continuous-smart-recorder-plan.md` around lines 27 - 29, Update the fenced code block containing the RecorderMode type declaration to specify the TypeScript language with a ts fence, ensuring the documentation satisfies MD040.Source: Linters/SAST tools
docs/plans/locket-commit-plan.md-30-36 (1)
30-36: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winAdd blank lines around the table.
Separate the preceding text from the table and the following text to satisfy MD058.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/locket-commit-plan.md` around lines 30 - 36, Add blank lines immediately before and after the “Do NOT commit” table in the plan document, keeping the table contents unchanged so it satisfies the Markdown table-spacing rule MD058.Source: Linters/SAST tools
docs/plans/insights-page-design-brief.md-9-153 (1)
9-153: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winAdd a language to the embedded prompt fence.
Use
```textso the document passes MD040.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/insights-page-design-brief.md` around lines 9 - 153, Update the embedded prompt fence in the design brief to specify the text language by changing the opening fence to ```text, while leaving the prompt content and closing fence unchanged.Source: Linters/SAST tools
docs/plans/audio-recorder-notes.md-1-1 (1)
1-1: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winReplace the em dash in the title.
The documentation copy standard prohibits em dashes; use a colon or hyphen instead.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/audio-recorder-notes.md` at line 1, Replace the em dash in the “Audio / recorder work — current state” title with a permitted colon or hyphen, preserving the rest of the title unchanged.Source: Coding guidelines
docs/plans/auto-detect-recording-plan.md-71-71 (1)
71-71: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winAdd a language to the pseudocode fence.
Use
```textso the document passes MD040.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/auto-detect-recording-plan.md` at line 71, Update the pseudocode code fence in auto-detect-recording-plan.md to specify the text language as ```text, preserving the enclosed content unchanged.Source: Linters/SAST tools
docs/plans/continuous-smart-recorder-plan.md-1-1 (1)
1-1: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winReplace the em dash in the title.
The documentation copy standard prohibits em dashes.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/continuous-smart-recorder-plan.md` at line 1, Replace the em dash in the “Continuous-Smart Recorder Mode — Implementation Plan” title with an allowed non-em-dash separator, preserving the title wording.Source: Coding guidelines
docs/plans/audio-recorder-notes.md-24-32 (1)
24-32: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winAdd a language to the fenced block.
Use
```textor another appropriate language so the document passes MD040.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/audio-recorder-notes.md` around lines 24 - 32, Update the fenced code block containing the commit list in the audio recorder notes to include an explicit language identifier, preferably text, immediately after the opening fence; leave the listed commits unchanged.Source: Linters/SAST tools
docs/plans/locket-commit-plan.md-21-27 (1)
21-27: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winAdd a language to the fenced block.
Use
```gitignoreso the document passes MD040.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/locket-commit-plan.md` around lines 21 - 27, Update the fenced code block in the locket commit plan to specify the gitignore language, changing the opening fence to gitignore while preserving its existing ignore patterns.Source: Linters/SAST tools
docs/plans/auto-detect-recording-plan.md-1-1 (1)
1-1: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winReplace the em dash in the title.
The documentation copy standard prohibits em dashes.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/auto-detect-recording-plan.md` at line 1, Replace the em dash in the “Auto-detect (VAD-gated) recording” title with an approved non-em-dash separator, preserving the title wording and meaning.Source: Coding guidelines
docs/plans/insights-page-design-brief.md-1-1 (1)
1-1: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winReplace the em dash in the title.
The documentation copy standard prohibits em dashes.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/insights-page-design-brief.md` at line 1, Update the “Insights Page — Design Brief” heading to replace the em dash with an approved non-em-dash separator, preserving the title wording.Source: Coding guidelines
docs/plans/locket-commit-plan.md-10-10 (1)
10-10: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winList the required commit-time checks explicitly.
“As per coding guidelines, staged Kotlin/Gradle changes must run
compileDebugKotlin,lintDebug, andnpm run test:android, while staged JavaScript/TypeScript changes must run staged-file eslint,tsc --noEmit, andnpm test.”The current shorthand is too vague to enforce those gates reliably.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/locket-commit-plan.md` at line 10, Update the commit-time quality-gates statement in the plan to explicitly require compileDebugKotlin, lintDebug, and npm run test:android for staged Kotlin/Gradle changes, and staged-file eslint, tsc --noEmit, and npm test for staged JavaScript/TypeScript changes; retain the instruction to pause and report native hook failures instead of using --no-verify without approval.Source: Coding guidelines
docs/plans/combined-intelligence-layer-plan.md-116-127 (1)
116-127: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winEm dashes used in the "Important flags" section.
Several bullets use em dashes (e.g., "biggest cost saver;" / "turn it on for" / "extra work if iOS matters" / "must add device" / "ingest into
meetings"), which the docs brand-tone/copy standard disallows. As per coding guidelines,docs/text changes "must followdocs/brand_tone_voice.mdand its copy standards (proof-first, no exclamation marks, no em dashes, no curly quotes, etc.)."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/combined-intelligence-layer-plan.md` around lines 116 - 127, Update the “Important flags” bullets in the combined intelligence layer plan to remove all em dashes and replace them with approved punctuation or sentence structure, while preserving each bullet’s meaning and existing proof-first tone. Ensure the edited text follows the copy standards in docs/brand_tone_voice.md, including no em dashes.Source: Coding guidelines
docs/plans/transcript-quality-experiment.md-82-82 (1)
82-82: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winEm dash in decode-params bullet.
"
--no-context) — the loop killer, test first" uses an em dash, against the docs brand-tone standard of no em dashes. As per coding guidelines.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/transcript-quality-experiment.md` at line 82, Update the condition_on_previous_text bullet in transcript-quality-experiment.md to replace the em dash between “--no-context” and “the loop killer” with approved punctuation, preserving the existing meaning and wording.Source: Coding guidelines
docs/plans/transcript-quality-findings.md-260-262 (1)
260-262: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winStale diarization conclusion in the DELIVERABLES summary contradicts the doc's own later findings.
Item 5 says diarization needs a FIXED cluster count (auto-count fails 23 vs 2, cost ~10× transcription). But the body's later "Diarization status" section (Lines 149-177) reports auto-count SOLVED via NME-SC, and the "Cost correction" (Lines 186-190) revises the cost to ~⅓ of transcription, not 10×. This is the actionable summary readers will follow — leaving it stale risks shipping the older FastClustering/fixed-count approach.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/transcript-quality-findings.md` around lines 260 - 262, The diarization summary in item 5 is stale and contradicts the later “Diarization status” and “Cost correction” findings. Update item 5 to describe NME-SC auto-count as solved and reflect the revised cost of approximately one-third of transcription, removing the fixed-cluster-count and outdated 10× cost claims.docs/plans/transcript-quality-findings.md-11-11 (1)
11-11: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winEm dashes throughout violate the brand tone guideline.
This doc (and its sibling handoff doc) use em dashes ("—") extensively, e.g. Lines 11, 26, 93, 106. As per coding guidelines, "Any change to website copy, essays, docs text, UI strings, or marketing content must follow
docs/brand_tone_voice.mdand its copy standards (proof-first, no exclamation marks, no em dashes, no forbidden words, no curly quotes, etc.)."Also applies to: 26-26, 93-93, 106-106
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/transcript-quality-findings.md` at line 11, Replace all em dashes in docs/plans/transcript-quality-findings.md and its sibling handoff document with brand-compliant punctuation or wording, including the occurrences near the referenced lines. Preserve the original meaning and ensure no em dashes remain in either document.Source: Coding guidelines
docs/plans/session-handoff-transcription-intelligence.md-26-27 (1)
26-27: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winEm dashes throughout violate the brand tone guideline.
Same pattern as the sibling findings doc — e.g. Lines 26, 27, 65. As per coding guidelines, docs text "must follow
docs/brand_tone_voice.mdand its copy standards (proof-first, no exclamation marks, no em dashes, no forbidden words, no curly quotes, etc.)."Also applies to: 65-65
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/session-handoff-transcription-intelligence.md` around lines 26 - 27, Update the prose in the session handoff transcription intelligence plan, especially the bullets “On-device / privacy by architecture, not by promise” and “Honest limitations over hype,” to remove em dashes and replace them with brand-compliant punctuation or sentence structure. Apply the same change to the text around the additionally referenced line, preserving the meaning and all other brand-tone requirements.Source: Coding guidelines
docs/plans/session-handoff-transcription-intelligence.md-207-208 (1)
207-208: 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick winRedact the personal email and device UDID from this planning doc. They are unnecessary in tracked repo text and expose personal identifiers.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/session-handoff-transcription-intelligence.md` around lines 207 - 208, Update the planning document’s “Commit co-author” and iOS device build instructions to remove the personal email address and device UDID while preserving the surrounding attribution and build guidance in anonymized form.
🧹 Nitpick comments (9)
src/components/DevGrammarModal.tsx (1)
176-186: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueExtract the nested ternary placeholder for readability.
The LiteRT constraint placeholder uses a nested ternary that's hard to scan at a glance. Extracting it to a variable or helper would improve clarity.
♻️ Optional refactor
- placeholder={litertType === 'json_schema' ? '{"type":"object","properties":{...}}' : litertType === 'regex' ? '(TITLE: .+\\n)+' : 'start: ...'} + placeholder={ + litertType === 'json_schema' + ? '{"type":"object","properties":{...}}' + : litertType === 'regex' + ? '(TITLE: .+\\n)+' + : 'start: ...' + }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/components/DevGrammarModal.tsx` around lines 176 - 186, In DevGrammarModal, extract the nested ternary used by the TextInput placeholder into a clearly named variable or helper based on litertType, then pass that value to placeholder while preserving the existing json_schema, regex, and default placeholder strings.docs/plans/locket-insights-handoff.md (1)
106-106: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocument the UI icon instead of an emoji.
The
⚡notation conflicts with the repository’s no-emoji UI rule. Specify the intended vector-icon name so the implementation does not copy the glyph.As per coding guidelines, UI icons must use the shared icon library and UI text must not contain emojis.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/locket-insights-handoff.md` at line 106, Update the LocketTodayScreen.tsx entry in the handoff table to replace the ⚡ emoji notation with the intended shared vector-icon name, preserving the described hub-entry behavior and avoiding emoji UI text.Source: Coding guidelines
src/services/devInference.ts (2)
27-27: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winAvoid the backtracking-prone grammar scan.
Sonar flags this regex as potentially super-linear. Scan individual lines instead so pasted grammars cannot cause avoidable JS-thread work.
Proposed fix
- if (!/(^|\n)\s*root\s*::=/.test(g)) return 'no "root" rule'; + const hasRootRule = g.split(/\r?\n/).some(line => /^\s*root\s*::=/.test(line)); + if (!hasRootRule) return 'no "root" rule';🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/services/devInference.ts` at line 27, Replace the backtracking-prone regex in the grammar validation flow with a line-by-line scan of g, checking each line for a root rule after optional whitespace and confirming the rule delimiter. Preserve the existing 'no "root" rule' return when no line matches, and avoid applying a multiline regex to the entire grammar.Source: Linters/SAST tools
31-76: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winSplit
applyDevGrammarOverridesto satisfy the complexity gate.Sonar reports cognitive complexity 19, above the configured limit of 15. Extract grammar handling, scalar overrides, and logging/error cleanup into focused helpers.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/services/devInference.ts` around lines 31 - 76, Reduce the cognitive complexity of applyDevGrammarOverrides by extracting focused helpers for grammar validation/application, scalar overrides, and final logging/error cleanup. Keep applyDevGrammarOverrides responsible for the enabled-state check and orchestration, preserving existing tool removal, invalid-grammar fallback, temperature/prefix/maxWords updates, logging, stale-error clearing, and return behavior.Source: Linters/SAST tools
docs/plans/transcript-quality-experiment.md (1)
10-10: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd a language hint to the main instructional fenced block (MD040).
📝 Suggested fix
-``` +```text You are a researcher solving a hard, open problem: ...Also applies to: 165-165
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/transcript-quality-experiment.md` at line 10, Add a language identifier such as text to the main instructional fenced code block in transcript-quality-experiment.md, including the corresponding fenced block at the other reported location, so both satisfy MD040 while preserving their content.Source: Linters/SAST tools
src/services/llm.ts (1)
20-20: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMissing test coverage for new dev-grammar/retry logic.
The new grammar-override, error-recording, and single-retry-without-grammar paths in
generateResponseandgenerateWithMaxTokens(including the reset bug above) have no accompanying unit tests in this cohort (onlyllmHelpers.test.ts,whisperService.test.ts,whisperStore.test.tsare present). As per coding guidelines, "Coverage for new code must include both core andpro/, and new code should reach 100% statements, branches, functions, and lines with all branches exercised."Want me to draft unit tests for
LLMService.generateResponse/generateWithMaxTokenscovering the grammar-apply, grammar-fail-retry, and non-grammar-fail-rethrow branches?Also applies to: 235-264, 304-345
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/services/llm.ts` at line 20, Add unit tests for LLMService.generateResponse and generateWithMaxTokens covering grammar application, grammar-override failure with one retry without grammar, error recording, reset behavior, and non-grammar failure rethrow paths. Provide equivalent coverage in both core and pro test suites, exercising all new branches, functions, statements, and lines.Source: Coding guidelines
docs/plans/locket-llm-insights-plan.md (1)
53-56: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd a language hint to fenced code blocks (MD040).
Three fenced blocks (pipeline diagrams / data-model snippet) have no language specified, flagged by markdownlint.
📝 Suggested fix
-``` +```text conversation/meeting ends (or time-of-day digest) -> local notification(apply the same
textlanguage tag to the blocks at Lines 80 and 93.)Also applies to: 80-86, 93-101
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/plans/locket-llm-insights-plan.md` around lines 53 - 56, Specify the text language on the three fenced code blocks in the plan document: the pipeline diagram near the conversation/meeting flow and the blocks near the referenced data-model sections. Add the text fence tag while preserving each block’s existing content.Source: Linters/SAST tools
src/services/transcriptSummarizer.ts (1)
185-204: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMissing test coverage for
abort()and dynamic context sizing.No test file for
TranscriptSummarizerService(coveringresolveContextTokens, dynamicINPUT_CONTEXT_FRACTIONchunking, grammar/repeatPenalty wiring, orabort()) is included in this cohort. As per coding guidelines, "Coverage for new code must include both core andpro/, and new code should reach 100% statements, branches, functions, and lines with all branches exercised," and per the same guideline, tests here "must cover approved behavior changes ... and assert outcomes/invariants rather than merely asserting that a gate or mock was called."Happy to draft tests for
resolveContextTokensbranch coverage (litert/remote/local) and anabort()-mid-map-loop scenario if useful.Also applies to: 225-329
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/services/transcriptSummarizer.ts` around lines 185 - 204, Add comprehensive tests for TranscriptSummarizerService covering resolveContextTokens across litert, remote, and local branches; dynamic INPUT_CONTEXT_FRACTION chunk sizing; grammar and repeatPenalty propagation; and abort() during an active map loop, asserting resulting behavior and invariants rather than only mock calls. Provide equivalent coverage in both core and pro test suites, including all relevant statements, branches, functions, and lines.Source: Coding guidelines
src/services/whisperService.ts (1)
367-391: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDesign is sound; minor duplication risk flagged by the code's own comment.
Asset-presence gating (not a user toggle) is well-reasoned and documented. One nit: the
.bin→-encoder.mlmodelcderivation is duplicated betweencoreMLPathFor(Line 100-102) and the inline regex at Line 375 — the comment at Line 98-99 already flags the "keep in lockstep" risk. Consider havingcoreMLPathFordelegate to a shared path-only helper and reuse it here.♻️ Optional consolidation
- private coreMLPathFor(modelId: string): string { - return this.getModelPath(modelId).replace(/\.bin$/i, '-encoder.mlmodelc'); - } + private coreMLPathForFile(binPath: string): string { + return binPath.replace(/\.bin$/i, '-encoder.mlmodelc'); + } + private coreMLPathFor(modelId: string): string { + return this.coreMLPathForFile(this.getModelPath(modelId)); + }- const coreMLPath = modelPath.replace(/\.bin$/i, '-encoder.mlmodelc'); + const coreMLPath = this.coreMLPathForFile(modelPath);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/services/whisperService.ts` around lines 367 - 391, Consolidate CoreML encoder path derivation by reusing the existing coreMLPathFor helper or a shared path-only helper in the model-loading block around useCoreML, instead of duplicating the inline “.bin” to “-encoder.mlmodelc” regex. Preserve the current iOS-only asset-presence gating and backfill behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/plans/auto-detect-recording-plan.md`:
- Around line 76-93: Update the recording state flow so the IDLE-to-CAPTURING
path flushes only preroll data before writing the current buffer once, removing
the duplicate write caused by flushing the current buffer. In the max-duration
branch, require state == CAPTURING before rolling so a silence-triggered close
cannot reopen a file in the same iteration.
In `@docs/plans/continuous-smart-recorder-plan.md`:
- Line 64: Update the continuous smart recorder plan’s roll behavior to never
silently discard finalized all-noise recordings. Replace the discard requirement
with preserving the recording as an unprocessed item, or retaining a recoverable
original before excluding it from the main feed, while keeping the existing 0:00
guard.
- Around line 104-110: Resolve the contradiction in section “The ‘nothing lost’
guarantee” by renaming the guarantee and revising its contract to acknowledge
that crashes may lose only the last few unflushed seconds. Keep the existing
backup, duration gate, rolling, and crash-safety details, but do not claim that
no audio can ever be lost unless durable flushing before capture acknowledgment
is implemented.
- Around line 46-53: Update the periodic Silero checkpoint flow in section 2b so
the AAC restore snapshot from ensureCompressedBackup reflects the complete
current open file immediately before any prune. Refresh the backup after each
append or prune, or keep 2b disabled until finalization; do not allow
restoration to omit audio recorded after an earlier backup.
In `@src/services/devInference.ts`:
- Around line 84-88: Update noteDevGrammarError to restore the original tools
and tool_choice metadata removed by applyDevGrammarOverrides before
generateWithToolsImpl retries. Preserve the existing grammar cleanup, error
recording, and warning behavior, ensuring the fallback retains the tool request
fields.
In `@src/services/llmToolGeneration.ts`:
- Around line 157-161: Update the catch branch in the completion flow around
safeCompletion so fallback is attempted only when the error is confirmed to be a
dev-grammar rejection. Preserve noteDevGrammarError and the ungrammared retry
for that case, but immediately rethrow cancellation, model availability,
OOM/native, and other non-grammar failures; do not use devGrammarApplied alone
as the retry condition.
- Around line 154-161: Reset the streaming state before the fallback retry in
the catch block of the generateWithTools flow: recreate fullResponse,
collectedToolCalls, tokenCount, firstReceived, and toolCallFilter so the second
safeCompletion call starts clean and cannot duplicate partial output. Preserve
the existing retry behavior and dev-grammar error recording.
In `@src/services/whisperService.ts`:
- Around line 97-170: Prevent ensureCoreMLEncoder from moving or removing the
shared small.en encoder when processing small.en-tdrz. Extract reused encoders
into a per-model scratch location, or otherwise copy/retain the canonical source
directory while placing the target at the requested model path; preserve
ggml-small.en-encoder.mlmodelc for subsequent small.en loads.
- Around line 260-264: Update the iOS CoreML backfill in downloadModel so
ensureCoreMLEncoder(modelId) is launched fire-and-forget rather than awaited,
while retaining its non-fatal error handling and allowing downloadModel to
resolve immediately after the main model is validated.
---
Minor comments:
In `@docs/plans/audio-recorder-notes.md`:
- Line 1: Replace the em dash in the “Audio / recorder work — current state”
title with a permitted colon or hyphen, preserving the rest of the title
unchanged.
- Around line 24-32: Update the fenced code block containing the commit list in
the audio recorder notes to include an explicit language identifier, preferably
text, immediately after the opening fence; leave the listed commits unchanged.
In `@docs/plans/auto-detect-recording-plan.md`:
- Line 71: Update the pseudocode code fence in auto-detect-recording-plan.md to
specify the text language as ```text, preserving the enclosed content unchanged.
- Line 1: Replace the em dash in the “Auto-detect (VAD-gated) recording” title
with an approved non-em-dash separator, preserving the title wording and
meaning.
In `@docs/plans/combined-intelligence-layer-plan.md`:
- Around line 116-127: Update the “Important flags” bullets in the combined
intelligence layer plan to remove all em dashes and replace them with approved
punctuation or sentence structure, while preserving each bullet’s meaning and
existing proof-first tone. Ensure the edited text follows the copy standards in
docs/brand_tone_voice.md, including no em dashes.
In `@docs/plans/continuous-smart-recorder-plan.md`:
- Around line 27-29: Update the fenced code block containing the RecorderMode
type declaration to specify the TypeScript language with a ts fence, ensuring
the documentation satisfies MD040.
- Line 1: Replace the em dash in the “Continuous-Smart Recorder Mode —
Implementation Plan” title with an allowed non-em-dash separator, preserving the
title wording.
In `@docs/plans/insights-page-design-brief.md`:
- Around line 9-153: Update the embedded prompt fence in the design brief to
specify the text language by changing the opening fence to ```text, while
leaving the prompt content and closing fence unchanged.
- Line 1: Update the “Insights Page — Design Brief” heading to replace the em
dash with an approved non-em-dash separator, preserving the title wording.
In `@docs/plans/locket-commit-plan.md`:
- Around line 30-36: Add blank lines immediately before and after the “Do NOT
commit” table in the plan document, keeping the table contents unchanged so it
satisfies the Markdown table-spacing rule MD058.
- Around line 21-27: Update the fenced code block in the locket commit plan to
specify the gitignore language, changing the opening fence to gitignore while
preserving its existing ignore patterns.
- Line 10: Update the commit-time quality-gates statement in the plan to
explicitly require compileDebugKotlin, lintDebug, and npm run test:android for
staged Kotlin/Gradle changes, and staged-file eslint, tsc --noEmit, and npm test
for staged JavaScript/TypeScript changes; retain the instruction to pause and
report native hook failures instead of using --no-verify without approval.
In `@docs/plans/locket-continuous-smart-recorder-handoff.md`:
- Line 168: Update the “Whole-day robustness” heading in the documented plan to
use approved brand vocabulary, replacing “robustness” with “whole-day
reliability” or another concrete description while preserving the heading’s
meaning.
- Around line 136-137: Add a blank line between the “Decided constants” heading
and its table, and ensure the table has the required surrounding blank lines to
satisfy Markdownlint MD058.
In `@docs/plans/locket-insights-handoff.md`:
- Around line 29-37: Update the fenced code blocks in the handoff document,
including the blocks around the pipeline and the referenced later section, to
specify text as the language identifier. Preserve each block’s contents and
formatting while ensuring every fence has an explicit language.
- Line 1: Remove every em dash from the documentation copy in the handoff
document, including the title and the referenced “TL;DR,” “pure heuristics,” and
“llama.rn only” text. Replace each with an appropriate colon, semicolon, or
standard hyphen while preserving the original meaning.
In `@docs/plans/locket-insights-screen-plan.md`:
- Around line 47-53: Add the text language identifier to the fenced
documentation blocks showing the loose output format and router pseudocode,
including the corresponding block at the other referenced section, so every
fence has an explicit language for Markdownlint MD040.
In `@docs/plans/session-handoff-transcription-intelligence.md`:
- Around line 26-27: Update the prose in the session handoff transcription
intelligence plan, especially the bullets “On-device / privacy by architecture,
not by promise” and “Honest limitations over hype,” to remove em dashes and
replace them with brand-compliant punctuation or sentence structure. Apply the
same change to the text around the additionally referenced line, preserving the
meaning and all other brand-tone requirements.
- Around line 207-208: Update the planning document’s “Commit co-author” and iOS
device build instructions to remove the personal email address and device UDID
while preserving the surrounding attribution and build guidance in anonymized
form.
In `@docs/plans/transcript-quality-experiment.md`:
- Line 82: Update the condition_on_previous_text bullet in
transcript-quality-experiment.md to replace the em dash between “--no-context”
and “the loop killer” with approved punctuation, preserving the existing meaning
and wording.
In `@docs/plans/transcript-quality-findings.md`:
- Around line 260-262: The diarization summary in item 5 is stale and
contradicts the later “Diarization status” and “Cost correction” findings.
Update item 5 to describe NME-SC auto-count as solved and reflect the revised
cost of approximately one-third of transcription, removing the
fixed-cluster-count and outdated 10× cost claims.
- Line 11: Replace all em dashes in docs/plans/transcript-quality-findings.md
and its sibling handoff document with brand-compliant punctuation or wording,
including the occurrences near the referenced lines. Preserve the original
meaning and ensure no em dashes remain in either document.
In `@src/screens/ChatScreen/ChatScreenComponents.tsx`:
- Around line 89-90: Update the devActive selector in ChatScreenComponents to
consider both configured constraint sources: keep requiring s.enabled, but treat
a non-empty trimmed grammar or a non-empty trimmed litertConstraintString as
active. Preserve the inactive state when neither constraint is configured.
In `@src/services/litert.ts`:
- Around line 142-155: The new LiteRT paths lack test coverage. Add tests for
getContextTokens() and for resetConversation() with __DEV__ enabled, covering
the setConstrainedDecoding branch and its expected invocation when the dev
constraint is armed, while preserving existing behavior for disabled or empty
constraints.
---
Nitpick comments:
In `@docs/plans/locket-insights-handoff.md`:
- Line 106: Update the LocketTodayScreen.tsx entry in the handoff table to
replace the ⚡ emoji notation with the intended shared vector-icon name,
preserving the described hub-entry behavior and avoiding emoji UI text.
In `@docs/plans/locket-llm-insights-plan.md`:
- Around line 53-56: Specify the text language on the three fenced code blocks
in the plan document: the pipeline diagram near the conversation/meeting flow
and the blocks near the referenced data-model sections. Add the text fence tag
while preserving each block’s existing content.
In `@docs/plans/transcript-quality-experiment.md`:
- Line 10: Add a language identifier such as text to the main instructional
fenced code block in transcript-quality-experiment.md, including the
corresponding fenced block at the other reported location, so both satisfy MD040
while preserving their content.
In `@src/components/DevGrammarModal.tsx`:
- Around line 176-186: In DevGrammarModal, extract the nested ternary used by
the TextInput placeholder into a clearly named variable or helper based on
litertType, then pass that value to placeholder while preserving the existing
json_schema, regex, and default placeholder strings.
In `@src/services/devInference.ts`:
- Line 27: Replace the backtracking-prone regex in the grammar validation flow
with a line-by-line scan of g, checking each line for a root rule after optional
whitespace and confirming the rule delimiter. Preserve the existing 'no "root"
rule' return when no line matches, and avoid applying a multiline regex to the
entire grammar.
- Around line 31-76: Reduce the cognitive complexity of applyDevGrammarOverrides
by extracting focused helpers for grammar validation/application, scalar
overrides, and final logging/error cleanup. Keep applyDevGrammarOverrides
responsible for the enabled-state check and orchestration, preserving existing
tool removal, invalid-grammar fallback, temperature/prefix/maxWords updates,
logging, stale-error clearing, and return behavior.
In `@src/services/llm.ts`:
- Line 20: Add unit tests for LLMService.generateResponse and
generateWithMaxTokens covering grammar application, grammar-override failure
with one retry without grammar, error recording, reset behavior, and non-grammar
failure rethrow paths. Provide equivalent coverage in both core and pro test
suites, exercising all new branches, functions, statements, and lines.
In `@src/services/transcriptSummarizer.ts`:
- Around line 185-204: Add comprehensive tests for TranscriptSummarizerService
covering resolveContextTokens across litert, remote, and local branches; dynamic
INPUT_CONTEXT_FRACTION chunk sizing; grammar and repeatPenalty propagation; and
abort() during an active map loop, asserting resulting behavior and invariants
rather than only mock calls. Provide equivalent coverage in both core and pro
test suites, including all relevant statements, branches, functions, and lines.
In `@src/services/whisperService.ts`:
- Around line 367-391: Consolidate CoreML encoder path derivation by reusing the
existing coreMLPathFor helper or a shared path-only helper in the model-loading
block around useCoreML, instead of duplicating the inline “.bin” to
“-encoder.mlmodelc” regex. Preserve the current iOS-only asset-presence gating
and backfill behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 092b8228-c4c2-4d23-a464-32b49093f7e0
⛔ Files ignored due to path filters (6)
android/app/src/main/assets/ggml-hexagon/libggml-htp-v69.sois excluded by!**/*.soandroid/app/src/main/assets/ggml-hexagon/libggml-htp-v73.sois excluded by!**/*.soandroid/app/src/main/assets/ggml-hexagon/libggml-htp-v75.sois excluded by!**/*.soandroid/app/src/main/assets/ggml-hexagon/libggml-htp-v79.sois excluded by!**/*.soandroid/app/src/main/assets/ggml-hexagon/libggml-htp-v81.sois excluded by!**/*.soandroid/app/src/main/assets/whisper-vad/ggml-silero-v5.1.2.binis excluded by!**/*.bin
📒 Files selected for processing (39)
.gitignore__tests__/unit/locket/recordingImportance.test.ts__tests__/unit/locket/recordingInsights.test.ts__tests__/unit/locket/recordingInsightsExtractive.test.ts__tests__/unit/services/llmHelpers.test.ts__tests__/unit/services/whisperService.test.ts__tests__/unit/stores/whisperStore.test.tsandroid/app/src/main/java/ai/offgridmobile/litert/LiteRTModule.ktdocs/plans/audio-recorder-notes.mddocs/plans/auto-detect-recording-plan.mddocs/plans/chat-grammar-test-harness-plan.mddocs/plans/combined-intelligence-layer-plan.mddocs/plans/continuous-smart-recorder-plan.mddocs/plans/insights-page-design-brief.mddocs/plans/locket-commit-plan.mddocs/plans/locket-continuous-smart-recorder-handoff.mddocs/plans/locket-insights-handoff-v2.mddocs/plans/locket-insights-handoff.mddocs/plans/locket-insights-product-design-handoff.mddocs/plans/locket-insights-screen-plan.mddocs/plans/locket-insights-transcript-quality-handoff.mddocs/plans/locket-llm-insights-plan.mddocs/plans/locket-recorder-handoff.mddocs/plans/session-handoff-transcription-intelligence.mddocs/plans/transcript-quality-experiment.mddocs/plans/transcript-quality-findings.mdprosrc/components/DevGrammarModal.tsxsrc/screens/ChatScreen/ChatScreenComponents.tsxsrc/services/devInference.tssrc/services/litert.tssrc/services/llm.tssrc/services/llmHelpers.tssrc/services/llmToolGeneration.tssrc/services/transcriptSummarizer.tssrc/services/whisperModels.tssrc/services/whisperService.tssrc/stores/devInferenceStore.tssrc/stores/whisperStore.ts
| on each PCM buffer: | ||
| isVoice = amplitude(buffer) > threshold # phase 1: RMS/avg-abs | ||
| if state == IDLE: | ||
| preroll.push(buffer) | ||
| if isVoice: | ||
| openNewFile() | ||
| flush preroll -> file # <-- onset preserved | ||
| write(buffer); lastVoiceMs = now | ||
| state = CAPTURING | ||
| else: # CAPTURING | ||
| write(buffer) | ||
| if isVoice: lastVoiceMs = now | ||
| else if now - lastVoiceMs > silenceGapMs: # <-- hangover | ||
| closeCurrentFile(announce=true) # finalize; triggers | ||
| state = IDLE # auto-prune + calendar | ||
| if maxFileMs > 0 and durationSoFar > maxFileMs: | ||
| roll file (close + open), keep CAPTURING | ||
| ``` |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Make the state transitions mutually exclusive.
preroll.push(buffer) followed by flush preroll already writes the current buffer, so write(buffer) duplicates it. Also, after closing on silence, the max-duration branch can immediately reopen a file because it is still evaluated in the same iteration. Flush only prior preroll data and guard the roll branch with state == CAPTURING.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/plans/auto-detect-recording-plan.md` around lines 76 - 93, Update the
recording state flow so the IDLE-to-CAPTURING path flushes only preroll data
before writing the current buffer once, removing the duplicate write caused by
flushing the current buffer. In the max-duration branch, require state ==
CAPTURING before rolling so a silence-triggered close cannot reopen a file in
the same iteration.
| ### 2b. Periodic Silero checkpoint (every ~60-120s) | ||
| On a timer, run Silero (GGML v5, via `vadDetect`) over the **recent unflushed slice** of the open file. Purpose: prune **confidently-non-speech, long** windows in place so a multi-hour file doesn't grow unbounded. | ||
|
|
||
| **Safety rule (critical):** the checkpoint prune is **conservative and reversible**: | ||
| - Only prune a window if it is BOTH amplitude-silent AND Silero-confident-non-speech, AND the run is longer than the prune gap (§4). Anything shorter or borderline stays. | ||
| - Keep the compressed **AAC backup** (`ensureCompressedBackup`) before any in-place removal, so a Silero false-negative is recoverable. Never destructively cut with no backup. | ||
|
|
||
| If the checkpoint prune adds risk without a clear disk win on target devices, it is acceptable to **skip 2b entirely** and rely only on the roll-time post-pass (§3 + §5). Capture-continuous + trim-at-roll already gives the full guarantee; the checkpoint is a disk optimization, not a correctness requirement. Build §3/§5 first, add 2b only if long files prove a disk problem. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Map the relevant plan file and search for related terms across docs.
git ls-files 'docs/plans/continuous-smart-recorder-plan.md' 'docs/**/*.md' | sed -n '1,120p'
echo
echo '--- outline ---'
ast-grep outline docs/plans/continuous-smart-recorder-plan.md --view expanded || true
echo
echo '--- related search ---'
rg -n --hidden -S 'ensureCompressedBackup|checkpoint prune|roll-time post-pass|unflushed slice|finalized|backup|AAC backup|prune gap|continuous-smart-recorder' docsRepository: off-grid-ai/off-grid-ai-mobile
Length of output: 10493
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the target section with line numbers.
sed -n '35,70p' docs/plans/continuous-smart-recorder-plan.md | cat -n
echo
echo '--- nearby references ---'
rg -n --hidden -S '2b\.|3 \+ 5|ensureCompressedBackup|open file|unflushed slice|prune|backup|append' docs/plans/continuous-smart-recorder-plan.mdRepository: off-grid-ai/off-grid-ai-mobile
Length of output: 7132
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find implementation or references to ensureCompressedBackup and pruning logic.
rg -n --hidden -S 'function ensureCompressedBackup|ensureCompressedBackup\(|vadDetect|prune gap|checkpoint prune|roll-time post-pass|capture-continuous|trim-at-roll|AAC backup|compressed backup' .Repository: off-grid-ai/off-grid-ai-mobile
Length of output: 6503
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Read the plan section and look for any language that says the backup is refreshed
# or tied to finalization/rollover.
sed -n '42,58p' docs/plans/continuous-smart-recorder-plan.md | cat -n
echo
rg -n --hidden -S 'refresh.*backup|backup.*refresh|restore snapshot|finalized|finalization|rollover|roll-time|append.*backup|ensureCompressedBackup' docs/plans docsRepository: off-grid-ai/off-grid-ai-mobile
Length of output: 5535
Refresh the restore snapshot before pruning the open file. A backup taken before 2b will not include audio recorded afterward, so restoring it can drop later clips. Rebuild the AAC backup after each append/prune, or keep 2b disabled until the file is finalized.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/plans/continuous-smart-recorder-plan.md` around lines 46 - 53, Update
the periodic Silero checkpoint flow in section 2b so the AAC restore snapshot
from ensureCompressedBackup reflects the complete current open file immediately
before any prune. Refresh the backup after each append or prune, or keep 2b
disabled until finalization; do not allow restoration to omit audio recorded
after an earlier backup.
| - Track "ms since last speech frame" (Silero from the checkpoint pass, or a lightweight running check). | ||
| - When trailing no-speech exceeds **ROLL_SILENCE_MS (~45-60s)** after there was speech in the file, **roll**. The next speech becomes a new clip. | ||
| - Also keep a **hard cap** roll (existing `maxFileMs`) so a never-ending conversation still rolls periodically. | ||
| - If a file has **no speech at all** by the time it would roll, discard it (don't finalize an all-noise clip) — subject to the 0:00 guard already in place. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Do not silently discard all-noise recordings.
Discarding a finalized file conflicts with the non-destructive rule in Line 167 and the project rule to never auto-delete user audio. Preserve it as an unprocessed recording, or retain a recoverable original before hiding it from the main feed.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/plans/continuous-smart-recorder-plan.md` at line 64, Update the
continuous smart recorder plan’s roll behavior to never silently discard
finalized all-noise recordings. Replace the discard requirement with preserving
the recording as an unprocessed item, or retaining a recoverable original before
excluding it from the main feed, while keeping the existing 0:00 guard.
| ## 6. The "nothing lost" guarantee (design contract) | ||
|
|
||
| - **Capture:** continuous, so no onset is ever clipped — better than the gate. | ||
| - **Trim:** non-destructive. The trimmed clip is what the user *sees*; the full audio is retained in the compressed AAC backup and restorable via `restoreOriginal`. A Silero miss is "the clip cut a bit early," not "audio gone." Honors "never auto-delete user data." | ||
| - **The only path to real loss** is destructive in-place pruning with no backup + a 15s+ continuous VAD false-negative. We close it by (a) keeping the backup and (b) the 15s duration gate. So: no permanent loss. | ||
| - **Rolling is not loss:** a pause longer than ROLL_SILENCE_MS splits a conversation into two clips, it does not drop audio. | ||
| - **Crash safety:** continuous writing means a crash loses only the last few unflushed seconds. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Resolve the “nothing lost” guarantee contradiction.
Line 110 explicitly allows crash loss of unflushed audio, so this section cannot promise that nothing is lost. Either implement durable crash-safe flushing before acknowledging capture, or rename the guarantee to state the bounded-loss behavior precisely.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/plans/continuous-smart-recorder-plan.md` around lines 104 - 110, Resolve
the contradiction in section “The ‘nothing lost’ guarantee” by renaming the
guarantee and revising its contract to acknowledge that crashes may lose only
the last few unflushed seconds. Keep the existing backup, duration gate,
rolling, and crash-safety details, but do not claim that no audio can ever be
lost unless durable flushing before capture acknowledgment is implemented.
| export function noteDevGrammarError(params: Record<string, any>, error: unknown): void { | ||
| const msg = error instanceof Error ? error.message : String(error); | ||
| useDevInferenceStore.getState().setLastError(msg); | ||
| delete params.grammar; | ||
| logger.warn(`[DevGrammar] completion failed, retrying ungrammared: ${msg}`); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Restore tool metadata on the ungrammared fallback.
applyDevGrammarOverrides removes tools and tool_choice, but noteDevGrammarError only deletes grammar. Therefore, generateWithToolsImpl retries without tool definitions, silently converting a tool request into plain-text generation. Preserve the original tool fields and restore them before retrying.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/devInference.ts` around lines 84 - 88, Update
noteDevGrammarError to restore the original tools and tool_choice metadata
removed by applyDevGrammarOverrides before generateWithToolsImpl retries.
Preserve the existing grammar cleanup, error recording, and warning behavior,
ensuring the fallback retains the tool request fields.
| let completionResult: any; | ||
| try { | ||
| completionResult = await safeCompletion(deps.context, () => deps.context.completion(completionParams as any, onCompletionData), 'generateWithTools'); | ||
| } catch (e) { | ||
| // A bad dev grammar must never brick chat: record it and retry ungrammared. | ||
| if (!devGrammarApplied) throw e; | ||
| noteDevGrammarError(completionParams, e); | ||
| completionResult = await safeCompletion(deps.context, () => deps.context.completion(completionParams as any, onCompletionData), 'generateWithTools-fallback'); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
set -euo pipefail
git ls-files src/services/llmToolGeneration.ts
wc -l src/services/llmToolGeneration.ts
sed -n '1,260p' src/services/llmToolGeneration.ts
printf '\n--- SEARCH ---\n'
rg -n "fullResponse|collectedToolCalls|tokenCount|firstReceived|toolCallFilter|safeCompletion|generateWithTools" src/services/llmToolGeneration.tsRepository: off-grid-ai/off-grid-ai-mobile
Length of output: 10589
🏁 Script executed:
set -euo pipefail
sed -n '1,260p' src/services/llmHelpers.ts
printf '\n--- DEV INFERENCE ---\n'
sed -n '1,260p' src/services/devInference.ts
printf '\n--- SEARCH PARTIAL/RETRY ---\n'
rg -n "partial|retry|safeCompletion|onCompletionData|tool_calls|firstReceived|tokenCount|fullResponse|toolCallFilter" src/servicesRepository: off-grid-ai/off-grid-ai-mobile
Length of output: 35941
🏁 Script executed:
set -euo pipefail
sed -n '1,260p' src/services/llmSafetyChecks.ts
printf '\n--- LLM RETRY SECTION ---\n'
sed -n '300,350p' src/services/llm.tsRepository: off-grid-ai/off-grid-ai-mobile
Length of output: 9109
🌐 Web query:
llama.rn completion callback throws after streaming partial tokens grammar error
💡 Result:
In recent versions of llama.cpp, errors in completion callbacks during streaming—or specifically, exceptions thrown during the final parse—are frequently linked to regressions in the chat parser's PEG implementation [1][2][3]. The issue typically manifests in the following way: 1. During streaming, the server parses output with the is_partial flag set to true, which allows the parser to fall back to partial AST extraction when it encounters tokens that do not fully match the grammar [2][3]. 2. When the generation finishes (or reaches max_tokens), the server performs a final parse on the accumulated text with is_partial set to false [1][2]. 3. In recent builds, this final parse can fail if the parser encounters unexpected input, causing the server to throw a std::runtime_error ("Failed to parse input at pos...") instead of utilizing the partial fallback used during streaming [1][2][3]. This results in an HTTP 500 error, discarding the valid content already sent to the client [1][2]. Technical context and mitigation: - Regression Source: This behavior has been identified in issues following major refactors to the chat parser and PEG implementation (e.g., commit 566059a26) [1][2]. - Error Handling: Earlier issues regarding grammar error reporting in streaming mode have been partially addressed by wrapping sampling functions (like common_sampler_accept) in try/catch blocks [4]. However, the specific "final parse" regression has been the subject of active bug reports and pull requests aimed at ensuring the parser behaves consistently (i.e., gracefully) in non-partial mode [2][3][5]. - Troubleshooting: If you are encountering this, ensure you are using the latest version of llama.cpp, as maintainers have been actively merging patches to improve parser leniency and error handling for partial outputs [3][5]. For development purposes, some users have implemented local patches to catch these specific exceptions and treat them as valid completions rather than server errors [2]. If the crash is due to specific model tokens (e.g., "Unexpected empty grammar stack"), it may also indicate a vocabulary-grammar mismatch specific to certain model variants [6].
Citations:
- 1: Misc. bug: Autoparser throws on final parse after streaming succeeds ggml-org/llama.cpp#20814
- 2: chat: new parser should not crash inference ggml-org/llama.cpp#20708
- 3: Fix chat parser regressions: inference crashes/frozen; output backtracked ggml-org/llama.cpp#20660
- 4: fix: GLM 4.5 streaming tool-call parsing + grammar error handling ggml-org/llama.cpp#19612
- 5: common : gracefully handle incomplete output ggml-org/llama.cpp#20191
- 6: Eval bug: Grammar sampler crash: "Unexpected empty grammar stack" on Gemma 4 <unused> tokens ggml-org/llama.cpp#23677
Reset stream state before retrying. The fallback path reuses fullResponse, collectedToolCalls, tokenCount, firstReceived, and toolCallFilter; if the first attempt emits any tokens or tool calls before failing, the second pass can append stale data and duplicate output. Recreate those buffers for the retry, or only retry before any callback fires.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/llmToolGeneration.ts` around lines 154 - 161, Reset the
streaming state before the fallback retry in the catch block of the
generateWithTools flow: recreate fullResponse, collectedToolCalls, tokenCount,
firstReceived, and toolCallFilter so the second safeCompletion call starts clean
and cannot duplicate partial output. Preserve the existing retry behavior and
dev-grammar error recording.
| } catch (e) { | ||
| // A bad dev grammar must never brick chat: record it and retry ungrammared. | ||
| if (!devGrammarApplied) throw e; | ||
| noteDevGrammarError(completionParams, e); | ||
| completionResult = await safeCompletion(deps.context, () => deps.context.completion(completionParams as any, onCompletionData), 'generateWithTools-fallback'); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Do not retry every completion failure as a grammar failure.
This branch retries after any rejected completion whenever a grammar was applied. That includes cancellation, model unloads, OOM/native failures, and other availability errors, so an abort can immediately start generation again. Restrict fallback to a confirmed grammar-rejection case and rethrow cancellation and infrastructure failures.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/llmToolGeneration.ts` around lines 157 - 161, Update the catch
branch in the completion flow around safeCompletion so fallback is attempted
only when the error is confirmed to be a dev-grammar rejection. Preserve
noteDevGrammarError and the ungrammared retry for that case, but immediately
rethrow cancellation, model availability, OOM/native, and other non-grammar
failures; do not use devGrammarApplied alone as the retry condition.
| // Path where whisper.cpp looks for a model's CoreML encoder: it derives it | ||
| // from the ggml filename, `.bin` -> `-encoder.mlmodelc`. Keep in lockstep with | ||
| // the load-time check below. | ||
| private coreMLPathFor(modelId: string): string { | ||
| return this.getModelPath(modelId).replace(/\.bin$/i, '-encoder.mlmodelc'); | ||
| } | ||
|
|
||
| /** True when this model's CoreML encoder is present on disk (iOS only). */ | ||
| async hasCoreMLEncoder(modelId: string): Promise<boolean> { | ||
| if (Platform.OS !== 'ios') return false; | ||
| return RNFS.exists(this.coreMLPathFor(modelId)); | ||
| } | ||
|
|
||
| /** | ||
| * iOS only: download + unzip a model's CoreML encoder next to its .bin so | ||
| * whisper.cpp can run the encoder on the Apple Neural Engine (~2-3x faster | ||
| * encode, frees the CPU). Non-fatal - on any failure the model still works on | ||
| * CPU. No-op on Android, when the model has no published encoder, or when it's | ||
| * already present. | ||
| */ | ||
| async ensureCoreMLEncoder(modelId: string, onProgress?: (p: number) => void): Promise<boolean> { | ||
| if (Platform.OS !== 'ios') return false; | ||
| const model = WHISPER_MODELS.find(m => m.id === modelId); | ||
| if (!model?.coreMLUrl) return false; | ||
| const targetDir = this.coreMLPathFor(modelId); // ggml-<id>-encoder.mlmodelc | ||
| if (await RNFS.exists(targetDir)) return true; | ||
| await this.ensureModelsDirExists(); | ||
| const zipPath = `${this.getModelsDir()}/ggml-${modelId}-encoder.mlmodelc.zip`; | ||
| await RNFS.unlink(zipPath).catch(() => {}); // clear any partial from a prior run | ||
| try { | ||
| logger.log(`[Whisper][CoreML] START download ${modelId} from ${model.coreMLUrl}`); | ||
| const t0 = Date.now(); | ||
| let lastPct = -1; | ||
| const { promise } = RNFS.downloadFile({ | ||
| fromUrl: model.coreMLUrl, | ||
| toFile: zipPath, | ||
| progressInterval: 500, | ||
| progress: (r) => { | ||
| if (r.contentLength <= 0) return; | ||
| const frac = r.bytesWritten / r.contentLength; | ||
| onProgress?.(frac); | ||
| const pct = Math.floor(frac * 10) * 10; // log each 10% | ||
| if (pct !== lastPct) { | ||
| lastPct = pct; | ||
| logger.log(`[Whisper][CoreML] ${modelId} ${pct}% (${(r.bytesWritten / 1e6).toFixed(0)}/${(r.contentLength / 1e6).toFixed(0)} MB)`); | ||
| } | ||
| }, | ||
| }); | ||
| const res = await promise; | ||
| if (res.statusCode && res.statusCode >= 400) throw new Error(`HTTP ${res.statusCode}`); | ||
| const zipMB = (Number((await RNFS.stat(zipPath)).size) / 1e6).toFixed(0); | ||
| logger.log(`[Whisper][CoreML] downloaded ${modelId} (${zipMB} MB) in ${((Date.now() - t0) / 1000).toFixed(1)}s — unzipping`); | ||
| await unzip(zipPath, this.getModelsDir()); | ||
| await RNFS.unlink(zipPath).catch(() => {}); | ||
| // The zip's top-level dir is named after the SOURCE encoder in the URL. For | ||
| // most models that already equals targetDir; when a model reuses another's | ||
| // encoder (tdrz -> small.en) the names differ, so rename it into place. | ||
| const extractedName = model.coreMLUrl.split('/').pop()!.replace(/\.zip$/i, ''); | ||
| const extractedDir = `${this.getModelsDir()}/${extractedName}`; | ||
| if (extractedDir !== targetDir && (await RNFS.exists(extractedDir))) { | ||
| await RNFS.unlink(targetDir).catch(() => {}); // clear any stale target | ||
| await RNFS.moveFile(extractedDir, targetDir); | ||
| logger.log(`[Whisper][CoreML] reused ${extractedName} → ${targetDir.split('/').pop()}`); | ||
| } | ||
| const ok = await RNFS.exists(targetDir); | ||
| logger.log(`[Whisper][CoreML] ${ok ? `READY for ${modelId} — next load will use the Neural Engine` : `FAILED for ${modelId}: no encoder dir after unzip`}`); | ||
| return ok; | ||
| } catch (e) { | ||
| logger.warn(`[Whisper][CoreML] fetch FAILED for ${modelId} (staying CPU-only): ${String(e)}`); | ||
| await RNFS.unlink(zipPath).catch(() => {}); | ||
| return false; | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== file list ==\n'
git ls-files 'src/services/whisperService.ts' 'src/services/whisperModels.ts' | cat
printf '\n== outlines ==\n'
ast-grep outline src/services/whisperService.ts --view expanded || true
ast-grep outline src/services/whisperModels.ts --view expanded || true
printf '\n== relevant line ranges ==\n'
nl -ba src/services/whisperService.ts | sed -n '80,190p'
printf '\n--- whisperModels ---\n'
nl -ba src/services/whisperModels.ts | sed -n '1,220p'
printf '\n== url reuse checks ==\n'
python3 - <<'PY'
from pathlib import Path
import re, json
text = Path('src/services/whisperModels.ts').read_text()
# naive parse of model entries to inspect ids and coreMLUrl fields
for m in re.finditer(r"id:\s*'([^']+)'.{0,300}?coreMLUrl:\s*'([^']+)'", text, re.S):
print(m.group(1), '=>', m.group(2))
PYRepository: off-grid-ai/off-grid-ai-mobile
Length of output: 3870
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== whisperModels relevant lines ==\n'
sed -n '1,120p' src/services/whisperModels.ts | cat -n
printf '\n== coreMLUrl occurrences ==\n'
python3 - <<'PY'
from pathlib import Path
import re
text = Path('src/services/whisperModels.ts').read_text()
for line in text.splitlines():
if 'coreMLUrl' in line or "id: 'small.en" in line or "id: 'small.en-tdrz'" in line:
print(line)
PY
printf '\n== path derivation check ==\n'
python3 - <<'PY'
def core_ml_path_for(model_id):
return f"/models/ggml-{model_id}.bin".replace(".bin", "-encoder.mlmodelc")
for mid in ["small.en", "small.en-tdrz"]:
print(mid, "->", core_ml_path_for(mid))
core_url = "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.en-encoder.mlmodelc.zip"
extracted = core_url.split('/')[-1].removesuffix('.zip')
print("extracted dir from shared URL:", extracted)
PYRepository: off-grid-ai/off-grid-ai-mobile
Length of output: 7488
Avoid moving the shared CoreML encoder out from under small.en src/services/whisperService.ts:117-160
small.en-tdrz reuses small.en’s CoreML zip, but ensureCoreMLEncoder() unzips it under the shared top-level dir and then renames that dir to the tdrz path. If small.en is already installed, this removes its canonical ggml-small.en-encoder.mlmodelc directory, and the next small.en load falls back to CPU. Extract into a per-model scratch dir, or skip the move when the source encoder is shared.
🧰 Tools
🪛 GitHub Check: SonarCloud Code Analysis
[warning] 162-162: Refactor this code to not use nested template literals.
[warning] 162-162: Refactor this code to not use nested template literals.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/whisperService.ts` around lines 97 - 170, Prevent
ensureCoreMLEncoder from moving or removing the shared small.en encoder when
processing small.en-tdrz. Extract reused encoders into a per-model scratch
location, or otherwise copy/retain the canonical source directory while placing
the target at the requested model path; preserve ggml-small.en-encoder.mlmodelc
for subsequent small.en loads.
| // iOS: also fetch the CoreML encoder so the ANE can run the encoder. Non-fatal | ||
| // and no-op off-iOS / when unpublished - the model is already usable on CPU. | ||
| if (Platform.OS === 'ios') { | ||
| await this.ensureCoreMLEncoder(modelId).catch(() => {}); | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Map the service and locate the relevant symbols
ast-grep outline src/services/whisperService.ts --view expanded || true
printf '\n--- relevant matches ---\n'
rg -n "ensureCoreMLEncoder|downloadModel|downloadFile|Platform.OS === 'ios'|fire-and-forget|backfill" src/services/whisperService.ts
printf '\n--- surrounding lines around the awaited call ---\n'
sed -n '230,300p' src/services/whisperService.ts
printf '\n--- surrounding lines around the fire-and-forget backfill mention ---\n'
sed -n '360,420p' src/services/whisperService.tsRepository: off-grid-ai/off-grid-ai-mobile
Length of output: 10727
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the CoreML helper and its timeout/error handling
rg -n "function ensureCoreMLEncoder|ensureCoreMLEncoder\\(|downloadFile\\(|setTimeout|timeout|RNFS\\.downloadFile" src/services/whisperService.ts src -g '!**/node_modules/**'
printf '\n--- CoreML helper region ---\n'
sed -n '1,220p' src/services/whisperService.ts
printf '\n--- remainder if helper is lower in file ---\n'
sed -n '220,520p' src/services/whisperService.tsRepository: off-grid-ai/off-grid-ai-mobile
Length of output: 41463
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Read only the relevant implementation to see whether the CoreML fetch has its own timeout / early return.
rg -n "ensureCoreMLEncoder|downloadFile|timeout|connectionTimeout|readTimeout|catch\\(\\) \\{\\}" src/services/whisperService.ts
printf '\n--- core implementation slice ---\n'
sed -n '300,460p' src/services/whisperService.tsRepository: off-grid-ai/off-grid-ai-mobile
Length of output: 8035
Don't block downloadModel() on the CoreML backfill.
ensureCoreMLEncoder() calls RNFS.downloadFile() with no timeout, so a stalled iOS encoder fetch can keep the model-download promise open even after the .bin is validated. Make this fire-and-forget like the load-time backfill.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/whisperService.ts` around lines 260 - 264, Update the iOS CoreML
backfill in downloadModel so ensureCoreMLEncoder(modelId) is launched
fire-and-forget rather than awaited, while retaining its non-fatal error
handling and allowing downloadModel to resolve immediately after the main model
is validated.
When a dev-grammar completion fails and retries without the grammar, the streaming state carried over from the failed attempt, duplicating/garbling the retry's output (and re-emitting via onStream). Reset fullContent/fullReasoningContent/streamed* in generateResponse, and fullResponse/collectedToolCalls + the ToolCallTokenFilter buffer in generateWithToolsImpl. Matches the existing reset in generateWithMaxTokens. (Gitar code-review finding.) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…-transcribe off + recorder comment) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
… delete deleteModel updated downloadedModelId (fallback) but left the deleted model in presentModelIds, so the models list kept showing a model whose file is gone. Recompute presentModelIds from disk, matching deleteModelById. (Gitar re-review finding.) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
|
Re: deleteModel leaves deleted model in presentModelIds — fixed in fd8df7e. deleteModel now recomputes presentModelIds from disk after deleting the active model (dropping the just-deleted one), matching deleteModelById. |
…ropout) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…-gated-recorder # Conflicts: # .gitignore # android/app/src/main/assets/ggml-hexagon/libggml-htp-v69.so # android/app/src/main/assets/ggml-hexagon/libggml-htp-v73.so # android/app/src/main/assets/ggml-hexagon/libggml-htp-v75.so # android/app/src/main/assets/ggml-hexagon/libggml-htp-v79.so # android/app/src/main/assets/ggml-hexagon/libggml-htp-v81.so # pro # src/services/llmHelpers.ts # src/stores/whisperStore.ts
Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Summaries prefer a remote provider whenever active (offloads generation off-device to save battery/RAM); does not gate on a local model being loaded. Chat generation keeps its own local-first policy. Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Forwards a short proper-noun/jargon list to whisper.cpp as its initial prompt so names are spelled the user's way. Pairs with the pro-side vocabulary setting. Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…GC-time UAF Insight generation churns conversations (reset->close->recreate). Each finished generation leaves fbjni HybridData peers to be reclaimed on the GC 'HybridData Dest' thread; under churn that reclaim can fire mid-decode of the next conversation and dereference freed memory (observed SIGSEGV, poison 0x01). Drain the reference queue at the quiescent teardown point (gc + runFinalization + brief yield) so reclaims do not overlap live native work. Includes temporary DIAG-UAF lifecycle logging to confirm the fix on-device; strip before merge. Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…, permissions, iOS recorder, VAD teardown fix) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
| import React, { useEffect, useRef } from 'react'; | ||
| import { Animated, Text, TouchableOpacity, StyleSheet } from 'react-native'; | ||
| import { useSafeAreaInsets } from 'react-native-safe-area-context'; | ||
| import Icon from 'react-native-vector-icons/Feather'; | ||
| import { create } from 'zustand'; | ||
| import { useTheme, useThemedStyles } from '../theme'; | ||
| import type { ThemeColors, ThemeShadows } from '../theme'; | ||
| import { SPACING, TYPOGRAPHY } from '../constants'; | ||
|
|
||
| /** | ||
| * One cross-platform toast (not ToastAndroid, which is Android-only): a brief, | ||
| * non-blocking message that slides up from the bottom and auto-dismisses. A | ||
| * single host is mounted once at the app root; anywhere in the app (screens or | ||
| * services) calls `showToast(message)` imperatively - no per-screen wiring. | ||
| */ |
There was a problem hiding this comment.
⚠️ Quality: New Toast component and summarizer fallback logic ship without tests
Per the project's testing standard (100% coverage on new code, both unit and integration, plus a fails-before/passes-after regression test), the following new/changed logic has no dedicated tests:
src/components/Toast.tsx(entirely new: show/hide store, nonce-based timer restart, fade-in/out lifecycle) has no test file.- The new remote-preferred selection (
isRemoteActive,resolveContextTokens) and the remote-before-streaming fallback ingenerateSummaryText(emittedAnygate, fall-through to LiteRT/local) are only exercised indirectly via a mocked summarizer in__tests__/unit/locket/recordingInsights.test.ts, not unit-tested directly.
Add unit tests covering: (1) remote preferred even when a local model is loaded; (2) remote failure BEFORE any token → falls through to on-device; (3) remote failure AFTER a token → error surfaced (no double-write); (4) Toast timer reset on repeated showToast with identical text (nonce). Include a regression test for the abort/remote gap above.
Was this helpful? React with 👍 / 👎
There was a problem hiding this comment.
Acknowledged — deferring to a follow-up. Toast and the summarizer fallback are earlier-branch code; adding their tests is out of scope for this review-fix pass, tracked separately.
There was a problem hiding this comment.
Understood — tracking Toast and summarizer fallback test coverage as a follow-up rather than blocking this PR.
Pure selectTextModelToLoad(models, budgetMB, { activeId, footprintMB }): active-if-fits -> largest-that-fits -> smallest. Footprint is injected (caller passes hardwareService.estimateModelRam) so weights + mmproj + overhead count the one canonical way. Unit-tested for every branch.
Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
loadTextModel({ textOnly: true }) skips loading the mmproj (saving its RAM) and sizes the residency budget on the gguf weights alone. Gated - chat still loads vision by default. Guards the post-load mmproj-clear so a text-only load never erases a model's persisted vision link.
Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…n prompt) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…RecordingVisible) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…e->retry / hard-error->no-evict) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…force (skipped pending dynamic-import env) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…s, auto-prune) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
… redesing.md) android/temp.txt was a 1532-line AI planning transcript; redesing.md a stray design note with broken image refs. Neither belongs in the repo (flagged by review). Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Summaries prefer a remote provider, so a local-only stopGeneration left a remote completion running after Stop. abort() now also calls the active provider's stopGeneration. Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…ot path Removes the DIAG-UAF create/send/close Log.i lines and the convSeq/currentConvSeq bookkeeping added to diagnose the GC-time HybridData UAF. Keeps the mitigation itself (gc + runFinalization drain at close). Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…prompt) Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…nscription
cleanTranscription only kept text containing [a-z0-9], so a Hindi/Arabic/CJK
transcript (no ASCII letters) was wiped to '' - silent data loss for non-English
users. Match any-script letters/digits (\p{L}\p{N}) instead. Also: the single
shared Whisper context can't run two file transcriptions at once, so a second
overlapping request now throws WhisperBusyError instead of orphaning the first
job's cancel handle, and unloadModel cancels an in-flight file transcribe before
releasing the context (prevents a use-after-free SIGSEGV).
Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
…budget Expose the residency manager (the one owner of the RAM cap + load gate) from the services barrel so callers that auto-pick a model to load can budget against the SAME number the loader enforces (modelResidencyManager.getBudgetMB), instead of a divergent hardcoded fraction. Used by memory-aware auto-load callers. Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Morning now covers 0-12 (early-hours clips group first instead of showing as Night at the top of the day), so hours 0-4 map to Morning, not Night. Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
Code Review
|
| Auto-apply | Compact |
|
|
Was this helpful? React with 👍 / 👎 | Gitar
The pro recorder/insights feature lives in the private pro/ submodule; its architecture, product-design handoffs, and transcript-quality R&D notes must not ship in the public core repo. Remove the 22 locket/recorder/insights/transcript planning + handoff docs from docs/plans/. Keeps only genuinely-core planning (backend-per-device, whisper-download-sync). Tree-only removal; a full history scrub, if needed, is a separate step at the public cutover. Co-Authored-By: Dishit Karia <hanmadishit74@gmail.com>
|



Stacked onto
feat/locket-pro(PR #433). This session's core-side work:transcriptSummarizer: dynamic context sizing (scales to the active backend), token streaming (onToken across local/LiteRT/remote), abort()litert: expose getContextTokens for backend-aware chunk sizingwhisperStore: fall back to another on-disk model when the active one is deleted (+ tests)prosubmodule to the insights-redesign / continuous-smart tipNOTE (WIP): branch cut Jul 9, behind
feat/locket-proby ~455 commits (it merged dev). GitHub's PR diff (merge-base) shows only these changes; needs rebase/cherry-pick onto the current tip before merge. Draft for review of the intended changes.Summary by Gitar
TranscriptSummarizerServiceto use dynamic, backend-aware context sizing based onINPUT_CONTEXT_FRACTION.abort()support toTranscriptSummarizerServiceto interrupt active local, LiteRT, or remote generations.grammarandrepeatPenaltyparameters togenerateSummaryTextandsummarizeOnefor improved output control.CoreMLencoder backfilling and integration to accelerate Whisper inference on the Neural Engine.whisperStoreto automatically fall back to alternative models if the active one is deleted.WhisperServiceto single-instance file transcription to prevent native context race conditions.DevGrammarModalandDevGrammarStoreto test GBNF/LLGuidance constraints behind__DEV__.Toastcomponent for imperative, non-blocking UI notifications.System.gc()and finalization drains in LiteRT to mitigate potential nativeSIGSEGVmemory crashes.This will update automatically on new commits.
Summary by CodeRabbit