Real-time, on-device speech-to-text. Speak and watch words appear as you talk — small streaming model, Chinese + English, no cloud, no API key, audio never leaves the machine.
在实时 ASR 之上,仓库里还有一个完整的 Web 应用 VoxMemo:说一段话,两个 Agent 把它整理成干净备忘。
- Agent ① · 端侧语音转写 —— sherpa-onnx 小模型,本机离线,输出原始(粗糙)文字
- Agent ② · DeepSeek-V4-Pro 润色 —— 纠错、补标点、去口语杂质与重复、按逻辑整理成通顺备忘
这演示了一个很实际的端云协同:端侧免费/离线拿到粗文字,云端强模型把它救成可用备忘。全过程(两 Agent 状态、模型「思考中」、逐字润色)在 Web 端实时可见,备忘存 SQLite 可搜索。
实测(一条真实备忘,端侧 ASR 很糙、DeepSeek 救回):
| Agent① 原始转写 | Agent② 润色后 |
|---|---|
这是第一种第二种叫呃与 ALWAYS ALWAYS什么意思啊 |
这是第一种，第二种叫 Always。Always 是什么意思？ |
cp .env.example .env # 填入 DEEPSEEK_API_KEY
bash scripts/download_model.sh # 下载端侧 ASR 模型(一次)
docker compose up # → http://localhost:8010不用 Docker:pip install -r requirements.txt && uvicorn server.app:app --port 8010(需 DEEPSEEK_API_KEY)。后端在 server/:asr.py(Agent①)、polish.py(Agent② DeepSeek 流式)、store.py(SQLite)、app.py(FastAPI + SSE)。
MVP 范围:分段式录音(说完→整理)+ 双 Agent 实时可视化 + 备忘记录/搜索。路线图:真流式(WebSocket)、归类、配置面板、端侧 VAD 降噪。
VoiceFlow Live turns your microphone into a live transcription stream. It uses a small streaming ASR model (sherpa-onnx zipformer transducer, int8) that emits partial words while you are still speaking and commits a segment on each natural pause — the behaviour a real voice-input tool needs.
Because the engine is plain ONNX, the exact same model runs on macOS, Linux, Windows, Android, iOS and embedded/edge devices — so this isn't just a laptop toy, it's a blueprint for putting voice input on the terminal itself.
Streaming a test clip through the pipeline (partials … grow in real time, ✓ = committed segment):
… 昨天 … 昨天是 … 昨天是 MONDAY ✓ 昨天是 MONDAY
… TODAY IS … THE DAY AFTER TOMORROW … 是星期三 ✓ TODAY IS THE DAY AFTER TOMORROW 是星期三
Note the seamless 中英混说 (Chinese⇄English code-switching) in a single utterance.
- Real-time streaming — not chunked 30s windows (Whisper); tokens emit as you speak.
- Small — int8 model is ~193 MB total, runs real-time on CPU (no GPU required).
- Private / offline — zero network calls; nothing is uploaded.
- Portable — ONNX runtime → the same model runs on phones, cars, and embedded boxes.
git clone https://github.com/<you>/voiceflow-live.git
cd voiceflow-live
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
bash scripts/download_model.sh # ~500 MB download, one time
# Live microphone → text (grant your terminal Microphone permission first):
python -m voiceflow.realtime_mic
python -m voiceflow.realtime_mic --paste # + copy each finalized segment to clipboard
# No mic handy? Verify the pipeline on a WAV:
python -m voiceflow.transcribe_file models/sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20/test_wavs/0.wavmic ──► 16 kHz mono frames ──► sherpa-onnx OnlineRecognizer (streaming zipformer transducer)
│ greedy decode on each 100 ms chunk → partial text (…)
└─ endpoint detection (trailing silence) → commit segment (✓)
voiceflow/recognizer.py builds the recognizer (prefers int8 model files); the mic tool and
the file tool share it, so what you verify on a WAV is exactly what runs live.
sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20
— bilingual zh/en streaming transducer. int8 ≈ 193 MB, fp32 ≈ 346 MB. Swap in a smaller
mono-lingual streaming model by editing MODEL_NAME in voiceflow/recognizer.py.
- Type-at-cursor (accessibility keystroke injection) instead of clipboard
- Custom vocabulary / hotwords (domain terms, names)
- Optional local-LLM post-polish (de-filler + punctuation) — kept off the hot path
- Native menu-bar wrapper (Swift) for a background always-on tool
Apache License 2.0.
VoiceFlow Live 是一个完全本地、实时流式的语音转文字工具:边说边出字,自然停顿即落段。 引擎是小体积的流式 ASR 模型（sherpa-onnx zipformer transducer,int8), 不联网、不需要 API key、音频不出本机。
因为引擎是纯 ONNX,同一个模型能跑在 macOS / Linux / Windows / Android / iOS / 车机 / 嵌入式 —— 所以它不只是台式机玩具,而是"把语音输入做进真实终端"的可迁移原型。
特点:实时流式(非 Whisper 那种 30s 分块) · 小模型(int8 约 193 MB,CPU 实时) · 隐私离线 · 跨端可移植 · 中英混说。
快速开始见上方 English → Quickstart。麦克风实时转写需要给终端授予"麦克风"权限(系统设置 → 隐私与安全性 → 麦克风)。
License: Apache 2.0。