Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,12 +1,30 @@
DATABASE_URL=postgresql://transcriber:transcriber@localhost:5433/transcriber
REDIS_URL=redis://localhost:6380/0
LLM_PROVIDER=ollama
# To use a local openai compatible server set the base url here, for example:
#OPENROUTER_BASE_URL=http://127.0.0.1:8000/v1/
# An API key may not be needed when using your own server
OPENROUTER_API_KEY=your_key_here
OPENROUTER_MODEL=anthropic/claude-sonnet-4

OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=qwen3:8b

# Whisper CLI path is not required if using the MLX backend.
WHISPER_CLI_PATH=/path/to/whisper-cli
WHISPER_MODEL_PATH=/path/to/kb_whisper_ggml_medium.bin
WHISPER_SMALL_MODEL_PATH=/path/to/kb_whisper_ggml_small.bin
STORAGE_PATH=./storage

# Hugging Face authentication token, required for some models and features. Get it from https://huggingface.co/settings/tokens
HF_AUTH_TOKEN=hf_your_token_here

# Supported backends: mlx, torch
#BACKEND=mlx

# When using a mixture of Docker and host services, it can be helpful to uncomment these two settings:
# LISTEN_HOST=0.0.0.0 # Allow requests from outside the docker network
# PROXY_HOST=host.docker.internal # Allow Docker containers to access services running on the host machine

# If a non-standard API port is needed, uncomment and set the value
#API_PORT=8082
46 changes: 43 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ AI-powered local meeting transcription with automatic speaker identification. Up
1. **Upload, record, or go live** through the web UI
2. **Audio extraction** - FFmpeg converts to 16kHz mono WAV
3. **Transcription** - whisper.cpp with KB-LAB Swedish models (Metal GPU accelerated)
4. **Speaker diarization** - pyannote.audio 3.1 separates speakers
4. **Speaker diarization** - pyannote.audio 3.1 on torch, or MLX sortformer on Apple Silicon
5. **Intro analysis** - LLM iteratively reads the transcript to detect introductions and count speakers
6. **Speaker identification** - Names matched to voices using LLM reasoning + SpeechBrain voice embeddings
7. **Results** - Color-coded transcript synced with audio playback, editable segments, AI-powered actions, export to 7 formats
Expand Down Expand Up @@ -50,7 +50,7 @@ The instructions below are for **macOS with Apple Silicon**. For other platforms
- **FFmpeg** (`brew install ffmpeg`)
- **whisper.cpp** compiled with Metal support
- **Ollama** with a model like `qwen3:8b` (recommended), or an OpenRouter API key
- **Hugging Face token** with access to `pyannote/speaker-diarization-3.1`
- **Hugging Face token** with access to `pyannote/speaker-diarization-3.1` when using the `torch` diarization backend

## Quick install

Expand Down Expand Up @@ -109,6 +109,32 @@ This starts:
- PostgreSQL on port **5433**
- Redis on port **6380**

### Docker networking gotchas (important)

When mixing host-native services with Docker containers, two rules prevent most connection issues:

1. A service that should be reachable from outside its own network namespace must listen on `0.0.0.0`.
2. A container cannot use `localhost` to reach services running on your host machine.

Examples:

- If you run a server inside Docker and want to access it from your Mac browser, ensure the process binds to `0.0.0.0` inside the container (not `127.0.0.1`).
- If a container needs to call a host service (for example Ollama running on macOS), use `host.docker.internal` from inside the container.

Use this in containerized configs:

```env
OLLAMA_BASE_URL=http://host.docker.internal:11434
```

And ensure the host service is actually listening on an external interface (typically `0.0.0.0`) rather than only `127.0.0.1`.

Quick reference:

- Host -> Container (published port): `http://localhost:<published-port>`
- Container -> Host service: `http://host.docker.internal:<port>`
- Inside one container, `localhost` means that same container only

### 5. Create the .env file

```bash
Expand All @@ -131,15 +157,23 @@ WHISPER_CLI_PATH=../whisper.cpp/build/bin/whisper-cli
WHISPER_MODEL_PATH=./models/kb_whisper_ggml_medium.bin
WHISPER_SMALL_MODEL_PATH=./models/kb_whisper_ggml_small.bin

# Shared ML backend switch (used by diarization and transcription): "torch" or "mlx"
BACKEND=torch
TORCH_DIARIZATION_MODEL=pyannote/speaker-diarization-3.1
MLX_DIARIZATION_MODEL=mlx-community/diar_sortformer_4spk-v1-fp16

STORAGE_PATH=./storage

# Hugging Face token (needed for pyannote.audio speaker diarization)
# Hugging Face token (needed only for the torch/pyannote diarization backend)
# Get yours at https://huggingface.co/settings/tokens
# You must accept the model terms at https://huggingface.co/pyannote/speaker-diarization-3.1
HF_AUTH_TOKEN=hf_your_token_here
EOF
```

Set `BACKEND=mlx` to use MLX for diarization and transcription.
Quick note: when using MLX, see the specific steps below for installing the python dependencies.

Edit the file and fill in your actual paths and tokens.

### 6. Set up the Python backend
Expand All @@ -150,6 +184,12 @@ source venv/bin/activate
pip install -r requirements.txt
```

If you plan to run with `BACKEND=mlx`, install MLX-specific dependencies instead:

```bash
pip install -r requirements_mlx.txt
```

### 7. Set up the frontend

```bash
Expand Down
18 changes: 10 additions & 8 deletions api/actions.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
from database import get_db
from models import Action, ActionResult, ActionResultStatus, Meeting
from tasks.action_task import run_action_task
from translations import get_translation

router = APIRouter(prefix="/api/actions", tags=["actions"])

Expand Down Expand Up @@ -40,7 +41,7 @@ def create_action(req: CreateActionRequest, db: Session = Depends(get_db)):
def update_action(action_id: str, req: UpdateActionRequest, db: Session = Depends(get_db)):
action = db.query(Action).filter(Action.id == action_id).first()
if not action:
raise HTTPException(404, "Action not found")
raise HTTPException(404, get_translation('errors.action_not_found'))
if req.name is not None:
action.name = req.name
if req.prompt is not None:
Expand All @@ -54,7 +55,7 @@ def update_action(action_id: str, req: UpdateActionRequest, db: Session = Depend
def delete_action(action_id: str, db: Session = Depends(get_db)):
action = db.query(Action).filter(Action.id == action_id).first()
if not action:
raise HTTPException(404, "Action not found")
raise HTTPException(404, get_translation('errors.action_not_found'))
db.delete(action)
db.commit()
return {"ok": True}
Expand All @@ -66,11 +67,11 @@ def delete_action(action_id: str, db: Session = Depends(get_db)):
def run_action(action_id: str, meeting_id: str, db: Session = Depends(get_db)):
action = db.query(Action).filter(Action.id == action_id).first()
if not action:
raise HTTPException(404, "Action not found")
raise HTTPException(404, get_translation('errors.action_not_found'))

meeting = db.query(Meeting).filter(Meeting.id == meeting_id).first()
if not meeting:
raise HTTPException(404, "Meeting not found")
raise HTTPException(404, get_translation('errors.meeting_not_found'))

result = ActionResult(
action_id=action_id,
Expand All @@ -87,9 +88,10 @@ def run_action(action_id: str, meeting_id: str, db: Session = Depends(get_db)):
db.commit()
except Exception as e:
result.status = ActionResultStatus.FAILED
result.error = f"Failed to queue task: {e}"
error_msg = get_translation('errors.failed_to_queue_task').format(error=e)
result.error = error_msg
db.commit()
raise HTTPException(503, f"Task queue unavailable: {e}")
raise HTTPException(503, get_translation('errors.task_queue_unavailable').format(error=e))

return result.to_dict()

Expand All @@ -112,7 +114,7 @@ def list_results(meeting_id: str, db: Session = Depends(get_db)):
for r in results:
d = r.to_dict()
a = actions.get(r.action_id)
d["action_name"] = a.name if a else "Deleted action"
d["action_name"] = a.name if a else get_translation('ui.deleted_action')
out.append(d)
return out

Expand All @@ -121,7 +123,7 @@ def list_results(meeting_id: str, db: Session = Depends(get_db)):
def delete_result(result_id: str, db: Session = Depends(get_db)):
result = db.query(ActionResult).filter(ActionResult.id == result_id).first()
if not result:
raise HTTPException(404, "Result not found")
raise HTTPException(404, get_translation('errors.result_not_found'))
db.delete(result)
db.commit()
return {"ok": True}
3 changes: 2 additions & 1 deletion api/analytics.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

from database import get_db
from models import Meeting, Speaker, Segment
from translations import get_translation

router = APIRouter(prefix="/api", tags=["analytics"])

Expand All @@ -11,7 +12,7 @@
def get_meeting_analytics(meeting_id: str, db: Session = Depends(get_db)):
meeting = db.query(Meeting).filter(Meeting.id == meeting_id).first()
if not meeting:
raise HTTPException(404, "Meeting not found")
raise HTTPException(404, get_translation('errors.meeting_not_found'))

speakers = db.query(Speaker).filter(Speaker.meeting_id == meeting_id).all()
segments = (
Expand Down
13 changes: 7 additions & 6 deletions api/encryption.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from models.segment import Segment
from models.action import ActionResult
from services.encryption_service import EncryptionService
from translations import get_translation

router = APIRouter(prefix="/api/meetings", tags=["encryption"])

Expand All @@ -26,11 +27,11 @@ class DecryptRequest(BaseModel):
def encrypt_meeting(meeting_id: str, req: EncryptRequest, db: Session = Depends(get_db)):
meeting = db.query(Meeting).filter(Meeting.id == meeting_id).first()
if not meeting:
raise HTTPException(404, "Meeting not found")
raise HTTPException(404, get_translation('errors.meeting_not_found'))
if meeting.is_encrypted:
raise HTTPException(400, "Meeting is already encrypted")
raise HTTPException(400, get_translation('errors.meeting_already_encrypted'))
if not req.password:
raise HTTPException(400, "Password is required")
raise HTTPException(400, get_translation('errors.password_required'))

svc = EncryptionService
salt = svc.generate_salt()
Expand Down Expand Up @@ -66,13 +67,13 @@ def encrypt_meeting(meeting_id: str, req: EncryptRequest, db: Session = Depends(
def decrypt_meeting(meeting_id: str, req: DecryptRequest, db: Session = Depends(get_db)):
meeting = db.query(Meeting).filter(Meeting.id == meeting_id).first()
if not meeting:
raise HTTPException(404, "Meeting not found")
raise HTTPException(404, get_translation('errors.meeting_not_found'))
if not meeting.is_encrypted:
raise HTTPException(400, "Meeting is not encrypted")
raise HTTPException(400, get_translation('errors.meeting_not_encrypted'))

svc = EncryptionService
if not svc.check_password(req.password, meeting.encryption_salt, meeting.encryption_verify):
raise HTTPException(403, "Wrong password")
raise HTTPException(403, get_translation('errors.wrong_password'))

salt = base64.b64decode(meeting.encryption_salt)
key = svc.derive_key(req.password, salt)
Expand Down
37 changes: 18 additions & 19 deletions api/export.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,10 @@
from database import get_db
from models import Meeting, Segment
from models.action import Action, ActionResult
from translations import get_translation

router = APIRouter(prefix="/api", tags=["export"])

UNKNOWN_SPEAKER = "Okand"


def _safe_filename(name: str) -> str:
"""Sanitize a string for use in Content-Disposition filename."""
Expand Down Expand Up @@ -54,7 +53,7 @@ def export_meeting(
):
meeting = db.query(Meeting).filter(Meeting.id == meeting_id).first()
if not meeting:
raise HTTPException(404, "Meeting not found")
raise HTTPException(404, get_translation('errors.meeting_not_found'))

segments = (
db.query(Segment)
Expand All @@ -79,13 +78,13 @@ def export_meeting(
elif format == "pdf":
return _export_pdf(meeting, segments)
else:
raise HTTPException(400, f"Unknown format: {format}")
raise HTTPException(400, get_translation('errors.unknown_format').format(format=format))


def _export_srt(meeting: Meeting, segments: list[Segment]) -> PlainTextResponse:
lines = []
for i, seg in enumerate(segments, 1):
speaker = seg.speaker.display_name if seg.speaker else UNKNOWN_SPEAKER
speaker = seg.speaker.display_name if seg.speaker else get_translation('common.unknown_speaker')
lines.append(str(i))
lines.append(f"{format_srt_time(seg.start_time)} --> {format_srt_time(seg.end_time)}")
lines.append(f"[{speaker}] {seg.text}")
Expand All @@ -103,7 +102,7 @@ def _export_srt(meeting: Meeting, segments: list[Segment]) -> PlainTextResponse:
def _export_vtt(meeting: Meeting, segments: list[Segment]) -> PlainTextResponse:
lines = ["WEBVTT", ""]
for seg in segments:
speaker = seg.speaker.display_name if seg.speaker else UNKNOWN_SPEAKER
speaker = seg.speaker.display_name if seg.speaker else get_translation('common.unknown_speaker')
lines.append(f"{format_vtt_time(seg.start_time)} --> {format_vtt_time(seg.end_time)}")
lines.append(f"<v {speaker}>{seg.text}")
lines.append("")
Expand All @@ -121,7 +120,7 @@ def _export_txt(meeting: Meeting, segments: list[Segment]) -> PlainTextResponse:
lines = []
current_speaker = None
for seg in segments:
speaker = seg.speaker.display_name if seg.speaker else UNKNOWN_SPEAKER
speaker = seg.speaker.display_name if seg.speaker else get_translation('common.unknown_speaker')
if speaker != current_speaker:
if lines:
lines.append("")
Expand All @@ -148,7 +147,7 @@ def _export_json(meeting: Meeting, segments: list[Segment]) -> JSONResponse:
{
"start": seg.start_time,
"end": seg.end_time,
"speaker": seg.speaker.display_name if seg.speaker else UNKNOWN_SPEAKER,
"speaker": seg.speaker.display_name if seg.speaker else get_translation('common.unknown_speaker'),
"text": seg.text,
}
for seg in segments
Expand All @@ -170,7 +169,7 @@ def _export_md(meeting: Meeting, segments: list[Segment]) -> PlainTextResponse:

current_speaker = None
for seg in segments:
speaker = seg.speaker.display_name if seg.speaker else UNKNOWN_SPEAKER
speaker = seg.speaker.display_name if seg.speaker else get_translation('common.unknown_speaker')
if speaker != current_speaker:
lines.append("")
ts = format_timestamp_short(seg.start_time)
Expand Down Expand Up @@ -204,7 +203,7 @@ def _export_docx(meeting: Meeting, segments: list[Segment]) -> StreamingResponse

current_speaker = None
for seg in segments:
speaker = seg.speaker.display_name if seg.speaker else UNKNOWN_SPEAKER
speaker = seg.speaker.display_name if seg.speaker else get_translation('common.unknown_speaker')
if speaker != current_speaker:
ts = format_timestamp_short(seg.start_time)
doc.add_heading(f"{speaker} [{ts}]", level=3)
Expand Down Expand Up @@ -272,7 +271,7 @@ def _export_pdf(meeting: Meeting, segments: list[Segment]) -> StreamingResponse:

current_speaker = None
for seg in segments:
speaker = seg.speaker.display_name if seg.speaker else UNKNOWN_SPEAKER
speaker = seg.speaker.display_name if seg.speaker else get_translation('common.unknown_speaker')
if speaker != current_speaker:
ts = format_timestamp_short(seg.start_time)
story.append(Paragraph(f"{speaker} [{ts}]", speaker_style))
Expand Down Expand Up @@ -306,15 +305,15 @@ def export_action_result(
):
result = db.query(ActionResult).filter(ActionResult.id == result_id).first()
if not result:
raise HTTPException(404, "Action result not found")
raise HTTPException(404, get_translation('errors.action_result_not_found'))
if not result.result_text:
raise HTTPException(400, "Action result has no content")
raise HTTPException(400, get_translation('errors.action_result_has_no_content'))

action = db.query(Action).filter(Action.id == result.action_id).first()
action_name = action.name if action else "Action"
action_name = action.name if action else get_translation('ui.action')

meeting = db.query(Meeting).filter(Meeting.id == result.meeting_id).first()
meeting_title = meeting.title if meeting else "Meeting"
meeting_title = meeting.title if meeting else get_translation('ui.meeting')

filename = _safe_filename(f"{meeting_title} - {action_name}")

Expand All @@ -324,7 +323,7 @@ def export_action_result(
headers={"Content-Disposition": f'attachment; filename="{filename}.txt"'},
)
elif format == "md":
md_content = f"# {action_name}\n\n**Meeting:** {meeting_title}\n\n---\n\n{result.result_text}"
md_content = f"# {action_name}\n\n**{get_translation('ui.meeting')}:** {meeting_title}\n\n---\n\n{result.result_text}"
return PlainTextResponse(
md_content,
headers={
Expand All @@ -337,7 +336,7 @@ def export_action_result(
elif format == "pdf":
return _export_action_pdf(result.result_text, action_name, meeting_title, filename)
else:
raise HTTPException(400, f"Unknown format: {format}")
raise HTTPException(400, get_translation('errors.unknown_format').format(format=format))


def _export_action_docx(text: str, action_name: str, meeting_title: str, filename: str) -> StreamingResponse:
Expand All @@ -347,7 +346,7 @@ def _export_action_docx(text: str, action_name: str, meeting_title: str, filenam
doc = Document()
doc.add_heading(action_name, level=1)
p = doc.add_paragraph()
run = p.add_run(f"Meeting: {meeting_title}")
run = p.add_run(f"{get_translation('ui.meeting')}: {meeting_title}")
run.font.size = Pt(10)
run.font.color.rgb = RGBColor(128, 128, 128)

Expand Down Expand Up @@ -391,7 +390,7 @@ def _export_action_pdf(text: str, action_name: str, meeting_title: str, filename

story = []
story.append(Paragraph(action_name, title_style))
story.append(Paragraph(f"Meeting: {meeting_title}", meta_style))
story.append(Paragraph(f"{get_translation('ui.meeting')}: {meeting_title}", meta_style))

for line in text.split("\n"):
safe = line.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")
Expand Down
Loading