Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions docs/memory/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -566,21 +566,25 @@ Package `services/compatibility/` mirrors the deterministic `canary/` library:

**Note**: Route ordering is critical — static routes (`/context-stats`, `/autonomy-status`) must be defined BEFORE the `/{name}` catch-all (Invariant #4).

### Voice (5 endpoints)
### Voice (6 endpoints)
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/agents/{name}/voice/start` | Start Gemini Live voice session; `workspace_mode` enables panel tools |
| POST | `/api/agents/{name}/voice/start` | Start Gemini Live voice session; `workspace_mode` enables panel tools. Resolves voice as request override → persisted `voice_name` → `Kore` (#28) |
| POST | `/api/agents/{name}/voice/stop` | Stop active voice session |
| GET/PUT | `/api/agents/{name}/voice/prompt` | Get/set per-agent voice system prompt |
| GET/PUT | `/api/agents/{name}/voice/name` | Get (any accessor; returns `available_voices`/`default_voice`) / set (owner-only; 400 on a voice outside `GEMINI_VOICE_NAMES`) the persisted per-agent Gemini voice. Applies to both the voice overlay/workspace and outbound VoIP calls (#28) |
| GET | `/api/agents/{name}/voice/{session_id}/panel` | Canvas panel state for workspace mode (ownership-gated; empty state when session gone, #699) |

### VoIP Telephony (VOIP-001, #1056 — flag-gated, default OFF)
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| GET/PUT/DELETE | `/api/agents/{name}/voip` | Owner | Twilio-voice binding status / configure / remove. 404 when `voip_available` off |
| GET/PUT/DELETE | `/api/agents/{name}/voip` | Owner | Twilio-voice binding status / configure / remove. 404 when `voip_available` off. Re-PUT preserves the `enabled` flag (#28) |
| PUT | `/api/agents/{name}/voip/enabled` | Owner | Enable/disable the binding without re-entering credentials (`{enabled: bool}`); 404 when no binding exists. Disabled ⇒ outbound calls refused (#28) |
| POST | `/api/agents/{name}/voip/call` | JWT/MCP (`AuthorizedAgent`) | Place outbound call; rate-limited + daily-capped; optional `Idempotency-Key`. Returns `{call_id, status:"ringing", twilio_call_sid}` |
| WS | `/api/voip/voice/{call_id}` | Call-bound ticket | Twilio Media Streams audio bridge — see [VoIP](#voip-telephony-voip-001-1056) |

The per-agent VoIP config + voice-picker UI lives in the agent Settings/Sharing tab (`components/VoipChannelPanel.vue`), shown only when the platform `voip_available` flag is true (frontend reads it via `stores/sessions.js`); the underlying CRUD/voice endpoints are OSS and ungated (#28).

### Activities (1 endpoint)
| Method | Path | Description |
|--------|------|-------------|
Expand Down Expand Up @@ -862,6 +866,7 @@ CREATE TABLE agent_ownership (
max_backlog_depth INTEGER DEFAULT 50, -- BACKLOG-001
group_auth_mode TEXT DEFAULT 'none',
voice_system_prompt TEXT,
voice_name TEXT, -- #28: persisted Gemini voice (NULL → 'Kore')
guardrails_config TEXT,
file_sharing_enabled INTEGER DEFAULT 0, -- FILES-001
circuit_breaker_enabled INTEGER DEFAULT 0, -- RELIABILITY-007 (#526): dispatch-breaker opt-in
Expand Down
19 changes: 19 additions & 0 deletions docs/memory/feature-flows/voice-chat.md
Original file line number Diff line number Diff line change
Expand Up @@ -443,6 +443,25 @@ Returns current canvas panel state. Returns empty state (not 404) for non-existe

---

## Persisted per-agent voice (trinity-enterprise#28)

The Gemini voice was historically hardcoded to `"Kore"` (`routers/voice.py::_get_voice_name`).
It is now a persisted per-agent setting, `agent_ownership.voice_name` (default `Kore`,
an edition-agnostic OSS primitive like `voice_system_prompt`):

- `GET/PUT /api/agents/{name}/voice/name` — GET returns the persisted voice plus
`available_voices`/`default_voice`; PUT is owner-only and validates against the
canonical `GEMINI_VOICE_NAMES` (`config.py`), shared with the frontend picker
list (`src/constants/voices.js`) and drift-guarded by a unit test.
- **Resolution at session start** (`voice_start`): per-session request override
(`VoiceStartRequest.voice_name`) → persisted `voice_name` → `Kore`. The read
path (`db.get_voice_name`) falls back to `Kore` for an unset or no-longer-valid
persisted value.
- The **Workspace** ephemeral picker (`AgentWorkspace.vue`) now defaults its
selection to the persisted voice; the picker list is the shared
`src/constants/voices.js` module. The persisted voice also drives outbound VoIP
calls — see `voip-telephony.md`.

## Related Documentation

- [Authenticated Chat Tab](./authenticated-chat-tab.md) — Existing chat implementation
Expand Down
32 changes: 28 additions & 4 deletions docs/memory/feature-flows/voip-telephony.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# Feature: VoIP Telephony — Outbound Calls over Gemini Live (VOIP-001)

> **Status (2026-06-04)**: Phase 1 (outbound) implemented for #1056. Behind a
> feature flag that is **OFF by default** (`VOIP_ENABLED`). Inbound (Phase 2)
> and UI/observability (Phase 3) are not yet built. Real PSTN path is
> manual-verify (needs a live Twilio voice number).
> **Status (2026-06-23)**: Phase 1 (outbound) implemented for #1056. Behind a
> feature flag that is **OFF by default** (`VOIP_ENABLED`). Phase 3 config UI +
> persisted voice picker added (trinity-enterprise#28, OSS — see below). Inbound
> (Phase 2) and call cost/duration observability are not yet built. Real PSTN
> path is manual-verify (needs a live Twilio voice number).

## Overview

Expand Down Expand Up @@ -216,6 +217,29 @@ PSTN barge-in latency and telephone-speech (8kHz μ-law) STT quality are
unvalidated until real calls happen — Phase 1 logs the barge-in/transcript
signals so they can be measured on the first live calls.

## Phase 3 — Config UI + persisted voice (trinity-enterprise#28)

Delivered as a plain **OSS** feature (not entitlement-gated — a UI gate over a
money-spending OSS backend would be cosmetic; gated on the existing
`voip_available` platform flag instead):

- **Panel**: `src/frontend/src/components/VoipChannelPanel.vue` (modeled on
`WhatsAppChannelPanel.vue`), mounted in `SharingPanel.vue` under
`v-if="sessionsStore.voipAvailable"` (`stores/sessions.js` now reads
`voip_available` from `/api/settings/feature-flags`). Create/update/remove the
binding over the existing `GET/PUT/DELETE /api/agents/{name}/voip`; the Auth
Token is write-only (never returned).
- **Enable/disable**: `PUT /api/agents/{name}/voip/enabled` (`{enabled}`,
owner-only, 404 when no binding) flips `voip_bindings.enabled` without
re-entering credentials. The upsert no longer forces `enabled=1`, so re-saving
credentials **preserves** a disabled state (the call path already refuses
disabled bindings in `voip_service.py`).
- **Persisted voice**: new OSS primitive `agent_ownership.voice_name`
(default `Kore`) via `GET/PUT /api/agents/{name}/voice/name` (PUT owner-only,
validated against `GEMINI_VOICE_NAMES`). `services/voip_service.py` reads it
per call instead of the old hardcoded `"Kore"`. See `voice-chat.md`.
- **Tests**: `tests/unit/test_28_voip_voice_config.py`.

## Related Flows

- **Upstream**: `voice-chat.md` (the Gemini Live bridge this reuses unchanged),
Expand Down
8 changes: 8 additions & 0 deletions docs/memory/learnings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Learnings — durable bug-class ledger

Append-only. One entry per *class* of error/pattern worth knowing before the next
plan or review. `/autoplan` reads this before planning; write for that reader.

## 2026-06-23 — pitfall — Adding a column needs BOTH schema.py AND tables.py; parity test guards only one
**Context**: `agent_ownership.voice_name` (trinity-enterprise#28). `tests/unit/test_schema_parity.py` compares `db/schema.py` (raw DDL) against `db/migrations.py` (the SQLite runner) — it does **not** import `db/tables.py` (the SQLAlchemy Core MetaData that Alembic autogenerates from and that every `db/*.py` accessor binds columns against). So a change that adds a column to `schema.py` + `migrations.py` but forgets `tables.py` keeps the parity test green, yet `select(agent_ownership.c.<col>)` blows up at runtime on the first call.
**Lesson**: A schema change touches **four** files — `db/schema.py`, `db/tables.py`, a SQLite migration in `db/migrations.py`, and an Alembic revision under `db/../migrations/versions/`. Don't trust schema-parity to catch a missing `tables.py` Column. The real guard is a db-accessor unit test that executes a live `select(table.c.<new_col>)` against a migrated DB (db_harness) — it fails loudly if `tables.py` was missed.
38 changes: 35 additions & 3 deletions docs/memory/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -2840,9 +2840,41 @@ Standalone mobile-friendly admin page for managing agents on the go. Designed as
Phase 2: Twilio voice webhook + `X-Twilio-Signature` + inbound
number→agent resolution on the same table; an `inbound_number` column
is shipped up-front so Phase 2 is additive); opt-in destination
allowlist (Phase 2); UI config surface, schedule-action trigger, and
call cost/duration observability (Phase 3). Real PSTN path is
manual-verify (needs a live Twilio voice number).
allowlist (Phase 2); schedule-action trigger and call cost/duration
observability (Phase 3). Real PSTN path is manual-verify (needs a live
Twilio voice number). (UI config surface delivered separately in §39.2.)

### 39.2 Per-agent VoIP config UI + persisted voice (trinity-enterprise#28)
- **Status**: 🚧 In Progress
- **Implements**: Issue trinity-enterprise#28 (the Phase-3 "UI config surface"
deferred by #1056)
- **Description**: A per-agent VoIP config panel in the agent Settings/Sharing
tab (`components/VoipChannelPanel.vue`) to create/update/remove the
`voip_bindings` row over the existing OSS `GET/PUT/DELETE /api/agents/{name}/voip`
endpoints (Twilio Account SID, write-only Auth Token, From number, optional
daily call cap), an **enable/disable toggle**, and a **persisted voice picker**.
- **OSS, not entitlement-gated**: shipped as a plain OSS capability gated purely
on the existing `voip_available` platform flag (the frontend reads it via
`stores/sessions.js`). No enterprise entitlement, no `register_module`, no
`trinity-enterprise` submodule change — a deliberate simplification over the
issue's original "entitlement-gated" framing (a UI gate over an OSS,
money-spending backend would be cosmetic; gate is the platform flag instead).
- **Enable/disable toggle**: `PUT /api/agents/{name}/voip/enabled`
(`{enabled: bool}`, owner-only, 404 when no binding) flips `voip_bindings.enabled`
without re-entering credentials; the call path already refuses disabled
bindings. Re-saving credentials via the binding PUT **preserves** the current
`enabled` state (the upsert no longer forces `enabled=1`).
- **Persisted per-agent voice**: new edition-agnostic OSS primitive
`agent_ownership.voice_name` (default `Kore`, like `voice_system_prompt`) with
`GET/PUT /api/agents/{name}/voice/name` (PUT owner-only, validated against the
canonical `GEMINI_VOICE_NAMES`). Replaces the two hardcoded `"Kore"` sites
(`routers/voice.py::_get_voice_name`, `services/voip_service.py`), so the chosen
voice applies to **both** the in-app voice overlay/workspace and outbound VoIP
calls. Resolution precedence at a voice start: per-session request override →
persisted `voice_name` → `Kore`; the read path falls back to `Kore` for an
unset or no-longer-valid persisted value. The Workspace ephemeral picker
defaults to the persisted voice. Dual-track migration (SQLite `db/migrations.py`
+ Alembic `0004_agent_ownership_voice_name` + `db/schema.py`/`db/tables.py`).

---

Expand Down
8 changes: 8 additions & 0 deletions src/backend/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,14 @@
VOICE_MODEL = os.getenv("VOICE_MODEL") or "models/gemini-3.1-flash-live-preview"
VOICE_MAX_DURATION = int(os.getenv("VOICE_MAX_DURATION", "300")) # seconds

# Per-agent voice selection (#28). Canonical set of Gemini Live prebuilt voices
# offered by Trinity; the single source of truth shared by the persisted-voice
# write validation and the read-path fallback (and mirrored by the frontend
# picker). DEFAULT_VOICE_NAME is the historical hardcoded default and the
# fallback for an unset or no-longer-valid persisted value.
DEFAULT_VOICE_NAME = "Kore"
GEMINI_VOICE_NAMES = ("Kore", "Zephyr", "Puck", "Aoede", "Charon", "Fenrir")

# Gemini text/audio models (#1130). Hardcoded `gemini-2.0-flash` was retired by
# Google (404 NOT_FOUND) with no config escape hatch — these env overrides make
# the next model retirement a config change instead of a code change. Same `or`
Expand Down
9 changes: 9 additions & 0 deletions src/backend/database.py
Original file line number Diff line number Diff line change
Expand Up @@ -741,6 +741,12 @@ def get_voice_system_prompt(self, agent_name: str):
def set_voice_system_prompt(self, agent_name: str, prompt: str):
return self._agent_ops.set_voice_system_prompt(agent_name, prompt)

def get_voice_name(self, agent_name: str):
return self._agent_ops.get_voice_name(agent_name)

def set_voice_name(self, agent_name: str, voice_name):
return self._agent_ops.set_voice_name(agent_name, voice_name)

# =========================================================================
# MCP API Key Management (delegated to db/mcp_keys.py)
# =========================================================================
Expand Down Expand Up @@ -1960,6 +1966,9 @@ def update_voip_webhook_url(self, agent_name, webhook_url):
def delete_voip_binding(self, agent_name):
return self._voip_ops.delete_binding(agent_name)

def set_voip_enabled(self, agent_name, enabled):
return self._voip_ops.set_enabled(agent_name, enabled)

def create_voip_call_log(self, call_id, agent_name, to_number, chat_session_id=None,
initiated_by_user_id=None, initiated_by_email=None,
direction="outbound"):
Expand Down
33 changes: 33 additions & 0 deletions src/backend/db/agents.py
Original file line number Diff line number Diff line change
Expand Up @@ -397,3 +397,36 @@ def set_voice_system_prompt(self, agent_name: str, prompt: Optional[str]) -> boo
.values(voice_system_prompt=prompt or None)
)
return result.rowcount > 0

# =========================================================================
# Voice Name (#28) — persisted per-agent Gemini Live voice
# =========================================================================

def get_voice_name(self, agent_name: str) -> str:
"""Get the persisted Gemini voice for an agent.

Falls back to DEFAULT_VOICE_NAME ('Kore') when unset, and ALSO when the
persisted value is not in the current GEMINI_VOICE_NAMES set (a voice
removed after it was saved) — defense-in-depth so the call path never
hands Gemini an unusable voice (#28, reviewer M1).
"""
from config import DEFAULT_VOICE_NAME, GEMINI_VOICE_NAMES

stmt = select(agent_ownership.c.voice_name).where(
agent_ownership.c.agent_name == agent_name,
agent_ownership.c.deleted_at.is_(None),
)
with get_engine().connect() as conn:
row = conn.execute(stmt).mappings().first()
value = row["voice_name"] if row else None
return value if value in GEMINI_VOICE_NAMES else DEFAULT_VOICE_NAME

def set_voice_name(self, agent_name: str, voice_name: Optional[str]) -> bool:
"""Set the persisted Gemini voice for an agent (None clears it → default)."""
with get_engine().begin() as conn:
result = conn.execute(
update(agent_ownership)
.where(agent_ownership.c.agent_name == agent_name)
.values(voice_name=voice_name or None)
)
return result.rowcount > 0
17 changes: 17 additions & 0 deletions src/backend/db/migrations.py
Original file line number Diff line number Diff line change
Expand Up @@ -971,6 +971,22 @@ def _migrate_agent_ownership_voice_prompt(cursor, conn):
)
conn.commit()

def _migrate_agent_ownership_voice_name(cursor, conn):
"""Add voice_name column to agent_ownership (#28).

Persists the per-agent Gemini Live voice (e.g. 'Kore', 'Puck'). NULL means
unset — the getter falls back to DEFAULT_VOICE_NAME ('Kore'), preserving the
prior hardcoded behaviour for existing agents.
"""
_safe_add_column(
cursor,
"agent_ownership",
"voice_name",
"ALTER TABLE agent_ownership ADD COLUMN voice_name TEXT",
)
conn.commit()


def _migrate_slack_channel_agents(cursor, conn):
"""Add multi-agent Slack support: workspace table + channel-agent bindings.

Expand Down Expand Up @@ -2564,4 +2580,5 @@ def _migrate_agent_compatibility_results_table(cursor, conn):
("operator_queue_cleared_at", _migrate_operator_queue_cleared_at),
("activities_created_index", _migrate_activities_created_index),
("agent_compatibility_results_table", _migrate_agent_compatibility_results_table),
("agent_ownership_voice_name", _migrate_agent_ownership_voice_name),
]
1 change: 1 addition & 0 deletions src/backend/db/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@
max_backlog_depth INTEGER DEFAULT 50,
group_auth_mode TEXT DEFAULT 'none',
voice_system_prompt TEXT,
voice_name TEXT,
guardrails_config TEXT,
file_sharing_enabled INTEGER DEFAULT 0,
circuit_breaker_enabled INTEGER DEFAULT 0,
Expand Down
1 change: 1 addition & 0 deletions src/backend/db/tables.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ def process_bind_param(self, value, dialect):
Column("max_backlog_depth", Integer),
Column("group_auth_mode", Text),
Column("voice_system_prompt", Text),
Column("voice_name", Text),
Column("guardrails_config", Text),
Column("file_sharing_enabled", Integer),
Column("circuit_breaker_enabled", Integer),
Expand Down
21 changes: 20 additions & 1 deletion src/backend/db/voip.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,11 @@ def create_binding(
"webhook_secret": webhook_secret,
"daily_call_cap": cap,
"display_name": display_name,
"enabled": 1,
# NOTE: `enabled` is deliberately NOT in the conflict set_ (#28).
# A fresh binding inserts enabled=1; re-saving credentials on an
# existing binding must PRESERVE the current enable state so an
# owner who disabled VoIP doesn't get it silently re-enabled by
# editing the from-number (reviewer H3). Toggle via set_enabled().
"updated_at": now,
},
)
Expand Down Expand Up @@ -186,6 +190,21 @@ def update_webhook_url(self, agent_name: str, webhook_url: str) -> None:
with get_engine().begin() as conn:
conn.execute(stmt)

def set_enabled(self, agent_name: str, enabled: bool) -> bool:
"""Flip a binding's enabled flag without touching credentials (#28).

Returns False when no binding row exists for the agent (caller 404s) so
the UI never shows a disabled-but-nonexistent state.
"""
stmt = (
update(voip_bindings)
.where(voip_bindings.c.agent_name == agent_name)
.values(enabled=1 if enabled else 0, updated_at=utc_now_iso())
)
with get_engine().begin() as conn:
result = conn.execute(stmt)
return result.rowcount > 0

def delete_binding(self, agent_name: str) -> bool:
stmt = delete(voip_bindings).where(voip_bindings.c.agent_name == agent_name)
with get_engine().begin() as conn:
Expand Down
30 changes: 30 additions & 0 deletions src/backend/migrations/versions/0004_agent_ownership_voice_name.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
"""Add voice_name column to agent_ownership (#28)

Persists the per-agent Gemini Live voice on the PostgreSQL backend. Mirrors the
SQLite ``agent_ownership_voice_name`` migration in ``db/migrations.py`` and the
DDL in ``db/schema.py`` / MetaData in ``db/tables.py``.

Fresh PG builds already get the column because ``0001_baseline`` iterates
``db/schema.py:TABLES``. This revision exists so an *existing* PG deployment —
stamped at an earlier revision and never re-running baseline — also picks the
column up on ``alembic upgrade head``.

Revision ID: 0004_agent_ownership_voice_name
Revises: 0003_agent_compatibility_results
Create Date: 2026-06-23
"""
from alembic import op

# revision identifiers, used by Alembic.
revision = "0004_agent_ownership_voice_name"
down_revision = "0003_agent_compatibility_results"
branch_labels = None
depends_on = None


def upgrade() -> None:
op.execute("ALTER TABLE agent_ownership ADD COLUMN IF NOT EXISTS voice_name TEXT")


def downgrade() -> None:
op.execute("ALTER TABLE agent_ownership DROP COLUMN IF EXISTS voice_name")
Loading
Loading