Aviary

Run every good open-source TTS engine on your Mac, behind one switchboard.

Aviary is a self-hosted manager for local text-to-speech engines on Apple Silicon. One hub process (management plane + WebUI on :5050) supervises per-engine worker processes, each in its own isolated uv venv — because the engines' Python dependencies are mutually incompatible. Every worker speaks the same OpenAI-style HTTP contract on a stable port, so any client that can call one engine can call them all.

中文说明 → · License: MIT (code — model weights have their own licenses, see below)

• WebUI: start/stop/monitor engines (live logs, memory, RTF), generate and download speech, clone voices from reference audio, A/B-compare engines side by side, browse history. Desktop + mobile (PWA), light/dark.
• Engines are plugins: a folder with a manifest and a ~60-line adapter. The UI renders itself from each engine's self-reported capabilities — adding an engine changes zero UI code.
• Memory-aware: engines start on demand (a generation request auto-starts its engine) and unload after a configurable idle period.

Supported engines

All verified end-to-end (whisper transcription round-trip). RTF = synthesis time ÷ audio duration, measured on an Apple Silicon Mac; lower is faster, < 1 is faster than real time.

Engine	Official repo	Port	Runtime	Voice paradigm	RTF
VoxCPM2	OpenBMB/VoxCPM	5077	mlx-audio, 8-bit	voice description + cloning	~0.44
MOSS-TTS-Nano	OpenMOSS/MOSS-TTS-Nano	5075	mlx-audio	cloning (+ legacy /api/generate dialect)	~1.5
MeloTTS	myshell-ai/MeloTTS	5065	PyTorch CPU (Py3.11)	fixed presets (ZH mix-EN, EN accents)	~0.48
Qwen3-TTS (1.7B Base)	QwenLM/Qwen3-TTS	5051	mlx-audio, 8-bit	default voice + 3-second cloning¹	~0.24 warm
IndexTTS2	index-tts/index-tts	5052	mlx-indextts, 8-bit	cloning + 8-emotion control	~1.4
GPT-SoVITS (v2ProPlus)	RVC-Boss/GPT-SoVITS	5053	official api_v2.py, PyTorch CPU (Py3.10)	cloning (fine-tune offline)	~0.39
F5-TTS	SWivid/F5-TTS	5054	f5-tts-mlx	cloning¹	~0.54
fake	— (test tones, ships with Aviary)	5099	none	presets + cloning	—

¹ Cloning with Qwen3-TTS and F5-TTS requires the reference voice's transcript; the WebUI warns when it's missing.

Model weight licenses differ from this repo's MIT code license. MeloTTS and GPT-SoVITS are MIT throughout; MOSS-TTS-Nano and Qwen3-TTS weights are Apache-2.0; IndexTTS2 weights are non-commercial (written permission needed for commercial use); F5-TTS weights are CC-BY-NC; VoxCPM2 — see its model card. Check the upstream license before commercial use.

Requirements

• Apple Silicon Mac (tested with 128 GB unified memory; a few engines resident at once take 5–20 GB — the idle auto-unload keeps that in check)
• uv (manages every Python version and venv; you never touch pip)
• ffmpeg on PATH (brew install ffmpeg) — mp3 encode + upload normalizing
• Per-engine extras, installed once and only if you use that engine: MeloTTS wants brew install mecab; GPT-SoVITS wants brew install cmake (each engine's engine.toml lists its own [install] hints)

Setup

git clone https://github.com/leoli-dev/aviary.git
cd aviary
cd hub && uv sync && cd ..     # the hub's own venv — the only manual install

That's the whole setup. Engine dependencies and model weights are not downloaded up front: the first time you start an engine, the hub creates its venv (uv sync), runs its one-time install steps, and the model downloads from Hugging Face on first load. Expect a few minutes per engine, once.

Start / stop the WebUI

./scripts/hub.sh start      # hub + WebUI on http://localhost:5050
./scripts/hub.sh status     # hub pid + engine states
./scripts/hub.sh logs       # tail the hub log
./scripts/hub.sh stop

Or run it in the foreground: cd hub && uv run aviary (Ctrl-C to stop). Stopping the hub stops all engine workers with it.

To keep it running across reboots, a launchd unit ships in deploy/com.aviary.hub.plist — install instructions are in the file's header comment.

The default config.toml binds 0.0.0.0, so other devices on your LAN (e.g. your iPhone) can open http://<mac-hostname>:5050 — the WebUI is responsive and installable as a PWA. Bind 127.0.0.1 instead if you want it local-only; there is no authentication.

Using it

1. Engines page — start an engine (or don't: generating auto-starts it). Each card shows status, memory, RTF, live logs, and its configuration.
2. Studio — pick an engine; the voice options and parameter sliders come from the engine itself. Type text, Generate, listen, download. Toggle A/B compare to send the same text to several engines side by side.
3. Voices — upload 3–30 s of clean speech (with its transcript!) once, then clone that voice with any engine that supports cloning.
4. History — every generation kept: re-listen, re-download, or load it back into Studio.

API

Everything the WebUI does is plain HTTP.

Hub API (:5050) — engine lifecycle, generation jobs, voices, history, config. Interactive docs at http://localhost:5050/docs. The essentials:

# submit a generation (202 + job id; auto-starts the engine if stopped)
curl -X POST localhost:5050/api/generate -H 'content-type: application/json' \
  -d '{"engine": "voxcpm", "text": "Hello from Aviary.", "response_format": "wav"}'

# poll, then download
curl localhost:5050/api/jobs/<job_id>
curl -o out.wav "localhost:5050/api/jobs/<job_id>/audio?download=true"

Worker API (one per engine, on the ports in the table) — every worker speaks the same unified contract:

Endpoint	What it does
GET /	health — 2xx means the model is loaded
GET /capabilities	voice modes, parameter schema, languages, formats
POST /v1/audio/speech	OpenAI-style synthesis: JSON in, raw audio bytes out (X-Audio-Duration / X-Sample-Rate headers)
POST /clone	multipart cloning: text + reference_audio file + optional prompt_text transcript

# direct worker call, no hub involved
curl -X POST localhost:5077/v1/audio/speech -H 'content-type: application/json' \
  -d '{"input": "Direct to the engine.", "response_format": "wav"}' -o direct.wav

# cloning
curl -X POST localhost:5077/clone \
  -F text="Say this in the cloned voice." \
  -F reference_audio=@voice.wav -F prompt_text="transcript of voice.wav" -o clone.wav

OpenAPI documents: every running worker serves Swagger UI at http://localhost:<port>/docs and its spec at /openapi.json. Static copies for all engines are committed under docs/api/ — one <engine>.openapi.json each, including that engine's exact parameter schema as the /capabilities response example. Regenerate after changing an adapter:

cd engines/<id> && uv run --no-sync python -m tts_hub_sdk.export_openapi \
  "$(grep '^module' engine.toml | cut -d'"' -f2)" > ../../docs/api/<id>.openapi.json

Engines with legacy clients also mount dialect routes — MOSS-TTS-Nano keeps the upstream multipart POST /api/generate (base64 JSON response), included in its OpenAPI doc.

Adding an engine

An engine is a folder under engines/ with three files. Copy engines/fake/ as a working template.

1. engine.toml — the manifest:

[engine]
name = "myengine"
title = "My Engine"
port = 5058
backend = "mlx"                 # informational label
python = "3.12"                 # uv provisions this
module = "adapter:MyEngine"     # class in adapter.py
autostart = false
idle_unload_min = 30            # 0 = stay resident

[install]                        # optional one-time steps on first start
brew = ["cmake"]                 # hints (checked by you)
post = ["python -m something_download"]   # run inside the fresh venv

2. pyproject.toml — the engine's own dependencies plus the shared SDK:

[project]
name = "tts-engine-myengine"
requires-python = ">=3.12"
dependencies = ["some-tts-lib", "tts-hub-sdk"]

[tool.uv.sources]
tts-hub-sdk = { path = "../../sdk", editable = true }

3. adapter.py — one class:

class MyEngine:
    capabilities = {
        "engine": "myengine",
        "voice_modes": ["preset_list", "reference_audio"],  # what the UI offers
        "voices": [{"id": "alto", "label": "Alto"}],
        "languages": ["en"],
        "params": [   # becomes sliders/selects in Studio, coerced by the SDK
            {"name": "speed", "type": "float", "min": 0.5, "max": 2.0,
             "default": 1.0, "step": 0.05, "label": "Speed"},
        ],
        "formats": ["wav", "mp3"],
        "sample_rate": 24000,
        "max_chunk_chars": 400,   # hub splits longer text; 0 = engine handles it
    }

    def load(self) -> None:
        ...  # blocking model load; runs once on the inference thread

    def synthesize(self, text, params, ref_audio=None, ref_text=None):
        ...  # return (waveform_ndarray, sample_rate)

Then click Settings → Rescan engines (or restart the hub). The first Start builds the venv, runs install.post, downloads weights — and the WebUI needs zero changes: forms render from capabilities.

Removing an engine: stop it, delete (or move away) its folder, rescan. Hiding without deleting: Disable engine in its configuration panel.

Configuration (two layers)

Engine defaults live in each engine's engines/<id>/engine.toml (shown above). Your overrides live in config.toml at the repo root, written by the Settings page or edited by hand:

[hub]
host = "0.0.0.0"        # 127.0.0.1 = local only
port = 5050
max_upload_mb = 100

[engines.melotts]
port = 5066             # override the manifest default
autostart = true
idle_unload_min = 0     # keep resident
enabled = false         # hide from Studio without deleting files
env = { MELO_LANG = "ZH" }   # worker env vars — model variants go here

Precedence: config.toml > engine.toml. Overrides for a running engine apply on its next start. Model choice is set via env vars read by each adapter (VOXCPM_MODEL, QWEN3_TTS_MODEL, INDEXTTS_MODEL, F5_MODEL, MOSS_MODEL, MELO_LANG — see each adapter.py header).

Repo layout

hub/        management plane: process supervision, SSE, jobs, SQLite, WebUI (Py3.12)
sdk/        tts_hub_sdk — the shared worker layer: unified HTTP contract,
            lenient param coercion, wav/mp3 encoding, single-thread
            inference executor, orphan self-exit, OpenAPI export
engines/    one folder per engine (manifest + pyproject + adapter)
docs/api/   static OpenAPI documents, one per engine
scripts/    hub.sh start/stop/status/logs
deploy/     launchd unit for the hub
config.toml hub settings + per-engine overrides
data/       runtime state (gitignored): audio, voices, logs, hub.db

Acknowledgements

Aviary stands on the engines' upstream teams and the Apple Silicon runtimes that host them: mlx-audio, mlx-indextts, f5-tts-mlx, and GPT-SoVITS's built-in API server.