Local, multi-speaker Japanese speech synthesis for AI agents — turn a multi-voice script into mastered audio (radio drama, audiobook, podcast, panel, briefing, …). Single-binary MCP server, AivisSpeech Engine backend.
Language: Japanese only. The AivisSpeech Engine synthesizes Japanese; other languages are not supported.
Producing scripted, multi-voice spoken audio takes two kinds of work: the intellectual part (script conversion, speaker attribution, performance direction) and the mechanical part (speech synthesis, retakes, mastering). AI agents such as Claude Code and Cowork handle the first part well — but they have no voice. voice-studio-mcp gives them one: a fully local MCP server that wraps AivisSpeech Engine (VOICEVOX-compatible API, Style-Bert-VITS2-family models) and turns an agent-authored script into mastered audio. Radio drama and audiobooks were its first use cases; the bundled multi-actor-narration skill adds podcasts, panels, news briefings, and lessons. No cloud APIs, no credentials.
The engine's voice models are Japanese, so voice-studio-mcp produces Japanese audio only — other languages are out of scope.
- Speaker catalog with license metadata — author-declared license text
is collected automatically from each model's AIVM manifest (
declared), and human-reviewed config entries mark modelsverified; thelicensessubcommand turns declarations into reviewable config skeletons. - Pronunciation dictionaries — register proper-noun readings per work so character names are never misread.
- Batch synthesis with a content-hash cache — edit three lines of a 200-line script and only those three lines are re-synthesized.
- Async jobs — long scripts synthesize in the background; poll with
check_job. - Mastering — ffmpeg concat with per-line pauses, one-pass loudness normalization (-18 LUFS default; configurable), mp3 or m4b with scene chapters, and an auto-generated credits file.
- Engine lifecycle management — spawns the AivisSpeech Engine on startup and reaps it on shutdown (or attaches to one that is already running).
- Japanese-language content only — the engine's voice models are Japanese; non-Japanese text is not supported
- macOS on Apple Silicon (v1 target)
- AivisSpeech installed (bundles the engine)
ffmpegin PATH (only for themastertool):brew install ffmpeg- Go 1.25+ (only when building from source)
make build # → dist/voice-studio-mcp
dist/voice-studio-mcp doctor # verify engine / ffmpeg / configRegister with an MCP client (e.g. Claude Code):
{
"mcpServers": {
"voice-studio": {
"command": "/path/to/dist/voice-studio-mcp",
"args": ["serve"]
}
}
}Configuration lives at ~/.config/voice-studio-mcp/config.toml
(see config.example.toml); everything has
sensible defaults.
| Command | Description |
|---|---|
serve |
Start the MCP stdio server (default when no subcommand is given) |
doctor |
Diagnose the environment (config, engine, ffmpeg, workspace dir). --fix repairs an unsigned AivisSpeech Engine macOS refuses to launch (ADR-0012) |
licenses |
Collect voice-model license declarations for review (--full <uuid> full text, --toml config skeleton) |
version |
Print the version |
Run voice-studio-mcp doctor first — it reports the engine, ffmpeg, and
config state read-only, so it will point at the specific problem.
"The engine exits immediately / macOS won't let it launch / it's killed on spawn." This is the most common report. AivisSpeech ships as an unsigned, un-notarized bundle whose nested libraries carry mismatched code signatures; modern macOS kills it at load time, and removing the quarantine flag alone is not enough on some machines. Fix it in one step:
voice-studio-mcp doctor # shows NG/warn engine signature if affected
voice-studio-mcp doctor --fix # strips quarantine + ad-hoc re-signs the engine--fix only touches the engine subtree (never the wider app) and is
idempotent. Re-run it after every AivisSpeech update (an update
restores the quarantine flag). macOS only; see
ADR-0012 for why this
is a local repair rather than a re-signed engine fork.
"initialize hangs for a long time on first use." The first model load
takes minutes. Raise engine.startup_timeout_seconds, or launch the
AivisSpeech GUI once to finish model setup (the server then attaches to it).
"engine command not found." AivisSpeech isn't installed, or it's in a
non-standard location — set engine.command in the config.
The full symptom→remedy table is in the setup guide.
| Tool | Description |
|---|---|
get_usage |
This server's operating manual (workspace model, schema, recovery table) — call it first from clients without the bundled skill |
list_speakers |
Installed voice models + styles + license metadata |
register_dictionary |
Register work-specific pronunciations (katakana readings, accent) |
synthesize_script |
Batch-synthesize a script JSONL → wav/<id>.wav (async; returns job_id) |
synthesize_line |
Synchronous single-line synthesis (retakes, voice auditioning via style_id) |
check_job |
Progress and per-line failures of a batch job |
master |
Concat + loudnorm + mp3/m4b (+ chapters) + credits file |
Tool results are compact JSON summaries (paths, counts, durations) — audio bytes are never returned to the client.
The repository bundles the multi-actor-narration skill — the operational form of the agent workflow (input → casting → dictionary → synthesis → retakes → mastering, with mandatory human checkpoints). One skill routes to five formats over the same voice-studio pipeline:
- talk-podcast / panel-discussion / news-briefing / lesson-narration — document or theme → multi-speaker explainer audio
- audio-drama — novel / script → narrated audio drama / audiobook (the generalized successor of the former radio-drama skill)
make install-skill # → ~/.claude/skills/multi-actor-narration
make package-skill # → dist/multi-actor-narration.skill (portable package)Then ask Claude Code / Cowork:
"/multi-actor-narration samples/manuscript.ja.md" (routes to audio-drama),
or point it at a PDF/theme for an explainer format. The skill source ships
here (not in a separate repo) so it always matches this server's schema and
tools — skills/skills_test.go enforces that across the multi-file skill
(ADR-0009). Releases attach multi-actor-narration.skill as a separate
asset next to the MCP binary, so users without the repo can import the
skill directly (ADR-0011).
One utterance per line; the schema is the contract for agent-side skills:
{"id":1,"scene":1,"speaker":"narrator","text":"夜のとばりが街を包んでいた。","pause_after_ms":800}
{"id":2,"scene":1,"speaker":"美咲","text":"……本当に、行くの?","style":"悲しみ","intensity":1.4,"speed":0.9}id(required, unique, >0) — output iswav/<id>.wav; retakes overwritespeaker(required) — resolved via the casting tablestyle/intensity(0–2) /speed(0.5–2) /volume(0–2) — performance direction (volumesets relative balance; overall loudness is governed by the masteringloudnorm_itarget)pause_after_ms— silence inserted at mastering time (does not invalidate the synthesis cache)scene— m4b chapter boundaries
[characters."narrator"]
speaker_uuid = "..." # from list_speakers
style_id = 888753760 # default style
credit = "AivisSpeech:Anneli"
license_checked = true # human confirmed the model's terms
[characters."美咲"]
speaker_uuid = "..."
style_id = 933744512
[characters."美咲".styles] # script line "style" → style id
"悲しみ" = 933744513list_speakers→ cast characters, writecasting.tomlregister_dictionary→ proper-noun readings- Write the script JSONL into
<workspace>/script/ synthesize_script→ pollcheck_jobsynthesize_linefor retakesmaster→ mp3/m4b + credits
Every workspace tool accepts an optional workspace_root — the absolute
path of a directory the agent prepared in its own writable area (e.g.
inside the project directory). This is how sandboxed MCP clients whose
writes are restricted to the project tree use the server; the server never
follows symlinks out of a workspace (kernel-enforced via os.Root,
ADR-0010). Omitted, the default root applies:
~/.voice-studio/<workspace_id>/
├── script/ script JSONL files (agent-authored)
├── casting.toml character → voice mapping
├── dict/words.json registered dictionary record
├── wav/<id>.wav per-line synthesized audio
├── cache/index.json synthesis cache index
└── master/ mastered outputs + credits + tmp
make test # unit tests (no AivisSpeech / ffmpeg required)
make test-e2e # build + drive the binary over stdio against a mock engine
VOICE_STUDIO_TEST_REAL_ENGINE=1 make test-e2e # opt-in: real engineThe server code is MIT. Voice models have their own terms, separate from any software license. The review workflow (ADR-0008):
voice-studio-mcp licenses— see what each installed model declaresvoice-studio-mcp licenses --full <speaker_uuid>— read the full termsvoice-studio-mcp licenses --toml— generate[[speaker_metadata]]skeletons; filllicense_url/commercial_useand delete the REVIEW note once accepted- Set
license_checked = trueincasting.tomlper work, and ship the generated credits file with your production
list_speakers reports verified (human-reviewed) / declared (manifest
text present, unreviewed) / unverified; only verified models belong in
published audio.
The registry is user data: it lives in your user config, reflects the models installed on your machine and your review, and must not be committed to shared repositories. Since speaker_uuid is model-intrinsic (not machine-local), you may sync it across your own machines via personal dotfiles.
docs/en/reference/agent-workflow.md— the agent workflow guide (hand this to Claude Code / Cowork; sample material insamples/)docs/en/reference/setup.md— setup guide (install → first production → troubleshooting)docs/en/reference/architecture.md— architecture overview and decision indexdocs/en/adr/— twelve ADRs recording the why behind non-obvious designsdocs/en/voice-studio-mcp-rfp.md— the original RFP- 日本語版:
docs/ja/(セットアップ / アーキテクチャ / ADR / RFP)
MIT — see LICENSE.