yanghhu/miragea-video-voiceover

AI Voiceover

"AI Voiceover" is an Agent skill that turns a video script into multi-speaker voiceover (WAV) and time-synced SRT subtitles in one command: use voices from your account's saved actors or voice cloning from a reference audio URL. It covers narration, movie commentary, novel promotion, sales, commercials, and audiobooks, with no local GPU required.

One-Line Definition

AI Voiceover = a script with [speaker] tags -> a full voiceover voiceover.wav + timed subtitles voiceover.srt + per-sentence timestamps manifest.json, in one command. Each sentence is generated by the Miragea open API, locally needing only Python 3.9+, no GPU, and no third-party packages.

Target Users and Scenarios

Scenario	Typical Need	Recommended Preset
On-Camera / Knowledge Creator	Daily cloned voice narration, vertical subtitles	--preset koubo
Movie / Short Drama Commentary	Multi-voice narration + character lines, timeline-aligned	--preset jieshuo
Novel Promotion / Audiobook	Multi-character reading, paragraph pauses, subtitles with speaker names	--preset youshengshu
E-commerce Short Videos	Fast-paced sales voiceover, large subtitles, A/B testing	--preset daihuo
Digital Human / AI Avatar	Clean vocal drive audio + per-sentence timeline	--preset shuziren
Commercials / Promos	Pauses, precise pacing, fixed brand voice	--preset guanggao
Add Subtitles to Existing Video	Audio/video recognition into timestamped SRT	miragea_audio.py recognize

The series also includes scenario-specific script-writing skills: On-Camera Voiceover, Movie Commentary Voiceover, Novel Promotion Voiceover, Text-to-Speech, Voice Cloning, and Subtitle Generation.

Core Capabilities

Item	Description
Output files	voiceover.wav (full dub), voiceover.srt (timed subtitles), manifest.json (per-sentence speaker, text, start/end seconds), lines/ (per-sentence audio, reused on edits)
Voice sources	Actor ID from your account (actor_id= — query with actors), or a server-accessible reference audio URL (ref=) plus the actual spoken text (ref_text=) for voice cloning
Generation mode	expressive (client "Default") / standard (client "More Like"); when omitted, server default is used, subject to real-time capabilities
Tone tags	tags=tag_name, fetch from capabilities' selected mode's tags[].label
Speed	speed= 0.25–4, 1 = normal speed, set per speaker
Multi-speaker	Define one voice per speaker in the script front matter; the skill calls the official client's generate per sentence and joins locally
Pauses	Inline [pause 0.6] / [pause 0.6] (seconds); --gap sets default pause between sentences
Subtitles	Generated during dubbing based on actual audio duration per sentence (--max-chars, --speaker-labels); recognize turns existing audio/video into SRT/TXT/JSON
Scene presets	--preset koubo / jieshuo / daihuo / shuziren / guanggao / youshengshu (on-camera/commentary/e-commerce/digital human/commercial/audiobook), adjusting only sentence pauses and subtitle length
Per-sentence text length, file/duration limits, pricing	Subject to real-time capabilities
Local dependencies	Python 3.9+ standard library, no pip install, no GPU
Offline dry run	--dry-run: no Key needed, no API calls, outputs silent placeholder audio and timeline to verify script format
Supported Agents	Claude Code, Codex, OpenClaw, Cursor, WorkBuddy — any Agent that can run Python scripts
Dubbing & Subtitle Service	Miragea Open API (https://miragea.ai/v1)

Capability Boundaries

Can Do	Won't Do
Single narrator, multi-speaker dialogue (one voice per speaker, generated sentence by sentence and joined locally)	Won't edit video or composite video (outputs audio track and subtitles, leaving them to CapCut/Premiere/ffmpeg)
Voices from your account's actors (actor_id) or reference audio URL voice cloning	Won't clone voices without authorized consent
Pauses, independent speed per speaker, generation mode and tone tags	Won't auto-write scripts (the Agent can write scripts using the methods below)
Audio/video recognition into SRT/TXT/JSON	Won't do background music mixing or translation (translation is done by the Agent)
--dry-run offline full workflow testing

Quick Start

python3 scripts/voiceover.py check                        # Offline check: Python version, Key set, service URL
export MIRAGEA_API_KEY=your_key                            # Generate at https://miragea.ai/workspace/api, keep in environment variable
python3 scripts/miragea_audio.py capabilities             # Current available generation modes, tone tags, text length limits, and pricing (subject to real-time returns)
python3 scripts/miragea_audio.py actors --limit 20        # Existing voices in your account and their actor_id (add --offset for pagination when has_more is true)

• Default API root is https://miragea.ai/v1; set MIRAGEA_BASE_URL (including /v1) only for a private deployment.
• Do not paste your Key into chat or save it in files. Without a Key, use --dry-run first.

python3 scripts/voiceover.py script examples/movie-recap.md -o out --dry-run      # Offline dry run of the example
python3 scripts/voiceover.py script script.md -o out --preset koubo                 # Full script -> out/voiceover.wav + voiceover.srt + manifest.json
python3 scripts/voiceover.py script script.md -o out --gap 0.3 --max-chars 16 --speaker-labels --model standard
python3 scripts/miragea_audio.py generate --text 'Listen to one sentence' --actor-id ACTOR_ID --wait --output test1.mp3
python3 scripts/miragea_audio.py recognize --file final-video.mp4 --wait --output final-video.srt

• --gap: seconds of pause between sentences (default 0.25). --max-chars: maximum characters per subtitle line (default 18, recommended 12–16 for vertical video, 40 for English).
• --preset sets both sentence pause and subtitle character count at once; you can still override with --gap / --max-chars.

What You Say -> Agent Executes

You Say	Agent Executes
"Dub this on-camera script into audio"	Write script.md (single speaker) -> script script.md -o out --preset koubo
"Movie commentary dub with narrator plus two characters"	Define narrator/character voices in front matter -> script ... --preset jieshuo
"Use my voice for dubbing"	Use actors to find your cloned actor_id; or provide a server-accessible reference audio URL and the original words -> ref= + ref_text=
"Make the tone happier"	Check capabilities for the selected mode's tags -> add tags=tag_name to the speaker
"Just read one sentence for preview"	miragea_audio.py generate --text "..." --actor-id ID --wait --output test1.mp3
"The fifth sentence is wrong"	Edit only that line and re-run the same command; the rest reuse out/lines/
"Generate subtitles for this video"	miragea_audio.py recognize --file video.mp4 --wait --output video.srt
"Don't spend money yet, test the flow"	script ... --dry-run

General Script-Writing Methodology for Voiceover

No matter the scenario, the quality of AI voiceover largely depends on whether the script is "written for the ear."

1. Six Universal Rules

1. One line, one sentence, one idea: Chinese sentences should not exceed about 20 characters; the dubbing sounds more natural and the subtitles fit one screen.
2. Write it as it should be spoken: Write ambiguous numbers, years, model numbers, and percentages in Chinese reading form; separate abbreviations with spaces if they must be read letter by letter.
3. Replace written language with spoken language: "therefore" -> "so", "make adjustments" -> "adjust", split long modifiers into two sentences.
4. Put pauses only before and after key points: 0.3 seconds before a key point, 0.4–0.6 seconds for a transition, 0.6–1.0 seconds for a reveal/slogan.
5. One speaker, one voice; one account, one persona: Write voice and speed into the front matter; do not change them within a series.
6. Listen to one sentence first, then generate the full script: Use single-sentence generate to test the opening sentence and difficult words, then run script.

2. Pacing Comparison by Scenario

Scenario	Sentence Pause (preset)	Characters per Subtitle (preset)	Recommended Speed speed	Script Focus
On-Camera	0.15s	14	1.05–1.1	Give the conclusion in the first 3 seconds, one point per clip
Movie Commentary	0.3s	16	0.95–1.05	Suspense opening, role aliases, pause before the reveal
E-Commerce	0.1s	12	1.1–1.2	Pain point -> visualize the selling point -> one clear call-to-action
Digital Human	0.2s	16	0.95–1.05	Pure voice, silence at start/end, one shot per segment
Commercial	0.4s	14	0.9–1.0	Imagery short sentences, white space, brand name + slogan at the end
Audiobook	0.5s (subtitles with speaker names)	20	0.9–1.0	Casting table, separated dialogue, three-level pauses

3. Same Voice, Two Speeds

speed is set per speaker. To have the same voice "talk fast for setup, slow down for emphasis," define two roles with the same actor_id:

---
narrator: actor_id=123 speed=1.05
narrator_slow: actor_id=123 speed=0.9
---
[narrator] He thought it was all over.
[pause 0.6]
[narrator_slow] Until that phone call.

4. How to Do Foreign Language Versions

1. After finalizing the Chinese script, ask the Agent to rewrite it in colloquial style for the target market (not word-for-word translation), localizing units and greetings.
2. One script file per language; choose a voice from your account suitable for that language (actors returns each actor's language), or use a reference audio URL ref= for that language, and optionally hint the language with language=.
3. Foreign subtitles: around --max-chars 40.
4. Preview the first sentence and difficult words; have a native speaker listen before generating the full script.

5. Pre-Delivery Quality Checklist

• Previewed the first sentence: is the voice and speed correct?
• Are numbers, English, names, and polyphonic characters pronounced correctly?
• Does manifest.json total duration match the video requirement?
• Does each subtitle line fit the screen aspect ratio?
• Is the voice authorized? Has the content been marked as AI-generated upon publication?

Script Syntax

• Between opening ---, each line defines a speaker: speaker_name: actor_id=ACTOR_ID or speaker_name: ref=REFERENCE_AUDIO_URL ref_text="actual words in the reference audio"; optional speed= (0.25–4), model=expressive|standard, tags=tag1,tag2, language= (language hint for the reference audio, used for automatic reference text recognition).
• ref_text= is only used with ref=; when omitted, the server automatically recognizes the reference text, with different billing — see capabilities.
• In the body, [speaker] line or [speaker]: line; if no tag is given, the previous speaker continues.
• [pause 0.6] / [pause 0.6] inserts a pause (seconds); lines starting with #, blank lines, and <!-- comments --> are ignored.
• A sentence exceeding audio.max_text_length in capabilities will be rejected by the API; break it into shorter sentences.
• Each sentence's audio is cached in out/lines/; re-running the same command with the same sentence and settings reuses the cache. Without cache, a new task is submitted with an idempotency key, and if the same key is used, the server returns the original task without creating duplicates. To force a new version, add --fresh.

Example Script (examples/movie-recap.md)

---
narrator: actor_id=YOUR_ACTOR_ID speed=1.05
hero: ref=https://example.com/your-sample.wav ref_text="actual words in the reference audio"
---
# Movie Commentary Example (replace actor_id / ref with your own voice)
[narrator] An ordinary delivery guy picked up a phone that could see the future.
[pause 0.5]
[hero]: If tomorrow's news mentions me, what should I do today?
[narrator] What he didn't know was that every time he looked into the future, reality would quietly change a little.

Integration with Editing

# Replace original video audio track with voiceover (requires local ffmpeg)
ffmpeg -i video.mp4 -i out/voiceover.wav -map 0:v -map 1:a -c:v copy -shortest dubbed-video.mp4
# Burn subtitles into video
ffmpeg -i dubbed-video.mp4 -vf subtitles=out/voiceover.srt -c:a copy final-video.mp4

• CapCut / Premiere: Import voiceover.wav into the audio track and voiceover.srt into the subtitle track.
• Per-sentence timing: manifest.json has start / end seconds for each sentence, useful for cutting shots, adding text overlays, or aligning digital human clips.
• Digital Human Tools: voiceover.wav is clean voice, ready to be used as drive audio; convert to MP3 or a specific sample rate with ffmpeg if needed.

Comparison: AI Voiceover vs. Human Voiceover vs. Local Model

Aspect	Human Voiceover	Local Open-Source TTS Model	This Skill
GPU Required	No, but requires recording environment and voice actor availability	Usually yes (some small models can run on CPU but slower); also requires downloading model weights and setting up environment	No, only local Python standard library needed; generation happens in the cloud
Script Revision Cost	Changing one sentence often requires re-recording and retiming	Edit and re-run, but splitting, concatenation, and alignment must be done manually	Edit only that line and re-run; the rest reuse out/lines/ cache; automatic re-joining and subtitle output
Multi-Speaker	Requires multiple voice actors recording separately	Depends on the model; usually requires manual splitting and concatenation	Use [speaker] tags in the script; one command outputs the full multi-speaker audio track
Subtitle Timeline	Requires separate transcription and syncing	Usually requires running speech recognition again	Outputs SRT and per-sentence timing manifest.json simultaneously
Complex Emotional Performance	A skilled human actor on a good day is still the best	Depends on the model and parameter tuning	Consistent and reliable, suitable for information delivery and bulk production; for top-level emotional performance, human recording is still recommended

Conclusion: For daily updates, multiple versions, and frequent script revisions, this skill offers the greatest advantage in revision cost and subtitle timing. For single high-quality pieces demanding highly emotional performance, human voiceover remains valuable.

Frequently Asked Questions (FAQ)

Does AI Voiceover require a GPU or local model deployment? No. Generation happens in the Miragea cloud; locally, only Python 3.9+ standard library is needed — no pip install of any packages.

How do I use my own voice for AI Voiceover? Two ways: use a saved actor in your account (query actors for the actor_id), or provide a server-accessible reference audio URL and write the actual words spoken in the audio (ref_text=). Sample requirements and pricing are subject to capabilities and official website documentation.

Can multiple people talk in one video with AI Voiceover? Yes. Define multiple speakers at the beginning of the script, each with a voice and speed, and switch between them in the body with [speaker] tags. The script generates each sentence and joins them into one audio track while outputting subtitles.

Can I adjust the tone and emotion of AI Voiceover? Yes, using tone tags: first check capabilities for the selected mode's tags[].label, then write tags=tag_name for the speaker. You can also adjust speed with speed= and control pauses with [pause].

Are the subtitles generated by AI Voiceover time-accurate? Each sentence's start and end times come from the actual generated audio length, with in-sentence timing allocated proportionally by character count. For more precise per-word timing, run recognize on the final audio.

Do I need to regenerate the entire audio if I change one sentence in the script? No. Edit that line and re-run the same command; only the changed sentence will be regenerated. The rest are reused from out/lines/, and the audio track and subtitles are automatically re-joined and output.

Does AI Voiceover cost money? How many characters can I generate at once? It consumes your account quota. The per-sentence character limit and billing rules are subject to the real-time capabilities. --dry-run does not call the API.

Will using AI Voiceover cause shadow-banning on Douyin / WeChat Channels? No tool can guarantee traffic. Labeling AI-generated content according to platform rules is mandatory. Traffic depends more on the content itself.

Can I automatically generate subtitles for an already recorded video? Yes, miragea_audio.py recognize --file video.mp4 --wait --output video.srt; add --subtitle-format txt to get plain text.

Which AI Agents are supported? Claude Code, Codex, OpenClaw, Cursor, WorkBuddy — any Agent that can run Python scripts.

Troubleshooting

Symptom	Action
missing_api_key / MIRAGEA_API_KEY is not set	Generate a Key and export MIRAGEA_API_KEY=...; or first add --dry-run
speakers without a voice	Add actor_id= or ref= for that speaker
401	Check your Key; do not write the Key into logs
403	Check your account's API access permissions
400 / 413	Correct parameters: mode, tags, speed, per-sentence text length; or reduce file size / shorten media
409	Idempotency key used for different inputs, or subtitle pricing has changed: re-confirm and resubmit
429	Insufficient quota or rate-limited; wait or add quota
Timeout / Network Interruption	The task may still be running: simply re-run the same command (the same idempotency key retrieves the original task), or use miragea_audio.py task / wait TASK_ID to continue querying. Do not resubmit with a different key.
WAV format mismatch during concatenation	Each sentence's audio is saved in out/lines/; you can concatenate them with editing software or ffmpeg

Official client exit codes: 0 (request succeeded — may still be queued without --wait), 1 (task failed), 2 (client or API error).

Compliance

• Only clone your own voice or voices you have explicit permission to clone. Do not imitate celebrities, streamers, public figures, or any unauthorized person.
• According to the "Methods for Identifying AI-Generated Synthetic Content" and the rules of Douyin, Kuaishou, WeChat Channels, Xiaohongshu, Bilibili, and other platforms, declare or label AI-generated content upon publication.
• scripts/miragea_audio.py is the official client, shipped unchanged; scripts/voiceover.py (script assembly layer) is original, MIT licensed (see LICENSE.txt). Dubbing and subtitle capabilities are provided by the Miragea Open API.

English Summary

• AI voiceover & subtitles for agents: write a script with [speaker] tags, run one command, get voiceover.wav, timed voiceover.srt and manifest.json.
• Voices: saved voices in your account (actor_id) or voice cloning from a reference audio URL plus its transcript; generation modes, tone tags, speed 0.25–4.
• Multi-speaker scripts are rendered line by line through the official client's generate and joined locally; recognize turns existing audio/video into SRT/TXT/JSON.
• Limits and pricing come live from capabilities. Python 3.9+ standard library, no GPU, --dry-run offline. Service: Miragea API (https://miragea.ai/v1).

Keywords: AI voiceover, video dubbing, voice cloning, subtitle generation, text-to-speech, speech synthesis, voice cloning, multi-speaker dubbing, short video dubbing, on-camera dubbing, movie commentary dubbing, novel promotion dubbing, commercial dubbing, English dubbing, AI narration, SRT, TTS, voiceover, voice cloning, text to speech, subtitles