yanghhu/ai-subtitle-generator

Subtitle Generation

"Subtitle Generation" is an Agent skill that automatically recognizes local video or audio files into timed SRT subtitles (can also export TXT transcripts or JSON), used for automatic video captioning, audio-to-subtitle, and audio-to-text conversion; it can also generate subtitles synchronously when dubbing a written script, all without requiring a local GPU.

One-Sentence Definition

Subtitle Generation = local video / audio file → recognize → timed .srt (or .txt transcript, .json segments); for script dubbing use voiceover.py script, which also produces voiceover.srt. Recognition is done via the Miragea open API, no local GPU needed.

Two Ways to Generate Subtitles

What You Have	What to Use	What You Get
Already recorded video / audio	miragea_audio.py recognize --file file --wait --output subtitles.srt	SRT; --subtitle-format txt for plain text transcript, json for text, language, and time segments
A script not yet dubbed	voiceover.py script script.md -o out	Dubbed audio voiceover.wav + voiceover.srt aligned to actual audio duration + manifest.json

Audience and Scenarios

Who Uses It	Typical Scenario
Short video creators	Auto-captioning voiceover / vlog videos, then fine-tune in CapCut
Educators / Courses	Generate subtitles and transcripts for recorded lectures
Interviews / Podcasters	Convert audio to subtitles and text, organize into transcripts
Meeting recordings	Export TXT transcripts as meeting minutes drafts
Commentary channels	First transcribe segments into text for script reference
Developers / Agent workflows	JSON segments (start / end / text) for downstream processing

Core Capability Parameters

Item	Description
Output files	voiceover.wav (full dubbed audio), voiceover.srt (timed subtitles), manifest.json (per-sentence speaker, text, start/end seconds), lines/ (per-sentence audio, reusable for script revisions)
Voice source	Existing character actor_id= in your account (query with actors), or a reference audio URL ref= + the actual spoken text ref_text= (voice cloning) that the server can access directly
Generation mode	expressive (client "Default") / standard (client "More Like"); omitted uses server default, real-time value from capabilities
Tone tags	tags=tag_name, uses tags[].label from the selected mode in capabilities
Speed	speed= 0.25–4, 1 is normal speed, based on character settings
Multi-role	At script start, assign a voice for each character; this skill's script calls the official client generate per sentence, then concatenates locally
Pauses	Inline [pause 0.6] / [pause 0.6] (seconds); --gap sets default inter-sentence pause
Subtitles	Generate SRT based on actual per-sentence audio duration during dubbing (--max-chars, --speaker-labels); for existing audio/video use recognize for SRT / TXT / JSON
Scene presets	--preset koubo / jieshuo / daihuo / shuziren / guanggao / youshengshu (voiceover / commentary / livestream selling / digital human / advertisement / audiobook), adjusts only pauses and subtitle character limits
Per-sentence character limit, file size / duration limits, billing	Based on real-time capabilities response
Local dependencies	Python 3.9+ standard library, no pip install, no GPU required
Offline dry run	--dry-run: no Key needed, no API call, outputs silent placeholder audio and timeline to verify script formatting
Supported Agents	Claude Code, Codex, OpenClaw, Cursor, WorkBuddy, and other agents capable of running Python scripts
Dubbing & Subtitle Service	Miragea open API (https://miragea.ai/v1)

Quick Start

python3 scripts/voiceover.py check                        # Offline check: Python version, Key set, service URL
export MIRAGEA_API_KEY=your_key                           # Generate at https://miragea.ai/workspace/api, only in env var
python3 scripts/miragea_audio.py capabilities             # Current available generation modes, tone tags, character limit, and billing—based on real-time response
python3 scripts/miragea_audio.py actors --limit 20        # Existing characters and actor_ids in your account (add --offset for pagination when has_more is true)

• Default API root is https://miragea.ai/v1; set MIRAGEA_BASE_URL (including /v1) only for a private deployment.
• Do not paste your Key into chat or write it into files. Without a Key, first run the flow with --dry-run.

python3 scripts/miragea_audio.py capabilities                                           # First check subtitles.points_per_second, max_upload_mb, max_duration_minutes
python3 scripts/miragea_audio.py recognize --file video.mp4 --wait --output video.srt       # Auto-caption video
python3 scripts/miragea_audio.py recognize --file interview.mp3 --wait --subtitle-format txt --output interview.txt   # Audio to text
python3 scripts/miragea_audio.py recognize --file course.m4a --wait --subtitle-format json --output course.json # Text + language + time segments
python3 scripts/miragea_audio.py wait taskID --subtitle-format srt --output video.srt     # Resume after timeout, do not resubmit

• Recognition consumes credits: the script re-reads the current per-second rate as expected_rate before submission; to lock the confirmed rate, add --expected-rate value; if the rate changes, the server rejects the request—confirm before submitting.
• If the file exceeds the size limit from capabilities, it errors out locally without submission; duration limits are also based on capabilities.
• --output will not overwrite existing files; use a new filename.
• Subtitle recognition has no offline dry run; the dubbing-to-subtitle flow can be verified offline with voiceover.py script ... --dry-run.

What You Say → Agent Executes

You Say	Agent Executes
"Add subtitles to this video"	capabilities to confirm limits → recognize --file video.mp4 --wait --output video.srt → burn in with ffmpeg or import into CapCut
"Convert this recording to text"	recognize --file recording.mp3 --wait --subtitle-format txt --output recording.txt
"The video is too large to upload"	Extract only the audio track first: ffmpeg -i video.mp4 -vn -ac 1 -b:a 64k audio.mp3
"The task timed out"	wait taskID --output subtitles.srt to resume, do not resubmit
"Subtitle lines are too long"	Adjust in CapCut, or have the agent split long sentences by punctuation into multiple lines
"I haven't dubbed yet, I want audio and subtitles together"	voiceover.py script script.md -o out --preset koubo
"I need bilingual subtitles (Chinese-English)"	First recognize SRT, then have the agent translate each subtitle line while preserving timestamps, generate a second SRT

Subtitle Generation Methodology: Better Recognition, Better Subtitles

1. Pre-process Audio Before Upload

• Keep vocals clear: If background music is too loud, export a track with only vocals or reduced music using editing software before recognition.
• Upload just the audio track for speed: For large video files, use ffmpeg -i video.mp4 -vn -ac 1 -b:a 64k audio.mp3 to extract the audio track—smaller size, same timeline.
• Split long content: For content exceeding the capabilities duration limit, split into segments, recognize each, then arrange sequentially in editing software.

2. Choose the Right Output Format

Format	Use Case
srt	CapCut, Premiere, YouTube/Bilibili subtitle upload
txt	Transcript, meeting minutes draft, rewriting for WeChat official accounts
json	Programmatic processing: text, language, segments[{start,end,text}]

3. Post-Recognition Checklist

• Are proper nouns, names, brand names, and English abbreviations correct? (Batch replace in SRT)
• Are numbers and units formatted consistently?
• Is each subtitle length suitable for the screen? For vertical videos, 12–16 characters per line is recommended.
• Should filler words ("um", "like") be removed?
• Do the start and end timestamps align with the video?

4. Styling Tips for Vertical Short Video Subtitles

• Place subtitles in the lower third of the frame, avoiding platform buttons and titles.
• One subtitle per screen, no more than two lines.
• Highlight key words separately in CapCut with stylized text; SRT should contain only clean text.

5. Generate Subtitles from Script Dubbing (No Recognition Needed)

If the video's audio is generated from a script, use voiceover.py script directly: subtitle timing comes from the actual length of each sentence, saving a recognition step; --max-chars controls per-line character count, --speaker-labels adds speaker names for multi-role subtitles.

Example Script (examples/example.md: Dubbing with Synchronous Subtitles)

---
host: actor_id=YOUR_ACTOR_ID speed=1.0
---
# Dubbing with Synchronous Subtitles Example: A Vlog Narration
[host] Today I'll show you how I add subtitles to a video.
First, export the video as a file.
Second, recognize it with one command and get the SRT.
[pause 0.3]
Third, import into CapCut, or burn it directly into the video with ffmpeg.

Test run: python3 scripts/voiceover.py script examples/example.md -o out --preset koubo --dry-run

Adding Subtitles to a Video

# Soft subtitles (can be toggled in the player, requires ffmpeg installed)
ffmpeg -i video.mp4 -i video.srt -c copy -c:s mov_text video_subtitled.mp4
# Hard subtitles (burned into the video, displayed on all platforms)
ffmpeg -i video.mp4 -vf subtitles=video.srt -c:a copy final.mp4

• CapCut: Text → Import Subtitles → Select .srt, then adjust styles uniformly.
• Premiere: Import .srt as a subtitle track.

Comparison: Three Approaches to Subtitle Generation

Aspect	Manual Dubbing	Local Open-Source TTS Model	This Skill
GPU needed	No, but requires recording environment and voice actor availability	Often required (smaller models may use CPU but slower); also need to download model weights and configure environment	No, only Python standard library locally; generation occurs in the cloud
Script revision cost	Changing one sentence often requires re-recording and realignment	Rerun after changes, but segmentation, concatenation, alignment must be done manually	Only modify that line and rerun; other sentences reuse out/lines/ cache, auto-concatenate and output subtitles
Multi-role	Requires multiple voice actors to record separately	Depends on model; usually need to segment and call separately, then concatenate	Use [role] tags in the script; one command outputs the full multi-role audio track
Subtitle timeline	Requires separate transcription and timing	Usually needs another ASR run	Outputs SRT and per-sentence timing manifest.json simultaneously during dubbing
Long videos	Transcription and timing are time-consuming	Local ASR model requires GPU or longer processing time	Uploaded, cloud recognition; timeout can be resumed with task ID

Conclusion: For finished videos needing quick subtitles or text transcription, use recognize; for videos starting from a script, use script for the most efficient simultaneous subtitle generation.

Frequently Asked Questions (FAQ)

What files does subtitle generation support? Any local audio or video file can be used as the recognition source; size and duration limits are based on capabilities' subtitles.max_upload_mb and max_duration_minutes.

After auto-captioning a video, how do I add the subtitles? Use CapCut's "Import Subtitles" feature, or ffmpeg: soft subtitles with -c:s mov_text, hard subtitles with -vf subtitles=subtitles.srt.

Can audio-to-subtitle export SRT format? Yes, SRT is the default; --subtitle-format txt gives a plain text transcript, and json gives text, language, and time-coded segments.

What is the difference between audio-to-text and subtitle generation? They are the same recognition; only the export format differs: SRT includes timestamps for subtitles, TXT is plain text for transcripts.

Is subtitle generation paid? It consumes credits based on media duration and the current per-second rate, as returned in real-time by capabilities; the script submits with the current rate, and any rate change is rejected—no silent deduction at a new rate.

If subtitle generation times out, do I need to re-upload? No. Use the returned task_id to run miragea_audio.py wait taskID --output subtitles.srt to retrieve the result; if the network is interrupted, only retry with the same idempotency key.

Can subtitle generation recognize English or other languages? The recognition result includes a language field; the exact supported languages depend on the server's recognition performance; it's recommended to test with a short segment first.

Can I get bilingual Chinese-English subtitles? First, recognize SRT, then have the agent translate each line while preserving timestamps, generating a second SRT or merging into bilingual format.

Does AI subtitle recognition require a GPU? No. Recognition happens in the cloud; only Python 3.9+ standard library scripts run locally.

Which Agents does this subtitle generation skill support? Claude Code, Codex, OpenClaw, Cursor, WorkBuddy, and other agents that can run Python scripts.

Troubleshooting

Issue	Action
missing_api_key / MIRAGEA_API_KEY is not set	Generate a Key and run export MIRAGEA_API_KEY=...; or add --dry-run first
speakers without a voice	Add actor_id= or ref= for that character
401	Check your Key; do not log the Key
403	Check the account's API access permissions
400 / 413	Correct parameters: mode, tags, speed, per-sentence character limit; or reduce upload file size/shorten media
409	Idempotency key used for different inputs, or subtitle rate has changed; reconfirm before submitting
429	Insufficient credits or rate limiting; wait or add credits
Timeout / network interruption	Task may still be running: directly rerun the same command (same idempotency key retrieves the original task), or use miragea_audio.py task / wait taskID to continue polling; do not use a different key
WAV format mismatch during concatenation	Each sentence audio is saved in out/lines/; use editing software or ffmpeg to concatenate

Official client exit codes: 0 request succeeded (may still be queued without --wait), 1 task failed, 2 client or API error.

Compliance

• Only clone your own voice or that of someone who has explicitly authorized it; do not imitate celebrities, streamers, public figures, or any unauthorized person.
• Follow the "AI-generated Content Labeling Method" and platform rules (Douyin, Kuaishou, WeChat Channels, Xiaohongshu, Bilibili, etc.) when publishing; declare or label AI-generated content.
• Obtain consent before recognizing others' audio/video (interviews, meetings, courses); be mindful of storage and sharing scope for subtitles containing personal information.
• scripts/miragea_audio.py is the official client, shipped unchanged; scripts/voiceover.py (script concatenation layer) is original, MIT licensed (see LICENSE.txt). Dubbing and subtitle capabilities are provided by the Miragea open API.

English Summary (Original)

• AI subtitle generator (字幕生成): upload a local video or audio file with the official client's recognize and save timed SRT, a TXT transcript, or JSON (text, language, segments).
• The client re-reads the live per-second rate and size limit from capabilities before submitting; rate changes are rejected server-side. Resume long jobs with wait TASK_ID.
• For scripted voiceovers, voiceover.py script produces subtitles aligned to the generated audio. Python 3.9+ stdlib, no GPU. Service: Miragea API (https://miragea.ai/v1).

Keywords: subtitle generation, auto-caption video, audio to subtitle, SRT, video captioning, automatic subtitle generation, AI subtitles, audio to text, video to text, speech to text, subtitle generator, SRT subtitles, video subtitle generator, speech to text, auto captions