Communitygithub.com

shushuitie2017/claude-video

视频转写与翻译:外语视频 → 带中文字幕的视频 + 中文文稿。完整管线 = 下载 → 本地 Whisper 词级转写 → 翻译润色 → 烧录字幕(可选双语中大英小/水印)→ 出 Markdown。当用户说"翻译视频""翻译这个视频""配字幕""加字幕""视频转写""转文字""视频转markdown""出文稿"时使用;说"翻译/配字幕"必须执行完整管线烧出视频,不能只出文档。

claude-video 是什么?

claude-video is a Claude Code agent skill that 视频转写与翻译:外语视频 → 带中文字幕的视频 + 中文文稿。完整管线 = 下载 → 本地 Whisper 词级转写 → 翻译润色 → 烧录字幕(可选双语中大英小/水印)→ 出 Markdown。当用户说"翻译视频""翻译这个视频""配字幕""加字幕""视频转写""转文字""视频转markdown""出文稿"时使用;说"翻译/配字幕"必须执行完整管线烧出视频,不能只出文档。.

兼容平台✓Claude Code✓Codex CLI~Cursor
npx skills add https://github.com/shushuitie2017/claude-video/tree/HEAD/skills/video-translate

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

/video-translate — Video Translation & Transcription

Capability & Intent Recognition (Strictly Enforced)

  • Transcription to document: Extract subtitles or audio transcription → output Markdown document (prioritizes subtitles when available, fast and free).
  • Translate video: Transcription → translation & polishing → burn Chinese subtitles → output video file with subtitles + Markdown document.

Decision rules:

  • User says "transcribe", "to text", "output document", "to markdown" → only generate Markdown document (use the "document only" flow).
  • User says "translate", "translate video", "add subtitles", "burn subtitles" → must execute the full translation pipeline (extract audio → Whisper transcribe to SRT → translate & polish → burn subtitles onto video → simultaneously generate Markdown), cannot end after outputting only a document.
  • Input is a URL → download first, then process; local file path → process directly.

Python interpreter: All commands in this skill use python as per Windows convention. On macOS/Linux, replace with python3.

Resolve SKILL_DIR (Do this before executing any command)

Set SKILL_DIR to the absolute path of the directory containing this SKILL.md file you just Read. Scripts are always in SKILL_DIR/scripts/. First, guard:

SKILL_DIR="<absolute path of the directory containing the SKILL.md you Read>"
if [ ! -f "$SKILL_DIR/scripts/transcribe_srt.py" ]; then
  echo "ERROR: scripts not found under SKILL_DIR=$SKILL_DIR" >&2
  exit 1
fi

Step -1: Resolve Output Root (⛔ Mandatory, Before Everything)

Before executing any command or writing any file, you must first read the configuration to obtain output_dir as <output root>:

python -c "import sys; sys.path.insert(0, r'${SKILL_DIR}/scripts'); import config; c = config.load_translate_config(); config.ensure_output_root(c['output_dir']); print(c['output_dir'])"
  • Configuration file is at ~/.config/video-skills/translate.json (first run python "${SKILL_DIR}/scripts/setup.py" to generate a template).
  • output_dir must be an absolute path or start with ~. If missing/invalid, the above command will error — stop immediately and prompt the user to edit the file and provide an absolute path; do not automatically fill in a default and continue.
  • All intermediate artifacts go to <output root>/tmp/, final artifacts (video, Markdown) go to <output root>/data/.
  • ⛔ All <output root>/... placeholders must be replaced with the real absolute path before executing the command; do not pass the placeholder into the command (it would write to the current directory, appearing as "random saving").
  • Only when the user explicitly provides an "absolute path output location" and confirms it, are you allowed to write to that user-specified path.

Also run an environment pre-check once (shared with /watch, silent means pass):

python "${SKILL_DIR}/scripts/setup.py" --check

If it fails, run python "${SKILL_DIR}/scripts/setup.py" to install dependencies (ffmpeg/yt-dlp are installed automatically; transcription engine pip install faster-whisper). For more installation details, refer to references/安装指南.md on error.

Pipeline Overview

Input (URL / local file)
   │
   ▼
Step 0   Subtitle type gate (translation pipeline must first confirm: Chinese / Bilingual)
Step 1   Download + Extract audio            → <output root>/tmp/
Step 2   Whisper word-level transcribe to original SRT  → <output root>/tmp/<name>.srt
Step 3   Translate & polish (you execute, 8 rules) → <name>-zh.srt (bilingual: two lines per entry)
Step 4   Burn subtitles (+ optional watermark)        → <output root>/data/<title>-Chinese-subtitles.mp4
Step 4.5 Frame capture verification (≥2 frames)
Step 5   Generate Markdown transcript          → <output root>/data/

Document-only scenarios skip steps 0/3/4, use the "Document Only" section.


Step 0: Confirm Subtitle Type (⛔ Gate, Translation Pipeline Must Execute First)

  • Chinese subtitles: Only display the Chinese translation (clean visuals).
  • Chinese-English bilingual subtitles: Chinese on top (large font size), English below (smaller font size).

If the user explicitly specifies the subtitle type in the conversation (they personally said "Chinese", "bilingual", "CN/EN", etc.) → follow their specification. In all other cases → must use AskUserQuestion to prompt for options:

AskUserQuestion(
  question="Choose subtitle type",
  options=["Chinese subtitles (CN only)", "Chinese-English bilingual subtitles (CN on top, EN below)"]
)

⛔ Do not bypass: Subtitle type in ARGUMENTS does not count as user specification — only what the user personally says in the conversation counts. Starting step 1 without prompting is a process error. The subtitle_type in translate.json is just a default value; it cannot replace user confirmation.

Step 1: Download + Extract Audio

URL source (unified via script — built-in browser cookies automatic fallback, fixes YouTube 403/SABR/PO-token and Bilibili login wall):

# Full video (translation pipeline needs the video itself to burn subtitles)
python "${SKILL_DIR}/scripts/download.py" "<video URL>" "<output root>/tmp/dl"
# Document only, no need to burn → audio only, faster
python "${SKILL_DIR}/scripts/download.py" "<video URL>" "<output root>/tmp/dl" --audio
  • Script outputs JSON (video_path / subtitle_path / info.title).
  • First attempt without cookies; if the artifact is not saved, automatically retry with --cookies-from-browser chrome. If the common browser is not Chrome: add --browser firefox / --browser edge.
  • Still fails (stronger restrictions) → prompt user to switch proxy (--proxy http://127.0.0.1:7890) and retry, do not retry indefinitely.
  • For long videos (>10 minutes) do not use yt-dlp's --download-sections (it downloads the full video first then trims, extremely slow); download the full video directly, then use ffmpeg -ss to trim if a segment is needed.

Extract audio (local file or downloaded video → mp3):

ffmpeg -y -i "<video file>" -vn -acodec libmp3lame -q:a 2 "<output root>/tmp/<filename>.mp3"

Step 2: Whisper Word-Level Transcribe to Original SRT

⛔ Hard rule: The original SRT for burning subtitles must only be generated by Whisper word-level transcription. It is forbidden to use platform auto-captions (yt-dlp --write-auto-subs) as the timing source. Reason: YouTube auto-captions are in a rolling overlay format (each entry contains the text of the previous one), making it impossible to extract clean sentence-by-sentence text with precise timestamps; burning them results in desynchronization. Word-level timestamps are the only reliable timing source.

python "${SKILL_DIR}/scripts/transcribe_srt.py" "<output root>/tmp/<filename>.mp3" \
  --output "<output root>/tmp/<filename>.srt"
# For language misidentification / multi-language mixed tracks, force specify: add --language en (or zh/ja/…)
  • Default engine = local faster-whisper (word-level timestamps, zero API cost, first run automatically downloads the model from HuggingFace).
  • If faster-whisper cannot be installed → configure GROQ_API_KEY/OPENAI_API_KEY then add --engine api (API word-level fallback, shares the same sentence-splitting logic as the local route). If the API does not return word-level timestamps, the script degrades to segment-level and prints "subtitle precision degraded" on stderr — relay this to the user.
  • ⛔ Do not write transcription Python code in the conversation — the script splits by "sentence + pause" (segment-based / splits by word only if >6s / merges only if <400ms). Bypassing the script with a greedy split will cram the tail of sentence A and the head of sentence B into the same frame (a real-world pitfall that has been fixed).
  • --max-line-ms controls the maximum duration of a single entry (default 6000ms), --pause-ms controls the pause threshold for sentence splitting (default 500ms).

Step 3: Translate & Polish (You Execute — This is an LLM Task, No Script)

Read the original SRT from step 2, translate and polish each entry, and output the polished SRT to <output root>/tmp/<filename>-zh.srt. Your role: Professional subtitle copy editor, skilled in Chinese writing, colloquial restoration, and subtitle formatting standards.

8 Polishing Rules (Core)

  1. Correct ASR recognition errors: Homophones, proper noun misspellings (e.g., Claude heard as cloud, MCP heard as NCP), numbers/abbreviations/units. Use video visuals as auxiliary context.
  2. Punctuation & written style: First add punctuation to convert spoken language to written form (preserving speaker tone), but the final subtitle text must remove all punctuation (commas, periods, question marks, exclamation marks all removed); add an English space between Chinese and English (e.g., 这是 Google 的产品).
  3. Remove redundancy: Delete filler words, repetitions, interjections ("um", "you know", "like") that carry no information while ensuring semantic continuity. Better slightly redundant than losing meaning.
  4. Timestamp alignment (strict): Strictly preserve original timestamps; do not shift them arbitrarily. Never manually advance the start time (this would make subtitles appear before the speaker). Trim overly long display: If an entry spans >6 seconds but has short text, Whisper has included silence/BGM after speech into the same segment. Use ffmpeg silence detection to precisely locate the speech end point, do not use formula estimation:
    ffmpeg -i <audio> -ss <start> -to <end> -af silencedetect=n=-30dB:d=0.5 -f null - 2>&1 | grep silence_start
    
    Take the first silence_start (add back the start offset) as the trimmed end; if the entire segment has sound, do not trim. If an entry is >8 seconds with long text, it can be split into multiple entries, with timestamps reasonably distributed within the original range. It is forbidden to merge adjacent entries; only splitting is allowed; renumber continuously after splitting; disallow timestamp overlaps.
  5. Line breaks & segmentation: Core principle: split only, never merge. One original line corresponds to one translated line. If exceeding 12 characters, split at semantic breakpoints into two lines (timestamps distributed proportionally). Prefer single lines; each line ≤ 12 Chinese characters (including spaces and English). Break at pauses in tone; do not split a complete phrase across two lines. Better to have many short lines than to stack multiple lines on screen.
  6. Proper nouns: Keep names of people/places/companies/products in the original (capitalize first letter); keep technical terms in English (API, GPU, AI, ASR); do not translate well-known brands (Google, iPhone, YouTube); for obscure terms, a brief Chinese definition can be added on first occurrence.
  7. Language style: Match contemporary youth usage; translate internet slang/colloquialisms naturally and informally, avoiding translationese ("this is very exciting" → "this is really cool"). Narration uses concise written language; dialogue/lyrics retain a colloquial feel.
  8. Lyrics: Enclose with ♪ (♪ lyrics ♪), focus on mood and rhyme rather than word-for-word translation; when alternating with narration, ensure the audience can distinguish.

Output Format (Strict SRT)

1
00:00:00,000 --> 00:00:03,500
This is the first subtitle line

2
00:00:03,500 --> 00:00:07,200
This is the second subtitle line

Numbering starts from 1 and is continuous; timestamps HH:MM:SS,mmm (comma for milliseconds); text has no punctuation; English spaces between Chinese and English; blank lines between entries; blank line at file end.

Bilingual Mode (When User Selected Bilingual in Step 0)

Produce a "bilingual SRT": each entry has two lines — Chinese on top, English (original) below, timestamps exactly aligned one-to-one with the original SRT (no re-splitting, no merging; the original segmentation is followed for Chinese). Chinese lines follow the punctuation removal and terminology rules; English lines retain the original text (only correcting ASR errors):

1
00:00:19,239 --> 00:00:21,239
大家好吗
Hello everyone, how are you?

Long Videos (>10 min) Segmented Parallel Processing

When there are many subtitle entries, segment and translate in parallel: each segment of about 500 entries, execute parallel subtasks for translation, then merge and renumber after translation. After merging, perform a global check: no timestamp overlaps or gaps, continuous numbering, semantic coherence, and format compliance.

⛔ Pipeline does not pause: After polishing is complete, immediately execute step 4 (burning); do not output intermediate reports, do not wait for user confirmation. The entire pipeline only reports once after all final artifacts (video + Markdown) are complete.

Step 4: Burn Subtitles (+ Optional Watermark)

Use burn.py uniformly — do not hand-craft ffmpeg commands. The script centrally handles: Windows path escaping in filters (C:\a.srt → 'C\:/a.srt', hand-crafted is error-prone), forced audio conversion to AAC (Opus tracks from yt-dlp cause upload failures on X/WeChat/Weibo, ⛔ forbid -c:a copy), watermark fontfile parsing and timing distribution, and subtitle+watermark in one -vf for single encoding (avoiding secondary quality loss).

Chinese monolingual (SRT direct burning):

python "${SKILL_DIR}/scripts/burn.py" "<video file>" \
  --subs "<output root>/tmp/<filename>-zh.srt" \
  --output "<output root>/data/<video title>-Chinese-subtitles.mp4"

Chinese-English bilingual (convert to ASS first then burn — ⛔ must use ASS):

python "${SKILL_DIR}/scripts/bilingual_ass.py" "<output root>/tmp/<filename>-zh.srt" \
  --output "<output root>/tmp/<filename>-zh.ass" [--cn-size N] [--height video height]
python "${SKILL_DIR}/scripts/burn.py" "<video file>" \
  --subs "<output root>/tmp/<filename>-zh.ass" \
  --output "<output root>/data/<video title>-Chinese-subtitles.mp4"
  • ⛔ Why bilingual must use ASS: The subtitles filter's force_style FontSize applies uniformly to the entire entry, cannot achieve Chinese large and English small within one line. Inline {\fsN} in SRT is stripped by ffmpeg's SRT decoder (tested three font sizes produced identical md5). Only real ASS fed into libass can support \N line breaks and inline font size changes within one line. bilingual_ass.py does exactly this, do not write ASS manually.
  • Font size: If the user explicitly gives a Chinese font size in the conversation (e.g., "font size 24") → pass --cn-size 24, English is automatically calculated as Chinese/1.7; if not given → the script selects a default based on resolution. Chinese:English ≈ 1.7 is a contrast ratio verified in practice.
  • ⛔ ASS FontSize/MarginV are relative to PlayResY=288, not pixels. libass automatically scales to video resolution; do not linearly scale font size by resolution (17 at 1080p multiplied by 2 for 4K would fill half the screen). At 1080p, MarginV=N ≈ N×3.75px from the bottom (MarginV=110 ≈ center of the screen, not 110px); do not adjust based on pixel intuition.

Watermark (disabled by default; enable when settings.watermark_enabled: true in translate.json or user requests):

python "${SKILL_DIR}/scripts/burn.py" "<video file>" --subs <subtitle> \
  --watermark-text "<watermark text>" --output <output>

Timings are automatically distributed by duration (<30min: 3 times for 4s / 30-60min: 5 times for 5s / >60min: once every 15 min for 10s; fixed at beginning and end), opacity 0.28, 0.5s fade in/out, top-left corner. Parameters can be overridden with --watermark-opacity/--watermark-fontsize etc.; parameters specified in the conversation take priority over configuration file.

Avoidance when original hardcoded subtitles exist at the bottom: New subtitles must be moved up to avoid covering the original subtitles:

# Single-line hardsub
python "${SKILL_DIR}/scripts/burn.py" <video> --subs <subtitle> --avoid-hardsub single ...
# Two-line or dynamically varying lines (one line in some entries, two in others) → avoid as double; Chinese being slightly higher in single-line segments is an acceptable trade-off
python "${SKILL_DIR}/scripts/burn.py" <video> --subs <subtitle> --avoid-hardsub double ...

Avoidance levels (single=MarginV 42/FontSize 19, double=MarginV 60/FontSize 19) are empirical values; if frame verification fails, fine-tune with --margin-v ±3-5 and reburn.

When unsure of the final command, first use --dry-run to print the command for review before actually running.

Step 4.5: Frame Capture Verification (⛔ Mandatory, ≥2 Frames)

After burning, you must capture frames and use Read to verify: subtitles are fully displayed, do not obscure original hardcoded subtitles, bilingual font size contrast is clear, Chinese shows no blocks (missing fonts):

ffmpeg -y -i "<output video>" -ss <timepoint1> -vframes 1 "<output root>/tmp/check1.jpg"
ffmpeg -y -i "<output video>" -ss <timepoint2> -vframes 1 "<output root>/tmp/check2.jpg"

When avoidance is needed, select one moment where the original subtitle is single-line and one where it is double-line; only consider it passed if both frames show no occlusion. Delivering after only checking one frame is a known pitfall. If there are issues, adjust --margin-v/--font-size and reburn and re-verify.

Step 5: Generate Markdown Transcript

General rules (all scenarios):

  • Read the transcribed text (SRT with sequence numbers and timestamp lines removed, or text cleaned by vtt_to_text.py), strictly forbid adding, deleting, or modifying the original text; only apply punctuation, sentence merging, and paragraph splitting. Must cover all content, no omissions.
  • Chinese content: Merge short sentences, standardize punctuation, split into paragraphs by semantics → output 1 file <output root>/data/<video title>-Chinese.md. ⛔ Chinese content must not use translation rules.
  • Non-Chinese content: First generate <output root>/data/<video title>-Original.md in the original language (also no additions/deletions/changes), then read references/深度文稿规则.md and create <output root>/data/<video title>-Chinese.md according to its rules (in-depth article style, not sentence-by-sentence literal translation).
  • Front matter only contains title, source_url.

Document-Only Scenario (User Only Wants Transcription/Transcript, No Subtitles Burned)

  1. If the video has subtitles, prioritize capturing them (free and fastest; ⛔ only for document output — subtitle burning forbids this route, see step 2 hard rule):
    python "${SKILL_DIR}/scripts/download.py" "<video URL>" "<output root>/tmp/dl" --captions-only
    python "${SKILL_DIR}/scripts/vtt_to_text.py" "<output root>/tmp/dl/<subtitle>.vtt" --output "<output root>/tmp/cleaned.txt"
    
    The cleaned plain text saves about half the tokens. Read cleaned.txt and output the document according to step 5.
  2. No subtitles → download audio (--audio) → transcribe with transcribe_srt.py → remove sequence numbers and timestamps from SRT to get plain text → output document according to step 5.

Temporary Files & Cleanup

  • Intermediate artifacts (audio, SRT, ASS, captured frames) are all in <output root>/tmp/; they can be cleaned after processing; final artifacts in <output root>/data/ are unaffected.
  • The transcribed SRT can be kept for debugging; users can manually clear tmp/ at any time.

Error Handling

SymptomHandling
Download failure (403/restrictions/login wall)Script has already retried with cookies automatically; if still failing, use --browser firefox/edge, or suggest user configure a proxy; confirm if it is a geographic lock/paywalled content and say so directly
faster-whisper not installedpip install faster-whisper; if not possible, use --engine api (requires API key)
Language misidentification in transcriptionForce retry with --language en (or the appropriate language)
Chinese characters appear as blocks after burningMissing fonts — burn.py selects by platform by default (Windows: YaHei, macOS: PingFang, Linux: Noto CJK); Linux requires sudo apt install fonts-noto-cjk; can also specify with --font-name
Subtitles obscure original hardcoded subtitles`--avoid-hardsub single
Insufficient disk spaceClean <output root>/tmp/
Dependency-related errorsRead references/安装指南.md, follow the instructions to install, then return to the main flow

Dependency check principle: Directly execute the main flow; check only when encountering errors — do not waste time pre-checking each dependency one by one; the default environment is assumed to be ready (the --check in step -1 already covers the basics).

相关技能