mathruffian-dot/claude-motion-explainer

Motion explainer — From a sentence to an explainer animation

This skill chains "web research → script → Chinese narration → frame-by-frame animation → subtitles → color scheme → self-review → MP4 output" into a fixed pipeline. It stands on the shoulders of three things:

• HyperFrames (HeyGen open-source, HTML + GSAP to MP4) and its official skills, especially the /hyperframes entry point and the /faceless-explainer workflow. Follow the official approach.
• This repo's motion-design (taste rules: easing, rhythm, pacing, Chinese typography) and review-loop (contact sheet self-review). Quality is judged by these two.
• The tools in this folder scripts/: they fill in what the official workflow misses for Chinese narration, subtitles, font consistency, and three-color theming.

references/lessons.md contains hard-learned lessons from real projects—read it before starting, every point has actually happened in the first video.

0. Environment check (first time use)

• Run python scripts/hw_check.py --project <project> first: It detects CPU, memory, GPU, and disk, and recommends which narration engine to use, how many --workers to set, and whether to dispatch sub-agents in batches. Explain the recommendations to the user as described in references/hardware.md; these are estimates, not benchmarks.
• Node.js 22+, ffmpeg/ffprobe, Python 3 + Pillow + fonttools (pip install fonttools; if brotli is installed, fonts output as woff2, otherwise as ttf).
• HyperFrames CLI: npm install -g hyperframes, then run hyperframes doctor (Docker, Whisper, Kokoro, MusicGen are optional).
• Official skills: hyperframes skills update, and for this type of video also hyperframes skills update faceless-explainer.
• Anonymous usage statistics are enabled by default; ask the user if they want hyperframes telemetry disable.
• TTS (choose one, ask the user):
  • edge-tts (default): pip install edge-tts, no GPU needed, requires internet during generation. Taiwan accents available: zh-TW-HsiaoChenNeural (female), zh-TW-HsiaoYuNeural (female), zh-TW-YunJheNeural (male).
  • Breeze TTS 2 (advanced): Local NVIDIA GPU (about 7.7 GB VRAM, recommended 12 GB+; model file about 7 GB). Can design voices via text descriptions, then clone voices; offers the finest control over accent and emotion.
  • Other engines are also fine: as long as they can output assets/voice/raw/NN.wav for each sentence. For English videos, HyperFrames' built-in Kokoro/HeyGen can be used.

1. Research (when web search is available)

• Search for the latest information on the topic from at least two independent sources. Note any mismatched numbers and use the one consistent across sources.
• Categorize each number into three types, to be labeled on screen: Official Claim / Measured or Reported / Own Estimate.
• Perform simple physics or math estimates yourself—this makes the video educational and valuable. Always compare estimates with real-world data (e.g., "normal cars at most 1.2 g" was contradicted by a production car doing 0–100 in 1.8 seconds; update to 1.5 g).
• Do not use brand photos or logos; use self-drawn illustrations and label them "Illustration."

2. Requirements interview (follow /hyperframes intent layer, ask all at once)

Must confirm: playback platform (determines aspect ratio: classroom/YouTube → 16:9), length (recommended 60–90 seconds, narration length determines actual duration), whether narration is needed (recommended), whether to review storyboards first, and automation vs. interactive tweaking.

Ask for three additional colors (new to this skill), like a slide template:

• Background color, Main color (text and line art), Accent color (only for answers, conclusions, key numbers, chapter cards).
• If no preference, recommend: black background #121212, white text #F2F0EA, orange accent #FF7A1A.
• Font: Use one font throughout the video. Default is GenSeki Gothic Bold (OFL licensed, used for Chinese, English, numbers, and subtitles). If the user specifies a different font, confirm it can be used freely.
• Write colors and fonts into the project's theme.json (format as in templates/theme.example.json with bg/fg/accent/font).

Compile into a requirements summary with two sections: "User's Answers" and "My Inferences". Only proceed after user confirmation.

3. Project setup and storyboard

1. hyperframes init videos/<kebab-name> --non-interactive --example=blank --skill=faceless-explainer, then write BRIEF.md, then follow /faceless-explainer Steps 0–2 (capture/extracted/, choose a frame preset to generate frame.md).
2. When selecting a preset, prioritize those with clear line art (e.g., cobalt-grid). It only serves as "source colors"; final theming will be overwritten by theme.py with the user's three colors.
3. Append Traditional Chinese overrides to frame.md (font, weight, letter-spacing, line-height, minimum font size for projectors), following motion-design Section 5.
4. Run bash scripts/fetch_fonts.sh <project> to download GenSeki Gothic Bold (from the official GitHub release), Latin fonts for frame workers, and Noto TC, all into assets/fonts/—do not rely on Google Fonts online loading.
5. Write STORYBOARD.md and SCRIPT.md (following the official storyboard-format / script-format). Split narration by clauses—each comma is a visual reveal timing point; write numbers in Chinese characters (e.g., "零點八一秒") so TTS reads them correctly.

4. Narration

1. Generate assets/voice/raw/NN.wav (one file per sentence, NN is the storyboard frame number):
   • edge-tts: python scripts/edge_tts_voice.py --project <p> --voice zh-TW-HsiaoChenNeural [--rate -5%]. Use the same voice for the whole video for natural consistency; --list zh-TW lists all voices.
   • Breeze TTS 2: python scripts/breeze_voice.py --project <p> --breeze-repo … --model … --voice "<voice description>" --only-ref. First design one reference voice for the user to listen to; after confirmation, remove --only-ref to generate all sentences (each clone of this reference voice). If a batch runner is available, use scripts/tts_jobs.py to generate a work list.
   • If the user hasn't installed it or is unsure: give recommendations per references/tts-setup.md, confirm step by step, and do not install without permission.
2. Let the user preview the first one or two sentences first. Let the user judge accent and emotion by ear, not by scores. If unsatisfied, change the voice or adjust speed, delete the old WAV files, and regenerate.
   • If a word is mispronounced after the whole video is done (e.g., English brand name, polyphonic character): Don't re-record the entire sentence—that would shift animation timing for that frame. Instead: generate 4–6 candidates from the same voice, same sentence (different seed or spelling, e.g., "Road-ster" or homophonic characters; subtitles still use the original text). Place them in assets/voice/fix/cand/NN_X.wav. Run python scripts/fix_voice.py --project . audition --line N --clauses <clause containing the word> to concatenate key segments into an audition file for the user to choose by ear. After selection, run apply --line N --take NN_X; it will resample the new sentence to the original length proportionally, match volume to the original, recalculate subtitle timings for that sentence, and keep the frame lengths unchanged. Then rebuild subtitles → patch_captions.py → delete .hyperframes/pre-theme/compositions/captions.html (otherwise theme.py reverts to old subtitles) → theme.py → re-render. Do not split the re-recorded sentence into clauses by pauses: the pause positions differ; splitting would cut characters in the middle, causing missing or repeated words (observed in practice). ASR can only filter obvious errors; it cannot detect tone correctness (example: "輪胎" was detected correctly by ASR but the user heard it as wrong).
3. python scripts/voice_meta.py --project <p>: Trim leading/trailing silence, measure each clause's timing, write audio_meta.json.
4. node <faceless-explainer>/scripts/audio.mjs sync-durations --audio-meta ./audio_meta.json --storyboard ./STORYBOARD.md
5. Loudness: final video should be about −14 LUFS, true peak ≤ −1 dBFS. If narration is too quiet, apply uniform gain with a limiter, preserving relative volumes between sentences.

5. Shot design (Step 4)

• In shots.json (see templates/shots.example.json), for each frame write "which clause → what appears on screen". Run python scripts/apply_shots.py --project <p> to convert to real timestamps and write back to STORYBOARD.md.
• The blueprint field must be only <id> (Adapt), <id> (Reproduce), or compose; explanations go in note.
• First search for existing components with hyperframes catalog --query; only draw custom ones if none found.
• Characters that appear repeatedly across multiple frames (car, person, instrument) should be drawn as a single shared SVG first, with fixed coordinates for all parts. All workers must copy it exactly. Otherwise, each frame's character will look different.

6. Parallel frame generation

1. node <faceless-explainer>/scripts/frame-packets.mjs --project <p> --storyboard <p>/STORYBOARD.md
2. Copy templates/dispatch_context.md to .hyperframes/frame-packets/_dispatch_context.md, fill in project values: Video direction, shared assets, timing rules, fonts.
3. Assign each frame to a sub-agent, all in parallel. The prompt reads in order: _role.md → _dispatch_context.md → that frame's packet → frame.md → shared assets.
4. When adjacent frames continue the same visual (e.g., the same scale ruler): the exact geometry reported by the finished frame must be immediately passed to the still-working frame.
5. After all frames are back: python scripts/unify_frames.py --project <p> to unify fonts and hairline positions.

7. Assembly, subtitles, theming

node <fe>/scripts/assemble-index.mjs --storyboard ./STORYBOARD.md --hyperframes .
node <fe>/scripts/captions.mjs build --storyboard ./STORYBOARD.md --audio-meta ./audio_meta.json --hyperframes . --out ./caption_groups.json
python scripts/patch_captions.py --project .     # Chinese subtitle font and weight (default bold 700)
python scripts/theme.py --project .              # Apply theme.json three colors and font (index, each frame, subtitles)
node <fe>/scripts/transitions.mjs inject --storyboard ./STORYBOARD.md --hyperframes .
node <fe>/scripts/transitions.mjs verify --storyboard ./STORYBOARD.md --index ./index.html
hyperframes lint && hyperframes check

(<fe> = the installed faceless-explainer skill folder.) Every time subtitles are rebuilt, re-run patch_captions.py then theme.py. The accents field in theme.json specifies which elements in each frame use the accent color (block/fill/text); font changes all text in each frame and subtitles to the same font and weight, overriding the frame worker's original font settings. After changing the font, text may become wider; re-run hyperframes check to detect overflow or overlap.

Font subsetting as the last step: python scripts/subset_fonts.py --project .. This keeps only the Chinese characters used in the video (CJK fonts go from 12–17 MB down to a few hundred KB) and updates each frame's @font-face to point to the subset file. Any changes to text, subtitles, or colors after this require re-running: theme.py always re-applies theming from the pre-color original files (which point to the full font), so if new characters are not in the subset file, they will fall back to system fonts.

8. Review (review-loop)

• First run hyperframes check, then render the full video at draft quality (hyperframes render --quality draft --workers <hw_check recommended value>; always use draft for review). Capture a frame 0.15 seconds before the end of each clause (to get the settled state), compile into a contact sheet using review-loop/scripts/sheet.py --video … --at …. Look at every single frame.
• Go through the checklist in references/lessons.md item by item. Fix, re-render, and re-review until one round passes without issues.
• Only then render the final version: hyperframes render --skill=faceless-explainer --quality high --output renders/video.mp4, and measure loudness again.

9. Delivery

Hand over: MP4 path, duration, one overview contact sheet, and list: which sources were used, which numbers are claims vs. estimates, any deviations from requirements (e.g., length overrun), and anything not done (e.g., no background music).