tzezhang1-crypto/claude-motion-explainer-video

Motion Explainer (Video Enhanced) — Animate the Principles, Use Photos as Evidence, AI Video Adds Emotion

This skill chains the process "Search the web → Write script → Chinese narration → Frame-by-frame animation → Subtitles → Color scheme → Find licensed photos & generate illustrations → Generate AI video → Self-review → Output MP4" into a fixed pipeline. Compared to the basic motion-explainer, it adds Section 7.5 (Image Layer), Section 7.6 (AI Video Layer), and Section 4 (Fixing mispronounced characters); all other steps are identical. Video footage is primarily AI-generated: news clips, promotional videos, and stock footage often carry copyrights and are not used in educational videos. It stands on the shoulders of three things:

• HyperFrames (HeyGen open source, HTML + GSAP to MP4) and its official skills, especially the /hyperframes entry point and /faceless-explainer workflow. Always follow the official implementation.
• This repo's motion-design (taste rules: easing, rhythm, pause, Chinese typography) and review-loop (contact sheet self-review). Quality is judged by these two.
• The tools in this folder scripts/: they fill gaps in the official flow for Chinese narration, subtitles, font consistency, and three-color theme.

references/lessons.md is a collection of real-world pitfalls; read it before you start — every item has actually happened in the first video.

0. Environment Check (First Time Use)

• Run python scripts/hw_check.py --project <project> first: it detects CPU, memory, GPU, and disk, and suggests which TTS engine to use, what --workers to set, and whether sub-agents should be dispatched in batches. Explain the suggestions to the user following references/hardware.md; these are estimates, not benchmarks.
• Node.js 22+, ffmpeg/ffprobe, Python 3 + Pillow + fonttools (pip install fonttools; if brotli is installed, fonts will be output as woff2, otherwise as ttf).
• HyperFrames CLI: npm install -g hyperframes, then run hyperframes doctor (Docker, Whisper, Kokoro, MusicGen are all optional).
• Official skills: hyperframes skills update. For this type of video, also run hyperframes skills update faceless-explainer.
• Anonymous usage statistics are enabled by default; ask the user if they want hyperframes telemetry disable.
• TTS (choose one, ask the user):
  • edge-tts (default): pip install edge-tts, no GPU required, needs internet connection for generation. Taiwan accents available: zh-TW-HsiaoChenNeural (female), zh-TW-HsiaoYuNeural (female), zh-TW-YunJheNeural (male).
  • Breeze TTS 2 (advanced): local NVIDIA GPU (approx. 7.7 GB VRAM, 12 GB+ recommended; model files ~7 GB). Allows voice design via text description and voice cloning, with the finest control over accent and emotion.
  • Other engines are also fine: as long as they can output assets/voice/raw/NN.wav for each line. For English videos, use HyperFrames' built-in Kokoro/HeyGen.

1. Research (When Web Search is Available)

• Search for the latest information on the topic, from at least two independent sources. Note any discrepancies in numbers and use the one consistent across sources.
• Categorize each number into three types, which must be labeled on screen: Official Claim / Measured or Reported / Own Estimate.
• Do simple physics or math estimates yourself so the audience can "calculate it"; this is the most valuable part of an educational video. Always compare estimates against reality (e.g., "most cars max out at 1.2 g" is contradicted by mass-market cars doing 0–100 km/h in 1.8 seconds, so it should be revised to 1.5 g).
• Do not use brand photos or logos; use custom illustrations and label them "Illustration."
• While researching, note down image leads: Are there any "precedents," "pioneers," or "experimental photos" that can serve as evidence (e.g., NASA historical photos, existing similar products)? These are the real photos to find in Section 7.5.

2. Requirements Interview (Following /hyperframes intent layer, ask all at once)

Must confirm: playback platform (determines aspect ratio: classroom/YouTube → 16:9), duration (recommended 60–90 seconds, actual length determined by narration), whether narration is needed (recommended), whether to review storyboard first, automatic or interactive fine-tuning.

Also ask about three colors (new in this skill), like a presentation template:

• Background color, Primary color (for text and line art), Accent color (only for answers, conclusions, key numbers, section cards).
• If no preference, recommend: black background #121212, white text #F2F0EA, orange accent #FF7A1A.
• Font: Use one font throughout the entire video. Default is GenSeki Gothic Bold (OFL license, used for Chinese, English, numbers, and subtitles). If the user specifies another font, make sure it can be freely used.
• Write colors and font into the project's theme.json (see templates/theme.example.json format for bg/fg/accent/font).

Compile into a requirements summary, divided into "User's answers" and "My inferences." Only proceed after user confirmation.

3. Project Setup and Storyboarding

1. hyperframes init videos/<kebab-name> --non-interactive --example=blank --skill=faceless-explainer, then write BRIEF.md, then follow /faceless-explainer Steps 0–2 (capture/extracted/, select a frame preset to generate frame.md).
2. When selecting a preset, prioritize those with clear line art (e.g., cobalt-grid). It is only a "source color scheme" — it will be replaced by the user's three colors in theme.py.
3. Add a Traditional Chinese override (font, weight, letter-spacing, line-height, minimum projector font size) to frame.md, following motion-design Section 5.
4. Run bash scripts/fetch_fonts.sh <project> to download GenSeki Gothic Bold (from official GitHub release), Latin fonts needed by frame workers, and Noto TC, all into assets/fonts/. Do not rely on Google Fonts online loading.
5. Write STORYBOARD.md and SCRIPT.md (following official storyboard-format / script-format). The narration must be split by "clauses" — each comma is a reveal point in the animation. Write numbers as Chinese characters (e.g., "zero point eight one seconds") so TTS pronounces them correctly.

4. Narration

1. Generate assets/voice/raw/NN.wav (one file per line, NN is the storyboard frame number):
   • edge-tts: python scripts/edge_tts_voice.py --project <p> --voice zh-TW-HsiaoChenNeural [--rate -5%]. Use the same voice for the entire video for consistency; --list zh-TW lists all voices.
   • Breeze TTS 2: python scripts/breeze_voice.py --project <p> --breeze-repo … --model … --voice "<voice description>" --only-ref. First design one reference voice, let the user listen, then remove --only-ref to generate all sentences (each sentence clones this reference voice). If a batch runner is available, use scripts/tts_jobs.py to create a job list.
   • If the user hasn't installed it or is unsure which to pick: give advice following references/tts-setup.md, confirm step by step, do not install without permission.
2. Let the user audition the first one or two lines first. Let the user's ears decide on accent and emotion, not scores. If unsatisfied, switch voice or adjust speed, delete old WAVs, and regenerate.
   • If a character is mispronounced after the whole video is done (e.g., English brand name, polyphonic character): Do not re-record the whole sentence — that would mess up the animation timing of that frame. Instead: generate 4–6 candidates from the same voice, same sentence, varying seed or spelling (e.g., "Road-ster" or homophones, keeping original text in subtitles); put them in assets/voice/fix/cand/NN_X.wav. Then run python scripts/fix_voice.py --project . audition --line N --clauses <the clause containing the word>, concatenate the key segment into an audition file for the user to choose by ear. After selection, run apply --line N --take NN_X. It will scale the entire new sentence proportionally to the original duration, align its volume to the original sentence, recalculate that sentence's subtitle timing, keeping all frame durations unchanged. Then rebuild subtitles → patch_captions.py → delete .hyperframes/pre-theme/compositions/captions.html (otherwise theme.py will revert to old subtitles) → theme.py → place_media.py → re-render. Do not split the re-recorded sentence into clauses and realign them: the pauses in the re-recording will be at different positions, causing cuts in the middle of words, leading to missing or duplicated characters (happened in testing). ASR can only filter out obvious errors; it cannot detect tone errors (example: "tire" was correct in ASR but sounded wrong to the user).
3. python scripts/voice_meta.py --project <p>: trim leading/trailing silence, measure each clause's duration, write audio_meta.json.
4. node <faceless-explainer>/scripts/audio.mjs sync-durations --audio-meta ./audio_meta.json --storyboard ./STORYBOARD.md
5. Loudness: final video should be approximately −14 LUFS, true peak ≤ −1 dBFS. If narration is too quiet, apply uniform gain with a limiter, preserving relative volume between lines.

5. Camera Design (Step 4)

• In shots.json (see templates/shots.example.json), for each frame, specify "Nth clause → what appears on screen". Run python scripts/apply_shots.py --project <p> to convert to real seconds and write back to STORYBOARD.md.
• The blueprint field can only be <id> (Adapt), <id> (Reproduce), or compose; additional explanation goes in note.
• First, use hyperframes catalog --query to find existing components; only create new ones if none exist.
• If a subject (car, person, instrument) appears in multiple frames, draw it as a shared SVG first, fixing coordinates of all parts. All workers must copy this exactly. Otherwise, the subject will look different in each frame.

6. Parallel Frame Creation

1. node <faceless-explainer>/scripts/frame-packets.mjs --project <p> --storyboard <p>/STORYBOARD.md
2. Copy templates/dispatch_context.md to .hyperframes/frame-packets/_dispatch_context.md, fill in project values: video direction, shared assets, timing rules, font.
3. Assign each frame to a sub-agent, all running in parallel. The prompt reads in order: _role.md → _dispatch_context.md → that frame's packet → frame.md → shared assets.
4. When adjacent frames continue the same scene (e.g., same scale ruler), the precise geometry reported by the finished frame must be immediately passed to the still-rendering frame.
5. After all frames are returned: python scripts/unify_frames.py --project <p> to unify font and hairline positions.

7. Assembly, Subtitles, Color Scheme

node <fe>/scripts/assemble-index.mjs --storyboard ./STORYBOARD.md --hyperframes .
node <fe>/scripts/captions.mjs build --storyboard ./STORYBOARD.md --audio-meta ./audio_meta.json --hyperframes . --out ./caption_groups.json
python scripts/patch_captions.py --project .     # Chinese subtitle font & weight (default Bold 700)
python scripts/theme.py --project .              # Apply theme.json's three colors & font (index, each frame, subtitles)
node <fe>/scripts/transitions.mjs inject --storyboard ./STORYBOARD.md --hyperframes .
node <fe>/scripts/transitions.mjs verify --storyboard ./STORYBOARD.md --index ./index.html
hyperframes lint && hyperframes check

(<fe> = the installed faceless-explainer skill directory.) Whenever subtitles are rebuilt, re-run patch_captions.py and then theme.py. In theme.json, accents specifies which elements in each frame use the accent color (block/fill/text); font replaces all text in every frame and subtitle with the same font and weight, overriding the worker's original font settings. After switching fonts, text may become wider; re-run hyperframes check to see if there are overflows or overlaps.

Only at the very end, subset the fonts: python scripts/subset_fonts.py --project .. It keeps only the Chinese characters actually used in the video (reducing CJK fonts from 12–17 MB to a few hundred KB) and updates each frame's @font-face to point to the subset file. Any subsequent changes to text, subtitles, or colors must re-run: theme.py always re-applies colors from the pre-theme originals (which point to the full font); newly added characters not in the subset file will fall back to system fonts.

7.5 Image Layer (New in This Skill)

Full rules in references/images.md. Key points:

1. Planning: 3–5 images per video. Real photos as evidence cards ("Someone really did this"), Codex-generated images as atmospheric backgrounds for opening and closing. Animation is the main body; images must not overshadow the line art explaining the principles.
2. Finding Photos: python scripts/find_images.py search "<keyword>" → get "File:…" --name <short-name> --project .. Only accept public domain, CC0, CC BY, CC BY-SA; sources are automatically written to assets/images/web/CREDITS.md. After download, visually inspect each photo; crop out passersby, other brand logos, and watermarks.
3. Generating Illustrations: Write prompts into gen_prompts.json (see templates/gen_prompts.example.json), run python scripts/codex_images.py --project .. Requires a locally logged-in Codex CLI using its built-in image_gen; no API key needed. After generation, visually inspect for any appearing text or logos. If Codex is not available, tell the user they can use another image generation tool and place the PNG into assets/images/gen/.
4. Color Treatment: In media.json, specify each image's crop, size, and mode (photos use duotone, generated images use keep); run python scripts/treat_images.py --project ..
5. Placement: After the contact sheet is finalized for each frame, find truly empty areas; in media.json's placements, write frame, mode (card/background), box, at, label, credit. To replace original line art with a photo, use hide. Then run python scripts/place_media.py --project . after theme.py.
6. On-screen labels: photos show "Photo: Author · License · Source", generated images show "AI-Generated Illustration".
7. Run hyperframes check (text contrast over photos, overlaps). Only then run subset_fonts.py, because captions and sources also contain text.

7.6 AI Video Layer (New in This Skill)

Full rules in references/ai-video.md. Key points:

1. Planning: 2–4 clips per video. Use for the opening hook, an action that is hard to capture but perfectly matches the narration (e.g., "only one jump" → car landing), and closing emotion. Scenes requiring precise numbers or force direction should still use line art.
2. First frame uses a Codex static image: First generate an image matching the theme colors per Section 7.5, then H3 animates from this image, ensuring style consistency between the video clip and the overall piece.
3. Write spec: assets/videos/gen/specs/<name>.json (example in templates/video-specs/). Each clip 4–15 seconds, subject: "No brand, no text"; action written as a single clear event, plus camera motion and sound description.
4. Generation: python scripts/ai_video.py prompt --project . (official six-prompt format) → run (requires runner in video_config.json, see templates/video_config.example.json). If no runner is available, give the user prompt.txt and the first frame, ask them to generate using ComfyUI's official MiniMax H3 R2V template, saving as assets/videos/gen/<name>/clip.mp4.
5. Inspection: python scripts/ai_video.py check --project .. Visually inspect every check-1.png: Check for deformations, extra parts, text or logos, and whether action timing matches the narration. If not passable, change seed and re-run.
6. Placement: In media.json's placements, use "video": "assets/videos/gen/<name>/clip.mp4", add duration, and optionally out (fade out the card before the video ends). Run place_media.py after theme.py, then hyperframes lint (video must have an id, cannot be placed inside timed elements) and hyperframes check.
7. On-screen label: "AI-Generated Video (H3)"; all videos are muted; only narration sound is used.

8. Review (review-loop)

• First, hyperframes check, then render the whole video at draft quality (hyperframes render --quality draft --workers <suggested by hw_check>; always use draft during review). Capture a frame 0.15 seconds before the end of each clause (showing the settled state), assemble into a contact sheet using review-loop/scripts/sheet.py --video … --at …. Visually inspect every frame.
• Go through the checklist in references/lessons.md item by item. Fix issues, re-render, re-inspect, until one pass has zero issues.
• Only after review is complete, render the final version: hyperframes render --skill=faceless-explainer --quality high --output renders/video.mp4. Then measure loudness again.

9. Delivery

Provide the MP4 path, duration, a summary contact sheet, and list: which sources were used, which numbers are claims and which are estimates, any deviations from requirements (e.g., longer/shorter video), and what wasn't done (e.g., no background music).