FnSGit/video-craft-skill

video-craft

Turn any topic into a narrated video. First, select a rendering backend based on content shape, then follow the shared production pipeline.

Formerly anything2explainer (Remotion explainer videos only). This split extracts all renderer-agnostic assets into core/, isolates renderer-specific logic in backends/, and adds routing/ for backend selection.

When to Use

• The user specifies a topic and wants an explanatory/derivation/tutorial/promotional video; or provides an article to be turned into a video.
• Not applicable: reproducing an existing video, real-person voiceovers, or primarily live-action footage.

Four Confirmation Points (Must Stop and Wait for User Response)

1. Duration, Language & Backend Recommendation (asked during Phase 1 while research is dispatched): Present the decision from routing/router.md together with the duration table in one question. If the user rejects the recommendation, follow their choice.
2. Narration Script Finalization (Phase 2, before TTS): Full text + chapter divisions + word count/estimated duration. Ask "Is this script acceptable?". List chapter names and HUD section names on separate lines. For multi-chapter videos, also list the two sentences at each chapter boundary side by side. After finalization, all frame numbers change; do not modify words.
3. Voiceover (before running tts_build.py): Default TTS: Chinese: edge-tts Yunxi; English: kokoro am_liam. If the user has a preference, let them provide a custom WAV file at script/audio.wav and manually fill the timeline.
4. First 30-Second Preview (Phase 5a, before dispatching remaining groups): Ask "Is the style/font/voiceover speed/rhythm acceptable?". Making changes here costs only one group; making changes after full render costs all groups.

Pipeline (8 Phases)

Phase 0  Project Creation  → Call the corresponding backend's new_project.sh based on routing result
Phase 1  Research          → core/research-brief.md (1 agent)               [Confirmation Point 1]
Phase 2  Narration + Timeline → core/narration-guidance.md + tts_build.py [Confirmation Points 2, 3]
Phase 3  Storyboard        → Written in backend-specific format, core/render_storyboard.py
Phase 4  Primitives        → Fill in per backend (remotion: ui.tsx / manim: figures.py)
Phase 5a Preview           → Build the first group first, output 30 seconds via preview.sh [Confirmation Point 4]
Phase 5b Parallel Build    → One agent per group, using the backend's agent-build-rules.md
Phase 6  Render            → backend's render.sh → mp4 + fin_frames/
Phase 7  QC & Fix          → core/motion_check.py + frame_metrics.py + backend's qc-rules
Phase 8  Delivery          → delivery-instructions.md (record used backend and decision rationale)

Routing: First Choose Backend

Must read routing/router.md. Short version:

Content Shape	Backend	Status
Continuous formula derivation / proof / symbolic manipulation	manim	✅ V1
Concept explainer (what X is / how it works / why)	remotion	✅ V1
Real terminal tutorial (installation / config / command execution)	terminal	⏸ V2
Product promo / social short video / PR walkthrough	hyperframes	⏸ V3

Decision is not based on topic name, but on whether the narration continuously manipulates symbols:

• "What does a Fourier transform do?" → remotion (explaining intuition)
• "Deriving the Fourier transform from Fourier series" → manim (continuous derivation)

When a not-yet-implemented backend matches, the user must be informed and a fallback plan must be provided — do not silently switch backends.

Prerequisites (Must-Read for New Machines)

All Backends

# core/ Python scripts need these. Use 3.13 to create the venv (3.14 lacks wheel, risky).
uv venv --python 3.13 ~/.venvs/video-craft
uv pip install --python ~/.venvs/video-craft/bin/python numpy pillow scipy edge-tts

⚠ Without this venv, tts_build.py / motion_check.py / frame_metrics.py will not run. This is a pitfall from anything2explainer: undocumented venv dependencies caused full pipeline freezes on new machines.

remotion Backend

brew install ffmpeg          # frame extraction
# Node 22+; project auto-run `npm install` via new_project.sh

manim Backend

brew install pkg-config cairo pango     # required for pycairo compilation; manim won't install without them
brew install --cask basictex            # required for true LaTeX formula typesetting; falls back to ASCII approximation

Without basictex, it still runs (auto MANIM_TEX=text), but formulas will use ASCII approximations like a/b. Derivation videos must have basictex installed — formula typesetting quality is critical for derivations.

Directory Structure

video-craft/
├── SKILL.md                   ← this file
├── routing/
│   └── router.md              backend decision tree + override mechanism + edge cases
├── core/                      renderer-agnostic, shared by all backends
│   ├── research-brief.md          research workflow and fact-checking rules
│   ├── narration-guidance.md      13 narration writing principles
│   ├── narration-storyboard.md    narration/storyboard format and token system
│   ├── tts_build.py               TTS voiceover + timeline.json
│   ├── render_storyboard.py       storyboard token replacement
│   ├── motion_check.py            rhythm QC (frame reading, cross-backend)
│   ├── frame_metrics.py           composition QC (frame reading, cross-backend)
│   ├── sheet.py                   thumbnail grid (frame reading)
│   └── lessons.md                 pitfall records and root causes
└── backends/
    ├── remotion/              concept explainers (originally anything2explainer)
    │   ├── template/               React project
    │   ├── scripts/                render/preview/still/test_render/new_project/selfcheck
    │   ├── reference/              style-guide/motion-vocabulary/composition-and-light/
    │   │                            agent-build-rules/agent-qc-rules/prompts/math-physics
    │   └── examples/rag/           sample video (quality benchmark)
    ├── manim/                 derivation videos
    │   ├── template/               Python project (common/ + scenes/)
    │   ├── scripts/
    │   ├── reference/
    │   └── README.md               environment test results (cairo/latex/pycairo)
    ├── hyperframes/           V3 reserved
    │   └── README.md
    └── terminal/              V2 reserved (user confirmed required)
        └── README.md

Quick Reference for Core Files

Goal	File
Determine which backend to use	routing/router.md
Dispatch research agent	core/research-brief.md
Write narration	Read core/narration-guidance.md first; format in core/narration-storyboard.md
Generate voiceover + timeline	core/tts_build.py
Generate storyboard table	core/render_storyboard.py
Rhythm QC	core/motion_check.py
Composition QC	core/frame_metrics.py
Full-video overview	core/sheet.py
Create a project	backends/<backend>/scripts/new_project.sh
Dispatch build agent	backends/<backend>/reference/agent-build-rules.md
Dispatch QC agent	backends/<backend>/reference/agent-qc-rules.md

Backend-Specific Storyboard Formats

Backend	Storyboard Unit	Format
remotion	A shot (one narration sentence with one scene)	Token {S12.from-8} from backends/remotion/reference/narration-storyboard.md
manim	A derivation step	DP format from backends/manim/reference/storyboard.md

QC Thresholds Per Backend

Default thresholds in core/motion_check.py are tuned for remotion explainer videos; other backends require adjustments:

# remotion (default)
python3 core/motion_check.py --frames fin_frames

# manim derivation: relax still upper limit to 5s (allow time to read after operations), disable hold criterion (no scene-exit concept)
python3 core/motion_check.py --frames fin_frames --shots DP01:1-170 --still-max 5.0 --no-hold

# terminal tutorial (V2): longer dwell during command output, expected 8-10s

Hard Rules (Universal Across All Backends)

1. Original: All visual elements must be code-generated or real recordings; no existing video frames or clips may be used.
2. Factual Attribution: Every number, term, year, or person name on screen must have a source URL documented in the research brief.
3. Single Running Example: Narration and visuals must use one consistent example throughout.
4. No Word Changes After Finalization: Frame numbers are hard-coded; changing a single word requires full realignment.
5. Inform User About Backend: Routing decisions and fallback plans must be communicated to the user; no silent decisions.

Sample Videos and Quality Benchmark

• remotion backend: backends/remotion/examples/rag/ (4′35″, 44 sentences, 44 shots) — Before starting, review examples/rag/frames/overview_*.jpg to set a quality benchmark.
• manim backend: V1 has no sample; only a minimal end-to-end test exists. Full benchmark samples are reserved for V2.
• Negative examples: backends/remotion/examples/contrast/

Migrated Projects from anything2explainer

Projects under ~/Projects/ai-video/ like linear-function / transformer / a2e-fourier / py-course were created by the original skill and still use the old layout (template/scripts/). They continue to work — the original skill is kept as deprecated but should not be deleted.

New projects must always use this skill's new layout.