Communitygithub.com

document-to-video

Turn any document (engineering report, paper, manual, science material) into a vertical 3:4 short science/promo video (1080×1440, 60–150 seconds). Pure code rendering: deterministic timeline in a single HTML file → Playwright frame-by-frame capture → ffmpeg encoding. No editing software, no AI video API, fully local and zero-cost, every frame reproducible.

What is document-to-video?

This skill transforms any written document (engineering record, report, thesis, manual, or popular science material) into a vertical 3:4 short science/promo video (1080×1440, 60–150 seconds) using deterministic code. It is purely code-driven: a single HTML file defines a deterministic timeline, Playwright captures every frame, and ffmpeg encodes the final video. No video editing software or AI video generation APIs are needed—everything runs locally at zero cost, and every frame is reproducible. The user provides a script, the skill generates a voiceover (using edge-tts by default), synchronizes the timeline, creates HTML animations with a blueprint-style design system, and renders the final video with automated checks. Key platforms supported include any environment with Python 3.10+, Playwright, ffmpeg, and a web browser for visual inspection. This is useful for developers, data storytellers, and anyone who needs batchable, auditable, and reproducible video production from text, with a strong emphasis on factual accuracy, determinism, and programmatic quality assurance.

Works with~Claude Code~Codex CLI~Cursor
npx skills add chrislitality-svg/script-to-video

Ask in your favorite AI

Open a new chat with this agent skill pre-loaded.

Documentation

document-to-video · Document → Vertical Science Video

Three core principles; all decisions return to them:

  1. Deterministic Timeline: Animation is a pure function setTime(t)—the same moment always renders the same frame. This enables arbitrary jump-frame auditing, multi-process segmented screenshotting, provable loops, and changing a single constant to alter the entire video's speed.
  2. Voiceover Duration Drives Timeline: Record the voiceover first, measure each line's length, then arrange scene boundaries—audio and visuals naturally sync without ASR alignment.
  3. The User's Eyes Shouldn't Be the First Quality Gate: Programmatic audits (boundary/overlap) must reach zero before screenshots are taken. Keyframe visual inspection is the final gate for aesthetics.

Pipeline (Nine Steps, Return Upstream if Any Fails)

Document ─▶ ① Data Constitution ─▶ ② Script + Subtitle Chunks ─▶ ③ Voiceover (Measured Duration)
     ─▶ ④ Timeline Sync ─▶ ⑤ Animation anim.html ─▶ ⑥ Double Audit to Zero + Keyframe Visual Inspection
     ─▶ ⑦ Music + Mix + Master ─▶ ⑧ Frame-by-Frame Screenshot → Encode → Composite ─▶ ⑨ Seven Checks → Delivery

① Data Constitution

Read the entire document, solidify all numbers/facts/names that will appear in the video into a single list (written into shots/facts.md). From then on, any imagery, script, or subtitles must only reference this list; no improvisation is allowed.

  • Amplify verifiable numbers; leave gaps for unverifiable details (better less information than false information).
  • Each number is mentioned only once, at its emotional peak.
  • Numbers must appear with a "happening" context: drawn growth, synced counter, falling impact.

② Script and Subtitle Chunks

Write the voiceover script (shots/vo_lines.json): one sentence per line, each 4–12 seconds, 8–16 lines total.

  • The first 5 seconds determine completion: money/conflict/suspense—pick one for the first sentence (e.g., "If I gave you X to build a road—would you take it?").
  • Structure reference: Hook → Problem → Rejection → Solution → Verification → Settlement → Character → Conclusion (can be trimmed).
  • Manually specify subtitle chunks per sentence ("cues": [...], each chunk ≤18 characters), more stable than mechanical punctuation splitting.

③ Voiceover

python templates/vo_tts.py (default edge-tts, free; can swap to a voice clone model if local GPU available, see references/audio.md). Outputs each sentence as a wav file plus measured duration audio/vo/vo_dur.json.

  • Write Chinese number years digit by digit (2022 → 二零二二); decimal points → "点".
  • TTS occasionally returns empty responses: the script must retry + support resume from breakpoint.

④ Timeline Sync

python templates/sync.py: scene duration = 0.25s lead time + sentence length + tail breathing (0.5–1.9s), automatically writes TL (timeline) and CUES (subtitles) back to anim.html, and generates shots/timeline.json for mixing. Must re-run after changing voiceover, and scale the timing of VO-aligned actions within scenes based on the old-to-new duration ratio.

⑤ Animation

Copy templates/anim.html and modify scenes. The skeleton already includes: blueprint-style design system, utility functions, auditor, scan-line transition, subtitle layer, and 3 example scenes. Core pattern:

  • Each scene: defscene(id, cam=>{...; return (t,d)=>{...}}), render is a pure function.
  • Two layers of motion: primary action (drawing a line/falling) + continuous layer (dashed line moving/floating/pulsing).
  • The camera always has a gentle 1.4%–3% zoom-in—the frame is never static.
  • "Exit before entry": old elements completely exit before new ones enter, avoiding any overlap.
  • Visual specifications and motion language: Read references/design-system.md and references/motion.md.

⑥ Audit + Visual Inspection (Quality Gate, Cannot Be Skipped)

Run auditText() (text overflow) and auditOverlap() (element overlap) in the browser; must reach zero before screenshots. The auditor cannot catch "ugliness": for each scene, capture the peak content frame and visually inspect (text clipping, composition voids, style color drift, graphic shapes). Fix any significant issues immediately. For intentional design overlaps, add data-okov="1" to the element for exemption.

⑦ Audio

python templates/music.py programmatically generates a light background track (zero copyright risk); or use your own track (check copyright). python templates/mix.py: normalize VO by RMS → ducking music near vocals → keep only accented sound effects (≤2 thuds per scene, reject whoosh fills—sound effects are punctuation, not background) → custom peak envelope limiter → ffmpeg loudnorm two-pass to -14 LUFS / TP -1.5.

⑧ Rendering

python templates/shoot.py 0 <dur> a --html anim.html   # ×N parallel segments
python templates/render.py                              # Encode→last frame check→concat→composite

Playwright frame-by-frame screenshot (device_scale_factor=2, JPEG q93) → per-segment x264 crf19 → per-segment last frame vs source image check (RMS<8, catch bad segments without errors) → concat → AV composite (pan explicit stereo copy, +faststart).

⑨ Seven Checks

python templates/check.py: seamless loop (first/last frame RMS<5) · audio track exists · subtitle visibility · style color order (each frame B>G>R to prevent drift) · blank frames · size/duration · faststart. Only deliver if all PASS. Deliverables: final video mp4 + srt subtitles + release copy (title ≤15 characters / one-line intro / tags).

Environment

python 3.10+, pip install playwright numpy soundfile pillow edge-tts
playwright install chromium
ffmpeg (available in PATH)

Directory Convention (Per Project)

<project>/
├── shots/vo_lines.json    # Script + subtitle chunks (sole content source)
├── shots/facts.md         # Data constitution
├── shots/timeline.json    # Generated by sync.py, used for mixing
├── tools/web/anim.html    # Animation (copy and modify from templates/anim.html)
├── audio/vo|music|mix/    # Audio
└── out/                   # Frames, segments, final video

Red Lines

  • No fabrication: data, names, events must have a source; [Pending verification] cannot be cleared until verified before voiceover.
  • Red is used only to express "judgment" semantics (elimination/acceptance chapter), never as decoration.
  • If using external music, confirm authorization first; default to programmatically generated background music.
  • For real people/companies with suffix names and numbers, verify each item against the data constitution before delivery.

References

FileContent
references/design-system.mdVisual system: color discipline, three-tier font size, figure captions, drafting grammar
references/motion.mdMotion language list: when to use which effect, parameters
references/audio.mdAudio pipeline: TTS selection, mixing formulas, mastering targets
references/pitfalls.mdPitfall checklist (sorted by cost; check here first if issues arise)
templates/Directly runnable skeleton and pipeline scripts

Related Skills