raymond9703/AgentMotion

AgentMotion — Unified Code-Based Video Generation Framework

Turn a "one-liner" into an MP4 rendered entirely by code. This skill is the orchestration layer: it selects the backend, picks the style, runs the workflow, and performs quality checks; specific drawing knowledge lives in references/ and styles/.

Core Invariants (Shared by All Backends)

Every frame is a pure function of its frame number: frame = f(t). No Math.random() (use seeded random), no Date.now(), no rAF cumulative state. This is the prerequisite for deterministic rendering, parallel frame generation, and frame-by-frame self-verification. Violating it will cause renders and previews to diverge.

Step 1: Choose Backend (Decision Tree)

Scenario	Backend	Reference
Short clips (≤60s), zero external assets, illustration/hand-drawn/stylized	Single-file HTML Canvas	references/backends.md §1
Data-driven, batch personalization (one video per data row), subtitle/narration burning, 3D/chart component ecosystem	React + Remotion	references/backends.md §2
Interactive animations played directly in a webpage (no MP4 needed)	Single-file HTML, deliver HTML instead of rendering	references/backends.md §3

When in doubt, choose Canvas single-file: minimal dependencies, easiest for the agent to get right in one pass.

Step 2: Choose Style

Consult styles/recipes.md (15 commercial-grade styles: palettes / core techniques / motion principles / pitfalls). If the user can't specify a style, let them pick or recommend based on the theme; by default, do not reuse the same look and music for two different videos.

Step 3: Standard Workflow (Don't Stop Midway to Ask)

1. Parse Brief: Infer video type from user input (story/explainer/reel/loop/data video), duration (default 15–20s), aspect ratio (default 1920×1080; for data-sharing videos use 1080×1920). Subtitle language follows the user's input language.
2. Storyboard: First determine BPM and segment data (SECTIONS/SHOTS/EVENTS). Both visuals and music read from the same timing data, ensuring natural audio-visual alignment (references/storyboard.md).
3. Scaffold: Copy from templates/starter.html (Canvas backend), replace palette and world drawing. Maintain the contract: the page exposes window.C2V = { fps, frames, width, height, draw(frame) }, with draw being a pure function.
4. Self-Check Then Render (order cannot be reversed):

node scripts/render.mjs film.html sheet.jpg --sheet 1          # 1 fps contact sheet, stepping off cuts
node scripts/render.mjs film.html still.png --stills 0,90,300  # stills at specific timestamps
node scripts/env-check.mjs                                      # environment check
node scripts/render.mjs film.html out.mp4 --audio music.mp3     # full render (optional audio)
node scripts/probe.mjs out.mp4 1920x1080 30                     # assert encoding parameters

5. Two-Round Visual Inspection (references/qc.md): ① Storyboard compliance — capture frames at each shot's action beat and verify against the storyboard rows; ② Director's pass — review the contact sheet, evaluate pacing/shot variety/dead frames/whether every action has sound. At least two rounds; re-render only the changed frames.
6. Deliver: MP4 + HTML (HTML playable directly in a browser). Include one or two sentences at the end offering change options: swap style, change music, adjust duration.

For Remotion backend, rendering and validation use npx remotion still → visual inspection → npx remotion render (see references/backends.md §2).

Anti-Failure Rules (Most Common Causes)

• Pure function. Any state accumulation will cause parallel rendering to misalign frames.
• Hand-drawn lines should "boil" not "wiggle": wobble seed changes every 4 frames (≈7.5 times/s); changing every frame reads as noise.
• Use ellipses and rounded rectangles for bodies; too many polygons read like street signs.
• Draw order explicitly back-to-front; most "face blocked" issues are drawing order problems.
• Text never collides: reserve space (claims) for text blocks and key objects; check layout frame-by-frame before delivery.
• Cut points and action points land on beats, and loudness jumps ≤+6dB, otherwise they read as shock (references/audio.md).

File Map

File	Contents
scripts/render.mjs	Deterministic rendering core: MP4 / stills / contact sheet, multi-worker parallel
scripts/env-check.mjs / probe.mjs	Environment check / MP4 encoding assertion
templates/starter.html	frame=f(t) scaffolding: contract, seeded random, easing, camera, beats
styles/recipes.md	15 style code-level recipes
references/*.md	Workflow / storyboard / techniques / quality control / music / backend selection
examples/hello-drop.html	Immediately runnable minimal example

Dependencies

Node 18+, ffmpeg (accessible via PATH), Chrome or Edge; npm i playwright-core (install in your project directory, not the skill directory).