Agent-Video-Driver
Does not rely on any asset library or editing software. All visuals and music are computed by code, then packaged with FFmpeg.
This skill defines only methods and discipline, not a fixed style. See references/styles.md for the style library.
1. Two Confirmation Gates (Neither May Be Skipped)
Gate 1: Startup Confirmation Gate
Upon receiving a video task, first report the costs to the user and obtain explicit consent before starting any rendering:
| Cost to Disclose | Typical Magnitude |
|---|---|
| Token consumption | ~50k-100k output tokens for a single 30s clip; higher for 90s+ segmented clips; long-session cache hits can accumulate to millions |
| Intermediate frame images | ~900 PNGs for 30s@30fps (~200MB at 1080p); >3000 for 90s+ |
| Render time | ~0.08-0.12 sec/frame at 1080p@30fps; ~5 minutes for 3000 frames on a single process |
| Machine load | Full rendering and final encoding will saturate the CPU for extended periods, causing noticeable slowdown on development machines; running within limits can mitigate |
| Session load | One session handles only one video task; interleaving other tasks will pollute context and slow every step |
Also confirm four things: duration and resolution, confirmation mode (step-by-step or one-time), whether author credit is required (default: no), and deliverable file name. Step-by-step confirmation is suitable for topics where the user needs a say in facts and aesthetics; one-time confirmation is suitable for objectively judged topics. In both modes, the "preview the first 10 seconds" checkpoint is not omitted. Before user confirmation, only perform environment probing and storyboard drafting; do not render.
Gate 2: Assembly Confirmation Gate (Confirm Separately Before Each Heavy Operation)
Consent from Gate 1 does not cover this step. Full rendering and final encoding are the only two operations that saturate the CPU for extended periods. Each must be confirmed again:
- First, probe local resources and obtain recommended limits and risk warnings:
python scripts/resources.py --for render --frames <frame_count>
python scripts/resources.py --for encode --frames <frame_count>
- Report the core count, current load, available memory, target disk free space, recommended thread count and encoding preset, and whether intervention is advised, all to the user.
- Only proceed after user explicit consent; if not consented, only run probing, no encoding or full rendering.
- When proceeding, always run within the probed limits, never at full core.
scripts/assemble.ps1 incorporates this gate into the script itself: without -ConfirmAssembly, it only prints the confirmation gate and exits with exit code 3, meaning "waiting for user secondary confirmation."
-ProbeOnly only probes without encoding. If probing detects insufficient space or high load, the script will refuse to start; to force start, explicitly add -Force.
2. Eight-Stage Workflow
0 Probe Environment -> 1 Brief & Prompting -> 2 Content & Copy -> 3 Style & Storyboard
-> 4 Music First -> 5 Segment Rendering -> 6 Encoding & Packaging -> 7 Objective QA
Details and deliverables for each stage can be found in references/workflow.md.
Each stage must end with a checkable artifact; do not write everything through and then run.
Prompts for stages 1-3 should be filled using the seven modules from references/prompt-scaffolding.md, then presented to the user along with outputs.
Before starting stages 5 and 6, each goes through the Assembly Confirmation Gate (see Section 1, Gate 2).
3. Directory and File Conventions
<work_dir>/
├── .skill/<skill_name>/ # Skill source, one directory per skill bundle
├── prompts/ # brief.md and stage prompts
├── script.md # Screen script table, input for timing.py
├── storyboard.md # Storyboard and startup confirmation gate
├── credits.md # Attribution and licenses for external assets
├── temp/ # All intermediate artifacts, frame sequences, plan.json, test renders, verification images
├── <output_dir>/ # Only final deliverables and directly reusable source scripts
└── audio/ # Music WAV (can be merged into output dir)
- Intermediate frames must be placed under
temp/and cleaned before delivery; do not leave 3000 PNGs in the root directory. - Prefer relative paths; do not hardcode absolute paths in scripts.
- Use one naming prefix per video, with frame directories matching the output file name, to avoid overwriting between tasks.
- More conventions can be found in
references/conventions.md.
4. Style Not Fixed, But Music Defaults to Orchestral
This skill provides an engine, not a template. Before starting, first choose a style, then select color and timbre, and only then write the storyboard.
references/styles.md provides eight ready-to-use styles with hex color palettes, fonts, motion grammar, music timbres, and suited themes:
Deep Space Neon / Cold Documentary / Warm Paper / Minimal White / Retro Terminal / Academic Chart / Hand-drawn Doodle / Cinematic Film.
Music defaults to the orchestral timbre library from scripts/orchestra.py. The reason is that it satisfies two hard requirements simultaneously:
- Not harsh: Timbre relies on harmonic stacking rather than broadband noise, with measured harsh zone (2-6kHz) occupancy at 1.2%, spectral centroid at 418Hz.
- Not monotonous: Ten-segment orchestration table plus motivic variations plus dynamic arcs, with measured RMS fluctuation of 2.80x over each 10 seconds.
Switch to keyboard and electronic timbres from scripts/music.py only when a lighter, more modern feel is needed.
Regardless of which is used, none of these three situations are allowed: using a single configuration throughout the entire piece, high-frequency content accounting for more than half, or a continuous voice covering the beat points.
5. Script Inventory
| Script | Role |
|---|---|
scripts/check_env.py | Probes ffmpeg, Python libraries, fonts, disk, sandbox restrictions, and rendering route availability |
scripts/resources.py | Resource probing and recommended limits before heavy operations: core count, load, memory, disk, recommended thread count and encoding preset |
scripts/check_text.py | Glyph health check: bitmap comparison to detect characters that will become tofu boxes |
scripts/beats.py | Reverse-compute BPM, beat points, bar lines, and per-bar loudness changes from audio |
scripts/timing.py | Align the screen script table to the beat grid, outputting plan.json and a timeline for storyboarding |
scripts/canvas.py | Rendering engine: supersampled canvas, easing, glow, text, common UI components, render driver |
scripts/dsp.py | Shared DSP primitives: filtering, reverb, bus compression, warm mastering, WAV writing |
scripts/orchestra.py | Orchestral music engine (recommended default) |
scripts/music.py | Keyboard and electronic music engine (alternative) |
scripts/sfx.py | Sound effect track mixing: mixes user-provided sound effects into one track according to a cue sheet |
scripts/check_audio.py | Objective music check: sync, frequency bands, pulse, monotonicity in one run |
scripts/preview.py | AI-blind visual acceptance: downsamples frames to colored character art, supports before/after frame comparison |
scripts/qa.py | Samples frames per screen from plan.json, compares anchor frames with adjacent frames, assembles overview image |
scripts/tokens.py | Counts token usage and cost for this session, for use in end credits |
scripts/assemble.ps1 | Encoding, mixing, loudness normalization, quality check |
templates/scene_module.py | Scene module skeleton: per-beat triggering, segment scheduling, interval rendering CLI |
templates/storyboard.md | Storyboard and confirmation gate template |
templates/brief.md | Brief template, the single source of background for the entire piece |
templates/stage-prompt.md | Stage prompt template, seven modules |
templates/screen-script.md | Screen script table template, input for timing.py |
Rendering defaults to Python + Pillow for per-frame rendering. The browser rendering route is unavailable under restricted sandboxes; for reasons and workarounds, see references/pitfalls.md.
6. Content Bottom Lines (Must Hold, Four Non-Negotiables)
- Facts have sources: Every year, name, number, and quote must be traceable to a reliable source, with the link recorded in the working directory; untraceable items must be directly removed, estimates must be labeled, and secondhand claims must state whose claim it is.
- Assets are usable: When incorporating external images, fonts, sound effects, or pre-existing music, verify licenses item by item, and write attributions in
credits.md; list unused candidate assets as well, with reasons. - No real human faces: Do not use AI to generate real human faces; when AI-generated images are used, mark them as "AI-generated content" per platform requirements.
- Visuals depend only on time: All randomness must be seeded; each frame must render identically every time.
For judgment rules, reporting format, and pre-release checklist, see references/content-and-rights.md.
7. Fourteen Engineering Hard Rules (All Learned the Hard Way)
- Create music before visuals; visual cut points must equal music cut points, never the other way around.
- All entrance animations must be hardcoded in beats, never approximated in seconds; viewers can spot a one-frame sync miss.
- Unit coordinates and pixel coordinates must be explicitly separated; mixing them will collapse the entire scene to the top-left corner.
- Use interval mapping for segment scheduling; after writing, always print the start and end frames for each segment to verify; missing one segment causes whole-piece misalignment.
- Frame sequences must not be written to the root directory, nor left on disk overnight; delete before delivery.
- For parallel rendering across segments, open independent processes each writing their own files; do not use shared queues; sandboxes will block named pipes.
- When mixing, first check frequency band distribution; high-frequency occupancy over half is harsh; low-frequency occupancy over half is muddy.
- The time constant of bus compression must not be shorter than 0.3 seconds, otherwise the compressor will flatten the beat points themselves.
- Any voice that is not shorter than a bar and appears every bar will become a sound wall, masking beat points.
- Text must pass three encoding layers; the rendering engine already does per-character font fallback, but characters beyond the fallback chain will still be silently rendered as tofu boxes, so after each rendering segment, check the glyph audit report; if glyphs are missing, rewrite the text or change the font.
- After encoding, PSNR must be measured; below 45dB indicates insufficient bitrate.
- When AI-blind, do not deliver by imagination; use character art to verify composition frame by frame, then back up with objective metrics.
- Only produce one video per session; start a new session for a second video, otherwise cache and context will become unmanageable.
- Before full rendering and final encoding, probe resources and run within limits, never at full core, and each of these two steps requires user re-confirmation.
For the complete pitfall table and reproduction conditions, see references/pitfalls.md; for text encoding, see references/text-and-encoding.md.
8. Acceptance Criteria (All Must Be Green Before Delivery)
| Item | Criterion |
|---|---|
| Duration | Accurate to the frame; a 30s clip must be exactly 30.000s |
| Beat Sync | Per-frame difference peaks fall within ±1 frame of bar lines, and the peak difference of frames before/after anchor screens falls on the cut frame |
| Loudness | Integrated loudness -16 to -14 LUFS, true peak no higher than -1.0 dBFS |
| Quality | PSNR between encoded output and source frames > 45dB |
| Composition | At least 2 frames from each segment checked via character art; at least 1 frame per screen in the overview image |
| File Size | 1080p every 30s kept within 15MB (CRF 18-20) |
| Glyph | No items in the rendering audit report showing "no glyph found in fallback chain" |
| Content | Every line of text on screen can be traced to a source in the verified content |
For methods, see references/verification.md.
9. Attribution Strategy
- Video deliverables default to no author information; the workflow asks once at the Startup Confirmation Gate; add only if requested by the user.
- If the user requests attribution, it appears only in the end credits, and provides a replaceable constant location.
- Skill author: JularDepick. Version: see frontmatter metadata.
10. Reference Document Index (In Recommended Reading Order)
| Order | Document | Content | When to Read |
|---|---|---|---|
| 1 | references/workflow.md | Eight-stage workflow, stage deliverables, segment experience values | First document after receiving a task |
| 2 | references/prompt-scaffolding.md | The seven modules for brief and stage prompts, role division, sub-agent handoff | When writing prompts to start work |
| 3 | references/narrative.md | Opening grammar, per-screen word count and reading speed, narrative flow and pacing | When writing copy |
| 4 | references/content-and-rights.md | Fact source determination, asset licensing, attribution, pre-release checklist | When researching materials and selecting images |
| 5 | references/styles.md | Eight styles and corresponding music, four judgment methods for style selection | When deciding on a style |
| 6 | references/beat-sync.md | Three-level alignment, per-segment event writing, beat sync verification | Before writing storyboard |
| 7 | references/audio-engine.md | Two music engines, timbre libraries, arrangement skeletons, objective criteria | When composing music |
| 8 | references/visual-engine.md | Rendering engine API, common components, graphic recipes, performance | When writing scenes |
| 9 | references/text-and-encoding.md | Three encoding layers and glyph health check | Before writing any on-screen text |
| 10 | references/verification.md | Various acceptance methods and delivery self-check checklist | Before delivery |
| 11 | references/pitfalls.md | Pitfall phenomena, causes, and workarounds | When encountering problems or reviewing |
| 12 | references/conventions.md | Directory, naming, code, documentation, collaboration conventions | When setting up project structure |
Sorting is only borne by this table; file names do not include sequence numbers.
When adding a new reference document, simply insert a row in the table; no need to rename any existing files.
New scripts go into scripts/, new skeletons into templates/, and add one row each to this table and the script inventory.