generate-footage
The plugin's other entry point. video-use edits footage you HAVE. This skill
builds videos from footage you DON'T have yet: narration plus still keyframes,
composed and animated by HyperFrames, rendered deterministically.
Pipeline (proven in production; see docs/rca/ for why the motion step is
deterministic):
script (with v3 audio tags)
→ eleven_v3 narration (scripts/tts_v3.py)
→ transcribe (npx hyperframes transcribe narration.mp3)
→ lock scene windows to sentence boundaries (word-level timestamps)
→ STILLS FROM ANYWHERE (one keyframe per scene — see below)
→ animate-stills composition (recipes/04-animate-stills.md, build_scenes.py)
→ lint → render → motion + audio QA (scripts/motion_check.py, contact sheet)
Order matters: voice FIRST, visuals second
Generate and transcribe the narration BEFORE building any composition or collecting keyframes. The spoken sentence boundaries are the scene windows. v3 reads are non-deterministic in pacing — a 120s estimate can come out 100s or 166s — so visuals built first always get re-timed.
Voiceover — ElevenLabs v3 with audio tags
scripts/tts_v3.py — stdlib-only, no pip deps. Key from $ELEVENLABS_API_KEY
or the plugin's skills/video-use/.env.
python3 scripts/tts_v3.py --probe # verify eleven_v3 + list voices
python3 scripts/tts_v3.py --voice <id> --stability 0.0 --script SCRIPT.md --out narration.mp3
- Audio tags go in the script text:
[excited],[whispers],[softly],[dramatic tone],[warmly]. v3 performs them; v2 ignores them. - Stability:
0.0= Creative (max expressiveness),0.5= Natural,1.0= Robust. A "flat" read is fixed by lowering stability and re-tagging the script, not by louder words. - Delivery levers: CAPS on power words,
...for dramatic pauses, contrast (drop to[whispers]right before a big line). - Audition 2-3 voices on the first sentence before committing. The first voice is never the pick.
- v3 may paraphrase slightly. Transcribe and READ the result; regenerate if a key line drifted.
Stills from anywhere
A keyframe is just a PNG. The pipeline does not care where it came from.
One still per scene, named kf-NN.png, listed in the scene manifest
(recipes/04-animate-stills.md documents the schema). Sources that work:
- AI image tools — any text-to-image tool you already have (Gemini, GPT image, Midjourney, local SD, or a hosted actor — see the optional section at the end). Generate from per-scene prompts with one locked style suffix.
- Screenshots and photos — product UI captures, archive material, scans, real photography. The Ken Burns treatment was invented for this.
- HyperFrames-rendered HTML — build a styled HTML frame and snapshot it; the engine produces its own stills.
Craft rules that hold regardless of source:
- Lock ONE style in DESIGN.md and apply it to every frame. Drift across frames reads as different shots once there's motion and grading.
- Verify the file type. Some tools serve JPEGs with .png names; check
magic bytes before composing (
file kf-01.png). - QA every frame (read the image). One off-style frame is a regenerate, not a keep. Check for: text artifacts, watermarks, anachronisms, faces in close-up (drift risk), wrong palette.
- No named IP, no real living person's likeness in generated frames for anything posted publicly.
Animation — deterministic, in-repo
Stills become motion with the animate-stills recipe
(recipes/04-animate-stills.md): a manifest-driven HyperFrames composition
applying camera language (push-ins, drifts, holds) and GSAP atmosphere
(mist, rain, light shifts, vignette) over each still, scene windows locked
to the transcript. scripts/build_scenes.py generates the scene layers from
the manifest; scripts/motion_check.py proves real motion via pixel-diff in
the rendered output. Renders identically every run, costs nothing, works
offline.
Requires Node >= 22 for the HyperFrames CLI.
Captions and assembly
- Captions burned in (muted autoplay on Reddit/X), phrase-level, timed from the transcript words.
- White flash (0.28s) at story turns; soft crossfades elsewhere; dark vignette grade unifies stills from mixed sources and makes captions read.
- Verify motion in the render: pixel-diff two frames 4s apart inside one
scene (
scripts/motion_check.py— mean gray diff > 3 = real motion). - Audio QA:
ffmpeg -af volumedetect— documentary speech sits near -26 dB mean.
Optional: hosted generative tools
These are NOT dependencies. They are one way among several to source stills or clips, and they can disappear without notice — plan accordingly.
- Apify text-to-image actors (e.g.
akash9078/ai-image-generator, ~$0.01/image) worked well for batch keyframe generation. Watch for silent failures (ok:truewith a null URL) and mislabeled file types. - Hosted image-to-video actors are where this pipeline previously kept its
motion step. On 2026-06-11 the actor
danitn11/wan22-lightning-image-to-video(2 total users) died upstream mid-production and every run failed instantly; the full analysis is indocs/rca/rca-generate-footage-animation-2026-06-11.md. Generative image-to-video now lives in a separate companion repo with a pluggable BYO-key backend; this plugin's motion step is deterministic and local.