visalnaqvi/video-generator-skill

Render — music, sound, captions, final export

Input: videos/<video>/ with an approved storyboard.json (fitted to the voice: has audio and timing) and an approved audio/voice-script.json. Output:

audio/music.wav        chosen or generated music
audio/mix.wav          final soundtrack (voice + ducked music + SFX), -14 LUFS
audio/mix.json         every sound event (shot, cue, file, time) for checking
render/final.mp4       1080x1920 (per format), 30 fps, H.264 + AAC
render/cover.png       full-resolution cover / thumbnail
render/qc.json         quality-check report

This stage doesn't change visuals or words. Wrong animation or timing → storyboard / voiceover skills. Free by default: library or code-generated music, code-generated sound effects. Paid music (AI generation) only if the user asks or approves an estimate.

Step 1 — Music

1. Read the storyboard's music.mood / notes and the video length.
2. Look in studio/library/music/ (index.json lists tracks with mood tags, bpm, licence). Pick the best match; if none fits, say so.
3. No match → generate the free fallback, with the lift ("drop") on the reveal shot's start (scene start time — compute from durations/transitions): node studio/scripts/make-music.mjs --out videos/<video>/audio/music.wav --duration <s> --mood upbeat|calm|cinematic|tense --bpm <n> --drop <s> [--silence <from>-<to>] (tense: minor key, heartbeat pulse, unresolved ending; --silence mutes a window before a reveal sound — in tense mood a ticking clock follows it). Universe video: follow universe.json → music / sfx. Library tracks: copy/convert to audio/music.wav; trimming, fading and ducking happen in the mix.
4. Record it in storyboard.json → music.file, music.source (track + licence, or "code-generated"), optional music.db (default -16 dB before ducking).

Step 2 — Sound effects

Cues already live in each shot's sfx (plain words, anchored to frames or spoken words by the voiceover stage). The mixer maps words → sounds via studio/library/sfx/index.json (generated by node studio/scripts/make-sfx.mjs; custom sounds go in custom, checked first). repeat + every play a cue several times; syncEnd: true places it at the shot's end; risers end on their cue. Transition whooshes are opt-in (--auto-whoosh) — the user prefers restrained sound: signature moments only (≈ 1 cue per 2 s at most, none on most transitions). If a storyboard has far more cues than that, thin them to the key moments before mixing and say so. Fix any "No sound matched" lines from the mixer by adding keywords or a custom sound — never by changing the storyboard's design.

Step 3 — Mix

node studio/scripts/mix.mjs videos/<video> (--dry to preview the event count first)

Voice stays on top; music is ducked by the voice (sidechain) and fades at the end; SFX at -12 dB (--sfx-db, or storyboard.sfxDb); master limited and normalised to -14 LUFS / -1.5 dBTP. Sets storyboard.finalAudio. Re-run after any audio change.

Step 4 — Captions

Default on for social formats. In storyboard.json: "captions": { "enabled": true, "y": 83, "size": 50, "skipLines": ["L11"] }

• Phrases (≤ 4 words, balanced, split on pauses) come from the voice timings; the spoken word is highlighted in the brand's highlight colour.
• skipLines: VO lines whose words are already on screen (phone numbers, URLs).
• Per shot: captionY (% of frame) to move them, captions: false to hide them where the same words are on screen (e.g. a name lower third).
• Keep them inside the safe zone (default 83%) and off important design elements.

Check with one contact sheet (captions show on frames where someone speaks): node studio/scripts/render.mjs videos/<video>/storyboard.json --sheet --scale 0.25 --out <scratchpad>/captions-sheet.png Look once; fix collisions with captionY / captions: false.

Step 5 — Final render, cover, QC

node studio/scripts/render.mjs videos/<video>/storyboard.json --final
node studio/scripts/render.mjs videos/<video>/storyboard.json --cover
node studio/scripts/qc.mjs videos/<video>

QC must pass: resolution, frame rate, codecs, length = storyboard, loudness -14 ±1.5 LUFS. Warnings (true peak, silent gaps, black frames) → explain or fix. Don't watch the MP4; send render/final.mp4 (and the cover) to the user with the QC summary and what's in the mix (music source, number of sound events, captions on/off).

Step 6 — Approve

User happy → "status": "final" in storyboard.json → render, done. Changes: music/sound/caption tweaks → Steps 1–5 here; visual or timing changes → storyboard / voiceover skill, then re-run this stage.