xiaocong3/ecommerce-video-skills

ffmpeg Auto-assemble · Auto-assemble with ffmpeg

storyboard.json → voiceover (edge-tts) → Ken Burns / video crop per shot → xfade transitions → ASS subtitles with stylized text → BGM auto-ducking mix → two-pass loudnorm (-14 LUFS) → out/video.mp4 + .srt.

Part of ecommerce-video-skills · All scripts are in the scripts/ directory: assemble.py (main program) · tts_edge.py · make_subs.py · make_bgm.py (royalty-free BGM synthesis) · make_cards.py (PIL product cards / end cards)

Inputs

• storyboard.json (can be generated by selling-point-storyboard), with asset paths relative to that file
• Environment: Python 3.9+, pip install pillow numpy edge-tts, ffmpeg (with libass), CJK font

Quick start

pip install pillow numpy edge-tts
python scripts/assemble.py examples/aura-bottle/storyboard.json -o out/aura.mp4       # Repository example, renders in ~30 seconds
python scripts/assemble.py storyboard.json --tts none -o out/preview.mp4             # Offline preview (no voiceover, estimate timing by character count)
python scripts/assemble.py storyboard.json --size 1080x1080 -o out/main-1x1.mp4      # Main image video 1:1
python scripts/assemble.py storyboard.json --bgm music.mp3                           # Use your own licensed music

storyboard Fields

Field	Description
Top-level voice / rate / pitch	edge-tts voice, speed (e.g., +10%), pitch
Top-level bgm	"auto" (built-in synthesis, royalty-free) / music file path / null
Top-level tag	Corner sticker (brand/model/"Demo") shown throughout
image or video (+start)	Asset; for video, start from start seconds and capture shot duration
line	Voiceover text; shot duration = voiceover + 0.15s lead-in + pad
sub	Subtitles (default same as line); "" means no subtitle for this shot; **highlight**, `
title / cta	Stylized text / call-to-action button
motion	zoom_in zoom_out pan_left pan_right pan_up pan_down punch static
fit	auto (crop to fill if aspect ratio is close, otherwise blur fill) / cover / blur
transition	xfade name for transition to next shot: fade slideleft slideup smoothleft circleopen wipeleft …
min / duration / pad	Minimum duration / fixed duration / trailing silence after sentence (seconds)

Steps

1. Check that all assets exist and have suitable resolution (product images ≥ 1000px); if no end card, generate one using make_cards.py: python scripts/make_cards.py --image main.png --name "Product Name" --points "Selling point 1|Selling point 2|Selling point 3" -o assets/end-card.png
2. First run with --tts none to preview the visual pacing, then do the full render.
3. After rendering, spot-check frames: ffmpeg -ss 3 -i out/video.mp4 -frames:v 1 check.png
4. Pass to platform-spec-export to generate platform-specific versions; use cover-title-ab for the cover image.

How it works

• Ken Burns: Uses scale(eval=frame)+crop per frame, which is less jittery than zoompan; punch = 0.35 second fast zoom-in (for hook shots).
• Transitions: Each shot renders an additional half transition length; xfade is centered on the shot boundary, so voiceover start is not eaten by transitions.
• Audio mixing: Voiceover is placed at the start of each shot; BGM defaults to -20 dB, compressed by an additional 10 dB when voiceover is present (smooth ducking); compression/limiting then two-pass loudnorm to approximately -14 LUFS, TP -1.5 dB.
• Subtitles: Real voiceover duration is used to generate build/timing.json → make_subs.py → libass burn-in, with simultaneous export of .srt.

Checklist

• Final video duration, resolution, and frame rate are correct (ffprobe out/video.mp4)
• Check 3 frames: subtitles do not overlap product or enter bottom UI area
• Listen once: voiceover is clear, BGM does not overpower, no clipping at start
• BGM has commercial license (the auto synthesized music is safe to use)
• Assets (images/video/fonts) have usage permissions

Common Issues & Fixes

Issue	Fix
ffmpeg failed	Check the end of stderr printed by the script; common causes are incorrect paths or ffmpeg missing libass/libx264
Blurry edges	Asset resolution too low; use images ≥ 1080px or reduce motion intensity (set to static)
Product is cropped	Set "fit": "blur" for that shot
Audio cut at transitions	Not possible (voiceover is aligned to shot start); if you set a very short custom duration, increase it
Slow rendering	Preset already uses veryfast for intermediate files; you can change the final encoding -preset medium to fast
Want 1:1 / 16:9	Use --size 1080x1080 / --size 1920x1080; subtitle font size auto-scales proportionally

Acknowledgments: The approach was inspired by MoneyPrinterTurbo (MIT) pipeline of "script → voiceover → subtitles → composition". This implementation is an independently written pure ffmpeg version and does not copy its code.