Music Beat-Sync Video (Beat Mashup)
Use librosa to detect background music beats, hard-cut scene transitions on each beat, and add beat-synced effects to create an energetic "beat-sync" short video. Everything goes through
skills/shared/scripts/beatsync.py– do not manually count beats or hand-stitch filters – the script handles beat detection, segment splitting, material looping, and audio-video alignment.
For a soft slideshow with fixed duration per image, see slideshow-video; for theme → AI-generated images + voiceover + final video, see auto-short-video.
Input
| Field | Required | Description |
|---|---|---|
| Background music | Yes | Music file that determines the beat grid (stronger rhythm = better effect; ask if not provided) |
| Materials | Yes | A list of images and/or short clips (in order, cycled) or a directory |
| Aspect ratio | Yes | Must ask and wait for confirmation if the user or upstream task has not specified landscape/portrait (or exact resolution); do not silently infer from platform, profile, or defaults; do not repeat if already clarified |
| Switch frequency | No | How many beats between switches (default: every 2 beats) |
| Effect | No | zoom push-in (default) / flash white flash / none |
Output (outputs/themeName/)
- Beat-sync final video (
*.mp4) - Report: number of beat segments, total duration, detected BPM, effect used
Execution Steps
Script path (relative to project root): skills/shared/scripts/beatsync.py (build -h / beats -h).
1. (Optional) Preview beats
python skills/shared/scripts/beatsync.py beats --music <music>
# Prints tempo (BPM) and beat time points, used to decide what --every should be
2. Generate beat-sync video
python skills/shared/scripts/beatsync.py build \
-i 1.jpg 2.jpg 3.jpg 4.jpg --music <music> \
-o outputs/themeName/show.mp4 \
--size 1080x1920 --every 2 --effect zoom
- Materials = images or short clips, can be fewer than beat segments (automatically cycled).
--every 1switches every beat (fast flash, needs many materials);--every 4switches every bar (gentle).--effect flashadds a white flash at segment start for more "fire";--effect noneclean hard cut.--max-duration 15limits output length (e.g., 15s for TikTok).- Alternatively
--images-dir <directory>to batch-feed materials (sorted by filename).
Profile Awareness
- With profile:
platforms.mdis only used to suggest aspect ratios – still requires user confirmation. Beat-sync style (fast flash / gentle zoom) and effects should match account personality (trendy/sports accounts → fast flash, lifestyle/healing accounts → zoom gentle). - Without profile: ask landscape/portrait first; default to every 2 beats, zoom effect.
Rules
- Clearer rhythm = more accurate beat detection; if there is no obvious beat, the script gracefully degrades to equal-interval switching and reports this in the output summary.
- Beat-sync uses hard cuts (no transitions) to preserve the rhythmic impact – this is intentional design.
- Materials are automatically cycled if insufficient; when abundant, use
--every 1~2so each segment gets a fresh image. - Output length defaults to music length; use
--max-durationto cap for platform requirements. - All outputs go into
outputs/themeName/.
Reference Sources
Beat detection uses librosa beat_track (onset strength + dynamic programming to find beat points). Hard-cutting images at beat points is the standard practice in beat mashup editing. The beat → segment → per-segment rendering → audio-video alignment pipeline is encapsulated into a deterministic script to avoid manual beat counting and alignment errors.