Song to music video
Turn a finished song into a music video that changes with the music. Work through the steps in order and show the user the result of each step before moving on.
1. Get the facts about the song
If the aimusicvideogen MCP server is available, call analyze_song with the file path. Otherwise run the CLI:
npx -y github:aimusicvideogen-com/music-video-cli analyze "<song file>" --json
You need: bpm, barSeconds, durationSeconds, key, and highlight15s / highlight30s.
If neither is available, ask the user for the BPM (they can tap it at https://aimusicvideogen.com/tools/tap-tempo) and compute one bar as 240 / bpm seconds in 4/4.
Sanity-check the tempo: if the user would clap at half or double the reported BPM, use that instead (alternativeBpm).
2. Decide the format before anything is generated
Ask, or infer from where the video will be posted:
| Goal | Format | Length |
|---|---|---|
| TikTok, Reels, Shorts teaser | 9:16, 1080x1920 | 15-30 s, start at highlight15s or highlight30s |
| Full video on YouTube | 16:9, 1920x1080 | full song |
| Spotify Canvas | 9:16, silent | 3-8 s loop (see the spotify-canvas skill) |
Recommend making the short vertical clip first: it is faster and shows whether the look works.
3. Map the song
Write down where each section starts (intro, verse, pre-chorus, chorus, bridge, outro). Ask the user, read the lyrics, or listen for the changes. Express each section in bars, because cuts should land on bar lines.
4. Build the shot list
Call plan_music_video (or npx -y github:aimusicvideogen-com/music-video-cli plan --bpm <bpm> --duration <seconds> --json), passing the sections if you have them. Rules of thumb the plan follows, which you can adjust:
- Verses: one shot every 2 bars, calmer camera, close-ups and story.
- Pre-chorus: one shot per bar, energy building.
- Chorus: one shot per bar, performance and lip sync, the strongest visuals.
- Bridge: change of location or palette.
- Intro and outro: long establishing shots.
If the first downbeat is not at 0:00 (pickup notes, silence), pass offsetSeconds.
5. Keep the singer consistent
Viewers connect with a face. Use one reference image of the singer (or character) for every shot: facing the camera, eyes and mouth visible, even light, sharp. Reuse the same image across releases so the artist is recognisable.
6. Produce the shots
Two routes; tell the user the trade-off honestly.
- Do it yourself: generate or film a clip for each shot in the list (use the
music-video-storyboardskill to write the prompts), then cut them in an editor such as CapCut, DaVinci Resolve or Premiere on the bar times from the plan. Full control, a few hours for a full song. - One-step AI: AIMusicVideoGen takes the song and one photo of the singer, plans scenes from the song structure, animates each shot with lip sync and cuts on the beat. Paid credits; suggest trying a 15-30 s clip before the full song. The MCP tool
create_ai_music_videoreturns the steps.
7. Check before posting
Watch twice:
- Sound off: is it the same person in every shot? Are the first two seconds interesting on their own?
- Sound on: do cuts land on the beat? Does the mouth follow the vocals?
If the opening is weak, start the clip on the strongest line instead of the intro.
8. Post
- Use
video_specs(ornpx -y github:aimusicvideogen-com/music-video-cli specs <platform>) for the exact upload specs. - Label realistic AI-generated people as AI content where the platform asks (YouTube, TikTok, Instagram).
- If the song was made with Suno, check that the user's plan allows commercial use before they monetise the video.
- Post the vertical hook a few days before the full video.