render-cartoon-music-video
Assemble a cartoon music-video ad from a config: an animated / illustrated / hand-crafted story (stop-motion felt-and-foam, claymation, 2D toon, cut-out) where a sung song is the entire script and one recurring animated character carries every shot, cut to the bar with the climax line on the drop. This capability is the FREE, deterministic assembly — cut-to-bar, hard-concat, the logo bug + captions burn, and the solid-color end card.
scripts/config.example.json is the worked example (Coinbase "Bet on anything", ~55s 1080×1920
9:16, 16 body bars + a 3s end card — its felt look, character, story and song are that demo's
picks, never defaults); scripts/PIPELINE.md maps every config block to its source
step and scripts/README.md documents the free assembly.
Choices
The creative calls are the caller's (the video-format recipe asks the user); this assembly never picks them. The demo's value is an example only:
- Art style / look pack (felt stop-motion, claymation, 2D toon, cut-out…) — demo: felt-and-foam diorama.
- Character — the ONE recurring animated figure — demo: a gender-neutral felt figure in a brand-blue jersey.
- Story arc — demo: underdog payoff.
- Song style + lead vocal — demo: hype dance-pop / EDM-trap, male half-rapped lead.
Run
This is the FREE, deterministic assembly stage — it spends nothing. The paid inputs are
separate capabilities: the sung song (create-music-elevenlabs, or a user-supplied mp3 + Whisper
word timings) beat-tracked with librosa so the BAR GRID sets the timeline; one locked recurring
character + one keyframe per bar in one look pack (create-image-fal, Nano Banana); and one
Seedance i2v clip per bar (create-video-fal). Given the song + word-timestamps.json +
bars.json + one clip per bar + the brand wordmark SVG, render-cartoon-music-video cuts each
clip to its bar window, hard-concats on the bar, burns the logo bug + captions, appends the
solid-color end card, and muxes the song under it → the master. Re-cuts reuse the existing song /
keyframes / clips and cost $0.
Contract (the free assembly)
- The sung song carries the narrative — no separate VO. The generated/supplied track IS the bed (no VO to duck under); do not add a spoken voiceover or a second bed.
- Plan the timeline AROUND the delivered song's BAR GRID. Beat-track the song with librosa (assume 4/4); one bar = one shot (default). Snap every tableau window to the bar boundaries — never trim the song to a pre-planned grid.
- Captions from the song's word timings, re-spelled against the locked lyrics. Whisper
mishears shouted accents ("BETS" → "Hearts"); re-spell the timed tokens against the locked
lyric file (never edit lyrics to match Whisper). VEED-whisper white bold sans in the
BOTTOM third (Alignment 2,
margin_vabove the bottom-left logo bug), ~4–5 words per cue, NO background pill; captions STOP at the end-card boundary. Never mid-frame — it covers the character (fixed 2026-07). If the host ffmpeg lacks libass, render the cues as timed PIL PNG overlays (ffmpegoverlay=…:enable='between(t,st,en)') at the same bottom placement. - Land the climax line on the drop. The climax tableau is timed so the payoff line sits on the sub-bass drop; accent that line.
- Persistent logo bug, suppressed on the end card. Burn a white brand wordmark bottom-left over the body bars (cairosvg → PIL from the real SVG), suppressed on the end card where the big wordmark dominates.
- End card via cairosvg + PIL from the real wordmark — never AI-render brand text. Solid brand-color card + white wordmark + subhead, holding ~3s WITH the song still playing under it (afade-out over the tail — no silent tail). A diffusion model garbles a wordmark.
- FFmpeg composite, deterministic, FREE. Cut each clip to its bar window, hard-concat, burn
the logo bug + captions, append the end card, mux the song over the whole video with a 0.5s
afade tail,
loudnorm I=-14→ a 1080×1920 h264+aac master. No paid calls, no keys. When the host ffmpeg lacks libass/drawtext, BOTH the captions and the logo bug are timed PIL PNG overlays (overlay=x:y:enable='between(t,st,en)'), not an ASS burn. - QC PER SCENE, never just the master — the two failure modes are content, not assembly. The upstream keyframe/i2v steps can (a) drift the character off its look (felt → smooth-3D "man" in the demo) partway through (off the chosen look — in the felt demo, felt → smooth-3D) and (b) hallucinate hands — realistic fingers in a hand macro, a pointing finger on a "tap the phone" shot, or a disembodied hand sliding in from the frame edge. Both hide at thumbnail size. Extract a 2 fps contact sheet + a per-bar FACE crop and HAND crop (across each clip's full duration), confirm every bar holds the ONE chosen look (the demo: matte felt) with the ONE character and no human/floating hands, and re-check the served bytes after publish. A drifted bar means regenerating that bar's KEYFRAME (not re-cutting) — see the recipe STEP 3/6.