chenlikun2010/product-promo-video-skill

Product Promo Video Production (Remotion)

Turn an "asset directory + creative ideas" into a playable MP4 promo video. The core is rendered locally using Remotion (React-code-driven video), which is reproducible and easy to fine-tune; scripting, TTS, and minor supplementary footage use Alibaba Cloud Bailian CLI bl as the primary tool.

Suitable for: motion-graphics-style promo videos centered on images, product photos, app screenshots, and short clips, with text + animations + voiceover (exhibition loops, product introductions, conference openers). Not suitable for: footage that requires extensive live-action filming or complex camera moves. A few missing shots can be supplemented using the Bailian Happy Horse model and then composited back into Remotion.

Environment Prerequisites

• node (>=18), npm, ffmpeg/ffprobe, python3 (with numpy, requests). Verify with which node ffmpeg python3 before starting.
• bl CLI installed and logged in: bl --version, bl auth status. Before using bl, follow the version pre-check rules in the bailian-cli skill.
• Internet connection required for rendering (first-time load of Noto Sans SC font).

Standard Workflow (execute in order)

1. Understand Assets & Intent

• Ask the user for the asset directory and creative ideas/selling points. First list the directory: find <dir> -type f \( -iname '*.jpg' -o -iname '*.png' -o -iname '*.mp4' -o -iname '*.mov' \).
• Read key images one by one (examine product, features, interface), use ffprobe on each video to get duration/codec/resolution.
• Identify issues: ① If screenshots contain unfavorable data (e.g., "medication adherence rate 4%"), plan to overlay positive data using React in the video; ② If the same product has multiple colors/models, select the primary visual; ③ Remove irrelevant assets.

2. Design the Storyboard & Align with User (Important)

• Use AskUserQuestion to confirm: Style, Duration (60–90s recommended for exhibition loops), Aspect ratio (landscape 1920×1080 / portrait 1080×1920), Voiceover method.
• Directly offer 3 preset style options (see styles/STYLES.md): warm (family-oriented warmth) / guochao (vibrant Chinese fashion) / tech (corporate technology). Each preset already implements color schemes, decorations, and storyboard layouts; after selection, apply the style step directly without designing visuals from scratch. For a new style, base it on the closest preset and modify.
• The copy tone should match the style: emotional narrative (warm/guochao) or corporate product intro (tech).
• Draft script/storyboard with Bailian first, then manually polish:

bl text chat \
  --system "Reply in 简体中文. 输出紧凑的产品宣传片分镜表." \
  --message "Based on these assets and selling points, write a 60-second product promo storyboard: <asset list + selling points>"

• Produce a storyboard table: each segment (shot number / duration / visual / big text card / narration / which asset to use). Exhibition video principles: understandable even without sound (big text cards as primary), grab attention in 3 seconds, ending should loop back to the beginning.

3. Scaffold the Project from Template

PROJ="$HOME/<project-name>"
cp -R ~/.claude/skills/product-promo-video/template "$PROJ"
cd "$PROJ" && npm install
mkdir -p public/assets public/audio out

The template includes: theme.ts (color scheme/fonts/SCENE_FRAMES), components.tsx (KenBurns/BigTitle/Subtitle/Narration/BgMusic/PhoneMock/Chip), Video.tsx (TransitionSeries dissolve), Root.tsx, scenes/AllScenes.example.tsx (complete example; after modification, save as AllScenes.tsx for Video to reference).

4. Process Assets into public/assets

• HEVC must be transcoded to H.264 (Chromium is unstable with HEVC decoding): ffmpeg -y -i in.mp4 -c:v libx264 -pix_fmt yuv420p -crf 20 -an public/assets/weather.mp4
• Copy images and rename to ASCII filenames (staticFile is unreliable with Chinese paths): intro.jpg / device.jpg / app_home.jpg ....
• Copy screenshots with unfavorable data as-is; plan to overlay React cards within storyboard components.

5. Optional: Bailian Happy Horse Supplementary Footage

• If critical shots are missing from assets, use bl video generate / bl video ref to generate supplemental footage; skip if assets are sufficient and use Remotion effects instead.
• Do not limit the Happy Horse video generation length in the skill. Set --duration according to user needs, budget, and model limitations; for longer final videos, generate multiple segments and composite them with Remotion/ffmpeg.
• To reduce trial-and-error costs, first test with a 720P, 5-second sample to confirm direction, then adjust parameters for the final requirement.
• Text-to-video example (low-cost test sample, adjustable duration):

bl video generate \
  --model happyhorse-1.1-t2v \
  --prompt "A bright modern exhibition hall, a smart health device slowly rotating under soft lighting, commercial tech texture, no text, no logo" \
  --ratio 16:9 \
  --resolution 720P \
  --duration 5 \
  --download public/assets/ai_device_5s.mp4

• Image-to-video example (low-cost test sample; local image can be passed directly with --image, CLI auto-uploads temporary file):

bl video generate \
  --model happyhorse-1.1-i2v \
  --image public/assets/device.jpg \
  --prompt "Keep product appearance consistent, camera gently pushes in, clean background, soft reflections, no text" \
  --ratio 16:9 \
  --resolution 720P \
  --duration 5 \
  --download public/assets/device_i2v_5s.mp4

• After generation, use ffprobe to check duration/codec; if HEVC or browser-unstable, transcode to H.264 as in Step 4.

6. Write Copy + Alibaba Bailian TTS (Primary)

• Write narration into narration.json: {"s1":"...","s2":"..."}, keys correspond to storyboard segments, matching Narration file="s1.mp3". Separate with commas to help TTS produce natural pauses.
• Primary: Bailian CosyVoice:

source ~/.claude/skills/product-promo-video/config/bailian.env
python3 ~/.claude/skills/product-promo-video/scripts/gen_tts_bailian.py \
  --script narration.json --out public/audio

• To directly generate a single sentence via CLI:

bl speech synthesize \
  --text "A sentence of product promo narration" \
  --voice longanwen_v3 \
  --language zh \
  --instruction "Commercial tech promo narration, confident, clear, warm tone, moderate speed" \
  --format mp3 \
  --out public/audio/s1.mp3

• Volcano Engine as fallback:

source ~/.claude/skills/product-promo-video/config/volcano.env
python3 ~/.claude/skills/product-promo-video/scripts/gen_tts_volcano.py \
  --script narration.json --out public/audio

• Error code 3001 resource not granted = account authorization issue: enable "Speech Synthesis" in Volcano console, add the voice to the app ID, verify app ID/token pairing. The first request may still fail right after service activation; the script has built-in retries. Re-run any individual failures separately.
• No network/authorization fallback: scripts/gen_tts_say.sh narration.json public/audio (macOS say, low audio quality, use as placeholder).
• Large-model voices (more natural, recommended): For *_bigtts / saturn_* / *_uranus_* etc., use the v3 interface: source config/seed.env then python3 scripts/gen_tts_seed.py --script narration.json --out public/audio. To change the voice, simply override with export SEED_SPEAKER=<voiceID>. This uses API Key authentication (X-Api-Key) and streaming response; it is a separate service/billing from v1 (BV voices).
  • Resource-Id uses seed-tts-2.0 (default in seed.env). Tested with saturn_* and *_uranus_* voices; incorrect usage (e.g., seed-tts-1.0/volc.megatts.default) will return code 55000000 resource ID is mismatched with speaker.
  • Occasional RemoteDisconnected due to network/proxy fluctuations; the script has built-in exception handling + retries (--retry 5).

7. Calibrate Shot Lengths Based on Real Audio Duration (Critical, otherwise audio gets cut off)

~/.claude/skills/product-promo-video/scripts/measure_audio.sh public/audio

In theme.ts's SCENE_FRAMES, ensure each segment:

Shot frames ≥ delayFrames of Narration + ceil(audio seconds × fps) + trailing padding (≥15 frames recommended) After changing durations, also check total frame count in Video.tsx (auto-summed, usually not needed to modify).

8. Apply Style / Write Storyboard Components

• First choice: Use a ready-made style preset (styles/STYLES.md). Copy the selected style's theme.ts, decoration modules, and AllScenes.tsx into the corresponding src/ location, then modify the on-screen big text cards according to the project's copy. Three presets: warm/guochao/tech.
• If handwriting/fine-tuning storyboards: use components.tsx's BigTitle (big text card), KenBurns, Subtitle, PhoneMock (supports crop), Chip, FeatureCard; for guochao/tech, overlay respective decorative elements (seal stamp/glass card/corner tag/HUD).
• Place one <Narration file="sX.mp3" delayFrames={..} /> per segment.
• Overlay unfavorable data: place a React card on top of the corresponding PhoneMock (see example MedsBadge).

9. Background Music

python3 ~/.claude/skills/product-promo-video/scripts/gen_bgm.py \
  --duration <video seconds+2> --out public/audio/bgm.mp3   # optional --semitones transposition --gain volume adjustment

Video.tsx already includes <BgMusic base={0.85} />, which will automatically fade out according to video length and sit under the narration. Reduce base if too loud.

10. Render + Self-Check

npx remotion render Promo "out/<project-name>.mp4" --concurrency=2
~/.claude/skills/product-promo-video/scripts/verify.sh "out/<project-name>.mp4"

• Use --concurrency=2: High concurrency (4) may cause sporadic Node crashes on local machines (output shows only Node.js v22.x stack tail, no final video). If it crashes, reduce concurrency and run again in the foreground; usually succeeds.
• When rendering in the background, crashes may still return exit code 0; always check that the mp4 file actually exists in out/, not just the exit code. Then use Read to view /tmp/vchk frame captures (verify Chinese/emoji/layout), check verify.sh output for volume (mean ~ -18 to -23 dB is normal, must not be silent). If necessary, use silencedetect to check if a narration sentence is fully pronounced.

11. Delivery & Variations

Report the output path/specs/duration. Common follow-ups: embed real-life short videos, create a portrait version (modify Root.tsx dimensions + rearrange layout), change copy/color scheme/music and re-render. You can also use npm run dev to open Remotion Studio for live preview.

Hard Pitfalls (Must Follow)

• Narration must be placed inside <Sequence from={delayFrames}>. Do not use if(frame<delay) return null to "delay" – that does not delay start, but instead skips the first delay seconds of audio, eating the beginning of each sentence (typical symptom: "爸妈的药" pronounces as "的药"). The template's Narration component already uses the correct implementation.
• HEVC video must be transcoded to H.264 before using <OffthreadVideo>, otherwise Chromium renders a black screen or throws an error.
• Use ASCII filenames for assets placed in public/assets, reference with staticFile('assets/xx.jpg').
• Chinese font uses @remotion/google-fonts/NotoSansSC (template theme.ts already configured, fallback to PingFang); colored emoji renders normally in Chromium, safe to use as icons.
• After generating voiceover, always re-measure audio duration and recalibrate SCENE_FRAMES; different voices/speeds change duration, otherwise the tail end of speech will be cut off.
• Do not write the 5-second sample as a production limitation. The --duration for Happy Horse supplemental footage is determined by user need, budget, and model support; longer videos can be composed from multiple AI segments, real assets, and Remotion/ffmpeg splicing.
• Avoid text/Logo in AI video prompts, as it tends to generate wrong characters or distorted logos. Brand names, selling points, and subtitles should be overlaid using Remotion exclusively.
• Volcano 3001 is usually an account authorization issue, not a code problem; follow the troubleshooting steps in the TTS section.
• Volcano credentials are in config/volcano.env (contains secret keys, do not share); the HTTP interface does not need the Secret Key.
• All remotion packages must be locked to the exact same version (remotion / @remotion/cli / @remotion/transitions / @remotion/google-fonts all 4.0.xxx without ^). Using a caret may let a sub-package install a newer version, causing bundling errors like export 'HtmlInCanvas' was not found in 'remotion'. The template is already locked; after adding a new package, use node -e "..." to verify version consistency.

File Map

• scripts/gen_tts_bailian.py – Alibaba Bailian CosyVoice TTS (primary, calls bl speech synthesize)
• scripts/gen_tts_volcano.py – Volcano Standard TTS (BV voices, v1 interface, reads credentials from env, read copy from JSON)
• scripts/gen_tts_seed.py – Volcano Large-Model TTS (bigtts/saturn/uranus voices, v3 interface, API Key authentication, recommended)
• scripts/gen_tts_say.sh – macOS offline fallback TTS
• scripts/gen_bgm.py – Generate warm background music
• scripts/measure_audio.sh – Measure narration duration → frames, calibrate SCENE_FRAMES
• scripts/verify.sh – Final video self-check (specs/volume/frame capture)
• config/bailian.env – Bailian TTS parameters · config/volcano.env – Volcano Standard TTS credentials · config/seed.env – Large-Model TTS credentials
• config/narration.example.json – Emotional narrative copy example · config/narration.corporate.example.json – Corporate product intro copy example
• styles/STYLES.md + styles/{warm,guochao,tech}/ – Three preset style packages (color scheme + decorations + storyboard), copy into project src/ after selection
• template/ – Remotion project scaffold ready for cp -R