ZJU-REAL/Easel

One-Click Short Video (End-to-End Orchestration)

Input a topic, output a finished short video. This SKILL is the orchestration layer that chains existing building blocks into a pipeline — script (video-script) → per-sentence images (ai-image-gen) or clips (ai-video-gen) → voiceover (tts-voiceover) → subtitles (auto-subtitle) → BGM (ai-music) → assembly (scripts/assemble.py). Inspired by the automated short-video engine approach from Pixelle-Video / MoneyPrinterTurbo.

Input

• Topic / script (required)
• Optional: target duration, style, whether to include voiceover/subtitles/BGM, use AI-generated images or user-provided assets
• Aspect-ratio mandatory confirmation gate: If the user or upstream task has not explicitly specified landscape/portrait (or 16:9/9:16/specific resolution), you must ask for and wait for confirmation before any production/payment calls; do not silently infer from platform, profile, or defaults. If already specified, do not ask again.

Output

The finished short video is written to outputs/<topic>/final.mp4; storyboard images, voiceover, subtitles, and storyboard JSON are written to outputs/<topic>/assets/.

Execution Steps (trim as needed; automatically degrade or query when API keys are missing)

1. Write script and storyboard: Use video-script to turn the topic into a talking-head script, split into N sentences (each a shot), with a visual description per sentence.

2. Generate visuals (one image or clip per shot):

   • With image API key → ai-image-gen for per-sentence text2img (using the confirmed aspect ratio)
   • For motion → ai-video-gen text2video/image2video
   • With user-provided assets → Use image-editing pad to match the confirmed aspect ratio
   • Without any → Use template cards scaled to the confirmed aspect ratio (portrait: card-xiaohongshu/poster-hero, landscape: card-quote) then pad, to avoid aspect-ratio mismatch.

3. Voiceover: tts-voiceover synthesizes the script into narration (outputs SRT alongside). If VOICE_PROVIDER is configured, it defaults to a premium closed-source voice (emotional, human-like); only fall back to edge (robotic) if no key is available — for natural-sounding talking-head audio, always set a closed-source key. Skip if voiceover is not needed.

4. Subtitles: Use the SRT from TTS, or run auto-subtitle on the voiceover; alternatively, let assemble auto-generate subtitles from each shot's caption.

5. BGM: ai-music generates music, or use user-provided music. Optional.

6. Assemble into final video: Write the above assets into a storyboard JSON, then call the composer:

   python skills/openclaw/auto-short-video/scripts/assemble.py assemble \
     --storyboard outputs/<topic>/assets/storyboard.json \
     -o outputs/<topic>/final.mp4
   

   Storyboard structure (choose image or video per shot; narration/bgm/subtitle optional; if duration is omitted, it is evenly split according to voiceover duration):

   {
     "size": "<confirmed dimensions, e.g., 1080x1920 or 1920x1080>",
     "image_motion": "ken-burns",
     "shots": [
       {"image": "outputs/<topic>/assets/shot1.png", "duration": 3, "caption": "First sentence", "motion": "static"},
       {"video": "outputs/<topic>/assets/clip2.mp4", "caption": "Second sentence"}
     ],
     "narration": "outputs/<topic>/assets/voice.mp3",
     "bgm": "outputs/<topic>/assets/bgm.mp3",
     "subtitle": "outputs/<topic>/assets/voice.srt"
   }
   

   image_motion sets the default motion for all images; per-shot motion can override: use ken-burns for photos, must use static for slides/charts/UI with text (proportional scale + padding, no cropping or panning). The composer automatically: generates still frames or Ken Burns based on image_motion, pads to aspect ratio, concatenates, mixes voiceover+BGM (auto-duck BGM), and burns in subtitles.

7. Deliver: Output final.mp4 with a production note describing which steps were used and which were degraded.

Orchestration Principles

• Missing components allowed: Steps without image/video/TTS API keys automatically degrade (fallback template cards / skip voiceover), do not block the whole pipeline, and truthfully inform the user what was degraded.
• Show plan first: When multiple paid APIs (image/video/music generation) are involved, first explain to the user which will be called, approximate time/cost, and get confirmation before proceeding.
• Preserve intermediate artifacts: Storyboard images, voiceover, subtitles, and storyboard JSON are all written to outputs/<topic>/assets/, enabling easy individual replacement and re-assembly.

Profile Awareness

• With profile: Read style.md for tone/visual style to use throughout the script and image prompts; platforms.md is used only for aspect ratio/duration suggestions (aspect ratio must still be confirmed); preferences.md applies content filters.
• Without profile: First confirm landscape/portrait mode, then use a generic talking-head style.