Communitygithub.com

Army161/OpenMontage

使用 Claude Code 影片工具包,自主建立專業級影片 — 包含 AI 旁白、圖片生成、音樂、虛擬主播以及 Remotion 渲染。

OpenMontage 是什麼?

Army161/OpenMontage 是一個 AI 代理工具包,可根據文字簡報,自動化產出專業解說影片。它整合了多種開源 AI 模型,透過雲端 GPU(如 Modal 或 RunPod)處理旁白(Qwen3-TTS)、圖片生成(Flux2)、背景音樂(MusicGen)及虛擬主播動畫(SadTalker、LTX2),最終使用 Remotion (React) 合成與渲染影片。該工具特別設計給 Claude Code 這類開發環境使用,要求所有指令都必須在固定的工具目錄下執行,以確保路徑正確。設定流程包括安裝 Python 依賴、設定 Modal 端點、部署微服務,並透過驗證腳本檢查所有工具是否就緒。使用時可選用多種範本(如產品展示、衝刺回顧等),透過設定檔調整場景、旁白腳本與生成資產,最後同步時間軸並渲染輸出。

相容平台✓Claude Code~Codex CLI~Cursor
npx skills add https://github.com/Army161/OpenMontage/tree/HEAD/.agents/skills/video-toolkit

在你喜歡的 AI 中提問

開啟一個已預先載入此 Agent Skill 的新對話。

說明文件

影片工具包

根據文字簡報建立專業解說影片。此工具包使用雲端 GPU(Modal 或 RunPod)上的開源 AI 模型處理旁白、圖片生成、音樂和虛擬主播動畫。Remotion (React) 負責組合與渲染。

重要:工具包路徑

工具包位於固定路徑。在執行任何工具指令前,請務必先 cd 到此目錄。

TOOLKIT=~/.openclaw/workspace/claude-code-video-toolkit
cd $TOOLKIT

絕對不要在專案目錄內執行工具指令。 工具會根據工具包根目錄解析路徑。

設定

步驟 1:檢查目前狀態

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/verify_setup.py

如果所有項目都顯示 [x],請跳至下方的「快速測試」。否則繼續設定。

步驟 2:安裝 Python 相依套件

cd ~/.openclaw/workspace/claude-code-video-toolkit
pip3 install --break-system-packages -r tools/requirements.txt

注意:在 Debian/Ubuntu 上使用受管理的 Python(PEP 668)時,需要 --break-system-packages。在容器內使用是安全的。

步驟 3:設定雲端 GPU 端點

工具包需要在 .env 檔案中設定雲端 GPU 端點 URL。檢查 .env 是否存在並包含 Modal 端點:

cat ~/.openclaw/workspace/claude-code-video-toolkit/.env | grep MODAL

如果 Modal 端點已設定完畢,您就可以開始使用。如果沒有,請要求使用者提供 Modal 端點 URL 或設定 Modal:

pip3 install --break-system-packages modal
python3 -m modal setup   # 開啟瀏覽器進行認證

# 部署每個工具 — 從輸出中擷取端點 URL
cd ~/.openclaw/workspace/claude-code-video-toolkit
modal deploy docker/modal-qwen3-tts/app.py
modal deploy docker/modal-flux2/app.py
modal deploy docker/modal-music-gen/app.py
modal deploy docker/modal-sadtalker/app.py
modal deploy docker/modal-image-edit/app.py
modal deploy docker/modal-upscale/app.py
modal deploy docker/modal-propainter/app.py
modal deploy docker/modal-ltx2/app.py      # 需要:modal secret create huggingface-token HF_TOKEN=hf_...

LTX-2 前置條件: 在部署 LTX-2 之前,請建立一個 HuggingFace 秘密並接受 Gemma 3 授權:

modal secret create huggingface-token HF_TOKEN=hf_your_read_access_token

將每個 URL 加入 .env:

MODAL_QWEN3_TTS_ENDPOINT_URL=https://...modal.run
MODAL_FLUX2_ENDPOINT_URL=https://...modal.run
MODAL_MUSIC_GEN_ENDPOINT_URL=https://...modal.run
MODAL_SADTALKER_ENDPOINT_URL=https://...modal.run
MODAL_IMAGE_EDIT_ENDPOINT_URL=https://...modal.run
MODAL_UPSCALE_ENDPOINT_URL=https://...modal.run
MODAL_DEWATERMARK_ENDPOINT_URL=https://...modal.run
MODAL_LTX2_ENDPOINT_URL=https://...modal.run

建議但非必須 — 使用 Cloudflare R2 以獲得可靠的檔案傳輸:

R2_ACCOUNT_ID=...
R2_ACCESS_KEY_ID=...
R2_SECRET_ACCESS_KEY=...
R2_BUCKET_NAME=video-toolkit

步驟 4:驗證與快速測試

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/verify_setup.py

所有工具應顯示 [x]。然後執行一個快速測試以確認 GPU 管道是否正常運作:

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/qwen3_tts.py --text "Hello, this is a test." --speaker Ryan --tone warm --output /tmp/video-toolkit-test.mp3 --cloud modal

如果您得到一個有效的 .mp3 檔案,則設定完成。如果失敗,請檢查:

  • .env 是否有正確的 MODAL_QWEN3_TTS_ENDPOINT_URL
  • 執行 python3 tools/verify_setup.py --json 並檢查 modal_tools 中缺少哪些端點

費用: Modal 包含每月 $30 美元的免費計算時數。一個典型的 60 秒影片費用為 $1-3 美元。


建立影片

步驟 1:建立專案

cd ~/.openclaw/workspace/claude-code-video-toolkit
cp -r templates/product-demo projects/PROJECT_NAME
cd projects/PROJECT_NAME
npm install

範本:product-demo(行銷/解說)、sprint-review、sprint-review-v2(可組合場景)。

步驟 2:編寫設定檔

編輯 projects/PROJECT_NAME/src/config/demo-config.ts:

export const demoConfig: ProductDemoConfig = {
  product: {
    name: 'My Product',
    tagline: 'What it does in one line',
    website: 'example.com',
  },
  scenes: [
    { type: 'title', durationSeconds: 9, content: { headline: '...', subheadline: '...' } },
    { type: 'problem', durationSeconds: 14, content: { headline: '...', problems: ['...', '...'] } },
    { type: 'solution', durationSeconds: 13, content: { headline: '...', highlights: ['...', '...'] } },
    { type: 'stats', durationSeconds: 12, content: { stats: [{value: '99%', label: '...'}, ...] } },
    { type: 'cta', durationSeconds: 10, content: { headline: '...', links: ['...'] } },
  ],
  audio: {
    backgroundMusicFile: 'audio/bg-music.mp3',
    backgroundMusicVolume: 0.12,
  },
};

場景類型:title、problem、solution、demo、feature、stats、cta。

時長規則: 將 durationSeconds 估算為 ceil(word_count / 2.5) + 2。您將在步驟 4 生成音訊後調整此值。

步驟 3:撰寫旁白腳本

建立 projects/PROJECT_NAME/VOICEOVER-SCRIPT.md:

## Scene 1: Title (9s, ~17 words)
Build videos with AI. The product name toolkit makes it easy.

## Scene 2: Problem (14s, ~30 words)
The problem statement goes here. Keep it punchy and relatable.

每個場景的字數預算: (durationSeconds - 2) * 2.5 個字。減 2 是為了計入 1 秒的音訊延遲加上 1 秒的補白。

步驟 4:生成素材

重要:以下所有指令都必須從工具包根目錄執行,而不是專案目錄。

cd ~/.openclaw/workspace/claude-code-video-toolkit

4a. 背景音樂

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/music_gen.py \
  --preset corporate-bg \
  --duration 90 \
  --output projects/PROJECT_NAME/public/audio/bg-music.mp3 \
  --cloud modal

預設樣式:corporate-bg、upbeat-tech、ambient、dramatic、tension、hopeful、cta、lofi。

4b. 旁白(每個場景)

為每個場景生成一個 .mp3 檔案。不要生成單一一個旁白檔案。

cd ~/.openclaw/workspace/claude-code-video-toolkit

# Scene 01
python3 tools/qwen3_tts.py \
  --text "The voiceover text for scene one." \
  --speaker Ryan --tone warm \
  --output projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --cloud modal

# Scene 02
python3 tools/qwen3_tts.py \
  --text "The voiceover text for scene two." \
  --speaker Ryan --tone warm \
  --output projects/PROJECT_NAME/public/audio/scenes/02.mp3 \
  --cloud modal

# ... 為每個場景重複

主播: Ryan、Aiden、Vivian、Serena、Uncle_Fu、Dylan、Eric、Ono_Anna、Sohee 語調: neutral、warm、professional、excited、calm、serious、storyteller、tutorial

用於語音複製(需要參考錄音):

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/qwen3_tts.py \
  --text "Text to speak" \
  --ref-audio assets/voices/reference.m4a \
  --ref-text "Exact transcript of the reference audio" \
  --output projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --cloud modal

4c. 場景圖片

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/flux2.py \
  --prompt "Dark tech background with blue geometric grid, cinematic lighting" \
  --width 1920 --height 1080 \
  --output projects/PROJECT_NAME/public/images/title-bg.png \
  --cloud modal

圖片預設樣式(使用 --preset 取代 --prompt --width --height): title-bg、problem、solution、demo-bg、stats-bg、cta、thumbnail、portrait-bg

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/flux2.py \
  --preset title-bg \
  --output projects/PROJECT_NAME/public/images/title-bg.png \
  --cloud modal

4d. 影片剪輯 — B-Roll 與動態背景(選用)

生成 AI 影片剪輯,用於 b-roll 切換、動態幻燈片背景或開場/結尾序列:

cd ~/.openclaw/workspace/claude-code-video-toolkit

# 從文字生成 B-Roll 剪輯
python3 tools/ltx2.py \
  --prompt "Aerial drone shot over a European city at golden hour, cinematic wide angle" \
  --output projects/PROJECT_NAME/public/videos/broll-europe.mp4 \
  --cloud modal

# 將幻燈片/截圖製作成動畫(圖片轉影片)
python3 tools/ltx2.py \
  --prompt "Gentle particle effects, soft ambient light shifts, very slight camera drift" \
  --input projects/PROJECT_NAME/public/images/title-bg.png \
  --output projects/PROJECT_NAME/public/videos/animated-title.mp4 \
  --cloud modal

# 抽象開場/結尾背景
python3 tools/ltx2.py \
  --prompt "Dark moody abstract background with flowing blue light streaks, bokeh particles, cinematic" \
  --output projects/PROJECT_NAME/public/videos/intro-bg.mp4 \
  --cloud modal

在 Remotion 合成中使用 <OffthreadVideo>:

<OffthreadVideo src={staticFile('videos/broll-europe.mp4')} />

LTX-2 規則:

  • 每個剪輯最多約 8 秒(24fps 下為 193 幀)。預設約 5 秒(121 幀)。
  • 寬/高必須能被 64 整除。預設:768x512。
  • 每個剪輯約 $0.20-0.25 美元,生成時間約 2.5 分鐘。
  • 冷啟動約 60-90 秒。後續在暖 GPU 上的剪輯生成更快。
  • 生成的音訊僅為環境音 — 請使用旁白/音樂工具來生成語音和音樂。
  • 約 30% 的生成結果可能帶有訓練資料的人工製品(如標誌/文字)。使用 --seed 參數重新執行以獲得變化。

4e. 虛擬主播(選用)

生成主播肖像,然後為每個場景產生動態剪輯:

cd ~/.openclaw/workspace/claude-code-video-toolkit

# 1. 生成肖像
python3 tools/flux2.py \
  --prompt "Professional presenter portrait, clean style, dark background, facing camera, upper body" \
  --width 1024 --height 576 \
  --output projects/PROJECT_NAME/public/images/presenter.png \
  --cloud modal

# 2. 為每個場景生成主播剪輯(每個場景一個,而不是一個長影片)
python3 tools/sadtalker.py \
  --image projects/PROJECT_NAME/public/images/presenter.png \
  --audio projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --preprocess full --still --expression-scale 0.8 \
  --output projects/PROJECT_NAME/public/narrator-01.mp4 \
  --cloud modal

# 為每個需要主播的場景重複

SadTalker 規則 — 請嚴格遵守:

  • 始終使用 --preprocess full(預設 crop 會輸出方形,畫寬比錯誤)
  • 始終使用 --still(減少頭部移動,看起來專業)
  • 始終生成每個場景的剪輯(6-15 秒),千萬不要生成一個長影片
  • 處理時間:在 Modal A10G 上,每 10 秒的音訊約需 3-4 分鐘
  • --expression-scale 0.8 使表情保持細微(範圍 0.0-1.5)

4e. 圖片編輯(選用)

從現有圖片建立場景變體:

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/image_edit.py \
  --input projects/PROJECT_NAME/public/images/title-bg.png \
  --prompt "Make it darker with red tones, more ominous" \
  --output projects/PROJECT_NAME/public/images/problem-bg.png \
  --cloud modal

4f. 放大(選用)

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/upscale.py \
  --input projects/PROJECT_NAME/public/images/some-image.png \
  --output projects/PROJECT_NAME/public/images/some-image-4x.png \
  --scale 4 --cloud modal

步驟 5:同步時間

在生成旁白後務必執行此步驟。 音訊時長與估算值不同。

cd ~/.openclaw/workspace/claude-code-video-toolkit
for f in projects/PROJECT_NAME/public/audio/scenes/*.mp3; do
  echo "$(basename $f): $(ffprobe -v error -show_entries format=duration -of csv=p=0 "$f")s"
done

將 demo-config.ts 中每個場景的 durationSeconds 更新為:ceil(實際音訊時長 + 2)。

範例:如果 01.mp3 是 6.8 秒,則將場景 1 的 durationSeconds 設為 9(ceil(6.8 + 2) = 9)。

步驟 6:檢查靜止幀

cd ~/.openclaw/workspace/claude-code-video-toolkit/projects/PROJECT_NAME
npx remotion still src/index.ts ProductDemo --frame=100 --output=/tmp/review-scene1.png
npx remotion still src/index.ts ProductDemo --frame=400 --output=/tmp/review-scene2.png

檢查事項:文字截斷、動畫時間、虛擬主播 PiP 位置、背景對比度。

步驟 7:渲染

cd ~/.openclaw/workspace/claude-code-video-toolkit/projects/PROJECT_NAME
npm run render

輸出: out/ProductDemo.mp4


合成模式

每個場景的音訊

使用帶有 1 秒延遲的每個場景音訊(from={30} = 30 幀 = 30fps 下的 1 秒):

<Sequence from={30}>
  <Audio src={staticFile('audio/scenes/01.mp3')} volume={1} />
</Sequence>

每個場景的虛擬主播 PiP

<Sequence from={30}>
  <OffthreadVideo
    src={staticFile('narrator-01.mp4')}
    style={{ width: 320, height: 180, objectFit: 'cover' }}
    muted
  />
</Sequence>

始終使用 <OffthreadVideo>,絕對不要使用 <video>。 Remotion 需要自己的元件以確保幀精確渲染。

轉場效果

import { TransitionSeries, linearTiming } from '@remotion/transitions';
import { fade } from '@remotion/transitions/fade';
import { glitch } from '../../../lib/transitions/presentations/glitch';
import { lightLeak } from '../../../lib/transitions/presentations/light-leak';

絕對不要從 lib/transitions 桶中匯入 — 直接從 lib/transitions/presentations/ 匯入自訂轉場效果。


錯誤復原

問題解決方案
工具指令失敗,顯示「No module named...」從工具包根目錄執行 pip3 install --break-system-packages -r tools/requirements.txt
「MODAL_*_ENDPOINT_URL not configured」檢查 .env 是否有端點 URL。執行 python3 tools/verify_setup.py
SadTalker 輸出為方形/被裁切您忘了使用 --preprocess full。加上該標誌重新執行
音訊相對於場景太短/太長重新執行步驟 5(同步時間)並更新設定檔
npm run render 失敗確認您在專案目錄中,而不是工具包根目錄。先執行 npm install
Remotion 中出現「Cannot find module」檢查匯入路徑。從 lib/transitions/presentations/ 直接匯入自訂轉場,不要使用桶索引

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

Army161/OpenMontage

Generate AI videos from text prompts using multiple provider gateways. Use when: (1) Generating videos from text descriptions, (2) Creating AI-generated video clips for content production, (3) Image-to-video generation with a reference image, (4) Choosing between video generation providers (VEO, Kling, Sora, Runway, Seedance, MiniMax). Supports two gateways: HeyGen API and fal.ai API.

Army161/OpenMontage

Create AI avatar videos with precise control over avatars, voices, scripts, scenes, and backgrounds using HeyGen's v2 API. Use when: (1) Choosing a specific avatar and voice for a video, (2) Writing exact scripts for an avatar to speak, (3) Building multi-scene videos with different backgrounds per scene, (4) Creating transparent WebM videos for compositing, (5) Using talking photos as video presenters, (6) Integrating HeyGen avatars with Remotion, (7) Batch video generation with exact specs, (8) Brand-consistent production videos with precise control.

Army161/OpenMontage

Use p5.js/canvas for local procedural character effects: particles, weather, squash/stretch, walk cycles, and environmental motion.

Army161/OpenMontage

Review local character animation with schema checks, Playwright browser previews, frame sampling, and FFmpeg/ffprobe final output checks.

Army161/OpenMontage

Create videos from a text prompt using HeyGen's Video Agent. Use when: (1) Creating a video from a description or idea, (2) Generating explainer, demo, or marketing videos from a prompt, (3) Making a video without specifying exact avatars, voices, or scenes, (4) Quick video prototyping or drafts, (5) One-shot prompt-to-video generation, (6) User says "make me a video" or "create a video about X".

Army161/OpenMontage

Generate Mandarin and multilingual narration with Volcengine Doubao Speech 2.0. Use when creating Chinese voiceovers, when the user prefers Doubao/Volcengine/火山引擎/豆包 TTS, or when narration needs character-level timestamp metadata for subtitles.

Army161/OpenMontage

Video and audio processing with FFmpeg. Use for format conversion, resizing, compression, audio extraction, and preparing assets for Remotion. Triggers include converting GIF to MP4, resizing video, extracting audio, compressing files, or any media transformation task.

Army161/OpenMontage

Use when implementing Disney's 12 animation principles with Framer Motion in React applications

Army161/OpenMontage

All animation knowledge for HyperFrames — atomic motion rules, multi-phase scene blueprints, scene transitions, broader motion-design techniques, AND the seven runtime adapters (GSAP default, plus Lottie, Three.js, Anime.js, CSS keyframes, Web Animations API, TypeGPU). Use for any motion or animation task: pick 2-4 rules and compose, or load a blueprint, or look up runtime-specific API (e.g. GSAP eases / Lottie player / Three.js mixer). HyperFrames-native: single paused timeline, seek-safe, deterministic.

Army161/OpenMontage

HyperFrames CLI dev loop. Use when running npx hyperframes init, add, catalog, capture, lint, validate, inspect, layout, snapshot, preview, play, render, publish, lambda, doctor, browser, info, upgrade, skills, compositions, docs, benchmark, telemetry, transcribe, tts, or remove-background, or when troubleshooting the HyperFrames build/render environment. Entry point for AWS Lambda cloud rendering (`hyperframes lambda deploy / render / progress / destroy / policies`).

Army161/OpenMontage

The HyperFrames composition contract — build one renderable project. Use for composition structure, the `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, deterministic-render rules, and validation. Read before writing composition HTML.

Army161/OpenMontage

Non-animation creative direction for HyperFrames videos. Use for design spec (frame.md / design.md) handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns, and brand / style decisions. For atomic motion patterns and scene blueprints, use `hyperframes-animation`.

Army161/OpenMontage

Audio and media assets for HyperFrames compositions, produced by one shared audio engine (`scripts/audio.mjs`) — multi-provider TTS (HeyGen / ElevenLabs / Kokoro local), background music + sound effects (HeyGen audio-library retrieval by default, with local Lyria / MusicGen BGM generation and a bundled SFX library as the no-credential fallback), Whisper transcription, background removal, and caption authoring. Use for voiceover / TTS, BGM, SFX / sound effects, transcription, captions / subtitles / lyrics / karaoke / per-word styling, voice + provider selection, and music-mood prompting.

Army161/OpenMontage

Install and wire registry blocks and components into HyperFrames compositions. Use when running hyperframes add, installing a block or component, wiring an installed item into index.html, or working with hyperframes.json. Covers the add command, install locations, block sub-composition wiring, component snippet merging, registry discovery, and authoring a new block or component to contribute upstream (idea → scaffold → validate → PR).

Army161/OpenMontage

Use when the user wants a short, design-led motion graphic where motion is the message: kinetic typography, stat or number count-up, chart/data-viz hit, logo sting, brand lockup, lower-third, callout, social overlay, animated headline/tweet/news item, motion poster, or quick captured-page highlight. Usually under 10s and up to ~30s, with no narration arc, voice-over, or live-action subject. Can render to MP4 or transparent overlay. Not for longer, multi-scene, narrated, or brand-reel pieces (use general-video), narrated website videos (website-to-video), topic explainers (faceless-explainer), product promos (product-launch-video), PR videos (pr-to-video), or captions on existing footage (embedded-captions). When unsure whether it's a quick motion-first piece or a longer / narrated treatment, see /hyperframes.

Army161/OpenMontage

Use when the user has a music track (an audio file, or a video to pull audio from) and wants a beat-synced HyperFrames video, calm to hard-hitting. The music drives everything: one analyzer reads it once, the orchestrator lays out the frames and fills a per-frame plan, and one sub-agent builds each frame. Typography and templates are the floor — a complete video needs zero assets — but any images or videos the user supplies are cut into the frames on the same beat grid (beat-cut / ken-burns). The genre (lyric video, slideshow, kinetic promo) falls out of the per-frame choices; the pipeline never branches on it.

Army161/OpenMontage

Best practices for Remotion - Video creation in React

Army161/OpenMontage

Toolkit-specific Remotion patterns — custom transitions, shared components, and project conventions. For core Remotion framework knowledge (hooks, animations, rendering, etc.), see the `remotion-official` skill.

Army161/OpenMontage

Port an existing Remotion (React) composition to HyperFrames HTML. Use ONLY when the user explicitly asks to port/convert/migrate/translate a Remotion source. Do NOT use: (a) authoring a new HyperFrames composition; (b) Remotion mentioned in passing; (c) Remotion code shared as reference only; (d) "same video as my Remotion one" without explicit migrate request — treat as fresh build. Doubt → `/general-video`. One-way, Remotion-only: no reverse export (HyperFrames→Remotion or any framework), no non-Remotion source (After Effects, Framer Motion, plain React/CSS) → out of scope, re-create via `/general-video`. Flags unsupported patterns (useState, useEffect, async calculateMetadata, third-party React libs, `@remotion/lambda`) and recommends runtime interop over lossy translation. Unsure whether to port vs. build fresh, or only a passing Remotion mention? → /hyperframes.

Army161/OpenMontage

Animate SVG character rigs with GSAP, CSS transforms, Remotion frame control, and HyperFrames-compatible browser previews.

相關技能