Communitygithub.com

Army161/OpenMontage

使用开源AI模型自主创建专业视频——AI配音、图像生成、音乐、虚拟主播和Remotion渲染。

OpenMontage 是什么?

这是一个强大的AI Agent技能,允许开发者利用开源AI模型在云端GPU上自动生成专业解说视频。该工具集成了文本转语音、图像生成、背景音乐和虚拟主播动画等模块,并通过Remotion(React)进行视频合成与渲染。用户只需提供文字简报,即可完成从脚本撰写、资产生成到最终渲染的全流程自动化。技能支持Modal和RunPod等云GPU平台,初始设置后,制作一个60秒的视频成本仅需1-3美元。它特别适合产品演示、营销视频和项目评审等场景,大大降低了专业视频制作的门槛。

兼容平台✓Claude Code~Codex CLI~Cursor
npx skills add https://github.com/Army161/OpenMontage/tree/HEAD/.agents/skills/video-toolkit

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

视频工具包

根据文字简报创建专业解说视频。该工具包使用云GPU上的开源AI模型(Modal 或 RunPod)进行配音、图像生成、音乐和虚拟主播动画。Remotion (React) 负责合成与渲染。

关键:工具包路径

工具包位于固定路径。在执行任何工具命令前,务必先 cd 到此目录。

TOOLKIT=~/.openclaw/workspace/claude-code-video-toolkit
cd $TOOLKIT

切勿在项目目录内运行工具命令。 工具会相对于工具包根目录解析路径。

设置

第一步:检查当前状态

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/verify_setup.py

如果一切显示 [x],请直接跳到下方的“快速测试”。否则继续设置。

第二步:安装 Python 依赖

cd ~/.openclaw/workspace/claude-code-video-toolkit
pip3 install --break-system-packages -r tools/requirements.txt

注意:在 Debian/Ubuntu 系统上使用受管理的 Python 时(PEP 668),需要使用 --break-system-packages。在容器内是安全的。

第三步:配置云 GPU 端点

工具包需要在 .env 文件中配置云 GPU 端点 URL。检查 .env 是否存在并且包含 Modal 端点:

cat ~/.openclaw/workspace/claude-code-video-toolkit/.env | grep MODAL

如果 Modal 端点已配置,则准备就绪。否则,要求用户提供 Modal 端点 URL 或设置 Modal:

pip3 install --break-system-packages modal
python3 -m modal setup   # 打开浏览器进行身份验证

# 部署每个工具 — 从输出中捕获端点 URL
cd ~/.openclaw/workspace/claude-code-video-toolkit
modal deploy docker/modal-qwen3-tts/app.py
modal deploy docker/modal-flux2/app.py
modal deploy docker/modal-music-gen/app.py
modal deploy docker/modal-sadtalker/app.py
modal deploy docker/modal-image-edit/app.py
modal deploy docker/modal-upscale/app.py
modal deploy docker/modal-propainter/app.py
modal deploy docker/modal-ltx2/app.py      # 需要: modal secret create huggingface-token HF_TOKEN=hf_...

LTX-2 前提条件: 在部署 LTX-2 之前,创建一个 HuggingFace secret 并接受 Gemma 3 许可:

modal secret create huggingface-token HF_TOKEN=hf_your_read_access_token

将每个 URL 添加到 .env 文件中:

MODAL_QWEN3_TTS_ENDPOINT_URL=https://...modal.run
MODAL_FLUX2_ENDPOINT_URL=https://...modal.run
MODAL_MUSIC_GEN_ENDPOINT_URL=https://...modal.run
MODAL_SADTALKER_ENDPOINT_URL=https://...modal.run
MODAL_IMAGE_EDIT_ENDPOINT_URL=https://...modal.run
MODAL_UPSCALE_ENDPOINT_URL=https://...modal.run
MODAL_DEWATERMARK_ENDPOINT_URL=https://...modal.run
MODAL_LTX2_ENDPOINT_URL=https://...modal.run

可选但推荐 — Cloudflare R2 用于可靠的文件传输:

R2_ACCOUNT_ID=...
R2_ACCESS_KEY_ID=...
R2_SECRET_ACCESS_KEY=...
R2_BUCKET_NAME=video-toolkit

第四步:验证和快速测试

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/verify_setup.py

所有工具都应显示 [x]。然后运行一个快速测试以确认 GPU 流水线正常工作:

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/qwen3_tts.py --text "Hello, this is a test." --speaker Ryan --tone warm --output /tmp/video-toolkit-test.mp3 --cloud modal

如果得到一个有效的 .mp3 文件,则设置完成。如果失败,请检查:

  • .env 文件中是否有正确的 MODAL_QWEN3_TTS_ENDPOINT_URL
  • 运行 python3 tools/verify_setup.py --json 并检查 modal_tools 以查看缺少哪些端点

费用: Modal 包含每月 30 美元的免费计算额度。一个典型的 60 秒视频费用为 1-3 美元。


创建视频

第一步:创建项目

cd ~/.openclaw/workspace/claude-code-video-toolkit
cp -r templates/product-demo projects/PROJECT_NAME
cd projects/PROJECT_NAME
npm install

模板:product-demo(营销/解说), sprint-review, sprint-review-v2(可组合场景)。

第二步:编写配置

编辑 projects/PROJECT_NAME/src/config/demo-config.ts:

export const demoConfig: ProductDemoConfig = {
  product: {
    name: 'My Product',
    tagline: 'What it does in one line',
    website: 'example.com',
  },
  scenes: [
    { type: 'title', durationSeconds: 9, content: { headline: '...', subheadline: '...' } },
    { type: 'problem', durationSeconds: 14, content: { headline: '...', problems: ['...', '...'] } },
    { type: 'solution', durationSeconds: 13, content: { headline: '...', highlights: ['...', '...'] } },
    { type: 'stats', durationSeconds: 12, content: { stats: [{value: '99%', label: '...'}, ...] } },
    { type: 'cta', durationSeconds: 10, content: { headline: '...', links: ['...'] } },
  ],
  audio: {
    backgroundMusicFile: 'audio/bg-music.mp3',
    backgroundMusicVolume: 0.12,
  },
};

场景类型:title, problem, solution, demo, feature, stats, cta。

持续时间规则: 将 durationSeconds 估算为 ceil(word_count / 2.5) + 2。在步骤四生成音频后,你将调整此值。

第三步:编写配音脚本

创建 projects/PROJECT_NAME/VOICEOVER-SCRIPT.md:

## 场景 1:标题 (9s, ~17 词)
Build videos with AI. The product name toolkit makes it easy.

## 场景 2:问题 (14s, ~30 词)
The problem statement goes here. Keep it punchy and relatable.

每个场景的字数预算: (durationSeconds - 2) * 2.5 个单词。减去 2 是为了应付 1 秒音频延迟加 1 秒填充。

第四步:生成资产

关键:以下所有命令必须从工具包根目录运行,而不是项目目录。

cd ~/.openclaw/workspace/claude-code-video-toolkit

4a. 背景音乐

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/music_gen.py \
  --preset corporate-bg \
  --duration 90 \
  --output projects/PROJECT_NAME/public/audio/bg-music.mp3 \
  --cloud modal

预设:corporate-bg, upbeat-tech, ambient, dramatic, tension, hopeful, cta, lofi。

4b. 配音(逐场景)

为每个场景生成一个 .mp3 文件。不要生成单个配音文件。

cd ~/.openclaw/workspace/claude-code-video-toolkit

# 场景 01
python3 tools/qwen3_tts.py \
  --text "The voiceover text for scene one." \
  --speaker Ryan --tone warm \
  --output projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --cloud modal

# 场景 02
python3 tools/qwen3_tts.py \
  --text "The voiceover text for scene two." \
  --speaker Ryan --tone warm \
  --output projects/PROJECT_NAME/public/audio/scenes/02.mp3 \
  --cloud modal

# ... 为每个场景重复

说话人: Ryan, Aiden, Vivian, Serena, Uncle_Fu, Dylan, Eric, Ono_Anna, Sohee 语气: neutral, warm, professional, excited, calm, serious, storyteller, tutorial

对于声音克隆(需要参考录音):

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/qwen3_tts.py \
  --text "Text to speak" \
  --ref-audio assets/voices/reference.m4a \
  --ref-text "Exact transcript of the reference audio" \
  --output projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --cloud modal

4c. 场景图像

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/flux2.py \
  --prompt "Dark tech background with blue geometric grid, cinematic lighting" \
  --width 1920 --height 1080 \
  --output projects/PROJECT_NAME/public/images/title-bg.png \
  --cloud modal

图像预设(使用 --preset 代替 --prompt --width --height): title-bg, problem, solution, demo-bg, stats-bg, cta, thumbnail, portrait-bg

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/flux2.py \
  --preset title-bg \
  --output projects/PROJECT_NAME/public/images/title-bg.png \
  --cloud modal

4d. 视频片段 — 补充镜头和动画背景(可选)

生成 AI 视频片段用于补充镜头切换、动画幻灯片背景或开场/结束序列:

cd ~/.openclaw/workspace/claude-code-video-toolkit

# 从文本生成补充镜头
python3 tools/ltx2.py \
  --prompt "Aerial drone shot over a European city at golden hour, cinematic wide angle" \
  --output projects/PROJECT_NAME/public/videos/broll-europe.mp4 \
  --cloud modal

# 动画化幻灯片/截图(图像到视频)
python3 tools/ltx2.py \
  --prompt "Gentle particle effects, soft ambient light shifts, very slight camera drift" \
  --input projects/PROJECT_NAME/public/images/title-bg.png \
  --output projects/PROJECT_NAME/public/videos/animated-title.mp4 \
  --cloud modal

# 抽象开场/结束背景
python3 tools/ltx2.py \
  --prompt "Dark moody abstract background with flowing blue light streaks, bokeh particles, cinematic" \
  --output projects/PROJECT_NAME/public/videos/intro-bg.mp4 \
  --cloud modal

在 Remotion 合成中使用 <OffthreadVideo>:

<OffthreadVideo src={staticFile('videos/broll-europe.mp4')} />

LTX-2 规则:

  • 每个片段最长约 8 秒(24fps 下 193 帧)。默认为约 5 秒(121 帧)。
  • 宽度/高度必须能被 64 整除。默认值:768x512。
  • 每个片段约 0.20-0.25 美元,生成时间约 2.5 分钟。
  • 冷启动约 60-90 秒。后续片段在热 GPU 上更快。
  • 生成的音频仅为环境音 — 语音和音乐请使用配音/音乐工具。
  • 约 30% 的生成结果可能包含训练数据工件(标志/文字)。使用 --seed 重新运行以获得变化。

4e. 虚拟主播解说(可选)

生成演示者肖像,然后为每个场景动画化片段:

cd ~/.openclaw/workspace/claude-code-video-toolkit

# 1. 生成肖像
python3 tools/flux2.py \
  --prompt "Professional presenter portrait, clean style, dark background, facing camera, upper body" \
  --width 1024 --height 576 \
  --output projects/PROJECT_NAME/public/images/presenter.png \
  --cloud modal

# 2. 为每个场景生成解说片段(每场景一个,而不是一个长视频)
python3 tools/sadtalker.py \
  --image projects/PROJECT_NAME/public/images/presenter.png \
  --audio projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --preprocess full --still --expression-scale 0.8 \
  --output projects/PROJECT_NAME/public/narrator-01.mp4 \
  --cloud modal

# 为每个需要解说的场景重复

SadTalker 规则 — 请严格遵循:

  • 始终使用 --preprocess full(默认的 crop 输出正方形,宽高比错误)
  • 始终使用 --still(减少头部移动,看起来更专业)
  • 始终为每个场景生成片段(6-15 秒),绝不生成一个长视频
  • 处理时间:在 Modal A10G 上,每 10 秒音频约需 3-4 分钟
  • --expression-scale 0.8 保持表情柔和(范围 0.0-1.5)

4e. 图像编辑(可选)

从现有图像创建场景变体:

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/image_edit.py \
  --input projects/PROJECT_NAME/public/images/title-bg.png \
  --prompt "Make it darker with red tones, more ominous" \
  --output projects/PROJECT_NAME/public/images/problem-bg.png \
  --cloud modal

4f. 放大(可选)

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/upscale.py \
  --input projects/PROJECT_NAME/public/images/some-image.png \
  --output projects/PROJECT_NAME/public/images/some-image-4x.png \
  --scale 4 --cloud modal

第五步:同步时间

生成配音后务必执行此步骤。 音频时长可能与估算不同。

cd ~/.openclaw/workspace/claude-code-video-toolkit
for f in projects/PROJECT_NAME/public/audio/scenes/*.mp3; do
  echo "$(basename $f): $(ffprobe -v error -show_entries format=duration -of csv=p=0 "$f")s"
done

将 demo-config.ts 中每个场景的 durationSeconds 更新为:ceil(actual_audio_duration + 2)。

例如:如果 01.mp3 是 6.8 秒,则将场景 1 的 durationSeconds 设置为 9 (ceil(6.8 + 2) = 9)。

第六步:检查静态帧

cd ~/.openclaw/workspace/claude-code-video-toolkit/projects/PROJECT_NAME
npx remotion still src/index.ts ProductDemo --frame=100 --output=/tmp/review-scene1.png
npx remotion still src/index.ts ProductDemo --frame=400 --output=/tmp/review-scene2.png

检查:文字截断、动画时间、虚拟主播画中画定位、背景对比度。

第七步:渲染

cd ~/.openclaw/workspace/claude-code-video-toolkit/projects/PROJECT_NAME
npm run render

输出: out/ProductDemo.mp4


合成模式

逐场景音频

使用带有 1 秒延迟的逐场景音频(from={30} 表示在 30fps 下为 30 帧 = 1 秒):

<Sequence from={30}>
  <Audio src={staticFile('audio/scenes/01.mp3')} volume={1} />
</Sequence>

逐场景虚拟主播画中画

<Sequence from={30}>
  <OffthreadVideo
    src={staticFile('narrator-01.mp4')}
    style={{ width: 320, height: 180, objectFit: 'cover' }}
    muted
  />
</Sequence>

始终使用 <OffthreadVideo>,绝不使用 <video>。 Remotion 需要自己的组件来实现帧精确渲染。

过渡

import { TransitionSeries, linearTiming } from '@remotion/transitions';
import { fade } from '@remotion/transitions/fade';
import { glitch } from '../../../lib/transitions/presentations/glitch';
import { lightLeak } from '../../../lib/transitions/presentations/light-leak';

绝不要从 lib/transitions 的 barrel 文件导入 —— 直接从 lib/transitions/presentations/ 导入自定义过渡。


错误恢复

问题解决方法
工具命令失败,提示 "No module named..."从工具包根目录运行 pip3 install --break-system-packages -r tools/requirements.txt
"MODAL_*_ENDPOINT_URL not configured"检查 .env 是否有端点 URL。运行 python3 tools/verify_setup.py
SadTalker 输出为方形/被裁剪你忘了加 --preprocess full。带着该标志重新运行
音频对于场景来说太短/太长重新运行第五步(同步时间)并更新配置
npm run render 失败确保你在项目目录中,而不是工具包根目录。先运行 npm install
"Cannot find module" 在 Remotion 中检查导入路径。自定义过渡应从 lib/transitions/presentations/ 直接导入

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

Army161/OpenMontage

Generate AI videos from text prompts using multiple provider gateways. Use when: (1) Generating videos from text descriptions, (2) Creating AI-generated video clips for content production, (3) Image-to-video generation with a reference image, (4) Choosing between video generation providers (VEO, Kling, Sora, Runway, Seedance, MiniMax). Supports two gateways: HeyGen API and fal.ai API.

Army161/OpenMontage

Create AI avatar videos with precise control over avatars, voices, scripts, scenes, and backgrounds using HeyGen's v2 API. Use when: (1) Choosing a specific avatar and voice for a video, (2) Writing exact scripts for an avatar to speak, (3) Building multi-scene videos with different backgrounds per scene, (4) Creating transparent WebM videos for compositing, (5) Using talking photos as video presenters, (6) Integrating HeyGen avatars with Remotion, (7) Batch video generation with exact specs, (8) Brand-consistent production videos with precise control.

Army161/OpenMontage

Use p5.js/canvas for local procedural character effects: particles, weather, squash/stretch, walk cycles, and environmental motion.

Army161/OpenMontage

Review local character animation with schema checks, Playwright browser previews, frame sampling, and FFmpeg/ffprobe final output checks.

Army161/OpenMontage

Create videos from a text prompt using HeyGen's Video Agent. Use when: (1) Creating a video from a description or idea, (2) Generating explainer, demo, or marketing videos from a prompt, (3) Making a video without specifying exact avatars, voices, or scenes, (4) Quick video prototyping or drafts, (5) One-shot prompt-to-video generation, (6) User says "make me a video" or "create a video about X".

Army161/OpenMontage

Generate Mandarin and multilingual narration with Volcengine Doubao Speech 2.0. Use when creating Chinese voiceovers, when the user prefers Doubao/Volcengine/火山引擎/豆包 TTS, or when narration needs character-level timestamp metadata for subtitles.

Army161/OpenMontage

Video and audio processing with FFmpeg. Use for format conversion, resizing, compression, audio extraction, and preparing assets for Remotion. Triggers include converting GIF to MP4, resizing video, extracting audio, compressing files, or any media transformation task.

Army161/OpenMontage

Use when implementing Disney's 12 animation principles with Framer Motion in React applications

Army161/OpenMontage

All animation knowledge for HyperFrames — atomic motion rules, multi-phase scene blueprints, scene transitions, broader motion-design techniques, AND the seven runtime adapters (GSAP default, plus Lottie, Three.js, Anime.js, CSS keyframes, Web Animations API, TypeGPU). Use for any motion or animation task: pick 2-4 rules and compose, or load a blueprint, or look up runtime-specific API (e.g. GSAP eases / Lottie player / Three.js mixer). HyperFrames-native: single paused timeline, seek-safe, deterministic.

Army161/OpenMontage

HyperFrames CLI dev loop. Use when running npx hyperframes init, add, catalog, capture, lint, validate, inspect, layout, snapshot, preview, play, render, publish, lambda, doctor, browser, info, upgrade, skills, compositions, docs, benchmark, telemetry, transcribe, tts, or remove-background, or when troubleshooting the HyperFrames build/render environment. Entry point for AWS Lambda cloud rendering (`hyperframes lambda deploy / render / progress / destroy / policies`).

Army161/OpenMontage

The HyperFrames composition contract — build one renderable project. Use for composition structure, the `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, deterministic-render rules, and validation. Read before writing composition HTML.

Army161/OpenMontage

Non-animation creative direction for HyperFrames videos. Use for design spec (frame.md / design.md) handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns, and brand / style decisions. For atomic motion patterns and scene blueprints, use `hyperframes-animation`.

Army161/OpenMontage

Audio and media assets for HyperFrames compositions, produced by one shared audio engine (`scripts/audio.mjs`) — multi-provider TTS (HeyGen / ElevenLabs / Kokoro local), background music + sound effects (HeyGen audio-library retrieval by default, with local Lyria / MusicGen BGM generation and a bundled SFX library as the no-credential fallback), Whisper transcription, background removal, and caption authoring. Use for voiceover / TTS, BGM, SFX / sound effects, transcription, captions / subtitles / lyrics / karaoke / per-word styling, voice + provider selection, and music-mood prompting.

Army161/OpenMontage

Install and wire registry blocks and components into HyperFrames compositions. Use when running hyperframes add, installing a block or component, wiring an installed item into index.html, or working with hyperframes.json. Covers the add command, install locations, block sub-composition wiring, component snippet merging, registry discovery, and authoring a new block or component to contribute upstream (idea → scaffold → validate → PR).

Army161/OpenMontage

Use when the user wants a short, design-led motion graphic where motion is the message: kinetic typography, stat or number count-up, chart/data-viz hit, logo sting, brand lockup, lower-third, callout, social overlay, animated headline/tweet/news item, motion poster, or quick captured-page highlight. Usually under 10s and up to ~30s, with no narration arc, voice-over, or live-action subject. Can render to MP4 or transparent overlay. Not for longer, multi-scene, narrated, or brand-reel pieces (use general-video), narrated website videos (website-to-video), topic explainers (faceless-explainer), product promos (product-launch-video), PR videos (pr-to-video), or captions on existing footage (embedded-captions). When unsure whether it's a quick motion-first piece or a longer / narrated treatment, see /hyperframes.

Army161/OpenMontage

Use when the user has a music track (an audio file, or a video to pull audio from) and wants a beat-synced HyperFrames video, calm to hard-hitting. The music drives everything: one analyzer reads it once, the orchestrator lays out the frames and fills a per-frame plan, and one sub-agent builds each frame. Typography and templates are the floor — a complete video needs zero assets — but any images or videos the user supplies are cut into the frames on the same beat grid (beat-cut / ken-burns). The genre (lyric video, slideshow, kinetic promo) falls out of the per-frame choices; the pipeline never branches on it.

Army161/OpenMontage

Best practices for Remotion - Video creation in React

Army161/OpenMontage

Toolkit-specific Remotion patterns — custom transitions, shared components, and project conventions. For core Remotion framework knowledge (hooks, animations, rendering, etc.), see the `remotion-official` skill.

Army161/OpenMontage

Port an existing Remotion (React) composition to HyperFrames HTML. Use ONLY when the user explicitly asks to port/convert/migrate/translate a Remotion source. Do NOT use: (a) authoring a new HyperFrames composition; (b) Remotion mentioned in passing; (c) Remotion code shared as reference only; (d) "same video as my Remotion one" without explicit migrate request — treat as fresh build. Doubt → `/general-video`. One-way, Remotion-only: no reverse export (HyperFrames→Remotion or any framework), no non-Remotion source (After Effects, Framer Motion, plain React/CSS) → out of scope, re-create via `/general-video`. Flags unsupported patterns (useState, useEffect, async calculateMetadata, third-party React libs, `@remotion/lambda`) and recommends runtime interop over lossy translation. Unsure whether to port vs. build fresh, or only a passing Remotion mention? → /hyperframes.

Army161/OpenMontage

Animate SVG character rigs with GSAP, CSS transforms, Remotion frame control, and HyperFrames-compatible browser previews.

相关技能