Communitygithub.com

Aseel1600/Edit01

使用 claude-code-video-toolkit 自主创建专业视频——AI 配音、图像生成、音乐、虚拟主播和 Remotion 渲染。

Edit01 是什么?

该工具包让 AI 开发者能够通过一个文本简报全自动地、端到端地制作专业解说视频。它利用开源 AI 模型和云端 GPU(如 Modal 或 RunPod)来处理语音合成(Qwen3-TTS)、图像生成(Flux2)、背景音乐生成以及虚拟主播动画(SadTalker)。视频合成和最终渲染由基于 React 的 Remotion 框架完成,支持每个场景的精确帧控制、同步配音和动态效果。它支持多场景模板(产品演示、冲刺回顾),并提供了严格的验证流程和故障恢复指南,确保在任何给定项目开始之前,所有云端端点、Python 依赖和环境变量都已正确配置。

兼容平台✓Claude Code~Codex CLI~Cursor
npx skills add https://github.com/Aseel1600/Edit01/tree/HEAD/.agents/skills/video-toolkit

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

视频工具包

根据文本简报创建专业的解说视频。该工具包在云端 GPU(Modal 或 RunPod)上使用开源 AI 模型进行配音、图像生成、音乐和虚拟主播动画。Remotion (React) 负责合成和渲染。

关键:工具包路径

工具包位于一个固定路径。在运行任何工具命令之前,务必先 cd 到此路径。

TOOLKIT=~/.openclaw/workspace/claude-code-video-toolkit
cd $TOOLKIT

切勿在项目目录内运行工具命令。 工具会相对于工具包根目录解析路径。

设置

步骤 1:检查当前状态

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/verify_setup.py

如果所有内容都显示 [x],请跳到下方的“快速测试”。否则,继续设置。

步骤 2:安装 Python 依赖

cd ~/.openclaw/workspace/claude-code-video-toolkit
pip3 install --break-system-packages -r tools/requirements.txt

注意:在具有托管 Python 的 Debian/Ubuntu 系统上(PEP 668),需要使用 --break-system-packages。在容器内是安全的。

步骤 3:配置云端 GPU 端点

工具包需要在 .env 文件中配置云端 GPU 端点 URL。检查 .env 是否存在并包含 Modal 端点:

cat ~/.openclaw/workspace/claude-code-video-toolkit/.env | grep MODAL

如果 Modal 端点已配置,则准备就绪。如果没有,请要求用户提供 Modal 端点 URL 或设置 Modal:

pip3 install --break-system-packages modal
python3 -m modal setup   # 打开浏览器进行身份验证

# 部署每个工具——从输出中获取端点 URL
cd ~/.openclaw/workspace/claude-code-video-toolkit
modal deploy docker/modal-qwen3-tts/app.py
modal deploy docker/modal-flux2/app.py
modal deploy docker/modal-music-gen/app.py
modal deploy docker/modal-sadtalker/app.py
modal deploy docker/modal-image-edit/app.py
modal deploy docker/modal-upscale/app.py
modal deploy docker/modal-propainter/app.py
modal deploy docker/modal-ltx2/app.py      # 需要:modal secret create huggingface-token HF_TOKEN=hf_...

LTX-2 先决条件: 在部署 LTX-2 之前,创建一个 HuggingFace 密钥并接受 Gemma 3 许可:

modal secret create huggingface-token HF_TOKEN=hf_your_read_access_token

将每个 URL 添加到 .env:

MODAL_QWEN3_TTS_ENDPOINT_URL=https://...modal.run
MODAL_FLUX2_ENDPOINT_URL=https://...modal.run
MODAL_MUSIC_GEN_ENDPOINT_URL=https://...modal.run
MODAL_SADTALKER_ENDPOINT_URL=https://...modal.run
MODAL_IMAGE_EDIT_ENDPOINT_URL=https://...modal.run
MODAL_UPSCALE_ENDPOINT_URL=https://...modal.run
MODAL_DEWATERMARK_ENDPOINT_URL=https://...modal.run
MODAL_LTX2_ENDPOINT_URL=https://...modal.run

可选但推荐——用于可靠文件传输的 Cloudflare R2:

R2_ACCOUNT_ID=...
R2_ACCESS_KEY_ID=...
R2_SECRET_ACCESS_KEY=...
R2_BUCKET_NAME=video-toolkit

步骤 4:验证并快速测试

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/verify_setup.py

所有工具应显示 [x]。然后运行一个快速测试以确认 GPU 流水线正常工作:

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/qwen3_tts.py --text "Hello, this is a test." --speaker Ryan --tone warm --output /tmp/video-toolkit-test.mp3 --cloud modal

如果您获得一个有效的 .mp3 文件,则设置完成。如果失败,请检查:

  • .env 包含正确的 MODAL_QWEN3_TTS_ENDPOINT_URL
  • 运行 python3 tools/verify_setup.py --json 并检查 modal_tools 以了解哪些端点缺失

成本: Modal 每月包含 30 美元的免费计算额度。一个典型的 60 秒视频成本为 1-3 美元。


创建视频

步骤 1:创建项目

cd ~/.openclaw/workspace/claude-code-video-toolkit
cp -r templates/product-demo projects/PROJECT_NAME
cd projects/PROJECT_NAME
npm install

模板:product-demo(营销/解说),sprint-review,sprint-review-v2(可组合场景)。

步骤 2:编写配置

编辑 projects/PROJECT_NAME/src/config/demo-config.ts:

export const demoConfig: ProductDemoConfig = {
  product: {
    name: 'My Product',
    tagline: 'What it does in one line',
    website: 'example.com',
  },
  scenes: [
    { type: 'title', durationSeconds: 9, content: { headline: '...', subheadline: '...' } },
    { type: 'problem', durationSeconds: 14, content: { headline: '...', problems: ['...', '...'] } },
    { type: 'solution', durationSeconds: 13, content: { headline: '...', highlights: ['...', '...'] } },
    { type: 'stats', durationSeconds: 12, content: { stats: [{value: '99%', label: '...'}, ...] } },
    { type: 'cta', durationSeconds: 10, content: { headline: '...', links: ['...'] } },
  ],
  audio: {
    backgroundMusicFile: 'audio/bg-music.mp3',
    backgroundMusicVolume: 0.12,
  },
};

场景类型:title,problem,solution,demo,feature,stats,cta。

时长规则: 将 durationSeconds 估算为 ceil(word_count / 2.5) + 2。您将在步骤 4 生成音频后调整此项。

步骤 3:编写配音脚本

创建 projects/PROJECT_NAME/VOICEOVER-SCRIPT.md:

## 场景 1: 标题 (9s, ~17 词)
使用 AI 构建视频。产品名称工具包使其变得简单。

## 场景 2: 问题 (14s, ~30 词)
问题陈述写在这里。保持简洁且 relatable。

每个场景的词汇预算: (durationSeconds - 2) * 2.5 个词。减去 2 是为了考虑 1 秒音频延迟和 1 秒填充。

步骤 4:生成素材

关键:以下所有命令必须从工具包根目录运行,而不是项目目录。

cd ~/.openclaw/workspace/claude-code-video-toolkit

4a. 背景音乐

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/music_gen.py \
  --preset corporate-bg \
  --duration 90 \
  --output projects/PROJECT_NAME/public/audio/bg-music.mp3 \
  --cloud modal

预设:corporate-bg,upbeat-tech,ambient,dramatic,tension,hopeful,cta,lofi。

4b. 配音(按场景)

为每个场景生成一个 .mp3 文件。不要生成单个配音文件。

cd ~/.openclaw/workspace/claude-code-video-toolkit

# 场景 01
python3 tools/qwen3_tts.py \
  --text "The voiceover text for scene one." \
  --speaker Ryan --tone warm \
  --output projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --cloud modal

# 场景 02
python3 tools/qwen3_tts.py \
  --text "The voiceover text for scene two." \
  --speaker Ryan --tone warm \
  --output projects/PROJECT_NAME/public/audio/scenes/02.mp3 \
  --cloud modal

# ... 为每个场景重复

发言人: Ryan,Aiden,Vivian,Serena,Uncle_Fu,Dylan,Eric,Ono_Anna,Sohee 语调: neutral,warm,professional,excited,calm,serious,storyteller,tutorial

对于语音克隆(需要参考录音):

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/qwen3_tts.py \
  --text "Text to speak" \
  --ref-audio assets/voices/reference.m4a \
  --ref-text "Exact transcript of the reference audio" \
  --output projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --cloud modal

4c. 场景图像

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/flux2.py \
  --prompt "Dark tech background with blue geometric grid, cinematic lighting" \
  --width 1920 --height 1080 \
  --output projects/PROJECT_NAME/public/images/title-bg.png \
  --cloud modal

图像预设(使用 --preset 代替 --prompt --width --height): title-bg,problem,solution,demo-bg,stats-bg,cta,thumbnail,portrait-bg

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/flux2.py \
  --preset title-bg \
  --output projects/PROJECT_NAME/public/images/title-bg.png \
  --cloud modal

4d. 视频片段——B-Roll 和动画背景(可选)

生成 AI 视频片段,用于 B-roll 切出、动画幻灯片背景或开场/结束序列:

cd ~/.openclaw/workspace/claude-code-video-toolkit

# 从文本生成 B-roll 片段
python3 tools/ltx2.py \
  --prompt "Aerial drone shot over a European city at golden hour, cinematic wide angle" \
  --output projects/PROJECT_NAME/public/videos/broll-europe.mp4 \
  --cloud modal

# 为幻灯片/截图添加动画(图像转视频)
python3 tools/ltx2.py \
  --prompt "Gentle particle effects, soft ambient light shifts, very slight camera drift" \
  --input projects/PROJECT_NAME/public/images/title-bg.png \
  --output projects/PROJECT_NAME/public/videos/animated-title.mp4 \
  --cloud modal

# 抽象开场/结束背景
python3 tools/ltx2.py \
  --prompt "Dark moody abstract background with flowing blue light streaks, bokeh particles, cinematic" \
  --output projects/PROJECT_NAME/public/videos/intro-bg.mp4 \
  --cloud modal

在 Remotion 合成中使用 <OffthreadVideo>:

<OffthreadVideo src={staticFile('videos/broll-europe.mp4')} />

LTX-2 规则:

  • 每个片段最长约 8 秒(24fps 下 193 帧)。默认约 5 秒(121 帧)。
  • 宽/高必须能被 64 整除。默认值:768x512。
  • 每个片段约 0.20-0.25 美元,生成时间约 2.5 分钟。
  • 冷启动约 60-90 秒。后续片段在热 GPU 上运行更快。
  • 生成的音频仅为环境音——语音和音乐请使用配音/音乐工具。
  • 约 30% 的生成可能会有训练数据伪影(标志/文本)。使用 --seed 重新运行以进行变化。

4e. 虚拟主播旁白(可选)

先生成主持人肖像,然后为每个场景生成动画片段:

cd ~/.openclaw/workspace/claude-code-video-toolkit

# 1. 生成肖像
python3 tools/flux2.py \
  --prompt "Professional presenter portrait, clean style, dark background, facing camera, upper body" \
  --width 1024 --height 576 \
  --output projects/PROJECT_NAME/public/images/presenter.png \
  --cloud modal

# 2. 为每个场景生成旁白片段(每个场景一个,不要一个长视频)
python3 tools/sadtalker.py \
  --image projects/PROJECT_NAME/public/images/presenter.png \
  --audio projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --preprocess full --still --expression-scale 0.8 \
  --output projects/PROJECT_NAME/public/narrator-01.mp4 \
  --cloud modal

# 为每个需要旁白的场景重复

SadTalker 规则——严格遵守:

  • 始终使用 --preprocess full(默认的 crop 会输出方形,宽高比错误)
  • 始终使用 --still(减少头部运动,看起来更专业)
  • 始终按场景生成片段(每个 6-15 秒),绝不生成一个长视频
  • 处理时间:在 Modal A10G 上,每 10 秒音频约 3-4 分钟
  • --expression-scale 0.8 保持表情微妙(范围 0.0-1.5)

4e. 图像编辑(可选)

从现有图像创建场景变体:

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/image_edit.py \
  --input projects/PROJECT_NAME/public/images/title-bg.png \
  --prompt "Make it darker with red tones, more ominous" \
  --output projects/PROJECT_NAME/public/images/problem-bg.png \
  --cloud modal

4f. 放大(可选)

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/upscale.py \
  --input projects/PROJECT_NAME/public/images/some-image.png \
  --output projects/PROJECT_NAME/public/images/some-image-4x.png \
  --scale 4 --cloud modal

步骤 5:同步时间

生成配音后务必执行此步骤。 音频时长与估算值不同。

cd ~/.openclaw/workspace/claude-code-video-toolkit
for f in projects/PROJECT_NAME/public/audio/scenes/*.mp3; do
  echo "$(basename $f): $(ffprobe -v error -show_entries format=duration -of csv=p=0 "$f")s"
done

在 demo-config.ts 中将每个场景的 durationSeconds 更新为:ceil(实际音频时长 + 2)。

例如:如果 01.mp3 为 6.8 秒,则将场景 1 的 durationSeconds 设置为 9 (ceil(6.8 + 2) = 9)。

步骤 6:检查静态帧

cd ~/.openclaw/workspace/claude-code-video-toolkit/projects/PROJECT_NAME
npx remotion still src/index.ts ProductDemo --frame=100 --output=/tmp/review-scene1.png
npx remotion still src/index.ts ProductDemo --frame=400 --output=/tmp/review-scene2.png

检查:文本截断、动画时间、旁白画中画位置、背景对比度。

步骤 7:渲染

cd ~/.openclaw/workspace/claude-code-video-toolkit/projects/PROJECT_NAME
npm run render

输出: out/ProductDemo.mp4


合成模式

按场景音频

使用带有 1 秒延迟(from={30} = 30 帧 = 30fps 下 1 秒)的按场景音频:

<Sequence from={30}>
  <Audio src={staticFile('audio/scenes/01.mp3')} volume={1} />
</Sequence>

按场景旁白画中画

<Sequence from={30}>
  <OffthreadVideo
    src={staticFile('narrator-01.mp4')}
    style={{ width: 320, height: 180, objectFit: 'cover' }}
    muted
  />
</Sequence>

始终使用 <OffthreadVideo>,切勿使用 <video>。 Remotion 需要其自己的组件来实现帧精确渲染。

转场

import { TransitionSeries, linearTiming } from '@remotion/transitions';
import { fade } from '@remotion/transitions/fade';
import { glitch } from '../../../lib/transitions/presentations/glitch';
import { lightLeak } from '../../../lib/transitions/presentations/light-leak';

切勿从 lib/transitions 包导入——直接从 lib/transitions/presentations/ 导入自定义转场。


错误恢复

问题解决方案
工具命令失败并显示 "No module named..."从工具包根目录运行 pip3 install --break-system-packages -r tools/requirements.txt
"MODAL_*_ENDPOINT_URL not configured"检查 .env 是否包含端点 URL。运行 python3 tools/verify_setup.py
SadTalker 输出为方形/被裁剪您忘记了 --preprocess full。使用该标志重新运行。
音频对于场景来说太短/太长重新运行步骤 5(同步时间)并更新配置
npm run render 失败确保您在项目目录中,而不是工具包根目录。首先运行 npm install
Remotion 中出现 "Cannot find module"检查 im

…

[为翻译截断]

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

Aseel1600/Edit01

Generate AI videos from text prompts using multiple provider gateways. Use when: (1) Generating videos from text descriptions, (2) Creating AI-generated video clips for content production, (3) Image-to-video generation with a reference image, (4) Choosing between video generation providers (VEO, Kling, Sora, Runway, Seedance, MiniMax, Gemini Omni). Supports gateways: HeyGen API, fal.ai API, Kling official direct API, and the Gemini API (Gemini Omni Flash).

Aseel1600/Edit01

Create AI avatar videos with precise control over avatars, voices, scripts, scenes, and backgrounds using HeyGen's v2 API. Use when: (1) Choosing a specific avatar and voice for a video, (2) Writing exact scripts for an avatar to speak, (3) Building multi-scene videos with different backgrounds per scene, (4) Creating transparent WebM videos for compositing, (5) Using talking photos as video presenters, (6) Integrating HeyGen avatars with Remotion, (7) Batch video generation with exact specs, (8) Brand-consistent production videos with precise control.

Aseel1600/Edit01

Use p5.js/canvas for local procedural character effects: particles, weather, squash/stretch, walk cycles, and environmental motion.

Aseel1600/Edit01

Review local character animation with schema checks, Playwright browser previews, frame sampling, and FFmpeg/ffprobe final output checks.

Aseel1600/Edit01

Create videos from a text prompt using HeyGen's Video Agent. Use when: (1) Creating a video from a description or idea, (2) Generating explainer, demo, or marketing videos from a prompt, (3) Making a video without specifying exact avatars, voices, or scenes, (4) Quick video prototyping or drafts, (5) One-shot prompt-to-video generation, (6) User says "make me a video" or "create a video about X".

Aseel1600/Edit01

Generate Mandarin and multilingual narration with Volcengine Doubao Speech 2.0. Use when creating Chinese voiceovers, when the user prefers Doubao/Volcengine/火山引擎/豆包 TTS, or when narration needs character-level timestamp metadata for subtitles.

Aseel1600/Edit01

Video and audio processing with FFmpeg. Use for format conversion, resizing, compression, audio extraction, and preparing assets for Remotion. Triggers include converting GIF to MP4, resizing video, extracting audio, compressing files, or any media transformation task.

Aseel1600/Edit01

Generate expressive, multilingual narration with fish.audio (S1 / S2-generation models) and reuse cloned voices via reference_id. Use when the user prefers fish.audio/Fish Audio TTS, wants a specific playground voice model, or needs high-emotion voice-clone narration.

Aseel1600/Edit01

Use when implementing Disney's 12 animation principles with Framer Motion in React applications

Aseel1600/Edit01

All animation knowledge for HyperFrames — atomic motion rules, multi-phase scene blueprints, scene transitions, broader motion-design techniques, AND the seven runtime adapters (GSAP default, plus Lottie, Three.js, Anime.js, CSS keyframes, Web Animations API, TypeGPU). Use for any motion or animation task: pick 2-4 rules and compose, or load a blueprint, or look up runtime-specific API (e.g. GSAP eases / Lottie player / Three.js mixer). HyperFrames-native: single paused timeline, seek-safe, deterministic.

Aseel1600/Edit01

HyperFrames CLI dev loop. Use when running npx hyperframes init, add, catalog, capture, lint, validate, inspect, layout, snapshot, preview, play, render, publish, lambda, doctor, browser, info, upgrade, skills, compositions, docs, benchmark, telemetry, transcribe, tts, or remove-background, or when troubleshooting the HyperFrames build/render environment. Entry point for AWS Lambda cloud rendering (`hyperframes lambda deploy / render / progress / destroy / policies`).

Aseel1600/Edit01

The HyperFrames composition contract — build one renderable project. Use for composition structure, the `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, deterministic-render rules, and validation. Read before writing composition HTML.

Aseel1600/Edit01

Non-animation creative direction for HyperFrames videos. Use for design spec (frame.md / design.md) handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns, and brand / style decisions. For atomic motion patterns and scene blueprints, use `hyperframes-animation`.

Aseel1600/Edit01

Audio and media assets for HyperFrames compositions, produced by one shared audio engine (`scripts/audio.mjs`) — multi-provider TTS (HeyGen / ElevenLabs / Kokoro local), background music + sound effects (HeyGen audio-library retrieval by default, with local Lyria / MusicGen BGM generation and a bundled SFX library as the no-credential fallback), Whisper transcription, background removal, and caption authoring. Use for voiceover / TTS, BGM, SFX / sound effects, transcription, captions / subtitles / lyrics / karaoke / per-word styling, voice + provider selection, and music-mood prompting.

Aseel1600/Edit01

Install and wire registry blocks and components into HyperFrames compositions. Use when running hyperframes add, installing a block or component, wiring an installed item into index.html, or working with hyperframes.json. Covers the add command, install locations, block sub-composition wiring, component snippet merging, registry discovery, and authoring a new block or component to contribute upstream (idea → scaffold → validate → PR).

Aseel1600/Edit01

Use when the user wants a short, design-led motion graphic where motion is the message: kinetic typography, stat or number count-up, chart/data-viz hit, logo sting, brand lockup, lower-third, callout, social overlay, animated headline/tweet/news item, motion poster, or quick captured-page highlight. Usually under 10s and up to ~30s, with no narration arc, voice-over, or live-action subject. Can render to MP4 or transparent overlay. Not for longer, multi-scene, narrated, or brand-reel pieces (use general-video), narrated website videos (website-to-video), topic explainers (faceless-explainer), product promos (product-launch-video), PR videos (pr-to-video), or captions on existing footage (embedded-captions). When unsure whether it's a quick motion-first piece or a longer / narrated treatment, see /hyperframes.

Aseel1600/Edit01

Use when the user has a music track (an audio file, or a video to pull audio from) and wants a beat-synced HyperFrames video, calm to hard-hitting. The music drives everything: one analyzer reads it once, the orchestrator lays out the frames and fills a per-frame plan, and one sub-agent builds each frame. Typography and templates are the floor — a complete video needs zero assets — but any images or videos the user supplies are cut into the frames on the same beat grid (beat-cut / ken-burns). The genre (lyric video, slideshow, kinetic promo) falls out of the per-frame choices; the pipeline never branches on it.

Aseel1600/Edit01

Best practices for Remotion - Video creation in React

Aseel1600/Edit01

Toolkit-specific Remotion patterns — custom transitions, shared components, and project conventions. For core Remotion framework knowledge (hooks, animations, rendering, etc.), see the `remotion-official` skill.

Aseel1600/Edit01

Port an existing Remotion (React) composition to HyperFrames HTML. Use ONLY when the user explicitly asks to port/convert/migrate/translate a Remotion source. Do NOT use: (a) authoring a new HyperFrames composition; (b) Remotion mentioned in passing; (c) Remotion code shared as reference only; (d) "same video as my Remotion one" without explicit migrate request — treat as fresh build. Doubt → `/general-video`. One-way, Remotion-only: no reverse export (HyperFrames→Remotion or any framework), no non-Remotion source (After Effects, Framer Motion, plain React/CSS) → out of scope, re-create via `/general-video`. Flags unsupported patterns (useState, useEffect, async calculateMetadata, third-party React libs, `@remotion/lambda`) and recommends runtime interop over lossy translation. Unsure whether to port vs. build fresh, or only a passing Remotion mention? → /hyperframes.

相关技能