Communitygithub.com

9018/mychat

使用 claude-code-video-toolkit 自主创建专业视频 — 支持 AI 语音旁白、图像生成、背景音乐、虚拟主播以及 Remotion 渲染。

mychat 是什么?

该技能(名称:9018/mychat)提供了一个完整的工具包,用于从文本简报中自主创建专业的解释型视频。它利用 Modal 或 RunPod 等云 GPU 平台上的开源 AI 模型,处理语音合成(TTS)、图像生成、背景音乐生成和虚拟主播动画。视频的合成与渲染通过 Remotion(一个基于 React 的框架)完成。该技能支持多种视频模板,如产品演示和冲刺回顾,并详细指导用户完成从环境配置、云端端点部署,到逐资产生成(每场景语音、图像、剪辑)以及最终时间同步和渲染的完整工作流。它特别强调所有命令必须从固定工具根目录执行,并建议使用 Cloudflare R2 进行可靠的跨平台文件传输。用户可以完全控制语音、音调和视觉风格,从而快速制作高质量、连贯的营销或解释型视频。

兼容平台✓Claude Code~Codex CLI~Cursor
npx skills add https://github.com/9018/mychat/tree/HEAD/OpenMontage/.agents/skills/video-toolkit

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

视频工具包

根据文本简报创建专业解释型视频。该工具包使用云 GPU(Modal 或 RunPod)上的开源 AI 模型进行语音旁白、图像生成、背景音乐和虚拟主播动画。Remotion(React)负责合成和渲染。

关键:工具包路径

工具包位于固定路径。在运行任何工具命令前,务必先 cd 到这里。

TOOLKIT=~/.openclaw/workspace/claude-code-video-toolkit
cd $TOOLKIT

永远不要在项目目录内运行工具命令。 工具相对于工具包根目录解析路径。

设置

步骤 1:检查当前状态

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/verify_setup.py

如果所有检查项都显示 [x],请跳到下面的“快速测试”。否则继续设置。

步骤 2:安装 Python 依赖

cd ~/.openclaw/workspace/claude-code-video-toolkit
pip3 install --break-system-packages -r tools/requirements.txt

注意:在 Debian/Ubuntu 上使用受管理的 Python(PEP 668)时需要 --break-system-packages 选项。在容器内是安全的。

步骤 3:配置云 GPU 端点

工具包需要在 .env 文件中配置云 GPU 端点 URL。检查 .env 是否存在并包含 Modal 端点:

cat ~/.openclaw/workspace/claude-code-video-toolkit/.env | grep MODAL

如果 Modal 端点已配置,则准备就绪。如果没有,请询问用户提供 Modal 端点 URL 或设置 Modal:

pip3 install --break-system-packages modal
python3 -m modal setup   # 打开浏览器进行身份验证

# 部署每个工具 — 从输出中捕获端点 URL
cd ~/.openclaw/workspace/claude-code-video-toolkit
modal deploy docker/modal-qwen3-tts/app.py
modal deploy docker/modal-flux2/app.py
modal deploy docker/modal-music-gen/app.py
modal deploy docker/modal-sadtalker/app.py
modal deploy docker/modal-image-edit/app.py
modal deploy docker/modal-upscale/app.py
modal deploy docker/modal-propainter/app.py
modal deploy docker/modal-ltx2/app.py      # 需要:modal secret create huggingface-token HF_TOKEN=hf_...

LTX-2 前提条件: 在部署 LTX-2 之前,创建一个 HuggingFace 密钥并接受 Gemma 3 许可证:

modal secret create huggingface-token HF_TOKEN=hf_your_read_access_token

将每个 URL 添加到 .env 文件:

MODAL_QWEN3_TTS_ENDPOINT_URL=https://...modal.run
MODAL_FLUX2_ENDPOINT_URL=https://...modal.run
MODAL_MUSIC_GEN_ENDPOINT_URL=https://...modal.run
MODAL_SADTALKER_ENDPOINT_URL=https://...modal.run
MODAL_IMAGE_EDIT_ENDPOINT_URL=https://...modal.run
MODAL_UPSCALE_ENDPOINT_URL=https://...modal.run
MODAL_DEWATERMARK_ENDPOINT_URL=https://...modal.run
MODAL_LTX2_ENDPOINT_URL=https://...modal.run

可选但推荐 — 使用 Cloudflare R2 实现可靠的文件传输:

R2_ACCOUNT_ID=...
R2_ACCESS_KEY_ID=...
R2_SECRET_ACCESS_KEY=...
R2_BUCKET_NAME=video-toolkit

步骤 4:验证与快速测试

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/verify_setup.py

所有工具应显示 [x]。然后运行快速测试以确认 GPU 管道工作正常:

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/qwen3_tts.py --text "Hello, this is a test." --speaker Ryan --tone warm --output /tmp/video-toolkit-test.mp3 --cloud modal

如果获得有效的 .mp3 文件,则设置完成。如果失败,请检查:

  • .env 文件中是否有正确的 MODAL_QWEN3_TTS_ENDPOINT_URL
  • 运行 python3 tools/verify_setup.py --json 并检查 modal_tools 以确定缺少哪些端点

费用: Modal 包含每月 30 美元的免费计算额度。一个典型的 60 秒视频成本为 1-3 美元。


创建视频

步骤 1:创建项目

cd ~/.openclaw/workspace/claude-code-video-toolkit
cp -r templates/product-demo projects/PROJECT_NAME
cd projects/PROJECT_NAME
npm install

模板:product-demo(营销/解释型)、sprint-review、sprint-review-v2(可组合场景)。

步骤 2:编写配置

编辑 projects/PROJECT_NAME/src/config/demo-config.ts:

export const demoConfig: ProductDemoConfig = {
  product: {
    name: 'My Product',
    tagline: 'What it does in one line',
    website: 'example.com',
  },
  scenes: [
    { type: 'title', durationSeconds: 9, content: { headline: '...', subheadline: '...' } },
    { type: 'problem', durationSeconds: 14, content: { headline: '...', problems: ['...', '...'] } },
    { type: 'solution', durationSeconds: 13, content: { headline: '...', highlights: ['...', '...'] } },
    { type: 'stats', durationSeconds: 12, content: { stats: [{value: '99%', label: '...'}, ...] } },
    { type: 'cta', durationSeconds: 10, content: { headline: '...', links: ['...'] } },
  ],
  audio: {
    backgroundMusicFile: 'audio/bg-music.mp3',
    backgroundMusicVolume: 0.12,
  },
};

场景类型:title、problem、solution、demo、feature、stats、cta。

时长规则: 估算 durationSeconds 为 ceil(word_count / 2.5) + 2。在步骤 4 生成音频后,你将调整此值。

步骤 3:编写旁白脚本

创建 projects/PROJECT_NAME/VOICEOVER-SCRIPT.md:

## Scene 1: Title (9s, ~17 words)
Build videos with AI. The product name toolkit makes it easy.

## Scene 2: Problem (14s, ~30 words)
The problem statement goes here. Keep it punchy and relatable.

每场景词数预算: (durationSeconds - 2) * 2.5 个单词。-2 考虑了 1 秒音频延迟加 1 秒填充。

步骤 4:生成素材

关键:以下所有命令必须从工具包根目录运行,而不是项目目录。

cd ~/.openclaw/workspace/claude-code-video-toolkit

4a. 背景音乐

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/music_gen.py \
  --preset corporate-bg \
  --duration 90 \
  --output projects/PROJECT_NAME/public/audio/bg-music.mp3 \
  --cloud modal

预设:corporate-bg、upbeat-tech、ambient、dramatic、tension、hopeful、cta、lofi。

4b. 旁白(每场景)

为每个场景生成一个 .mp3 文件。不要生成单个旁白文件。

cd ~/.openclaw/workspace/claude-code-video-toolkit

# 场景 01
python3 tools/qwen3_tts.py \
  --text "The voiceover text for scene one." \
  --speaker Ryan --tone warm \
  --output projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --cloud modal

# 场景 02
python3 tools/qwen3_tts.py \
  --text "The voiceover text for scene two." \
  --speaker Ryan --tone warm \
  --output projects/PROJECT_NAME/public/audio/scenes/02.mp3 \
  --cloud modal

# ... 为每个场景重复

演讲者: Ryan、Aiden、Vivian、Serena、Uncle_Fu、Dylan、Eric、Ono_Anna、Sohee 语调: neutral、warm、professional、excited、calm、serious、storyteller、tutorial

对于语音克隆(需要参考录音):

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/qwen3_tts.py \
  --text "Text to speak" \
  --ref-audio assets/voices/reference.m4a \
  --ref-text "Exact transcript of the reference audio" \
  --output projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --cloud modal

4c. 场景图像

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/flux2.py \
  --prompt "Dark tech background with blue geometric grid, cinematic lighting" \
  --width 1920 --height 1080 \
  --output projects/PROJECT_NAME/public/images/title-bg.png \
  --cloud modal

图像预设(使用 --preset 代替 --prompt --width --height): title-bg、problem、solution、demo-bg、stats-bg、cta、thumbnail、portrait-bg

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/flux2.py \
  --preset title-bg \
  --output projects/PROJECT_NAME/public/images/title-bg.png \
  --cloud modal

4d. 视频剪辑 — B 卷与动画背景(可选)

生成 AI 视频剪辑用于 B 卷切换、动画幻灯片背景或片头/片尾序列:

cd ~/.openclaw/workspace/claude-code-video-toolkit

# 根据文本生成 B 卷剪辑
python3 tools/ltx2.py \
  --prompt "Aerial drone shot over a European city at golden hour, cinematic wide angle" \
  --output projects/PROJECT_NAME/public/videos/broll-europe.mp4 \
  --cloud modal

# 将幻灯片/截图动画化(图像转视频)
python3 tools/ltx2.py \
  --prompt "Gentle particle effects, soft ambient light shifts, very slight camera drift" \
  --input projects/PROJECT_NAME/public/images/title-bg.png \
  --output projects/PROJECT_NAME/public/videos/animated-title.mp4 \
  --cloud modal

# 抽象片头/片尾背景
python3 tools/ltx2.py \
  --prompt "Dark moody abstract background with flowing blue light streaks, bokeh particles, cinematic" \
  --output projects/PROJECT_NAME/public/videos/intro-bg.mp4 \
  --cloud modal

在 Remotion 合成中使用 <OffthreadVideo>:

<OffthreadVideo src={staticFile('videos/broll-europe.mp4')} />

LTX-2 规则:

  • 每个剪辑最多约 8 秒(24fps 下 193 帧)。默认约 5 秒(121 帧)。
  • 宽度/高度必须能被 64 整除。默认:768x512。
  • 每个剪辑约 $0.20-0.25,生成时间约 2.5 分钟。
  • 冷启动约 60-90 秒。后续剪辑在 GPU 预热后更快。
  • 生成的音频仅为环境音 — 请使用旁白/音乐工具处理语音和音乐。
  • 约 30% 的生成可能包含训练数据伪影(徽标/文字)。使用 --seed 重新运行以变化。

4e. 虚拟主播旁白(可选)

生成一张主播肖像,然后为每个场景制作动画剪辑:

cd ~/.openclaw/workspace/claude-code-video-toolkit

# 1. 生成肖像
python3 tools/flux2.py \
  --prompt "Professional presenter portrait, clean style, dark background, facing camera, upper body" \
  --width 1024 --height 576 \
  --output projects/PROJECT_NAME/public/images/presenter.png \
  --cloud modal

# 2. 生成每场景主播剪辑(每个场景一个,而不是单个长视频)
python3 tools/sadtalker.py \
  --image projects/PROJECT_NAME/public/images/presenter.png \
  --audio projects/PROJECT_NAME/public/audio/scenes/01.mp3 \
  --preprocess full --still --expression-scale 0.8 \
  --output projects/PROJECT_NAME/public/narrator-01.mp4 \
  --cloud modal

# 为每个需要主播的场景重复

SadTalker 规则 — 务必严格遵守:

  • 始终使用 --preprocess full(默认 crop 会输出正方形,宽高比错误)
  • 始终使用 --still(减少头部运动,看起来更专业)
  • 始终生成每场景剪辑(每个 6-15 秒),绝不生成单个长视频
  • 处理时间:在 Modal A10G 上,每 10 秒音频约 3-4 分钟
  • --expression-scale 0.8 保持表情细微(范围 0.0-1.5)

4f. 图像编辑(可选)

从现有图像创建场景变体:

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/image_edit.py \
  --input projects/PROJECT_NAME/public/images/title-bg.png \
  --prompt "Make it darker with red tones, more ominous" \
  --output projects/PROJECT_NAME/public/images/problem-bg.png \
  --cloud modal

4g. 放大(可选)

cd ~/.openclaw/workspace/claude-code-video-toolkit
python3 tools/upscale.py \
  --input projects/PROJECT_NAME/public/images/some-image.png \
  --output projects/PROJECT_NAME/public/images/some-image-4x.png \
  --scale 4 --cloud modal

步骤 5:同步时间

生成旁白后务必执行此步骤。 音频时长可能不同于估算值。

cd ~/.openclaw/workspace/claude-code-video-toolkit
for f in projects/PROJECT_NAME/public/audio/scenes/*.mp3; do
  echo "$(basename $f): $(ffprobe -v error -show_entries format=duration -of csv=p=0 "$f")s"
done

在 demo-config.ts 中将每个场景的 durationSeconds 更新为:ceil(实际音频时长 + 2)。

例如:如果 01.mp3 是 6.8 秒,则将场景 1 的 durationSeconds 设置为 9(ceil(6.8 + 2) = 9)。

步骤 6:检查静态帧

cd ~/.openclaw/workspace/claude-code-video-toolkit/projects/PROJECT_NAME
npx remotion still src/index.ts ProductDemo --frame=100 --output=/tmp/review-scene1.png
npx remotion still src/index.ts ProductDemo --frame=400 --output=/tmp/review-scene2.png

检查:文本截断、动画时间、主播画中画定位、背景对比度。

步骤 7:渲染

cd ~/.openclaw/workspace/claude-code-video-toolkit/projects/PROJECT_NAME
npm run render

输出: out/ProductDemo.mp4


合成模式

每场景音频

使用 1 秒延迟的每场景音频(from={30} = 30 帧 = 1s at 30fps):

<Sequence from={30}>
  <Audio src={staticFile('audio/scenes/01.mp3')} volume={1} />
</Sequence>

每场景主播画中画

<Sequence from={30}>
  <OffthreadVideo
    src={staticFile('narrator-01.mp4')}
    style={{ width: 320, height: 180, objectFit: 'cover' }}
    muted
  />
</Sequence>

始终使用 <OffthreadVideo>,绝不使用 <video>。 Remotion 需要其自己的组件以实现逐帧精确渲染。

转场

import { TransitionSeries, linearTiming } from '@remotion/transitions';
import { fade } from '@remotion/transitions/fade';
import { glitch } from '../../../lib/transitions/presentations/glitch';
import { lightLeak } from '../../../lib/transitions/presentations/light-leak';

绝不从 lib/transitions 桶导入 — 直接从 lib/transitions/presentations/ 导入自定义转场。


错误恢复

问题解决方案
工具命令失败,提示 "No module named..."从工具包根目录运行 pip3 install --break-system-packages -r tools/requirements.txt
"MODAL_*_ENDPOINT_URL not configured"检查 .env 是否包含端点 URL。运行 python3 tools/verify_setup.py
SadTalker 输出为方形/被裁剪你忘记使用 --preprocess full。请使用该标志重新运行
音频对于场景来说太短/太长重新执行步骤 5(同步时间)并更新配置
npm run render 失败确保你在项目目录中,而不是工具包根目录。先运行 npm install
Remotion 中 "Cannot find module"检查导入路径是否正确。请确保从 lib/transitions/presentations/ 导入

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

9018/mychat

Generate AI videos from text prompts using multiple provider gateways. Use when: (1) Generating videos from text descriptions, (2) Creating AI-generated video clips for content production, (3) Image-to-video generation with a reference image, (4) Choosing between video generation providers (VEO, Kling, Sora, Runway, Seedance, MiniMax, Gemini Omni). Supports gateways: HeyGen API, fal.ai API, Kling official direct API, and the Gemini API (Gemini Omni Flash).

9018/mychat

Create AI avatar videos with precise control over avatars, voices, scripts, scenes, and backgrounds using HeyGen's v2 API. Use when: (1) Choosing a specific avatar and voice for a video, (2) Writing exact scripts for an avatar to speak, (3) Building multi-scene videos with different backgrounds per scene, (4) Creating transparent WebM videos for compositing, (5) Using talking photos as video presenters, (6) Integrating HeyGen avatars with Remotion, (7) Batch video generation with exact specs, (8) Brand-consistent production videos with precise control.

9018/mychat

Use p5.js/canvas for local procedural character effects: particles, weather, squash/stretch, walk cycles, and environmental motion.

9018/mychat

Review local character animation with schema checks, Playwright browser previews, frame sampling, and FFmpeg/ffprobe final output checks.

9018/mychat

Create videos from a text prompt using HeyGen's Video Agent. Use when: (1) Creating a video from a description or idea, (2) Generating explainer, demo, or marketing videos from a prompt, (3) Making a video without specifying exact avatars, voices, or scenes, (4) Quick video prototyping or drafts, (5) One-shot prompt-to-video generation, (6) User says "make me a video" or "create a video about X".

9018/mychat

Generate Mandarin and multilingual narration with Volcengine Doubao Speech 2.0. Use when creating Chinese voiceovers, when the user prefers Doubao/Volcengine/火山引擎/豆包 TTS, or when narration needs character-level timestamp metadata for subtitles.

9018/mychat

Video and audio processing with FFmpeg. Use for format conversion, resizing, compression, audio extraction, and preparing assets for Remotion. Triggers include converting GIF to MP4, resizing video, extracting audio, compressing files, or any media transformation task.

9018/mychat

Use when implementing Disney's 12 animation principles with Framer Motion in React applications

9018/mychat

All animation knowledge for HyperFrames — atomic motion rules, multi-phase scene blueprints, scene transitions, broader motion-design techniques, AND the seven runtime adapters (GSAP default, plus Lottie, Three.js, Anime.js, CSS keyframes, Web Animations API, TypeGPU). Use for any motion or animation task: pick 2-4 rules and compose, or load a blueprint, or look up runtime-specific API (e.g. GSAP eases / Lottie player / Three.js mixer). HyperFrames-native: single paused timeline, seek-safe, deterministic.

9018/mychat

HyperFrames CLI dev loop. Use when running npx hyperframes init, add, catalog, capture, lint, validate, inspect, layout, snapshot, preview, play, render, publish, lambda, doctor, browser, info, upgrade, skills, compositions, docs, benchmark, telemetry, transcribe, tts, or remove-background, or when troubleshooting the HyperFrames build/render environment. Entry point for AWS Lambda cloud rendering (`hyperframes lambda deploy / render / progress / destroy / policies`).

9018/mychat

The HyperFrames composition contract — build one renderable project. Use for composition structure, the `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, deterministic-render rules, and validation. Read before writing composition HTML.

9018/mychat

Non-animation creative direction for HyperFrames videos. Use for design spec (frame.md / design.md) handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns, and brand / style decisions. For atomic motion patterns and scene blueprints, use `hyperframes-animation`.

9018/mychat

Audio and media assets for HyperFrames compositions, produced by one shared audio engine (`scripts/audio.mjs`) — multi-provider TTS (HeyGen / ElevenLabs / Kokoro local), background music + sound effects (HeyGen audio-library retrieval by default, with local Lyria / MusicGen BGM generation and a bundled SFX library as the no-credential fallback), Whisper transcription, background removal, and caption authoring. Use for voiceover / TTS, BGM, SFX / sound effects, transcription, captions / subtitles / lyrics / karaoke / per-word styling, voice + provider selection, and music-mood prompting.

9018/mychat

Install and wire registry blocks and components into HyperFrames compositions. Use when running hyperframes add, installing a block or component, wiring an installed item into index.html, or working with hyperframes.json. Covers the add command, install locations, block sub-composition wiring, component snippet merging, registry discovery, and authoring a new block or component to contribute upstream (idea → scaffold → validate → PR).

9018/mychat

Use when the user wants a short, design-led motion graphic where motion is the message: kinetic typography, stat or number count-up, chart/data-viz hit, logo sting, brand lockup, lower-third, callout, social overlay, animated headline/tweet/news item, motion poster, or quick captured-page highlight. Usually under 10s and up to ~30s, with no narration arc, voice-over, or live-action subject. Can render to MP4 or transparent overlay. Not for longer, multi-scene, narrated, or brand-reel pieces (use general-video), narrated website videos (website-to-video), topic explainers (faceless-explainer), product promos (product-launch-video), PR videos (pr-to-video), or captions on existing footage (embedded-captions). When unsure whether it's a quick motion-first piece or a longer / narrated treatment, see /hyperframes.

9018/mychat

Use when the user has a music track (an audio file, or a video to pull audio from) and wants a beat-synced HyperFrames video, calm to hard-hitting. The music drives everything: one analyzer reads it once, the orchestrator lays out the frames and fills a per-frame plan, and one sub-agent builds each frame. Typography and templates are the floor — a complete video needs zero assets — but any images or videos the user supplies are cut into the frames on the same beat grid (beat-cut / ken-burns). The genre (lyric video, slideshow, kinetic promo) falls out of the per-frame choices; the pipeline never branches on it.

9018/mychat

Toolkit-specific Remotion patterns — custom transitions, shared components, and project conventions. For core Remotion framework knowledge (hooks, animations, rendering, etc.), see the `remotion-official` skill.

9018/mychat

Port an existing Remotion (React) composition to HyperFrames HTML. Use ONLY when the user explicitly asks to port/convert/migrate/translate a Remotion source. Do NOT use: (a) authoring a new HyperFrames composition; (b) Remotion mentioned in passing; (c) Remotion code shared as reference only; (d) "same video as my Remotion one" without explicit migrate request — treat as fresh build. Doubt → `/general-video`. One-way, Remotion-only: no reverse export (HyperFrames→Remotion or any framework), no non-Remotion source (After Effects, Framer Motion, plain React/CSS) → out of scope, re-create via `/general-video`. Flags unsupported patterns (useState, useEffect, async calculateMetadata, third-party React libs, `@remotion/lambda`) and recommends runtime interop over lossy translation. Unsure whether to port vs. build fresh, or only a passing Remotion mention? → /hyperframes.

9018/mychat

Animate SVG character rigs with GSAP, CSS transforms, Remotion frame control, and HyperFrames-compatible browser previews.

9018/mychat

Three.js animation - keyframe animation, skeletal animation, morph targets, animation mixing. Use when animating objects, playing GLTF animations, creating procedural motion, or blending animations.

相关技能