Communitygithub.com

ComeOnOliver/skillshub

多镜头AI视频生成管线,保持人物面部一致性。将脚本或创意转化为完整视频,包含角色提取、分镜设计、画面生成和视频合成。经过300次实验验证,面部距离改进70%。当用户要求从脚本、故事、创意生成视频,或需要角色一致的多镜头视频时使用。

skillshub 是什么?

这是一个名为ViMax的AI视频生成技能,专为HappyCapy AI Gateway设计,能够从文本脚本或简短创意自动生成完整的、多镜头视频,并确保所有场景中角色面部高度一致。该管线包含八个关键步骤:角色提取(带详细物理特征)、多角度肖像生成、逐镜头分镜设计、每个镜头的首尾帧与动作拆解、镜头关系相机树构建、基于参考帧(面部一致性为最高优先级)的图像生成、视频片段生成,以及最终视频拼接。技能已通过300次实验验证,面部相似度距离指标提升了70%(从0.74降至0.22)。它被实现为一个Python管线,使用LangChain与底层模型交互,支持多种配置(如GPT-4.1/ Gemini/ Veo等)。用户可直接运行预置脚本处理完整剧本或仅需一个想法,也可以编程方式集成,并可选择使用真实照片替代AI生成的肖像来进一步提升一致性。

兼容平台~Claude Code✓Codex CLI~Cursor✓Gemini CLI
npx skills add https://github.com/ComeOnOliver/skillshub/tree/HEAD/skills/happycapy-ai/Happycapy-skills/capy-video-gen-skill

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

Capy 视频生成技能 - 脚本转视频管线

从脚本或创意生成完整的、多镜头视频,所有场景中角色面部保持高度一致。专为 HappyCapy AI Gateway 构建。经过 300 次实验验证,面部距离改进 70%。

概述

ViMax 通过自动化管线将文本脚本转换为完整视频:

  1. 从脚本中提取角色,附带详细物理特征
  2. 为每个角色生成正面/侧面/背面肖像
  3. 设计逐镜头分镜板
  4. 将每个镜头分解为首帧、末帧和动作描述
  5. 构建用于镜头关系的相机树
  6. 使用参考图像选择(面部一致性为最高优先级)生成帧
  7. 从帧生成视频片段
  8. 拼接为最终视频

安装位置

ViMax 管线代码位于:/home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax/

所有命令都必须在此目录下使用虚拟环境运行:

cd /home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax

前提条件

  • AI_GATEWAY_API_KEY 环境变量(在 HappyCapy 中自动配置)
  • Python 虚拟环境位于 .venv/(已设置)

快速开始

脚本转视频

编辑入口脚本中的脚本、要求和样式,然后运行:

cd /home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax
.venv/bin/python main_happycapy_script2video.py

创意转视频

从简短创意生成视频(先自动生成脚本):

cd /home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax
.venv/bin/python main_happycapy_idea2video.py

编程方式使用

import asyncio
from langchain.chat_models import init_chat_model
from tools.render_backend import RenderBackend
from utils.config_loader import load_config
from pipelines.script2video_pipeline import Script2VideoPipeline

config = load_config("configs/happycapy_script2video.yaml")
chat_model = init_chat_model(**config["chat_model"]["init_args"])
backend = RenderBackend.from_config(config)

pipeline = Script2VideoPipeline(
    chat_model=chat_model,
    image_generator=backend.image_generator,
    video_generator=backend.video_generator,
    working_dir=config["working_dir"],
)

# 运行管线
asyncio.run(pipeline(
    script="你的脚本内容在此...",
    user_requirement="总共不超过8个镜头。",
    style="电影感,暖色调灯光"
))

管线

Script2VideoPipeline(脚本转视频管线)

  • 输入:格式化的剧本/脚本,包含角色对话和场景描述
  • 输出:拼接后的视频,位于 {working_dir}/final_video.mp4
  • 配置:configs/happycapy_script2video.yaml

Idea2VideoPipeline(创意转视频管线)

  • 输入:简短创意/概念(1-3段)
  • 输出:自动生成脚本,然后生成视频
  • 配置:configs/happycapy_idea2video.yaml

配置

HappyCapy 配置位于 configs/happycapy_script2video.yaml:

chat_model:
  init_args:
    model: gpt-4.1
    model_provider: openai
    api_key: ${AI_GATEWAY_API_KEY}
    base_url: https://ai-gateway.happycapy.ai/api/v1/openai/v1

image_generator:
  class_path: tools.ImageGeneratorHappyCapyAPI
  init_args:
    api_key: ${AI_GATEWAY_API_KEY}
    model: google/gemini-3.1-flash-image-preview

video_generator:
  class_path: tools.VideoGeneratorHappyCapyAPI
  init_args:
    api_key: ${AI_GATEWAY_API_KEY}
    model: google/veo-3.1-generate-preview

working_dir: .working_dir/script2video

关键组件

智能体(AI 处理)

智能体文件用途
CharacterExtractor(角色提取器)agents/character_extractor.py从脚本中提取角色,包含静态/动态特征
CharacterPortraitsGenerator(角色肖像生成器)agents/character_portraits_generator.py为每个角色生成正面/侧面/背面肖像
StoryboardArtist(分镜艺术家)agents/storyboard_artist.py设计逐镜头分镜板,包含首帧/末帧和动作
ReferenceImageSelector(参考图像选择器)agents/reference_image_selector.py为每帧选择最佳参考图像(面部一致性优先级最高)
CameraImageGenerator(相机图像生成器)agents/camera_image_generator.py构建相机树并生成过渡视频
BestImageSelector(最佳图像选择器)agents/best_image_selector.py从候选图像中选择最佳生成图像
Screenwriter(编剧)agents/screenwriter.py从创意生成脚本

工具(生成后端)

工具文件用途
ImageGeneratorHappyCapyAPItools/image_generator_happycapy_api.py通过 HappyCapy Gateway(Gemini)生成图像
VideoGeneratorHappyCapyAPItools/video_generator_happycapy_api.py通过 HappyCapy Gateway(Veo)生成视频
RenderBackend(渲染后端)tools/render_backend.py用于从配置实例化生成器的工厂

接口(数据模型)

  • CharacterInScene - 角色,包含标识符、静态特征、动态特征
  • ShotDescription - 镜头描述,包含首帧描述、末帧描述、动作描述、变化类型
  • Camera - 相机,包含父子关系
  • Frame - 帧,包含镜头索引、帧类型、可见角色
  • ImageOutput / VideoOutput - 生成输出,包含保存方法

面部一致性(关键)

本管线包含经过 257 次实验验证的面部一致性改进(面部距离从 0.74 提升至 0.22,改进幅度 70%):

内置保护机制

  1. 参考图像选择器:选择参考图像时,面部一致性的优先级最高。当角色面部可见时,始终包含正面肖像。

  2. 角色肖像:增强的提示会生成身份关键细节(精确的鼻子形状、眼距、下颚线、特征标记)以实现跨场景识别。

  3. 视频提示面部锁定:每个视频生成提示前都会加上面部一致性指令,要求角色面部在整个片段中与起始帧保持一致。

使用 ViMax 的最佳实践

  • 超详细角色描述:在脚本的角色介绍中包含种族、年龄、发质/发型/发色、眼睛形状、面部毛发、眼镜、肤色、体型和特征标记
  • 极端特写镜头:每个角色至少包含一个极端特写镜头以锚定身份
  • 一致的光照:在不同场景中指定相似的光照,防止面部漂移
  • 用户提供的参考照片:将照片放入工作目录,并通过 character_portraits_registry 参数传入,跳过 AI 肖像生成

不起作用的做法

  • 复杂的提示工程(viseme 变形、音素锚定)不会改善面部一致性
  • 包含详细物理描述的简单、直接提示优于精巧的提示
  • 无法与外部音频同步口型(Veo 会生成自己的内部音频)

详情请参阅 ViMax 目录中的 FACE_IDENTITY_GUIDE.md。

输出结构

运行后,工作目录内容如下:

.working_dir/script2video/
  characters.json                      # 提取的角色
  character_portraits_registry.json    # 肖像路径注册表
  character_portraits/                 # 生成的肖像
    0_角色名称/
      front.png
      side.png
      back.png
  storyboard.json                     # 镜头描述
  camera_tree.json                    # 相机关系
  shots/
    0/
      shot_description.json
      first_frame.png
      last_frame.png (如果为中/大变化)
      video.mp4
    1/
      ...
  final_video.mp4                     # 最终拼接输出

自定义

使用自己的参考照片

使用真实照片代替 AI 生成的肖像:

# 构建指向你照片的肖像注册表
character_portraits_registry = {
    "Alice": {
        "front": {"path": "/path/to/alice_front.png", "description": "Alice 的正面视图"},
        "side": {"path": "/path/to/alice_side.png", "description": "Alice 的侧面视图"},
        "back": {"path": "/path/to/alice_back.png", "description": "Alice 的背面视图"},
    }
}

# 传递给管线(跳过肖像生成)
await pipeline(
    script=script,
    user_requirement=user_requirement,
    style=style,
    character_portraits_registry=character_portraits_registry,
)

更换模型

编辑 YAML 配置以使用不同的模型:

  • 图像:google/gemini-3.1-flash-image-preview(推荐用于面部一致性)
  • 视频:google/veo-3.1-generate-preview(推荐)或 openai/sora-2
  • 聊天:gpt-4.1(推荐)或任何兼容 OpenAI 的模型

故障排除

“No module named 'tools'”或类似导入错误

从 ViMax 根目录运行:

cd /home/node/a0/workspace/527fb591-1439-4b5b-ad5d-90f972773f95/workspace/tmp/ViMax
.venv/bin/python main_happycapy_script2video.py

API 速率限制错误

在 YAML 配置中减小 max_requests_per_minute 的值。

生成的视频中出现面部一致性漂移

  • 在脚本中为角色描述添加更多物理细节
  • 使用用户提供的参考照片代替 AI 生成的肖像
  • 为重要角色包含极端特写镜头
  • 保持各场景的光照一致

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

ComeOnOliver/skillshub

Create stunning, animation-rich HTML presentations from scratch or by converting PowerPoint files. Use when the user wants to build a presentation, convert a PPT/PPTX to web, or create slides for a talk/pitch. Helps non-designers discover their aesthetic through visual exploration rather than abstract choices.

ComeOnOliver/skillshub

Next.js 16+ 和 Turbopack — 增量打包、文件系统缓存、开发速度,以及何时使用 Turbopack 与 webpack。

ComeOnOliver/skillshub

SwiftUI architecture patterns, state management with @Observable, view composition, navigation, performance optimization, and modern iOS/macOS UI best practices.

ComeOnOliver/skillshub

See, Understand, Act on video and audio. See- ingest from local files, URLs, RTSP/live feeds, or live record desktop; return realtime context and playable stream links. Understand- extract frames, build visual/semantic/temporal indexes, and search moments with timestamps and auto-clips. Act- transcode and normalize (codec, fps, resolution, aspect ratio), perform timeline edits (subtitles, text/image overlays, branding, audio overlays, dubbing, translation), generate media assets (image, audio, video), and create real time alerts for events from live streams or desktop capture.

ComeOnOliver/skillshub

AI-assisted video editing workflows for cutting, structuring, and augmenting real footage. Covers the full pipeline from raw capture through FFmpeg, Remotion, ElevenLabs, fal.ai, and final polish in Descript or CapCut. Use when the user wants to edit video, cut footage, create vlogs, or build video content.

ComeOnOliver/skillshub

Next.js 15 애플리케이션을 위한 프론트엔드 개발 가이드라인. React 19, TypeScript, Shadcn/ui, Tailwind CSS를 사용한 모던 패턴. Server Components, Client Components, App Router, 파일 구조, Shadcn/ui 컴포넌트, 성능 최적화, TypeScript 모범 사례 포함. 컴포넌트, 페이지, 기능 생성, 데이터 페칭, 스타일링, 라우팅, 프론트엔드 코드 작업 시 사용.

ComeOnOliver/skillshub

Provides Tamagui patterns for config v4, compiler optimization, styled context, and cross-platform styling. Must use when working with Tamagui projects (tamagui.config.ts, @tamagui imports).

ComeOnOliver/skillshub

Create distinctive, production-grade frontend interfaces with high design quality. Use this skill when the user asks to build web components, pages, or applications. Generates creative, polished code that avoids generic AI aesthetics.

ComeOnOliver/skillshub

Create high-converting, visually distinctive landing pages. Use when building marketing pages, product launches, SaaS homepages, or any single-page conversion-focused website. Guides section-by-section composition with anti-AI-slop principles.

ComeOnOliver/skillshub

Auto-loaded context for Portfolio Buddy 2 development. Use for ANY task involving: React 19 development, TypeScript, portfolio analysis features, metrics calculations, trading strategy comparison, or working with the Portfolio Buddy 2 codebase. Contains tech stack, known issues, and architectural constraints.

ComeOnOliver/skillshub

Next.js development tooling via MCP. Inspect routes, components, build info, and debug Next.js apps. Use when working on Next.js applications, debugging routing, or inspecting app structure. NOT for general React or non-Next.js projects.

ComeOnOliver/skillshub

Implement UI using Shadcn MCP (atoms/theme) and 21st.dev MCP (complex sections). Use when adding buttons, layouts, or generating landing pages.

ComeOnOliver/skillshub

A conceptual skill for building an API client in Next.js that handles JWT tokens

ComeOnOliver/skillshub

Best practices and patterns for Next.js App Router, Server Actions, and Routing in this project.

ComeOnOliver/skillshub

The Design System, Theme, and UX rules for the Physical AI Hub.

ComeOnOliver/skillshub

Comprehensive frontend development skill for building modern, performant web applications using ReactJS, NextJS, TypeScript, Tailwind CSS. Includes component scaffolding, performance optimization, bundle analysis, and UI best practices. Use when developing frontend features, optimizing performance, implementing UI/UX designs, managing state, or reviewing frontend code.

ComeOnOliver/skillshub

This document captures learnings from fixing keyboard navigation issues when floating components (Select, DropdownMenu, Popover) have CSS open/close animations.

ComeOnOliver/skillshub

Build Next.js 16 applications with correct patterns and distinctive design. Use when creating pages, layouts, dynamic routes, upgrading from Next.js 15, or implementing proxy.ts. Covers breaking changes (async params/searchParams, Turbopack, cacheComponents) and frontend aesthetics. NOT when building non-React or backend-only applications.

ComeOnOliver/skillshub

Comprehensive Framer Motion animation library for React. Covers motion components, variants, gestures, page transitions, and scroll animations. Use when adding animations to React/Next.js applications.

ComeOnOliver/skillshub

Build interfaces that resonate, not just render. This skill enforces bold, intentional

相关技能