Communitygithub.com

ZJU-REAL/Easel

AI Video Generation: Text-to-Video / Image-to-Video / Digital Human First-Frame Driving. Generates videos asynchronously via pluggable providers (Tongyi Wanxiang Wan / Volcengine Seedance / Kuaishou Kling / OpenAI compatible). Users bring their own API key. Use when the user mentions AI video generation, text-to-video, image-to-video, AI-generated video, AI short video, animating an image, digital human video, or generating a video clip. Distinguished from video-strategy (selection/strategy), video-editing (editing/processing), and clipify (segmentation): this skill generates new videos from scratch using AI.

What is Easel?

Easel is an AI agent skill designed for generating new videos from scratch using text prompts, images, or digital human first-frame driving. It encapsulates a shared Python script (`ai_video.py`) with a pluggable provider architecture, supporting asynchronous task submission, polling, and final download. Users must bring their own API keys configured in a `.env` file. Supported providers include Alibaba Tongyi Wanxiang (DashScope), Volcengine Seedance (Ark), Kuaishou Kling, and any OpenAI-compatible `/videos` endpoint. Additional providers like Agnes (with native audio support) and internal MaaS endpoints (e.g., xhs-maas) are also configurable. The skill strictly enforces a workflow: first, configure and check the provider; second, run capability probes (including a `probe-dialogue` to test speech fidelity for short dramas); third, write a detailed prompt following best practices; fourth, execute the generation command; and finally, optionally pass the output to downstream editing or assembly skills. It is particularly useful for AI content creators, short video producers, and storytellers who need to quickly generate visual assets from ideas without manual filming.

Works with~Claude Code✓Codex CLI~Cursor
npx skills add https://github.com/ZJU-REAL/Easel/tree/HEAD/skills/openclaw/ai-video-gen

Ask in your favorite AI

Open a new chat with this agent skill pre-loaded.

Documentation

AI Video Generation

Text-to-Video / Image-to-Video / Digital Human First-Frame Driving. Wraps shared/scripts/ai_video.py, with pluggable providers, asynchronous submission → polling → download. Users must bring their own API key (configured in .env).

Prerequisite: Configure API Key

Configuration Check Path Rule: First cd to the Easel project root specified at the end of AGENTS.md. Confirm the current directory contains .env and skills/shared/scripts/, then run registration, check, or generation commands. Do not use the workspace's ./shared/scripts/..., and do not assume it's unconfigured just because env/printenv doesn't show the variable.

First, choose a provider and fill in the corresponding key in .env, then run check for offline validation:

python skills/shared/scripts/ai_video.py check --provider dashscope
providerServiceRequired in .env
dashscopeAlibaba Tongyi Wanxiang WanDASHSCOPE_API_KEY (optional DASHSCOPE_VIDEO_MODEL/DASHSCOPE_BASE_URL; supports legacy DASHSCOPE_MODEL)
arkVolcengine SeedanceARK_API_KEY (optional ARK_MODEL/ARK_BASE_URL)
klingKuaishou KlingKLING_ACCESS_KEY + KLING_SECRET_KEY (JWT authentication)
openai-compatibleGeneric /videos endpointVIDEO_API_KEY + VIDEO_BASE_URL (optional VIDEO_MODEL)
xhs-maasXiaohongshu internal MaaS (happyhorse text/image-to-video)XHS_MAAS_API_KEY (optional XHS_MAAS_VIDEO_BASE/XHS_MAAS_T2V_MODEL/XHS_MAAS_I2V_MODEL). DashScope-style async + api-key header, direct intranet connection
agnesAgnes (agnes-video-2.5-flash)AGNES_API_KEY (optional AGNES_BASE_URL/AGNES_MODEL/AGNES_SIZE). OpenAI Videos compatible creation + custom endpoint polling; native audio by default (prompt describes sound); external network via proxy

You can also set VIDEO_PROVIDER to avoid needing --provider each time.

Before execution, run model_registry.py configured --group video --env-file .env: if only one is available, use it explicitly; if multiple are available and the user hasn't specified one, list the providers/models and ask which to use this time—do not pick arbitrarily from defaults.

Input

Aspect Ratio Confirmation Hard Rule: If the user or upstream task hasn't explicitly specified landscape/portrait (or 16:9/9:16/specific ratio), you must ask and wait for confirmation before any generation or paid call. Do not silently infer from the platform, profile, or script defaults. If already confirmed, do not ask again.

  • Text-to-Video: Description of scene/camera/style (prompt)
  • Image-to-Video: One input image (local path or URL) + optional motion description
  • Optional: --duration, --ratio (16:9 / 9:16 / 1:1), --model, --audio auto|on|off

Output

The generated video file; must use -o to specify the path under outputs/topic_name/. Asynchronous tasks automatically poll until completion before downloading.

Execution Steps

  1. Confirm Configuration & Capabilities: First run check, then run capabilities --provider <p> --model <m>. For short dramas, do not guess whether a model supports native audio based on brand name; new models register capabilities and request fields via VIDEO_CAPABILITIES_JSON without modifying the call flow.
    • probe-dialogue (for short dramas): Send 1 real generation + ASR to test whether the model can faithfully speak the specified lines word-for-word, judge dialogue_faithful, and cache the result. Short dramas then decide whether to use native dialogue or "no-dialogue generation + post-dubbing." Usage: probe-dialogue --provider <p> --model <m>.
  2. Write the Prompt: AI video is sensitive to prompts. Follow the AI Video Prompting Guide for camera, movement, style, and duration. For vertical short videos, use --ratio 9:16.
  3. Text-to-Video:
    python skills/shared/scripts/ai_video.py text2video --provider dashscope \
      --prompt "Sunset at the beach, slow zoom in, warm tones, cinematic" --ratio 9:16 --duration 5 \
      --audio auto \
      -o outputs/topic_name/clip.mp4
    
  4. Image-to-Video / Animate an Image / Digital Human First Frame:
    python skills/shared/scripts/ai_video.py image2video --provider kling \
      --image outputs/topic_name/cover.png --prompt "The person smiles and waves, hair gently flowing" \
      -o outputs/topic_name/clip.mp4
    
  5. Post-Processing: The generated clips can be passed to video_ops.py (stitching/subtitles/BGM/portrait-landscape conversion), auto-subtitle (subtitles), tts-voiceover (dubbing) to assemble into a final piece, or directly fed into the auto-short-video end-to-end pipeline.

Profile Awareness

  • With Profile: Extract visual style preferences from style.md and inject them into the prompt; platforms.md is only used to give aspect ratio suggestions, not to replace user confirmation.
  • Without Profile: First confirm landscape/portrait, then generate according to the confirmed ratio.

Notes

  • Video generation APIs are all asynchronous and time-consuming (tens of seconds to several minutes) + billed per usage; confirm with the user first.
  • Model names/fields vary across providers and versions; use --model or env to override. If errors occur, refer to the latest official documentation.
  • --audio auto only maps known fields according to the capability profile; capability declaration does not guarantee quality. After download, still use ffprobe/ASR/visual audit. If the gateway has audio by default but the toggle field is unknown, do not guess and inject parameters.

Individual skills in this repo

This repo contains 9 individual skills — each has its own dedicated page.

ZJU-REAL/Easel

通用音频处理:音频剪辑/裁剪、格式转码(mp3/wav/m4a/aac)、音量归一化、从视频提取音轨、多段拼接、淡入淡出、变速(保音高)。当用户说“剪音频”“裁一段”“转成 mp3”“调音量/响度”“提取音轨/扒音频”“拼接音频”“淡入淡出”“加速/减速音频”“变速不变调”时使用。与 audio-denoise 的区别:audio-denoise 专做降噪,audio-editing 做除降噪外的通用音频操作(也内置 denoise 作为兜底)。

ZJU-REAL/Easel

一句话主题 → 成品短视频:自动串联 文案→配图/AI视频→配音→字幕→BGM→合成,把 Easel 制作层零件编排成一条'一键出片'流水线。**单条视频、口播/资讯向,画面默认逐句配图 + Ken Burns 缓动,需要动态时才逐段图生视频**。当用户说 一键生成视频、自动做短视频、主题生成视频、帮我做条视频、口播视频一条龙、自动出片、短视频一键生成 时使用。**有剧情/角色/对白/反转/多集的短剧改用 short-drama(每镜强制图生视频、不用静态图冒充);只写脚本用 video-script;只生成单个片段用 ai-video-gen。**

ZJU-REAL/Easel

自动字幕 / 语音转字幕:把音频或视频里的语音识别成字幕文件(SRT/ASS/TXT/JSON),可选把字幕烧录进视频。当用户说自动字幕、语音转字幕、视频加字幕、上字幕、转录、听写、字幕文件、生成字幕、烧字幕时使用。

ZJU-REAL/Easel

音乐卡点视频 / 踩点视频:检测背景音乐的节拍,让图片或片段在节拍点上切换,配推进/白闪特效,做出燃系'卡点'短视频。当用户说 卡点视频、踩点视频、音乐卡点、节奏卡点、按音乐切换、鼓点视频、踩节奏、beat 卡点、图片卡点、卡点混剪 时使用。基于 shared/scripts/beatsync.py(librosa 节拍检测)。与 slideshow-video 区别:slideshow 每图固定时长、柔和转场,本 SKILL 切换点由音乐节拍决定、硬切踩点带特效。

ZJU-REAL/Easel

Easel clipify: long English talk → punchline cuts → optional 16:9→9:16 face-pan/split → opus word-by-word ASS burn. Ships scripts + assets/preview.png. Part of 1.6k★ Easel suite.

ZJU-REAL/Easel

>- 通用图像处理加工:改尺寸/缩放、裁剪、补边适配平台尺寸、格式转换(png/jpg/webp)、 压缩到目标大小、加文字或图片水印、圆角、多图拼接、生成缩略图、读图片信息。 基于 image_ops.py 确定性处理。 使用时机:用户说"改尺寸"、"缩放图片"、"裁剪"、"压缩图片"、"加水印"、"转格式"、 "圆角"、"拼图/多图联排"、"缩略图"、"适配小红书/朋友圈尺寸"、"把图 pad 成 1:1"。 和 card-*(卡片类)的区别:card-* 是"HTML 设计→渲染出图", image-editing 只加工已有图片,不负责视觉设计。

ZJU-REAL/Easel

多角色对话配音:按 cast 和逐行对白为不同角色分配音色与情绪,合成多声线音轨和带角色名字幕。 当用户说“多角色/双人/剧本/对话配音、多人对白、不同角色不同声音、有声剧配音”时使用。 单一公共音色用 tts-voiceover;克隆本人音色用 voice-clone;整部短剧制作由 short-drama 编排。

ZJU-REAL/Easel

Easel video-highlights: CN/livestream highlight extract + stable static vertical reframe (sibling to clipify).

ZJU-REAL/Easel

>- 视频制作策略与工具选型:AI 视频生成模型对比、视频脚本结构设计、制作流程规划,覆盖产品演示/解说/社媒短视频场景。 当用户说"视频怎么做""视频选型""用什么工具做视频""视频制作流程""视频策略""视频系列规划"时使用。 和 video-script 的区别:video-strategy 做内容规划与工具选型,video-script 写具体某条视频的脚本。

Related Skills