AI Video Generation
Text-to-Video / Image-to-Video / Digital Human First-Frame Driving. Wraps
shared/scripts/ai_video.py, with pluggable providers, asynchronous submission → polling → download. Users must bring their own API key (configured in.env).
Prerequisite: Configure API Key
Configuration Check Path Rule: First
cdto the Easel project root specified at the end ofAGENTS.md. Confirm the current directory contains.envandskills/shared/scripts/, then run registration,check, or generation commands. Do not use the workspace's./shared/scripts/..., and do not assume it's unconfigured just becauseenv/printenvdoesn't show the variable.
First, choose a provider and fill in the corresponding key in .env, then run check for offline validation:
python skills/shared/scripts/ai_video.py check --provider dashscope
| provider | Service | Required in .env |
|---|---|---|
dashscope | Alibaba Tongyi Wanxiang Wan | DASHSCOPE_API_KEY (optional DASHSCOPE_VIDEO_MODEL/DASHSCOPE_BASE_URL; supports legacy DASHSCOPE_MODEL) |
ark | Volcengine Seedance | ARK_API_KEY (optional ARK_MODEL/ARK_BASE_URL) |
kling | Kuaishou Kling | KLING_ACCESS_KEY + KLING_SECRET_KEY (JWT authentication) |
openai-compatible | Generic /videos endpoint | VIDEO_API_KEY + VIDEO_BASE_URL (optional VIDEO_MODEL) |
xhs-maas | Xiaohongshu internal MaaS (happyhorse text/image-to-video) | XHS_MAAS_API_KEY (optional XHS_MAAS_VIDEO_BASE/XHS_MAAS_T2V_MODEL/XHS_MAAS_I2V_MODEL). DashScope-style async + api-key header, direct intranet connection |
agnes | Agnes (agnes-video-2.5-flash) | AGNES_API_KEY (optional AGNES_BASE_URL/AGNES_MODEL/AGNES_SIZE). OpenAI Videos compatible creation + custom endpoint polling; native audio by default (prompt describes sound); external network via proxy |
You can also set VIDEO_PROVIDER to avoid needing --provider each time.
Before execution, run model_registry.py configured --group video --env-file .env: if only one is available, use it explicitly; if multiple are available and the user hasn't specified one, list the providers/models and ask which to use this time—do not pick arbitrarily from defaults.
Input
Aspect Ratio Confirmation Hard Rule: If the user or upstream task hasn't explicitly specified landscape/portrait (or 16:9/9:16/specific ratio), you must ask and wait for confirmation before any generation or paid call. Do not silently infer from the platform, profile, or script defaults. If already confirmed, do not ask again.
- Text-to-Video: Description of scene/camera/style (prompt)
- Image-to-Video: One input image (local path or URL) + optional motion description
- Optional:
--duration,--ratio(16:9 / 9:16 / 1:1),--model,--audio auto|on|off
Output
The generated video file; must use -o to specify the path under outputs/topic_name/. Asynchronous tasks automatically poll until completion before downloading.
Execution Steps
- Confirm Configuration & Capabilities: First run
check, then runcapabilities --provider <p> --model <m>. For short dramas, do not guess whether a model supports native audio based on brand name; new models register capabilities and request fields viaVIDEO_CAPABILITIES_JSONwithout modifying the call flow.probe-dialogue(for short dramas): Send 1 real generation + ASR to test whether the model can faithfully speak the specified lines word-for-word, judgedialogue_faithful, and cache the result. Short dramas then decide whether to use native dialogue or "no-dialogue generation + post-dubbing." Usage:probe-dialogue --provider <p> --model <m>.
- Write the Prompt: AI video is sensitive to prompts. Follow the AI Video Prompting Guide for camera, movement, style, and duration. For vertical short videos, use
--ratio 9:16. - Text-to-Video:
python skills/shared/scripts/ai_video.py text2video --provider dashscope \ --prompt "Sunset at the beach, slow zoom in, warm tones, cinematic" --ratio 9:16 --duration 5 \ --audio auto \ -o outputs/topic_name/clip.mp4 - Image-to-Video / Animate an Image / Digital Human First Frame:
python skills/shared/scripts/ai_video.py image2video --provider kling \ --image outputs/topic_name/cover.png --prompt "The person smiles and waves, hair gently flowing" \ -o outputs/topic_name/clip.mp4 - Post-Processing: The generated clips can be passed to
video_ops.py(stitching/subtitles/BGM/portrait-landscape conversion),auto-subtitle(subtitles),tts-voiceover(dubbing) to assemble into a final piece, or directly fed into theauto-short-videoend-to-end pipeline.
Profile Awareness
- With Profile: Extract visual style preferences from
style.mdand inject them into the prompt;platforms.mdis only used to give aspect ratio suggestions, not to replace user confirmation. - Without Profile: First confirm landscape/portrait, then generate according to the confirmed ratio.
Notes
- Video generation APIs are all asynchronous and time-consuming (tens of seconds to several minutes) + billed per usage; confirm with the user first.
- Model names/fields vary across providers and versions; use
--modelor env to override. If errors occur, refer to the latest official documentation. --audio autoonly maps known fields according to the capability profile; capability declaration does not guarantee quality. After download, still use ffprobe/ASR/visual audit. If the gateway has audio by default but the toggle field is unknown, do not guess and inject parameters.