Communitygithub.com

ZJU-REAL/Easel

多角色对话配音:按 cast 和逐行对白为不同角色分配音色与情绪,合成多声线音轨和带角色名字幕。 当用户说“多角色/双人/剧本/对话配音、多人对白、不同角色不同声音、有声剧配音”时使用。 单一公共音色用 tts-voiceover;克隆本人音色用 voice-clone;整部短剧制作由 short-drama 编排。

Easel 是什么?

Easel is a Codex agent skill that 多角色对话配音:按 cast 和逐行对白为不同角色分配音色与情绪,合成多声线音轨和带角色名字幕。 当用户说“多角色/双人/剧本/对话配音、多人对白、不同角色不同声音、有声剧配音”时使用。 单一公共音色用 tts-voiceover;克隆本人音色用 voice-clone;整部短剧制作由 short-drama 编排。.

兼容平台~Claude Code✓Codex CLI~Cursor✓Gemini CLI
npx skills add https://github.com/ZJU-REAL/Easel/tree/HEAD/skills/openclaw/multi-voice-dubbing

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

Multi-role / Dialogue Dubbing (Multi-voice Dubbing)

Synthesize "multi-person dialogue / multi-role scripts" into multi-voice audio tracks—each character gets a voice matching their persona, no longer "one voice throughout". The core engine is skills/shared/scripts/multivoice.py: it calls tts.py (free multi-voice via edge-tts) or voice_clone.py (cloud expressive provider / cloned voices) line by line, feeds each line's emotion into the provider's true emotion channel, and uses ffmpeg to concatenate everything into one track while generating aligned speaker-labeled subtitles.

Creativity (who says what, with what emotion) is produced by you, the LLM; voice mapping is in the cast. Deterministic IO (line-by-line synthesis/concatenation/subtitles) is handled by the engine. The output voice.mp3 can be directly used as narration for auto-short-video/assemble.py or added to any video; voice.srt is a subtitle track with speaker names.

Dubbing Quality Tiers (Important: Cures "flat AI reading")

edge-tts has no emotion engine; it can only adjust speed and pitch, and no matter what you do, it still sounds like a robot. To sound "human", you must use a cloud provider with emotion channels (user brings their own key, no GPU needed):

Engine (engine in cast)QualityRequirementsEmotion Mechanism
edge (default, free fallback)⚠️ Flat, robotic—draft onlyNo key, external networkOnly rate/pitch/volume micro-adjustments
clone+openai-compatible→SiliconFlow CosyVoice2 (recommended)Good, natural ChineseVOICE_API_KEY (cheap / free credits for new users)Inline `<
clone+geminiGood, truly freeGEMINI_API_KEY (Google free tier, requires proxy in China)Natural language prefix
clone+minimax / dashscopeGood, emotionalProvider-specific keysEmotion enumeration / instruct

Recommended: SiliconFlow (cloud CosyVoice2, no GPU needed, most stable for Chinese). Configure in .env:

VOICE_PROVIDER=openai-compatible
VOICE_BASE_URL=https://api.siliconflow.cn/v1
VOICE_API_KEY=<your key>
VOICE_MODEL=FunAudioLLM/CosyVoice2-0.5B
VOICE_INSTRUCT_MODE=inline

For a character in cast: --engine clone --provider openai-compatible --voice-id FunAudioLLM/CosyVoice2-0.5B:alex (8 voices: alex/anna/benjamin/…).

  • Line-by-line emotion auto-drives performance: Each line in lines.json has an emotion (angry / collapsing in tears / sneering / gentle…) → the engine converts it to the corresponding emotion parameter per provider. The more specific you write, the better the acting.
  • Want it "human-like" → give at least the main character / key characters a cloud provider (engine=clone); only with a key will you get emotions; without a key it automatically falls back to edge (flat) with a warning.
  • Provider configuration is at the top of voice_clone.py (env vars per provider); voice_clone.py check --provider <name> validates offline that the keys are complete.

Who would use it

Short drama dialogue (short-drama already delegates internally), paper Q&A with two speakers (paper-explainer: speaker + questioner), interview/podcast scripts, audio dramas, any spoken content with multiple speakers. For a single speaker's full narration, use tts-voiceover.

Input

FieldRequiredDescription
cast.jsonYesCasting table: each speaker → voice (edge voice + pitch/rate, or cloned voice). Includes entries for "Narrator / Main Speaker"
lines.jsonYesLine-by-line dialogue: ordered [{speaker, text, emotion}] (speaker must match a name in cast; emotion e.g. cold/angry/nervous/gentle, automatically matched to tone)
Output pathNovoice.mp3 (default agreed with caller); voice.srt with same name

Execution Steps

1. Casting (cast.json)—Read the casting guide to assign voices

First read skills/shared/references/voice-casting.md (voice ↔ persona archetype mapping + distinguishing same-gender voices + independent narrator + emotional prosody). Assign a voice matching each speaker's gender/age/temperament/identity. The narrator / main speaker should be separate and different from all characters:

python skills/shared/scripts/multivoice.py cast init  --cast <path>/cast.json
python skills/shared/scripts/multivoice.py cast add   --cast <path>/cast.json \
    --name Lin Ce --role male_lead --voice zh-CN-YunxiNeural --rate=-5% --pitch=-3Hz --note "Cool male lead"   # Use = for negative values
python skills/shared/scripts/multivoice.py cast add   --cast <path>/cast.json \
    --name Su Wan --role female_lead --voice zh-CN-XiaoxiaoNeural --note "Gentle female lead"
python skills/shared/scripts/multivoice.py cast check --cast <path>/cast.json    # Validate: voices valid / narrator independent / no voice collisions

cast.json can also be written directly (it's just JSON). If you want a character to sound human with emotion (cures AI flatness) → use a cloud provider for that character:

python skills/shared/scripts/multivoice.py cast add --cast <path>/cast.json \
    --name CEO --role male_lead --engine clone --provider minimax --voice-id <your voice id> --note "Cloned / expressive voice"

The corresponding provider key must be set in .env (voice_clone.py check --provider minimax); if the key is missing, that character automatically falls back to edge (flat) with a warning.

2. Line-by-line dialogue (lines.json)

Split the script / dialogue into ordered lines:

{"lines":[
  {"speaker":"Presenter","text":"This paper addresses a key problem.","emotion":"neutral"},
  {"speaker":"Questioner","text":"Wait, why can't existing methods work?","emotion":"surprised"},
  {"speaker":"Presenter","text":"Because they ignore temporal dependencies.","emotion":"firm"}
]}

speaker must match a name in cast (if not, it falls back to the narrator voice with a warning); emotion is optional.

3. Synthesize multi-voice track + subtitles

python skills/shared/scripts/multivoice.py dub \
    --cast <path>/cast.json --lines <path>/lines.json -o <path>/voice.mp3

Produces voice.mp3 (each character has an independent voice; emotions automatically modulate prosody) + voice.srt (with speaker names, timed to actual line durations). dub prints how many voice types were used—confirm ≥2 (a multi-person dialogue should not have only one voice).

4. Use in a video

  • voice.mp3 as narration + voice.srt as subtitles, feed into auto-short-video/scripts/assemble.py (image/video compositing).
  • Or mix with BGM (audio-mix), add to an existing video (video-editing).

Fallback

  • No external network / edge-tts fails → cannot synthesize; inform the user truthfully (multi-voice depends on edge-tts).
  • cast specifies a cloned voice but the key is missing / fails → that character automatically falls back to a free edge voice with a warning; does not block the process.
  • Speaker not in cast → fallback to narrator voice with a warning (suggest adding to cast).

Profile Awareness

  • With Profile: style.md influences voice timbre selection (character temperament → voice); preferences.md filters lines for redlines.
  • Without Profile: Use default voice-casting.md mapping.

Rules

  1. Multi-person dialogue must use multiple voices: each speaker has an independent voice; narrator / presenter is separate—never use one voice throughout (cast check + dub voice count guard).
  2. To sound human, use a cloud provider: edge is only for drafts; final / key characters use engine=clone+provider (with a real emotion channel), no GPU needed.
  3. Voice matches the character: follow voice-casting.md archetype mapping + distinguish same-gender voices with pitch/rate.
  4. Emotion annotations must be specific: each line in lines should have an emotion (angry / collapsing in tears / sneering / gentle…), which feeds into the provider's emotion channel to drive performance.
  5. Deterministic persistence: save cast.json / lines.json; to change lines or swap voices, simply re-run dub—no need to start over.

References

See EASEL-META.md. Multi-voice dubbing (cast + line-by-line lines + per-character voice synthesis + ffmpeg concatenation + speaker subtitles) is self-developed by Easel; underlying wrappers are edge-tts (tts.py) and cloud cloning (voice_clone.py).

Individual skills in this repo

This repo contains 9 individual skills — each has its own dedicated page.

ZJU-REAL/Easel

AI 视频生成:文生视频 / 图生视频 / 数字人首帧驱动。通过可插拔 provider(通义万相 Wan / 火山 Seedance / 快手可灵 / OpenAI 兼容)异步生成视频,用户自备 API key。当用户说 AI 视频生成、文生视频、图生视频、AI 生成视频、AI 短视频、让图片动起来、数字人视频、生成一段视频 时使用。与 video-strategy(选型/策略)、video-editing(剪辑处理)、clipify(切片)区别:本 SKILL 是从 0 用 AI 生成新视频。

ZJU-REAL/Easel

通用音频处理:音频剪辑/裁剪、格式转码(mp3/wav/m4a/aac)、音量归一化、从视频提取音轨、多段拼接、淡入淡出、变速(保音高)。当用户说“剪音频”“裁一段”“转成 mp3”“调音量/响度”“提取音轨/扒音频”“拼接音频”“淡入淡出”“加速/减速音频”“变速不变调”时使用。与 audio-denoise 的区别:audio-denoise 专做降噪,audio-editing 做除降噪外的通用音频操作(也内置 denoise 作为兜底)。

ZJU-REAL/Easel

一句话主题 → 成品短视频:自动串联 文案→配图/AI视频→配音→字幕→BGM→合成,把 Easel 制作层零件编排成一条'一键出片'流水线。**单条视频、口播/资讯向,画面默认逐句配图 + Ken Burns 缓动,需要动态时才逐段图生视频**。当用户说 一键生成视频、自动做短视频、主题生成视频、帮我做条视频、口播视频一条龙、自动出片、短视频一键生成 时使用。**有剧情/角色/对白/反转/多集的短剧改用 short-drama(每镜强制图生视频、不用静态图冒充);只写脚本用 video-script;只生成单个片段用 ai-video-gen。**

ZJU-REAL/Easel

自动字幕 / 语音转字幕:把音频或视频里的语音识别成字幕文件(SRT/ASS/TXT/JSON),可选把字幕烧录进视频。当用户说自动字幕、语音转字幕、视频加字幕、上字幕、转录、听写、字幕文件、生成字幕、烧字幕时使用。

ZJU-REAL/Easel

音乐卡点视频 / 踩点视频:检测背景音乐的节拍,让图片或片段在节拍点上切换,配推进/白闪特效,做出燃系'卡点'短视频。当用户说 卡点视频、踩点视频、音乐卡点、节奏卡点、按音乐切换、鼓点视频、踩节奏、beat 卡点、图片卡点、卡点混剪 时使用。基于 shared/scripts/beatsync.py(librosa 节拍检测)。与 slideshow-video 区别:slideshow 每图固定时长、柔和转场,本 SKILL 切换点由音乐节拍决定、硬切踩点带特效。

ZJU-REAL/Easel

Easel clipify: long English talk → punchline cuts → optional 16:9→9:16 face-pan/split → opus word-by-word ASS burn. Ships scripts + assets/preview.png. Part of 1.6k★ Easel suite.

ZJU-REAL/Easel

>- 通用图像处理加工:改尺寸/缩放、裁剪、补边适配平台尺寸、格式转换(png/jpg/webp)、 压缩到目标大小、加文字或图片水印、圆角、多图拼接、生成缩略图、读图片信息。 基于 image_ops.py 确定性处理。 使用时机:用户说"改尺寸"、"缩放图片"、"裁剪"、"压缩图片"、"加水印"、"转格式"、 "圆角"、"拼图/多图联排"、"缩略图"、"适配小红书/朋友圈尺寸"、"把图 pad 成 1:1"。 和 card-*(卡片类)的区别:card-* 是"HTML 设计→渲染出图", image-editing 只加工已有图片,不负责视觉设计。

ZJU-REAL/Easel

Easel video-highlights: CN/livestream highlight extract + stable static vertical reframe (sibling to clipify).

ZJU-REAL/Easel

>- 视频制作策略与工具选型:AI 视频生成模型对比、视频脚本结构设计、制作流程规划,覆盖产品演示/解说/社媒短视频场景。 当用户说"视频怎么做""视频选型""用什么工具做视频""视频制作流程""视频策略""视频系列规划"时使用。 和 video-script 的区别:video-strategy 做内容规划与工具选型,video-script 写具体某条视频的脚本。

相关技能