Context → Video
Transforms any input context into an MP4 with narration, slides, subtitles, and (optionally) a talking-head avatar.
When to Use
動画化,解説動画作って,ブログを動画に,記事を動画化,explainer video,context to video- Want to turn release blogs / PR explanations / meeting notes / design memos / release notes / learning content into short videos
- Need to mass-produce material for YouTube / internal study sessions / Teams / X in short time
- For customer-facing output, replace TTS with Azure Speech official version (default uses the free edge-tts endpoint)
Input Sources (any of these works)
| Input | How to Get It |
|---|---|
| Blog/article URL | Fetch body via web_fetch |
| Pasted text | Summarize directly |
| PR / commit diff | gh pr view + gh pr diff |
| Meeting notes / design memos | Read from file |
| Release notes | URL or Markdown |
| Free-form prompt | Ask LLM to structure the script |
→ After obtaining the source, processing is uniform (LLM writes notes/script.json).
Stack (all free, local)
| Role | Tool |
|---|---|
| Script generation | LLM (the skill caller) |
| Slides | Pillow (using BIZ UDGothic etc.) |
| Narration | edge-tts (Edge's Neural voice, free, unofficial) |
| Video compositing | ffmpeg (can use imageio-ffmpeg bundled binary) |
| Subtitles | Auto-generated .srt |
| Avatar (optional) | SadTalker (GPU recommended) + ffmpeg elliptical mask |
Quick Workflow (no avatar)
pip install edge-tts imageio-ffmpeg pillow
# 1. Generate script.json from input (LLM)
# 2. Generate video
python scripts\build_video.py --project D:\path\to\project
# → output\final.mp4 + final.srt
Details: references/script-schema.md / references/setup.md
Avatar Mode (talking-head in corner)
If you have a GPU (VRAM 8GB+), you can add an avatar in the bottom-right corner using SadTalker. Initial setup takes 30-40 minutes in a separate venv (Python 3.10), then +20-30 minutes per video.
# Existing video + 8 audio → avatar composition
python scripts\add_avatar.py --project D:\path\to\project --face D:\path\to\face.jpg
# Create elliptical mask version
python scripts\ellipse_overlay.py --project D:\path\to\project
# → output\final_with_avatar.mp4 and final_with_avatar_ellipse.mp4
Setup, pitfalls, and character selection: references/avatar.md
Output Layout
<project>/
├── notes/script.json Script (written by LLM)
├── slides/slide_XX.png 1920x1080
├── audio/audio_XX.mp3 Narration per slide
├── avatar/ (optional)
│ ├── input/face.jpg Avatar source photo
│ ├── SadTalker/ repo + checkpoints
│ ├── venv/ Python 3.10 + torch cu121
│ └── clips/audio_XX.mp4 Avatar video per narration
└── output/
├── final.mp4 Standard (slides + narration)
├── final.srt Subtitles
├── final_with_avatar.mp4 (optional) Avatar rectangle
└── final_with_avatar_ellipse.mp4 (optional) Avatar circular wipe
Defaults
- Resolution: 1920x1080 30fps (16:9)
- Voice:
ja-JP-NanamiNeural(female, calm). For male:ja-JP-KeitaNeural - Number of slides: Cover + 5-7 main slides (6-8 total)
- Color scheme: slate-900 / sky-400 / BIZ UDGothic
- Duration per slide: 15-25 seconds, total 2-4 minutes
To change voice, colors, font, or aspect ratio: references/customization.md
Production Use
The free edge-tts is unofficial, no SLA, legally gray for commercial use. For customer distribution or continuous operation:
- Narration → Replace with Azure Speech (Neural TTS)
- Commercial avatar → Azure TTS Avatar (Lisa etc., official, $0.30/min+)
Details: references/production-upgrade.md
Common Variants
- 9:16 Short version (X/Reels):
W,H = 1080, 1920, compress narration to 60 seconds - Burn subtitles into video: Use ffmpeg
subtitles=final.srtfilter to embed subtitles (ffmpeg-advanced-filters.md) - Add BGM: Mix a separate mp3 at low volume (ffmpeg-advanced-filters.md)
- Chapters / Course format (learning material): Split script.json into chapters, add common intro/outro
- Multi-language versions: Switch
voice+ translate script - Fixed channel mascot generation: Create mascot with SDXL and save seed (avatar-generation-sdxl.md)
- VOICEVOX + official character illustrations: For "relaxed explanation" channels like Zundamon, use PSD layer differences for full lip-sync (voiced-mascot.md)
- Circular avatar wipe: Use ffmpeg geq filter for circular mask + accent ring (ffmpeg-advanced-filters.md)
- YouTube auto-upload: Schedule publication via API + add thumbnail (youtube-upload.md)
- PoC → Regular channel upgrade: Redesign directory structure (scale-up-production.md)
Done Criteria
-
<out>/output/final.mp4(or avatar version) plays without audio-video desync - Slide body text ≥28pt / title ≥56pt, readable
- Narration correctly reflects the main message of the input
-
final.srtis synchronized with narration - Does not infringe copyright of the input source (summarized, not verbatim; includes citation scope)
- (For avatar) completes without GPU memory overflow, no extreme lip-sync errors