Paper → Explainer Video
Target audience: People with a technical background who haven't read this paper. After watching, they should be able to explain: what it changed, why it works, what we can learn, and be able to repeat a concrete example of what it does. Don't derive formulas, but make the logic clear.
Final output specs: Landscape 1920×1080, 30fps, H.264 + AAC, duration 6–8 minutes, clarity first (up to ~10 minutes is acceptable). The ending credits and call-to-action are configurable items (outro in the project directory film.json), not hard-coded.
This skill is paired with the code in this repository: engine/ (canvas shell, paper entity, style card ④, WebCodecs render page, concatenation & mixing, audio alignment, music & SFX), examples/demo (25-second minimal example), examples/always-on (one complete episode example). API docs: engine/README.md.
I. Hard Rules (each from rework; these take precedence in case of conflict)
- Before starting, load the full text of this skill. Paste the original checklist text into the brief for any subagent; do not paraphrase from memory. Briefs for writing and reviewing must also include the full text of
references/voiceover-selfcheck.md. The first line of the brief must specify this skill's path and require reading the full text. - The order cannot be reversed: logical chain → locked voiceover → storyboard (shot plan) → voiceover → rough cut to check flow → effects/art polish → final cover last. Don't do visuals before the voiceover is locked; don't generate any assets besides the voiceover (e.g., optional avatar) before the storyboard and script are locked. Setting the style before changing the content = rework.
- The writer does not review. Before submitting the voiceover for review, it must pass an independent 'listen-once' blind test and fact-check. Before showing the final video to the author, an agent not involved in the production must watch the complete dynamic video, check it against
references/review-checklist.md, and submit the report together. - Rendering follows only one path: in a local Chrome page, frame-by-frame
render(t)+ WebCodecs hardware encoding (engine/common/shell-wc.html). Write only this one path in the brief; do not offer a fallback of 'headless frame-by-frame screenshot → ffmpeg'. Test 10 seconds of ms/frame and the WebGL renderer before starting the full render; if the renderer is a software renderer like SwiftShader / llvmpipe, stop and report, don't switch to a different rendering method. Disablegeqfor compositing. Render in background agents, independent sections in parallel, while the main session remains able to chat at any time. - Voiceover: any TTS that outputs wav →
engine/voice/align.pyfor alignment. Re-generate incorrectly read sentences individually; do not manually edit timings. - Frame 0 has content. The first 20–30 seconds are the most important. Hook directly: authoritative name + the most counter-intuitive result + the audience's confusion (or 'explain X at once'), without suspense.
- Length follows clarity, up to ~10 minutes is acceptable; pauses between sentences should be tight (see Section 10).
- Visuals: readability first, warm white / beige editing style; default style card ④ Editorial (used in already published episodes). Prefer continuous camera moves, zooming into paper pages, soft composite music bed. Don't use 3Blue1Brown-style dark backgrounds as the whole video's base, nor Swiss grid poster style; don't do PPT-style hard-cut static cards.
- Optional presenter: the main content is the visuals. If you place the author's image, use only a small window in the bottom right, only at the beginning / chapter transitions / end; no large faces; don't render the voiceover text as large subtitles.
- Ending: a discussion question (related to the core mechanism, answerable in a sentence or two) → credits + call-to-action (
outrofromfilm.json); don't preview the next episode. - Copy: professional, concise, with reasoning process; minimal emoji use; avoid 'not... but...'; don't write unfounded 'I tested'; separate official claims, test results, and inferences. Submit the voiceover for author review in sections. Self-check list per sentence:
references/voiceover-selfcheck.md(overrides any older wording in this file). - Base plans on proven approaches; don't invent new formats yourself. For new camera movements, first find similar examples in
examples/.
Steps the author must personally do (the agent stops here, explains what the author needs to do, and waits for confirmation): review the logical chain and voiceover (Section 7), generate the voiceover wav (Section 9, unless the author has already provided a working TTS method), review the shot plan (Section 8), review the rough cut and final video (Sections 11–12), and publish. While the author hasn't finished reviewing, the same video doesn't proceed to the next stage (other videos can continue).
Save tokens: each review version gets at most 1 round of fact-checking and 1 round of blind testing; accumulate small issues and fix them together with the author's feedback. Use small models for searching and organizing files; use strong models for writing, fact-checking, and review. Write large content to files, only return paths and conclusions.
II. Overall Workflow
- Topic filter: paper overview + topic popularity on your publishing platform; hard requirement: the paper has a complete case study
- Deep reading → paper card (
references/paper-card.md) - Find the hook
- Three-question outline + running example + audience question chain (
references/question-chain.md) - Write voiceover (Section 4 +
references/voiceover-selfcheck.md) - 'Listen-once' blind test (two roles, subagent,
references/listen-test-brief.md) → revise script - Independent fact-check + devil's advocate (subagent, parallel,
references/factcheck-brief.md) → revise script, then re-test comprehension - Author reviews voiceover section by section (logical chain one page + segmented voiceover, wait for confirmation)
- Storyboard: choose style card + shot plan (
references/shot-plan.md) + top navigation progress bar → (Author reviews shot plan) - Voiceover (any TTS)
- Alignment, audio tracks (voice + music bed + SFX)
- Common components → write scene.js per chapter in parallel → rough cut → independent review → full video → chapter-by-chapter art polish → re-review → .1 revision
- Frame-drop self-check → delivery; after the author watches and proposes issues, first review them before making changes
For multi-source 'explain at once' videos, also use this skill
Surveys / multi-paper collations also go through steps 0–12. Consequences of not loading this skill: sentences crammed with 'organization + year + number' read in one breath, chapters not connected, no real paper pages, no animation or music, no comprehension test, entire video rework. Adaptations:
- Use one running example + one mother graphic throughout the whole video; each chapter re-walks the mother graphic with the same vocabulary. Prioritize using an existing classification from the paper for the mother graphic, organize chapters by this classification; don't invent your own framework.
- Each chapter tells only one main story and one main number; put other numbers as source cards on screen; the voiceover only says 'X found'.
- Chapters progress (simple to deep / pipeline order), last chapter returns to the opening hook.
- Don't put organization names and years in long voiceover sentences. One piece of information per sentence.
- The authority of a survey comes from 'paper + open-source code from big companies'; keep only one subject in the hook.
0. Topic Filter
- When there are multiple candidates, first check the paper's AI overview (e.g., alphaXiv); only use it to decide whether to do a deep read, not as a factual source.
- Check the topic's popularity on your publishing platform: has anyone already explained it thoroughly? How did those thorough explanations perform? Academic popularity (likes, citations, number of papers) doesn't equal platform popularity; what's popular on a platform is usually 'clear concept', 'one video is enough', and names/companies. Don't say 'recently very popular' without evidence.
- Hard requirement: the paper must have a complete case study (input → what each layer did → before/after scores), something you can 'walk through'. If not, choose carefully.
- Before finalizing the topic, decide four things: the reader's question, our claim, the evidence, and the delivery (what the viewer can take away).
1. Deep Reading: Paper Card
- Get the full text:
https://arxiv.org/pdf/<id>; if possible, also get the TeX source (/src/<id>, more accurate for numbers and figure captions, original figures can be used directly); if PDF download fails, first readarxiv.org/html/<id>. Only download materials you have the right to use, note the license (CC BY, etc.). - PDF:
pdftoppm -r 150to render all pages for content overview; for pages and regions used in the video, use 300dpi; for figures that need to be zoomed into, use 720dpi vector rendering (camera may zoom in 2–3x, 150dpi or video frames will be blurry). For word bounding boxes (for highlighter positioning), usepdftotext -bbox; it may crash with some poppler versions, then switch to PyMuPDF (seeexamples/demo/get-paper.sh, remember NFKC normalization, otherwise fi ligatures cannot be found). For web pages, useengine/tools/capture.mjs(3× screenshot + phrase bounding boxes). - Read the main text, appendix, and figure captions thoroughly. The appendix, ablation studies, failure cases, and limitations must be reviewed separately. Must read prompts, decision rules, and pseudocode in the appendix: the core mechanism 'on what basis does it judge' is often only written here.
- Write the paper card according to the 12 items in
references/paper-card.md. For single paper analysis, first grab the paper's own core figures and structure, don't impose your own framework.
2–3. Hook, Three Questions, Running Example, Audience Question Chain
Hook (each sentence ≤3 seconds, one piece of information per sentence):
- Shorten the authoritative name and put it first ("Google's new paper:").
- Follow immediately with the most unique, most counter-intuitive point, describe the scenario in plain language ("Helped a producer reconcile scheduling, personnel, and budget sheets simultaneously; scores went from 43 to 78"); for a single case, keep qualifiers like 'a real...', don't claim it's the average.
- Can end with a short reversal ("Not a single weight in the model was changed.").
- Then give the full name and organization, plus a preview ("Next, we'll break down its complete architecture").
- Don't string multiple numbers with the same unit in one sentence ("8B", "one-seventieth", "128,000" can be misheard).
- Don't make the audience understand a technical term before they can get the hook: use plain language for the event first, put the original English sentence only on screen; unfamiliar terminology can appear, but define it in the next sentence immediately.
- Verify the comparison object in the hook, write the limiting conditions; for every keyword in the hook and title, the main text must have a paragraph that directly addresses it (if you say 'self-evolving', there must be a paragraph 'Is this self-evolution? Where is it reflected?').
- Frame 0 is full-screen with the main title or the most counter-intuitive image (132px Songti, already in place, not pushed up from blank); key numbers appear on screen only when the voiceover mentions them and become the protagonist of that segment (≥200px), don't flash them at the beginning.
Three Question Outline: What was changed (level) | Why did it work (author's thinking + mechanism + why it doesn't fail, this should be over 50%) | What can we learn.
Running Example: Choose an example from the paper itself, covering every layer, and relate all mechanisms back to it.
Audience Question Chain (must do before writing the script): Write per paragraph 'What will the audience ask at this point? → How to answer in the next sentence'; template and required coverage items are in references/question-chain.md. Submit the question chain together with the logical chain for author review. Must include a 'cold water' section.
4. Voiceover
Structure: Hook → Problem (predecessors name a specific work) → If you were the author → One-sentence essence → Architecture (grouped according to the paper's grouping) → Walk through a paper example step by step (each step with 'why') → Summary card (one sentence connecting all steps) → Title concept Q&A → Results (question → answer) → Cold water (numbered) → What we can learn → Ending (discussion question → credits + call-to-action).
Hard Rules for Clarity
- A name is not an explanation. For every term/mechanism, upon first appearance, give a one-sentence operational definition: who looks at what and how they decide. Translating a paper term is not an explanation.
- Don't skip steps. There must be causal links between actions ("Seeing these two problems → we modified Harness's files").
- Concrete first, then abstract. First say which files were changed and what was written, then say what category or name it falls under.
- New concepts: set them up before using them. Don't let terms like 'candidate' or 'blind selection' appear for the first time in the cold water section.
- Put evidence after the steps, not numbers in the middle of the steps; when describing a controlled experiment, first state what the control group did, then the result.
- Make roles clear: when there are multiple models, explicitly say 'There are two models: X is the worker, Y is the one modifying the Harness'; clarify whether weights change, whether there is reinforcement learning.
- When a benchmark name first appears, explain what it is; use consistent units throughout the video ('points', not mixing 'percentage points'); 'it' must not be ambiguous.
- On screen, use only the terminology that appears in the voiceover.
Style
- Use English technical terms familiar to the audience (Harness, agent, diff) directly, with one consistent name throughout the whole video, don't coin new Chinese translations. Use industry terms (RL environment, trajectory, verifier), don't invent metaphors.
- Describe scenarios in plain language; translate tasks and benchmark names into a plain-language sentence first.
- If you were the author: after asking 'What would you think?', leave a 1-second pause (show a '?' on screen), then say 'I guess...'. Speculation must be preceded by 'I guess'.
- One-sentence essence: must be repeatable, followed by a supplementary sentence; don't write slogan-like short phrases (see self-check list).
- For long lists, group by the paper's own grouping, based on the paper's original table. Talk directly about the paper, don't force-fit it into the audience's tools.
- Walk-through: separate by 'First step... Second step...', making clear each time who has what and what they are doing, add a 'why'. For chosen examples, note 'This is a showcase example picked by the paper; the average improvement is not this large'.
- For each mechanism, follow: 'How it was done before → What was the problem → What it was changed to → Why this is better'.
- For technical mechanisms, include minimal before/after code comparison; use the original text if available (mark 'Original text excerpt · abridged'), otherwise write pseudocode marked 'Schematic'.
- Use analogies only as supporting material, placed after the example. Discuss approaches that were 'tried but not kept'. Before deleting information, ask: 'Will the audience misunderstand this if I delete it?'
- Results section: each number answers one question; don't read a scorecard; don't cram several numbers into one sentence. When the same 'max/best' comes from different comparisons, specify the scope.
- Cold water section: 'Three reservations. First...' with complete numbering; material from the devil's advocate subagent; when citing favorable numbers, also mention the counter-directional ones.
- Ending: don't lecture. 'What can we learn from this?' 'My biggest takeaway is...'; don't use command-style sentences like 'You can take away'.
- Conversational, short sentences; use Arabic numerals in subtitles.
- Before submitting for review, go through every sentence with
references/voiceover-selfcheck.md.
Word Count and Rhythm: Overall about 5.8 characters/second (Chinese characters, English words, and numbers each count as 1); 6–8 minutes ≈ 2000–2700 characters. Don't rely on speeding up; reduce pauses. Within a paragraph, string sentences together in one breath; leave a full beat only before questions, reversals, and key numbers; mark full beats with '‖' (3–5 per chapter). Target pause ratio 10–12%, average pause every 5–6 seconds.
Voiceover File Format (directly read by the engine): Section title ## 00 · Section Name, spoken sentences one per line starting with - ; > Screen card and explanatory text are not spoken. Sentence id = s<section>-<number>.
5–6. Blind Test + Fact-Check + Devil's Advocate (Independent Subagents, Run in Parallel)
- Listen-once blind test: two roles (A non-expert, B expert), fixed questions, misunderstanding checklist, scoring. Threshold: A ≥ 8 before proceeding to voiceover. Brief template
references/listen-test-brief.md. Address all issues raised in the blind read one by one; don't just add a single sentence. - Fact-check: check the voiceover against the paper, line by line for numbers, attribution strength, limiting conditions, comparison objects, whether 'speculation' is marked, whether groupings match the paper's original table; output must-fix items (directly replaceable sentences).
- Devil's advocate: given only the paper, take the reviewer's perspective: weakest assumption, unfair comparisons, cells where it loses to the baseline, sample size, counterexamples, one-week reproducibility; produce cold water section material. Brief templates for fact-check and devil's advocate:
references/factcheck-brief.md. - **After every round of script revision (especially after fact-check added numbers and limiting conditions), re-run the blind test + fact-check on the changed sentences. Pass accuracy and comprehensibility together. Calibrate thresholds against published scripts.
7. Author Reviews Voiceover (Stop and Wait for Confirmation)
- First give a one-page logical chain: one line per paragraph 'Audience's question → This paragraph's answer'.
- Don't paste a whole block of text: divide by section, with subheading + one-sentence explanation + voiceover one line per sentence; bold key conclusions.
- Review section by section, 1–3 sections at a time, write 'Now → Problem → Revised', ask yes/no questions that can be answered directly, provide context in plain language; if there is a reasonable default, go with it and state what you did.
- Record the author's preferences and update this skill when done. Proceed to voiceover and production only after confirmation.
8. Storyboard: Style Card, Shot Plan, Progress Bar
Style Card
Visual = Skeleton (structure, camera movement, zooming into paper, large canvas, transition timing, fixed for the whole video) + Style Card (color palette, fonts, materials, signature actions, fixed element styles). To change the style, just change the card. Use only one card per video; keep the same card within a series. Beat follows the voiceover; recompose each shot's composition according to the card.
Card ④ Editorial Data Journalism (Default, implemented in engine/common/style.js)
- Color Palette: Paper #FFF1E5 | Ink #33302E | Secondary #66605C | Thin Line #CCC1B7 | Wine Red #990F3D = Improved / key numbers / our annotations | Oxford Blue #0F5499 = Baseline / original | Cyan #0D7680 = match, link, ✓ | Gray #B9AFA6 = Rejected / neutral | Wheat Panel #F2DFCE.
- Fonts: Chinese title Noto Serif SC Black (hook 132px, chapter/shot main title 92px, cards 56–64px, leading 1.1); body text, labels Noto Sans SC; numbers Inter 800 + tnum. Kicker Bold 28–30px, with a 56×5 wine red short line above. Fonts are downloaded by
style-cards/editorial/get-fonts.sh(OFL). - Layout: kicker + large title + lead + figure; left margin 120px, strict column grid; 1px thin lines and 4px thick lines for hierarchy; charts draw only horizontal grid lines, labels placed directly on lines and points, no legend; annotations = thin callout line + one-sentence short explanation; source line in gray at the bottom left of each screen (≥26px, drawn uniformly by chrome.js).
- Signature Actions: title mask push up line by line (easeOutQuart 0.55s,
ED.maskLine); lead wipe in from left (wipeText); thin line grow from left (growLine); outer frame stroke clockwise (strokeCW); line chart draw from left to right, numbers scroll synchronously; key phrase underline sweep in wine red (underlineSweep); chart camera slow push 3–4%. No bouncing, no glows. - Transitions: between chapters, whole page slide left like flipping a newspaper page (0.6s, drawn uniformly by chrome.js); within chapters, continuous morph / large canvas.
- Fixed Elements: top newspaper section navigation progress bar, wheat-colored subtitle bar (Songti Bold 44px), source line—all drawn by
engine/common/chrome.js, chapters don't do these themselves. - Drawbacks: somewhat cold, many text layers; on mobile, 28px annotations may still be too small, delete when possible.
Other Cards (no implementation provided in repository; write your own style.js as needed): ② Hand-drawn Whiteboard (Excalidraw style, suitable for architecture breakdown); ① Math Animation (3Blue1Brown style, suitable for formula derivation; Hard Rule 8 recommends against using dark background for the whole video, but if really needed, dark background over white PDF pages, add brightness and vignette); ⑤ Dark Developer (suitable for engineering walkthroughs); ③ New Brutalism (suitable for short hooks, can be tiring for long videos); A Warm White Explanation (Background #FAFAF7, Focus Orange #D4552A = improved, Slate Blue #3E6A8A dashed line = baseline). When changing cards: animation methods stay the same, only change the easing character (④ no bounce) and chapter transition appearance; color semantics are fixed: improved = card's accent color, baseline = cool color, rejected = gray.
High-Density Paragraph Rules (All Cards)
- One protagonist per screen: whoever is being talked about takes ~65% width, camera push to 1.3–1.5x; when changing protagonists, the previous one shrinks to a top summary thin strip; only pull back to see the whole scene for 1–2 seconds during the sentence that makes the relationship clear.
- Text limit: ≤ about 25 Chinese characters per screen, besides subtitles; ≤ 4 visual blocks; don't write things the voiceover already said as labels; at most one label per element.
- No Chinese-English bilingual labels; all text that needs to be read ≥28px, source line can be 26px.
- Don't put illegible paper clippings: if you want to reference something, zoom in to full size and let it be seen for ≥1.5s, after that only leave the source line or a small angle mark.
- Restrained placeholders: for the next column, only leave a thin line + small column header, don't spread large gray skeleton areas.
- Top only leaves one navigation: steps for this chapter as sub-scales in the progress bar, no longer occupying a separate row.
All-Card Common Storyboarding Rules
- Core beat: original text page → yellow highlighter rgba(250,199,117,0.7) paints from left to right with the voiceover (with marker pen sound) → original text clearly stays on screen for ≥3 seconds (not counting entry, exit, and push); before re-drawing a chart, first show the original image and circle the points to be discussed.
- Two types of highlighters: marking key points on the original text = yellow (same for all cards); our annotations (callouts, circles, underlines) = card's accent color (④ wine red). The audience can immediately distinguish what is from the original and what is our interpretation.
- Fixed shots: walk-through shot | code comparison shot | hierarchy table shot | author's thinking shot (mark 'Speculation') | controlled experiment shot (how the experimental group vs. control group was done) | summary card.
- Subtitle bar background opacity ≥0.97, one line ≤ 22 characters, matching the voiceover word for word.
- Font size: text that needs to be read ≥28px; use ink color for informative text, gray only for metadata.
- No empty screens: content fills ~70%; at any moment (including intermediate frames during morphing and camera changes), the screen should not be half empty for more than 2 seconds. The landing frame must have its protagonist.
- Screen semantics must not be misleading: vertical arrows would be read as data flow; things that don't flow into the next character should be placed to the side, with dashed boxes and annotations.
- Mark schematic diagrams as 'Schematic'; charts must be to scale, axes must have labels; cover up paper logos, use only text for organization names.
- No random noise fill (Math.random grayscale grids, barcodes, skeleton strips); use 3–4 levels of smooth color steps for schematic diagrams.
- Titles must not overlap the old and new (new one enters after the old one exits). When a new image appears, the old image must completely exit, leaving no ghosting.
- Layers containing text must not be scaled using
scale()>1on bitmap, rebuild the layout at the new size. - 3D must use Three.js with lighting (rounded boxes, matte materials, hemisphere light + main light + fill light, baked contact shadows, render at 2× then downscale); CSS 3D is forbidden.
Safe Zone (Layout according to this during storyboarding)
- y 0–64: only for progress bar; top boundary of video elements ≥ y80, regular content within y 136–930, left/right 96px.
- y 900–940: source line; y 952–1040: subtitle bar. Chart numbers and labels don't enter these areas.
- Optional avatar: bottom right, approximately x>1660, y>840; content to be read placed at x<1640, y<940.
- Don't later overlay the progress bar / avatar onto the video (would only shrink the main content, and the subtitles would shrink too).
Camera Movement and Transitions (Three Techniques, Divided by Content)
Forbid 'fade to blank page and build from scratch' and full-screen cross-dissolve (like turning a PPT slide).
| Content | Technique | How It's Done |
|---|---|---|
| Most explanations: concepts, background, one-sentence essence, results, cold water | Continuous Morph (3Blue1Brown style) | Elements from the next shot grow out of elements from the previous shot: labels fly to the center and become the main graphic, title words shrink and merge into formulas, child elements pop out of parent elements when the voiceover names them, ? turns into a light bulb, question card flips into cold water card. Every incoming element must have an origin |
| Referencing paper original text, original figures | Dive into Paper (physical paper model) | The paper is a real sheet of paper with a shadow, about 1.5–2° tilt (real PDF page, no gray placeholder strip), usually scrunched in a corner; when needed for reference, it slides in and flips (0.8s, showing the back of the page), pushes in 2–3x to that sentence or region, highlighter passes; when talking about a figure, that part of the figure 'floats' out and flies to become a re-drawn animated card. The floating screenshot stays to the side without covering the target card, then returns to a corner minimap. The push-in shouldn't exceed the screenshot resolution |
| Paragraphs with sequential steps (walk-through, workflow) | Large Canvas Camera Move (Kurzgesagt style) | Steps are laid out left to right on one large canvas; the camera moves between steps (0.5–0.9s easeInOutCubic + whoosh); a paper minimap in the corner. The skeleton for the next station is pre-built, visible during the move; off-screen, the adjacent card cut in half is dimmed. Use only 1–2 segments per video. After finishing, pull back to see the whole picture, then shrink into a summary chain |
- Full video skeleton: continuous morph as a base → dive into paper every time the paper is mentioned → large canvas for steps → chapter transitions of ≤0.6s between major sections, only 5–6 times.
- Animation syntax: entry ease-out (④ no bounce), child elements offset by 60–120ms; camera easeInOutCubic; the same object across shots remains the same element (interpolate position/scale), don't copy in a new one with a fade-in; when a card scales, the content scales with it, don't clear it first and then scale.
- Motion blur: 12–16 equal-weight samples, trail ≤ half the frame's displacement and ≤40px; no directional blur during push/pull; for whip pan, full-screen blur ≤0.35s.
- First produce the shot plan (
references/shot-plan.md): for each paragraph, write which technique is used, what element from the previous shot remains, which element the next shot grows from, the original text source, anchor words. Only after the author has seen it, proceed to visuals.
Top Navigation Progress Bar
- Divided into 6–8 columns according to voiceover paragraphs, on screen use two-character newspaper section names (Introduction / Background / Thinking / Breakdown / Results / Doubts / Insights...); 'Hook' and 'Cold Water' are internal names, not on screen.
- Configuration is in the project directory
film.jsonundernav:[{ "n": "01", "name": "Introduction", "sections": ["00"] }, ...]; start/end times come from timeline.json. Current column wine red Songti, played progress wine red line, last 2 seconds of each column pulse next column as a preview. Implementation:engine/common/chrome.js.
Avatar (Optional)
Not shown by default. When used, use an image you have the right to use; prepare a transparent background avatar track, composite it as the last layer in the bottom right safe zone; show it throughout the whole video or only at the beginning / chapter transitions / end; when the camera pushes into the paper (≥1.4x, or when the original text would fall into the bottom right), slide the entire avatar out of the screen; when pulling back, slide it back in, each 0.35s, no semi-transparency. Layout according to the safe zone during storyboarding.
9. Voiceover (Any TTS)
- Use any TTS that can output wav/mp3 (commercial webpage, API, local model), generate per paragraph (each batch ≤ about 450 characters; longer ones tend to miss words). Write numbers and version numbers in their Chinese reading form (forty-three point one, Opus four point eight); keep English terms; use Arabic numerals in subtitles (numfix in chapter.json).
- For each batch, first do speech recognition to check the first and last words and English proper nouns (
engine/voice/align.py asr); re-generate incorrectly read sentences individually, write into theoverridesin voice.json. - One verified approach (example): a certain commercial TTS webpage, pick a tech explanation style voice, 1x speed; each segment: 'paste script → auto-add emotion tags → generate → download the 1st result'; when using browser automation, open a new tab for itself, don't touch the tab the user is looking at. The voice and account are your own, not distributed with the repository.
- Before delivery, do a word-by-word transcription check for the first and last words of the original audio; fade-out ≤ 2 frames.
10. Alignment and Audio Tracks
uv run --with numpy python engine/voice/align.py voice.json: recognize results, align voiceover by character → snap sentence boundaries to silence → remove leading and trailing silence from each sentence, compress internal long pauses to ≤0.30s, atempo 1.05, 8ms fade in/fade out → concatenate outputvoice/voice.wav,timeline.json,clauses.json.- Pause parameters (latest published video): intro 0.3s, between sentences 0.35s, between paragraphs 0.7s, outro 2.5s; before questions, '‖' marks, and key numbers, can increase to 0.8–1.0s.
- timeline.json is the sole time source for the whole video: progress bar, subtitles, anchor words, SFX cues all come from it. To add sentences, just add a new segment and re-run alignment. When there is no voiceover yet, use
engine/tools/mk_dummy_timeline.pyto generate a placeholder timeline for making visuals first. - Background music (
engine/audio/audiolib.py music): synthesized with numpy, no copyrighted material. Warm pad + soft arpeggios + bass, F major Fmaj7–Dm7–Bbmaj7–C7, 84 BPM; approximately −28 LUFS; ducking about 7 dB when voice is present (attack 60ms / release 450ms); intro/outro fade in/fade out. - Sound Effects (
sfx, about 19 dB lower than voice; 24 dB is inaudible on phone speakers): camera movement / chapter transition = whoosh; highlighter = marker; cards / labels / ✓✗ = pop (aligned to the frame the element actually enters); number scrolling = tick; paper floating / page flip = paper. Don't play whoosh when there is no visual movement. Timing is registered bydata.cue()in the scene, using the same time source asrender(t). - Mixing (
mix): two passes of loudnorm I=-16:TP=-1.8:LRA=11 → AAC 192k; after encoding, measured true peak ≤ −1.5 dBTP. - Before delivery, listen once with headphones and once with a phone speaker (adjusting only by measured values doesn't count; if the agent can't listen, ask the author to listen and point out the densest parts of the voiceover).
11. Production and Rendering
First build the common components, then assign tasks (API docs engine/README.md)
timeline.json/clauses.json: timing for the whole video; chapter times = section start/end for this chapter, chapter uses local seconds.film.json: chapter order, progress bar columns, ending credits.engine/common/chrome.js: progress bar, subtitles, source line, chapter start newspaper flip. Chapters only drawrenderContent, never make fixed elements themselves, thus the whole thing has only one set of UI.- Chapter brief: full text of style card + high-density rules + API + fact rules + budget; write from the start: 'layout fills ~70% of the screen, no frame (including intermediate frames during morphing and camera changes) can be nearly empty, shot main title 92px, no gray placeholders'.
- Mother graphic: write the code for the shared mother graphic as a spec file (e.g.,
examples/always-on/MOTHER.md), each chapter copies it verbatim.
Parallel Per Chapter: one production agent per chapter, the main controller only does common components, unified rendering, and concatenation.
- The
renderContent(ctx, t, data)function in scene.js must be a pure function oft: no CSS animations, no timers, random numbers use a seeded PRNG. Cut points can only be calculated from anchor words (data.A(sentence_id, substring)), don't hardcode seconds. - Chapter transitions: at
t=0of each chapter, the layout must already be complete; the last 0.5s should be static.engine/tools/prep_render.mjsexports the last frame of the previous chapter without chrome to use as the background for the next chapter's page flip (prev.png). The first chapter can implementrenderPrevto draw its own 'previous page'. - For very long chapters (>90s), split into two agents, use a single whip pan for the handover: the element coordinates, canvas coordinates, and final velocity on the handover frame must be hardcoded in the brief.
- If common components are changed, broadcast to all chapter agents; re-render the affected frames for already rendered chapters; if unsure, re-render the entire chapter.
- Self-checks by production agents: only grab static frames (headless open shell page
seek(t), screenshot, ≤10 frames per round); final video is only done via WebCodecs.
Rendering (Sole Path)
node engine/tools/serve.mjs 8771(listens only on 127.0.0.1).- Local Chrome, new tab:
http://127.0.0.1:8771/engine/common/shell-wc.html?ch=<chapter directory>. - Console
await probe(): checkwebglRenderer(should be a real GPU, e.g., Apple M series / Metal, NVIDIA, AMD) and whetherprefer-hardwareis supported; ifsoftware: true, stop and report. startRender({ to: 10, download: false })to test 10 seconds, check__prog.msPerFrame(typically 5–15 ms/frame on GPU).startRender({ name: 'chNN.mp4' })→ poll for__prog.state === 'done'→ browser download. Each tab downloads only once (Chrome silently blocks the second download), one output per new tab, close after render.- After download,
cp(do notmv) to<chapter>/build/video.mp4,ffprobe -count_framesto check frame count =ceil(T×30). node engine/common/assemble.mjs --project <project directory>: frame-by-frame concat, music, SFX, mixing, package; also output a <30MB lightweight version. Don't use -shortest for mixing; audio/video sync error < 1 frame.
Order: Rough Cut → Independent Review → Full Video → Art Polish → Re-review → .1 Revision
- First produce two fully composited rough cut segments: A = hook + first dive into paper (about 30 seconds); B = first one or two stations on the large canvas (about 40 seconds). Show to the author for visual and volume feedback, and give to the review agent.
- The review agent is independent of the production (only sees the final video + this skill + the paper card), checks against
references/review-checklist.mditem by item, outputs must-fix / should / nice, with timecodes and specific fixes. - Apply must-fix items uniformly across the whole video, not just within the rough cut scope.
- Art polish per chapter: after content and timeline are locked, each chapter goes to a visuals-only agent (who doesn't change voiceover, shot order, or numbers). For each shot, grab 3 frames (entry mid-point / settled / exit beginning). The shot must be able to stand alone as a poster. Check: four levels of fonts, protagonist big enough (key numbers ≥200px), signature actions used, color semantics not confused, aligned to column grid, no 'default gray frame + small text' PPT cards. If content issues are found, log them to the review and return them to the production team.
- Full video review again, fix, output .1 version. Don't send a 'final version' before one round of review.
12. Self-Check and Delivery
- Follow the items in
references/review-checklist.mdfor the final check before delivery.