geoqiao/short-video-skills

Creating Short Videos with Code

The visuals are determined solely by time: the page exposes window.render(t), and given a second, it draws the scene for that moment. The rendering script uses a headless Chrome to call it frame by frame and take screenshots, while ffmpeg combines the screenshots and voiceover into an MP4. Changing the visuals means changing the code and re-rendering.

This approach originates from a post by xilo on X (2026-09-29, https://x.com/xilo2991/status/2104912748794589515). This skill builds upon it by adding voiceover alignment, covers, still frame checks, and a set of conventions for scripting and visual design.

Getting Started

You'll need Node 20+, uv, ffmpeg, and locally installed Google Chrome. The default fonts are macOS's Songti, PingFang, and SF Mono. For other systems, replace them in brand.json's fonts with fonts available on your machine.

1. Copy the entire template/ directory from this skill into a new project directory.
2. Run npm install (installs only playwright-core) in the new directory.
3. Edit brand.json: series is the series name in the top-left of the cover, handle is the credit on the cover and end screen, avatar is the relative path to an avatar image (can be null), colors and fonts define the palette and typefaces.
4. Run the example:

uv run tools/voice.py demo                # Voiceover, background music, timeline -> episodes/01-demo/build/
node tools/render.mjs demo --stills auto  # Export a still frame at the end of each shot
node tools/render.mjs demo                # Render the full video -> episodes/01-demo/out/video.mp4 and cover.png
sh tools/covers.sh demo                   # Generate 4:3 and 3:4 covers
sh tools/sheet.sh demo                    # Combine cover and last frames of each shot into a contact sheet build/sheet.png

The command only needs the slug; the script automatically finds episodes/<episode-number>-<slug>/. The voiceover sends the narration text to Microsoft's online TTS service.

Project Structure

• brand.json: Branding settings. Edit this, not the engine.
• src/lib.js: Drawing utilities, mounted on window.E.
• src/run.js: Timeline scheduling. Frame 0 is the cover, pausing 0.3 seconds before panning into the first shot; shots are arranged horizontally and connected by pans; subtitles and progress bars are also drawn here.
• src/stage.html: The rendering page.
• tools/voice.py: Synthesizes voiceovers line by line, uses word-level timestamps to determine clause start points, adds background music and alert sounds, and normalizes loudness to -14 LUFS. Retries on incomplete synthesis, errors after three failures.
• tools/render.mjs: Fetches frames from four parallel pages, ffmpeg encodes at 30 fps.
• tools/covers.sh, tools/cover-vertical.mjs: Generate 4:3 horizontal cover (16:9 cover with top/bottom background color fill) and a separately laid out 3:4 vertical cover.
• tools/sheet.sh: Extracts the cover and the last frame of each shot from the final video to create a contact sheet.
• episodes/<episode-number>-<slug>/: One directory per episode, containing script.json, shots.js, and generated build/ and out/ folders.

Workflow for One Episode

Follow these steps in order; don't move to the next step until the current one is complete.

Content and facts are your responsibility; this skill starts from writing the script.

1. Script: Create episodes/<episode-number>-<slug>/ and write script.json. Write the episode number into cover.no.
2. Visuals: Write shots.js in the same directory. The number of shots must equal the number of narration lines.
3. Voiceover: uv run tools/voice.py <slug>. Re-run if you change the narration, voice, or cover text.
4. Still Frame Check: See "Still Frame Check" below.
5. Final Video: node tools/render.mjs <slug>, then run covers.sh and sheet.sh. Review the contact sheet.
6. Delivery: See the last section of this document.

When adjusting episode numbers, change both the directory name prefix and cover.no, then re-run voiceover and rendering. To change only the cover text without redoing the voiceover: edit the cover in script.json, replace the cover in build/timeline.js with the same content, then re-run rendering and covers.sh.

script.json

{
 "title": "This video is drawn by code",
 "cover": { "no": "01", "lines": ["This video", "is drawn by code"], "hot": 1, "tag": "No editing software, no video model" },
 "voice": "zh-CN-YunxiNeural",
 "rate": "+6%",
 "lines": [["Every frame you see", "is drawn by code."], ["The page exposes just one function:", "give it a time,", "it draws the scene for that moment."]]
}

• A line is a shot, and each clause within a line is a visual beat. Clauses can be written as [spoken text, subtitle text]: say "seventy-five percent" while displaying "75%"; use this to correct the pronunciation of English terms.
• Cover: lines has at most two lines, hot is the line using the main color, tag is a hook with a specific number or conclusion, no is a two-digit episode number. The cover title, platform title, and first line of narration should all convey the same thing.

How to Write the Script

• For vertical-scrolling short videos, 75-90 seconds and 8-9 shots are appropriate. With a +6% speech rate, about 4.8 words per second (including pauses), narration should be around 380-430 words (Chinese characters plus English words). Cut text before voiceover if it exceeds this.
• Lead with the most compelling point in the first two seconds. Before writing the script, answer: what is the single most captivating point of this episode? Typically, it's a counterintuitive number, a specific result, or a problem the audience is facing. In the template, narration starts at 0.5 seconds (HEAD in tools/voice.py), so by 2 seconds only 8-9 words can be spoken. Therefore, the first clause must be short, with its opening words being that point; the first shot should display it in large text before 1.2 seconds, and the cover's tag should also mention it. Use the audience's own language, including the topic keyword (e.g., "Don't max out inference intensity," then explain "Going from High to Max costs 2.9x but scores only 0.6 more points"), before moving into the main content. No preamble, no self-introduction, no "Today we'll talk about," no bare questions like "What is X?" and no filler like "Same A, same B." This is an observation from limited video data, treated as pending validation.
• The following structures can serve as starting points; adjust based on content.
• Tutorial: Final result → One shot per step → Common pitfalls → One-sentence conclusion.
• Product introduction: Problem it solves → Two or three key features, one per shot → How to start → One-sentence conclusion.
• Concept explanation: Conclusion and numbers → Old methods or common misconceptions → Where the problem lies → New approach → Specific mechanism → Benchmarks or official data → One-sentence conclusion.
• Research finding: Conclusion and numbers → Common misconceptions → How the experiment was done → Results → Follow-up research → Reason → One-sentence conclusion.
• End with a one-sentence summary of the conclusion or actionable advice, followed by a question the audience can answer from their own experience ("Which setting do you usually use?") to encourage comments. The question should echo the pinned comment. This is also pending validation; monitor comment count changes. Don't create a separate shot to discuss limitations; write qualification conditions in the data shot's source footnote.
• Don't say "In the next episode" or "As mentioned in the previous episode" since release order may change. To cross-reference, say "the episode about [topic]."
• When the original source gives a margin of error, describe differences within the margin as "almost unchanged." Any derived numbers should be flagged as weak evidence during delivery.
• Don't include URLs in the spoken narration, and don't say "Go to website X" or "Search for Y" (platforms treat this as outbound traffic; see the video-publish skill).

How to Draw the Visuals

shots.js is an immediately invoked function that ends with window.SHOTS = [...]. Each shot is a (c, k) => {}:

• c is a 1920×1080 Canvas 2D context.
• k.at(i, delay, dur) returns an eased progress from 0 to 1 for the i-th clause, starting delay seconds after the clause begins, over dur seconds; k.raw is the same but linear; k.t is the time elapsed within the shot, k.d is the duration of the narration line, k.segs are the start times of each clause relative to the shot's start.
• Drawing utilities are in window.E: title (shot title with brand stroke), txt, measure, rise (fade-in plus displacement), pop (pop-in), rrect, path (draw a polyline based on progress), arrow, chip (label block), code (typewriter code block), stroke (brand stroke), mark (avatar), sign (end screen credit), fmt, with colors C and fonts F. Read the function signatures in src/lib.js before use. For missing tools, write them in the episode's own shots.js; don't modify lib.js or run.js for a single shot.

Rules:

• Each clause should correspond to a new visual element. Something must change within 2-4 seconds. Each shot covers one idea, expressed visually: process arrows, dot matrices for quantities, comparison bars, code blocks, tables—never a full screen of text.
• Left-aligned and asymmetrical. Avoid dark backgrounds with large centered text, avoid fading all elements in together, and avoid glow effects.
• Use the primary emphasis color to mark "the number to look at this frame," not as a value judgment; use the accent color at most once or twice per screen.
• Determine font sizes for mobile vertical viewing of a horizontal frame: txt boosts sizes below 56 (e.g., 24 becomes 34px, 30 becomes 38px). Informational text should have an input value of at least 30; source footnotes can use 24. Width estimation uses the boosted size, with monospace characters roughly 0.6 times the font size.
• Layout boundaries: subtitles are below y=932, one line holds about 30 full-width characters; the cover hook string hits the avatar at about 14 full-width characters; shots have a slow 1.025x zoom-in; edge elements maintain at least 20px padding; a 24px footnote line holds about 48 full-width characters.
• Songti Black lacks the "÷" glyph, and "×" renders as a large cross: use PingFang for operators, or write out "divided by" or "times." Descenders of English letters in 200px headlines may overlap the next line.
• If the first clause is very short, the beginning of the shot may feel empty: place the visual skeleton (axes, table headers, outlines) at the first clause's entry, with data following in later clauses.
• Icons not drawn from measured data should be labeled "schematic." Numbers read from source charts should have footnotes stating "Read from original chart." Data shots require source footnotes at the bottom, containing only the name ("Source: Official Documentation of X")—no domain names or URLs.
• Every shot should use E.title for its title; the final shot uses E.sign for credits.

Still Frame Check

Run node tools/render.mjs <slug> --stills auto to export the last frame of each shot; --stills 0,t1,t2,… exports frames at specific seconds (0 is the cover). Intermediate frames are taken about 0.4 seconds after each line's start and about 1 second after each clause, with timestamps read from build/timeline.js. Each --stills run clears the previous batch, so either check in batches or combine timestamps into a single command. For many frames, use ffmpeg to create a contact sheet; check the cover and last frames at full resolution.

Verify frame by frame:

• Is text overflowing cards, code blocks, table columns, or being obscured by the subtitle bar or progress bar?
• Are right-aligned numbers touching left-side labels? Are axis labels overlapping with curve labels?
• Is the last frame of each shot the most information-rich? Are there any elements not yet entered?
• Are the two cover title lines too small? Has the hook string exceeded the width limit?
• Are there 2-3 second stretches with nothing new, especially at the beginning of a shot?

Fix any issues found; don't render the final video with known overflows. For visual-only changes, simply re-run rendering.

Known Limitations

• edge-tts uses an unofficial Microsoft API. Version 7.2.8 started returning 403 from October 2026 (rany2/edge-tts #490). tools/voice.py pins to a commit based on the fix PR #491; when a new official version is released, switch back to edge-tts==<new version>. If 403 appears again, check the upstream repository's issues.
• The agent cannot hear the voiceover: the pronunciation and speed of English terms require human verification. Mid-synthesis stream interruptions are handled by voice.py's duration check.
• No motion blur or beat-matching to music.

What to Include in the Delivery

• Note that no one has reviewed the voiceover.
• Results of the still frame check, including any remaining known issues.
• Paths to the final video, three cover images, and the contact sheet, along with the video duration.