clip-builder

Clip builder

Build a short video from three parts: an HTML page that renders the picture (the scene), a list of script lines in clip.json (the cues), and a command-line tool that captures the page frame by frame and encodes an MP4.

The scene page defines draw(s), which sets the page to its state at second s, computed from s alone. The capture calls it for each frame (second 0, then 1/30, and so on) and takes a screenshot. As a result, output does not depend on machine speed, any frame can be rendered on its own for review, and a change to one part of the clip can be re-captured with --from and --to.

The CLI is scripts/clip.mjs in this skill's directory. In this document clip means node "<skill dir>/scripts/clip.mjs", where <skill dir> is the absolute path of the directory containing this file. (Claude Code substitutes that path here: ${CLAUDE_SKILL_DIR}. In other agents, use the directory this file was read from.) Run commands from the clip's project directory, which contains clip.json. clip help lists every command and option.

Requirements

Node 20 or later, ffmpeg, and Playwright with Chromium. clip doctor checks them and prints install commands for anything missing. Install Playwright in the directory where clips are built (npm install --save-dev playwright, then npx playwright install chromium), not in this skill's directory. Narration and generated music are optional. They are the only steps that cost money or need an API key.

Sandboxed commands (the default in Codex): clip stills, clip capture and clip serve start an HTTP server on 127.0.0.1, and the first two also start Chromium. A command sandbox blocks both. The command then exits with status 1 and a message that names the sandbox. Request approval to run these three commands outside the sandbox, and state the reason. Do not substitute another capture method. The other commands need neither a server nor a browser and run inside the sandbox; tts --generate and music --generate also need network access.

Brief

Establish the following before starting. Ask only for what is missing. For anything the user has not specified, use the default and state it in one line.

• Purpose and audience: promotion, product demo, staff how-to, lesson. This sets the pace. Promotional clips move quickly; training clips leave time to follow each step.
• Length. Default: 20 to 45 seconds for a promotion or demo, 60 to 120 for training.
• Destination, which determines the frames. Default: wide 1920×1080, plus tall 1080×1350 for social feeds. See references/delivery.md.
• Visual approach: new graphics or an app screen. For an app: which screens, and whether its source code or design system is available as the source for colors, type and spacing.
• Sound. Default: silent with captions. This is free, and most feed video plays muted. Narration and music are each opt-in.
• Script. Use the user's script if they have one. Otherwise draft one and get their approval before building the scene.

Workflow

1. Check the tools: clip doctor.
2. Create the project: clip init <dir> --template graphics or --template app-ui. This writes clip.json, a scene/ directory with a working example, and audio/. Both templates render without changes.
3. Write the cues in clip.json, one per beat. A beat is a segment of the clip with one line and one visual state. Write the script first: the scene is timed to it, and every wording change after recording is another paid request. references/narration.md covers writing for speech.
4. Build the scene: edit scene/index.html and scene/scene.js. Read references/scenes.md first. It lists the constraints that keep a scene deterministic: no Math.random(), no clocks or timers, no CSS animations, all state computed from s. For an app screen, also read references/app-ui.md.
5. Review stills: clip stills writes frames at selected seconds and a contact sheet to out/review/. Open the contact sheet as an image and inspect it. Check that text is legible, nothing overlaps, and each beat is on screen while its caption is shown. Repeat for every frame size (--frame tall). Do most of the iteration at this step: stills are fast and cost nothing. clip serve serves a live preview for the user.
6. Capture a draft: clip capture --draft renders at two-thirds size for checking motion and pacing. Give the user the file path and ask them to watch it.
7. Add sound, if requested. Narration: references/narration.md. Music and the mix: references/sound.md. Run clip tts without flags first; it lists the planned requests and character counts. Get the user's approval, then run clip tts --generate. After any new recording, run clip timing, which places the cues and captions and, if configured, stretches the scene to the narration. Then review stills again.
8. Capture the final: clip capture --final for each frame, then clip mix if there is sound, then clip web <file> for a smaller copy to upload.
9. Verify and report: review stills from the final, read the loudness in the mix report, and run clip scan out. Report what was produced, the file paths, what was checked and what was not.

A silent clip skips step 7. Its captions come from each cue's at and until values, and clip timing writes them for the page.

Safety rules

A clip job involves the user's money, API keys and published identity.

• Paid requests require explicit user approval. The provider bills narration and music. The CLI sends nothing without --generate; pass that flag only after the user approves that specific take. Approval for one take does not cover the next. Show the dry-run output (lines and character counts) when asking. If a request is interrupted, its receipt stays attempted. Do not retry until the user has checked the provider's usage history, because the request may have been billed.
• API keys come from the environment only. The CLI reads ELEVENLABS_API_KEY or OPENAI_API_KEY and no other source. Pass a key inline for a single command from the user's secret store (references/keys-and-paid-calls.md has the commands). Do not ask the user to paste a key into the conversation, do not print one (echo, env, cat), and do not write one to a file. To confirm a key is set, check its length; clip doctor does this.
• Deliver files; do not publish. Posting or uploading a clip is the user's action unless they request that specific action.
• Use only material the user has the right to use. Music, fonts, logos and footage need a license or the user's ownership. Show only an app the user owns or has permission to show, and do not style a mock-up as another company's product. Both providers' terms cover commercial use and the disclosure of generated voices; point the user to them before a narrated clip is published.
• Keep private data out of the picture and the files. Use fictional names and numbers in app screens. Figures and quotes presented as real must be real, and examples must be identifiable as examples. Run clip scan on anything before it is shared. It checks for API keys, the local user name and paths, email addresses, the user's list of private terms, and metadata embedded in media files.

Editorial guidelines

• Cover one topic per clip. Split a script that covers two.
• Start the narration or the first caption within the first few seconds.
• Use one cue per beat, and keep a beat's visual on screen while its line is spoken.
• Always include captions. Most feed video plays muted.
• Review tall frames at phone size. The scene unit is larger in tall frames for this reason.
• Use motion to mark a state change or to direct attention. Avoid continuous movement.
• Hold the closing card for at least 2.5 seconds.
• For training: show the action, then its result, at a pace the viewer can follow while performing it. Number the steps on screen, and state at the start what the viewer will be able to do.

Commands

Command	Function
clip doctor	Checks Node, ffmpeg and Playwright, and reports which keys are set (length only)
clip init <dir> --template graphics|app-ui	Creates a project from a working example
clip serve	Serves a live preview; ?t=5 in the URL holds at 5 s
clip stills [--at 2,5,9] [--frame tall] [--final]	Renders review stills and a contact sheet
clip capture [--frame wide] [--draft] [--final] [--from s --to s] [--web]	Renders every frame and encodes an MP4
clip capture --mode clock --url … --seconds N [--actions file.mjs]	Captures a running app (experimental; see references/app-ui.md)
clip tts [--generate]	Lists the narration still to record; with --generate, records it (paid)
clip music [--generate]	The same for a generated score (paid)
clip timing	Computes cue and caption times; writes scene/timing.json and an .srt
clip mix	Builds the soundtrack at a standard loudness and muxes it into each captured video
clip web <video>	Writes a smaller copy for uploading
clip scan <dir> [--terms-file words.txt]	Reports private data before files are shared
clip strip <media file>	Writes a copy with all embedded metadata removed

Output goes to out/, named <name>-<take>-<frame>.mp4, with the suffixes -draft, -part (a range captured with --from/--to), -sound and -web added as they apply. To keep an earlier version next to a new one, increment "take" in clip.json (a, b, c, …).

Reference files

Read the file that the current step needs.

File	Read it when
references/scenes.md	Writing or changing a scene: the page contract, the helper library, units, tall frames, fonts, and the constraints that keep a capture deterministic
references/app-ui.md	The clip shows an app screen: rebuilding it from design tokens, the pointer, typing, camera moves, and capturing a running app
references/narration.md	Writing lines, choosing a provider and voice, pronunciation, re-recording one line, fitting the scene to the narration, captions
references/sound.md	Music and the mix: levels, ducking, fitting a track to the clip, loudness, fixing a mix
references/keys-and-paid-calls.md	Before any paid request: passing a key safely, what is sent, receipts, interrupted requests
references/project-file.md	Any field of clip.json
references/delivery.md	Frame sizes, the final render, checks before delivery, what to report
references/cookbook.md	The request matches a common job. Twelve worked recipes: a silent clip, an app screen from design tokens, several aspect ratios, a step-by-step training clip, a bar chart from JSON, narration with each provider, re-recording a line, background music, a custom font and logo, a running app, final checks. Also draw() snippets

Troubleshooting

Symptom	Likely cause
"This looks like a command sandbox", listen EPERM, or the browser fails to start	The agent's sandbox is blocking the HTTP server or the browser. Run stills, capture and serve outside it, with the user's approval
"The page never set window.ready"	A script error in the scene (the message lists the page's errors), or the page does not call the kit's clip()
Two captures of the same scene differ	Some state in the scene is not computed from s: Math.random(), Date, a CSS animation or transition, a <video>, a timer
Text renders in a different font on another machine	The scene uses system fonts. Put font files in the scene directory (references/scenes.md)
The picture is ahead of or behind the narration	clip timing was not run after a new recording, or the video was captured before it. clip mix refuses to run in the first case and warns in the second
attempted in the tts list	An earlier request did not complete. Check the provider's usage history before using --retry-attempted
Music level rises and falls between words	Set "mix": { "master": "limit" } (references/sound.md)