Canvas Film
You are making a short film where every frame is drawn by code as a pure function of time. A voice drives the timing, the kit in this folder draws the frames, synthesizes the sound and renders the MP4. Your job is the film: the story, the script and the drawing file.
Two looks, one pipeline. Pick with "kit" in motion.json:
| Look | "kit" | What it is | Start from |
|---|---|---|---|
| Block talk | "block" | Two 3D block characters in conversation, a step pill and a card at the top, big word captions | example-3d/ |
| Ink world | leave out | One hand-drawn 2D world on a ground line, a camera moving through it, a headline | example/ |
Paths below are relative to this skill's folder. FILM is the folder you create for the user's film.
python scripts/film.py voice FILM # voice.mp3 + words.json: the time of every spoken word
python scripts/film.py build FILM # motion.html (open it: live preview), captions.srt, score.wav
python scripts/film.py check FILM # rebuilds, then shot stills, contact sheets, layout and quiet-stretch report
python scripts/film.py render FILM # rebuilds, then final.mp4
What to run again after a change:
| You changed | Run |
|---|---|
any say text, a voice, a gap | voice, then check |
cues, heads, pill, snd, or film.js | check (it rebuilds first) |
Start every new film by copying an example folder to the user's folder and running voice and check once, so you know
the tools work before you write anything. If a tool is missing, give the user the install line from README.md; don't
work around it. python scripts/film.py narration FILM prints the word count and a length estimate.
1. The story sentence comes first
Write one sentence with four parts before any script: who wants what, what is in the way, what they do about it, what
changes because of it. Put it in film.js as const STORY = "...". An explainer has a story too: the viewer's
question is the goal, the confusing part is the obstacle.
- If you can't write the sentence, the idea is a list of facts, not a film. Find the one thing that has to get somewhere.
- Give the idea a body: an actor the eye can follow from the first frame to the last. In a block film that is the cast.
- Each beat causes the next. Show the obstacle before it is beaten.
- Use a different structure for each film. Don't reuse the last film's order of shots with new nouns.
- End resolved: the last picture shows what changed. Then one ask, not three.
Facts: use only what the user gave you or what you have read at a primary source in this session. Keep the source's own hedges ("may", "expects", "plans to") in the script and in on-screen text. Never invent a number. Never claim the film does something it doesn't: if the script says "it jumps on the word", it must.
2. The script: motion.json
{
"title": "A dot, a door and three words",
"tag": "your channel",
"lines": [
{"say": "This is a canvas film. A dot walks in, and a door draws itself.",
"gap": 0.35,
"cues": {"dot": "dot", "door": "door"},
"snd": {"dot": "pop", "door": "print"},
"heads": [["A *canvas* film", ""], ["It *draws* itself", "door"]]}
]
}
title: the page title.tag: the small channel name at the top left. Ask the user for theirs, or leave it out.say: one or two spoken sentences. Voices read 2.5 to 3.3 words a second; the film adds each line'sgapand a 1.7-second tail. Line one is the hook: the subject and the surprise in the first five words.cues: a name for each visible event, mapped to the spoken word that event lands on."token#2"is the second time the word is said in that line. Spell the word as it is insay.buildwarns when a cue names a word that isn't in its line.snd: one sound for a cue:pop pop2 tick ding thud buzz lock scan print swish slam boing. Silence is fine.cap: caption text when it should differ fromsay("$108 billion" for "108 billion dollars").gap: silence after the line, in seconds.
Ink world only:
heads: the headline. Each is[text, spoken word that swaps it in]; the first appears at the line start, and the last one stays only until the next line starts, so give each at least a second. Two to four words.*word*is the green accent; no punctuation straight after a closing*.src: a small source line at the bottom.end: trueon the last line clears the headline for an end card.
Block talk only:
"kit": "block", and"cast": {"dev": {"col": "#FFC83D", "say": "Daniel", "edge": "en-US-AndrewNeural"}, ...}: one entry per character, with the colour its captions light in and its voice for each engine.whoon every line: which character says it. Short lines, 4 to 11 words, and swap speakers often.pill: the label at the top for that line, like"STEP 2 · *SCRIPT + VOICES*"."steps": 5at the top level plus"step": 2on a line lights the progress dots.shout: words to show in capitals in the captions, like["jump"].- Captions are automatic: up to three words at a time, each lighting as it is spoken.
3. Voice
voice makes one take per line and times every word. Read what it prints: a low transcript match or "heard N words"
means a word was dropped or misread, so re-take. Engines: --engine say (macOS, offline, the default on a Mac),
--engine edge (Microsoft Edge voices through edge-tts), or --file take.mp3 for the user's own recording of the
whole script. A block film takes each character's voice from cast. Ask the user which voices they want before the
final render. Re-voicing moves every time in the film, which is fine: nothing in film.js is a fixed second.
4. The drawing: film.js
Read the example's film.js first, then reference/api.md. Every film file has:
- Times from words.
const A = cues();thenA.dooris when the cuedoorlands. Other moments:T(i)(line start),E(i)(line end),wordAt(i, "word"). Offsets from those are fine (A.door + .3); a bare number of seconds for when something happens is not. A misspelt cue name prints a warning incheck: fix it, don't ignore it. - The storyboard as data.
shot(...)for each camera setup andhit(...)for each impact. The shot'snotesays what the frame must show;checksaves a still at itsbeatso you can compare the two. - Actors. Anything that moves has one continuous function of
t, so it can never teleport between shots. const FILM = {...}: the functions the engine calls every frame.
Block talk: FILM = { frame(t, cam), card(t, lineIndex), overlay(t) }
- Build the cast once at the top from
box,faceBoxandpivot(see the example'sdevanddot). Design your own characters: change proportions, colours, one signature prop (a hat, an antenna). Faces are painted on a small 2D canvas every frame, which is how mouths move:talk("dev", t)is how wide that character's mouth is open now. frameposes everyone for timet: who is speaking leans in and gestures, who is listening looks at the speaker. Give each line one physical gag tied to a word (a jump on "jump", a hat on "hat", coming apart into blocks).- Screen layout to keep: pill at y 236, card from y 318 to 650, captions at y 742, heads below y 820. Aim the camera
so faces sit in the lower half; hide a character (
g.visible = false) rather than leave a sliver of them in a close-up. carddraws what the current line is about in the card area, with the 2D drawing calls: typed code, a prompt, a waveform, checked frames, a big call to action. One idea per card, and something on it changes on a spoken word.- Cut on every change of speaker (
cut: trueatT(i) - .08), and vary close-up, two-shot and wide.
Ink world: FILM = { sky(t, cam), world(t, cam), screen(t, cam) }
- One world. Lay the places out on one ground line (y = 0, up is negative) and move the camera between them.
- Draw things in. Lines arrive with
drawOn. Planned or imaginary things aredashed. Solid things aremass: a flat colour, a shadow side, one ink line. No gradients on objects. - Text. World text goes through
wtxt(id, ...). Keep it out of the headline band (screen y 250 to 450) and off the source line (y 1540 to 1590). The top shade covers y 0 to 640 and the bottom shade starts at y 1400, so keep the ground line above about y 1400 on screen. Text under 21 screen pixels is skipped on purpose. - Shot sizes vary: wide to place, medium to act, close for the moment that matters.
Rules for both
- Timing on the word. Tie each event to the word that names it, not to the end of the sentence.
- Weight. A thing that lands gets a
hitand a squash (spring)."large"hits two or three times a film at most. - Never an empty frame. Something is happening at t = 0, and something new happens at least every second and a half.
- Deterministic. Never
Math.random()orDate. Usehash2(a, b)orrng(seed). - Props keep their facts. A count, a code, a size, a hat: the same in every shot.
5. Check, then direct
Run check after every change. Fix in this order:
PAGE:lines (script errors, "no cue named", "no word") andbuild's "cue wants the word" warnings: the film is broken until these are gone.layout:must say clean. It lists screen text that overlaps or is cut by the frame edge, and world text that overlaps. Anote:line about world text sitting half out of frame is a reframing job. Move the thing; don't shrink text to hide it.motion:quiet stretches. For each, find a word spoken in that window and give it something to do. A deliberate hold on a finished picture can stay; say why.- Look at the pictures. The report cannot see composition. Open every
check/NN-shot.pngnext to itsnoteincheck/storyboard.md: does the frame show what the note says, is the subject large enough, is anyone cut by the edge? Then readcheck/sheet-N.pngleft to right as a viewer with the sound off: can you follow the story sentence? Look for neighbouring frames that are the same, cuts that repeat a framing, an actor that jumps position, a sliver of something at the edge, big empty areas, and anything touching the captions or the headline. - Use
--at 12.4,30for extra stills at exact times while fixing.
Write down what you changed and check again. Stop when a pass finds nothing, not when you are tired of it.
6. Render and hand over
render writes final.mp4. Give the user the file, the captions.srt, and one honest paragraph: what the film shows,
which voices it uses, and what you did not verify (you cannot hear the audio; say so and ask them to listen once before
they post).
Do not publish anything yourself unless the user asks you to in plain words.