Auto-Loom
Turn any web app into a narrated, self-driving MP4. The page scrolls, highlights elements, and navigates between routes while an AI voice narrates — timed to the millisecond. No manual screen recording, no fumbled Loom takes. Edit one line of narration and re-render in minutes.
How it works: ElevenLabs generates one audio clip per storyboard beat.
ffprobe measures each clip's exact duration. Playwright drives the browser
for exactly that long per beat while recording the window. ffmpeg muxes the
final MP4 with a caption track. Audio duration is ground truth — visuals are
slaved to it, so sync is exact.
This folder is the whole engine. Sibling skills are thin entry points over these same scripts:
- auto-loom-proof — Software Factory proof video: performs a build's acceptance criteria on camera, one criterion per beat
This SKILL.md covers the engine plus developer mode: you have the codebase, the app runs on localhost, and you can add stable selectors to the JSX.
Setup (one-time, ~3 minutes)
cd <this-skill-folder>
bash scripts/setup.sh # installs Playwright + Chromium, creates .env.local
# edit .env.local → add your ElevenLabs API key
node scripts/doctor.mjs # verifies node, ffmpeg, playwright, key, voice
No ElevenLabs account yet? Sign up with the maintainer's referral link: https://try.elevenlabs.io/atherial.
doctor.mjs prints the exact fix for anything missing. Requirements: Node 18+,
ffmpeg (brew install ffmpeg), an ElevenLabs API key. Voice options:
references/voices.md.
Smoke test (30 seconds, records example.com):
node scripts/run.mjs --beats examples/smoke-beats.json
Step 0 — Interview the user first
Do not start building. Ask:
1. What URL or app are we recording? Is it running locally?
2. Who is the recipient — name and context?
3. What's the story? What changed, or what are we showing for the first time?
4. Which specific features, metrics, or sections should we highlight?
5. What's the tone? (client update, internal demo, prospect showcase)
6. Does the app require login? (see references/auth.md — never ask for passwords in chat)
7. Target length — 30s, 60s, or 75s?
In a hurry: ask 1–3 and infer the rest from the app.
Step 1 — Write the storyboard (beats.json)
Write the narration before touching any code and get it approved verbatim.
Script rules and the beat framing that works: references/tts-rules.md.
{
"meta": {
"title": "Acme dashboard walkthrough",
"slug": "acme-walkthrough",
"cta": "Send me your feedback and we move to the next phase.",
"targetLengthSec": 70
},
"baseUrl": "http://localhost:3000",
"viewport": { "width": 1920, "height": 1080 },
"beats": [
{
"id": 1,
"page": "/dashboard/insights",
"text": "Hey Jim. We shipped some updates to your A-I visibility tracker.",
"actions": [
{ "fn": "waitFor", "sel": "[data-vn=kpi-row]" },
{ "fn": "scrollToEl", "sel": "[data-vn=kpi-row]", "opts": { "instant": true } },
{ "fn": "hl", "sel": "[data-vn=kpi-row]" }
]
}
]
}
Full schema: schema/beats.schema.json. Action table and selector guidance:
references/actions.md. Login options (scripted steps or saved session):
references/auth.md.
Check it before spending anything:
node scripts/validate.mjs --beats <file> # structure + TTS + pacing lint
node scripts/run.mjs --beats <file> --script-only # print the narration and stop
The validator catches unknown actions, missing selectors, TTS problems (em dashes, bare initialisms, run-on sentences), and pacing that won't fit the target length.
--script-only is the money-saving review gate: it prints every beat's
narration with per-beat timing estimates and exits before any ElevenLabs
call or recording. Get the script approved here, then re-run without the
flag to spend credits.
Step 2 — Instrument the page (developer mode)
Playwright needs stable selectors. With codebase access, add data-vn
attributes to the component JSX:
<div data-vn="kpi-row" className="grid grid-cols-4 gap-4">
<Card data-vn="competitor-chart" ...>
Then reference them: { "fn": "hl", "sel": "[data-vn=kpi-row]" }.
Suppress interactive drilldowns while recording. Clickable cards with
role="button" / tabIndex={0} can get auto-focused during highlights and
open a drawer mid-video. Gate them on your demo-mode flag:
{!isDemoMode() && <MetricBreakdownSheet ... />}.
If the app requires auth locally, either wire a demo mode (skip auth, fixture
data that matches the narration numbers exactly) or use an auth path from
references/auth.md.
Note on CSP: the recorder launches Chromium with bypassCSP: true so it
can inject presenter.js into strict-CSP pages. If you point it at a
production app with a live session, know that Content-Security-Policy is
disabled inside that automation browser context.
Step 3 — One command
node scripts/run.mjs --beats <file>
That runs: validate → audio (cached by text hash) → preflight (headless
pass resolving every selector before anything is recorded) → record → captions
→ mux → verify. Output lands in demo-video-out/<slug>/:
<slug>-demo.mp4— final video (H.264/AAC, soft caption track)<slug>-demo-contact-sheet.png— sampled frames for eyeballingqa/beat-NN.png— exact screenshot at each beat's hold pointcaptions.srt,audio/,video/recording.webm
All flags: --script-only (print the narration script and exit before any
ElevenLabs call or recording — the review gate from Step 1),
--preflight-only (check selectors, spend nothing), --skip-audio
(visual-only changes — audio is reused), --skip-preflight (skip the headless
selector check — only when preflight just passed and nothing changed),
--burn-captions (hardcode subtitles into pixels for platforms that strip
tracks), --no-captions, --out <dir>, --env <envfile>.
Step 4 — Verify before sending
run.mjs checks duration (MP4 = sum of audio within 0.5s), audio level (silent-track
detection), and drops QA screenshots. You still eyeball:
- Contact sheet +
qa/beat-NN.png: each frame shows what its beat narrates — no login pages, error states, or surprise modals. - First beat names the recipient; last beat delivers the CTA verbatim.
Iterating
- Narration edit → re-run; only the changed beat's audio regenerates.
- Visual/pacing edit →
--skip-audio. - Selector broke →
--preflight-onlyshows exactly which beat and selector.
Known gotchas
- Hot reload wipes the injected presenter. Already handled — presenter re-injects before every beat. But avoid editing the app while recording.
- Port drift.
yarn devtakes 3001 if 3000 is busy — updatebaseUrl. - Free vs paid voices. Cloned/library voices throw
payment_requiredon free tier.doctor.mjscatches this before render time. - i18n redirects.
/en/dashboard→/dashboardredirects are followed correctly; same-page beats don't re-navigate. - Delivery. The MP4 is self-contained — email, Slack, LinkedIn DM, Notion embed, or import into Descript for a personal intro on top.