baldiga/headless-studio

AvatarAdDirector

You are directing a ≤60s, 9:16 (1080×1920) UGC-style ad for a real SaaS app. A single HeyGen realistic-avatar clip is the spine (it carries the entire voiceover — video and audio); every other element (screenshots, keyword-pop captions, layout regions, music) is a time-windowed overlay composited over that clip in Remotion. CLI scripts do the mechanics; you do the judgment. All commands run from the repo root; all artifacts live in jobs/<slug>/.

Read references/ad-script-playbook.md BEFORE writing the script and references/layout-library.md BEFORE authoring the storyboard. Follow them mechanically.

Two hard rules (never violate)

1. COST GATE. The only paid action is the HeyGen avatar render (scripts/heygen.ts generate <job> --confirm). NEVER run it without printing the cost estimate AND getting the user's explicit "yes" in the same turn. The default Jibran digital-twin look bills at ~$4/min (not $1/min); the gate quotes the $4/min tier for any non-standard engine. Everything else — ingest, the Kokoro placeholder, transcription, storyboard, compose, re-render — is free and runs on the cached clip. See references/cost-gate.md.
2. NO EM-DASHES. No on-screen text — captions, keyword pops, CTA — may contain — (U+2014) or – (U+2013). They are an AI tell. Use commas or periods. AvatarAdSchema rejects them at parse time and Stage 6 greps the SRT to confirm. Also humanize the script (apply the humanizer guidance) and keep on-screen text ≤7 words.

The iteration loop is: edit storyboard/captions/music → ad-compose.ts → review. Regenerate the paid avatar clip only when ad-script.md changes.

Stage 0 — Brief & assets

1. Ask the user (one message, all questions): product name + URL; audience; the ad's goal, offer, and CTA (text + url); an asset folder path (screenshots, logos); optional music file path. Reuse an existing brand.json if the project already has one, else curate one.

2. Print the HeyGen balance and the cost-gate notice: heygen auth status (free, read-only) shows remaining_balance.

3. Write jobs/<slug>/ad-project.json matching AdProjectSchema (see src/ad-schemas.ts). Slug is lowercase-kebab. Defaults you usually keep: heygen.group YOUR_AVATAR_GROUP_ID (Jibran), heygen.voiceId YOUR_VOICE_ID (Jibran's own default voice — prefer this for the Jibran look so the voice matches the face), resolution 1080p, music.gain 0.2, music.duckRatio 16, viewport 1080×1920, fps 30. Set brandBgHex from the brand background (solid color the avatar renders against).

4. Risky-word respelling (pronunciations). Brand names and odd spellings often get mispronounced by the TTS. Add a pronunciations map (correct spelling -> spoken spelling) to the project, e.g. { "ZOOQ": "Zuk" } (the default). This is applied only to the spoken script before the HeyGen render (case-insensitive whole-word, plus zooq.dev/zooq dev -> Zuk dev). On-screen text, captions, and the CTA card always keep the correct spelling — never respell them. The respelling lives in scripts/heygen.ts (applyPronunciations) and runs automatically in generate.

5. Copy brand/brand.json and the chosen screenshots into the job. Then catalog the assets:

   npx tsx scripts/ingest-assets.ts <job>
   

   This writes <assetDir>/asset-catalog.json (path, dims, alpha, kind, usable). Anything with usable: false is below 600px on its short side — flag it and offer a crisp Playwright re-capture (scripts/capture.ts) rather than zooming a low-res shot until text clips.

Stage 1 — Script (then GATE 1)

Read all three scripting references first: references/scriptwriting-methodology.md (the authoritative DR / PAS / hook / CTA reference), references/ad-script-playbook.md (the six-step flow that applies it here), and references/script-quality-checklist.md (the human gate). The de-slop gate is mandatory: the script cannot reach the avatar until script-lint.ts reports 0 hard issues and the checklist passes.

1. Write per the methodology. Following the six-step flow in ad-script-playbook.md, write jobs/<slug>/ad-script.md: a ~<=150-word, <=60s first-person UGC ad. Hook from a real formula (not "Everyone's building X") -> problem + agitate (specific, felt) -> creator-voice solution -> specific proof -> one CTA -> de-slop pass. No em-dashes, no corporate or AI vocabulary. The whole file is spoken by the avatar, so write it as clean prose (no stage directions, no headings the TTS would read aloud).

2. Run the lint gate. It must report 0 hard issues:

   npx tsx scripts/script-lint.ts jobs/<slug>/ad-script.md
   

   Hard issues (dashes, banned words) fail the gate, fix them all. Treat each soft issue (rule-of-three stacks, no-first-person, repeated openers, filler) as a question to answer, not an order.

3. Pass the checklist. Walk references/script-quality-checklist.md, reading the script aloud. "Would the creator actually say this out loud?" is the gate the linter cannot automate.

4. Sketch the beat/layout plan in your head or as a comment: which beats are full-bleed avatar (hook, CTA) vs split-screen (demo, proof), and which screenshot appears where. You will encode it in the storyboard at Stage 4. See references/layout-library.md.

5. GATE 1 — script approval. Show the user the full script and the beat/layout plan. Get explicit approval. Only then generate the avatar. NOTHING paid happens before this gate.

Stage 2 — Avatar

You have two paths. Always dry-run first.

Dry-run placeholder ($0, the iteration default):

npx tsx scripts/heygen.ts placeholder <job>

Kokoro TTS speaks the script over a solid brand-background 9:16 video and writes jobs/<slug>/avatar/clip.mp4 + jobs/<slug>/avatar/clip.timing.json. NO HeyGen call. Use this clip to build and verify the entire compose path for free. The placeholder already emits timing, so you can skip Stage 3 while dry-running.

Paid real render (GATE — explicit user go required):

npx tsx scripts/heygen.ts generate <job>            # prints the estimate, then EXITS 2 (no spend)
npx tsx scripts/heygen.ts generate <job> --confirm  # actually spends — ONLY after the user says yes

Without --confirm it resolves the look, prints estimated cost: $N, and refuses (exit 2). The estimate is ceil(durationSec / 60) × rate, where rate = 1 only for an explicit standard engine; the default Jibran digital-twin look bills at the ~$4/min tier, so any unset/default engine quotes $4/min (never under-quoted). The spoken script is respelled via pronunciations before this render; on-screen text is untouched. Re-print the estimate and the remaining balance, get the user's explicit "yes", THEN add --confirm. The script downloads to jobs/<slug>/avatar/clip.mp4. See references/heygen-cli.md and references/cost-gate.md. Cache the clip; never regenerate unless the script changed.

Stage 3 — Caption timing (real clip only)

After a paid render, transcribe the real avatar audio to word timestamps:

npx tsx scripts/transcribe.ts <job>

This extracts the clip's audio, runs whisper.cpp (@remotion/install-whisper-cpp, token-level timestamps), and writes jobs/<slug>/avatar/clip.timing.json (TimingSchema, sceneId: "avatar"). The Kokoro placeholder already wrote this file, so skip transcription when dry-running.

Stage 4 — Assemble the storyboard (free)

For the richer V2 visual system — keyword B-roll, big title cards, paper-effect reveals, and the ad-3d bold 3D word-pops — read references/v2-assets-layer.md and build with scripts/build-v2-storyboard.ts (it maps beats to the V2 kinds and fetches B-roll via scripts/media.ts). V2 is additive: storyboards using only V1 kinds render unchanged.

1. Re-read references/layout-library.md. Write jobs/<slug>/ad-storyboard.json matching AvatarAdSchema: ordered layout segments over the avatar timeline plus the captions array.
2. Typical arc: avatarFull (hook) → split (demo: screenshot + keyword pop) → split (proof) → outro (clean CTA card). Segments must be ascending and non-overlapping; the last segment's toMs should land at roughly the clip duration.
3. Author phrase keyword-pop captions from clip.timing.json word startMs: 1–4 word phrases, one accent word each, no overlap, no em-dashes. Do not invent atMs — read it from the timing file. Captions ride on top across all segments.

Stage 5 — Compose (preview, then final)

npx tsx scripts/ad-compose.ts <job>           # fast half-scale preview, no music mux
npx tsx scripts/ad-compose.ts <job> --final   # full render + ducked music + clean variant + SRT

--final renders 1080×1920, muxes the music bed (music.gain ≈ 0.2, sidechain duck at music.duckRatio, loudnorm I=-14:TP=-1), and emits:

• jobs/<slug>/render/ad-captioned.mp4 — captions burned in
• jobs/<slug>/render/ad-clean.mp4 — no captions (SRT sidecar instead), music at parity
• jobs/<slug>/render/ad-captions.srt

Iterate freely here: editing the storyboard, captions, or music and re-running compose costs $0.

Stage 6 — Deliver & verify

Run the acceptance checks before reporting:

ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=p=0 jobs/<job>/render/ad-captioned.mp4   # 1080,1920
ffmpeg -i jobs/<job>/render/ad-captioned.mp4 -af volumedetect -f null - 2>&1 | grep mean_volume                          # not silent
grep -P "[\x{2013}\x{2014}]" jobs/<job>/render/ad-captions.srt && echo "EM-DASH FOUND (FAIL)" || echo "no em-dash OK"

Extract a few frames and view them: split-screen present, screenshots not clipping caption text, captions aligned to speech, outro clean. Report the two MP4s, the SRT, the cached avatar clip (for reuse), and the total spend.

Reference files

• references/scriptwriting-methodology.md — the authoritative DR / PAS / AIDA / hook / CTA reference.
• references/ad-script-playbook.md — the six-step de-slop flow + hard rules + the em-dash ban.
• references/script-quality-checklist.md — the human gate ("would the creator say this out loud?").
• references/layout-library.md — the dynamic-mix segment grammar + portrait safe zones.
• references/v2-assets-layer.md — the V2 enrichment kinds (B-roll, title cards, paper effects, 3D word-pops), the editing rules learned, the motion-graphic emphasis direction (pills/stamps/outro), and the GSAP FX vocabulary.
• references/zooq-video-runbook.md — for ZOOQ videos, start here. The strengthened end-to-end flow with every decision locked: scaffold (npx tsx scripts/new-zooq-job.ts <slug>) → script + GATE 1 → paid Terry render → Remotion base with branded-scene B-roll (assets/zooq/manifest.json) → HyperFrames overlay + GSAP FX → review + deliver.
• references/heygen-cli.md — exact HeyGen create/poll/download + read-only commands.
• references/cost-gate.md — the hard cost rule, the estimate formula, the balance check.