shaharsha/deck-to-video

---

name: deck-to-video description: Use when turning a slide deck plus per-slide narration audio into an mp4 video — a self-playing/self-presenting deck video, "slides to video", "presentation to mp4", a shareable WhatsApp/Slack/Drive video of a narrated deck, or when a slide video needs a per-slide progress bar / countdown / slide counter. Also use when a slide video should highlight, glow or spotlight the element being talked about in time with the voice, or when narration and on-screen animation need syncing. Also use when an ffmpeg-drawn progress bar renders full/static from the first frame, or ffmpeg errors "No such filter: 'drawtext'". Also use when burning or hardcoding subtitles into a deck video, when subtitles do not show in a Slack or Teams inline player, when VLC shows two sets of subtitles at once, or when ffmpeg reports "No option name near" on a subtitles filter. Also use when a final ffmpeg mux cuts off the end of a video, or ffmpeg 9 rejects -filter_complex_script.

Deck to Video (slides + narration → self-playing mp4)

Overview

Turn N slide images (or a PDF) + N per-slide audio clips into one mp4: each slide holds for exactly its narration's length plus a short pause, then hard-cuts to the next. Overlays: a filling progress bar + seconds-remaining countdown (bottom-right) and an "N / M" slide counter (top-right).

The one command (the script ships in this skill's own scripts/ directory — call it by its full path there):

python3 <this-skill-dir>/scripts/build_video.py SLIDES AUDIO_DIR out.mp4
# SLIDES = slides.pdf (rasterized at 200dpi) OR a directory of per-slide images
#          (numeric-aware ordering: s2 before s10; zero-padded names still recommended)
# AUDIO_DIR = slide01.mp3, slide02.mp3, … — count MUST equal slide count
# options: --size 1920x1080 · --pad 0.7 · --no-bar · --no-counter · --bar-color HEX · --crf 20
#          --highlights hl.json · --anim-fps 30   (see "Highlighting elements" below;
#          scripts/make_highlights.py builds hl.json)

On success it prints one line per slide — that per-slide length already includes the pad — and ends with duration matches segment sum (it exits non-zero otherwise).

Getting the inputs right

1. Slides: if starting from a pptx, rasterize from real PowerPoint, not LibreOffice. Export the pptx to PDF via actual PowerPoint (the office-render skill does this on macOS), then hand the PDF to the script — LibreOffice substitutes fonts and re-flows layouts, so the video would not match what the deck's author sees. 200 dpi keeps dense slide text crisp at 1080p. (Already have per-slide images? Use them directly — skip this step.)
2. Audio: one clip per slide, named slideNN.mp3. If the clips came from elevenlabs-tts' generate_tts.py (which narrating-pptx uses), they are already in this exact shape — reuse it so the pptx and the video sound identical.
3. If narration pace was adjusted (e.g. ffmpeg -filter:a atempo=1.1), build the video from the adjusted files.

Why the overlays are baked with PIL, not drawn by ffmpeg (paid for in blood)

• drawbox with a t-based width expression renders a FULL bar from frame 0 in several ffmpeg builds — the expression is evaluated once at init, not per frame. The failure is silent: the video encodes fine and the bar simply never moves. drawtext expressions do update per frame in the same builds, which makes the broken bar easy to miss next to a working countdown.
• Homebrew's ffmpeg bottles (2026+) are built without libfreetype — No such filter: 'drawtext' — and the keg versions (ffmpeg@7, ffmpeg@8) are the same. Don't spend an install cycle discovering this; if a full build is needed for anything else, uv tool install static-ffmpeg ships one (the script auto-falls-back to it).

So the script bakes frames with PIL (bar fill, countdown, counter) and feeds ffmpeg an image sequence encoded at -r 30. Deterministic, font-controlled, and cheaper to encode than filtering a 30 fps stream. One frame per second is the default and is right while nothing on the slide moves — second-granularity updates read naturally for a countdown. Slides carrying --highlights bake at 30 fps instead, for the reason in the next section.

Encoding choices the script already makes (don't undo them)

Choice	Why
apad=pad_dur=PAD + -t seg_dur	exact tail padding; naive -shortest cuts the pause after speech
yuv420p + even dimensions	anything else fails to play in QuickTime/WhatsApp/browsers
-tune stillimage, CRF 20	static slides compress well without text shimmer
concat demuxer + -c copy	lossless join of identically-encoded segments
-movflags +faststart	video streams before it finishes downloading

Validate — assume the overlay is broken until frames prove otherwise

The script asserts final duration ≈ sum of segments. The overlay needs eyes:

ffmpeg -y -ss T1 -i out.mp4 -frames:v 1 a.jpg   # early in some slide
ffmpeg -y -ss T2 -i out.mp4 -frames:v 1 b.jpg   # late in the same slide

Pick T1/T2 from the script's printed per-slide lengths (cumulative-sum them for slide boundaries; remember they include the pad). Read both images. The bar must be visibly fuller in b.jpg and the countdown lower — and it should reset at each new slide. A full-looking bar at an early timestamp = the static-bar bug — this exact check is what caught it originally. You cannot verify audio headlessly — say so; never claim you heard it.

Highlighting elements in time with the narration

Narration that says "on the left… in the middle… on the right" reads like an audio-description track (see self-presenting-decks). The fix is to strip the verbal pointer and glow the element instead — but only if the timing comes from the audio.

The audio is the master; the animation follows it. Never write narration to hit a mark, and never estimate when a phrase lands. Generate the clip normally, then use ElevenLabs forced alignment (POST /v1/forced-alignment, audio + transcript → per-character start/end; run it with elevenlabs-tts' scripts/align_narration.py) to read back when each thing was actually said, and derive the windows from that. Re-record a clip later and re-aligning self-corrects.

• Anchor on positions in the script, not on keywords. A lone keyword breaks the moment it appears twice ("tags" appears four times on one slide). Mark where the narration turns to each element with a short phrase copied from the script, in spoken order, and search each one only after the previous match, which is what make_highlights.py does; a repeated word earlier in the script can't capture it. (Keeping per-element segment offsets into the joined script works too.)
• Strip audio tags before aligning — all of them. [confident] is performed, not spoken; send it to the aligner and it hunts for the word in the audio and drags every later offset with it. Eleven v4 tags are free-form ([Pause, dry amusement]), so strip anything in brackets, not just letters-and-spaces (elevenlabs-tts' aligner does). Compute offsets against the same stripped string.
• Check the returned character count equals the text length, and watch loss (0.04-0.10 across a healthy 17-clip English deck, 0.59-0.77 across a healthy 26-clip Hebrew one; a clear outlier within the deck is the signal, not an absolute threshold). A mismatch means every window on that slide is wrong.

Build the --highlights JSON with scripts/make_highlights.py from three inputs: markers (per slide, each region with the phrase where the narration turns to it), boxes (per slide, region → [x, y, w, h]) and the alignment/ files:

python3 <this-skill-dir>/scripts/make_highlights.py markers.json boxes.json narration/alignment/ hl.json \
  --size 1920x1080 [--inches 13.333]      # --inches: boxes are in slide inches, page this wide
# markers.json: {"09": [["chart", "Last quarter"], ["callout", "The real problem"], ["_end", "Now, to"]],
#                "12": {"marks": [["a", "First"], ["b", "Second"]], "dim": "black", "colour": "#E07A5F"}}

A region's window runs from its phrase to the next marker's (_end closes one without opening another; otherwise the last runs to the end of the speech), starting 0.12 s early. Windows under 1.4 s are skipped with a warning, since a 0.5 s fade in and out leaves nothing to see. It stops, writing nothing, when a phrase isn't in the script, a region has no box, a box leaves the frame or an alignment file doesn't match its script. It warns when a box sits within 19 px of the edge, where the glow is clipped.

The resulting JSON, keyed by zero-padded slide number, boxes in output pixels:

{"09": [{"card": "chart", "box": [87, 468, 549, 374], "colour": "#B45309",
         "dim": "white", "start": 13.6, "end": 22.9}]}

Take box from the deck's own layout code (inches → pixels), never by eyeballing the render — a card's geometry is already exact in the builder that drew it. dim is white on light slides, black on dark ones; check the glow colour against a dark slide too (a dark-plum glow vanished on a near-black closing slide). A region may be highlighted more than once on a slide. Slides absent from the file bake at 1 fps as usual.

Subtitles come free once aligned. The same word timings make an .srt with no second transcription pass:

python3 <this-skill-dir>/scripts/make_srt.py narration/alignment/ narration/audio/ out.srt --pad 0.7

--pad must match the pad the video was built with, or cues drift further out with every slide.

Three ways to ship them, and the destination decides which.

Form	How	What the viewer gets
Sidecar .srt	ship the file beside the mp4	subtitles only if they open both in a real player
Soft track	-c copy -c:s mov_text -disposition:s:0 default — lossless mux, ~12 KB, seconds	player-dependent; needs a CC control to exist
Burned in	re-encode through libass	always visible, everywhere, cannot be turned off

Default to the sidecar or the soft track — both are reversible and neither touches the picture. Burn only when the video will be played somewhere with no subtitle control. A Slack or Teams thread's inline player is the case that forces it: a viewer who just hits play there gets nothing from the other two forms.

Burning them in

ffmpeg -i deck.srt /tmp/subs.ass          # convert FIRST; do not pass force_style
# …edit the Style: line in /tmp/subs.ass (see below), then:
<static-ffmpeg> -i master.mp4 -vf "ass=/tmp/subs.ass" \
  -c:v libx264 -crf 20 -preset medium -tune stillimage -pix_fmt yuv420p \
  -c:a copy -sn -movflags +faststart out.mp4

Four things that each cost a cycle:

• Homebrew's ffmpeg has no libass — the same gap as drawtext/libfreetype, and the tell is misleading. You get No option name near '<filename>', a filtergraph parse error rather than No such filter, so it reads as a quoting bug and sends you rewriting escapes that were already correct. Check with ffmpeg -filters | grep -E ' (ass|subtitles) ' and use the static-ffmpeg build.
• Style the .ass file; never cram force_style into the filter string. Its commas are read as filter separators, and backslash-escaping them is fragile enough that it fails twice before it works.
• Map ASS style fields by the Format: header, never by counting positions. An off-by-one writes Alignment=0 (invalid) and leaves MarginV unset, and libass renders it without complaining.
• Burn from the pre-burn master and keep that master. Restyling from an already-burned file stacks a second generation and bakes the old text in permanently.

Placement — band or overlay. The deck decides, not taste.

	How	Cost
Band below the slide	pad=1920:1200:0:0:black then ass=, MarginV inside the band	nothing on the slide is ever covered; output stops being 16:9
Overlay (the familiar VLC look)	ass= alone; bottom-centred, bold ~48 px at 1080p, heavy black outline, MarginV ~40	classic and self-contained, but it will cover whatever the deck puts in its bottom strip

Overlay is what most people mean by "subtitles". On a text-dense deck, extract a frame and look before committing: it lands on footer callouts, legends and the progress bar.

1 fps is a sync limit, not just a smoothness one. The default bake is one frame per second, so a highlight can only change on an integer second — up to a full second off the word it points at, which reads as broken. Animated slides need ~30 fps.

30 fps is cheap if you cache frames. A 60 s slide is 1,800 frames, but nearly all are pixel-identical — the glow only moves during the fades. Key each frame on its visual state and hardlink the duplicates:

key = (active_element, round(fade_alpha, 2), int(t))   # int(t) = the bar's step
if key in cache: os.link(cache[key], path); continue

120-200 real composites per slide instead of ~1,800, and no extra disk. But quantise the progress bar to whole seconds first — a continuously sliding bar makes every frame unique and defeats the cache entirely. (At 1 fps it already behaved that way, so nothing is lost.)

Invert the dim on dark slides. Washing inactive elements with translucent white pushes them back on a light slide and brightens them on a dark one. Carry the dim colour per slide, not per deck.

Don't animate what is said too fast. Measure the spoken span before committing: four list items rattled off in ~1.2 s each leave ~0.2 s of glow after a 0.5 s fade in and out, and strobe. Either give each item its own beat in the narration — usually an improvement, since one-breath lists are the densest thing in any deck — or leave the slide static.

Common mistakes

Mistake	Consequence	Fix
ffmpeg drawbox animated by t	bar full from frame 0, silently	PIL-baked frames (the script)
Assuming drawtext exists	No such filter on Homebrew builds	script falls back to static-ffmpeg / PIL text
LibreOffice rasterization	fonts/layout differ from the real deck	real PowerPoint PDF via office-render
-shortest for segment length	eats the pause after narration	apad + explicit -t
Audio count ≠ slide count	desynced or missing slides	script hard-fails; fix inputs, don't skip slides
Judging output by "it encoded fine"	static bar / wrong slide slips through	extract early/late frames and look
Deck prints its own slide numbers	number appears twice (deck footer + counter)	render the video from a numberless deck variant, or pass --no-counter
ffmpeg hangs at 0% CPU after writing a segment	build stalls forever; a killed attempt leaves an INVALID partial file	inside an agent's shell ffmpeg can wait on stdin: the script passes -nostdin with stdin closed, and still times out each segment (300s) and retries once, deleting the partial first. Do the same in any ffmpeg you run by hand (-nostdin or < /dev/null)
Estimating when a phrase lands instead of aligning	highlights drift off the words, worst on the longest slides	forced alignment on the finished clip; make_highlights.py
A box at the edge of the slide	the glow, drawn 19 px outside the box, is cut off by the frame	inset the region, or accept it; make_highlights.py warns
One glow colour for every slide	it disappears on a dark slide	set colour (and dim: black) per dark slide in the markers
Reading card geometry off a rendered image	boxes are a few px out and the glow sits crooked	compute from the deck's own layout constants
Piping the build through | tail inside an && chain	pipeline exit = tail's 0 — a failed build "succeeds" and stale artifacts get delivered	run the script unpiped (or set -o pipefail); keep delivery/sync in a separate step gated on verified output
Leaving the sidecar .srt beside a burned-in mp4	VLC auto-loads any .srt matching the video's basename — the viewer sees two sets of subtitles stacked	move the sidecar out of that folder; a burned video is self-contained
force_style='…' with commas, in the filter string	No option name near … — looks like a shell quoting bug, is actually the filtergraph splitting on those commas	convert to .ass and set the style in the file
Counting ASS Style: field positions	Alignment/MarginV land on the wrong fields and libass renders it silently	map fields by the Format: header line
-filter_complex_script file on ffmpeg 9	Unrecognized option - the option was removed	-/filter_complex file (the -/ prefix reads any option's value from a file; ffmpeg 7+)
Final mux with -shortest when the mixed audio ends early	the video's last seconds (an end caption, a fade-out) are cut	pad the audio: apad in the graph plus -t <video length> on the mix, then check both stream durations with ffprobe -show_entries stream=codec_type,duration
A music bed that stops before the video ends	the music cue is shorter than the stretch it scores	generate a longer cue (or add a second cue for the last section), and check every cue's real length against its window before mixing - see elevenlabs-tts, Music and sound effects

Caveats

• Hard cuts between slides by design — crossfades break the lossless -c copy concat and add little.
• ~13–17 min of 1080p mostly-static slides ≈ 40–50 MB.
• The video is a third artifact beside the clean pptx and the narrated pptx — it forces the narration's pace on the viewer, so keep the pptx variants for people who prefer reading.
• Burning re-encodes the whole video (~60 s for 10 min at CRF 20 on an M-series); the sidecar and soft-track forms are seconds and lossless. Keep the pre-burn master so a restyle is one re-encode from source, not two stacked.

Related skills

• producing-animated-episodes: voiced, animated episodes with characters, built on the same ffmpeg lessons.