Shortform Editing — Cut the Footage Into a Posted-Ready Short
You are the edit-bay technician. Someone hands you real footage — a talking-head take, a screen recording, a voiceover, some B-roll — and you turn it into a vertical short that is ready to post: silences gone, cuts fast, captions burned in word-by-word, motion snapped to the beat, exported to spec. You encode pixels and timing. You do not write the words or pick the hook.
The rigor is a checkable file: a 1080x1920 H.264 MP4 with burned captions that ffprobe confirms, plus the intermediate artifacts (a cut list, an .ass/.srt, a beat grid). If you cannot ffprobe the result and see the specs, you are not done.
Scope: three skills touch "a vertical video" — know your lane
| The ask | Goes to | Artifact |
|---|---|---|
| Write the script, hook, beats, on-screen caption text, edit decision sheet | ../video-shorts/SKILL.md | a .md script |
| Cut/caption/export real footage, tool-agnostic | here | edited .mp4 + caption/cut files |
Build a Remotion React composition project (<TransitionSeries>, headless render) | ../remotion-video/SKILL.md | a .tsx project |
One-line router: footage in → cut, captioned, exported → this skill. Words or a plan → video-shorts. A React codebase → remotion-video. You are the only one of the three that is tool-portable.
Pick your route first
Do not start cutting before you have chosen the tool. The choice is about who runs it and whether it must reproduce.
| Situation | Route | Why |
|---|---|---|
| Reproducible / batch / CI / "do this to 40 clips" | ffmpeg + WhisperX | scriptable, deterministic, no GUI — same input, same output |
| The repo is already a Remotion project | @remotion/captions | reuse the existing composition graph; don't shell out |
| A human is polishing one video by hand | CapCut click-path | fastest for a single hands-on pass with auto-captions + auto beat markers |
The rest of this skill is the ffmpeg + WhisperX pipeline (the reproducible path), with the Remotion and CapCut routes called out where they diverge. Long command blocks live in references/ffmpeg-pipeline.md; the caption detail lives in references/captions.md.
The reproducible pipeline, in order
This is the spine. Run it top to bottom; each step feeds the next.
1. Extract a clean mono 16 kHz audio track for transcription.
ffmpeg -i in.mp4 -vn -ac 1 -ar 16000 a.wav
Why mono 16 kHz: that is what Whisper models expect; feeding stereo 48 kHz wastes time and changes nothing about accuracy.
2. Transcribe with word-level timestamps. Segment timestamps are not enough — word-by-word captions need per-word timing.
whisperx a.wav --model large-v3 --output_format json --highlight_words True
WhisperX (v3.8.5) runs faster-whisper plus wav2vec2 forced alignment, giving word timing accurate to under 100 ms — the floor for karaoke highlight. faster-whisper alone with word_timestamps=True works (~10% overhead) but drifts more on word boundaries. Why forced alignment: Whisper's native word stamps are interpolated and jitter; forced alignment pins each word to the audio. See references/captions.md for the install and the JSON shape.
3. Detect silence, then cut video AND audio together. This is the step people get wrong.
# Pass 1: find the silent ranges (speech threshold)
ffmpeg -i in.mp4 -af silencedetect=noise=-25dB:d=0.3 -f null - 2> silence.log
silenceremove alone is an audio-only filter and desyncs the video — never use it as a one-shot. Parse the silence_start / silence_end lines from silence.log, invert them into keep-ranges, and build a select + aselect filter (or a trim+concat graph) that drops the same ranges from both streams. Thresholds: speech noise=-25dB:d=0.3; music beds noise=-40dB:d=1. The filter-generation snippet is in references/ffmpeg-pipeline.md.
4. Build the beat grid (only if you are cutting to music).
python -c "import librosa; y,sr=librosa.load('music.wav'); \
_,b=librosa.beat.beat_track(y=y,sr=sr,units='time'); print(*b,sep='\n')" > beats.txt
beats.txt is a list of beat times in seconds. CapCut drops these markers on the waveform automatically; librosa (or aubio) gives you the same grid in a script.
5. Snap cut points and zoom keyframes to the nearest beat. Round each cut time to its closest value in beats.txt. A zoom-punch on the beat reads as energy; see Beat sync below for when NOT to.
6. Burn the karaoke .ass captions.
ffmpeg -i tightened.mp4 -vf "ass=captions.ass" -c:a copy captioned.mp4
The .ass carries the per-word \k timing built from step 2's word timestamps and the safe-zone margins. Generation detail and the template are in Karaoke captions below.
7. Export the platform master (see Export specs).
Karaoke captions
Word-by-word captions are the signature look. Build them from the word timestamps, not from segment timestamps.
- Timing. Each word's on-screen highlight comes from its forced-alignment start/end. In
.ass, the\k<centiseconds>tag advances the karaoke fill per word. Word-by-word from segment timing jitters and lands the highlight on the wrong word — always use the per-word stamps from step 2. - Legibility for sound-off. Big bold sans, heavy outline/stroke, optional drop shadow. People watch muted: if it is not readable at a glance on a phone, it failed. Keep it to ≤ 2 lines / ~6 words on screen at once.
- Safe zone — the non-obvious one. Place captions in the vertical middle band, not the bottom. Platform UI (right-rail icons, bottom caption/CTA/profile) covers the bottom ~15–20% and the right edge. Keep captions and key text inside the central ~80% of the
1080x1920frame. Captions burned at the bottom get hidden behind the TikTok buttons — this is the #1 "my captions are broken" cause.
Minimal .ass style block (full template in references/captions.md):
[V4+ Styles]
Style: Karaoke,Montserrat,64,&H00FFFFFF,&H0000FFFF,&H00000000,&H64000000,-1,0,0,0,100,100,0,0,1,5,2,5,80,80,640,1
MarginV (the trailing 640) lifts the line into the middle band; the heavy Outline (5) keeps it readable on any background.
Remotion route: if you are inside a Remotion project, use @remotion/captions — createTikTokStyleCaptions({ captions, combineTokensWithinMilliseconds }) pages the tokens. Low (200–500 ms) = word-by-word pop; high (1200–2000 ms) = phrase pages. Page durationMs was added in v4.0.261. CapCut route: Text → Auto captions → Generate, then restyle.
Beat sync
A beat grid (step 4) is a list of seconds. Snap your cut points and zoom keyframes to the nearest beat so motion lands on the music.
- Bad → Good. Cutting mid-word to land a cut exactly on a beat reads as a mistake → nudge the cut to the nearest beat that also respects the speech boundary. Honor the words first, the beat second.
- When NOT to beat-cut. A talking-head explainer where the words carry the value: clarity beats rhythm. Beat-cutting a person mid-sentence chops their delivery. Reserve beat sync for B-roll montages, transitions, and music-led segments.
- Transitions stay cheap and fast. Hard cut is the default. Whip-pan or zoom-punch only on a beat. Avoid 1-second crossfades in a 30-second short — they kill the pace; a short lives on fast hard cuts.
Export specs
The universal vertical master — one file that plays cleanly across all three platforms:
ffmpeg -i captioned.mp4 \
-vf "scale=1080:1920:force_original_aspect_ratio=decrease,pad=1080:1920:(ow-iw)/2:(oh-ih)/2,setsar=1" \
-c:v libx264 -profile:v high -pix_fmt yuv420p -r 30 \
-c:a aac -b:a 192k -ar 48000 -ac 2 \
-movflags +faststart out.mp4
yuv420p (broad playback), +faststart (web streaming starts before full download), AAC-LC 48 kHz. Use 30 fps; bump to 60 fps for high-motion Shorts.
| Platform | Video bitrate | Notes |
|---|---|---|
| TikTok | ~2,000–4,000 kbps | lightest of the three |
| Instagram Reels | ~3,500–4,500 kbps | |
| YouTube Shorts (1080p) | ~8,000–15,000 kbps | up to ~53,000 for 4K/60 |
A single 1080x1920 H.264 MP4 @30 fps under ~250 MB is safe everywhere. Verify before you ship:
ffprobe -v error -select_streams v:0 \
-show_entries stream=width,height,codec_name,avg_frame_rate out.mp4
Expect width=1080 height=1920 codec_name=h264. The full per-platform encode flags and ffprobe assertions are in references/ffmpeg-pipeline.md.
Anti-patterns
| Bad | Why it breaks | Good |
|---|---|---|
silenceremove as a one-shot filter | It is audio-only; the video keeps playing → audio/video desync | silencedetect → invert ranges → select+aselect cut both streams |
| Word captions from segment timestamps | Highlight jitters and lands on the wrong word | Forced-alignment per-word stamps (WhisperX / word_timestamps) |
| Captions burned in the bottom 15–20% | Platform UI (buttons, CTA, profile) covers them | Place in the vertical middle band, central ~80% |
| One export reused for every platform | Over/under the bitrate target somewhere | Per-platform bitrate, or the safe universal master |
| 1-second crossfades in a 30s short | Kills the pace; reads slow | Hard cuts; whip/zoom only on a beat |
| Beat-cutting a talking-head mid-sentence | Chops the delivery; clarity lost | Beat-sync B-roll/montage only; honor speech boundaries |
| Re-running the encode to "check the specs" | Wastes minutes; the file already exists | ffprobe the output; verify, don't re-render |
| Writing the caption copy or the hook here | That is the script, not the cut | Route to ../video-shorts/SKILL.md; you only burn it in |
ffmpeg overwriting the source in place | One bad flag and the original is gone | Always write to a new file; keep in.mp4 untouched |
Verify & references
bash scripts/verify.sh <output.mp4>— runsffprobeand assertswidth=1080,height=1920(9:16), video codech264, audio codecaac, fps in{30, 60}, and that a sibling.srt/.asscaption file exists and is non-empty. With noffprobeit falls back to a path/extension check. Read-only; exits 0 on an empty/clean target so it never false-fails. The "does it feel on-beat" judgment is not mechanical — that lives in the capability eval.references/ffmpeg-pipeline.md— the full silence-detect → keep-range inversion →select/aselectfilter generation that cuts video+audio together, the concat/trim approach, per-platform encode flags,ffprobeassertions, and desync/audio-drift/faststart troubleshooting.references/captions.md— WhisperX install + run, the word-timestamp JSON shape, the full.asskaraoke template with safe-zone margins, the RemotioncreateTikTokStyleCaptions()snippet withcombineTokensWithinMillisecondsguidance, and the faster-whisper vs WhisperX tradeoff.