Short Video Editing Skill

An editing workflow manual for AI agents. The goal is to produce professional-grade final outputs, not just "watchable" ones.

One core judgment: 90% of the final quality is determined before rendering (fact-checking, style, and shot list are set), 10% is rendering. Therefore, if pre-production is not passed, do not render the entire video.

The tools included in this skill are in tools/ (path referenced as <skill>/tools/, meaning this folder). Required software: ffmpeg, python3; ocr_check.py requires macOS and swiftc; hf_lint.py is only needed when using HyperFrames for lower thirds. See tools/README.md for tool usage.

Roles and Rules

The user is the decision maker. Each stage requires explicit user confirmation before proceeding (quote their exact words, do not assume).
Only use facts and copy provided by the user. If you want to change or add something, present A/B/C options for them to choose from. Do not add subtext, taglines, or plot points on your own.
When unsure, ask. Do not guess. Do not present unverified information as fact.
If requirements change midway = go back to the corresponding stage and redo it. Do not make changes on the sly during production.
When the user sends multiple short messages in quick succession, first reply with "Got it, waiting for you to finish," and only act once they are done.

Routing (First determine the type, then read the corresponding section)

A single video can belong to multiple types; read all relevant sections.

If the task looks like this	Read this
"Cinematic feel", "Trailer", "Montage" — multiple clips cut into one emotionally driven piece	`references/情境_预告片混剪.md` + `references/剪辑语法与叙事.md`, `跨镜头连续性.md`, `声画节奏与调色.md`, `退件与验收格式.md`
Primarily a person talking, needs content editing and subtitles	`references/情境_口播演讲片.md`
The main character is an illustrated character requiring interaction	`references/情境_角色动画片.md`
Working with static image cards/infographics to turn into a video	`references/情境_图卡转动态影片.md`
Picture-in-picture, split screen, montage, freeze frame, speed ramp, reverse, Ken Burns, etc.	`references/技法库.md` (includes tested ffmpeg commands, marked as tested or common practice)
Transitions: whether to use, which type, how to do ffmpeg `xfade`	`references/转场指南.md`
Animated lower thirds, numeric graphics, transition effects, effect layers, 3D/Three.js	`references/动态图形与3D.md` (dynamic graphic feel, numbers, transition recipes, choosing animation engines, when 3D is worth it)
"Year N" rolling montage, time-lapse montage	`references/情境_时间流逝蒙太奇.md`
Don't know what to do: need to choose editing techniques, animation, or tools	`references/技法与工具目录.md` (each item marked as tested or common practice)
Process-based (cooking, assembly, construction, etc. step-by-step)	Stage 1's "Three questions for process-based projects"
Involves pets, family, real experiences	"Tools and Taboos": Don't use generated imagery for real-life experiences

Workflow Overview

Confirm Goal: One-sentence goal, target audience and purpose, platform and specs (aspect ratio, duration, resolution), asset list, verifiable success criteria, what NOT to do.
Script: For creative projects, first give three different storylines in text outline for the user to choose one. Final draft = sections, duration, subtitles, sound effect direction.
Rough Storyboard: Shot list + storyboard, leave technique column blank. Replace adjectives with numbers ("joyful" → bounce amplitude, action beat).
Audit Work and Risk Test: Asset audit (contact sheet); check each shot's technique against "verified capabilities"; for techniques never tried before, do a 5-second test first; if it fails, switch to a simplified backup; lock the style pack (Stage 2) here. Pass = user locks the storyboard.
Rough Dynamic Edit: Arrange the storyboard by time and sound effects into a rough edit. Confirm rhythm and length before proceeding to production.
Production: Filming, post-production.
Self-Check: Mine for issues before rendering → render → machine check → comprehensive checklist, images inspected one by one.
Delivery and Wrap-Up.

Phase 1: Pre-Production (User must approve all three stages)

Stage 1 | Facts and Script (Ask once, store answers in shot list meta)

Address, audio rules (default: no background music, only sound effects; which original audio to keep), aspect ratio, duration, resolution, video title.
First create a contact sheet for assets (tools/storyboard.py) to review content, do not rely on file names; for audio, check the spectrum or listen first.
For knowledge/educational content, verify before action: Look up each definition from official sources (official docs first, third-party summaries only as supporting evidence). Next to the shot list, note "this sentence matches which source." Mark anything unverified as "unverified." Before removing any part of the original script, determine if that part prevents misunderstanding; if you simplify, clearly state "this is my simplification, not the original wording." If you only read a search summary and not the full article, say so explicitly.
For process-based projects, first ask three questions: ① The process order; ② Which asset belongs to which final product and which step (don't guess based on shooting order); ③ Key moments that must appear.
Before saying "it's not in the assets / can't be shot," watch through the entire footage frame by frame, not just the action frames. When the user sends a screenshot, use tools/find_frame.py to find which asset and which second it corresponds to.
Selection criteria: If the footage has verbal comments from the shooter, transcribe and use them as additional evidence; otherwise, use only explicit rejection conditions (blurry, jittery, too dark). Write which clip and which shot the rejection applies to. These two methods are only aids, not guarantees.

Stage 2 | Style Pack (One page, show to user first)

1–2 reference videos or images, plus a sentence: "This is what I want to emulate from it." When analyzing reference material, specify the dimensions; otherwise, the AI might only look at color. Options include: video type, font, colors, lower thirds, elements, modules (reusable components like lower third entry/exit, overlapping cards, markers), free assets (which free assets are used, their licenses must be checked), motion, rhythm, transitions.
Colors: Background / Text / One accent color (accent color ≤15% of frame, text contrast ≥4.5:1); for narrative videos, also define a color arc (target brightness for each segment).
Fonts and lower third language: At most two sets.
Transition vocabulary: ≤3 types across the entire video; each shot has one primary effect.
Provide a style reference image (one key frame), not just text.
Technique selection: Before starting, check which techniques can make the edit more refined. No upper limit on quantity, but each must be explained with "which shot does it apply to, and why it fits." When presenting the list to the user, use plain-language names with a one-sentence description of the effect (e.g., "Push in shot: slowly zoom toward the main character"), not English codes. At delivery, cross-check: Do the techniques used match this list?
Common packaging techniques for talking-head / social explainer videos: Masked color grading (color grade only the background), background replaced with video wall, oversized text placed behind the person, before/after slider, typing input box with progress bar, freeze frame with speed ramp, keyword floating tags, picture-in-picture, number animation, cutting breaths. Don't promise techniques you haven't tested—do a 5-second test first.

Stage 3 | Shot List + Storyboard

For each shot, write: action (what actually happens on screen), shot_size (wide / medium / close / extreme close-up), subtitles, lower thirds, sound effects (store only relative timecodes). For lower thirds, write evidence (where on screen it is supported).
For each cut point, write cut_reason (why cut here, why this shot); for match cuts or cuts on action, also write anchor (what the previous shot ends with, what the next shot begins with). If you can't write a reason for a cut point, delete it first.
Subtitles: 4–5 characters per second, maximum 6; appearance time should not be earlier than the shot's entry; subtitle length determines shot length; every subtitle must be supported by the image; if not, ask the user for material.
Arrange shots with a story arc + shot size variation: First define the main character and the narrative arc (grouping similar actions together is not narrative); deliberately vary shot sizes within each segment, commonly "medium → close-up → wide" or "wide → medium → extreme close-up."
For process videos, maintain consistency: Large sections follow the actual process order; within the same section, multiple final products must progress simultaneously; completed final products cannot be inserted in the middle of ongoing sections; final products appear only in the final setup segment (except for opening preview). To "hide hands," use crop to frame an extreme close-up, only on the shots specifically named by the user.
When the user says "I want soul/atmosphere" = color grade: unify one look across the entire video.
Generate the storyboard by extracting frames from assets or finished cuts using tools/storyboard.py. Machine check only verifies text; confirming that the lower third matches the image requires human inspection.

Before submitting the shot list to the user, tick each box yourself. Do not submit if any item is missing:

Have you asked about the emotion (inspirational / heartwarming / suspenseful)? Have you asked if they want background music?
Lower thirds: First provide 3× 5-second comparison clips (three different "styles," not the same style with different fonts). First write each lower third's function, then pick from the "Lower Third Menu" in references/剪辑语法与叙事.md; the user chooses one as primary, then mix and match by segment, with one unified font across the video.
Techniques: Did you actually check? Is the list in plain language? Does each one specify which shot it applies to?
Audio: Use real recorded sound effects, not synthesized ones; each sound effect corresponds to a visual action. If not in the library, find it on a free sound effects website, confirm the license yourself, write the source in the filename, and record it in the sound effects folder's README. If no exact match, find the closest and explain.
Are all assets (including photos) reviewed? Are rejection criteria clearly written? Dates checked by shooting time, not file name.
Are facts and copy sourced only from the user? Are your own lower third ideas presented only as options, not placed directly?
Is the style reference image (three key frames) submitted together? Is the cover frame saved separately, without total duration added?
For short user replies ("A," "OK," "whichever") that could be interpreted two ways, have you confirmed first?
Does every cut point have a cut_reason?
For videos with background music: Before mixing, create a Sound Map (layers, start/end, priority, source).
For emotional lower thirds, first ask "Can I replace this with a shot?": For every lower third that "draws an emotional conclusion" for the viewer (funny, touching, awkward, shocking), note next to it whether it can be replaced by "character looks → reaction" or "shot A cut to shot B"; if yes, write the replacement shot; only keep the lower third if material is missing or time is tight. Informational lower thirds (numbers, years, names, locations) are unaffected. The effectiveness of this technique is still debated in academia; treat it as an A/B option, not a guaranteed formula. For real-life documentary footage, do not edit together reactions from different times to create an emotion that didn't happen.

Phase 2: Production

Video: First produce a low-resolution draft; once finalized, render at final quality; if changing one shot, only re-render that shot. For HLG/HDR footage processed with ffmpeg, convert to SDR first; measure brightness (signalstats) for each shot, matching the color arc target.
Real screen high resolution: Capture website/app screens using browser automation at 3× resolution; for 4K output, set devicePixelRatio. If the source asset resolution is insufficient, tell the user the upper limit before starting. When recording your own demo site, don't open pages too quickly in succession—you may hit your own rate limiting and record error pages.
Special effects: search asset library first, then hand-code: Only hand-code after searching the library, and note the gap. Common cinematic effects: push-in / pull-out reveal / rack focus, whip pan, light leak, film grain, vignette, soft-focus entry lower thirds. Don't use shake, whip pan, diagonal swoop, or focal impact for medical or heartwarming scenes.
Lower thirds (HyperFrames or similar HTML animation): Write timeline into window.__timelines; for fromTo and subsequent from calls, add immediateRender:false; for elements that should fill the frame frame-by-frame, first set opacity 0; keep subtitles at the same height across the entire video; don't block main subjects; completely clear all elements from the previous shot when transitioning. Before rendering, scan with tools/hf_lint.py.
Chinese lower third layout (based on 1920 width): Main title: 900 font weight, ≥120px, letter spacing 0.01–0.03em, ≤14 characters per line, manually break between words, no orphans at line start or punctuation; body text: 500–700 font weight, ≥32px; half-width spaces between Chinese, English, and numbers; line height ≥1.18 inside masks. Minimum lower third hold time = character count × 0.18s + 0.6s. For on-screen labels, use solid background with dark bold text, placed on empty areas.
Rhythm: Something should move within 0.3 seconds of the beginning; the first text should appear within 1.5 seconds; the next action should start 0.1–0.3 seconds before the previous one ends; only one main subject should move at a time; exit duration should be about half of entry duration.
After generating a modified script, confirm the file wasn't truncated before running.

Character Keying and Masking

Edge approval = zoom in to check fingers, hair, and shoulders; don't just look at the thumbnail. Mask processing: first blur about 1.6, then increase contrast, shrink by 2px, for smooth edges without white fringes.
Next to the keyed segment, don't place cards overlapping the character or burned-in subtitles (original subtitles on top of the person will be treated as "not a person" and become transparent).
Don't guarantee untested results: there may still be a few pixels of color edge at the fingertips—say so directly.

Implementation Pitfalls (GSAP and ffmpeg)

Total duration must be an integer number of frames (duration × fps is an integer), otherwise the frame count check will be off by 1.
Original audio sync points should be calculated from the timing file, not hardcoded seconds (changing shot length will desync the audio).
When the subject is black, don't use the zoomin transition (it will zoom into complete black). Use smoothleft or fade instead. For horizontal photos in a vertical video, crop to vertical first to save the face; don't use blurred background filling.
Three GSAP bugs that cause "appear → disappear → reappear" or freeze the entire video: ① If tl.set(groupSubElements, {opacity:1}, 0) is placed before a later tl.from(...), it will override the hidden starting point of from; to hide, use tl.set(allElements, {opacity:0}, 0), then use to/from to reveal. ② A selector matching two elements causes one to be animated prematurely; only animate the intended element, or use an id. ③ Variable names in a new scene that collide with another scene cause onUpdate to throw errors and freeze the video from that shot onward. Before adding new variables, grep the entire file for duplicates; after changes, use actual rendered frames each shot, extracting one frame before and one after each cut change.
During color grading in post-production, don't use colorbalance highlights (it will tint white backgrounds yellow). To add vignette, use vignette=angle=PI/8. After color grading a pure white web page, remeasure the white point.
Voiceover: Only do it if the user explicitly requests it (default is only sound effects). For each sentence, apply silenceremove, and if necessary atempo (≤1.2). Then use a Python array to layer each segment into a same-length wav file based on timecodes; a single ffmpeg amix with many adelay inputs will truncate the output. Before mixing, confirm the sound effect file length matches the final video length. If subtitles change, regenerate the audio accordingly.
If the user complains it's "too loud," only delete the sounds they point out; before deleting, list "what's being removed" so they know. For something emotionally important like original audio, it's better to keep one.
After the video is finished, the user may insert a shot midway: only modify the shot list (add shot, extend subtitle, move sound effect positions), re-run checks, re-assemble shots, re-render, and re-mix; subsequent start and end points should shift automatically. When inserting a photo shot, first check the composition; don't place subtitles over a high-saturation main subject.

Phase 3: Post-Production

Color grade according to the color arc. Use one unified look system across the entire video, not a different filter for each shot.
Default cinematic color grading: S-curve, slightly lifted blacks (matte finish), vignette, film grain placed on the final layer. Split toning (cool shadows, warm highlights) only applies to color LUTs; for black-and-white segments, use a slightly cool tone instead, or the B&W will look like yellow-green old photos.
Sound effects: start minimal; each sound corresponds to a visual action. Before delivery, ask yourself "what would be missing if I removed this sound?" Remove those that don't fit the situation; use loudnorm with alimiter to avoid distortion.
Always add level=disabled to alimiter (the default auto-gain pushes peaks back close to 0dB); in the same step, apply atrim to the video length, and add -shortest to the output; otherwise, the audio track will be longer than the video, and the frame count check will fail. Finish with loudnorm=I=-16:TP=-1.5.
Don't let sped-up/slow-motion shots stretch audio with atempo (it will produce distorted sounds): for slow-motion/fast-forward shots, use original-speed audio; take video and audio as two separate inputs.
Always ask the user to listen with headphones for the final audio; measurements and speech recognition are only aids. Low frequencies (below 150Hz) are not played on laptops and phones; sound effects must have energy above 300Hz.
For lightweight version ≤9MB: if -maxrate can't compress enough, switch to x264 two-pass encoding.

Removing Vocals / Retaining Ambient Sound

Don't use speech recognition to find segments and then mute them (voices are often mislabeled as non-speech, and timestamps are only to the nearest second). Instead, use a source separation tool (e.g., demucs) to extract a vocal-free audio track—one pass is enough. After separation, use speech recognition to scan again as a cross-check: content in parentheses is non-speech and not considered residue; repeating the same sentence or "remember to subscribe" are recognition hallucinations.
Separation tools may mistake sound effects (like a spray gun burst or a knife strike) for vocals and remove them: for the few seconds of such critical sound effects, add the vocal track back, but strip the 250–3800Hz vocal frequency band.

Auxiliary Tools: Find Cut Points, Cut Breaths, Detect Beats

All three have been functionally validated using synthetic footage (results attached); when using real footage, thresholds and effectiveness must be adjusted based on the material.

PySceneDetect: scenedetect -i video.mp4 detect-adaptive list-scenes, lists shot timecodes. Validation: a 6-second video composed of three hard cuts of different content detected 3 shots, with cut points at 2.0s and 4.0s, matching reality. Animated footage may produce "shots" that are not what you want.
Auto-Editor: auto-editor video.mp4 --edit audio:threshold=5% --margin 0.15sec -o output.mp4, cuts silent gaps in talking-head video. Validation: a 6-second video with three 1-second audio segments interleaved with 1-second silence was cut to about 3.85 seconds. Threshold must be tested per asset: the same synthetic footage at 15% was entirely classified as silence and cut to nothing. After cutting, re-check subtitles, lip sync, and audio-visual sync; subtitle timestamps must be re-generated from the cut version. Cutting out breath does not mean you can remove filler words; subtitles must not have fewer characters than what was spoken. Use the official release (e.g., from Homebrew), not similar-named shell packages.
librosa: librosa.beat.beat_track retrieves beat positions and BPM. Validation: synthetic drum beats at 120 BPM measured ~117.5 BPM and 23 beats, within 2% error; still ask the user to listen to the beats before delivery. Only use for videos where the user wants beats; no need to follow the beat for leisurely videos.

Phase 4: Pre-Delivery Self-Check (Each item must have evidence to pass)

First, run machine check: tools/qa_video.py final.mp4 output_folder --size WxH --fps N --share share_version.mp4 (checks resolution, fps, integer frame count, audio track length, clipping, loudness, black frames, freeze frames, before/after cut frames, share version size). Numbers are checked by the tool, you must inspect the images one by one. Tick each item:

Cleanliness (do this for every delivery): For each shot, extract the first frame, middle frame, and last frame, combine into one image, and inspect each. Any hand or shadow entering frame, unexplained action, entire shot with nothing happening, flickering shadows from speed changes—shorten or replace immediately.
Cut continuity (required for multi-shot emotional videos): Use tools/cuts_sheet.py to extract 2 frames before and after each cut point, inspect each for direction, eye line, action phase, shot size, lighting. Write issues with image paths following references/退件与验收格式.md. Self-correct up to 2 rounds; if still not passing, give the user the final decision.
Specs: Resolution, frame count, aspect ratio match what the user requested; final output is downscaled from 4K or higher.
Copy: Every word on screen is from the user. When there is an original script to compare against, use tools/ocr_check.py to convert key frames of the final video and the original script into text, then compare sentence by sentence; if mismatches are found, go back to the original text (recognition error ≠ screen error). Text matching does not guarantee content correctness—verify content against the source separately.
Lower third = image: Every lower third's name matches what actually happens in that shot. When selecting segments and cropping, reference the actual frames from that segment, not your own description.
Subtitles: Character count per second is acceptable, not started before the previous shot, same height across video, no automatic line breaks.
Readability: All text is still clear on the share version (compressed).
Clean: No leftover elements after a cut, no double images, no recorded error pages or test data.
Consistency: ≤3 transition types, color follows the color arc, ≤2 subtitle languages.
Rhythm: Something moves within 0.3 seconds of the opening; no two main subjects move at the same time; lower thirds hold long enough.
Audio: No clipping, every sound effect has a corresponding action, no inappropriate sound effects.
Reproducibility: Render twice at the same time point; screenshot hashes match (program animations must be pure functions of time).
Honesty: For shots where quality cannot be saved, or items that have not been verified, say so directly at delivery.

Phase 5: Delivery and Wrap-Up

Delivery folder (you choose the location; the following is a suggested structure), one folder per video, date first, then name (YYYYMMDD_Title):

成品/: High-resolution version, lightweight version (for chat apps, ≤9MB), cover image, subtitle file.
初剪/: Process versions and input assets.
定稿图表/: Style pack, storyboard, shot list (machine-readable file plus a human-readable .md), 原始码/ (scripts for redoing, plus README).
Test items (new tools, 5-second samples) go into a separate 测试/date_name/ folder; only official projects go into the project folder.
Use tools/deliver.sh to combine sound effects, compress lightweight version, copy, and verify md5; do not overwrite files with the same name.
After the final video is revised, update the final materials: shot list and storyboard timestamps and start points must match the final video, or you'll leave outdated documentation.

Keep only final versions: Intermediate products go into the project's .work/ folder. When cleaning, use tools/safe_trash.sh to move to the trash—do not delete directly; first list everything for user confirmation. Do not touch other people's work areas.

Five Mandatory Wrap-Up Tasks (Every Video)

Finalization: First verify the cover image. The video header must have a cover frame (title plus main subject); save the cover image separately; do not add total duration to the cover; deduct from the opening assets. Derive the cover copy from the final conclusion lower third in the video, not from the file name or process segments. Rename the final video to <Title>_<Duration>.mp4, lightweight version ≤9MB. Rebuild the 定稿图表/ folder's storyboard and shot list to match the final video. Verify md5.
Clean Up: Move replaced old storyboards, old scripts, various draft versions, and working directories all to the trash using safe_trash.sh; write in the README what's in the trash so it can be retrieved if needed.
Close Tasks: Mark all to-dos or tasks opened for this video as closed.
Post-Mortem: For each revision, write what the user changed (quote their exact words), the root cause, and how to prevent it next time; list your own mistakes separately; reusable approaches; mark unverified deductions as "unverified."
Optimize This Skill: From the post-mortem, if any "how to prevent next time" can be turned into a rule, write it back into the corresponding section of this SKILL.md—don't just pile it at the end.

Tools and Taboos

Prioritize real footage (ffmpeg plus HTML animation). For what wasn't captured: static image generation (don't generate Chinese text; overlay Chinese text with HTML) → other external assets. Don't use generated imagery for real-life experiences (pets, family, client cases).
Before installing any new tool, package, or model, first check the source and license, ask the user before installing, and don't install while working.
Do not execute commands that send free-form text externally (e.g., the tool's feedback command); keep tool statistical reporting disabled by default.
Do not write unverified content, the user's private assets or personal data into public samples or examples.

josenjc9/short-video-edit

short-video-edit 是什么？

在你喜欢的 AI 中提问

文档