pj1433139082-hue/video-design-skills

remotion-build — Generate Remotion Videos

Use React code from DESIGN.md (produced by remotion-consult) to generate an MP4. Subtitles are the sole timeline, animations are imported from the master animation library, and assets use cross-harness routes.

Workflow

Step 0 — Mode Selection (Choose One of Four, Decide Before Acting)

1. By Category Template: User wants a predefined style, or picks from the four categories in remotion-shared/categories/ (voiceover subtitles / data visualization / product demo / brand animation) → Read that category's DESIGN.template.md + motion-list.md, inherit the design system, only change content and subtitles.
2. By Reference: User provides a reference video (highest priority) or picks from remotion-shared/video-real-library.md → Deconstruct its subtitle style / transitions / rhythm, recreate using components from the master animation library.
3. Free Design: No reference, no template → Only avoid the reverse prompt words in video-anti-slop.md.
4. By Existing DESIGN (consult output): Consult has already set the DESIGN + style samples approved → Strictly follow DESIGN tokens/§8, do not re-ask about style direction (Step 0.5 only aligns acceptance criteria, does not repeat Phase 1 clarification). If unsure, list options for the user.

Render Engine Binary Choice (Fifth Dimension, marked in DESIGN): In addition to the four modes above, decide on the render backend (this skill defaults to the Remotion engine; only if DESIGN specifies or user requests, use the HyperFrames engine):

• Remotion Engine (Default): React complex animations / shotcraft shot cards / src/ layering + Sequence + useCurrentFrame. Use when maximum animation freedom is needed.
• HyperFrames Engine (HeyGen open-source HTML→MP4, optional): index.html data attributes + GSAP timeline (npx hyperframes init/render, free locally) → closer to web development feel, built-in npm run check (lint + contrast + motion). Assets/BGM/subtitles/acceptance all reuse, only the rendering method changes. Rules see references/hyperframes-guide.md.
• Decision: Complex shot cards (flip cards/particles/3D) or existing Remotion project → Remotion; Pure HTML/GSAP animations, desire web feel, need built-in lint → HyperFrames. Do NOT mix the two engines in the same project.

Step 0.5 — Requirement Alignment + Acceptance Criteria (grill-me, Mandatory, ≤10 Rounds)

Run the grill-me skill to align: one-sentence requirement, aspect ratio / duration / platform, subtitle source (.srt/.vtt/script from scratch), mode, style direction. Must produce acceptance criteria in four categories:

• Structure: Category / Aspect ratio / Duration / fps
• Visual: Subtitle font / Color scheme / Composition consistent with DESIGN.md or reference (for Free Design: "no AI feel")
• Content: All subtitle sentences present, no typos, timestamps cover the entire video, authentic copy
• Technical: Subtitles frame-aligned (subtitles exist at sentence start/end frames), still frame extraction passes, MP4 render successful

If not aligned after 10 rounds → Record unresolved items and proceed with the build with known issues.

Step 1 — Subtitle-Driven Process (Core)

Follow references/captions-guide.md:

1. Source Routing: Existing .srt/.vtt → Use directly; Has script .md → Draft a rough .srt; From scratch → First draft a script, then .srt.
2. parseSrt({input}) → Caption[], validate empty segments / time overlap / out-of-bounds.
3. Frame Alignment: startFrame = Math.round((startMs/1000)*fps), endFrame = Math.round((endMs/1000)*fps), each sentence as <Sequence from={startFrame} durationInFrames={endFrame-startFrame}>.
4. Word-level/Sentence-level Highlighting: createTikTokStyleCaptions → CaptionLayer component; For Chinese, a single token per whole sentence → degrade to sentence-level (decision based on DESIGN.md highlightMode).
5. Screen rhythm is bound to subtitle sentence start times; Full video durationInFrames = round(max(endMs)/1000*fps) + trailing.
6. After modifying words, write back with serializeSrt.

Step 1.5 — Voiceover-Driven srt Reconstruction (Timing Assumption Truly Activated, Voiceover Before Frame Extraction)

For voiceover videos, voiceover is a mandatory process (no need to ask the user): TTS generation → ffprobe actual measurement → rebuild srt to finalize timeline, the timing is based on the actual TTS measurement (DESIGN §4 timing assumption truly activated). Voiceover before frame extraction — prevents Step 1 srt alignment from being overturned by voiceover reconstruction, making previous steps wasted.

1. python scripts/media/tts_gen.py --provider edge-tts --lines lines.txt --save-dir public/audio/ --voice <voice> (defaults to a keyless fallback; when MiniMax is available, explicitly switch to --provider minimax, for domestic cloud TTS see mmx-media-pipeline.md §2).
2. ffprobe actual duration per sentence (ffprobe -show_entries format=duration), with 0.8s breathing gap between sentences.
3. Rebuild srt (rebuild_srt.py): Each sentence's start/end = cumulative actual duration + breathing gap; write to public/captions.srt.
4. Root SHOTS frame intervals updated according to the new srt (round(ms/1000*fps)), each Sequence mounts <Audio src={staticFile("audio/s{n}.mp3")}>.
5. BGM: When AI music generation is not needed, obtain from the project's licensed asset library; when needed based on prompt, default to local generation with ACE-Step, with cloud music backend as an optional alternative. After obtaining, trim to full length (ffmpeg -t video seconds), lower volume so it doesn't overpower voiceover (0.22-0.3).
6. Scene Frame Budget + Pre-Timing Check (Prevent "Key Animation Not Appearing", Tested 3 Times): The to-from of each sentence output by rebuild_srt is the frame budget for that scene. Before writing shot cards / data animations, verify: the last keyframe inside the card ≤ scene budget, otherwise compress the card's timing (don't wait until acceptance to discover this) —
   • Short sentences (2-3s≈60-90f): Stamp/scan circle/periodic lighting etc. tail animations most likely exceed budget (S2 red stamp originally 70-88f exceeded 68f budget → compressed to 55-65f; S6 periodic lighting originally 46-70 exceeded 70f budget → compressed to 46-64; S9 5 circles scan originally 136f exceeded 128f budget → compressed to 108f).
   • Rule: Card last keyframe = start + cascade total length + tail hold, must be ≤ sentence budget; if exceeded, proportionally compress stagger/single animation duration, keep tail hold ≥4f.

Step 2 — Find References

User-provided reference is highest priority → video-real-library.md → Free design. With reference, use "deconstruction animation method" (reference → combine master animation library components, see motion-patterns-video.md).

Step 3 — Reverse Prompt Words (Video Version Condensed, Only Hard Constraint)

• Do not use the same subtitle rhythm throughout (every sentence same length, word-level highlighting flashing indiscriminately)
• Do not center the composition in every scene, do not use only fades or only hard cuts throughout
• Do not cut scenes evenly (cut every N seconds = no breathing feel)
• Do not default to purple-indigo gradient backgrounds, do not use cliché AI fonts
• Do not use empty voiceover subtitles ("Welcome to watch", "Let's start")
• Animations must be present: At least an entrance animation per scene + a transition between scenes
• That's it.

Step 4 — Build (Technical Baseline)

Follow references/remotion-build-guide.md:

• Project init: Copy skeleton from _dev/remotion-tutorial + npm i @remotion/captions@<aligned version> + keep Chrome path.
• Import components from the master animation library (Workspace/Video/_AnimationLibrary/) or copy to src/components/.
• Shot card optional integration: If DESIGN §8.2 selects cards, follow the "Shot Card Integration Steps" in remotion-shared/video-shots-spec.md (including [copy]/[borrow idea] intent markers); Chinese videos default to borrowing ideas.
• src layering: Root.tsx / theme.ts / captions.ts / components/.
• Deterministic animations (useCurrentFrame/interpolate/spring, forbid Math.random and browser timers).
• If DESIGN.md exists, strictly follow tokens.
• Assets follow references/video-image-spec.md cross-harness routing (Codex image_gen / GPT prompt round-trip / user-provided images).
• Asset generation post-check outline (mandatory): After generating assets, verify against DESIGN §8 outline sentence by sentence — ① Does the image match the sentence semantics/layout (vision evaluation of fit, <7 regenerate or adjust prompt) ② Do large keyword graphics coordinate with the scene image without obstruction ③ Is the layer z-order consistent with the outline design. Non-compliance → adjust keywords or regenerate assets, do not "generate and use"; select the best among multiple candidates for each screen element.

Step 5 — Acceptance Closure (grill-me Node Table)

Node	Trigger	Max Rounds	Exit Condition
Start	Step 0.5	10 rounds	Acceptance criteria finalized
Modify	User requests change	5 rounds/instance	What to change is clear, each round checks against acceptance criteria
Final	Before delivery	4 rounds	Acceptance criteria checked item by item

Hard Acceptance Actions (Frame Extraction Before Rendering, Prevent Long Render Waste): Layered Quantification (4 Layers, Proven on Keyframes at Sentence Start/Middle/0.82 End State in DESIGN §8.2) :

• Layer 1 Background Image (scene image itself): python scripts/media/layout_calc.py <scene image> — composition/whitespace/suggested text area (find empty areas based on image whitespace structure, do not force top area to be pristine; if top area is invaded → move text to text_zone suggested area, or regenerate a cleaner image);
• Layer 2 Background Image + Text (still with text): layout_calc text placement + frame_judge --subtitle-hex true WCAG contrast (large text≥3.0/body text≥4.5);
• Layer 3 + Animation (consecutive frames): Frame difference/consecutive frames vision judge animation truly happening + animation does not overlap with scene image (card is a complete screen, not overlaid on scene image, visual mode see §8.2);
• Layer 4 Overall (final video): frame_judge full video + audio_sync_check.py audio-video sync + occlusion check + four-skill scoring + action proof.

1. npx remotion compositions confirm correct id;
2. Extract 3-5 still frames (sentence start/end/transition frames) — extract "end state frames" not mid-animation frames (lesson learned 2026-08-17): Select frames after animation completion (0.82 end state / after tail hold), do not extract frames mid-animation — mid-animation frames cause vision/human eyes to misjudge as "missing/static/black frame" (S2 flip card mid-way no red stamp, S6 odometer rolling mid-way no periodic point, S7 timeline mid-way no portrait card = wrong extraction timing, extract end state frames instead). Subtitles exist and are aligned, composition intact;
3. Subtitle alignment verification: Subtitle present at sentence start frame, not vanished early at sentence end frame;
4. Automated hard metric judgement: After frame extraction, run python scripts/media/frame_judge.py preview/*.png --subtitle-hex <DESIGN subtitle color, e.g., E8833A> --subtitle-threshold 3.0 --text-bg-hex <text capsule background, e.g., F2F0EB> (use --text-region when text is in a corner, pass actual position tl/tr/bl/br/center, to prevent false bottom area alarms; use --text-bg-hex when capsule background is known to directly calculate subtitle vs background true contrast, skip mode calculation — mode captures scene color, inaccurate when text overlaps scene; large text threshold 3.0 / body text 4.5; any FAIL triggers revision); optionally pip install iqa-pytorch then add NIMA aesthetic score (>6.5 recorded as reference grade); metric thresholds see references/video-quality-metrics.md; dark design brightness FAIL does not block (see dark design explanation in metrics, contrast is the standard);
5. Check each extracted still against checklist (score <3 triggers revision):
   • Composition: ≥2 visual elements, not centered in every scene, correct layer z-order (bg/mid/fg)
   • Subtitles: Present and aligned, legible (contrast ≥3:1), no broken characters
   • Transitions: Transitions/entrance animations between sentences, not bare cuts or static
   • Assets: No AI garbled text, layers match §8.1, transparent overlays clean
   • Animation: At least 2 animation types present per scene (asset entry/exit + font + background choose two out of three), background has animation
   • No AI feel: Check against video-anti-slop.md reverse prompt words
6. Occlusion check + visual mode verification (inspired by paper-collage-video spatial-contract, expanded 2026-08-16): Verify extracted stills sentence by sentence — ① Large text capsule in reserved area (top whitespace) ② Main subject does not encroach on text/subtitle area ③ Foreground does not occlude mid/background subjects ④ Card animation does not overlap with scene image: Full-screen cards (odometer/dial/particle beam/focus dark field) must never be overlaid on a scene image (choose one of three visual modes, see DESIGN §8.2) ⑤ Animation meaning must match sentence semantics (vision judge what the particle beam/spotlight/digit scrolling expresses in that scene, whether it aligns with the voiceover text; unclear meaning triggers revision). Occlusion/unclear meaning → change visual mode or remove unnecessary card.
7. Four-skill review hard gate (any gate not passed → revision, not reference): Executed by sub-agents independently (hallmark/taste executed by independent agents, not self-review by main model) —
   • taste-skill Pre-Flight (40+ taste checks)
   • hallmark audit (57 gates anti-AI slop)
   • ui-ux-pro-max (UX principles: readability/hierarchy/consistency)
   • open-design/anti-ai-slop ("Seven Deadly Sins" de-templating) Review against extracted stills/final video, score by items passed (supplemented by reviewer, prevent self-review): hallmark 57 gates pass ≥90% (≥51 items) to proceed, taste 40+ items pass ≥85% to proceed, ui-ux-pro-max / open-design all key items pass; list failed items triggering revision, proceed to render only after all gates passed.
8. npx remotion render <id> out/final.mp4;
9. Action proof moments (inspired by paper-collage-video proof moments): According to DESIGN §8.2, for each sentence, extract proof moments (sentence start/middle/0.82 end state) + verify action types actually occurred (entrance has happened / continuous motion / end state static); ffmpeg -vf "fps=4" extract consecutive frames → vision judge actions occurred according to DESIGN — a single static frame of a translate animation ≠ action proof (particles/light beam/digits must be confirmed by consecutive frames); transitions smooth / entry-exit no jumps (video-quality-metrics §4.6);
10. Audio-video sync check (main axis for voiceover videos, supplemented by reviewer): Run python scripts/media/audio_sync_check.py out/final.mp4 public/captions.srt on the final video — within ±0.15s of each srt sentence start, there should be voiceover audio (RMS≥0.01); silence at sentence start = voiceover misaligned/missing/end truncated → revision;
11. Check acceptance criteria item by item → STATUS PASS/FAIL/BLOCKED, NEXT_OWNER.

Red Lines

• Subtitle-driven is a hard constraint: If captions.srt exists, it is the sole source of the timeline; duration is derived from it.
• Extract frames before rendering: Still frames not passed, do not render full video.
• Shot card borrowing not mandatory: If DESIGN §8 selects a card, integrate it (copy shotcraft/demos/{category}/{card} and adapt); no selected card or borrowing ideas for Chinese videos does not count as omission; not integrating a card is not BLOCKED.
• Assets use real image generation, do not generate SVGs: Primary assets prioritize Codex or configured image backend; transparent assets use --transparent/remove_white_bg.py.
• Procedural data cards prioritize true copy of shotcraft full version, do not hand-write simplified SVGs (lesson learned 2026-08-17): Self-built data cards (dial/ring/odometer/chart) if hand-written as simplified SVGs (bare circle/line/path, no ticks/red zone/hub/glow details) will appear shoddy (S9 ring hand-drawn version mmx vision 3.5/10, true copy of shotcraft NeedleSweepSelftest full details scored 8.5/10). For data cards, always true copy the full version from shotcraft demos/data, typography (include all details: ticks/primary-secondary/glow/dial hub/pointer tail etc.), then reskin colors; only simple decorations (background light spots/base tracks) can be procedurally drawn.
• Voiceover-driven srt reconstruction is mandatory (Step 1.5, voiceover before frame extraction): For voiceover videos, Step 1.5 MUST do TTS→ffprobe→rebuild srt to finalize timeline (timing based on actual TTS measurement, DESIGN §4), frame extraction and acceptance happen after voiceover reconstruction srt, do not ask the user.
• Do not change design direction: Once DESIGN.md is set, follow it; if direction needs to change, go back to remotion-consult.
• Forbid fabrication: Subtitles/data not fabricated.

Reference Files

File	When to Read
references/captions-guide.md	Step 1 Subtitle timestamp alignment (Core)
references/remotion-build-guide.md	Step 4 Engineering specifications
references/motion-patterns-video.md	Step 2/5 Animation implementation
references/video-image-spec.md	Step 4 Asset routing
references/codex-imagegen.md	Step 4 When assets use Codex
references/mmx-media-pipeline.md	Step 4 Media backend (MiniMax optional + TTS/BGM fallback chain + known pitfalls)
references/video-quality-metrics.md	Step 5 Automated metric thresholds (frame_judge.py + VMAF/NIMA/ASL thresholds)
references/hyperframes-guide.md	Step 0 When render engine=HyperFrames: HTML data attributes/GSAP/check rules + CLI
references/hyperframes-blocks.md	Step 0 When HyperFrames engine: select animation blocks, 154 registry blocks by category (equivalent to shotcraft)
.claude/skills/remotion-shared/video-anti-slop.md	Step 3 Reverse prompt baseline
.claude/skills/remotion-shared/video-tokens-spec.md	When writing theme.ts
.claude/skills/remotion-shared/categories/{category}/motion-list.md	Selecting animation combinations
.claude/skills/remotion-shared/video-shots-spec.md	Shot card system + authoritative techniques (build time: copy from demos/{category}/{card} into project)
.claude/skills/remotion-shared/aesthetic-judgment/	Step 5 Acceptance: six-axis judgment of photography/editing/front-end typography (<3 triggers revision)
.claude/skills/ui-ux-pro-max/SKILL.md	Design rule engine: Step 4 Build time check UX principles + priority
.claude/skills/taste-skill/SKILL.md	Step 5 Final check: Pre-Flight 40+ items (self-check before delivery)
.claude/skills/hallmark/SKILL.md	Step 5 Design review: 57 gates / hallmark audit (before acceptance)
资料库/AI工具/design-skills/open-design/craft/anti-ai-slop.md	Step 3 De-templating rules ("Seven Deadly Sins") + 151 design system alternatives
.claude/skills/design-shared/review-panel.md	Review panel: agency-agents perspective + design system library/aesthetic reference library criteria (Step 5 Acceptance scoring)

Output and Wrap-up

STATUS: PASS | FAIL | BLOCKED
Output: out/final.mp4 path + preview/ extracted frames path
Acceptance Criteria: item by item Passed / Not Passed / Unresolved
NEXT_OWNER: user (view effect) | remotion-consult (change direction)
Pending: Unresolved items