remotion-broll

Material Engineering (Executable Pipeline)

Positioning: This is a working skill, not a rule document—judgment, commands, and acceptance are executed in one go. The complete rule source is in remotion-assembly/references/broll.md (Level 4 priority details/every case material list obligation/reference specifications), and in case of conflict, that document prevails.

When to Call

• Step 2 of the clip workshop determines a "real material segment" (product experience visuals: typing/generation/interface/demo/operation)
• Material list collection before finalizing the assembly plan (1 product screenshot or 5s screen recording per case, AI self-collected without waiting for user input)
• User provides screen recordings or real footage to embed into the final video

Execution Pipeline (each step has verification, proceed step by step)

① Leveling: Select route by four-level priority—user screen recording > official video > web screenshot > UI mockup card. Cases with public websites/official channels must not drop directly to Level 4.

② Acquisition

• Official video: yt-dlp --print "%(channel)s | %(title)s | %(duration)ss | %(id)s" "ytsearch8:<product name> official" identify the official channel → yt-dlp -f "bv*[height<=1080][vcodec^=avc1]+ba[ext=m4a]/b[vcodec^=avc1]/b" -o "public/broll/<name>.mp4" <url> ([vcodec^=avc1] filters out default AV1, same as HEVC not stored, see broll.md 2c); on 403 error, try UA / --cookies-from-browser / proxy in turn.
• Web screenshot: node_modules/.remotion/chrome-headless-shell/*/chrome-headless-shell*/chrome-headless-shell --headless --screenshot=<out.png> --window-size=1920,1080 --virtual-time-budget=10000 <url>.

③ Verification (mandatory): file <file> to prevent "PNG is actually HTML"; video ffprobe to verify codec, transcode HEVC to H.264 (prevent black screen); visually Read screenshot to confirm not a login wall/cookie popup.

④ Selection: ffmpeg -vf fps=0.5 extract frames to create a contact sheet, Read frame by frame, select a 3-8s shot segment synchronized with the narration semantics (input → action → wait → result), record start and end seconds.

⑤ Embedding: Inside a Takeover, use <Sequence from={beat(x)}> + <OffthreadVideo muted src={staticFile('broll/….mp4')} startFrom={F(start second)}> (muted must be added); takeover without PiP (user decides); screenshots use ShotCard/WindowCard. For media sequences, add premountFor≈fps to prevent first frame black, but that Sequence must not have layout="none" (after removing wrapping layer, premount fails); if the outer layer uses return null to gate container visibility, null must open early ≥premountFor frames (seg06 real test: container frame<enterAt-30 return null with inner premountFor={30}), otherwise parent null cuts off the preload window, making premount ineffective.

⑥ Color Grading Unification: Add filter: brightness(0.9-0.92) saturate(0.95) to the material layer (fine-tune based on main footage brightness), raw output = sticker feel (mistake #18); takeover segments use a fixed setting: brightness(.92) saturate(.95) (seg06 real test fixed setting, do not adjust per segment by eye).

⑦ Editing Processing: Emphasized parts use punch-in (crop+scale to key area, key sentence character height ≥40px), frame selection, speed changes; reference type go with a two-stage approach: "full card 2-3s → punch-in key sentence". Takeover window matches material natural duration, prioritize natural rate—if material is shorter than the window, shorten the window + cross-dissolve earlier, don't slow down to fill the window; playbackRate<1 only for interface scrolling types and only as last resort when window cannot be shortened (≥0.7). For burned-in highlights/translated subtitles with no clean frame material → use mask reveal method (white mask scaleX 1→0 reveal, and must register in SOURCES.md with "mask reveal method", see broll.md 8.5).

⑧ Segment Render Verification: Render only that segment with --frames=a-b, extract frames and Read to check: sticker feel (color grading), readability (≥16px), crossing lines (whiteboard safe zone), sync with narration.

Storage Convention

Video public/broll/; screenshots public/assets/; missing items written into the assembly plan table for user to supply. Copyright note: Official promotional materials used for commentary videos.

Names Must Have Portraits (User Decision 2026-07-22, Iron Rule)

When the narration mentions a specific person, the avatar circle in the person card should use that person's real photo as much as possible, do not use letter monograms as a substitute (user original: "When a person is mentioned, try to put an image as much as possible. You can search the internet for that person's most representative image and put it in the avatar circle"). Monograms/icons are only fallbacks when no image is available.

Image Sourcing Rules (two categories, don't mix):

• Public figures (Jobs, Musk, etc.): AI may source images independently, but only from sources explicitly marked with permissible licenses—preferably Wikimedia Commons entries with CC BY / CC BY-SA / Public Domain, followed by official press release materials. Do not grab images from search engines casually (most are commercial photography with unclear rights; the user's video will be published). After sourcing, must register in public/broll/<title>/SOURCES.md: filename / source URL / license type / photographer attribution. CC BY-SA requires attribution in the credits or description; inform the user upon delivery.
• Non-public figures (e.g., teachers, friends, colleagues mentioned in narration): AI must not search for images online—can neither confirm identity nor have authorization. Follow the "name requires material" process to ask the user to provide; if not provided, keep hand-drawn name card/monogram and explain why upon delivery.

This overrides the old standard: The early approach of "no portrait for Jobs, use text/icon chip only" was a conservative copyright measure. Now per user decision, it becomes "use clearly licensed real photos," with copyright issues resolved by selecting authorized sources rather than omitting images.

AI-Generated Collage B-roll (Emotional/Metaphor Scenes Without Real Material, 0722-01 Finalized)

When none of the four real material levels apply but emotional or abstract metaphorical visuals are needed (e.g., repetition, surgery, phone scrolling, fragmented attention—scenes impossible to screen record or find official videos), follow this process: Codex image_gen generates static frames → Grok image_to_video animates them → embed into the final video. Execution follows the gbro-collage-broll skill's three gates (Gate1 Metaphor / Gate2 Static Frame / Gate3 Video), but pacing, container, and composition rules here take precedence (overriding gbro defaults).

① Container: Default is a right-zone card, not full screen. Place B-roll in the right-zone evidence card of the three-column layout (16:9 proportional objectFit:contain full display, no cropping—user decision "scale down proportionally"), the person remains on screen, the left zone retains sidebar + timeline HUD. Full-screen takeover only when "no information needs to be juxtaposed with the material" (A-roll iron rule: full screen is the only time to cut the person, see layout §2). Reference component: src/koubo/scenes-0722-01/CollageBroll.tsx (card + OffthreadVideo, border follows chapter color). Don't default to full screen—0722-01 mistakenly did full screen and was rejected by the user, requiring rework.

② Pacing: One image per narration sentence, cut points for rhythm, no in-shot reveals. In dense narration, each image lives only 1-3s. Prohibit assemble-from-empty entrance animations (starting from emptiness and only complete on the last frame = information always lags behind speech rhythm, causing 8 video renders to be scrapped). Instead, "animate"—the static frame composition must be complete from frame 0; Grok only adds continuous micro-motion (paper floating/gears spinning/light strips sliding/fog surging); the composition must not change at all from frame 0.

③ Static Frame Composition Rules (Codex image_gen):

• Proportion by type (one-size-fits-all fails): Object-driven metaphors (hospital bed/pill/calendar/fragments/book pile—people are just elements): person occupies 45-60% of height; Person-driven metaphors (working/sitting/button pressing/daydreaming—person is the subject): must be close-up 65-80%, wide shots are ineffective.
• Hard line: No fake text/readable alphanumeric characters/logo/watermark/UI; check all four edges equally (Codex self-check often misses fake text at edges; the main session must use ffmpeg to crop four edges and enlarge for independent re-verification).
• Identity reference image specified by user (0722-01 uses my-face-studio-default.JPG); expression follows narrative, use more silhouettes/looking down/reclining in low-energy segments to avoid mismatch, no smiles or sadness.
• After each batch generation, use md5 for deduplication to prevent file overlap.

④ Grok Animation Three Pitfalls (Detailed in Mistake #39):

• Resolution: The grok CLI image_to_video resolution_name only has two options: 480p/720p, no 1080p (confirmed 2026-07-24 real test; requesting 1080p silently falls back to 720p); the xAI API side grok-imagine-video-1.5 supports 1080p i2v (official docs). Full screen requires 1080p = CLI cannot do it: use xAI API 1.5 tier, or switch to real material or other generators (Sora 2 / Veo / Kling / Seedance; ChatCut MCP includes Seedance/Kling). The instruction must explicitly hardcode the resolution to get the full tier—otherwise it defaults to 736×400 (much blurrier). Is 720p enough depends on container mode: for cards/picture-in-picture (display area often <720px), 720p is more than enough and clear; full-screen filling 1920×1080 at 720p magnified 1.5× becomes soft. The final output is 1080p; B-roll does not need 4K (won't show unless heavily zoomed in).
• Morph proliferation: Elements prone to deformation (gears, rings, calendar squares) should have a locking constraint at the end of the prompt (keep individual original size/grid count and colors unchanged/ring continuous and unbroken).
• Hard filling: "Sparse elements + large empty background + hollow human figures + exposed face fragments" trigger the model to fill empty spaces/fill holes/generate faces, changing the composition rather than being blocked by prohibitions.
• Invocation: Headless mode needs --always-approve --permission-mode bypassPermissions --max-turns 10; multiple agents sharing cwd must disable grok --continue (would confuse sessions, writing material A's output to material B's path); before delivery, batch md5 deduplication.

⑤ Environment Setup: Codex projects must be created inside its writable working directory (local = ~/Documents/remotion剪辑/); creating under ~/hyperframes-projects/ will result in image_gen success but save failure; for multiple tasks, use one agent per task in parallel, don't let one agent process all tasks serially (it may declare completion halfway through).