Reference Video Replication
Status: v0.9. v0.5 verified the closed loop from material to finished product and delivery audit; v0.6 (2026-10-30) added the script rewrite branch: new voiceover track, AI voiceover tools, royalty-free material fill-ins, new graphics and subtitle styles, three samples passed audit; v0.7 (2026-10-30) integrated Volc Speech Synthesis 2.0, tool for re-timing after voice change, three subtitle styles differentiated by video, three revisions passed audit—voice pleasantness still requires human ears; v0.8 (2026-10-30) renderer supports 1080×1920 and low-res material enhancement, three 1080 versions passed audit. v0.9 (2026-11-18) audit supports host-camera main shots (a_roll). Business effectiveness and release status not yet verified.
Replication Boundaries
- First determine reference ownership: For the user's own channel's original footage, the original lines can be closely matched; for other channels (competitors, bloggers), default to replicating only the structure, pacing, and shot formulas, with the script rewritten according to the target narrative and without using the original audio—see the "Script Rewrite Branch". Follow the user's explicit choice; do not default on their behalf.
- The baseline is the user's currently confirmed reference footage; structure, order, sentence patterns, and pacing are preserved by default. The script is not rewritten to sound "better" or "more like an ad".
- Propose only the minimum necessary modifications in cases of factual errors, inapplicability of the target product/service, or when the user explicitly requests a change of target. Keep the original line and the reason for each modification item by item. Unverified does not mean non-existent; first mark as "pending confirmation", do not replace prematurely.
- Record transcription corrections and creative modifications separately. Brand/model names can be corrected based on on-screen visuals, but do not fill screen titles with unverified spoken content.
- Visuals can be replaced as long as they match the original line's meaning; try to preserve the original segment's structure and pacing. When the user allows paraphrased visuals, use topic-related usage scenarios, tool operation B-rolls to support the narration, without requiring a verbatim re-enactment for every word. Generic scene materials cannot prove the existence of a specific tool, platform, or service.
- The presence of a feature in the original video does not imply that the current promoted product/service necessarily includes it; keep "Original video statement", "Target service pending confirmation", and "Verified fact" separate.
- Do not add pain point openings, consultation call-to-actions, discounts, or effect promises without authorization. Whether the original video was viral or drove conversions requires corresponding data.
Workflow
- Obtain the original video path, media parameters, and hash. Check the complete time range, including intro, title, subtitles, voiceover, and outro. Original file is read-only.
- Extract frames for visual inspection, transcribe the audio track and verify; if tools are missing, use a locally runnable solution. When using machine transcription, retain the raw result, specify the model, approximate timestamps, and manual verification status. Words/phrases that are unclear or inconsistent with the model should be marked separately; do not fill in gaps with smooth guesses.
- Output per line: time, original line, evidence type & confidence status, kept or minimally modified, reason, original video shot, replacement shot formula, pending service items. Keep the original text layer immutable; list proposed changes separately.
- The shot formula should at minimum include subject, action, framing, and information that must be visible; separate original observation from replacement suggestions. Frame extraction gives only approximate times; precise edit points need re-localization.
- Summarize pending product/service items into a short checklist; do not stop completing the original video organization due to unknown facts. Unverified service lines can remain in drafts but cannot be marked as ready for publication.
- After confirming the original lines and necessary changes, proceed with matching existing materials, filling gaps, voiceover, and editing; record actual outputs and shortfalls at each step. Final check of subtitles, audio-visual correspondence, export stream, and duration.
Script Rewrite Branch: Structural Replication of Others' Videos
Use this branch when the user selects "keep structure, rewrite script". The original video only provides segment order, each segment's function, pacing, and shot formulas; all lines, audio, and visuals are replaced with the user's own.
- Break down each segment and write its "function", not a word-for-word rewrite of the original lines. New lines are written from scratch based on the target product's narrative; cannot be a few word swaps from the original. The original video's taglines and ending phrases are also rewritten. The machine transcription of the original text stays only in the analysis directory as evidence and is not used in the rewrite.
- Write only evidence-based service statements: details captured in the target brand's own account materials (e.g., a specific store, episode, or batch) can be written, but only prove that instance; otherwise, write as "illustrative" animations and add to pending confirmation. Do not transfer endorsements like years of history, qualifications, headcount, effect numbers, etc. from the original video.
- If the original video is a real-event on-location recap (headcount, location, scene) and you have no real visuals from the target brand's similar event, change to "process illustration": omit headcount, location, date, and scores; mark material visuals as "scene illustration"; add an end slate saying "refer to official announcements". "Visuals don't need to be real shots" does not mean you can fabricate an event.
- First-person user stories with effect numbers count as user testimonials: mark the effect card as "scene illustration"; before release, replace with real users and obtain authorization, or keep the "scene reproduction" label; do not fabricate result numbers.
- Markers like "[pending confirmation]" are only written in documents, not in on-screen subtitles.
- Voiceover:
scripts/tts_lines.py <script.json> <new_output_dir>, requires internet. Synthesize each line separately, remove leading/trailing silence, arrange sequentially into a timeline, outputvoice.wavandtimeline.json; then convert to AAC (m4a) for direct copying by the mixing tool. Default engine edge-tts (depends on edge-tts), commercial licensing unclear, only for discussion samples. When the user provides a Volc Speech Synthesis account, write"engine":"volc"in the script JSON and use seed-tts-2.0; voice and tone specifications are in references/fullscreen-mix.md. Volc's release permissions follow the user account's service terms; do not mark as authorized. - Credentials are stored only as runtime environment variables (VOLC_APP_ID, VOLC_TOKEN); not in script JSON, scripts, documentation, or commits; after use, inform the user that the chat log contains plain text and recommend rotation.
- The agent cannot judge voice pleasantness: choose a primary voice based on the video's audience, and for each video, generate 3–4 alternate samples of the same first few lines into
voice-samples-vN/; changing voice only requires re-running voiceover and re-timing. Use a different voice for each video; writetonefor speaker and context (effectiveness not compared by human ear). - After export, use
transcribe_local.pyto transcribe and compare against the script; if the recognition model's homophone errors and voiceover reading errors are indistinguishable, mark as "pending human ear verification", do not claim verified. The small model often mis-hears homophones and writes proper nouns as similar-sounding characters; such differences cannot be used as evidence of voiceover reading errors. - On-screen subtitles and segment timings are determined by the voiceover timeline: each segment starts about 0.1 seconds before the line begins and ends when the next line starts; remove punctuation from subtitle text, and break long sentences into two lines based on semantics.
- When changing voice, do not manually edit frame numbers:
scripts/retime_plan.py <old_plan> <timeline.json> <new_plan> --audio <new_track> --version <vN>re-times subtitles and cut points according to the new timeline; for live-shot segments, do not carry over the original source interval—this avoids dragging in the next shot: if the duration becomes shorter, trim the tail at original speed; only slow down if it becomes longer. For lines originally without voiceover and now adding voiceover to screen text, use--silent-lines 1,9to mark them only taking time without generating subtitles. First adjust the overallrateto roughly bring the total duration back to the original (easier than per-segment tweaking: +10 on the whole reduces a 31.0→33.9s video's 5th segment to 29.7s), then check the ratio printed by the script; for segments exceeding 1.25x, first extend source intervals by moving cut points outward; if the same line's first and second half match two different shots, split into two segments (tested: a one-line split into two action shots reduced a 1.75x ratio back to original speed); only accept slowdown as a last resort and ensure it is not slower than the previous version. - Adding voiceover to a music-only video: use the existing on-screen text as the script; read only what appears on screen for intro and end slate; do not add new phrases. Normalize voiceover loudness, lower background music by approximately 14 dB for backing, fade out at the end, and mix into one m4a for
audio. - For original videos without voiceover, use royalty-free music: Mixkit Stock Music Free License allows social media videos and online ads, but not TV broadcasts, CDs, games, or standalone remixes; trim to target duration, add fade in/out, and normalize loudness (-16 LUFS).
- The mixing plan uses the
audiofield to point to the new track; the renderer copies it instead of the original video's audio track; the audit also verifies against it; the original video visuals are disabled by default.
Filling Visuals: Royalty-Free and Douyin Materials
- First search through already-downloaded materials from the target brand's own account; use a 1-second contact sheet to find shots—vlogs, store visits, or event recordings often cover many real-world scenarios and match the script better than generic materials.
- For generic scenes (daily life, office, study, kitchen, etc.) use Pexels: free for commercial use, no attribution required; add
?orientation=portraitto the search page URL for vertical videos; download URL ishttps://www.pexels.com/download/video/<ID>/, redirects to the original file. First create a contact sheet with thumbnails with IDs to select, then download. Most subjects are foreign faces, suitable for shots that don't require identification. - Douyin's web search and account profile full video list require login; when not logged in, only known video links can be opened. Login must be done by the user via QR code; the agent does not enter credentials.
- After opening a video detail page, the media address loaded during normal playback can be read from the video element's
currentSrcor the performance resource list: some are single MP4, others have separate video and audio streams (media-video/media-audio). Download with Refererhttps://www.douyin.com/; for separate streams, use ffmpeg to losslessly merge (-c copy); blob addresses are not download-able results. Still no circumventing CAPTCHA or calling hidden APIs. - Log each material's source, ID, authorization type, and local path; material from brand accounts may be downloadable but not necessarily publishable—release authorization is a separate pending item.
Delivery Audit Is a Mandatory Step
The user should not act as the first quality check. Export success does not mean deliverable; first follow references/delivery-audit.md to execute preparation check, export, actual visual review, and final release. If issues are found, the agent should adjust framing, replace shots, or add materials on its own, then re-audit; reversible fixes do not require user re-consent. Only return the sample when the final audit.json for the current file shows delivery_allowed=true. Do not write "passed" if unresolved issues exist.
- Brands must be fully visible or fully avoided; no remnants of half characters, partial icons, or small orange text; besides top imprints, also check walls, screens, etc. for moving brands. After improving framing, re-check whether source subtitles were brought back.
- Different shots can be selected from the same source; the same source time interval cannot be reused. Same action/camera position belongs to the same shot family, even if crop, nearby in-point, or speed changes are applied; it is not a new shot. Each shot family appears once consecutively; continuous normal action progression is not considered reuse. Do not rename the same visuals to pass the audit.
- Exception: voiceover videos' host main shots (confirmed by user 2026-11-18). When the host is on camera, the visuals and the finished track are from the same moment (lip sync), the host shot can be interleaved with other shots repeatedly, but the source time can only go forward, not overlap; mark this shot family with
"a_roll":truein the log. Only for lip-sync host shots; B-roll, product shots cannot be marked this way. - First detect reuse using source interval overlap and logged shot families, then supplement with perceptually similar candidate detection and actual visual recheck. Cannot claim the algorithm can automatically understand all repeated compositions. If materials are insufficient, add materials or rearrange; do not loop fill to meet duration.
- Automatically checkable: source hash, crop vs. logged enclosure area, source subtitle exclusion zone, timeline, source segment overlap, shot family return, original audio track PCM consistency, and full decoding. Brand movement, unknown subtitles, subject completeness, visual naturalness, and client presentation feel must be visually reviewed by the agent from exported start/middle/end frames, and more if necessary—this is a visual review, not a frame-by-frame human inspection.
- The audit and visual review must be bound to the current plan and exported file hash. After changing plan, subtitles, framing, source files, or re-exporting, old conclusions are invalid; without actual visual evidence, unaudited shots, unprocessed similar candidates, or expired conclusions, block delivery.
- Final delivery includes: deliverable video, brief revision notes, and necessary limitations; audit details go with supporting files; internal test and failed files are kept but not pushed to the user as the latest version. Separately log product/service fact confirmation status from production quality pass/fail.
Web Download Branch (No API)
When the user specifies a Douyin URL and a download website, prioritize searching for the account in the browser, verifying authentication and works, obtaining visible video links, then submitting to the specified website for parsing and download—do not ask the user to paste links one by one. First verify one, then expand collection. The required number of candidates is determined by script shot gaps; do not blindly download entire accounts.
Candidate selection criteria are whether visuals match the original line; account name, video title, or high engagement are not substitutes for viewing. One video that seemed relevant by title was sampled and found to be mostly host talking, lacking needed shots, so it was not counted as matched material.
Log each record: source account, video ID, original link, original file & local file, download status, media check, usable segment time, and corresponding sentence number. Statuses are separate: candidate, parsed, downloaded, media check passed, shot matched. Parsing success or button click does not count as download complete; being playable does not count as shot matched.
Use temporary download links promptly as per page instructions; for example, one site indicates about 5 minutes expiration; download immediately after parsing each link. If encountering login/interactive CAPTCHA, save the breakpoint and hand it over to the user per browser rules; do not blindly retry. Automatic safety checks completing does not mean the human verification code is handled. After the user completes, read the page status; if still showing failure, do not assume passage.
If the download site is persistently stuck, check the original Douyin detail page for video elements that the user can normally see and play; use the browser's supported downloadMedia to save public media. In this case, the detail page showed actual MP4 links that downloaded successfully, whereas search result pop-ups could be blobs; do not treat blob addresses as downloaded videos, and do not bypass CAPTCHA to call hidden parsing APIs.
After downloading, check for non-empty file, audio/video streams, duration, and dimensions; perform full decoding and frame extraction for verification. The site's watermark removal capability does not confirm the material's scope of use. Both download paths were verified on 2026-10-21, but long-term stability or complete material readiness is not proven.
Per-Sentence Material Matching and Time Verification
- First extract frames at 1–2 second intervals covering the entire video to find candidate ranges; then re-seek near the candidate start/end and extract frames at at least 2 fps for verification. The first fps sample label may deviate from the actual shot by about one second; cannot be used as the edit point directly—this was why the start/end times of a headphone side shot were corrected.
- Before setting in/out points, measure cut points:
ffmpeg -ss A -to B -i src -vf "select='gt(scene,0.25)',showinfo" -an -f null -; the output time plus A gives the source time. Place the in-point at least 0.05 seconds after the cut point; the out-point at least 0.15 seconds before the next cut point; for slow-motion segments (source_span shorter than target duration), the read source frames will be about 0.1–0.2 seconds more than planned—three times on 2026-10-30, the next shot leaked in and was only caught during export review, so slow-motion segments need extra margin. - Log shot families based on the measured cut points; continuous long shots from the same camera position (e.g., walking from one zone to another) are one family and cannot be split into two names for non-adjacent segments. If adjusting the in-point exceeds the logged range, first supplement the log before auditing.
- Use local file time stamps for in-point and out-point, and distinguish from original video's line timing. Visual continuity, existing large subtitles, lip sync, clarity, and brand/city markers all affect usability—include in restrictions; do not treat "no watermark" on the download page as "no subtitles in the visual".
- Mark separately: visuals available, partial match, conditional candidate, no visuals. Related product names, generic scenes, or similar tools do not automatically prove the specific function in the original line; other brands' interfaces cannot impersonate the target product's functionality.
- Check if the candidate shot length covers the original line; multiple short shots can be combined; do not mark a 1-second screen shot as sufficient to cover an entire line. When multiple lines share the same material, inform about duplication risk; final editing must allocate.
- Keep original lines per sentence; even if partial visuals are available, list uncovered information; do not just give download counts. "Not found" is the conclusion of this round's search, not the product's non-existence.
- Original video is read-only preserved. Create a curated copy for user convenience, verified by hash; exclude materials and candidate retention reasons from curated set. Without explicit user request, do not install the skill globally or publish videos.
When User Allows Gaps to Remain from Original: Rough Cut First
After the user adopts "replaceable visuals are replaced; gaps keep original", proceed to the first version; do not wait for all shots to be complete. Prioritize keeping script, segment order, original audio track, and total duration; if material exists but has irrelevant large subtitles, poor quality, or mouth interference, keeping the original and logging the reason is allowed—do not sacrifice content match for replacement rate.
Build a continuous per-frame timeline based on the source video's actual frame rate. For each segment, clearly define: original video target frame range, source file, source in-point, source duration, framing, and subtitles. Retained segments use the same position as the original; replaced segments only process visuals; necessary minor slowdowns are recorded separately. Original audio from material is not mixed into the final track.
If the source video has subtitles, first check the actual visuals; crop to avoid irrelevant text, protect the subject and brand; do not assume a fixed percentage will cleanly crop everything. If subtitles need to be reordered, follow the original video's on-screen text; do not turn machine guesses into new voiceover. When the user requests full screen, select the subject, crop to the target aspect ratio and fill the frame; do not use blur extension. If the subject cannot be preserved or subtitles cannot be avoided, change shots or use illustrative animations; do not zoom in until semantic loss occurs.
Before export, keep the original file hash; after export, verify the source file unchanged, copied audio track's decoded PCM hash, continuous frame count, audio/video streams, and full decoding. After re-encoding retained segments, cannot claim pixel or byte identity; can perform SSIM check on corresponding frames and explain the range. Visual sampling cannot be claimed as frame-by-frame acceptance. If original subtitles appear, correct framing, re-export, and protect first-round process files.
Deliver a complete playable rough cut, replacement/retention records, and still-pending items, then refine based on user feedback. When the audio track is directly carried over, there is no need to wait for all machine transcription name readings to be confirmed; do not re-record pending names.
When Visual Difference Is Needed: Mix of Live Shots and Self-Created Animations
When the user finds too many original visuals retained and allows creating images with animations, keep confirmed script, audio, paragraphs, and timing, and redesign visuals. Separately record duration of new live shots, self-created graphics, and retained original visuals; do not count re-encoding or simple cropping as new content.
Purpose affects visual design: if the user says it's for client pitching and requires natural trust, prioritize real usage scenarios and product shots; keep self-created parts to restrained animations like keyword emphasis, simple highlights. Do not automatically add cartoons, synthetic characters, lengthy concept flowcharts, or fictitious interfaces just because the user once mentioned generating images. If necessary, a small number of real product screens from the original can be kept and logged; do not always pursue zero original. Internal test versions are not delivered as the latest client presentation.
- Use real users, scenarios, and tool details to carry characters and actions; use self-created keyword cards, soundwave archives, flow records, sentence highlights to explain abstract processes. Alternate between the two; avoid continuous long stacks of static cards.
- Simple icons, flowcharts, and text analysis are suitable for code-native drawing with animations; only consider image generation when true photo-style illustrations are needed. Do not automatically interpret "illustrations" as generated photos.
- Clearly mark self-created graphics as "function illustration" and note in source records as non-actual interfaces. Do not fabricate effect numbers, user reviews, actual reports, or expert identities; keep source video statements separate from target product confirmation status. Unconfirmed items cannot be disguised as confirmed through visuals.
- The original video retention mode and this branch are different version choices. When the user explicitly requests visual difference, do not automatically fill in large amounts of original footage; for missing shots, list and use authorized illustration solutions. Before claiming no direct reference to original video visuals, check the timeline source; cannot claim the platform will necessarily recognize it as original.
- Only one set of subtitles is kept; do not mix source subtitles with new subtitles; timecodes are unified across the whole video. Do not use the same style for all videos: batch videos each use a different style; within the same video, vary styles for keywords and sentence patterns based on content (user feedback on 2026-10-30: "don't be so uniform"), but only change style, not words. Framing must avoid source subtitles; review start, middle, and end frames of each segment, not just one center frame. Subtitle text corrections are listed separately with evidence; do not re-record or modify the audio track.
- Full-screen cropping maintains the target aspect ratio; no stretching. Use real images to cover the entire frame; do not use blur filling. If only to increase subtitle contrast, a slight brightness gradient is acceptable, but do not add new frame borders.
Full-Screen Mixing Tool
scripts/render_fullscreen_mix.py <edit-plan.json> [--preview], depends on ffmpeg, ffprobe, Python, and Pillow. Output dimensions are determined by the plan's width/height (must be 9:16); if not specified, 576×1024; delivery versions always use 1080×1920 and set "enhance":true—do not wait for the user to complain about quality (2026-10-30, three samples were first rendered at 576 before being flagged). Graphics and subtitles are redrawn at output resolution; layout coordinates do not need changing. Example schema and sample paths in references/fullscreen-mix.md.
Check quality when selecting material: calculate magnification per segment (output width ÷ crop width); preferentially replace segments exceeding 2x or crop less. enhance only applies mild denoising and sharpening to magnified segments; cannot salvage low-bitrate material: a 720p/714kbps Douyin download upscaled to 1080 is still noticeably softer than 4K source—to truly fix, need original file. AI upscaling is not verified in this project; it risks distortion on real people's faces and hands and requires user consent.
Supports two segment types: footage and graphic. Live shots use integer 9:16 cropping; graphics natively draw soundwaves, keyword cards, concept flows, and sentence analysis; v0.6 added score cards, icon cards, large keywords, and brand text end slates. Live shots can use accent.labels for lightweight keyword emphasis; also used when material visuals need a "scene illustration" label. By default, original video visual paths are disabled; when explicitly retaining necessary real screens, set allow_reference_pictures=true in the plan and log reason and statistics. Global subtitles are burned in only once (caption_style options: default bottom white, box yellow background black text, v0.7 added pop, band, paper—see references); finally copy audio track: default original track; for script rewrite, copy plan's audio. --preview generates representative frames and contact sheet only; after formal export, still recheck each segment's edge frames and visual motion. Output must be a new file; changing plan requires a new work directory. Even if renderer code changes but plan doesn't, increment version and re-export: audit is bound to plan and output hash, not renderer code. Formal execution checks continuous frames, source hash, decoded audio track consistency, and full file decoding; tool passing does not equal product fact or automatic visual acceptance.
Rough Cut Tool
scripts/render_picture_replacement.py <edit-plan.json>. Depends on ffmpeg, ffprobe, Python, Pillow, and a readable Chinese font. Plan includes reference, width, height, fps, total_frames, font, work, output, and segments; each segment includes id, start_frame, end_frame, path, mode, source_in, source_span. Replacement segments also require normalized roi=[x, y, width, height] and caption, with optional font_size, caption_y. Only local video processing, no external API.
First use --only <segment-id> to output representative frames and check framing; after confirmation, output full. Use independent version directories for work; output must not exist. Cache is identified by input source hash and configuration; configuration changes require a new directory; do not reuse old visuals based solely on same frame count. After encoding, copy the reference audio track; output verification.json verifying decoded audio track consistency, source file hash, and frame count.
Material Tools
- AI Voiceover:
scripts/tts_lines.py—see "Script Rewrite Branch". - Voiceover Change and Re-timing:
scripts/retime_plan.py—see "Script Rewrite Branch". - Audio Transcription:
scripts/transcribe_local.py. Depends on Python and faster-whisper; first-time model loading requires network download. Outputs machine-recognized original text JSON, not a post-acceptance verbatim transcript; does not automatically standardize brand names. - Media Check/Preview:
scripts/scan_media.py <video> <new-output-dir> [--step 1]. Depends on ffmpeg, ffprobe, Python, and Pillow. Outputs media parameters, hash, preview images with approximate timestamps; output directory must not exist to protect old evidence. It does not automatically judge semantic match. - Material List Validation:
scripts/validate_material_map.py <material-delivery.json>, verifies files, hashes, segment boundaries, sentence numbers, and shot references. - Minimum Delivery: original video folder, per-sentence match and gap list, source/video ID/file/time range correspondence, pending service items; provide curated copy and portable skill archive if needed.
- Carry unresolved listening items, unconfirmed services, material gaps, and actual verification status to the next round; do not write "document complete" as "video replication complete".