AI Product Video

What does AI Product Video do?

Core Division of Labor:

• 3D or Product Photos: Determines where the product is, its size, how it moves, and when it changes.
• Image Model: Makes the image look realistic.
• Video Model: Makes the image move.
• Code: Uses product facts and 3D as a ruler to measure every image and video from the AI; if it doesn't measure up, don't spend the money.

Editing, subtitles, callouts, and sound are data-driven overlay layers; changing a single word does not require regenerating a single frame.

This methodology comes from a real project: a demonstration video for an investor presentation of a smart hardware product. The route was changed four times before it worked. An anonymous post-mortem is in examples/case-study.md. What follows is the general version that worked, applicable to explainer videos and ads.

Why You Can't Just Copy the AI Comic/Animation Pipeline

The AI comic/animation pipeline is: concept → storyboard → asset library → lock still frames → video model makes the frames move → voiceover and editing. There's no 3D model, and the audience tolerates detail inaccuracies, so the AI-generated image is the ground truth.

Product videos sell the idea: "This is what it looks like, and this is how you use it." If an interface orientation, label, or light position is wrong, the video is lying. So:

• Borrow the skeleton: Assets first, still then moving, choose modes per shot, only write changes in prompts, test at low resolution, QC each shot individually.
• Change only one thing: The ground truth is not an AI image, but product facts + 3D (or product photos/CAD models).

Another key insight: Do not use 3D render pixels directly in the final video. No matter how detailed, they will always have a CG look. Have the image model repaint over the 3D base image, then use that image as the first and last frame of the video to achieve realism.

Step 1: Project Initiation – Ask These 8 Questions First

Details and phrasing for the user can be found in references/01-intake.md.

1. Time Estimate: In the first reply, state approximately how many minutes or days, and by what time, the project will be finished. List it by phase, including steps the user needs to do. Update immediately if there is a delay.
2. Video Type and Distribution: Demo, brand ad, short vertical video, e-commerce video, or launch/pitch video? Where will it be played? What is the target duration and aspect ratio? Choose a strategy based on references/02-formats.md; one production run can produce multiple delivery versions.
3. Audience is Internal or Public: Internal audiences and investors can see exploded views, cross-sections, and internal structures. Publicly released versions must not show any internal structure or name internal components. This determines which shots are possible.
4. Product Fact Confirmation Sheet (templates/product_facts.md): For each feature, what part makes it work? Where does it come from on the product? What does it look like in a shot? What must it NOT look like? Is it a design goal or already implemented? Can it be shown publicly? Any items you infer must be marked "Pending Confirmation" until the responsible person confirms.
5. Quality Benchmark: Ask the user to provide a reference video. Measure its motion level and production method. Learn the visual quality, not the structure.
6. User's Available Tools:
   • Image model: Is it a subscription? Does the user need to generate images manually, or is there an API for scripts?
   • Video model: API account (scriptable) or web/app membership (manual generation only: use manual video packs, check commercial terms first, don't assume drafts can be upgraded)?
   • Does the user have a 3D model or CAD of the product? If not, is there a physical sample or any render? Without 3D, use the "No 3D Route" (references/03-previs.md section 10).
7. Budget and Billing: Pricing formula, packages, how much coupons actually cover, account balance, total cap. Before large-scale generation or HD upgrades, always ask the user to check their balance first.
8. Test Before Deciding the Route: Generate two low-resolution drafts of the same shot. One uses the 3D or product photo as the first frame, the other uses a realistic keyframe from the image model. This costs a few dollars and a few minutes and shows the difference clearly.

Project Folder Convention

All scripts use --project as the root folder (default is the current directory). New projects should be structured as follows:

<project>/
  .env                    Keys (ARK_API_KEY, etc.), do not commit, do not show in chat
  brief/                  creative_brief.md, product_facts.md, benchmark.md, budget.md, decisions.md, feedback_brief.md, claims.md, licenses.md
  shots/shot_list.json    Shot list, the single source of truth for the timeline
  shots/prompts/<id>.txt  Video prompt for each shot (add version number when changing, keep old versions)
  plates/<id>/            3D bases, masks, camera coordinates (for no-3D projects, put product reference images here)
  packs/                  look_bible.txt (unified settings, single source of truth), manual image packs, manual video packs
  keyframes/keyframes.json   Keyframe manifest (masters, first/last frames, upload order)
  keyframes/raw/          Raw images from the image model
  keyframes/norm/         Keyframes resized to output dimensions
  footage/                Real footage, screen recordings, real UI screenshots, etc. (optional)
  gen/                    Video model output, and video_log.jsonl (invocation and cost ledger)
  qc/                     Frame-by-frame mosaic, Gate K / Gate V reports
  edit/                   edl.json, captions.json, callouts.json, approved_drafts.json, approved_bridges.json
  renders/                Silent video from the editor and .timeline.json (default output location for edit.py)
  sound/                  Music and sound effect tracks, cue sheet, mixing sessions
  deliver/                Final videos (public and internal versions in separate folders)
  work/                   Project-specific scripts and workflows (EDL generation, image packs, checks), write each step's result to disk

Copy templates from templates/ into the corresponding folders: creative_brief.md, product_facts.md, benchmark.md, budget.md, feedback_brief.md, claims.md, licenses.md go into brief/; decision_list.md is saved as brief/decisions.md; shot_list.json goes into shots/; keyframes.example.json is saved as keyframes/keyframes.json; the filled-in prompts/look_bible.md code block is saved as packs/look_bible.txt; sfx_cue_sheet.md goes into sound/; the five *.example.json files (edl, captions, callouts, approved_drafts, approved_bridges) are renamed by removing .example and placed in edit/.

Write the aspect ratio in the data, not by remembering commands:

• Video generation: The aspect ratio of first/last frame tasks follows the first frame (first use scripts/qc/normalise_keyframes.py --size 1080x1920 to normalize keyframes); for reference mode, add --aspect 9:16 for vertical.
• Editing: Write "size": [1080, 1920] in edit/edl.json, then uv run scripts/edl_editor/edit.py --project <project> --edl edit/edl.json --dry will arrange it vertically; --res 480 will output a small preview at the same aspect ratio; --size WxH outputs a different aspect ratio.
• Vertical feed: In edit/captions.json, set "style": {"preset": "feed_safe"} to keep subtitles and end credits within a conservative safe area. Use --dry to check each item.

git init on day 1, and add .env, gen/, renders/, .cache/ to .gitignore.

Each type of data recognizes only one file as its single source of truth:

• Product appearance: spec file or 3D master; unified settings: packs/look_bible.txt;
• Timeline: shots/shot_list.json;
• Subtitles: edit/captions.json;
• Editing: edit/edl.json;
• Approved shots: edit/approved_drafts.json, edit/approved_bridges.json;
• Claims and licenses: brief/claims.md, brief/licenses.md;
• User feedback: brief/feedback_brief.md.

Pipeline

Two tracks run in parallel: the visual track and the sound track. Don't wait for video to be upgraded to HD before starting on sound; start as soon as the editing structure is locked.

#	Phase	Output	User Check	Automated Check/Record	Details
1	Initiation & Facts	Creative brief, product fact confirmation sheet, delivery checklist	Confirm facts, video type, budget	—	01, 02
2	Creative & Shot List	Structure, beats, shot list (per shot: camera position, action, effect, product state, mode, transition, subtitle, sound effect)	Lock structure and wording direction	Check if duration and reading time fit	02, rulebook §1
3	Pre-vis & Base Images	3D master, base image contract for each keyframe, grey-model video when needed; product reference images for no-3D projects	Review base image overview	Check orientation, cables, clipping from each real camera position	03
4	Master Settings	Product master, scene master, character design (assets first, before all shot keyframes)	Review each against checkpoints	—	04 §2, §3
5	Keyframes	One image pack per keyframe, or scripted API calls; resize to uniform size	Generate manually or review images	Gate K (project script): outline, proportion, effect state, scale consistency, duplicate files	04
6	Video Drafts	480p drafts	User clicks to activate and pay; check balance before batch; review drafts	Rehearsal & rejection conditions (including prompt check); frame-by-frame mosaic every 0.25s; Gate V (project script)	05
7	Review & Ledger	edit/approved_drafts.json, edit/approved_bridges.json, rough cut	Provide feedback, numbered and placed in brief	Manual approval binds file fingerprint	05 §13
8	Editing & Subtitles	EDL preview: transitions, AI transition shots, subtitles, callouts, intro/outro cards	Change wording and order	--dry check spacing and reading time; subtitle still frame review	06
9	HD Upgrade	Approved shots, unchanged, upgraded to 1080p	Approve after finalizing order and subtitles; check balance first	Check duration and intermediate frame consistency	05 §6
S1	Sound Design (parallel with 8 & 9)	Music selection and licensing, sound effect cue sheet, product-specific sounds, voiceover when needed	Select music (send playable files), audition transitions	License records	07
S2	Mixing	Balanced final audio track	Listen to the final product	Measure dB difference "SFX minus music" by time window; loudness target	07
10	Delivery	Master, shareable version under 30MB, various platform aspect ratios, 15/6-second cut-downs	Finalize intro text, request final video	Duration, video/audio streams, loudness, keyframe visuals	08
11	Multi-language	One copy file generates parallel subtitles and EDL	Finalize copy	Original version remains pixel-perfect; zero residual characters from original language	08
12	Compliance Final Check	brief/claims.md, brief/licenses.md, AI identifiers	Sign off	Glossary check + line-by-line comparison with product facts	09

The 12 Most Important Rules

For the full 139 rules, see references/rulebook.md.

1. Write the intake form first: Video type, audience (internal or public), platform, duration, aspect ratio – then talk about creative ideas.
2. Product facts are documented and confirmed first; one correction means updating all downstream items once (3D, base images, image prompts, video prompts, subtitles, callouts, brief), then do a global search for leftover terms.
3. 3D only sets position; let the AI draw the final pixels; for every shot, ask "Where do the final pixels come from?"
4. AI keyframes must drive shots as the first and last frame, not be just one image among many in a reference set.
5. Every pixel fed to the video model will be faithfully copied: use a uniform grey for grey-models and denoise; don't put images with different camera angles in the reference pack; the reference images themselves must be correct.
6. Video prompts should be a second-by-second script; numbers should come from the 3D or shot list, not handwritten; only describe changes, don't repeat the appearance description.
7. Describe effects as they would be captured by a camera, and state what they are NOT; a constantly lit light should say "constant brightness"; use visually verifiable standards to lock down quantities.
8. Spend money in two stages: pass free checks first, generate a 480p draft; only after approval, upgrade to HD exactly as is; check the balance before upgrading.
9. Review original footage frame by frame: the first/last frame mode can secretly crossfade in the middle; independent review must look at the generated original video.
10. Subtitles, callouts, and sound are all in the editing layer, generated from timeline data; changing text, reordering, or upgrading to HD does not require regenerating frames.
11. Two tiers of compliance: first, check against a glossary (prohibited terms, efficacy, medical, numbers); second, compare line by line against product facts to catch "implied capabilities." The default for AI identification is to mark it once on screen at the beginning (for at least ~2 seconds) and explain it in small text at the end, plus the platform's required AI disclaimer; do not add a permanent corner tag unless the platform requires it.
12. Number each piece of user feedback by original wording and write it into the brief; all agents and the final reviewer must refer to it. Long conversations get compressed; files do not.

Collaborating with the User

• Time: Give an estimate first; for long tasks with no visible output, actively report the current phase, the time of the last activity, and the new expected time.
• When it's the user's turn, deliver everything at once: a complete set of documentation files, materials they can directly open or play, sent to their phone (individual files under 30 MB). Tell the user which folder to put master files in.
• Understand their working style: If they say "just analyze," only analyze. If they say "don't stop in the middle," go all the way to delivery. If they say "final round," skip intermediate auditions.
• Conclusion first: If the user asks "Do we need to scrap everything and start over?", state the conclusion in the first sentence, then explain the reasons, and finally list the decisions you need from the user.
• Safety:
  • Do not read, display, or ask for keys (only put them in .env);
  • Do not register accounts, enter passwords, or payment information;
  • For agreement to terms, payment, activation, top-ups: stop before the button, explain the cost clearly, and let the user click themselves;
  • Ask before downloading files.
• Long Tasks:
  • Before starting, ask the user to plug in their power source and set system sleep to "Never," then request to keep the system awake;
  • Workflows and batch processing must be resumable from disk;
  • Separate submission and result retrieval into two steps;
  • Only run one 3D rendering process at a time.

Details in references/10-working-with-user.md.

Public Release

For anything public, including ad placements, social media platforms, and public presentation visuals:

• Do not show exploded views, cross-sections, or shots revealing internal structures; do not name internal components or working principles;
• Do not name specific AI products in social media posts or images (write "image LLM" or "video model" or "3D software");
• Do not disclose costs;
• Describe pitfalls as general process issues, not personal experiences;
• Add any required AI generation disclaimers for the platform.

See references/09-compliance.md for details.

Tools

All scripts are run with uv run. Dependencies are declared in PEP 723 blocks at the top of each script. ffmpeg comes from imageio-ffmpeg.

Script	Use
scripts/edl_editor/edit.py	Assemble video from EDL: cards, crossfades, black, light flares, whip pans, AI transition shots, subtitles, callouts, corner logos; supports 16:9, 9:16, 1:1; --dry for rehearsal, outputs timeline.json. Use make_demo.py for one-click demo project generation
scripts/edl_editor/typeset.py	Subtitle and card typography engine: hierarchy for Chinese/English, punctuation hanging, safe zones, cross-platform font fallback; --specimen to check local fonts
scripts/video_api/seedance.py	Volcano Ark Seedance client: first/last frame, reference image, and other modes; draft and HD upgrade; resumable queries; cost estimation and ledger; --dry for offline checks only (including prompt checks); can be used as a template for other video model adapters
scripts/audio/mix.py	Balance music and sound effect tracks by time window, measure dB difference, limiter and double loudness normalization, then composite into the video
scripts/qc/strip.py	Extract one frame every 0.25s and stitch into a long strip, plot motion and brightness curves, mark hard cuts, jumps, and holds
scripts/qc/contact_sheet.py	Grid of labeled images for reviewing base images, keyframes, and drafts
scripts/qc/normalise_keyframes.py	Normalize keyframes from raw to norm: crop shot frames to output aspect ratio; only resize (not crop) master and reference images; warn about enlargement and hard cropping
scripts/qc/lint_prompts.py	Scan video prompts for forbidden term families (including product mechanism forbidden terms); errors if no file is found

Scripts not included, which must be written by the project in work/: Gate K and Gate V decisions (seedance.py does not read the Gate K report; the project's own submission step ensures "no report, no submit"), image pack generation (keyframes.json → packs/), shot_list.json → EDL and subtitle draft. Requirements are in 04, 05, 06 and templates/prompts/keyframe_pack.md.

See each directory's README for usage instructions. Model capabilities, prices, and limitations change rapidly; always check the official documentation before using, and note the verification date at the end of each reference file's "Model Notes" section.

When to Read Which File

File	When to Read
references/01-intake.md	First day of a new project
references/02-formats.md	When deciding video type, duration, aspect ratio, and structure, or when creating ads and short vertical videos
references/03-previs.md	Before setting up 3D previsualization or base images; when there is no 3D model
references/04-keyframes.md	When creating image packs, defining unified settings, or reviewing keyframes
references/05-video-gen.md	When writing video prompts, choosing generation modes, submitting, reviewing drafts, or upgrading to HD
references/06-edit-type.md	Editing, transitions, subtitles, callouts, intro/outro cards
references/07-sound.md	Music selection, sound effects, voiceover, mixing
references/08-delivery.md	Delivery, platform-specific versions, cut-down versions, multi-language
references/09-compliance.md	Before writing any on-screen text or voiceover; before any public release
references/10-working-with-user.md	Collaborating with non-technical users, long tasks, safety boundaries, environment issues
references/rulebook.md	When uncertain about a specific phase, check the corresponding section
references/pitfalls.md	When something goes wrong, look up the symptom first
templates/	At project start, copy into the project, fill out, and use
examples/case-study.md	When you want to see how a complete project unfolded