kinetic-typography-video

Kinetic Typography Video

A visual language + zero-dependency frame-by-frame renderer. Every frame is a pure function of time: visuals, subtitles, sound effects, and music share a single timeline, with no manual keyframing. It is distilled from two viral kinetic typography videos (see breakdown in references/viral-laws.md) and has been validated through a complete video in tasks/opus-kinetic-20261001/ (see examples/opus-kinetic/).

Five Laws (Apply to Every Shot)

1. One Image per Semantic Beat: Give each semantic beat a concrete object the audience can recognize at a glance—content creators will spot selection boxes, waveforms, keyframes, timelines, render bars, mouth shapes, and prompt boxes. Make the motion interesting: cross out "none," knock over "overthrow," and literally hide an object behind a person.
2. On-Beat Snapping: Large text lands exactly on the syllable where it's spoken (at('word')); pure visual scene cuts land on music beats (beat(k)); every visible action has a corresponding sound effect, which only responds to visible actions.
3. One Light Thread Through: A single light runs through the entire video—starting as an ignited scan line, becoming a selection line, a waveform baseline, or a visual effect moving through the scene, and ending as a single point that fades out after the final title. Transform rather than cut wherever possible.
4. Tension and Release: Dense sections must be followed by pauses (a full silent beat, a small text on bone-white background); a continuously changing number builds suspense (in this film: "Manual keyframes: 0").
5. Visual Quality: Strict four-color palette, sub-frame motion blur, glow effects, chromatic dispersion pulses, cluster particles; small tool-mark annotations invite the viewer to pause.

Technical Specifications

Item	Specification
Palette	Night #0A0A0D · Bone White #F1EDE4 · Signal Orange #FF5A1F · Cobalt Blue #3448FF (for "machine/data") · Gray #8B8983 · Graphite #26262C
Fonts	Noto Sans SC (Chinese, 900 for heavy text / 500 for body) · Archivo VF (Latin and numeric display) · JetBrains Mono (annotations) · Songti only for pause beats
Minimum Font Size (1920×1080)	Annotations ≥28px, explanations ≥36px, large text ≥76px; must remain legible when scaled to 390px width
Eight Actions → Sound Effects	Ignite: click · Pull line: zip · Scan: scan · Selection: blip/select · Text impact: impact/hit · Decode: tick · Suck in: suck · Shockwave: drop
Character States	Full frame (W 1.0 / M ~1.2 / C ~1.4 jump cut push/pull) · Selection drag into window · Corner small window (bottom right) · Chroma key into scene (text behind person)
Subtitles	One natural clause per card, no punctuation, model joined (e.g., Opus5.5); single-layer night solid background, 8px rounded corners, 46px; unread gray → read bone white continuous color sweep, with signal orange light ahead of sweep front
Safety	Any text/graphic in front of a person must maintain ≥30px distance from the actual head bounding box (including hat); text behind the person (under chroma key layer) is unrestricted. Verify with qa_face.py, not by eye

Pipeline (Command Details in references/pipeline.md)

1. Script → Beat Table: Write natural spoken script first, then select an object for each semantic beat (see references/devices.md). Do not modify the spoken script for visual effects.
2. Voiceover and Character: Generate per project workflow (Fish Audio → HeyGen, etc.). For live footage, use the original video directly.
3. Alignment: tools/align_script.py — script-first, ASR for timing only; CJK global alignment + silence-aware interpolation (ASR may pull the first character of a sentence into the previous pause, and treat the dot in 5.5 as a period).
4. Editing: tools/build_edit.py — overall speed change (syncs video and audio, lip sync unaffected), compresses long breaths into jump cuts, inserts pure visual sections, outputs a single timeline.json (all downstream tools use this).
5. Character Data: Extract frames → tools/vision_analyze.swift (Vision person mask + face/mouth bounding boxes) → tools/prep_media.py mattes|faces|thumbs|barcode.
6. Writing the Film: A film.js module (see examples). Each element anchored with at('word') or beat(k); use vocabulary from engine/devices.js; ev(t, 'sfx') registers sound effects and post-processing pulses at the same point.
7. Validation: render.mjs --mode stills for frame extraction and visual review → --mode qa + qa_face.py (zero violations required to proceed).
8. Audio: render_dialog.py (match loudness across sources) → --mode cues → sfx.py → mix.py (cut music at beat points, lower voice, raise voice when absent, silent beats, static gain + true peak limiting to target loudness).
9. Final Render: render.mjs --mode frames for lossless master segments → single H.264 encode → project mechanical check.

How to Use with Live Footage

Same method: chroma key the person, let large text pass behind them, turn numbers from the spoken script into visible counters/progress bars/circles, add push/pull on jump cuts, match overall loudness of original audio. The 10-second example is a 4K live footage clip (see references/pipeline.md).

Cover Art (3:4 / 4:3)

Generated by the same engine as still frames, with the same look and feel (see commands in references/pipeline.md, example in examples/opus-kinetic/cover.js):

• Character: Pick one frame from the digital human source—full face, open eyes, natural smile. First filter with faces.jsonl using mouth bounding box aspect ratio and face orientation, then visually confirm. Lossless frame extraction → vision_analyze … 1.0 (full-resolution mask) → prep_media.py cutout (separate wall color from hair edges and hat brims to avoid white edges on dark backgrounds).
• Layout: Title ≥150 px (at 1080 width), must remain legible when scaled to 180 px width; final text labels hidden behind the character (3:4 uses magazine cover style, with hat top overlapping the lower edge of text); a single light dot at the end of the title sentence. Add at most 2–3 small objects from the film (selection handles, layer label, crossed-out REC, a statistics line). Do not let text and hat brim touch: either clearly overlap or leave ≥40 px.
• Safety and Platform: --mode qa + qa_face.py must have zero violations; the bottom ~10% of the frame is reserved for platform play counts and duration labels.
• The first line of the module calls setSize(w, h); layout numbers are positioned relative to the face center; changing a frame does not require code changes.

Known Pitfalls (Don't Step Here Again)

• ASR may miss the first few characters of a clip: add 1 second of silence before sending to ASR, then subtract it back.
• Dots in version numbers (e.g., 5.5) are not punctuation; alignment and subtitles must treat them as decimal points.
• ffmpeg -ss … -c copy seeks to the previous keyframe, causing inaccurate clip start points; for precise starts, decode frames/audio and verify with cross-correlation.
• Frame cache recycles ImageBitmap: long-used images (thumbnails, point cloud sources) must be held separately with loadImage.
• TTS voice peaks are very sharp (PLR up to 24 dB); increasing volume directly will cause clipping: apply speech-style dynamic loudness normalization (loudnorm with true peak limit) before mixing.
• zsh does not split variables by spaces: use bash scripts for render segment commands.
• Dynamic loudnorm causes voice/music balance to "breathe"; master only with static gain + limiting.
• After switching to full WAV music, content above 19 kHz (including limiter-generated harmonics) can cause the AAC-encoded true peak to be about 1 dB higher than the master: set loudness.lowpass: 19000 in mix.json at the end of the master chain to still meet the limit after encoding.
• Before replacing the preview version with the full version, verify alignment with cross-correlation (Epidemic preview and WAV are sample-aligned), then fine-tune fullDb based on the loudness difference between the two; the voice-to-music ratio will match the reviewed version.