syydaniel/kazi

kazi 是做什么的？

Beat-sync videos lock to the drum; explainer videos lock to every word. An animation-writing AI can't hear audio, so it can only guess where the viewer's eyes should be at any second.

Kazi flips the order: audio first, then visuals, and visuals always follow the audio. It calculates exactly which second each word is spoken, then looks up the table to know which frame each action should land on.

This document is for you (the AI). Each step's details are in docs/; read each step's doc only when you reach that step—no need to read everything at once.

The tools/, starter/, and docs/ folders mentioned below are all located in this skill's own directory (same level as this file); substitute the full path when using them.

Six Steps, Four Pauses

Step	What to Do	Output	Details
1 Write Script	Write as you speak, one keyword per sentence, separate version for dubbing and subtitles	script.txt	docs/01_WriteScript.md
⏸ Pause 1	Show the script to the user, wait for their approval
2 Voiceover	Generate several takes of each sentence, another AI listens back and scores them, keep the best	vo/lines/	docs/02_Voiceover.md
⏸ Pause 2	Let the user listen to the full voiceover
3 Align	Alignment model marks start and end times for each character	timeline.json	docs/03_Align.md
4 Visuals	Draw verbs; animation lands 3 frames before the first character of the keyword	Code + Render	docs/04_Visuals.md
⏸ Pause 3	Render a 20-30 second sample for user review	out/sample.mp4
5 Sound	Music ducks under narration; sound effects are cued from animation timestamps	Mixed audio track	docs/05_Sound.md
6 Review & Publish	Measure frame-sync rate, black frames, flash frames, loudness; prepare title, cover, and description	out/final.mp4 and inspection report	docs/06_ReviewAndPublish.md
⏸ Pause 4	Deliver final video and inspection report together to the user

If the user explicitly says "don't pause, just finish it", you may skip the pauses, but step 6's inspection must still run, and results must be included in the final response.

If the user already has a recorded narration, skip step 2 and start from step 3 (timeline.py --whole).

Before Starting

1. Run python tools/doctor.py to check for missing dependencies. Dubbing and alignment are currently tested only on Apple Silicon Macs; for other platforms, follow the workarounds it provides and inform the user.
2. Ask (only if the user hasn't already told you): what is the topic, who is the audience, how long should it be, landscape or portrait, and does the user have a voice sample (about ten seconds of clean recording with a word-for-word transcript).
3. Create the project directory structure as shown below. Keep all intermediate files; save each version of the sample separately and do not overwrite.

project/
  script.txt             Script (format in docs/01)
  vo/                    Voiceover: lines/ one wav per sentence, takes/ all takes
  timeline.json          Start and end times for each sentence and each character
  film/                  Visuals project (default uses starter/ Remotion template)
  audio/                 Music, sound effects, mix
  out/                   Samples, final video, inspection report

Key Rules for Each Step

Write Script (details in docs/01)

• Write as you speak. No more than 22 characters between two punctuation marks; mix long and short sentences. Ears only hear it once.
• Mark exactly one keyword per sentence: *keyword*. The visuals will land on this word. At most one keyword per subtitle.
• When pronunciation differs from the subtitle text, write {pronunciation=>subtitle}, e.g., {F F M peg=>FFmpeg}.
• Write complete sentences: do not omit subjects, objects, or measure words; verbs should be at least two characters long, alternate single and multi-character ones; keywords should not be single characters. Determine pauses by meaning, see the pause table in docs/01.
• Replace easily mispronounced polyphone characters during script writing (docs/02 has a substitution table).
• After writing, run python tools/lint_script.py script.txt; it checks for overly long breath phrases, overly long subtitles, multiple keywords per sentence, and estimates duration.

Voiceover

• python tools/voice.py script.txt vo --ref voice_sample.wav: generate 4 takes per sentence, transcribe each via speech recognition and compare to the original script; discard mispronounced ones and keep the best.
• Polyphone characters and words that might sound like others will show up in recognition results; rewrite the sentence or adjust pronunciation rather than keeping a problematic take.
• To speed up the entire track use tools/tempo.py (change speed without changing pitch); don't regenerate.
• python tools/pace.py vo_fast vo_paced: normalize pauses. Measure silences at sentence start/end and around punctuation, pad short ones and trim long ones, only modify silence not sound.
• tools/polyphone.py picks out potentially mispronounced polyphone characters and compiles them into a test audio clip; it may produce false positives and false negatives, final check relies on human listening.

Align

• python tools/timeline.py script.txt vo_paced timeline.json --wav vo_paced/narration.wav --speech-gap: produces start/end times for each sentence and each character, plus a keyword list. --speech-gap ensures pauses in the script match the silent gaps heard in the audio.
• All visual timings related to narration must be looked up from this table. Never estimate by character count. Estimating by character count can be off by over 3 seconds after just ten sentences.

Visuals

• Decide on visual style first. If unsure, browse the code animation style gallery at https://github.com/syydaniel/awesome-code-animation ; pick a style, then pick a tool. Unless there's a strong reason, use starter/ (Remotion).
• Draw verbs, not nouns. For "volume increases", draw the action of increasing, not an illustration of an ice cube. See the table of common verb visualizations in docs/04.
• Time constants (at 60fps): the frame where the action lands should be placed 3 frames before the first character of the keyword starts (hit()); allowed range: 6 frames early to 1 frame late. For newly appearing elements, the first frame should be at least 12 frames before the keyword.
• On the keyword hit, the visuals must show a noticeably stronger motion than the previous second: more motion than what happened a second ago. If the whole segment moves constantly, or only moves once at the beginning of a sentence, it won't sync properly.
• Limit new on-screen information to no more than 4 blocks per screen; do not display full sentences that the narration has already spoken—show only the keyword.
• Choose one element that runs through the entire video (a small character, a color, a shape); use camera zooms and pans to follow key points; transition between scenes by using elements already in the frame.
• Subtitles: no more than 16 characters per line, remove commas and periods; characters should fade in according to the actual speaking time, and the keyword changes color.
• After each segment, render a short clip and measure its quality with tools/kazi_check.py before continuing to the next segment.

Sound

• Keep music 12 to 18 LU below the narration; use per-character timing for ducking: push music down 0.2 seconds before a sentence starts, and only bring it back up when the gap between sentences exceeds 1.2 seconds.
• Stop the music 2 to 4 times during the whole video, for the most important sentences.
• Export sound effects from animation cues, split into four layers: hit points, transitions, ambiance, and foley. The texture of the sound should match the visual style.
• Final video loudness: -14 LUFS, true peak not exceeding -1 dBTP, 48 kHz.

Review

• python tools/kazi_check.py out/final.mp4 --timeline timeline.json: measures frame-sync rate, random baseline, black frames, flash frames, prolonged static segments, loudness, audio tracks, and duration.
• If any check fails, go back and fix, then measure again. Only call the user when all checks pass, and present the exact numbers from the report—don't claim "completed" for anything that hasn't been measured.
• When the user requests 4K or HDR, output these after checks pass: tools/master.sh to render a 4K master, tools/hdr/hdr_encode.sh for Dolby Vision 8.4 and HLG. If the color scheme is changed, provide the new background and glow colors to hdr_mask.py; see docs/06_ReviewAndPublish.md.

Tool Overview

Tool	Purpose
tools/doctor.py	Check environment: what's missing, how to install
tools/lint_script.py	Lint the script
tools/voice.py	Clone voice, dub sentence by sentence, pick the best of multiple takes
tools/tempo.py	Change overall speed without changing pitch
tools/pace.py	Normalize pauses: silence at sentence start/end and around punctuation
tools/polyphone.py	Pick out potentially mispronounced polyphone characters, compile into a test audio clip
tools/timeline.py	Character-level alignment, generates timeline.json; --whole for a single pre-recorded narration
tools/kazi_check.py	Measure frame-sync rate and various hard issues
tools/caption_check.py	Verify subtitle appearance, disappearance, per-character highlighting, and alignment with narration
tools/mix.py	Mix narration, music, and sound effects; duck music according to per-character timing; normalize loudness
tools/thin_cues.py	Filter sound effect cues by priority when there are too many
tools/master.sh	Render 4K master in segments, resumable if interrupted
tools/hdr/	Convert SDR master to HDR: Dolby Vision 8.4 (HLG base layer) and HLG
starter/	Remotion template: time.ts for time lookups, anim.ts for curves and springs, subtitle components

What to Tell the User Upon Delivery

• Final video path, duration, resolution.
• Frame-sync rate, random baseline, how much it exceeds random; which keywords were missed and why.
• Any issues found during inspection and how they were resolved; clearly state any unresolved issues.