Yari-tuber/math-rap-video

Mnemonic Instructional Video (Math Rap)

Turns a knowledge point into an approximately 1-minute instructional video. It's memorable because three layers are stacked together, all synchronized to the same audio track:

• Ears: Steady rhythm, one lyric line per measure, single rhyme throughout – easy to follow
• Language: 8–10 character mnemonic, repeated 2–3 times, pinned at the top of the screen
• Eyes: As each word is sung, the graphic advances step-by-step, showing the student "why it's correct"

The narrative is fixed: Pose a problem → Set a trap (clumsy method fails) → Deliver the mnemonic → Demonstrate → Explain the principle → Conclude.

Division of Labor: Who Provides What

Part	Who Does It	Notes
Mnemonic, lyrics, storyboard	You (AI) write, user confirms	Rules in references/lyrics.md
Audio (vocals + music)	User or client provides it	They decide which model, whose voice. Give them the "Client Requirements" from references/audio.md. This skill does not choose a platform for the user, nor does it assume paid generation
Subtitle timing	Script calculates	Any tool's SRT export works, even with typos
Background footage	Auto-selected from library	assets/backgrounds/, tagged by use; can add your own
Graphics, mnemonic, formulas, subtitles	Script draws frame-by-frame	You only write the project-specific storyboard.json and graphics.py

Environment

First time, run the self-check – it will tell you what's missing and how to install each dependency:

python $SKILL/scripts/check_env.py

Requires: Python 3.10+ (pip install pillow numpy librosa pypinyin), ffmpeg / ffprobe, a Chinese bold font (common system fonts are auto-detected), and a background footage library. The library is searched in this order: environment variable MATH_RAP_LIBRARY → this skill's assets/backgrounds/ → shared location ~/.math-rap-video/backgrounds/. If the library is missing, run python $SKILL/scripts/install.py --library <library.zip>.

Hereafter, $SKILL refers to the directory containing this skill (the folder where this SKILL.md resides). It is not a pre-existing environment variable: when running commands, replace it with the actual path, or set it once (bash: export SKILL="/path/math-rap-video"; PowerShell: $SKILL="C:\path\math-rap-video"). Use quotes if the path contains spaces or Chinese characters. All scripts are ordinary command-line programs (all support --help); any AI or person who can run commands can follow them – no dependency on specific tools.

Workflow

1 Write lyrics & storyboard ──→ 2 User provides audio ──→ 3 prepare_audio calculates timing & approves
                                                               │
4 Write storyboard.json + graphics.py ←───────────────────────┘
        │
5 Check + preview images (confirm with images) ──→ 6 build_video produces final video ──→ 7 Review and deliver

After each step, briefly inform the user of the result. Steps 1 and 5 require user confirmation before proceeding.

If the user arrives with pre-made lyrics and audio (lyrics are final, audio made accordingly): Do NOT change a single word of the lyrics, otherwise the visuals won't match the vocals. Step 1 only does three things: create the project, place the lyrics verbatim into lyrics.md, and append the storyboard table at the end; the check_lyrics.py results are for reference only – do not modify lyrics based on them. Then proceed directly from step 3.

1. Create project, write mnemonic, lyrics, storyboard table

python $SKILL/scripts/new_project.py <project_directory>          # Copy the example (counting line segments) as a starting point
python $SKILL/scripts/new_project.py <project_directory> --blank  # Or start from a blank template

Follow references/lyrics.md to write lyrics.md: one line = one measure, each line 11–15 characters (principle section 9–11 characters), single rhyme throughout; structure is Verse 1 (14 lines) + Hook (2 lines, mnemonic spoken solo) + Verse 2 (8 lines).

The storyboard table is written in the ## Storyboard section at the end of lyrics.md (a table: line number, screen action, which word triggers the action). It's for user confirmation and the basis for step 4 code; the script does not read it.

After writing, run a lyrics check; fix all "must fix" items, review each "suggestion":

python $SKILL/scripts/check_lyrics.py <project_directory>     # Structure, character count per line, rhyme, easily misheard writing

Manually verify every number in the example problems (scripts can't check math). Then present the mnemonic, lyrics, and storyboard to the user for confirmation.

2. User provides audio

Give the confirmed lyrics and the "Client Requirements" from references/audio.md to the user. They can use any method to produce audio: AI music platforms, human rap, TTS or human narration with backing track. What they return:

• One or more audio files (mp3 / wav) with vocals. For AI-generated audio, recommend 2–3 versions to choose from
• Ideally, also provide a subtitle file (SRT). If not, use any speech recognition tool on the audio to generate one and export as SRT

If the user explicitly asks you to generate the song on their behalf and provides an ElevenLabs API Key, you may use scripts/gen_music.py (usage in references/audio.md). This is optional – each generation consumes their quota; explain this before generating.

3. Process audio: calculate timing and determine if this version is usable

python $SKILL/scripts/prepare_audio.py <project_directory> --audio song.mp3 --srt recognized.srt --bpm 90 --intro-bars 2

It aligns the recognition result to the lyrics (typos and different phrasing are OK), detects speed and beat, writes timing.json, and copies the audio file into the project directory (the project directory will now contain the audio file – this is normal). The final line gives the conclusion:

• Usable: Continue. If there are lines with "low match rate", note them down and remind the user to listen to those lines during delivery
• Suggest trying another version (exit code 3): Lists reasons, common ones being a missed line, vocals starting too late, or overall compression. Swap to another audio version – do not force it

If there are multiple audio versions, write each to a different file for comparison, then copy the chosen one to timing.json:

python $SKILL/scripts/prepare_audio.py <dir> --audio generated_01.mp3 --srt 01.srt --bpm 90 --intro-bars 2 --out timing_01.json
python $SKILL/scripts/prepare_audio.py <dir> --audio generated_02.mp3 --srt 02.srt --bpm 90 --intro-bars 2 --out timing_02.json
cp <dir>/timing_02.json <dir>/timing.json

How to choose – see references/audio.md section 3. In short: first exclude versions labeled "try another version", then pick the one where the mnemonic's two lines have the highest match rate, and only then consider rhythmic regularity. When the two conflict, mnemonic clarity takes priority over rhythm: what the audience remembers is the mnemonic.

Other cases: Pure narration audio without beat – add --no-beat; fallback when no SRT is available – use --grid --entry <vocals_entry_seconds>, estimating one line per measure; --intro-bars should match the number of intro bars agreed with the client (the 2 in the example is our default client requirement), do not include if no agreement was made. On Windows PowerShell, cp works; in cmd, use copy.

To swap audio, just re-run this step – no need to change the storyboard.

4. Write storyboard and problem-specific graphics

Read references/visuals.md, then edit two files in the project directory:

• storyboard.json: Plans for mnemonic banner, formulas, text bursts, arrows, background transitions
• graphics.py: Problem-specific graphics, implementing setup(ctx) and draw(canvas, t, ctx)

The most important rule: All times must be written as expressions relative to the lyrics – never hardcode seconds.

Syntax	Meaning
L(6) / E(6)	Start / End of line 6
W(9,'四') / W(9,'四',2)	Time when character '四' (2nd occurrence) is sung in line 9
B, L(6)+2*B	One beat; 2 beats after line 6 starts

This ensures visuals truly sync with the vocals, and if the client swaps the audio, re-running step 3 directly produces the final video.

Graphics are the most time-consuming step. The engine already provides common drawing methods (dots, lines, arcs, sector angles, right angle symbols, number blocks, parentheses, multi-color single-line formulas, whole-block rotation and translation, movement along a path, equidistant spacing) – first look for existing ones in references/visuals.md section 4, don't reinvent from scratch. Generate a small piece and view it (next step) – don't write everything before checking.

Based on real-world testing, avoid these two common pitfalls from the start:

• Graphics are too small and too thin. This is for mobile viewing: main graphics should occupy more than half the screen width (width ≥ 800 or height ≥ 400), use ctx.spread(n) to fill the width for a row of elements, line width ≥ 9, font size ≥ 44. Size references in references/visuals.md section 1
• Every line must have visual action. At least one of graphic, text burst, or formula should be in the center of the frame; don't let any line have only background and subtitles

5. Check and preview

python $SKILL/scripts/overlay_engine.py <project_directory> check          # Mandatory: 0 layout issues, 0 rendering errors; review each "layout warning"
python $SKILL/scripts/overlay_engine.py <project_directory> sheet          # storyboard_preview.jpg: one grid per line overview
python $SKILL/scripts/overlay_engine.py <project_directory> sheet --pages  # Also generate storyboard_preview_p1.jpg...: large paginated view to check if text touches edges (overview image updated too)
python $SKILL/scripts/overlay_engine.py <project_directory> frames "L(18)" "L(19)" 0.2   # storyboard_frames.jpg: spreading a section of animation frame by frame

Open the images yourself, check each frame against the storyboard:

• Overview image: Did the expected elements appear for each line? Are any frames empty?
• Large paginated view: Are any text elements overlapping? Touching the mnemonic banner or subtitles? Are graphics thick and large enough?
• Areas with movement, rotation, or fly-ins: Use frames to verify nothing collides mid-transition
• For time points flagged by check as "layout warnings", inspect each one (it will flag: touching banner, overlapping subtitles, overlapping with formula, touching edges, graphics too small, empty center area, text overlapping)
• What check can't detect: text blending into lines or blocks and becoming unreadable, wrong graphic count, insufficiently vibrant colors. Only the large images can reveal these

If issues are found, go back to step 4 and fix. Once everything looks good, present the overview image to the user for confirmation, then build the final video.

6. Build final video

python $SKILL/scripts/build_video.py <project_directory> bg            # Select and concatenate backgrounds; prints which footage was used per segment
python $SKILL/scripts/build_video.py <project_directory> final 20 30   # Optional: generate a 10-second preview first
python $SKILL/scripts/build_video.py <project_directory> final         # Full video, approx. 6–12 minutes (depends on computer speed), run in background
python $SKILL/scripts/build_video.py <project_directory> sheet         # final_video_overview.jpg
python $SKILL/scripts/make_vertical.py <project_directory> --title "Title"   # Optional: vertical version (Douyin, Kuaishou, WeChat Channels)

bg automatically checks if each segment contains problem-specific graphics – if so, it only selects "quiet" clips (no bright close-up faces) and notes this in the printout. After previewing everything, you can also use build_video.py <project_directory> all to run check + bg + final + overview image in one command. The vertical version places the landscape final video in the center of a 1080×1920 canvas, adds a title at the top, small text at the bottom, and fills the background with a blurred magnified version of the video; it does not rearrange graphics vertically. If --title is omitted, it reads vertical.title from storyboard.json; if that's also missing, it uses the lyrics file title.

7. Review and deliver

Check final_video_overview.jpg: Is the background dark enough? Are graphics and text clear? Is there a bright face from the background accidentally blocking a text burst or explanatory text? (If so, add :quiet or specify a clip for that line in the background plan, then re-run bg and final.) Then deliver final_video.mp4 to the user with the following notes:

• The visuals and timing are script-verified, but the audio needs human review: please pay special attention to whether the mnemonic pop-up and result numbers appear simultaneously with the sung words
• For lines flagged as low match rate in step 3, listen to them specifically

Reference Files

File	When to Read
references/lyrics.md	Before writing mnemonic, lyrics, storyboard
references/audio.md	When users ask "what tool to use for music/voiceover"; when sending audio requirements to client; when approving and selecting audio
references/visuals.md	Before writing storyboard.json and graphics.py (layout, fields, drawing tools, common problem types)
references/library.md	When adding clips to the library, generating new backgrounds, or when background selection is inappropriate
references/troubleshooting.md	When any step errors or results are incorrect
references/teaching.md	When the user wants to understand principles, teach others, or avoid writing code (CapCut approach)
references/compatibility.md	When users ask "can other AIs / models do this"; when passing this skill to others
examples/shuxianduan/	Complete example: lyrics, storyboard, graphics. Read before writing a new problem

Script Overview

Script	Purpose
check_env.py	Environment self-check
new_project.py	Create a new project directory
check_lyrics.py	Lyrics health check
prepare_audio.py	Audio + SRT → timing.json and issues usable / try-another-version verdict
overlay_engine.py	Check, preview, render overlay layers
build_video.py	Select backgrounds, composite final video, export overview image
make_vertical.py	Landscape final video → vertical version
library_tool.py	Manage background footage library (add, delete, retag, overview)
gen_music.py	Optional: generate song using ElevenLabs
install.py	Install this skill into other AI tools (Codex, Gemini CLI, opencode, etc.), place the footage library
chat_pack.py	Generate prompt packs for chat-only AIs (write lyrics / write visuals / modify based on check results)