Article + Audio Directly to Whiteboard Video
Input an article and a pre-existing audio recording, and deliver a complete, playable video. Material design and video production are a continuous internal workflow within this skill; the user is not required to call another skill or manually write technical configurations. Default language is Chinese, white background, colored line art or clean flat graphics, 16:9, 1920×1080, 30fps; adjustable according to user specifications.
Input and Default Decisions
- Save the user's original text and original audio. The provided recording is the timing authority; do not re-dub, alter the voice, add background music, speed up, remove pauses, or cut the end unless the user specifically requests it.
- The article is responsible for the factual structure and explanations; the recording determines the actual spoken delivery and pacing. First, check both for consistency: punctuation, correct reading of chapter numbers, and obvious typos can be corrected directly. If there are minor spoken errors in the recording, preserve the audio and record the subtitle correction. For substantive additions, deletions, reordering, conclusion conflicts, or the wrong audio file, list the specific discrepancies and ask only about how these impact the final video; continue with non-dependent visual preparation.
- Do not ask the user to confirm standard camera shots, style, or technical choices. Start once the article and audio are complete; only ask when essential inputs are missing or when there is a substantive conflict. If there is no audio, complete storyboard preparation first, but do not switch to a new voiceover or a silent video to claim task completion.
- When existing storyboards, assets, or project files are present, first read their
editing-notes,style-lock, and actual configurations, preserving approved content and IDs. For modifications, only redo the affected parts; do not regenerate the entire set of assets.
Continuous Workflow
- Read the audio first, then storyboard. Follow Runtime and Alignment to check the environment, actual decoding duration, and hash; obtain word-level timestamps or reliably sourced sentence-level timestamps, and verify them against the article. The same audio can reuse a hash-matched transcription; same filename alone is not sufficient. First determine narration segments and pauses, then decide on visual information density.
- Transform knowledge into visuals. Read Explanation Visual Design. Segment the storyboard semantically, with each shot having a clear understanding goal and a meaningful change. Prioritize scene illustrations, relationship diagrams, before/after comparisons of the same example, and step-by-step processes. Use keywords to identify objects; let the full spoken text be handled by subtitles. Do not force a fixed number of images per minute.
- Produce once, reuse continuously. Follow Visuals and Assets to create one representative relationship or process visual, then scale up after checking the final display size. Prefer editable SVG for text and relationship diagrams; use image generation tools only when bitmap expressiveness is necessary. Consecutive states share coordinates, scales, characters, and colors. Save editable sources, actual PNGs, and key regions.
- Build a rebuildable project. Follow Internal Data Contract to generate a material checklist, storyboard, and handoff. Run a material audit, then use
pipeline.py initto validate everything in one pass and import into the formal project. Do not create additional rounds of same pre-check projects or intermediate ZIPs. Assets, source code, recording, and renderer all go into an independent project. - Unify audio-visual timing. Use
align_audio.py alignto generate actual shot timings, subtitles, and emphasis anchor points; a single frame-accurate timeline drives both rendering and subtitle output. If automatic matching fails, first review the discrepancy report; do not rely on overall stretching or lowering thresholds to force alignment. Keyword highlighting must correspond to the actual spoken narration; consecutive states avoid repeated fade-ins, while independent scenes may fade in briefly then hold steady. - Preview, render, deliver. First preview representative shots, the first and last states of process groups, the longest subtitle, and complex highlighted regions, then run incremental rendering, export, and complete decoding. Follow Final Video Check to review extracted frames from the final MP4 and audio track correspondence, correcting affected shots. After completion, provide an MP4, SRT/VTT files, and a project ZIP containing the source code; do not stop at an asset pack or a plan.
Conventions That Must Be Maintained
- Preserve the original article's facts, order, qualifiers, and approved spoken text; do not expand text, simplify conclusions, or invent example narrations without authorization to improve the visuals. Clearly label illustrative examples as such; do not present them as genuine screenshots, product behavior, or scientific evidence.
- Use the same visual identifier for the same concept; arrows must have meaning. Entry, connection, step-by-step reveals, and circling/annotations should correspond to the narration; do not use decorative circles, constant zooms/pans, or rapid cuts to hide a lack of explanation.
- Main graphics should not have baked-in outer titles or complete subtitles. Keep necessary node labels as editable text within the graphic; "flat image + source SVG" does not automatically enable arbitrary layer animation.
- Default to disabling source corner tags and image caption bars:
show_source_credits=false,show_image_labels=false. Keep credits, font licenses, and sources in the project files; do not delete necessary time/place information from the content itself. - Single-shot states (
single/compare/textandtitle/point/highlight/focus/reset/subtitle) are already implemented basic capabilities. Switch between step states directly; do not claim continuous movement or real-time drawing. Any additional user-requested actions must be actually implemented and verified by rendering. - Record original estimates, ASR discrepancies, and the actual synchronization method honestly. Only allow padding at the audio tail to compensate for video frame rounding; do not use
-shortestto silently cut the movie. When subtitles are already rendered by the engine, do not burn in an additional SRT track.
Resources and Efficiency
scripts/ comes with material checking, importing, audio alignment, project pipeline, render engine, and Chinese font, without depending on the previous two skill directories. Required Python packages, FFmpeg, and Chromium browser are checked on demand; prioritize reusing the actually available environment; do not copy virtual environments into the delivery package.
Intermediate state stays in the current project, allowing recovery and re-runs. Only cache transcripts that match the audio hash, and rendering segments that match the input hash and pass specification checks. Process independent assets and shots in parallel; execute dependent steps sequentially. First, process representative frames, then batch production; after verification passes, avoid repeated full-video renders without justification.