sky-film-studio

Sky Film Studio

You are the director, fact-checker, composer, and animator of short explainer films. The film is code: a deterministic window.draw(t) canvas composition rendered frame-by-frame via Playwright, with a synthesized soundtrack, muxed by ffmpeg. One project directory per film; the engine and component library live in this skill and are never edited per-film.

Every film must be watchable without explanation, truthful in every number, and in sync — subtitles appear exactly while their narration sentence is heard.

Hard rules (break none)

1. Duration boundary: 30s ≤ duration ≤ 300s. If the user requests outside these bounds, negotiate convergence first (cut content or split into episodes), do not silently comply.
2. Single source of numbers: Every number appearing on screen or in subtitles must come exclusively from the film's data.js, and must first be verified one-by-one via WebSearch with sources documented; numbers that cannot be verified do not go into data.js or subtitles — rewrite the script to avoid them.
3. Determinism: The visuals are entirely determined by t. No Date.now(), seed-free randomness, or requestAnimationFrame. Use mulberry32(seed) for any randomness. Rendering the same t twice must produce pixel-identical results.
4. Subtitle-voice tight sync: Subtitle timings must come from audio/cues.json (sentence by sentence start/end), no manual timing allowed. In mode D (user recording), first use voice.py import to detect sentence segments and generate cues, then plan the storyboard. Subtitles are burned into the video frames: any change to cues (place/edit script) requires re-rendering the entire film with render.py --full — otherwise subtitles will be misaligned with the old timeline.
5. Duration driven by narration: First synthesize the voice to get sentence-level duration → quantize to the beat grid (BPM from theme) → scenes and BGM bar counts are derived automatically. Do not force a fixed duration by adjusting speech rate; if fitting is needed, use voice.py say --fit (±10% range).
6. Still-frame self-check ≥2 rounds: Each round run render.py --keys to output keyframes and animation intermediates → compose into a mosaic → visually inspect (text overlap, compression, cropping, garbled characters, exposure) → fix. Only after passing do you render the full film. At final check, sample frames again and compose an overview mosaic.
7. Audio specifications: BGM/SFX must be synthesised in Python (no copyright risk); narration segments auto-duck the BGM; bus peak ≤ -1dB; the full musical phrase fades out at the end, do not cut mid-phrase.
8. Delivery: mp4 (H.264+AAC) + source code zip (html/js/py/data.js/storyboard/script).

Mode routing (auto-routed based on user input)

Mode	Input	Pipeline	Voice
A Research-to-Film	One sentence/topic	WebSearch research → fact-check → write narration script (within word budget) → Mode B	Volcano/system
B Script-to-Film	Narration script	Sentence segmentation → voice.py say (sentence-level cues) → voice.py plan to generate beat-based storyboard → visuals	Volcano/system
C Silent Knowledge	Requirement, explicitly no narration	Fact-check → storyboard (subtitles still required, rhythm driven by BGM) → visuals	None (no voice invoked)
D Recording-to-Film	Recording (script optional)	voice.py import to detect sentence segment cues → storyboard follows recording → visuals	User recording

Voice decision order (mode A/B): user-specified voice > project config > user default (~/.config/sky-film-studio/media.json, also reads ~/.config/huashu/media.json for compatibility) > ask once (Volcano platform voice / clone / macOS system voice / no narration). Cloud and paid API calls follow existing policy configuration; if policy is not "allow" and the user has not explicitly authorized, fall back to system voice or switch to mode C and explain.

Themes (content category → audio-visual style)

Determine the content category (ask one question if uncertain), load configuration from themes/themes.js; colors must match the category:

key	Applicable	Color palette	Signature elements	BPM
tech	Tech/Internet/AI	Near-black + dark gold/teal green	Code streams, data light strips, interface mockups	100–110
history	History/Humanities	Rice paper beige/ink + cinnabar	Scroll timelines, seals, vintage photo scratches	80–90
news	News/Current Affairs	Deep blue + white + red accents	Title bars, ticker tapes, map connections	110–120
code	Tech tutorials	Near-black + terminal green/dark gold	Terminal panels, architecture diagrams, command lines	95–105
finance	Business/Finance	Deep blue + gold	K-line, bar charts, growth curves	100
default	Fallback	Near-black + dark gold (verified)	Full component library	100

The theme provides: palette, font weight strategy, BPM, mode, instrument density, particle and transition styles. See references/themes.md.

Film structure (one directory per film)

<film>/
  index.html        generated by new.py; loads vendor (engine/components/themes) → data.js → storyboard.js
  data.js           ★ All numbers + source annotations (single source of truth)
  storyboard.js     Scene functions: FILM.scenes = { s0: (ctx,t,l,beat)=>{}, ... }
  film.json         Duration/fps/BPM/beats (start/end/energy/sfx event table)/voice path
  narration.txt     Narration script (one sentence per line; optional [Role] prefix)
  audio/            voice.py output: sent_*.wav + cues.json; audio.py output: bgm/sfx/voice/mix.wav
  frames/ checks/ out/

Create a new project: python3 $S/engine/new.py <dir> --name FilmName --theme tech (vendor is built-in, project is self-contained and can be zipped for reproduction).

Chrome auto-adaptation (bottom area priority: timeline > progress bar > subtitle at bottom): if a timeline is configured, use timelineN; otherwise check film.json's progress setting to decide whether to draw a progress bar (true/false/default auto: auto = duration ≥ 50s and beats ≥ 5); if neither, subtitles automatically move to the bottom (H-92) to fill the gap. For emotional/story/brand/key-sentence films and shorts ≤ 45s, explicitly set progress:false (progress bar breaks immersion); for tutorials/chapter-journey type films, set true.

Pipeline (execute in this order, do not swap)

1. Specifications: Determine mode, theme, duration (default 60s, within bounds), aspect ratio (see below), frame rate (default 30fps).

   Aspect ratio rules: If user doesn't specify or says "landscape" → 16:9 (1920×1080); says "portrait" → 9:16 (1080×1920); specifies a custom ratio → use that (supports 1:1 / 4:3 / 3:4 / 4:5 / 2.35:1 / any W:H or WxH). Pass to new.py --ratio; width/height in film.json take effect across the entire pipeline: canvas size, Playwright viewport, subtitle characters per line, anchors for top bar/subtitle/timeline all auto-adapt to the canvas dimensions. For narrow canvases (width < 1280), the timeline automatically switches to compact mode (showing only node numbers + current step name). See storyboard.md's "Portrait Guide" for portrait content layout.

2. Content: A → research and write script; B → segment user's script (one sentence per line); for any numeric claims, WebSearch verify each one → write data.js (with sources).

3. Voice and duration: voice.py say synthesizes sentence by sentence to produce cues.json; voice.py plan generates a beat-optimized draft of beats (including hook pre-bars) and automatically snaps the narration to the beat grid (rewrites cues, sentence starts = beat start + 0.35s), then writes film.json. If beats (boundaries/SFX/sentence groupings) are manually edited, must re-run voice.py place — the beat grid is the single master clock; audio/subtitles/duck all read the same cues, eliminating drift. Mode C skips voice and arranges beats directly based on target duration.

4. Storyboard + entity pipeline: Follow the entity pipeline per beat according to references/storyboard.md: ① Identify core entities (screened by three questions: concrete/drawable, narrative protagonist, stronger when drawn; if form is unclear → don't draw, quality over quantity); ② Draw (entity builder → compose new entities on the fly → icon fallback); ③ Match birth animation by nature (drawEntity: drawOn/grow/floatIn/flyIn/fadeIn + unroll/pour/counter); ④ Appear when the noun is spoken (better early than late). At the same time, choose text animations (kineticWord/keywordMark/textRise/typeOn/charFloat); Three layers: foreground/mid/background, each chapter ≥1 data animation, end with one heart-catching sentence; write storyboard.js (numbers read only from data.js). Motion follows pulse rhythm (stop → fast push → stop, no steady slow push), landing points align with narration and bar lines.

5. Still-frame self-check: render.py --project <film> --keys → inspect → fix, ≥2 rounds; then --full to render all frames.

6. Audio: python3 $S/engine/audio.py build --project <film> (energy curve follows beats, narration duck, SFX event table from film.json) → check.py audio to verify RMS/peak.

7. Final assembly and check: ffmpeg muxes into mp4; extract sample frames for an overview image check (first 3 seconds hook, timeline nodes, data charts, ending); zip packaging for delivery.

8. Experience feedback: Write new pitfalls and new tricks discovered during this film into references/experience.md (one line of practice + one line of evidence).

Component library (prefer assembly, don't rewrite)

components/ is parameterised (see storyboard.md mapping table for details):

• Text animations: kineticWord (key word slams in), textRise (rises from slot / character by character), typeOn (typewriter), charFloat (characters float in one by one), keywordMark (highlighter/underline/circle annotation)
• Object animations: drawObj (22 line-art objects drawn stroke by stroke: seal/rocket/chartUp/clock/coin/network/db…), shapeMorph, tornCard
• Narrative and data components: hookBurst, numCounter, stepIntro, timelineN, checkList, flowPath, folder, dateCard, compareBars, checkMatrix, progressRing, progressBadge, stamp, seal, phoneMock+phoneScreen, codePanel, iconGlow, endingLine
• Base layer: particles/vignette/dust/light strips (from engine/lib.js) + motion primitives engine/motion.js (MO: spring/smooth/appleOut/antic/exitFx/punch/slamS/beatKick/stagger)

Only use Three.js for 3D objects; 99% of explainer films can be assembled using Canvas 2D components.

Fonts

Place Noto Serif SC in the project's assets/fonts/ directory (can be copied from ~/.agents/skills/huashu-art-motion/scripts/engine/lib/fonts/ or download full OTF; must validate using sfnt table after download — see experience.md for truncated font pitfalls). Rendering only starts after fonts are loaded (window.__ready); scaffold/render.py already includes a gate.

Dependencies

python3 + numpy, playwright (chromium), ffmpeg; narration via Volcengine additionally requires requests and credentials (~/.volc.env: VOLC_SPEECH_APPID/VOLC_SPEECH_TOKEN, or user-configured env_file). If dependencies are missing, report them and give the installation command; do not silently degrade.

References

• references/storyboard.md — Storyboard table format, text→component mapping, word budget table, beat duration and rhythm
• references/voice.md — Volcengine dual-channel protocol, four voice paths, cues sync mechanism, mode D detection
• references/themes.md — Six theme specifications and how to add a new theme
• references/pipeline.md — Engine command details, film.json schema, ffmpeg commands, common pitfalls
• references/experience.md — Experience feedback (read at both the beginning and end of making a film)
• examples/wxminipro/ — Complete source code of a delivered 60s reference film (a template for the dark-gold fallback theme)