chulegexiaojiu/novel-storyboard

What does novel-storyboard do?

Creates a storyboard for AI short dramas—the first layer in the pipeline that directly faces the video model. The premise is ingrained: shots are generated, and the cost of cutting more shots is nearly zero. Therefore, we don't skimp on shot count; the only upper limit is the video model's single segment generation length (default 15 seconds).

Core mechanism: shots claim beats. Each shot declares which consecutive beats of a script scene it covers (sceneIndex + beats: [start, end]). Shots must not cross scenes—a scene change always triggers a new shot. This makes the relationship between storyboard and script mechanically auditable:

Deliverable	Solves
Beat claiming	Every beat is claimed by exactly one shot, in order—modify the script and re-run validate; invalid shots are flagged immediately
Single shot ≤ 15 s	AI video single segment generation limit; long dialogues are forcibly split (params.maxShotSeconds is adjustable per model)
Dialogue fits	Duration of claimed beats' dialogue ≤ shot duration—checked per shot, not guessed
Frame prompt + shot text	frame (Chinese, with character name) for image model with reference images; shot (Chinese, generic identity) describes what happens. Shot scale is an enum, and Chinese scale terms must appear in the frame prompt
H3 video prompt (one per segment)	MiniMax H3 I2VA structure: fixed alignment instruction + integrated_multimodal_description + overall_soundscape + non_diegetic_music. Dialogue from claimed beats goes verbatim into <d>[Chinese] …</d> blocks—dialogue, soundscape, and music all in one prompt
Seedance video prompt (one per segment, procedurally built)	Generated alongside H3. The script assembles the official structure: reference image declarations, segment blocking, per‑shot quantitative fields, dialogue {}, sound effects <>, music (), numbering constraints. The model only writes the shot for each cut, not timings, numbers, or image references
Quantitative composition fields	Per segment: blocking (who's left/right/distance). Per shot: focal length / camera position / composition / gaze / focus / stability—when no storyboard art exists, these are the sole source for composition
Generation batch list	Shots from the same scene + same lighting are grouped into one batch, sharing one environment reference image—AI version of a continuity breakdown, summarised automatically
Dubbing alignment sheet	Every dialogue line mapped to a shot number—where the TTS audio clips attach to which video segment, auto‑generated

{baseDir} = directory containing this file. Script {baseDir}/scripts/novel-storyboard.mjs, zero dependencies, run directly with node.

Boundaries (what we don't do): We don't write dialogue or change the script (that's novel-script's job). We don't produce scene/character/prop reference images (novel-art / novel-characters's job). We don't generate video or do compositing. Lip sync is not managed yet—that belongs to the generation pipeline.

---

Step 0 — Define Input & Scope

script.json is a hard prerequisite—a storyboard without a script is meaningless. Both validate and render require --script. Other upstream inputs are used if provided:

• --outline / --cast: enables prohibited name checks in prompts and shows character names as C01 in the report
• --art: enables scene names as S01 in the report and embeds scene reference images in batch lists
• --shots <card directory>: optional mount a shot‑recipe card library (point to shot-recipes/references/cards, only accepts a directory, not an exported JSON file). This enables the 18th gate shot-recipe. If shot-recipes is not installed, don't use it—this skill is self-contained and does not depend on it

How many episodes per pass: Follows the script's batch (storyboard wherever the script is written), default batch size ≤ 3 episodes.

Step 1 — Seed the Working Draft

node {baseDir}/scripts/novel-storyboard.mjs seed <script.json> --eps 1-3 > <workdir>/storyboard.json

Deterministic expansion: per scene, a beat list (number, action/dialogue, duration per beat, speaker) goes into seedScenes. This is the working draft for cutting shots. Each beat's duration is calculated, not estimated by the model. shots is left empty; shot‑cutting is the model's job.

Step 2 — Cut Shots per Episode, within Segments

One task per episode, can be concurrent. Each task receives:

• {baseDir}/references/storyboard-pass.md and {baseDir}/references/schema.md (read and follow them)
• The episode's seedScenes draft + scene cards (art.json's anchors and lighting prompts) + character cards (cast.json's appearance highlights)

Process: first group beats into segments (each 9–15 s, does not cross scenes), then cut 2–5 s shots within each segment (shot‑reverse‑shot for dialogue, key action insert close‑ups, establishing three‑shot—cutting grammar is in storyboard‑pass.md). For each cut, write a frame prompt (Chinese, use character names directly), a shot description shot (Chinese, generic identity), and six composition fields. Per segment, write blocking, soundscape, and optionally music.

Write each segment's shot descriptions following {baseDir}/references/shot-writing.md (protocol‑agnostic: one cut = one camera movement, actions must be completable, dialogue verbatim, layered sound, no style descriptions). Whichever video service you send to, wrap them in that service's protocol syntax: MiniMax H3 see {baseDir}/references/h3-prompt.md, Seedance see {baseDir}/references/seedance-prompt.md.

Write one h3Prompt per segment following {baseDir}/references/h3-prompt.md (internalised version of the official methodology, no dependency on any external skill). Official language is English by default (promptLang can switch to Chinese), each shot on its own line. Key points: the first line (alignment instruction) and [Shot k] cut timestamps are derived from shot durations, not a single character off (validate checks each character); claimed dialogue verbatim into <d>[Chinese] …</d> blocks; camera movement words go in their own line; soundscape and music go into the last two fields—**soundscape is also an action instruction; if the scene changes, update the soundscape.

Don't write Seedance prompts: the render report's prompt panel and export --protocol seedance both assemble them from the fields above. The visual style is provided by the caller, global constraints via --constraints <file>.

After cutting, delete seedScenes.

Step 3 — Validation ⛔ Must Not Be Skipped

node {baseDir}/scripts/novel-storyboard.mjs validate <storyboard.json> \
  --script <script.json> --outline <outline.json> --cast <cast.json> \
  [--shots </path/to/cards>]

All 18 quality gates are pure code: full beat coverage (at shot level, exactly once, in order, contiguous), segment 0 < total seconds ≤ 15, each cut 2–5 s, dialogue fits in shot, per‑episode total duration within script target ±15%, on‑screen ≤ 3 people (if exceeded, must include a splitting explanation), segment numbers in E01-01 continuous format, Chinese scale terms appear in frame prompts, camera movement uses H3 vocabulary and appears in its own [Shot k] paragraph, H3 alignment instruction derived from storyboard structure—character‑by‑character check + cut‑timestamps check, claimed dialogue verbatim in <d> blocks, prompt language consistent with promptLang (bidirectional check: Chinese written as English is blocked, English mixed into Chinese is blocked), frame prompts non‑null in Chinese, video prompts contain no character names (H3 text and Seedance shot texts are checked regardless of language; frame prompts are allowed to use names directly), all quantitative composition fields present, Seedance shot texts compliant (non‑null Chinese, no time/shot number/image references/protocol symbols), scene/character/prop alignment with script, shot recipe alignment (optional gate, see below).

Fix any violations one by one, re‑run until all pass.

The 18th gate shot-recipe (optional mount): only checked if --shots is provided; otherwise explicitly noted as skipped. Each cut can have an optional recipe (card id, at cut level, not segment level; multi‑card recipes are expressed by consecutive cuts with the same id, not an array). The gate checks three conditions: id exists in the card library, every must_phrases from the card appears in the cut's frame (both lowercased and .includes), and if the card's cuts lower bound ≥ 2, the number of consecutive cuts with that id must be at least that bound. The card's suggested shot scale and camera movement are not enforced—they only appear as hints in the report's "Recipe" column and at the end of checkup. A recipe is a vocabulary, not a law; making an optional mount too strict means nobody will mount it.

Step 4 — Output & Report

cd <output directory>
node {baseDir}/scripts/novel-storyboard.mjs render <drama-name>-storyboard.json --md \
  --script <script.json> --outline <outline.json> --art <art.json> > <drama-name>-storyboard.md
node {baseDir}/scripts/novel-storyboard.mjs render <drama-name>-storyboard.json --html \
  --script <script.json> --outline <outline.json> --art <art.json> > storyboard-report.html

Report interface language is set via --lang zh|en (priority: --lang > JSON top‑level lang field > default Chinese)—only affects UI labels, independent of promptLang (H3 prompt language). render uses --frames <directory> to locate storyboard frames (<directory>/<segment>/f<cut number>.png, defaults to current directory) and --images <directory> to locate scene reference images for batch lists (<directory>/<scene slug>-sheet.png, defaults to ./images). Both can point to any path—this skill does not produce these files; after the downstream graphist generates them, re‑run render to embed them in the report. export --frames <directory> copies any found frames into the production package, no manual copying needed. The report includes: KPI bar, shot‑rhythm bar (thick separators = segment boundaries, bar width = shot duration share, color intensity = shot scale distance, clickable for segment navigation), per‑episode shot table (main frame + sub‑frame strips + per‑cut shot rows + frame/H3 prompt copy buttons), generation batch list, dubbing alignment sheet, quality gates, and JSON export. The Markdown version includes the full H3 prompt per segment, ready to copy.

In your summary, state clearly: how many episodes and shots, total duration vs target, number of generation batches, report path; explicitly mention any gates that failed.

Don't say "images generated". That step no longer exists—the deliverable is prompts and a list of reference images to attach.

Final output structure:

<output directory>/
├── <drama-name>-storyboard.json
├── <drama-name>-storyboard.md
├── storyboard-report.html         ← double‑click to open
├── manifest.json                  ← generated by export
└── E01-01/                        ← one folder per segment = all materials for one H3 generation
    ├── f1.png                     ← main frame (placed here by downstream)
    ├── f2.png …                   ← sub frames
    └── prompt.md                  ← H3 prompt (generated by export)

---

The Five‑Skill Relay (Pipeline Closes Here)

novel-outline    → outline.json    (what: structure & episode summaries)
novel-characters → cast.json       (who: character reference images)
novel-art        → art.json        (where: scene/prop reference images)
novel-script     → script.json     (action: scenes, beats, dialogue)
novel-storyboard → storyboard.json (how to shoot: shots, first frames, batches)

The storyboard is the consumer: seed ingests script.json; storyboard frame generation (downstream) uses art and characters reference images; H3 prompts go directly to the video model; dubbing alignment sheet connects scripts' TTS output. Each of the five JSON files' reports has an export button, and any modified file can be fed back into its respective render/validate.

Boundaries

• Report UI is bilingual (Chinese/English via --lang, default Chinese); prompt language is independently controlled by promptLang (default English)
• Segment duration is the generation duration sent to the video model, not an estimate—adjust params.maxSegmentSeconds per your model; cut rhythm interval adjust min/maxCutSeconds
• Lip sync is not managed yet—that belongs to the generation pipeline
• Frame prompts don't need to be perfect on first pass; they are composition anchors for the video model. As long as composition and assets are correct, minor refinements can be left to re‑generation

Gate Failures Accumulate

Both validate and checkup append gate results to .gates.jsonl in the current directory. Run stats to summarise:

node {baseDir}/scripts/novel-storyboard.mjs stats

Answers three questions: which gate fires most often (the rule the model most frequently ignores—update your phrasing), which gate never fires (might be a dead gate, or the rule is internalised), what do failed details look like (recurring issues not caught by a gate need human review).

To skip logging, add --no-log. If the log file can't be written, it silently skips; validation still passes/fails normally.

Self‑Test

node {baseDir}/scripts/selftest.mjs

323 assertions, no model calls, no credit expenditure. Every one of the 18 quality gates has a test case that breaks it. Run this after any script changes.

Built‑in Example

{baseDir}/examples/Ferry-storyboard.json: Complete storyboard for episode 1 of "Ferry"—10 segments, 34 cuts, claiming all 35 beats of the script, average 3.5 seconds per cut, total 119 s / target 120 s, 2 generation batches, each segment with a complete H3 video prompt (multi‑frame alignment + cut timestamps all pass validation). Serves as a quality benchmark and test fixture.