ci-story

What does ci-story do?

Each lesson is a short video. A small cast acts out a simple story while a calm voice narrates short sentences, and each sentence repeats many times with the picture making its meaning obvious. Captions in the language's script plus romanization highlight the word being spoken. The last ~15% runs without captions, for pure listening.

Work out the plan yourself from the course data and the learner's request, then present it and wait for the learner's go before generating anything (step 5). Apart from that, don't ask questions mid-run, except during first-run setup and before large downloads.

Locations

• <skill>: this skill's folder (the one containing this file), with scripts/, references/ and assets/.

• Config: ~/.config/ci-story/config.json holds workspace, tts_project (only for a local qwen voice) and optional env. If it is missing, or a path in it does not exist, follow references/setup.md first.

• A course: <workspace>/courses/<lang>/, holding:

  • course.json: voice, fonts
  • notes.md: learner overrides
  • art/drawn/, art/generated/
  • audio-cache/
  • lessons/NNN-slug/

  When the learner doesn't name a course and only one exists, use that one.

• Never run uv lock --upgrade in the tts project, and never change a torch path source there. A multi-GB torch wheel may be pinned as a local file.

Workflow for "a new story"

1. Read the course. Run python3 <skill>/scripts/curriculum.py report <course>. Read notes.md, and the last one or two lesson.json files plus their story.py, for continuity of cast, props and helpers you can reuse.

2. Record what the learner said.

   • A word that is hard or forgotten: add a line struggling: 给.
   • A word they know well: add solid: 苹果.
   • Words they want more of: add focus: ….

   Tag lines hold only words, comma-separated. A later line overrides an earlier one for the same word, and remarks in (parentheses) are ignored.

3. Plan, like a teacher:

   • Honor the request first: its topic, focus words or review target.
   • Pick 2–3 new words. Each must be concrete and showable (an object, a visible action, a number, a colour, or a contrast such as has/doesn't have or big/small), common in everyday speech, and able to combine with known words into simple sentences. A course's first lesson uses only 5–7 words in total (for example 这 是 一个 plus 2–3 nouns), each heard 10+ times. Since every word in a first lesson is new, list them all in new_words.
   • Weave in every due word at least 3 times. Use fragile and struggling words more often. Leave solid words out unless the story needs them.
   • Every new word is heard at least 10 times, each time with the picture showing what it means.
   • Story shape: introduce (这是…), contrast (有 / 没有), an action or change, then a recap that combines everything. Keep the existing cast (man, woman) unless the story needs someone new.
   • Length follows from the content. By default, use as many sentences as it takes to give each new word 10+ hearings and cover the due words. That is usually 30–50 sentences of 2–6 words, roughly 2–4 minutes (about 4 s per sentence). If the learner asks for a longer story ("about 6 minutes", "a long one"), keep the same 2–3 new words and add scenes: retell the story, put the same words in a new situation, and bring back older words. Never add more new words to fill time.
   • Prefer repeating a sentence exactly to inventing a variant. Exact repeats sound identical (they come from the cache) and help listening.

4. Draft <course>/lessons/NNN-slug/lesson.json. NNN comes from the report. This draft is the plan's source of truth, and you'll revise it until the learner approves. Put each beat on one line, so beat numbers are easy to see:

   {"title": "…", "art": "drawn", "new_words": ["…"],
    "beats": [
     {"words": [["这","zhè"],["是","shì"],["一个","yí ge"],["人","rén"]], "show": "man alone", "target": ["man"], "caption": true}
    ]}
   

   • words holds the word units with tone-marked romanization, using sandhi as spoken (一个 = yí ge). No punctuation inside words. Every beat is read as a statement ending in 。, so don't plan questions.
   • show is prose describing the picture, for you and for the art.
   • target lists the sprite ids that glow while the beat is spoken.
   • caption is false for roughly the last 15% of beats.
   • pause is optional: the silence after the beat, default 1.5 s, minimum 1.2. Make it longer (3–3.5 s) on a beat followed by movement, such as a handover, so the motion happens in the silence.
   • art is drawn when you are Claude and generated when you are Codex.

   Then run python3 <skill>/scripts/curriculum.py check <lesson.json>. Fix every ERROR and every WARNING. It also prints a numbered list of beats; keep it for step 8.

5. Present the plan and wait for the go. Before any art, narration or rendering, show the learner a short plan built from the draft and the check output:

   • Story: two to four sentences on what happens, scene by scene, in plain English.
   • New words, each with its romanization, meaning and how the picture shows it (for example "香蕉 xiāngjiāo, banana: the woman holds one, then hands it over"), and how many times it is heard.
   • Review words, with why each is in (due, fragile, struggling or focus) and how many times it is heard. Mention any due word you left out, and why.
   • Length: the number of sentences and about how long the video runs (about 4 s per sentence), plus the art set and the voice.
   • Anything new to draw or generate.

   Then ask, politely and in one short line, whether they want changes or are happy for you to generate. Stop and wait for their reply. Treat any feedback (a different topic, other words, longer or shorter, a different twist) as a revision: update lesson.json and notes.md as needed, re-run check, and present the revised plan the same way. Generate only once the learner clearly approves ("ok", "go", "generate it", "looks good"). If they said up front that they don't want to review the plan ("just make it", "surprise me"), skip this step. If they drop the idea, delete the draft lesson folder.

6. Art. Reuse the sprites in <course>/art/<set>/. For anything missing, follow references/art.md exactly, then add new files only:

   • Claude: write SVG by hand into art/drawn/<name>.svg, modelled on the course's existing files (or references/examples/ in a new course). A new character needs all three expressions. Look at it before using it:

     uv run --no-project --with 'cairosvg>=2.7,<3' python -c "import cairosvg,sys; cairosvg.svg2png(url=sys.argv[1], write_to=sys.argv[2], output_width=512)" <svg> /tmp/preview.png
     

   • Codex: use the image generator. Save transparent PNGs to art/generated/ and append the prompt to art/generated/prompts.md.

7. Narration. Use the command that matches voice.engine in course.json:

   qwen (local GPU): <env from config> uv run --project <tts_project> --no-sync python <skill>/scripts/audio.py <lesson dir>
   edge (online):    uv run <skill>/scripts/audio.py <lesson dir>
   

   Run it in the background. Cached sentences are reused with no GPU or network. New ones take about 30 s each with qwen on a GPU (3 takes plus alignment), or about a second each with edge. If the run is interrupted, re-run it: finished sentences are cached. This writes timing.json (real beat and word times) and narration.wav. If a cached clip sounds wrong (a misread word, a dropped syllable), delete its two files and re-run; grep -l '<sentence>' <course>/audio-cache/*.json finds them.

8. Choreography: write <lesson dir>/story.py. It defines scene(t, timing) and returns (sprites, marks). The contract is in the docstring of scripts/engine.py.

   • Key all movement to beat times (timing["beats"][i]["start"] and ["end"]), never to literal seconds.
   • Hand positions come from e.anchor / e.on_hand with e.HAND_L / e.HAND_R. On the mirrored right character, HAND_R is the hand toward the centre.
   • e.ease gives smooth motion. For rings and anything tied to "the sentence being spoken", use index, _ = e.active_beat(t, timing).
   • Look at the course's earlier story.py files first, and at references/examples/story-002-banana.py: a complete 44-beat choreography with track (keyframes with arcs), eat (to the mouth, then shrink), a MOUTH point, handovers and rings. Copy what fits. When a helper has been copied into a second lesson, move it into engine.py in a separate step.

   A skeleton:

   import engine as e
   
   def scene(t, timing):
       s = [beat["start"] for beat in timing["beats"]]
       man = {"id": "man", "sprite": "man-0", "x": e.LEFT_X, "y": e.CHARACTER_Y, "h": e.CHARACTER_H}
       woman = {"id": "woman", "sprite": "woman-0", "x": e.RIGHT_X, "y": e.CHARACTER_Y,
                "h": e.CHARACTER_H, "flip": True, "opacity": e.ease((t - s[2] + 0.8) / 0.8)}
       ax, ay = e.on_hand(man, e.HAND_R, 84)
       return [man, woman, {"id": "apple", "sprite": "apple", "x": ax, "y": ay, "h": 84}], []
   

   Conventions:

   • Use expression -1 (disappointed) while a character is empty-handed.
   • Draw a ring (marks) over an empty hand while 没有 is said about it.
   • Bring a character or object in just before the beat that names it.
   • Route objects so they never pass over a face.

9. Render: uv run <skill>/scripts/render.py <lesson dir> takes about 1–2 minutes. Fix any WARNING beat N: target not on screen it prints.

10. Verify before reporting.

• Open every contact-sheet-N.jpg: one tile per beat, mid-sentence. Check that the glowing target matches the sentence, objects sit in hands, rings sit on empty hands, faces match, and nothing overlaps the caption strip or leaves the frame.
• Movement happens in the pauses, so check a few handovers or arrivals: ffmpeg -v error -ss <seconds> -i lesson.mp4 -frames:v 1 /tmp/frame.png. Beat times are in timing.json.
• Fix story.py (or target/caption in lesson.json; both are read at render time) and re-render. If you change words, re-run step 7 first.
• Then tell the learner the lesson path, the new words, and the words reviewed.

Other requests

• "Review …" / "I forgot …": make a lesson with no new words, or one, built around the named words or that lesson's words. Mark them struggling if the learner says they are slipping.

• Vocabulary or progress: run the report and summarize it.

• Voices: the course's voice is in course.json: either qwen (local, for example Serena with a slow-teacher instruction) or edge (online, for example zh-CN-XiaoxiaoNeural at rate -25%).

  • Qwen speakers: <env> uv run --project <tts_project> --no-sync python -c "from qwen_tts import Qwen3TTSModel as M; print(M.from_pretrained('Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice').get_supported_speakers())". This loads the model.
  • edge-tts voices: uvx edge-tts --list-voices | grep zh-.

  Switching engine or voice is an edit to the voice block (see references/setup.md, step 3). It starts a fresh set of cached clips.

Rules

• A run writes only inside its lesson folder, the course's audio-cache/ and notes.md, plus new files under <course>/art/<set>/.
• Changes to <skill>/scripts, existing art or the tts project are a separate step, made when the learner asks.
• Imported lessons without a story.py are never re-rendered.