Score Sync Guide (Score-Following Video)

Score-Following Guide (score-sync-guide)

Turn a MusicXML score + an audio recording into a video: on the left, a single-line horizontally scrolling score with a fixed playhead, red highlighting for sounding notes, and only the currently active parts shown; on the right, guided analysis cards and the movement's formal structure. The entire workflow runs on native Windows, no WSL, Node, or Playwright required.

This approach comes from the actual production of a Mahler Symphony No. 6 (four movements, 83 minutes). The onset alignment algorithm is ported from tofucurd/score-sync-video (MIT), and several player design ideas (fixed cursor, stable part sets, two-level form structure, render-by-block resumability) also draw from it. This skill is its Windows localization and extension (analysis cards, recording alignment workflow, cross-checking tools).

Read These Rules First

1. Original files stay untouched. The score and recording go in the project root, read-only. All score corrections go into scores_fixed/corrections.json, generated by tools/fix_mxl.py to produce a copy, with justification (original score page, screenshot).
2. Ask the user before changing notes. Community MusicXML and printed scores are often different versions; "inconsistency" is not necessarily an error. Only fix obvious import errors.
3. Do not upload commercial recordings to any third-party service, and do not include them in the repository.
4. New Python packages are installed by the user: Just list the package name and command; do not run pip install yourself.
5. Every factual claim in an analysis card must have a basis: score data (specify measure number) or verifiable literature (references.json + superscript). If unverifiable, don't write it or mark it as "unverified." Automatic harmony analysis is unreliable for large orchestral scores; use only as a hint.
6. If the user cannot read music, verification relies on listening: listening against the score (via collected disputed excerpts) is more reliable than listening for a "click" sound.
7. Run fine onset alignment first, then ask the user to listen, then add pins. Doing it the other way (asking the user to report timestamps one by one) wastes effort on within-bar offsets that fine alignment can usually resolve automatically.
8. The user's personal opinions are only embellishments; do not fabricate or exaggerate views on their behalf.

Workflow Overview

Step	What to do	Main files	Details
0	Create project: copy template/ to project root, write manifest.json	template/, examples/mahler6/manifest.json	workflow
1	Score health check: parts, repeats, tempo, rehearsal marks, placeholder characters	analyze.py, check_all.py	workflow
2	Corrected copy, Chinese part names and merges	tools/fix_mxl.py, tools/build_labels.py, labels.json	workflow
3	Bar-by-bar DTW → immediately run fine onset alignment	align/align.py, align/ssv_align.py	alignment
4	Listen-check: disputed excerpts; only for places still wrong after fine alignment, add pins and local re-alignment, then re-run fine alignment	demo/make_disputes.py, align/manual_regions.json, align/refine_local.py	alignment
5	Form, motives, analysis cards	form/, motifs.json, cards/	cards
6	Cross-check cards against score, review dwell times	tools/cards_xcheck.py, tools/cards_review.py	cards
7	Layout, player page, test clip	demo/build_score.py, demo/player.html, demo/render_frames.py	rendering
8	Full video rendering and assembly	demo/render_film.py	rendering

Gotchas on Windows (encoding, processes, ports, batch newlines, ffmpeg sample rate, etc.) are collected in windows-notes — skim it before starting.

Shortest Path (One Movement)

# System Python (with verovio, music21, librosa, numba installed)
$py = "python"
& $py -X utf8 analyze.py                                   # Health check: out/<score>_structure.json, report.txt
& $py -X utf8 align/align.py --movement 1                  # ① Bar-by-bar DTW → align/m1_measures.json (opening anchor in align/anchors.json)
& $py -X utf8 align/ssv_align.py 1                         # ② Immediately run fine onset alignment → align/m1_ssv_chain.json
& $py -X utf8 align/make_checklist.py                      # Listen-check checklist
& $py -X utf8 demo/make_disputes.py                        # ③ Disputed excerpts: listen against score
# ④ Only for still-wrong places, add pins/beats to align/manual_regions.json, if needed:
& $py -X utf8 align/refine_local.py --movement 1
& $py -X utf8 align/ssv_align.py 1                         # ⑤ Re-run fine alignment if pins or measure table changed
& $py -X utf8 demo/build_score.py --movement 1 --range 1-40 --out demo/build   # Layout (positions start at 1)
& $py -X utf8 demo/render_frames.py --data build/data.js --out demo/test.mp4  # Test clip

Full film:

& $py -X utf8 demo/render_film.py plan --res 1920x1080 --fps 60 --name sample --single 4
& $py -X utf8 demo/render_film.py worker 1 --only 4        # Multiple workers can run simultaneously (number determines debug port)
& $py -X utf8 demo/render_film.py worker 2
& $py -X utf8 demo/render_film.py status
& $py -X utf8 demo/render_film.py assemble

For long renders, start the process detached from the current terminal (see the WMI section in windows-notes), otherwise closing the terminal or session will break the render.

Pre-Delivery Self-Check

• Timeline is strictly increasing, no missing measures; measures after unfolding repeats match play_order.
• Part names are all in Chinese, no "Unclassified."
• Suspicious items from cards_xcheck.py have been judged individually; all items in cards_review.py pass; after modifying cards, re-run build_refs.py.
• After changing measure tables or pins, re-run ssv_align.py (build_score.py uses fingerprints to catch outdated fine alignments).
• Check frame counts segment by segment in the final film; spot-check rendered frames visually.