Communitygithub.com

ysyecust/lecture-to-notes

Use when users provide YouTube, Bilibili, or X/Twitter lecture URLs and want reader-first Chinese LaTeX/PDF notes with source-faithful claims, fluent authored prose, and verified teaching figures, especially requests phrased as lecture notes, 视频转PDF, 课程笔记, 讲义, YouTube笔记, B站笔记, X/Twitter lecture notes, or BV号; do not produce transcript dumps, quota-padded prose, or screenshot galleries.

lecture-to-notes 是什么?

lecture-to-notes is a Claude Code agent skill that use when users provide YouTube, Bilibili, or X/Twitter lecture URLs and want reader-first Chinese LaTeX/PDF notes with source-faithful claims, fluent authored prose, and verified teaching figures, especially requests phrased as lecture notes, 视频转PDF, 课程笔记, 讲义, YouTube笔记, B站笔记, X/Twitter lecture notes, or BV号; do not produce transcript dumps, quota-padded prose, or screenshot galleries.

兼容平台✓Claude Code✓Codex CLI~Cursor
npx skills add https://github.com/ysyecust/lecture-to-notes/tree/HEAD/skills/lecture-to-notes

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

Lecture to Notes

Convert a YouTube, Bilibili, or X/Twitter lecture video into a complete, compilable .tex note set and a rendered PDF.

Dependencies

Check before starting (use which). Prompt the user to install any missing tools.

ToolRequiredPurpose
yt-dlpAlwaysVideo/subtitle/metadata download (supports YouTube + Bilibili + X/Twitter). Check yt-dlp --version: a build older than ~90 days fails on current Bilibili and YouTube pages
ffmpegAlwaysFrame extraction, audio extraction
xelatexAlwaysLaTeX compilation (TeX Live + CTeX for Chinese)
magickAlwaysFrame montage and contact sheets
ffprobeAlwaysDuration and frame height (ships with ffmpeg)
pdftotextAlwaysRendered-page checks in verify_notes.py (poppler)
python3AlwaysInstalled helper scripts and local ASR support
sherpa-onnx + X ASR modelOptional zh/en ASRFast local Chinese/English transcription with token timestamps
Whisper backendASR fallback / non-zh-en / no-CCtranscribe_whisper.py picks mlx-whisper (macOS arm64) → faster-whisper → openai-whisper
rapidocr-onnxruntimeBilibili / any burned-in subtitlesocr_hardsubs.py reads the subtitle band and overlay geometry
Pillow + numpyAlwaysframe_filter.py overlay and composite-panel crops, talking-head scores

Install the Python side with pip install rapidocr-onnxruntime Pillow numpy plus one transcription backend: pip install mlx-whisper on Apple silicon, otherwise pip install faster-whisper (or openai-whisper as the slowest fallback). /ABSOLUTE/PATH/TO/lecture-to-notes/assets/smart_crop.py remains an optional experiment.

LaTeX package check (do NOT skip — which xelatex alone is insufficient)

which xelatex passing does not mean the required LaTeX packages are installed. On minimal TeX installs (e.g. MacTeX Basic, TeX Live scheme-basic), the binary exists but ctex, tcolorbox, and other packages are missing. This causes silent failures: long Chinese lines overflow \textwidth (no CJK line breaking) or compilation aborts with File '...' not found.

Check required packages before starting:

MISSING=0
for pkg in ctex tcolorbox environ trimspaces listings hyperref booktabs float subcaption etoolbox; do
  if ! kpsewhich "$pkg.sty" >/dev/null 2>&1; then
    echo "❌ Missing LaTeX package: $pkg"
    MISSING=1
  fi
done
if [ "$MISSING" -ne 0 ]; then
  echo "Install missing packages:"
  echo "  tlmgr install ctex tcolorbox environ trimspaces etoolbox"
  echo "Or install full TeX distribution:"
  echo "  macOS:  https://www.tug.org/mactex/  (~4 GB, includes everything)"
  echo "  Linux:  sudo apt install texlive-full"
fi

If ctex cannot be installed (e.g. due to l3kernel version conflicts on minimal installs), the notes-template.tex includes a fallback that uses XeTeX's built-in CJK line breaking (\XeTeXlinebreaklocale "zh") with system fonts — no ctex or xeCJK needed.

Cookies and yt-dlp freshness

Every supported platform can reject an unauthenticated download, and the two failures look different:

SymptomPlatformMeaning
Sign in to confirm you're not a botYouTubebot detection
HTTP Error 412 on probe or downloadBilibilianti-scraping check
WARNING: Your yt-dlp version (...) is older than 90 daysanyextractors predate the site's current markup

Fix the version first, because a stale extractor produces the same errors that cookies fix: brew upgrade yt-dlp, pipx upgrade yt-dlp, or pip install -U yt-dlp, then confirm with yt-dlp --version. Homebrew disables the self-updater, so yt-dlp -U reports success without changing anything.

Then supply cookies. video_source.py probe takes them directly:

python3 "/ABSOLUTE/PATH/TO/lecture-to-notes/assets/video_source.py" probe \
  --cookies-from-browser chrome "<URL>" > metadata.json

Use --cookies FILE instead when the browser profile is unavailable, for example on a headless host. A probe that fails for an authentication reason now names the flag to retry with, so treat that message as the next action rather than a dead end.

The rest of the workflow calls yt-dlp directly. Rather than editing every command, write a yt-dlp config file once and point XDG_CONFIG_HOME at it, so downloads, subtitle fetches, and thumbnail fetches all pick the cookies up:

mkdir -p "$WORKDIR/.xdg/yt-dlp"
printf -- '--cookies-from-browser chrome\n' > "$WORKDIR/.xdg/yt-dlp/config"
# then prefix every later yt-dlp call in this lecture's workdir:
XDG_CONFIG_HOME="$WORKDIR/.xdg" yt-dlp --no-playlist ... "<URL>"

Browser cookies also unlock Bilibili 1080P+ for logged-in accounts, so the same setup improves frame quality.

Goal

Produce a professional Chinese lecture note from a YouTube, Bilibili, or X/Twitter video URL. The output must:

  • use the video's actual teaching content, not just subtitle transcription
  • place the video's original cover image on the front page
  • include selected full-frame teaching figures chosen by contact-sheet review; when the video places a camera feed and slides side by side, each figure keeps one measured panel so the slide text stays readable
  • let a capable first-time reader understand the question, mechanism, evidence, consequence, and boundary without decoding the transcript
  • achieve source-fit information density — every figure, box, formula, and paragraph earns its space
  • be structurally organized with \section{} / \subsection{}
  • end with a synthesis section combining speaker's conclusions and your own distillation
  • be a complete .tex from \documentclass to \end{document}
  • compile successfully to PDF

Non-negotiable quality bar (Codex / production mode)

STOP and do not claim completion unless every item below is true. Skeleton outlines, image-heavy PDFs with thin prose, or missing intermediate artifacts are automatic failures.

Mandatory workdir artifacts (must exist on disk before saying "done")

FileRequired content
metadata.jsonFrom video_source.py probe (or equivalent full dump)
audio.srtFinal subtitle track used for writing (manual CC, cleaned auto, hard-sub OCR, X ASR, or Whisper)
audio_corrected.srtCopy of final track, or glossary/LLM-corrected track when local ASR was used
hardsub_ocr.srtocr_hardsubs.py extract output when ocr_hardsubs.py detect reported burned-in subtitles; absent otherwise
bands.jsonOverlay geometry (navigation strip, subtitle band) from ocr_hardsubs.py detect --geometry or frame_filter.py bands
layout.jsonPanel layout from frame_filter.py layout over the dense sample; always measured, composite: false records a single-picture video
cover.jpgFront-page cover
video.mp4Source for frames (may omit only if user forbids download and provides frames)
frames/Dense sample, default 1 frame / 15s
frame_scores.jsonframe_filter.py score over the dense sample; mandatory when the host cannot show you images
figures/Selected figure assets with semantic names (fig_01_topic.jpg, …), overlays cropped via bands.json, composites cropped to one panel via layout.json
figure_manifest.tsvHeader figure\tframe\tstart\tend\ttopic\tpanel — one row per figure image; panel names the layout.json panel it keeps (main, left, …) or full
figure_verification.txtFull stdout of verify_figures.py over all manifest timestamps
lecture_profile.jsonReader and source-fit profile; required fields are described below
teaching_atoms.tsvHeader atom\tstatus\tevidence — every teaching atom mapped to the notes
numerical_claims.tsvGenerated by extract_claims.py extract, filled by extract_claims.py check --write; header-only is valid when the lecture contains no numerical claims
notes.texComplete Chinese lecture notes
notes.pdfTwo-pass xelatex output
verify_notes.txtFull stdout of verify_notes.py, ending in OVERALL PASS

Source-fit profile and reader contract (hard)

After subtitle correction and contact-sheet review, create lecture_profile.json before outlining the notes:

{
  "mode": "technical-slide",
  "audience": "capable first-time reader",
  "central_question": "What should the reader be able to explain after reading?",
  "reader_outcome": "A concrete capability, decision, or mental model",
  "visual_teaching_atoms": 24,
  "formula_teaching_atoms": 18
}

Choose exactly one mode:

  • technical-slide: slides, board work, formulas, code, or diagrams carry most teaching content;
  • conceptual-talk: a talk, interview, or discussion carries most content through claims, examples, and reasoning rather than visual mechanisms;
  • mixed: both modes contribute substantial teaching content.

Count visual_teaching_atoms and formula_teaching_atoms from the source, not from the draft. A visual atom is a distinct source visual that helps teach a point. A formula atom is a source equation or derivation worth preserving. Do not inflate these counts to justify more assets, and do not classify a conceptual talk as technical merely to trigger quotas.

For every mode, the hard gates are source fidelity, complete teaching-atom coverage, traceable numerical claims, clear source attribution, and a coherent reader path. The numeric density gates below are source-fit backstops. They never authorize invented equations, low-value talking-head screenshots, repetitive boxes, or synonym padding.

Density and structure gates (source-adaptive)

For a lecture of duration $T$ minutes (from metadata):

Gate vs target (do not confuse): the numbers below are hard gates (fail = no delivery). A high-quality Codex-class note often lands ~25–40% above the CJK gate; that is a target, not a second hard floor. Raising the hard floor to the measured gold value encourages padding, not more teaching atoms. (Validated on CS336 L3: $T\approx 89$ → gate CJK 6246 / figs 25; gold ~8035 CJK / 27 figs / 31 pages.)

  1. Chinese character count in the notes.tex body (CJK unified ideographs only):

    • technical-slide: at least $\max(5000,\ \mathrm{round}(70\times T))$;
    • mixed: at least $\max(3500,\ \mathrm{round}(55\times T))$;
    • conceptual-talk: at least $\max(2500,\ \mathrm{round}(45\times T))$. Do not count English jargon, LaTeX commands, or captions alone. These are completeness floors, not invitations to repeat the same idea.
  2. Figures: include every distinct visual teaching atom that materially improves understanding, up to the technical target $\max(20,\ \mathrm{round}(T/3.5))$. A conceptual-talk with four real visual atoms should contain four verified figures, not twenty talking-head frames. A slide-led lecture with thirty distinct mechanisms must not stop at twenty.

  3. Sections for $T\ge 60$: use at least 8 for technical-slide, 6 for mixed, and 5 for conceptual-talk, unless fewer reader questions produce a demonstrably clearer structure. Each major section answers one reader question and ends with \subsection{本章小结} that states the answer and prepares the next question.

  4. Judgment boxes: use boxes only when they separate a definition, background dependency, decision rule, or failure boundary from the main flow. The technical default is 12 for $T\ge60$; mixed and conceptual-talk have no box quota. Never split continuous reasoning into boxes merely to raise a count.

  5. Teaching atom shape (adapt, do not stamp out a template):

    • state what the reader needs to understand and why it matters;
    • explain the mechanism or reasoning in the order needed to follow it;
    • attach source evidence: a number, example, quotation-level paraphrase, formula, code, table, or verified figure as appropriate;
    • state the consequence or boundary when it changes interpretation. Use only the components that the source and the reader need. After every display formula with at least two symbols, add an immediate symbol explanation list.
  6. No outline-only sections. If a 5-minute span of the lecture introduces a distinct mechanism, number, or design choice, it must appear as its own subsection or a clearly labeled paragraph with evidence — not a bullet in a summary list.

  7. Coverage audit before compile (timeline): walk the timeline in 3–5 minute steps using the SRT; list any gap without corresponding prose or figure; fill gaps before xelatex.

  8. Teaching-atom checklist (topic coverage, hard for technical lectures): Before delivery, extract a lecture-specific atom list from SRT + slides (15–40 atoms). Each atom must map to a subsection, a labeled paragraph, or a figure+caption in notes.tex. Missing atoms → expand prose; do not "pass" on CJK alone. Example atoms for an LLM-architecture lecture: Pre/Post-Norm, LN vs RMSNorm, FLOPs≠runtime, SwiGLU, serial vs parallel block, RoPE, FFN ratio, head dim, width/depth, vocab size, dropout vs weight decay, z-loss, QK-Norm, Prefill vs Decode, MQA/GQA, sliding window / interleaved / hybrid attention. Persist the checklist as teaching_atoms.tsv (atom\tstatus\tevidence).

  9. Mechanism density (hard — L2 gold comparison 2026-07-12): CJK/fig/box gates alone are not enough. CS336 L2 first Grok pass: 5463 CJK / 8 formulas / 0 code (gate PASS, gold incomplete); after claim-driven补全: ~6450 CJK / 31 formulas / 3 code / 33 pages (matches Codex). Also require:

    • Display math: for source formula atoms, preserve at least $\min(N_{\mathrm{formula\ atoms}},\ \max(10,\ \mathrm{round}(T/4)))$ blocks (\[ ... \] or equation), each with an immediate symbol explanation list.
    • Code blocks: for source code or pseudocode, preserve at least $\min(N_{\mathrm{code\ atoms}},\ \max(1,\ \mathrm{round}(T/15)))$ lstlisting blocks (or verbatim + color), each with a prose walkthrough.
    • Tables: for multi-parameter comparisons or dimension specifications, preserve at least $\min(N_{\mathrm{table\ atoms}},\ \max(0,\ \mathrm{round}(T/30)))$ tabular blocks.

Platform-specific subtitle workflow

PlatformPreferred trackFallback
YouTubeManual CC (language preference: zh-Hans, zh, en)Auto-generated CC → Whisper ASR (if auto CC is too sparse or misaligned)
BilibiliBurned-in subtitle OCR via ocr_hardsubs.pyCC API (when available and authorized) → Whisper ASR
X/TwitterSometimes embedded CCWhisper ASR only

After obtaining the subtitle track, always run verify_subtitles.py to check timestamp alignment, misrecognized tokens (e.g. "attention" → "a tension"), and dropped lines.

Verification scripts (always run; fix all failures before delivery)

ScriptPurpose
verify_metadata.pyChecks metadata.json has all required fields
verify_subtitles.pyChecks SRT has at least 80% of expected lines, no gaps >15s
verify_figures.pyCompares every figure timestamp against the dense frame to confirm the timestamp is within the figure's start–end window
verify_notes.pyCompiles notes.tex, checks output page count, CJK character count, section count, figure count, box count; enforces all hard gates above; exits with OVERALL PASS only when every gate passes

Workflow summary (implement as sequential steps)

  1. Probe: video_source.py probe to get metadata, download options
  2. Download: video.mp4, cover image, subtitles
  3. Subtitle preparation: OCR, ASR, or CC extraction; correction and verification
  4. Frame extraction: Dense sample at 1 frame / 15s
  5. Layout analysis: frame_filter.py layout to determine panel geometry
  6. Frame scoring: frame_filter.py score for keyframe selection
  7. Contact-sheet review (human-assisted or LLM-reviewed with frame images): Select teaching figures, edit figure_manifest.tsv, verify with verify_figures.py
  8. Profile creation: Write lecture_profile.json based on source analysis
  9. Atom extraction: Create teaching_atoms.tsv from SRT + slides
  10. Note drafting: Write notes.tex following the goal, density gates, and atom checklist
  11. Compilation: xelatex notes.tex (two passes)
  12. Verification: Run all verification scripts; fix any failures
  13. Delivery: Confirm notes.pdf is readable and complete

Shell integration (recommended practice)

Set a workdir for the session and keep all intermediate artifacts there:

WORKDIR="/path/to/lecture-output/$(date +%Y%m%d_%H%M%S)"
mkdir -p "$WORKDIR"
cd "$WORKDIR"
export XDG_CONFIG_HOME="$WORKDIR/.xdg"

Use absolute paths for asset scripts throughout to avoid $PATH resolution surprises: /ABSOLUTE/PATH/TO/lecture-to-notes/assets/video_source.py.

Final note

This skill is designed to produce the highest-quality Chinese lecture notes from online video sources. It demands rigor at every step — from dependency checking and subtitle acquisition to figure selection, prose density, and verification. Follow the workflow sequentially, respect the hard gates, and never deliver before all verification scripts pass.

相关技能