Bilibili / YouTube Audio to Subtitles
Use the codex Conda environment to transcribe in the original language. The default model is Qwen/Qwen3-ASR-1.7B + Qwen/Qwen3-ForcedAligner-0.6B, invoked via the official qwen-asr package with return_time_stamps=True. The ASR model itself only generates text; the ForcedAligner provides word/character-level timestamps. The SRT is organized by punctuation, pauses, roughly 6 seconds, and 42 characters. Boundaries come from the actual alignment results; duration is not equally divided, and no text is discarded.
--engine whisper preserves the native long-form audio mode of openai/whisper-large-v3-turbo, using return_timestamps=True in the pipeline without passing chunk_length_s. Experimental external chunking for Whisper once produced 5-minute subtitle segments and consecutive repetitions; it should not be enabled by default. The Qwen official SDK processes segments of up to 180 seconds by low-energy boundaries for alignment, automatically restoring global timing; this is a different implementation from Whisper's experimental 30-second pipeline chunking.
Qwen timestamps support Chinese, English, Cantonese, French, German, Italian, Japanese, Korean, Portuguese, Russian, and Spanish; for other languages, explicitly select Whisper. Do not assume alignment support just because ASR supports the language. By default, language is auto-detected. When the language is known, pass --language chinese or --english; common codes such as zh/en are also accepted.
Running and Output
First, record the user's current project's absolute path. Do not treat the skill installation directory as the project directory. Replace <skill-dir> with the directory containing this SKILL.md, and <project-dir> with the project path at invocation time; pass the URL as a single quoted argument.
conda run --no-capture-output -n codex python "<skill-dir>/scripts/bilibili_audio_subtitles.py" \
'https://www.bilibili.com/video/BV1w88n6uE1X/' \
--project-dir '<project-dir>'
The same command also accepts YouTube links, e.g., 'https://youtu.be/jNQXAC9IVRw'. Keep the original skill name and script name; existing $bilibili-audio-subtitles invocations can continue to work.
Bilibili audio is downloaded by yutto, saved as ~/Downloads/<BV-or-av-number>-p<N>.m4a; YouTube audio is downloaded by yt-dlp, saved as ~/Downloads/youtube-<video-ID>.m4a. The project directory outputs only one timestamped SRT with the subtitle body per run; only this recommended subtitle is delivered. No VTT, TXT, JSON, or model comparison files are delivered additionally. Necessary quality warnings are provided as brief suggestions for review but no additional delivery files are generated. Raw model text/chunks and reproduction diagnostics are saved in a separate JSON in ~/.cache/bilibili-audio-subtitles/runs/; the run log provides the path. This is an internal diagnostic record, not an additional subtitle delivery file.
Existing audio is reused. When an SRT with the same name already exists, the new result automatically uses a filename that includes the model, script version, and run number, preserving the old version. Do not overwrite or delete old subtitles. After retrying, explicitly point the user to the recommended SRT version, not just list all paths.
Common options:
--audio '/absolute/path/to/audio.m4a': Transcribe local audio; do not pass a URL at the same time.--language chinese: Explicitly specify when the language is known; defaults to auto-detect.--engine qwen|whisper: Defaults to Qwen; Whisper is used for comparison, speed priority, or languages not supported by Qwen alignment.--project-dir/--download-dir: Override project output directory or Downloads.--device auto|cpu|cuda:0: Defaults to CUDA priority; do not automatically alter system drivers.--download-only: Only download; continue later with--audio.--output-name 'filename': Custom SRT filename, without path or extension.--diagnostics-dir '/absolute/path': Override internal raw results and diagnostics directory.--auth-file '/absolute/path': Only for user-provided Bilibili/yutto authentication file.--cookies-file '/absolute/path': Only for user-provided YouTube Netscape Cookie file; do not scan browsers.--max-cue-seconds 12: Alert threshold for overly long subtitles; default is 12 seconds.--limit-seconds 30: For user-requested segments or smoke tests; filename is marked sample, and report explicitly states it is not a full transcription.--threads 8/--batch-size 1: Control threads and inference batch size.--engine whisper --chunk-length-seconds 30: Only enable when there is a clear reason and experimental risks are explained; Qwen does not accept this parameter.
Supports Bilibili standard /video/BV…, /video/av… links and bare BV/av numbers. Preserves ?p=N specified parts; removes tracking parameters. Only processes one part by default; call per part for multiple parts. Short Bilibili links are resolved to standard links first.
YouTube supports youtube.com/watch?v=…, youtu.be/…, shorts/…, live/…, and embed/… (including www, m, music subdomains). Unify to a single video standard link, removing tracking, start-at, and playlist parameters, and explicitly use --no-playlist; playlist-only links are directly rejected. Download selects bestaudio, extracts to M4A via FFmpeg; do not download platform subtitles, covers, or videos as deliverables. Do not scan browsers, print cookies, or automatically expand to an entire collection.
On Download Failure, Check for Updates First
This rule applies to both platforms simultaneously. When an actual download fails, or the downloader returns success but without valid audio, the script first checks the corresponding downloader package for a compatible stable version: check yutto for Bilibili, check yt-dlp for YouTube. Use the current codex Python's pip index versions <package> --json to compare local and package index versions, respecting existing pip mirror configuration.
If a newer version exists, the script upgrades only that downloader package and its necessary dependencies in that environment, then retries the download exactly once. Use yt-dlp[default] for yt-dlp, keeping YouTube EJS dependency matching. If there is no newer version, the query fails, the upgrade fails, or the second download still fails, save the original error and update event, then stop; do not retry indefinitely or automatically switch to nightly/master. Only after checking for package updates, check the real error for network, login, video removed, regional restrictions, and JS runtime.
Updates require network and environment write permissions. Request necessary permissions for this run when restricted; do not replace system Python or update transcription models or PyTorch for download issues. A failed update query does not mean it is already the latest version. If an audio file is known to be corrupt, clearly report it and preserve it; do not overwrite directly. New invalid audio from a failed retry also keeps an independent name.
Structural Check and Quality Check
Both checks must be reported separately:
- Structural check: Full audio decoding duration match, non-empty subtitles, timestamps finite, ordered, valid boundaries, and not exceeding transcription duration. On failure, preserve original diagnostics; do not generate an SRT.
- Quality check: Subtitles exceeding 12 seconds, single entry exceeding 120 characters, text density exceeding 25 characters/second, lasting at least 8 seconds but below 0.5 characters/second, single entry repeating words/phrases at least 5 times, at least 3 consecutive identical subtitle entries. Timestamps needing correction are also flagged for review. Thresholds and alert details are written to diagnostics.
Exit code 0 means structural check passed and no significant quality anomalies found; exit code 2 means SRT was generated but requires review; exit code 1 means run or structural failure. Do not claim exit code 2 as a successful check. Quality rules are heuristic hints, not proof of voice content correctness; false positives must be judged against the audio. Do not arbitrarily increase thresholds, delete text, re-segment by equal duration, or retry indefinitely to suppress alerts.
When quality anomalies exist, provide the alert time window and diagnostics path. If anomalies come from explicit Whisper external chunking, retry once with --engine whisper native mode, preserving both runs' output; if still anomalous, state that review is needed; do not automatically switch models or retry indefinitely. Terminology corrections are handled separately; do not globally replace words like "订单流" or "鼠标" based on guesswork. Even without alerts, refer to it only as a machine transcription draft, not as a sentence-by-sentence proofread subtitle.
Diagnostics JSON preserves raw text/chunks, normalized segments, structural status, quality status and alerts, correction records, recommendation status, script version, transcription mode, generation parameters, model GenerationConfig and revision, library versions, device, duration, and inference time. Also record source platform, downloader versions, update checks, and retry events. Qwen additionally preserves the raw ASR string before SDK post-processing, complete alignment units, detected language, zero-duration unit count, alignment model revision, and segmentation configuration; repetitions and near-generation-length-limit in raw output also trigger review, do not let SDK deduplication obscure anomalies. Zero-duration timestamps in quantized timestamps are merged into adjacent subtitles with text preserved; do not fabricate word-level duration. If an entire segment has no valid duration or text does not match, preserve raw results and fail. Failed downloads or transcriptions also save independent diagnostics for reproduction and comparison.
Environment
Check Python in the codex environment, platform-specific downloaders, torch, transformers, accelerate, numpy, qwen-asr, and ffmpeg/ffprobe in PATH. Install missing dependencies only into that environment:
conda run --no-capture-output -n codex python -m pip install yutto 'yt-dlp[default]' 'qwen-asr==0.0.6' numpy
Qwen tooling pins Transformers 4.57.6 and accelerate 1.12.0; avoid upgrading to Transformers 5 or installing vLLM for this task. Keep any existing CUDA PyTorch. Verified environment on this machine: Python 3.12, qwen-asr 0.0.6, yutto 2.3.1, Transformers 4.57.6, PyTorch 2.14.1+cu130; under CUDA, Qwen uses bfloat16 and SDPA; FlashAttention compilation is not required. To rebuild the local GPU environment:
conda run --no-capture-output -n codex python -m pip install 'torch==2.14.1+cu130' --index-url https://download.pytorch.org/whl/cu130
If the shell cannot find conda, locate its absolute path. This machine can use /home/leev/miniforge3/bin/conda or /home/leev/miniforge3/envs/codex/bin/python. If GPU detection fails in a restricted sandbox, re-detect in an environment with access to NVIDIA devices first; do not directly assume driver corruption or switch to a CPU-only build. Only use matching CPU PyTorch on machines that truly have no GPU.
YouTube also requires yt-dlp's default EJS support dependencies and a supported JS runtime. The script searches for Deno, Node, Bun, or QuickJS in PATH and explicitly passes it to yt-dlp; this machine has verified Node 24 with yt-dlp 2026.08.19. If no runtime is found or versions are incompatible, report explicitly; do not repeatedly install other packages due to download failures.
The first run downloads models to the standard Hugging Face cache, respecting HF_HOME / HF_HUB_CACHE; if the full cache exists and the network is unstable, set HF_HUB_OFFLINE=1. For Bilibili, only use yutto to download; disable video, danmaku, platform subtitles, covers, and chapter information. For YouTube, only use yt-dlp to download audio. On download failure, first apply the update rule, then report the real error; on transcription failure, preserve the audio. When permissions are restricted, request necessary directory and GPU execution permissions.
Regression Verification
Fixed example: BV1w88n6uE1X. Run the full default Qwen transcription on the same audio; when comparison is needed, use --engine whisper, preserve both runs' diagnostics, and invoke:
python -m unittest discover -s '<skill-dir>/scripts' -p 'test_*.py'
python '<skill-dir>/scripts/check_regression.py' \
--candidate '<new-cache-diagnostics-JSON>' --baseline '<old-JSON>'
Key time windows are 00:43.240–05:46.060 and 08:10–08:33. Measured: Qwen produces 282 segments, longest 5.76s; Whisper native produces 438 segments, longest 5.18s; no overly long or repetitive alerts in either window. Do not use fixed segment counts as pass conditions for other environments. In old results with 117 segments, a 302.82s subtitle and 23 consecutive "好" entries should be flagged by new quality checks for review. Regression pass only proves that known segmentation and repetition issues have not regressed; semantics and terminology still require verification against the audio.
YouTube regression includes watch/short link/Shorts routing, downloading audio of only a single video, checking for updates on failure, retrying at most once after upgrade, and stopping when there is no update or query fails. Measured: ~19s audio for jNQXAC9IVRw is fully downloaded, Qwen generates a 7-segment SRT (Whisper 4 segments); a video that is no longer available also tested update checks and failure diagnostics paths.
Model selection, measured time, and limitations are in references/model-evaluation.md; read it when comparing models or re-evaluating. The current default biases toward Chinese terminology recognition quality; do not claim it is superior to Whisper for all languages or all audio.
Implementation references: yutto official project, yt-dlp official project, Qwen3-ASR official project, Whisper model description.