Video-to-Text — Douyin / Local Video → Text → Content Intelligence (Full Loop in One Command)
Douyin share link → automatic download → speech-to-text → structured Markdown → viral breakdown, all in one command. No need to manually save videos, no third-party API key required (defaults to local faster-whisper), and privacy stays on your machine.
📁 Install:
hermes skills install jiawood2006/hermes-skills/skills/video-to-textor./install.sh(one-click install + global commandvtt) → Scripts are in~/.hermes/skills/utilities/video-to-text/scripts/.
Trigger Conditions
Use this skill when the user sends:
- A Douyin share link (v.douyin.com / www.douyin.com/video/…) and asks to "transcribe / extract text / analyze / break down viral content / view content"
- A Bilibili link and asks to "see video info"
- A local video file (.mp4 / .mov / .m4a) and asks to "speech-to-text / subtitles / summary / analysis"
Usage Steps
1. Douyin Link → Fully Automated (Default Behavior)
# One command: metadata + auto download + local transcription
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py "https://v.douyin.com/xxxx/"
# Full pipeline + LLM summary + viral structure breakdown
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py "https://v.douyin.com/xxxx/" --all
# Metadata only (no download, no transcription)
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py "https://v.douyin.com/xxxx/" --meta-only
# Switch transcription engine to SiliconFlow SenseVoice (better for spoken Chinese/noisy audio, requires SILICONFLOW_API_KEY, fails back to local)
SILICONFLOW_API_KEY=sk-xxx python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py "https://v.douyin.com/xxxx/" --engine sensevoice
Technical chain: curl resolves short link to real video ID → Playwright listens to aweme/detail API to get metadata + watermark-free play_url → auto download → extract audio → transcribe.
2. Local Video File → Transcription + Content Intelligence
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 # just transcribe
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --summary # + content summary
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --analyze # + viral structure breakdown
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --engine sensevoice # switch to SenseVoice
3. Bilibili Link → Metadata (Zero-dependency Public API)
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py "https://www.bilibili.com/video/BV1..."
Outputs: title/author/duration/views/likes/description; for transcription, download the video first and run on the local file.
4. Output Artifacts (Structured Markdown)
A file named <title>_transcript.md is saved by default, ready for Obsidian or any knowledge base:
---
platform: douyin
url: https://v.douyin.com/xxx
title: Video Title
author: Author Name
duration_s: 42
digg: 12345
date: 2026-09-04
tags: ["tag1", "tag2"]
---
# Video Title
| Field | Value |
|:---|:---|
| duration_s | 42 |
...
## Full Transcript
[00:00] Hello everyone today I will...
[00:05] This product...
5. Output Subtitle Files (SRT / VTT, with Real Timestamps)
When you want to edit a video, upload subtitles to YouTube/Bilibili, or do sentence-by-sentence proofreading, directly generate subtitle files:
# Transcript + accompanying SRT subtitle (most common)
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --subtitles srt
# Output both SRT and VTT
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --subtitles srt,vtt
# Subtitles only, no transcript
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --format srt
- Outputs
<title>_transcript.srt(compatible with CapCut / Premiere / YouTube) or.vtt(compatible with web players) - Chinese defaults to Simplified + punctuation, and is segmented by sentence (no subtitle spans more than ~18 characters)
- Segmentation threshold adjustable via environment variables:
VTT_MAX_CHARS=18(characters per subtitle),VTT_MAX_DUR=7(maximum seconds per subtitle)
6. Specify Transcription Language
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py english_video.mp4 --asr-lang en
Defaults to zh (Mandarin). Chinese subtitles additionally inject a prompt to guide Simplified output.
7. Analyze an Existing Transcript
python3 ~/.hermes/skills/utilities/video-to-text/scripts/analyze.py transcript.md --all
--summaryoutputs: one-sentence summary / key points / quotable lines / target audience / takeaways--analyzeoutputs: opening hook (0-10s) / pace-structure ratio / emotional curve / selling points / CTA / reusable patterns--format txtoutputs plain text (legacy format compatibility);--out-dirspecifies output directory
Validation (Self-test after Installation)
# Quick self-test with a local file: no video? Use macOS built-in speech synthesis
say -v "Ting-Ting" "Hello everyone, this is a test voice" -o /tmp/t.m4a # Linux users can skip
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py /tmp/t.m4a --out-dir /tmp/vtt_test
# Expected: /tmp/vtt_test contains *_transcript.md with text near "test voice"
# Self-test subtitle function (should get Simplified Chinese with punctuation, sentence-segmented SRT)
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py /tmp/t.m4a --subtitles srt --out-dir /tmp/vtt_test
# Expected: /tmp/vtt_test contains *_transcript.srt with --> timestamp lines
Dependencies (Installed on First Use)
pip3 install faster-whisper # default local transcription (first download of tiny model ~75MB)
# macOS has avconvert built-in; Linux needs ffmpeg
# Douyin parsing requires playwright + chromium (used internally by douyin_extract)
# Content summary / viral breakdown requires an LLM key (LLM_API_KEY environment variable or ~/.deai_writer.conf)
# SenseVoice engine optional: SILICONFLOW_API_KEY (SiliconFlow, free tier available)
Known Pitfalls
- duration_ms=0 false negative: SSR metadata duration may be 0 even though it's a video. If there is a download URL, download first and verify; do not immediately classify as image-only.
- play_addr URL expiry: Video URL resolved via SSR may return 404. In that case, the download fails and the tool prompts you to provide the file manually (automatic graceful degradation, no interruption).
- Image-only content: duration=0 and downloaded file <1MB is an image post (no audio); only metadata is output, no fabricated content.
- Whisper transcription is slow: On an Intel Mac, a 3-minute video takes about 5 minutes; automatic
language='zh'+ VAD silence filtering are enabled. For faster / more accurate results, use--engine sensevoice. - Chinese defaults to Traditional (fixed): Whisper often outputs Traditional Chinese with little punctuation. This skill injects an
initial_promptby default to guide Simplified + punctuation. To customize, set the environment variableWHISPER_INITIAL_PROMPT(set to empty to disable the guidance). - Accuracy depends on model size: Default is
WHISPER_MODEL=tiny(~75MB, fast but error-prone). For usable transcription quality, switch tosmall/medium:
The model will be downloaded on first use (for users in China, it is recommended toWHISPER_MODEL=small python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --subtitles srtexport HF_ENDPOINT=https://hf-mirror.com). - Subtitle segmentation granularity is adjustable: Default splits by punctuation / 18 characters / 7 seconds. Use environment variables
VTT_MAX_CHARSandVTT_MAX_DURfor finer or coarser segmentation. - SenseVoice path has no real timestamps: This engine does not return timing information; duration is estimated by speaking speed, so subtitle timelines are approximate. For precise subtitles, use faster-whisper.
- Analysis requires an LLM key:
--summary/--analyzeneedLLM_API_KEY. If not configured, only transcription is performed (clear degradation, no error thrown).