Communitygithub.com

Video-to-Text — Content Intelligence for Short Video Platforms

Turn Douyin (TikTok China) links, Bilibili links, or local video/audio files into structured text, subtitles (SRT/VTT), summaries, and viral-content analysis — all in one command, with no third-party API key required outside the LLM.

What is Video-to-Text — Content Intelligence for Short Video Platforms?

This skill is a comprehensive video-to-text intelligence pipeline designed for AI agents on Hermes. It accepts a Douyin/TikTok share link, a Bilibili link, or a local video/audio file, and automatically downloads (for Douyin), transcribes speech into text using a local Whisper model or a cloud SenseVoice engine, and produces structured Markdown transcripts, SRT/VTT subtitle files with real timestamps, and optionally generates content summaries and viral-structure breakdowns via an LLM. The skill is fully offline-safe (default faster-whisper runs locally, no data leaves the machine), supports Chinese (Simplified with punctuation, sentence-segmented) and other languages, and integrates seamlessly with Hermes agent environments. It also falls back gracefully if the video is an image-only post or if the download URL is expired, and provides environment variables to tune subtitle segmentation and model size. The skill is ideal for content researchers, video analysts, and agent developers who need to extract written content and insights from short-form videos without manual steps.

Works with~Claude Code~Codex CLI~Cursor
npx skills add https://github.com/jiawood2006/hermes-skills/tree/HEAD/skills/video-to-text

Ask in your favorite AI

Open a new chat with this agent skill pre-loaded.

Documentation

Video-to-Text — Douyin / Local Video → Text → Content Intelligence (Full Loop in One Command)

Douyin share link → automatic download → speech-to-text → structured Markdown → viral breakdown, all in one command. No need to manually save videos, no third-party API key required (defaults to local faster-whisper), and privacy stays on your machine.

📁 Install: hermes skills install jiawood2006/hermes-skills/skills/video-to-text or ./install.sh (one-click install + global command vtt) → Scripts are in ~/.hermes/skills/utilities/video-to-text/scripts/.

Trigger Conditions

Use this skill when the user sends:

  • A Douyin share link (v.douyin.com / www.douyin.com/video/…) and asks to "transcribe / extract text / analyze / break down viral content / view content"
  • A Bilibili link and asks to "see video info"
  • A local video file (.mp4 / .mov / .m4a) and asks to "speech-to-text / subtitles / summary / analysis"

Usage Steps

1. Douyin Link → Fully Automated (Default Behavior)

# One command: metadata + auto download + local transcription
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py "https://v.douyin.com/xxxx/"

# Full pipeline + LLM summary + viral structure breakdown
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py "https://v.douyin.com/xxxx/" --all

# Metadata only (no download, no transcription)
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py "https://v.douyin.com/xxxx/" --meta-only

# Switch transcription engine to SiliconFlow SenseVoice (better for spoken Chinese/noisy audio, requires SILICONFLOW_API_KEY, fails back to local)
SILICONFLOW_API_KEY=sk-xxx python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py "https://v.douyin.com/xxxx/" --engine sensevoice

Technical chain: curl resolves short link to real video ID → Playwright listens to aweme/detail API to get metadata + watermark-free play_url → auto download → extract audio → transcribe.

2. Local Video File → Transcription + Content Intelligence

python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4                 # just transcribe
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --summary       # + content summary
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --analyze       # + viral structure breakdown
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --engine sensevoice  # switch to SenseVoice

3. Bilibili Link → Metadata (Zero-dependency Public API)

python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py "https://www.bilibili.com/video/BV1..."

Outputs: title/author/duration/views/likes/description; for transcription, download the video first and run on the local file.

4. Output Artifacts (Structured Markdown)

A file named <title>_transcript.md is saved by default, ready for Obsidian or any knowledge base:

---
platform: douyin
url: https://v.douyin.com/xxx
title: Video Title
author: Author Name
duration_s: 42
digg: 12345
date: 2026-09-04
tags: ["tag1", "tag2"]
---

# Video Title

| Field | Value |
|:---|:---|
| duration_s | 42 |
...

## Full Transcript

[00:00] Hello everyone today I will...
[00:05] This product...

5. Output Subtitle Files (SRT / VTT, with Real Timestamps)

When you want to edit a video, upload subtitles to YouTube/Bilibili, or do sentence-by-sentence proofreading, directly generate subtitle files:

# Transcript + accompanying SRT subtitle (most common)
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --subtitles srt

# Output both SRT and VTT
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --subtitles srt,vtt

# Subtitles only, no transcript
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --format srt
  • Outputs <title>_transcript.srt (compatible with CapCut / Premiere / YouTube) or .vtt (compatible with web players)
  • Chinese defaults to Simplified + punctuation, and is segmented by sentence (no subtitle spans more than ~18 characters)
  • Segmentation threshold adjustable via environment variables: VTT_MAX_CHARS=18 (characters per subtitle), VTT_MAX_DUR=7 (maximum seconds per subtitle)

6. Specify Transcription Language

python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py english_video.mp4 --asr-lang en

Defaults to zh (Mandarin). Chinese subtitles additionally inject a prompt to guide Simplified output.

7. Analyze an Existing Transcript

python3 ~/.hermes/skills/utilities/video-to-text/scripts/analyze.py transcript.md --all
  • --summary outputs: one-sentence summary / key points / quotable lines / target audience / takeaways
  • --analyze outputs: opening hook (0-10s) / pace-structure ratio / emotional curve / selling points / CTA / reusable patterns
  • --format txt outputs plain text (legacy format compatibility); --out-dir specifies output directory

Validation (Self-test after Installation)

# Quick self-test with a local file: no video? Use macOS built-in speech synthesis
say -v "Ting-Ting" "Hello everyone, this is a test voice" -o /tmp/t.m4a   # Linux users can skip
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py /tmp/t.m4a --out-dir /tmp/vtt_test
# Expected: /tmp/vtt_test contains *_transcript.md with text near "test voice"

# Self-test subtitle function (should get Simplified Chinese with punctuation, sentence-segmented SRT)
python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py /tmp/t.m4a --subtitles srt --out-dir /tmp/vtt_test
# Expected: /tmp/vtt_test contains *_transcript.srt with --> timestamp lines

Dependencies (Installed on First Use)

pip3 install faster-whisper        # default local transcription (first download of tiny model ~75MB)
# macOS has avconvert built-in; Linux needs ffmpeg
# Douyin parsing requires playwright + chromium (used internally by douyin_extract)
# Content summary / viral breakdown requires an LLM key (LLM_API_KEY environment variable or ~/.deai_writer.conf)
# SenseVoice engine optional: SILICONFLOW_API_KEY (SiliconFlow, free tier available)

Known Pitfalls

  • duration_ms=0 false negative: SSR metadata duration may be 0 even though it's a video. If there is a download URL, download first and verify; do not immediately classify as image-only.
  • play_addr URL expiry: Video URL resolved via SSR may return 404. In that case, the download fails and the tool prompts you to provide the file manually (automatic graceful degradation, no interruption).
  • Image-only content: duration=0 and downloaded file <1MB is an image post (no audio); only metadata is output, no fabricated content.
  • Whisper transcription is slow: On an Intel Mac, a 3-minute video takes about 5 minutes; automatic language='zh' + VAD silence filtering are enabled. For faster / more accurate results, use --engine sensevoice.
  • Chinese defaults to Traditional (fixed): Whisper often outputs Traditional Chinese with little punctuation. This skill injects an initial_prompt by default to guide Simplified + punctuation. To customize, set the environment variable WHISPER_INITIAL_PROMPT (set to empty to disable the guidance).
  • Accuracy depends on model size: Default is WHISPER_MODEL=tiny (~75MB, fast but error-prone). For usable transcription quality, switch to small/medium:
    WHISPER_MODEL=small python3 ~/.hermes/skills/utilities/video-to-text/scripts/vtt.py video.mp4 --subtitles srt
    
    The model will be downloaded on first use (for users in China, it is recommended to export HF_ENDPOINT=https://hf-mirror.com).
  • Subtitle segmentation granularity is adjustable: Default splits by punctuation / 18 characters / 7 seconds. Use environment variables VTT_MAX_CHARS and VTT_MAX_DUR for finer or coarser segmentation.
  • SenseVoice path has no real timestamps: This engine does not return timing information; duration is estimated by speaking speed, so subtitle timelines are approximate. For precise subtitles, use faster-whisper.
  • Analysis requires an LLM key: --summary / --analyze need LLM_API_KEY. If not configured, only transcription is performed (clear degradation, no error thrown).

Individual skills in this repo

This repo contains 1 individual skill — each has its own dedicated page.

Related Skills