Communitygithub.com

ComeOnOliver/skillshub

使用 OpenAI 的 Whisper——最先进的语音识别模型——进行音频转录。此技能涵盖本地 Whisper(Python)、faster-whisper(CTranslate2,速度快 4 倍)、whisper.cpp(针对 CPU 优化的 C++)以及 OpenAI Whisper API。包括字幕生成(SRT/VTT/JSON)、多语言转录、翻译成英语、说话人分离、词级时间戳,以及适用于播客、会议和视频字幕的生产管道模式。

skillshub 是什么?

这个技能为 AI 代理(如 Claude Code、Cursor、Codex)提供了全面的音频转文本解决方案,基于 OpenAI 的 Whisper 模型。它支持四种不同的运行时:原始的 Python Whisper 实现、使用 CTranslate2 加速(快 4 倍)的 faster-whisper、专为 CPU 优化的轻量级 whisper.cpp,以及无需本地 GPU 的 OpenAI Whisper API。该技能能够生成 SRT、VTT 和 JSON 格式的字幕,支持 99 种语言的自动检测和转录,并能将任何语言翻译成英语。它还通过集成 pyannote-audio 实现了说话人分离功能,可以识别“谁说了什么”。此外,该技能提供了生产级别的管道模式,用于处理播客、会议和视频字幕的批量转录,包括模型选择指南和生成词级时间戳的功能。用户可以根据可用硬件(CPU 或 GPU)和精度要求(从快速草稿到专业质量)灵活选择模型。

兼容平台~Claude Code✓Codex CLI~Cursor
npx skills add https://github.com/ComeOnOliver/skillshub/tree/HEAD/skills/TerminalSkills/skills/whisper

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

Whisper

概述

使用 OpenAI 的 Whisper——最先进的语音识别模型——进行音频转录。此技能涵盖本地 Whisper(Python)、faster-whisper(CTranslate2,速度快 4 倍)、whisper.cpp(针对 CPU 优化的 C++)以及 OpenAI Whisper API。包括字幕生成(SRT/VTT/JSON)、多语言转录、翻译成英语、说话人分离、词级时间戳,以及适用于播客、会议和视频字幕的生产管道模式。

说明

步骤 1:选择您的运行时

选项 A — OpenAI Whisper(原始 Python):

pip install openai-whisper
# 模型:tiny (39M), base (74M), small (244M), medium (769M), large-v3 (1.5G)

选项 B — faster-whisper(建议本地使用,速度快 4 倍):

pip install faster-whisper
# 使用 CTranslate2 — INT8 量化,在 CPU 上运行良好

选项 C — whisper.cpp(最适合 CPU,依赖最少):

git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp && make
# 下载模型
bash models/download-ggml-model.sh base.en

选项 D — OpenAI API(无需本地 GPU):

pip install openai
export OPENAI_API_KEY="sk-..."

步骤 2:基本转录

faster-whisper(推荐):

from faster_whisper import WhisperModel

model = WhisperModel("base", device="cpu", compute_type="int8")
# GPU:model = WhisperModel("large-v3", device="cuda", compute_type="float16")

segments, info = model.transcribe("episode.mp3", beam_size=5)
print(f"语言:{info.language}(概率:{info.language_probability:.2f})")

for segment in segments:
    print(f"[{segment.start:.2f}秒 → {segment.end:.2f}秒] {segment.text}")

OpenAI Whisper(原始):

import whisper

model = whisper.load_model("base")  # tiny, base, small, medium, large-v3
result = model.transcribe("episode.mp3")

print(result["text"])
for segment in result["segments"]:
    print(f"[{segment['start']:.1f}秒 - {segment['end']:.1f}秒] {segment['text']}")

whisper.cpp(CLI):

./main -m models/ggml-base.en.bin -f episode.wav -otxt -osrt -ovtt
# 输出:episode.txt, episode.srt, episode.vtt

OpenAI API:

from openai import OpenAI
client = OpenAI()

with open("episode.mp3", "rb") as f:
    transcript = client.audio.transcriptions.create(
        model="whisper-1",
        file=f,
        response_format="verbose_json",
        timestamp_granularities=["segment", "word"],
    )

print(transcript.text)
for segment in transcript.segments:
    print(f"[{segment.start:.1f}秒] {segment.text}")

步骤 3:字幕生成(SRT/VTT)

使用 faster-whisper 生成 SRT:

from faster_whisper import WhisperModel

model = WhisperModel("small", device="cpu", compute_type="int8")
segments, info = model.transcribe("episode.mp3", beam_size=5)

def format_timestamp(seconds):
    h = int(seconds // 3600)
    m = int((seconds % 3600) // 60)
    s = int(seconds % 60)
    ms = int((seconds % 1) * 1000)
    return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"

with open("episode.srt", "w") as f:
    for i, seg in enumerate(segments, 1):
        f.write(f"{i}\n")
        f.write(f"{format_timestamp(seg.start)} --> {format_timestamp(seg.end)}\n")
        f.write(f"{seg.text.strip()}\n\n")

print("已生成 episode.srt")

生成 VTT(用于 HTML5 视频):

with open("episode.vtt", "w") as f:
    f.write("WEBVTT\n\n")
    for i, seg in enumerate(segments, 1):
        start = format_timestamp(seg.start).replace(",", ".")
        end = format_timestamp(seg.end).replace(",", ".")
        f.write(f"{start} --> {end}\n")
        f.write(f"{seg.text.strip()}\n\n")

词级时间戳(用于卡拉 OK 风格字幕):

segments, info = model.transcribe("episode.mp3", word_timestamps=True)
for segment in segments:
    for word in segment.words:
        print(f"  [{word.start:.2f}秒 → {word.end:.2f}秒] {word.word}")

步骤 4:语言检测与翻译

# 自动检测语言
segments, info = model.transcribe("foreign_audio.mp3")
print(f"检测到:{info.language}(概率:{info.language_probability:.0%})")

# 强制指定语言
segments, info = model.transcribe("german.mp3", language="de")

# 翻译成英语(任何语言 → 英语)
segments, info = model.transcribe("german.mp3", task="translate")
for seg in segments:
    print(seg.text)  # 英语翻译

支持的语言: 99 种语言,包括 en、zh、de、es、ru、ko、fr、ja、pt、tr、pl、nl、ar、sv、it、hi 等。

步骤 5:说话人分离

将 Whisper 与 pyannote-audio 结合使用,实现“谁说了什么”:

from faster_whisper import WhisperModel
from pyannote.audio import Pipeline
import torch

# 转录
model = WhisperModel("small", device="cpu", compute_type="int8")
segments, info = model.transcribe("meeting.wav", beam_size=5)
whisper_segments = list(segments)

# 分离说话人(需要具有 pyannote 访问权限的 HuggingFace 令牌)
diarization = Pipeline.from_pretrained(
    "pyannote/speaker-diarization-3.1",
    use_auth_token="hf_YOUR_TOKEN"
)
diarization_result = diarization("meeting.wav")

# 合并:为每个 whisper 片段分配说话人
def get_speaker(start, end, diarization_result):
    """查找时间范围内的主要说话人。"""
    speakers = {}
    for turn, _, speaker in diarization_result.itertracks(yield_label=True):
        overlap_start = max(start, turn.start)
        overlap_end = min(end, turn.end)
        if overlap_start < overlap_end:
            duration = overlap_end - overlap_start
            speakers[speaker] = speakers.get(speaker, 0) + duration
    return max(speakers, key=speakers.get) if speakers else "未知"

for seg in whisper_segments:
    speaker = get_speaker(seg.start, seg.end, diarization_result)
    print(f"[{speaker}] [{seg.start:.1f}秒 → {seg.end:.1f}秒] {seg.text}")

步骤 6:批量处理与管道

转录目录中的所有剧集:

import os
from pathlib import Path
from faster_whisper import WhisperModel

model = WhisperModel("small", device="cpu", compute_type="int8")
episodes_dir = Path("episodes")
output_dir = Path("transcripts")
output_dir.mkdir(exist_ok=True)

for audio_file in sorted(episodes_dir.glob("*.mp3")):
    print(f"正在转录:{audio_file.name}")
    segments, info = model.transcribe(str(audio_file), beam_size=5)
    
    # 纯文本
    txt_path = output_dir / f"{audio_file.stem}.txt"
    with open(txt_path, "w") as f:
        for seg in segments:
            f.write(seg.text.strip() + "\n")
    
    # SRT
    srt_path = output_dir / f"{audio_file.stem}.srt"
    segments2, _ = model.transcribe(str(audio_file), beam_size=5)  # 重新迭代
    with open(srt_path, "w") as f:
        for i, seg in enumerate(segments2, 1):
            h, m = divmod(int(seg.start), 3600)
            m, s = divmod(m, 60)
            ms = int((seg.start % 1) * 1000)
            start_ts = f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"
            h2, m2 = divmod(int(seg.end), 3600)
            m2, s2 = divmod(m2, 60)
            ms2 = int((seg.end % 1) * 1000)
            end_ts = f"{h2:02d}:{m2:02d}:{s2:02d},{ms2:03d}"
            f.write(f"{i}\n{start_ts} --> {end_ts}\n{seg.text.strip()}\n\n")
    
    print(f"  → {txt_path},{srt_path}")

步骤 7:模型选择指南

模型大小VRAM速度 (CPU)准确度最佳用途
tiny39M~1GB非常快低快速草稿、实时
base74M~1GB快中等适用于 CPU 的良好平衡
small244M~2GB中等好播客、清晰音频
medium769M~5GB慢非常好噪声音频、口音
large-v31.5G~10GB非常慢最佳生产质量

建议:

  • 仅 CPU,速度重要:tiny 或 base 配合 faster-whisper
  • 仅 CPU,准确度重要:small 配合 faster-whisper
  • 有 GPU:large-v3 配合 faster-whisper(float16)
  • 无本地计算:OpenAI API(whisper-1)

示例

示例 1:转录播客季并生成 SRT 字幕

用户提示:“转录 ./episodes/ 下所有 20 个 MP3 剧集,并生成纯文本转录和 SRT 字幕文件。我有一个 10GB VRAM 的 GPU。”

代理将:

  1. 通过 pip install faster-whisper 安装 faster-whisper。
  2. 使用 device="cuda" 和 compute_type="float16" 加载 large-v3 模型以利用 GPU。
  3. 创建 ./transcripts/ 输出目录。
  4. 循环遍历 ./episodes/ 中的所有 .mp3 文件,使用 beam_size=5 转录每个文件。
  5. 为每个剧集写入一个 .txt 文件(纯文本)和一个 .srt 文件(带正确格式的时间戳)。
  6. 报告每个剧集的检测语言和总处理时间。

示例 2:将德语采访翻译成英语并带有说话人标签

用户提示:“我有一个 45 分钟的德语采访录音文件 meeting.wav,有两个说话人。将其转录为英语并标记谁说了什么。”

代理将:

  1. 安装 faster-whisper 和 pyannote-audio(pip install faster-whisper pyannote.audio)。
  2. 使用 task="translate" 转录 meeting.wav,从德语音频获取英语文本。
  3. 运行 pyannote 说话人分离以识别说话人片段(需要具有 pyannote 模型访问权限的 HuggingFace 令牌)。
  4. 通过匹配时间范围将 Whisper 片段与分离结果合并,以分配说话人标签。
  5. 输出格式化的转录,在每个片段前加上 [Speaker 1] 和 [Speaker 2] 标签。

指南

  • 对于本地转录,使用 faster-whisper 而非原始 OpenAI Whisper;它通过 CTranslate2 的 INT8 量化速度快 4 倍,且使用更少内存。
  • 根据您的硬件选择模型大小:tiny/base 用于 CPU 速度,small 用于 CPU 准确度,large-v3 用于 GPU 生产质量。
  • 当使用 MP3/M4A 输入时,始终将音频转换为 WAV 或确保安装了 ffmpeg;Whisper 依赖 ffmpeg 解码非 WAV 格式。
  • 对于说话人分离,pyannote 管道需要接受模型条款的 HuggingFace 访问令牌;在尝试多说话人转录之前进行设置。
  • 生成 SRT 文件时,使用 beam_size=5 以获得更准确的片段边界;默认的贪婪解码可能会产生时间不均匀的字幕断点。

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

ComeOnOliver/skillshub

Create stunning, animation-rich HTML presentations from scratch or by converting PowerPoint files. Use when the user wants to build a presentation, convert a PPT/PPTX to web, or create slides for a talk/pitch. Helps non-designers discover their aesthetic through visual exploration rather than abstract choices.

ComeOnOliver/skillshub

Next.js 16+ 和 Turbopack — 增量打包、文件系统缓存、开发速度,以及何时使用 Turbopack 与 webpack。

ComeOnOliver/skillshub

SwiftUI architecture patterns, state management with @Observable, view composition, navigation, performance optimization, and modern iOS/macOS UI best practices.

ComeOnOliver/skillshub

See, Understand, Act on video and audio. See- ingest from local files, URLs, RTSP/live feeds, or live record desktop; return realtime context and playable stream links. Understand- extract frames, build visual/semantic/temporal indexes, and search moments with timestamps and auto-clips. Act- transcode and normalize (codec, fps, resolution, aspect ratio), perform timeline edits (subtitles, text/image overlays, branding, audio overlays, dubbing, translation), generate media assets (image, audio, video), and create real time alerts for events from live streams or desktop capture.

ComeOnOliver/skillshub

AI-assisted video editing workflows for cutting, structuring, and augmenting real footage. Covers the full pipeline from raw capture through FFmpeg, Remotion, ElevenLabs, fal.ai, and final polish in Descript or CapCut. Use when the user wants to edit video, cut footage, create vlogs, or build video content.

ComeOnOliver/skillshub

Next.js 15 애플리케이션을 위한 프론트엔드 개발 가이드라인. React 19, TypeScript, Shadcn/ui, Tailwind CSS를 사용한 모던 패턴. Server Components, Client Components, App Router, 파일 구조, Shadcn/ui 컴포넌트, 성능 최적화, TypeScript 모범 사례 포함. 컴포넌트, 페이지, 기능 생성, 데이터 페칭, 스타일링, 라우팅, 프론트엔드 코드 작업 시 사용.

ComeOnOliver/skillshub

Provides Tamagui patterns for config v4, compiler optimization, styled context, and cross-platform styling. Must use when working with Tamagui projects (tamagui.config.ts, @tamagui imports).

ComeOnOliver/skillshub

Create distinctive, production-grade frontend interfaces with high design quality. Use this skill when the user asks to build web components, pages, or applications. Generates creative, polished code that avoids generic AI aesthetics.

ComeOnOliver/skillshub

Create high-converting, visually distinctive landing pages. Use when building marketing pages, product launches, SaaS homepages, or any single-page conversion-focused website. Guides section-by-section composition with anti-AI-slop principles.

ComeOnOliver/skillshub

Auto-loaded context for Portfolio Buddy 2 development. Use for ANY task involving: React 19 development, TypeScript, portfolio analysis features, metrics calculations, trading strategy comparison, or working with the Portfolio Buddy 2 codebase. Contains tech stack, known issues, and architectural constraints.

ComeOnOliver/skillshub

Next.js development tooling via MCP. Inspect routes, components, build info, and debug Next.js apps. Use when working on Next.js applications, debugging routing, or inspecting app structure. NOT for general React or non-Next.js projects.

ComeOnOliver/skillshub

Implement UI using Shadcn MCP (atoms/theme) and 21st.dev MCP (complex sections). Use when adding buttons, layouts, or generating landing pages.

ComeOnOliver/skillshub

A conceptual skill for building an API client in Next.js that handles JWT tokens

ComeOnOliver/skillshub

Best practices and patterns for Next.js App Router, Server Actions, and Routing in this project.

ComeOnOliver/skillshub

The Design System, Theme, and UX rules for the Physical AI Hub.

ComeOnOliver/skillshub

Comprehensive frontend development skill for building modern, performant web applications using ReactJS, NextJS, TypeScript, Tailwind CSS. Includes component scaffolding, performance optimization, bundle analysis, and UI best practices. Use when developing frontend features, optimizing performance, implementing UI/UX designs, managing state, or reviewing frontend code.

ComeOnOliver/skillshub

This document captures learnings from fixing keyboard navigation issues when floating components (Select, DropdownMenu, Popover) have CSS open/close animations.

ComeOnOliver/skillshub

Build Next.js 16 applications with correct patterns and distinctive design. Use when creating pages, layouts, dynamic routes, upgrading from Next.js 15, or implementing proxy.ts. Covers breaking changes (async params/searchParams, Turbopack, cacheComponents) and frontend aesthetics. NOT when building non-React or backend-only applications.

ComeOnOliver/skillshub

Comprehensive Framer Motion animation library for React. Covers motion components, variants, gestures, page transitions, and scroll animations. Use when adding animations to React/Next.js applications.

ComeOnOliver/skillshub

Build interfaces that resonate, not just render. This skill enforces bold, intentional

相关技能