Communitygithub.com

ComeOnOliver/skillshub – Whisper 語音辨識

使用 OpenAI 最先進的語音辨識模型 Whisper 進行音訊轉錄。涵蓋本地 Whisper(Python)、faster-whisper(CTranslate2,4 倍速)、whisper.cpp(CPU 優化 C++)與 OpenAI Whisper API,支援字幕產生(SRT/VTT/JSON)、多語言轉錄、英文翻譯、說話者區分、詞級時間戳,以及播客、會議與影片字幕的生產流程。

skillshub – Whisper 語音辨識 是什麼?

這個技巧圍繞 OpenAI Whisper 語音辨識模型,提供完整的多執行緒轉錄解決方案。它支援四種執行環境:本地 Python 的 openai-whisper、透過 CTranslate2 加速 4 倍且支援 INT8 量化的 faster-whisper、純 CPU 優化且相依性極低的 whisper.cpp,以及無需本機 GPU 的 OpenAI API。使用者可以根據硬體與精確度需求選擇模型大小(從 tiny 到 large-v3 共 5 種),並能輸出 SRT、VTT 或 JSON 格式的字幕,包含詞級時間戳以達成卡拉 OK 風格效果。此外,技巧整合 pyannote-audio 實現說話者區分(辨別「誰說了什麼」),並支援自動語言偵測與任意語言轉譯成英文。最後,透過批次處理與生產流程模式,使用者可以一次轉錄整個目錄的音訊檔案,適用於播客、會議記錄與影片字幕等實際場景。

相容平台~Claude Code✓Codex CLI~Cursor
npx skills add https://github.com/ComeOnOliver/skillshub/tree/HEAD/skills/TerminalSkills/skills/whisper

在你喜歡的 AI 中提問

開啟一個已預先載入此 Agent Skill 的新對話。

說明文件

Whisper

概述

使用 OpenAI 最先進的語音辨識模型 Whisper 進行音訊轉錄。本技巧涵蓋本地 Whisper(Python)、faster-whisper(CTranslate2,4 倍速)、whisper.cpp(CPU 優化 C++)與 OpenAI Whisper API。支援字幕產生(SRT/VTT/JSON)、多語言轉錄、英文翻譯、說話者區分、詞級時間戳,以及播客、會議與影片字幕的生產流程模式。

指示

步驟 1:選擇執行環境

選項 A — OpenAI Whisper(原始 Python):

pip install openai-whisper
# 模型:tiny (39M)、base (74M)、small (244M)、medium (769M)、large-v3 (1.5G)

選項 B — faster-whisper(建議本地使用,4 倍速):

pip install faster-whisper
# 使用 CTranslate2 — INT8 量化,在 CPU 上運行良好

選項 C — whisper.cpp(最適合 CPU,相依性最少):

git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp && make
# 下載模型
bash models/download-ggml-model.sh base.en

選項 D — OpenAI API(無需本機 GPU):

pip install openai
export OPENAI_API_KEY="sk-..."

步驟 2:基本轉錄

faster-whisper(建議使用):

from faster_whisper import WhisperModel

model = WhisperModel("base", device="cpu", compute_type="int8")
# GPU:model = WhisperModel("large-v3", device="cuda", compute_type="float16")

segments, info = model.transcribe("episode.mp3", beam_size=5)
print(f"語言:{info.language}(機率:{info.language_probability:.2f})")

for segment in segments:
    print(f"[{segment.start:.2f}s → {segment.end:.2f}s] {segment.text}")

OpenAI Whisper(原始):

import whisper

model = whisper.load_model("base")  # tiny、base、small、medium、large-v3
result = model.transcribe("episode.mp3")

print(result["text"])
for segment in result["segments"]:
    print(f"[{segment['start']:.1f}s - {segment['end']:.1f}s] {segment['text']}")

whisper.cpp(命令列):

./main -m models/ggml-base.en.bin -f episode.wav -otxt -osrt -ovtt
# 輸出:episode.txt、episode.srt、episode.vtt

OpenAI API:

from openai import OpenAI
client = OpenAI()

with open("episode.mp3", "rb") as f:
    transcript = client.audio.transcriptions.create(
        model="whisper-1",
        file=f,
        response_format="verbose_json",
        timestamp_granularities=["segment", "word"],
    )

print(transcript.text)
for segment in transcript.segments:
    print(f"[{segment.start:.1f}s] {segment.text}")

步驟 3:字幕產生(SRT/VTT)

使用 faster-whisper 產生 SRT:

from faster_whisper import WhisperModel

model = WhisperModel("small", device="cpu", compute_type="int8")
segments, info = model.transcribe("episode.mp3", beam_size=5)

def format_timestamp(seconds):
    h = int(seconds // 3600)
    m = int((seconds % 3600) // 60)
    s = int(seconds % 60)
    ms = int((seconds % 1) * 1000)
    return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"

with open("episode.srt", "w") as f:
    for i, seg in enumerate(segments, 1):
        f.write(f"{i}\n")
        f.write(f"{format_timestamp(seg.start)} --> {format_timestamp(seg.end)}\n")
        f.write(f"{seg.text.strip()}\n\n")

print("已產生 episode.srt")

產生 VTT(適用於 HTML5 影片):

with open("episode.vtt", "w") as f:
    f.write("WEBVTT\n\n")
    for i, seg in enumerate(segments, 1):
        start = format_timestamp(seg.start).replace(",", ".")
        end = format_timestamp(seg.end).replace(",", ".")
        f.write(f"{start} --> {end}\n")
        f.write(f"{seg.text.strip()}\n\n")

詞級時間戳(卡拉 OK 風格字幕):

segments, info = model.transcribe("episode.mp3", word_timestamps=True)
for segment in segments:
    for word in segment.words:
        print(f"  [{word.start:.2f}s → {word.end:.2f}s] {word.word}")

步驟 4:語言偵測與翻譯

# 自動偵測語言
segments, info = model.transcribe("foreign_audio.mp3")
print(f"偵測到:{info.language}(機率:{info.language_probability:.0%})")

# 強制指定語言
segments, info = model.transcribe("german.mp3", language="de")

# 翻譯成英文(任何語言 → 英文)
segments, info = model.transcribe("german.mp3", task="translate")
for seg in segments:
    print(seg.text)  # 英文翻譯

支援的語言: 99 種語言,包括 en、zh、de、es、ru、ko、fr、ja、pt、tr、pl、nl、ar、sv、it、hi 等。

步驟 5:說話者區分

結合 Whisper 與 pyannote-audio 來識別「誰說了什麼」:

from faster_whisper import WhisperModel
from pyannote.audio import Pipeline
import torch

# 轉錄
model = WhisperModel("small", device="cpu", compute_type="int8")
segments, info = model.transcribe("meeting.wav", beam_size=5)
whisper_segments = list(segments)

# 說話者區分(需要具有 pyannote 存取權限的 HuggingFace token)
diarization = Pipeline.from_pretrained(
    "pyannote/speaker-diarization-3.1",
    use_auth_token="hf_YOUR_TOKEN"
)
diarization_result = diarization("meeting.wav")

# 合併:為每個 whisper 區段指定說話者
def get_speaker(start, end, diarization_result):
    """找出時間範圍內的主要說話者。"""
    speakers = {}
    for turn, _, speaker in diarization_result.itertracks(yield_label=True):
        overlap_start = max(start, turn.start)
        overlap_end = min(end, turn.end)
        if overlap_start < overlap_end:
            duration = overlap_end - overlap_start
            speakers[speaker] = speakers.get(speaker, 0) + duration
    return max(speakers, key=speakers.get) if speakers else "未知"

for seg in whisper_segments:
    speaker = get_speaker(seg.start, seg.end, diarization_result)
    print(f"[{speaker}] [{seg.start:.1f}s → {seg.end:.1f}s] {seg.text}")

步驟 6:批次處理與流程

轉錄目錄中的所有音訊集:

import os
from pathlib import Path
from faster_whisper import WhisperModel

model = WhisperModel("small", device="cpu", compute_type="int8")
episodes_dir = Path("episodes")
output_dir = Path("transcripts")
output_dir.mkdir(exist_ok=True)

for audio_file in sorted(episodes_dir.glob("*.mp3")):
    print(f"正在轉錄:{audio_file.name}")
    segments, info = model.transcribe(str(audio_file), beam_size=5)
    
    # 純文字
    txt_path = output_dir / f"{audio_file.stem}.txt"
    with open(txt_path, "w") as f:
        for seg in segments:
            f.write(seg.text.strip() + "\n")
    
    # SRT
    srt_path = output_dir / f"{audio_file.stem}.srt"
    segments2, _ = model.transcribe(str(audio_file), beam_size=5)  # 重新迭代
    with open(srt_path, "w") as f:
        for i, seg in enumerate(segments2, 1):
            h, m = divmod(int(seg.start), 3600)
            m, s = divmod(m, 60)
            ms = int((seg.start % 1) * 1000)
            start_ts = f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"
            h2, m2 = divmod(int(seg.end), 3600)
            m2, s2 = divmod(m2, 60)
            ms2 = int((seg.end % 1) * 1000)
            end_ts = f"{h2:02d}:{m2:02d}:{s2:02d},{ms2:03d}"
            f.write(f"{i}\n{start_ts} --> {end_ts}\n{seg.text.strip()}\n\n")
    
    print(f"  → {txt_path}、{srt_path}")

步驟 7:模型選擇指南

模型大小VRAM速度(CPU)準確度最佳應用
tiny39M~1GB非常快低快速草稿、即時
base74M~1GB快中CPU 的良好平衡
small244M~2GB中等好播客、清晰音訊
medium769M~5GB慢非常好噪音音訊、口音
large-v31.5G~10GB非常慢最佳生產品質

建議:

  • 僅使用 CPU、速度優先:tiny 或 base 搭配 faster-whisper
  • 僅使用 CPU、準確度優先:small 搭配 faster-whisper
  • 有 GPU 可用:large-v3 搭配 faster-whisper(float16)
  • 無本機運算能力:OpenAI API(whisper-1)

範例

範例 1:轉錄播客季度並產生 SRT 字幕

使用者提示:「請轉錄 ./episodes/ 中所有 20 個 MP3 音訊集,並產生純文字轉錄檔與 SRT 字幕檔。我有一個 10GB VRAM 的 GPU。」

代理程式會:

  1. 透過 pip install faster-whisper 安裝 faster-whisper。
  2. 使用 device="cuda" 與 compute_type="float16" 載入 large-v3 模型以充分利用 GPU。
  3. 建立 ./transcripts/ 輸出目錄。
  4. 迴圈處理 ./episodes/ 中所有 .mp3 檔案,並以 beam_size=5 進行轉錄。
  5. 為每個音訊集寫入一個 .txt 檔案(純文字)與一個 .srt 檔案(含正確格式的時間戳)。
  6. 回報每個音訊集的偵測語言與總處理時間。

範例 2:將德語訪談翻譯成英文並附上說話者標籤

使用者提示:「我有一個 45 分鐘的德語訪談錄音 meeting.wav,有兩位說話者。請將其轉錄成英文並標註誰說了什麼。」

代理程式會:

  1. 安裝 faster-whisper 與 pyannote-audio(pip install faster-whisper pyannote.audio)。
  2. 使用 task="translate" 轉錄 meeting.wav,從德語音訊取得英文文本。
  3. 執行 pyannote 說話者區分以識別說話者區段(需要具有 pyannote 模型存取權限的 HuggingFace token)。
  4. 透過比對時間範圍將 Whisper 區段與區分結果合併,以指定說話者標籤。
  5. 輸出格式化的轉錄稿,每個區段前加上 [Speaker 1] 與 [Speaker 2] 標籤。

指南

  • 對於本地轉錄,建議使用 faster-whisper 而非原始的 OpenAI Whisper;它透過 CTranslate2 的 INT8 量化,速度提升 4 倍且使用更少記憶體。
  • 根據硬體選擇模型大小:tiny/base 適合 CPU 速度需求,small 適合 CPU 準確度需求,large-v3 適合 GPU 生產品質。
  • 時刻將音訊轉換為 WAV 格式,或確保已安裝 ffmpeg 以處理 MP3/M4A 輸入;Whisper 依賴 ffmpeg 解碼非 WAV 格式。
  • 對於說話者區分,pyannote 流程需要具有已接受模型條款的 HuggingFace 存取權杖;在嘗試多說話者轉錄前請先設定好。
  • 產生 SRT 檔案時,使用 beam_size=5 以獲得更準確的區段邊界;預設的貪婪解碼可能產生時間軸不佳的字幕斷點。

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

ComeOnOliver/skillshub

Create stunning, animation-rich HTML presentations from scratch or by converting PowerPoint files. Use when the user wants to build a presentation, convert a PPT/PPTX to web, or create slides for a talk/pitch. Helps non-designers discover their aesthetic through visual exploration rather than abstract choices.

ComeOnOliver/skillshub

Next.js 16+ 和 Turbopack — 增量打包、文件系统缓存、开发速度,以及何时使用 Turbopack 与 webpack。

ComeOnOliver/skillshub

SwiftUI architecture patterns, state management with @Observable, view composition, navigation, performance optimization, and modern iOS/macOS UI best practices.

ComeOnOliver/skillshub

See, Understand, Act on video and audio. See- ingest from local files, URLs, RTSP/live feeds, or live record desktop; return realtime context and playable stream links. Understand- extract frames, build visual/semantic/temporal indexes, and search moments with timestamps and auto-clips. Act- transcode and normalize (codec, fps, resolution, aspect ratio), perform timeline edits (subtitles, text/image overlays, branding, audio overlays, dubbing, translation), generate media assets (image, audio, video), and create real time alerts for events from live streams or desktop capture.

ComeOnOliver/skillshub

AI-assisted video editing workflows for cutting, structuring, and augmenting real footage. Covers the full pipeline from raw capture through FFmpeg, Remotion, ElevenLabs, fal.ai, and final polish in Descript or CapCut. Use when the user wants to edit video, cut footage, create vlogs, or build video content.

ComeOnOliver/skillshub

Next.js 15 애플리케이션을 위한 프론트엔드 개발 가이드라인. React 19, TypeScript, Shadcn/ui, Tailwind CSS를 사용한 모던 패턴. Server Components, Client Components, App Router, 파일 구조, Shadcn/ui 컴포넌트, 성능 최적화, TypeScript 모범 사례 포함. 컴포넌트, 페이지, 기능 생성, 데이터 페칭, 스타일링, 라우팅, 프론트엔드 코드 작업 시 사용.

ComeOnOliver/skillshub

Provides Tamagui patterns for config v4, compiler optimization, styled context, and cross-platform styling. Must use when working with Tamagui projects (tamagui.config.ts, @tamagui imports).

ComeOnOliver/skillshub

Create distinctive, production-grade frontend interfaces with high design quality. Use this skill when the user asks to build web components, pages, or applications. Generates creative, polished code that avoids generic AI aesthetics.

ComeOnOliver/skillshub

Create high-converting, visually distinctive landing pages. Use when building marketing pages, product launches, SaaS homepages, or any single-page conversion-focused website. Guides section-by-section composition with anti-AI-slop principles.

ComeOnOliver/skillshub

Auto-loaded context for Portfolio Buddy 2 development. Use for ANY task involving: React 19 development, TypeScript, portfolio analysis features, metrics calculations, trading strategy comparison, or working with the Portfolio Buddy 2 codebase. Contains tech stack, known issues, and architectural constraints.

ComeOnOliver/skillshub

Next.js development tooling via MCP. Inspect routes, components, build info, and debug Next.js apps. Use when working on Next.js applications, debugging routing, or inspecting app structure. NOT for general React or non-Next.js projects.

ComeOnOliver/skillshub

Implement UI using Shadcn MCP (atoms/theme) and 21st.dev MCP (complex sections). Use when adding buttons, layouts, or generating landing pages.

ComeOnOliver/skillshub

A conceptual skill for building an API client in Next.js that handles JWT tokens

ComeOnOliver/skillshub

Best practices and patterns for Next.js App Router, Server Actions, and Routing in this project.

ComeOnOliver/skillshub

The Design System, Theme, and UX rules for the Physical AI Hub.

ComeOnOliver/skillshub

Comprehensive frontend development skill for building modern, performant web applications using ReactJS, NextJS, TypeScript, Tailwind CSS. Includes component scaffolding, performance optimization, bundle analysis, and UI best practices. Use when developing frontend features, optimizing performance, implementing UI/UX designs, managing state, or reviewing frontend code.

ComeOnOliver/skillshub

This document captures learnings from fixing keyboard navigation issues when floating components (Select, DropdownMenu, Popover) have CSS open/close animations.

ComeOnOliver/skillshub

Build Next.js 16 applications with correct patterns and distinctive design. Use when creating pages, layouts, dynamic routes, upgrading from Next.js 15, or implementing proxy.ts. Covers breaking changes (async params/searchParams, Turbopack, cacheComponents) and frontend aesthetics. NOT when building non-React or backend-only applications.

ComeOnOliver/skillshub

Comprehensive Framer Motion animation library for React. Covers motion components, variants, gestures, page transitions, and scroll animations. Use when adding animations to React/Next.js applications.

ComeOnOliver/skillshub

Build interfaces that resonate, not just render. This skill enforces bold, intentional

相關技能