Communitygithub.com

Cody-W-Tucker/skills

对视频和音频进行感知、理解与操作。支持本地文件、URL、RTSP/桌面直播流的接入,实时上下文提取,以及可播放的流链接生成。

skills 是什么?

该技能旨在为开发者提供一个全面的视频和音频处理框架,支持从多种来源(本地文件、URL、RTSP直播流、桌面录制)进行数据采集。它能够对视频内容进行深度理解,包括提取帧、构建视觉/语义/时间索引,并支持基于时间戳的精确搜索和自动剪辑生成。在操作层面,该技能提供了丰富的视频编辑能力,如转码、归一化、字幕生成与翻译、文本/图片叠加、音频配音等,并可基于直播流或桌面录制内容创建实时告警。该技能主要面向使用VideoDB Python SDK的用户,支持在Claude Code、Cursor等AI Agent平台中通过自然语言指令驱动,例如“录制屏幕并在出现密码字段时发出警报”。其核心优势在于将复杂的视频处理流程(从感知、记忆到行动)整合为简洁的API调用和自动化工作流,极大减少了手动编码和本地工具依赖。

兼容平台✓Claude Code~Codex CLI~Cursor
npx skills add https://github.com/Cody-W-Tucker/skills/tree/HEAD/skills/videodb

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

VideoDB 技能

视频、直播流和桌面会话的感知、记忆与操作。

使用场景

桌面感知

  • 启动/停止 桌面会话,采集 屏幕、麦克风和系统音频
  • 流式传输 实时上下文 并存储 情景会话记忆
  • 对屏幕上的语音和事件运行 实时告警/触发器
  • 生成 会话摘要、可搜索的时间线以及 可播放的证据链接

视频接入与流媒体

  • 接入 文件或 URL 并返回 可播放的 Web 流链接
  • 转码/归一化:编解码器、比特率、帧率、分辨率、宽高比

索引与搜索(时间戳与证据)

  • 构建 视觉、语音 和 关键词 索引
  • 搜索并返回带 时间戳 和 可播放证据 的精确时刻
  • 从搜索结果中 自动创建剪辑

时间线编辑与生成

  • 字幕:生成、翻译、烧录
  • 叠加:文字/图像/品牌标识,动态字幕
  • 音频:背景音乐、画外音、配音
  • 通过 时间线操作 进行程序化合成和导出

直播流(RTSP)与监控

  • 连接 RTSP/直播源
  • 运行 实时视觉和语音理解,为监控工作流生成 事件/告警

工作原理

常见输入

  • 本地 文件路径、公共 URL 或 RTSP URL
  • 桌面采集请求:开始 / 停止 / 总结会话
  • 所需操作:获取上下文以进行理解、转码规范、索引规范、搜索查询、剪辑范围、时间线编辑、告警规则

常见输出

  • 流 URL
  • 包含 时间戳 和 证据链接 的搜索结果
  • 生成的资产:字幕、音频、图像、剪辑
  • 直播流的 事件/告警负载
  • 桌面 会话摘要 和记忆条目

运行 Python 代码

在运行任何 VideoDB 代码前,切换到项目目录并加载环境变量:

from dotenv import load_dotenv
load_dotenv(".env")

import videodb
conn = videodb.connect()

这将从以下位置读取 VIDEO_DB_API_KEY:

  1. 环境变量(如果已导出)
  2. 当前目录下的项目 .env 文件

如果密钥缺失,videodb.connect() 会自动引发 AuthenticationError。

当简短的內联命令足以完成任务时,不要编写脚本文件。

当编写内联 Python (python -c "...") 时,始终使用格式正确的代码——使用分号分隔语句并保持可读性。对于超过约 3 条语句的情况,改用 heredoc:

python << 'EOF'
from dotenv import load_dotenv
load_dotenv(".env")

import videodb
conn = videodb.connect()
coll = conn.get_collection()
print(f"视频数量: {len(coll.get_videos())}")
EOF

设置

当用户要求“设置 videodb”或类似操作时:

1. 安装 SDK

pip install "videodb[capture]" python-dotenv

如果 videodb[capture] 在 Linux 上安装失败,则安装不包含 capture 扩展的版本:

pip install videodb python-dotenv

2. 配置 API 密钥

用户必须使用 任一 方法设置 VIDEO_DB_API_KEY:

  • 在终端中导出(启动 Claude 前):export VIDEO_DB_API_KEY=your-key
  • 项目 .env 文件:将 VIDEO_DB_API_KEY=your-key 保存到项目的 .env 文件中

在 console.videodb.io 获取免费 API 密钥(50 次免费上传,无需信用卡)。

不要自行读取、写入或处理 API 密钥。始终让用户设置它。

快速参考

上传媒体

# URL
video = coll.upload(url="https://example.com/video.mp4")

# YouTube
video = coll.upload(url="https://www.youtube.com/watch?v=VIDEO_ID")

# 本地文件
video = coll.upload(file_path="/path/to/video.mp4")

转录与字幕

# force=True 可在视频已索引时跳过错误
video.index_spoken_words(force=True)
text = video.get_transcript_text()
stream_url = video.add_subtitle()

在视频内搜索

from videodb.exceptions import InvalidRequestError

video.index_spoken_words(force=True)

# search() 在未找到结果时引发 InvalidRequestError。
# 始终用 try/except 包裹并将“未找到结果”视为空。
try:
    results = video.search("产品演示")
    shots = results.get_shots()
    stream_url = results.compile()
except InvalidRequestError as e:
    if "No results found" in str(e):
        shots = []
    else:
        raise

场景搜索

import re
from videodb import SearchType, IndexType, SceneExtractionType
from videodb.exceptions import InvalidRequestError

# index_scenes() 没有 force 参数——如果场景索引已存在则会引发错误。
# 从错误中提取现有的索引 ID。
try:
    scene_index_id = video.index_scenes(
        extraction_type=SceneExtractionType.shot_based,
        prompt="描述此场景的视觉内容。",
    )
except Exception as e:
    match = re.search(r"id\s+([a-f0-9]+)", str(e))
    if match:
        scene_index_id = match.group(1)
    else:
        raise

# 使用 score_threshold 过滤低相关性噪声(建议:0.3 或更高)
try:
    results = video.search(
        query="一个人在写白板",
        search_type=SearchType.semantic,
        index_type=IndexType.scene,
        scene_index_id=scene_index_id,
        score_threshold=0.3,
    )
    shots = results.get_shots()
    stream_url = results.compile()
except InvalidRequestError as e:
    if "No results found" in str(e):
        shots = []
    else:
        raise

时间线编辑

重要: 在构建时间线之前始终验证时间戳:

  • start 必须 >= 0(负值会被静默接受但会产生损坏的输出)
  • start 必须 < end
  • end 必须 <= video.length
from videodb.timeline import Timeline
from videodb.asset import VideoAsset, TextAsset, TextStyle

timeline = Timeline(conn)
timeline.add_inline(VideoAsset(asset_id=video.id, start=10, end=30))
timeline.add_overlay(0, TextAsset(text="结束", duration=3, style=TextStyle(fontsize=36)))
stream_url = timeline.generate_stream()

转码视频(分辨率/质量更改)

from videodb import TranscodeMode, VideoConfig, AudioConfig

# 在服务端更改分辨率、质量或宽高比
job_id = conn.transcode(
    source="https://example.com/video.mp4",
    callback_url="https://example.com/webhook",
    mode=TranscodeMode.economy,
    video_config=VideoConfig(resolution=720, quality=23, aspect_ratio="16:9"),
    audio_config=AudioConfig(mute=False),
)

重新调整宽高比(用于社交平台)

警告: reframe() 是一个慢速的服务端操作。对于长视频可能需要几分钟,并可能超时。最佳实践:

  • 尽可能使用 start/end 将操作限制在短片段上
  • 对于完整长度的视频,使用 callback_url 进行异步处理
  • 先在 Timeline 上修剪视频,然后重新调整较短结果的宽高比
from videodb import ReframeMode

# 始终优先重新调整短视频片段:
reframed = video.reframe(start=0, end=60, target="vertical", mode=ReframeMode.smart)

# 完整长度视频的异步重新调整(返回 None,结果通过 webhook 返回):
video.reframe(target="vertical", callback_url="https://example.com/webhook")

# 预设:"vertical" (9:16), "square" (1:1), "landscape" (16:9)
reframed = video.reframe(start=0, end=60, target="square")

# 自定义尺寸
reframed = video.reframe(start=0, end=60, target={"width": 1280, "height": 720})

生成式媒体

image = coll.generate_image(
    prompt="山上的日落",
    aspect_ratio="16:9",
)

错误处理

from videodb.exceptions import AuthenticationError, InvalidRequestError

try:
    conn = videodb.connect()
except AuthenticationError:
    print("检查你的 VIDEO_DB_API_KEY")

try:
    video = coll.upload(url="https://example.com/video.mp4")
except InvalidRequestError as e:
    print(f"上传失败: {e}")

常见陷阱

场景错误消息解决方案
为已索引的视频建立索引Spoken word index for video already exists使用 video.index_spoken_words(force=True) 在已索引时跳过
场景索引已存在Scene index with id XXXX already exists使用 re.search(r"id\s+([a-f0-9]+)", str(e)) 从错误中提取现有的 scene_index_id
搜索未找到匹配项InvalidRequestError: No results found捕获异常并视为空结果 (shots = [])
重新调整宽高比超时在长视频上无限阻塞使用 start/end 限制片段,或传递 callback_url 进行异步处理
时间线上的负时间戳静默生成损坏的流在创建 VideoAsset 前始终验证 start >= 0
generate_video() / create_collection() 失败Operation not allowed 或 maximum limit计划限制功能——告知用户有关计划限制的信息

示例

规范提示词

  • "启动桌面采集,并在出现密码字段时发出警报。"
  • "录制我的会话,并在结束时生成可操作的摘要。"
  • "接入此文件并返回一个可播放的流链接。"
  • "索引此文件夹并找到所有包含人物的场景,返回时间戳。"
  • "生成字幕,烧录进去,并添加轻量背景音乐。"
  • "连接此 RTSP URL,并在有人进入区域时发出警报。"

屏幕录制(桌面采集)

使用 ws_listener.py 在录制会话期间捕获 WebSocket 事件。桌面采集仅支持 macOS。

快速开始

  1. 选择状态目录:STATE_DIR="${VIDEODB_EVENTS_DIR:-$HOME/.local/state/videodb}"
  2. 启动监听器:VIDEODB_EVENTS_DIR="$STATE_DIR" python scripts/ws_listener.py --clear "$STATE_DIR" &
  3. 获取 WebSocket ID:cat "$STATE_DIR/videodb_ws_id"
  4. 运行采集代码(完整工作流程参见 reference/capture.md)
  5. 事件写入:$STATE_DIR/videodb_events.jsonl

每次开始新的采集运行时使用 --clear,这样旧的字幕和视觉事件不会泄漏到新会话中。

查询事件

import json
import os
import time
from pathlib import Path

events_dir = Path(os.environ.get("VIDEODB_EVENTS_DIR", Path.home() / ".local" / "state" / "videodb"))
events_file = events_dir / "videodb_events.jsonl"
events = []

if events_file.exists():
    with events_file.open(encoding="utf-8") as handle:
        for line in handle:
            try:
                events.append(json.loads(line))
            except json.JSONDecodeError:
                continue

transcripts = [e["data"]["text"] for e in events if e.get("channel") == "transcript"]
cutoff = time.time() - 300
recent_visual = [
    e for e in events
    if e.get("channel") == "visual_index" and e["unix_ts"] > cutoff
]

附加文档

参考文档位于此 SKILL.md 文件旁边的 reference/ 目录中。如有需要,请使用 Glob 工具定位。

除非 VideoDB 不支持该操作,否则不要使用 ffmpeg、moviepy 或本地编码工具。 以下所有操作均由 VideoDB 在服务端处理——裁剪、组合剪辑、叠加音频或音乐、添加字幕、文本/图像叠加、转码、分辨率更改、宽高比转换、为平台要求调整大小、转录和媒体生成。仅在 reference/editor.md 中“限制”部分列出的操作(转场、速度更改、裁剪/缩放、调色、音量混合)时才回退到本地工具。

何时使用什么

问题VideoDB 解决方案
平台拒绝视频宽高比或分辨率video.reframe() 或使用 VideoConfig 的 conn.transcode()
需要为 Twitter/Instagram/TikTok 调整视频大小video.reframe(target="vertical") 或 target="square"
需要更改分辨率(例如 1080p → 720p)使用 VideoConfig(resolution=720) 的 conn.transcode()
需要在视频上叠加音频/音乐在 Timeline 上使用 AudioAsset
需要添加字幕video.add_subtitle() 或 CaptionAsset
需要合并/修剪剪辑在 Timeline 上使用 VideoAsset
需要生成画外音、音乐或音效coll.generate_voice()、generate_music()、generate_sound_effect()

来源

此技能的参考材料已在本地以 skills/videodb/reference/ 形式提供。请使用本地的副本,而不是在运行时遵循外部仓库链接。

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

Cody-W-Tucker/skills

Landing page quality assessment for paid advertising campaigns. Evaluates message match, page speed, mobile experience, trust signals, form optimization, and conversion rate potential. Use when user says landing page, post-click experience, landing page audit, conversion rate, or landing page optimization.

Cody-W-Tucker/skills

TikTok Ads deep analysis covering creative quality, tracking, bidding, campaign structure, and TikTok Shop. Evaluates 28 checks with emphasis on creative-first strategy, safe zone compliance, and Smart+ campaigns. Use when user says TikTok Ads, TikTok marketing, TikTok Shop, Spark Ads, Smart+, or TikTok campaign.

Cody-W-Tucker/skills

YouTube Ads specific analysis covering campaign types, creative quality, audience targeting, and measurement. Evaluates video ad performance across skippable, non-skippable, bumper, Shorts, Demand Gen, and Connected TV formats. Covers VAC→Demand Gen migration, Shorts creative requirements, and CTV shoppable ads. Use when user says YouTube Ads, video ads, pre-roll, bumper ads, YouTube campaign, Shorts ads, or CTV ads.

Cody-W-Tucker/skills

Craft the story that makes investors say \"I want in.\" Master the narrative structure that Reid Hoffman, Sequoia, and top founders use to raise capital. Use when: **Preparing to fundraise** to develop your story before the deck; **Investor meetings** to structure conversations that convert; **Pitch practice** to refine your narrative arc; **Cold outreach** to write compelling investor emails; **Pivoting** to reframe your story after a change

Cody-W-Tucker/skills

Identify the best GTM motions and tools across 7 motion types: Inbound, Outbound, Paid Digital, Community, Partners, ABM, and PLG. Use when selecting marketing channels, choosing between inbound and outbound strategy, or planning cross-channel campaigns.

Cody-W-Tucker/skills

Produce a print-grade single-page kami (紙 / 纸) document — warm parchment canvas, ink-blue accent, serif at one weight, no italic, no cool grays. The output reads like a professional white paper or studio one-pager, not an app UI. Multilingual by design (EN · zh-CN · ja). One self-contained HTML file, zero dependencies.

Cody-W-Tucker/skills

Optimize landing pages for maximum conversions using proven frameworks from Unbounce and Oli Gardner—apply the \"one goal, one message, one action\" principle with data-driven design, copy, and CTA best practices. Use when: **Create a new landing page** for a campaign; **Optimize an existing landing page** that isn't converting; **Review landing page design** before launch; **Improve form conversion rates** on lead gen pages; **A/B test landing page elements** systematically

Cody-W-Tucker/skills

Advanced motion patterns for React / Next.js — drag & drop, gestures, text animations, SVG path drawing, custom hooks, imperative sequences (useAnimate), loaders, and the full API decision tree. Requires motion-foundations.

Cody-W-Tucker/skills

Motion tokens, spring presets, performance rules, device adaptation, accessibility enforcement, and SSR safety for React / Next.js using motion/react. Foundation layer — all other motion skills depend on this.

Cody-W-Tucker/skills

A single-frame motion-design composition with looping CSS animations — rotating type ring, animated globe, ticking timer, parallax labels. Renders as a hero video poster you can hand straight to HyperFrames or any keyframe-based exporter. Use when the brief asks for "motion design", "animated hero", "loop", "video poster", "title card", or pairs Open Claude Design with HyperFrames for a kinetic export.

Cody-W-Tucker/skills

Production-ready animation patterns for React / Next.js — button, modal, toast, stagger, page transitions, exit animations, scroll, and layout — built on motion-foundations tokens and springs.

Cody-W-Tucker/skills

Production-ready UI motion system for React/Next.js. Use when implementing animations, transitions, or motion patterns.

Cody-W-Tucker/skills

Produce a single-file slide deck in the Atelier Zero visual language (warm-paper background, italic-serif emphasis spans, coral terminating dots, surreal collage plates) — Open Design's brand deck recipe. The deck uses **horizontal magazine-style swipe pagination** (←/→, wheel, swipe), a per-slide chrome strip with brand mark and slide counter, an ESC overview grid, a coral progress bar, and inherits the canonical stylesheet + 16-slot image library from the sister `open-design-landing` skill.

Cody-W-Tucker/skills

Produce a world-class single-page editorial landing site in the Atelier Zero visual language (Monocle / Apartamento / Études editorial collage) — the same aesthetic Open Design uses for its own marketing surface. The agent fills a typed `inputs.json` from a brand brief, optionally generates 16 collage assets via gpt-image-2, then runs a pure-function composer that emits a self-contained HTML file; a separate path can mirror the Astro marketing site in `apps/landing-page/`. Drop-in scroll-reveal motion and a Headroom-style sticky nav are wired automatically.

Cody-W-Tucker/skills

Best practices for Remotion - Video creation in React. 29 domain-specific rules covering 3D, animations, audio, captions, charts, transitions, and more.

Cody-W-Tucker/skills

Single-page SaaS landing with hero, features, social proof, pricing, and CTA. Respects the active DESIGN.md color/typography/layout tokens. Trigger keywords: "saas landing", "marketing page", "product landing".

Cody-W-Tucker/skills

Single-page SaaS landing with hero, features, social proof, pricing, and CTA. Respects the active DESIGN.md color/typography/layout tokens. Trigger keywords: "saas landing", "marketing page", "product landing".

Cody-W-Tucker/skills

Master April Dunford's 8-step sales narrative from \"Sales Pitch\" (2023). Transform your positioning into a compelling story that wins enterprise deals. Use when: Structuring B2B sales presentations and demos; Creating pitch decks for complex products; Training sales teams on narrative-driven selling; Converting positioning strategy into sales conversations; Winning against entrenched competitors

Cody-W-Tucker/skills

Swiss-style user-research narrative template in warm-paper editorial aesthetics. Use when users ask for a premium research deck or story-first live artifact with minimalist typography, high-clarity layout, subtle motion, donut breakdowns, and keyboard/click navigation across slides in a single HTML file.

Cody-W-Tucker/skills

Download YouTube videos with customizable quality and format options. Use this skill when the user asks to download, save, or grab YouTube videos. Supports various quality settings (best, 1080p, 720p, 480p, 360p), multiple formats (mp4, webm, mkv), and audio-only downloads as MP3.

相关技能