Communitygithub.com

1345AbhishekKumar/starter-supabase

为 HyperFrames 合成创建音频和媒体资产,包含统一的音频引擎,支持多提供商 TTS、背景音乐、音效、Whisper 转录、背景去除和字幕创作。

starter-supabase 是什么?

该技能提供了一个统一的音频引擎 (`scripts/audio.mjs`),用于处理 HyperFrames 合成中所需的所有音频和媒体资产,包括语音合成(TTS)、背景音乐(BGM)、音效(SFX)、Whisper 转录、背景去除和字幕创作。其核心设计是“一个引擎,无供应商拷贝”,通过一个 `audio_request.json` 请求文件生成 `audio_meta.json` 输出。引擎的关键切换点在于是否存在 HeyGen 凭证:有凭证时优先使用 HeyGen 的 REST API(TTS 支持原生词时间戳),否则优雅降级到 ElevenLabs 和本地 Kokoro TTS,BGM 从检索切换为本地 Lyria/MusicGen 生成,SFX 则回退到内置的 21 个音效库。技能支持 Cursor、Codex 等主流 AI 编码平台,通过 CLI 命令 `npx hyperframes auth status` 进行身份验证预检,确保用户知情选择。对于字幕,技能提供了详细的指导,包括样式检测、布局、词分组、卡拉 OK 动效以及音频反应效果。

兼容平台~Claude Code~Codex CLI~Cursor
npx skills add https://github.com/1345AbhishekKumar/starter-supabase/tree/HEAD/.agents/skills/hyperframes-media

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

HyperFrames 媒体资产

创建合成所需的音频和媒体资产——语音旁白(TTS)、背景音乐 + 音效、转录、字幕、背景去除——然后在 HTML 中消费并动画化这些数据。关于如何将资产放入合成,请参见 hyperframes-core。

音频引擎——TTS · BGM · SFX 的单一来源

工作流不自行处理音频或引入供应商拷贝。只有一个引擎——scripts/audio.mjs——它接受一个中立的 audio_request.json 并写入 audio_meta.json(以及 assets/voice|bgm|sfx 下的资产):

# <MEDIA_DIR> = 此技能的目录
node <MEDIA_DIR>/scripts/audio.mjs --request ./audio_request.json --hyperframes . --out ./audio_meta.json

所有三种能力根据一个开关降级——是否存在 HeyGen 凭证(从 $HEYGEN_API_KEY / $HYPERFRAMES_API_KEY / ~/.heygen 解析,不是 CLI):

能力HeyGen 凭证存在不存在
TTSHeyGen Starfish REST(原生词时间戳)→ ElevenLabs → Kokoro(链接 transcribe 用于获取词)
BGMHeyGen 音乐检索Lyria → MusicGen 本地生成(分离进程)
SFXHeyGen 音效检索(min_score 0.4)内置 21 文件库(assets/sfx/)
  • 请求(audio_request.json):{ provider?, lang?, speed?, lines: [{ id, text, sfx?: [names] }], bgm: { mode?, query?, prompt? } }。id 将每行关联回调用者的模型(帧号、场景 ID 等)。bgm.mode = retrieve | generate | none;省略则自动选择(有凭证时检索,否则生成)。显式的 retrieve 是严格的——它跳过而不是启动分离的生成(针对没有 wait-bgm 步骤的调用者)。
  • 输出(audio_meta.json,按 id 键索引):{ tts_provider, voice_id, bgm, bgm_pending, …, voices: [{ id, path, duration_s, words }], sfx: [{ id, name, file, source, offset_s, duration_s, volume }], total_duration_s }。
  • --only tts,bgm,sfx 运行子集并合并到现有的 --out(例如,早期运行 TTS+BGM,等线索确定后再运行 SFX)。
  • BGM 生成以分离方式启动(bgm_pending: true)——在组装前运行 scripts/wait-bgm.mjs。
  • scripts/heygen-tts.mjs 是同一代码的单次 CLI 版本(一个文本 → wav + 词),适用于只需要 HeyGen TTS 而无需请求文件的场景。

完整的标志列表和 audio_meta.json 模式位于 scripts/audio.mjs 头部注释中。以下参考涵盖每种能力背后的提供商细节和边界情况。

预检——在任何音频生成前显示登录状态

在生成语音或 BGM 之前始终运行此操作——无论是在完整工作流中还是单独的“给我生成一个 BGM/旁白”请求。 不存在 HeyGen 凭证不是静默回退到本地引擎的理由:首先建议登录,让用户决定。运行共享预检并逐字传递其输出——不要即兴编写自己的“缺少密钥”提示,也不要提议将密钥写入每个仓库的 .env:

npx hyperframes auth status
  • 已登录 → 它打印账户信息;继续。
  • 未登录(exit 1 在此是预期的——“未登录”是正常状态,不是失败)→ 它打印注册优先的指导。建议登录:npx hyperframes auth login 是浏览器 OAuth——它登录并创建账户(始终可通过此仓库的 CLI 使用)。要使用现有的 HeyGen API 密钥(来自 app.heygen.com/settings/api),运行 npx hyperframes auth login --api-key——它保存到共享的 ~/.heygen(没有每个仓库的 .env)。输出还列出了本地引擎语音/BGM 将回退到的信息,以及缺少依赖时的 pip 提示。按原样传递此输出——不要用自己的措辞改写。 然后停止并等待用户选择——登录,或者说“go”/“local”以脱机继续——在生成任何内容之前。 这是一个真正的决策点,不是附带说明:不要将其合并到另一个问题中,也不要自己越过去。(例外:在自主/非交互模式下,记录状态并继续脱机。)
  • npx hyperframes auth status --json 返回 { configured, recommended_action, offline_engines } 以进行确定性分支。
  • 如果 CLI 无法运行(不在 PATH 上且 npx 无法获取它)→ 仍然建议登录(npx hyperframes auth login)并停止等待用户选择——不要将“无凭证”视为本地生成的静默绿灯。

凭证解析、完整密钥优先级和本地依赖列表在 references/requirements.md 中。

供应商链(引擎的详细信息)

TTS——第一个可用的供应商胜出(引擎,或 npx hyperframes tts "..."):

顺序供应商检测条件词时间戳
1HeyGen (Starfish)$HEYGEN_API_KEY / hyperframes auth login是,原生——传递 --words narration.words.json 以捕获
2ElevenLabs设置 $ELEVENLABS_API_KEY否——之后链接 transcribe
3Kokoro-82M(本地,54个语音)始终可用(无需密钥)否——之后链接 transcribe

已发布的 hyperframes tts CLI 通常是仅本地构建(其 --help 显示“Kokoro-82M”,没有 --provider/--words),即使设置了 $HEYGEN_API_KEY 也会静默回退到 Kokoro。这就是为什么引擎的 HeyGen 路径是自包含的 scripts/heygen-tts.mjs(REST),而不是 CLI;CLI 仅用于 Kokoro 路径。请参见 references/tts.md。

BGM & SFX——默认从 HeyGen 音频库(/v3/audio/sounds)检索,与 HeyGen TTS 使用相同的凭证,无凭证时回退到上述开关:

资产HeyGen type存放位置回退(无凭证)
BGMmusicassets/bgm/track.mp3(检索)· track.wav(生成)Lyria / MusicGen 生成
SFXsound_effects(min_score 0.4)assets/sfx/<slug>.mp3内置 21 文件库(assets/sfx/* + manifest.json)

请参见 references/bgm.md 和 references/sfx.md。

路由

任务阅读
音频引擎——请求/元数据模式、--only、开关scripts/audio.mjs(头部注释)
npx hyperframes tts / heygen-tts.mjs——供应商、语音、词references/tts.md
BGM——HeyGen 检索 + 本地 Lyria / MusicGen 生成references/bgm.md
SFX——HeyGen 检索(min_score 0.4)+ 内置本地库references/sfx.md
npx hyperframes transcribe——Whisper、模型规则、输出形状references/transcribe.md
npx hyperframes remove-background——透明切出references/remove-background.md
TTS → 转录 → 字幕(无录制旁白)references/tts-to-captions.md
字幕创作——样式检测、布局、词分组、退出references/captions/authoring.md
转录处理——输入格式、质量门控、清理、APIreferences/captions/transcript-handling.md
字幕动画——卡拉 OK、标记效果、音频响应式references/captions/motion.md
模型缓存、系统依赖、故障排除references/requirements.md

不可协商的规则

  • 一个引擎,无供应商拷贝。 通过 scripts/audio.mjs(或用于单次 HeyGen TTS 的 heygen-tts.mjs)生成音频。不要在工作流中重新实现 TTS/BGM/SFX——编写一个 audio_request.json 适配器并调用引擎。
  • “HeyGen 可用” = 可解析的凭证,不是 CLI。 整个开关取决于 heygenCredential();已发布的 hyperframes tts 可能仅是 Kokoro,并且根本没有 hyperframes bgm / hyperframes sfx 命令。
  • 语音 ID 是特定于供应商的。 am_michael 仅 Kokoro 可用;HeyGen UUID 在 Kokoro 上不起作用。如果传递 --voice,还要固定 --provider,以避免用户环境变化时供应商静默漂移。
  • 始终将 --model 传递给 transcribe。 CLI 默认值 small.en 会静默翻译非英语音频。请参见 references/transcribe.md → “语言规则”。
  • HeyGen 返回词时间戳;ElevenLabs / Kokoro 不返回。 引擎会自动为后两者链接 transcribe;独立使用时,向 HeyGen 传递 --words 或对音频文件运行 transcribe。
  • 字幕消费扁平词数组格式,包含 { id, text, start, end }。请参见 references/transcribe.md → “输出形状”。
  • remove-background --background-output 是抠洞,不是修补。 对于“没有人的场景”,需要不同的工具。请参见 references/remove-background.md → “何时不是正确的工具”。
  • BGM/SFX 默认使用 HeyGen 检索;无凭证回退是生成(BGM)或内置库(SFX)。 /audio/sounds 按文本查询排序——具体命名效果(glass shatter,而不是 dramatic sound);无匹配则跳过,绝不阻塞渲染。SFX 在语音 + BGM 下音量约为 ~0.35。请参见 references/sfx.md / references/bgm.md。
  • 将工作流字幕 HTML 视为生成输出。 对于基于预设的视频,可重用皮肤源位于 .hyperframes/caption-skin.html,工作流脚本写入 compositions/captions.html;不要编辑生成的 compositions/captions.html 来修复皮肤。通过工作流的 captions.mjs 重建,或在该工作流存在显式覆盖机制时使用该机制。

Individual skills in this repo

This repo contains 13 individual skills — each has its own dedicated page.

1345AbhishekKumar/starter-supabase

Add captions to a talking-head video. ONE catalog (CATALOG.md) of 32 visual identities behind two engines: column-flow (captions composited INTO the scene — matte occlusion + mix-blend; cream/ink/editorial/keynote/documentary/loud/neon/glitch/chrome/velocity) and themed constitutions (anchor/ordnance/terminal/neonsign/stardust/stomp/scoreboard/transit/vhs/arcade/dossier/laser/thunder/hologram/biolume/aurora/spectrum/papercut/popup/chalkboard/graffiti/brush/inkwater/ransom/lastpage/nightcity — e.g. a glyph-decode climax, a neon sign WRITTEN stroke by stroke, or the quiet `anchor` rail default). Route by identity, never by mode. Trigger on "captions/subtitles", "embed/cinematic captions", "VFX captions", "炸/特效/酷炫字幕", a named identity, or top-tier motion-graphics asks. Embedding every word is wrong for most talking-head content — `anchor` is the verbatim default. Pipeline: transcription → hyperframes remove-background matting → HTML render → ffmpeg overlay. Requires hyperframes and a single-subject clip.

1345AbhishekKumar/starter-supabase

The fallback workflow for authoring custom HyperFrames video compositions at any length or format — longer or multi-scene pieces, brand / sizzle reels, montages, title cards, static loops, and freeform compositions. Input- and length-agnostic. If a specialized workflow clearly fits the input — a marketed product, a website, a topic explainer, a GitHub PR, existing footage, a short motion graphic, or a Remotion port — prefer it (see /hyperframes); use this only as the general fallback when none fit.

1345AbhishekKumar/starter-supabase

All animation knowledge for HyperFrames — atomic motion rules, multi-phase scene blueprints, scene transitions, broader motion-design techniques, AND the seven runtime adapters (GSAP default, plus Lottie, Three.js, Anime.js, CSS keyframes, Web Animations API, TypeGPU). Use for any motion or animation task: pick 2-4 rules and compose, or load a blueprint, or look up runtime-specific API (e.g. GSAP eases / Lottie player / Three.js mixer). HyperFrames-native: single paused timeline, seek-safe, deterministic.

1345AbhishekKumar/starter-supabase

HyperFrames CLI dev loop. Use when running npx hyperframes init, add, catalog, capture, lint, validate, inspect, layout, snapshot, preview, play, render, publish, lambda, doctor, browser, info, upgrade, skills, compositions, docs, benchmark, telemetry, transcribe, tts, or remove-background, or when troubleshooting the HyperFrames build/render environment. Entry point for AWS Lambda cloud rendering (`hyperframes lambda deploy / render / progress / destroy / policies`).

1345AbhishekKumar/starter-supabase

The HyperFrames composition contract — build one renderable project. Use for composition structure, the `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, deterministic-render rules, and validation. Read before writing composition HTML.

1345AbhishekKumar/starter-supabase

Non-animation creative direction for HyperFrames videos. Use for design spec (frame.md / design.md) handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns, and brand / style decisions. For atomic motion patterns and scene blueprints, use `hyperframes-animation`.

1345AbhishekKumar/starter-supabase

Install and wire registry blocks and components into HyperFrames compositions. Use when running hyperframes add, installing a block or component, wiring an installed item into index.html, or working with hyperframes.json. Covers the add command, install locations, block sub-composition wiring, component snippet merging, registry discovery, and authoring a new block or component to contribute upstream (idea → scaffold → validate → PR).

1345AbhishekKumar/starter-supabase

Use when the user wants a short, design-led motion graphic where motion is the message: kinetic typography, stat or number count-up, chart/data-viz hit, logo sting, brand lockup, lower-third, callout, social overlay, animated headline/tweet/news item, motion poster, or quick captured-page highlight. Usually under 10s and up to ~30s, with no narration arc, voice-over, or live-action subject. Can render to MP4 or transparent overlay. Not for longer, multi-scene, narrated, or brand-reel pieces (use general-video), narrated website videos (website-to-video), topic explainers (faceless-explainer), product promos (product-launch-video), PR videos (pr-to-video), or captions on existing footage (embedded-captions). When unsure whether it's a quick motion-first piece or a longer / narrated treatment, see /hyperframes.

1345AbhishekKumar/starter-supabase

Use when the user has a music track (an audio file, or a video to pull audio from) and wants a beat-synced HyperFrames video, calm to hard-hitting. The music drives everything: one analyzer reads it once, the orchestrator lays out the frames and fills a per-frame plan, and one sub-agent builds each frame. Typography and templates are the floor — a complete video needs zero assets — but any images or videos the user supplies are cut into the frames on the same beat grid (beat-cut / ken-burns). The genre (lyric video, slideshow, kinetic promo) falls out of the per-frame choices; the pipeline never branches on it.

1345AbhishekKumar/starter-supabase

turn a product or marketing URL, pasted script, or brief into a product launch video, including SaaS promos, feature reveals, app launches, company promos, and product marketing videos. Use this skill when the user wants to market, launch, promote, or reveal a product. Do not use it for general non-launch website tours, non-product topic explainers, GitHub pull requests, captioning existing footage, or short unnarrated motion graphics. If the intent is unclear, route through /hyperframes first. This is the new shot-sequence architecture: every visual frame is authored as a time-coded shot sequence picked from a menu of golden blueprints, so frames develop over their full duration instead of freezing after entrance.

1345AbhishekKumar/starter-supabase

turn a GitHub pull request (a PR URL like github.com/<owner>/<repo>/pull/<N>, an <owner>/<repo>#<N> ref, or 'this PR' in a checked-out repo) into a code-change explainer video, up to ~3 min (sweet spot 30-90s) — changelog, feature reveal, fix, or refactor walkthrough, rendered from the diff / commits / files. The input is a CODE CHANGE read via the gh CLI; there is no website capture. Use this skill for a GitHub PR. Do not use it for a product launch/promo (use /product-launch-video), a tour of a real website (use /website-to-video), a topic explainer with no PR (use /faceless-explainer), captions on existing footage (use /embedded-captions), or a short unnarrated motion graphic (use /motion-graphics). If the intent is unclear, route through /hyperframes first.

1345AbhishekKumar/starter-supabase

Port an existing Remotion (React) composition to HyperFrames HTML. Use ONLY when the user explicitly asks to port/convert/migrate/translate a Remotion source. Do NOT use: (a) authoring a new HyperFrames composition; (b) Remotion mentioned in passing; (c) Remotion code shared as reference only; (d) "same video as my Remotion one" without explicit migrate request — treat as fresh build. Doubt → `/general-video`. One-way, Remotion-only: no reverse export (HyperFrames→Remotion or any framework), no non-Remotion source (After Effects, Framer Motion, plain React/CSS) → out of scope, re-create via `/general-video`. Flags unsupported patterns (useState, useEffect, async calculateMetadata, third-party React libs, `@remotion/lambda`) and recommends runtime interop over lossy translation. Unsure whether to port vs. build fresh, or only a passing Remotion mention? → /hyperframes.

1345AbhishekKumar/starter-supabase

Capture a general website/URL and turn it into a HyperFrames video (site tour, showcase, or social clip from the site's own visuals). Uses headless Chrome screenshots + brand assets. Use when intent is general — portfolio/blog/landing-page showcase or social clip from the site. NOT for: product/SaaS launch or promo (→ /product-launch-video, even from a URL); topic explainer with no site (→ /faceless-explainer); GitHub PR (→ /pr-to-video); adding captions to existing video (→ /embedded-captions); short unnarrated page-highlight motion graphic (→ /motion-graphics). Unclear launch-vs-general-site? Ask one question or start at /hyperframes.

相关技能