Communitygithub.com

1345AbhishekKumar/starter-supabase

HyperFrames 媒體資產引擎 — 語音合成、背景音樂、音效、轉錄、字幕及去背,由單一音頻引擎驅動。

starter-supabase 是什麼?

此技能提供一個統一的音頻與媒體資產生成引擎,專為 HyperFrames 組合設計。其核心是 `scripts/audio.mjs` 腳本,能夠根據一份中性的 `audio_request.json` 請求,自動選用 HeyGen、ElevenLabs 或本地 Kokoro 等 TTS 提供者,並處理背景音樂(HeyGen 檢索或 Lyria/MusicGen 生成)與音效(HeyGen 檢索或內建 21 個音效庫)。輸出結果為結構化的 `audio_meta.json`,包含逐字時間戳、語音路徑及總時長等資訊。 技能涵蓋完整的媒體生產流程:從語音旁白、背景音樂與音效生成,到使用 Whisper 進行語音轉錄、自動字幕生成(支援卡拉 OK 與逐字樣式),以及影像去背。所有功能皆圍繞一個關鍵開關:是否配置 HeyGen 憑證。有憑證時優先使用 HeyGen 的雲端服務以獲得高品質與原生逐字時間戳;無憑證時則優雅降級至本地模型(Kokoro、Lyria、MusicGen)或內建音效庫,無需任何 API 金鑰即可運作。 此技能支援 Claude Code、Cursor 及 Codex 等 AI 代理平台。使用前務必執行 `npx hyperframes auth status` 檢查登入狀態,系統會明確提示使用者選擇登入或離線運作,而非默默降級。對於開發者與 AI 代理使用者而言,這是一個強大且靈活的媒體資產生產線,能夠在單一請求中同時產出旁白、背景音樂、音效與字幕,並確保所有資產的格式與時間軸完美對齊。

相容平台~Claude Code~Codex CLI~Cursor
npx skills add https://github.com/1345AbhishekKumar/starter-supabase/tree/HEAD/.agents/skills/hyperframes-media

在你喜歡的 AI 中提問

開啟一個已預先載入此 Agent Skill 的新對話。

說明文件

HyperFrames 媒體資產

為組合建立所需的音頻與媒體資產 — 語音旁白(TTS)、背景音樂 + 音效、轉錄、字幕、去背 — 然後在 HTML 中消費並動畫化這些資料。若要將資產放入組合,請參閱 hyperframes-core。

音頻引擎 — TTS · BGM · SFX 的單一來源

工作流程不應手動拼湊音頻或複製供應商套件。只有一個引擎 — scripts/audio.mjs — 它接受中性的 audio_request.json 並寫出 audio_meta.json(以及 assets/voice|bgm|sfx 下的資產):

# <MEDIA_DIR> = 此技能的路徑
node <MEDIA_DIR>/scripts/audio.mjs --request ./audio_request.json --hyperframes . --out ./audio_meta.json

所有三項能力取決於一個開關 — 是否存在 HeyGen 憑證(從 $HEYGEN_API_KEY / $HYPERFRAMES_API_KEY / ~/.heygen 解析,非 CLI):

能力HeyGen 憑證存在憑證不存在
TTSHeyGen Starfish REST(原生逐字時間戳)→ ElevenLabs → Kokoro(鏈式 transcribe 取得逐字時間)
BGMHeyGen 音樂檢索Lyria → MusicGen 本地生成(分離執行)
SFXHeyGen 音效檢索(min_score 0.4)內建 21 個檔案音效庫(assets/sfx/)
  • 請求(audio_request.json):{ provider?, lang?, speed?, lines: [{ id, text, sfx?: [names] }], bgm: { mode?, query?, prompt? } }。id 將每一行連接回呼叫端的模型(影格編號、場景 ID 等)。bgm.mode = retrieve | generate | none;省略時自動(有憑證時檢索,否則生成)。明確指定 retrieve 是嚴格的 — 它會跳過而非啟動分離的生成(適用於沒有 wait-bgm 步驟的呼叫端)。
  • 輸出(audio_meta.json,以 id 為鍵):{ tts_provider, voice_id, bgm, bgm_pending, …, voices: [{ id, path, duration_s, words }], sfx: [{ id, name, file, source, offset_s, duration_s, volume }], total_duration_s }。
  • --only tts,bgm,sfx 執行子集並合併到現有的 --out(例如,先 TTS+BGM,等音效提示存在後再加 SFX)。
  • BGM 生成以分離方式啟動(bgm_pending: true)— 在組裝前運行 scripts/wait-bgm.mjs。
  • scripts/heygen-tts.mjs 是相同程式碼的單次 CLI(一個文字 → wav + 逐字時間),適合只需 HeyGen TTS 而無需請求檔案的場景。

完整標誌列表 + audio_meta.json 結構位於 scripts/audio.mjs 的標頭註解中。以下參考資料涵蓋各項能力背後的提供者細節與邊界情況。

預檢 — 在任何音頻之前顯示登入狀態

在生成語音或 BGM 之前務必執行此步驟 — 無論是在完整工作流程還是單次「幫我生成 BGM/語音旁白」請求中。 沒有 HeyGen 憑證不是默默降級到本地引擎的理由:首先建議登入,讓使用者決定。執行共享的預檢並逐字傳達其輸出 — 不要自行編寫「遺失金鑰」提示,也不要把金鑰寫入每個儲存庫的 .env:

npx hyperframes auth status
  • 已登入 → 它會顯示帳戶資訊;繼續。
  • 未登入(此處預期 exit 1 —「未登入」是正常狀態,不是失敗)→ 它會顯示註冊優先指引。建議登入:npx hyperframes auth login 是瀏覽器 OAuth — 它會登入並建立帳戶(始終可透過此儲存庫的 CLI 使用)。若要使用現有的 HeyGen API 金鑰(來自 app.heygen.com/settings/api),請執行 npx hyperframes auth login --api-key — 它會儲存到共享的 ~/.heygen(無需每個儲存庫的 .env)。輸出還會列出語音/BGM 將降級到的本地引擎,以及在缺少依賴時的 pip 提示。逐字傳達此輸出 — 不要用自己的措辭重新解釋。 然後停止並等待使用者選擇 — 登入,或說「go」/「local」以繼續離線運作 — 在生成任何內容之前。 這是一個真正的決策點,不是過場備註:不要把它併入其他問題,也不要自行跳過它。(例外:在自主/非互動模式下,記錄狀態並繼續離線運作。)
  • npx hyperframes auth status --json 會回傳 { configured, recommended_action, offline_engines },供確定性分支使用。
  • 如果 CLI 無法執行(不在 PATH 上且 npx 無法擷取)→ 仍然建議登入(npx hyperframes auth login)並停下來等待使用者選擇 — 不要把「沒有憑證」當作本地生成的默認綠燈。

憑證解析、完整金鑰優先順序以及本地依賴列表位於 references/requirements.md。

提供者鏈(引擎背後的細節)

TTS — 第一個可用的提供者勝出(引擎,或 npx hyperframes tts "..."):

順序提供者檢測時機逐字時間戳
1HeyGen(Starfish)$HEYGEN_API_KEY / hyperframes auth login有,原生 — 傳入 --words narration.words.json 以擷取
2ElevenLabs設定 $ELEVENLABS_API_KEY無 — 之後鏈式 transcribe
3Kokoro-82M(本地,54 種語音)始終可用(無需金鑰)無 — 之後鏈式 transcribe

已發布的 hyperframes tts CLI 通常是僅限本地版本的建置(其 --help 顯示「Kokoro-82M」,沒有 --provider/--words),即使設定了 $HEYGEN_API_KEY 也會默默降級到 Kokoro。這就是為什麼引擎的 HeyGen 路徑是獨立的 scripts/heygen-tts.mjs(REST),而不是 CLI;CLI 僅用於 Kokoro 路徑。請參閱 references/tts.md。

BGM & SFX — 預設從 HeyGen 音頻庫(/v3/audio/sounds)檢索,使用與 HeyGen TTS 相同的憑證,並使用上方開關的無憑證降級方案:

資產HeyGen type存放位置降級(無憑證)
BGMmusicassets/bgm/track.mp3(檢索) · track.wav(生成)Lyria / MusicGen 生成
SFXsound_effects(min_score 0.4)assets/sfx/<slug>.mp3內建 21 個檔案音效庫(assets/sfx/* + manifest.json)

請參閱 references/bgm.md 和 references/sfx.md。

路由

任務閱讀
音頻引擎 — 請求/中繼資料結構、--only、開關scripts/audio.mjs(標頭註解)
npx hyperframes tts / heygen-tts.mjs — 提供者、語音、逐字時間references/tts.md
BGM — HeyGen 檢索 + 本地 Lyria / MusicGen 生成references/bgm.md
SFX — HeyGen 檢索(min_score 0.4)+ 內建本地音效庫references/sfx.md
npx hyperframes transcribe — Whisper、模型規則、輸出格式references/transcribe.md
npx hyperframes remove-background — 透明去背references/remove-background.md
TTS → 轉錄 → 字幕(無錄製語音旁白)references/tts-to-captions.md
字幕編寫 — 樣式偵測、佈局、單字分組、退出references/captions/authoring.md
轉錄稿處理 — 輸入格式、品質門檻、清理、APIreferences/captions/transcript-handling.md
字幕動態 — 卡拉 OK、標記效果、音頻反應式references/captions/motion.md
模型快取、系統依賴、疑難排解references/requirements.md

不可協商的規則

  • 單一引擎,不得複製供應商套件。 透過 scripts/audio.mjs(或單次 HeyGen TTS 使用 heygen-tts.mjs)產生音頻。請勿在工作流程中重新實作 TTS/BGM/SFX — 請撰寫 audio_request.json 適配器並呼叫引擎。
  • 「HeyGen 可用」= 可解析的憑證,而非 CLI。 整個開關依賴 heygenCredential();已發布的 hyperframes tts 可能僅支援 Kokoro,而且根本沒有 hyperframes bgm / hyperframes sfx 指令。
  • 語音 ID 是提供者專用的。 am_michael 僅限 Kokoro;HeyGen UUID 在 Kokoro 上無效。如果你傳入 --voice,也請固定 --provider,以避免使用者環境變更時提供者默默漂移。
  • 務必傳入 --model 給 transcribe。 CLI 預設的 small.en 會默默翻譯非英文音頻。請參閱 references/transcribe.md →「語言規則」。
  • HeyGen 會回傳逐字時間戳;ElevenLabs / Kokoro 不會。 引擎會為後兩者自動鏈式 transcribe;單獨使用時,請為 HeyGen 傳入 --words,或對音頻檔案執行 transcribe。
  • 字幕使用扁平單字陣列格式,包含 { id, text, start, end }。請參閱 references/transcribe.md →「輸出格式」。
  • remove-background --background-output 是挖洞,而非修補。 對於「沒有人的場景」,需要使用不同的工具。請參閱 references/remove-background.md →「何時不是正確工具」。
  • BGM/SFX 預設使用 HeyGen 檢索;無憑證降級為生成(BGM)或內建音效庫(SFX)。 /audio/sounds 根據文字查詢排序 — 具體命名音效(如 glass shatter,而非 dramatic sound);無匹配則跳過,絕不阻擋渲染。SFX 在語音 + BGM 下的音量約為 0.35。請參閱 references/sfx.md / references/bgm.md。
  • 將工作流程字幕 HTML 視為產生的輸出。 對於預設背景的影片,可重複使用的皮膚原始碼位於 .hyperframes/caption-skin.html,工作流程腳本會寫入 compositions/captions.html;請勿編輯產生的 compositions/captions.html 來修正皮膚。請透過工作流程的 captions.mjs 重建,或使用該工作流程的明確覆寫機制(如果存在的話)。

Individual skills in this repo

This repo contains 13 individual skills — each has its own dedicated page.

1345AbhishekKumar/starter-supabase

Add captions to a talking-head video. ONE catalog (CATALOG.md) of 32 visual identities behind two engines: column-flow (captions composited INTO the scene — matte occlusion + mix-blend; cream/ink/editorial/keynote/documentary/loud/neon/glitch/chrome/velocity) and themed constitutions (anchor/ordnance/terminal/neonsign/stardust/stomp/scoreboard/transit/vhs/arcade/dossier/laser/thunder/hologram/biolume/aurora/spectrum/papercut/popup/chalkboard/graffiti/brush/inkwater/ransom/lastpage/nightcity — e.g. a glyph-decode climax, a neon sign WRITTEN stroke by stroke, or the quiet `anchor` rail default). Route by identity, never by mode. Trigger on "captions/subtitles", "embed/cinematic captions", "VFX captions", "炸/特效/酷炫字幕", a named identity, or top-tier motion-graphics asks. Embedding every word is wrong for most talking-head content — `anchor` is the verbatim default. Pipeline: transcription → hyperframes remove-background matting → HTML render → ffmpeg overlay. Requires hyperframes and a single-subject clip.

1345AbhishekKumar/starter-supabase

The fallback workflow for authoring custom HyperFrames video compositions at any length or format — longer or multi-scene pieces, brand / sizzle reels, montages, title cards, static loops, and freeform compositions. Input- and length-agnostic. If a specialized workflow clearly fits the input — a marketed product, a website, a topic explainer, a GitHub PR, existing footage, a short motion graphic, or a Remotion port — prefer it (see /hyperframes); use this only as the general fallback when none fit.

1345AbhishekKumar/starter-supabase

All animation knowledge for HyperFrames — atomic motion rules, multi-phase scene blueprints, scene transitions, broader motion-design techniques, AND the seven runtime adapters (GSAP default, plus Lottie, Three.js, Anime.js, CSS keyframes, Web Animations API, TypeGPU). Use for any motion or animation task: pick 2-4 rules and compose, or load a blueprint, or look up runtime-specific API (e.g. GSAP eases / Lottie player / Three.js mixer). HyperFrames-native: single paused timeline, seek-safe, deterministic.

1345AbhishekKumar/starter-supabase

HyperFrames CLI dev loop. Use when running npx hyperframes init, add, catalog, capture, lint, validate, inspect, layout, snapshot, preview, play, render, publish, lambda, doctor, browser, info, upgrade, skills, compositions, docs, benchmark, telemetry, transcribe, tts, or remove-background, or when troubleshooting the HyperFrames build/render environment. Entry point for AWS Lambda cloud rendering (`hyperframes lambda deploy / render / progress / destroy / policies`).

1345AbhishekKumar/starter-supabase

The HyperFrames composition contract — build one renderable project. Use for composition structure, the `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, deterministic-render rules, and validation. Read before writing composition HTML.

1345AbhishekKumar/starter-supabase

Non-animation creative direction for HyperFrames videos. Use for design spec (frame.md / design.md) handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns, and brand / style decisions. For atomic motion patterns and scene blueprints, use `hyperframes-animation`.

1345AbhishekKumar/starter-supabase

Install and wire registry blocks and components into HyperFrames compositions. Use when running hyperframes add, installing a block or component, wiring an installed item into index.html, or working with hyperframes.json. Covers the add command, install locations, block sub-composition wiring, component snippet merging, registry discovery, and authoring a new block or component to contribute upstream (idea → scaffold → validate → PR).

1345AbhishekKumar/starter-supabase

Use when the user wants a short, design-led motion graphic where motion is the message: kinetic typography, stat or number count-up, chart/data-viz hit, logo sting, brand lockup, lower-third, callout, social overlay, animated headline/tweet/news item, motion poster, or quick captured-page highlight. Usually under 10s and up to ~30s, with no narration arc, voice-over, or live-action subject. Can render to MP4 or transparent overlay. Not for longer, multi-scene, narrated, or brand-reel pieces (use general-video), narrated website videos (website-to-video), topic explainers (faceless-explainer), product promos (product-launch-video), PR videos (pr-to-video), or captions on existing footage (embedded-captions). When unsure whether it's a quick motion-first piece or a longer / narrated treatment, see /hyperframes.

1345AbhishekKumar/starter-supabase

Use when the user has a music track (an audio file, or a video to pull audio from) and wants a beat-synced HyperFrames video, calm to hard-hitting. The music drives everything: one analyzer reads it once, the orchestrator lays out the frames and fills a per-frame plan, and one sub-agent builds each frame. Typography and templates are the floor — a complete video needs zero assets — but any images or videos the user supplies are cut into the frames on the same beat grid (beat-cut / ken-burns). The genre (lyric video, slideshow, kinetic promo) falls out of the per-frame choices; the pipeline never branches on it.

1345AbhishekKumar/starter-supabase

turn a product or marketing URL, pasted script, or brief into a product launch video, including SaaS promos, feature reveals, app launches, company promos, and product marketing videos. Use this skill when the user wants to market, launch, promote, or reveal a product. Do not use it for general non-launch website tours, non-product topic explainers, GitHub pull requests, captioning existing footage, or short unnarrated motion graphics. If the intent is unclear, route through /hyperframes first. This is the new shot-sequence architecture: every visual frame is authored as a time-coded shot sequence picked from a menu of golden blueprints, so frames develop over their full duration instead of freezing after entrance.

1345AbhishekKumar/starter-supabase

turn a GitHub pull request (a PR URL like github.com/<owner>/<repo>/pull/<N>, an <owner>/<repo>#<N> ref, or 'this PR' in a checked-out repo) into a code-change explainer video, up to ~3 min (sweet spot 30-90s) — changelog, feature reveal, fix, or refactor walkthrough, rendered from the diff / commits / files. The input is a CODE CHANGE read via the gh CLI; there is no website capture. Use this skill for a GitHub PR. Do not use it for a product launch/promo (use /product-launch-video), a tour of a real website (use /website-to-video), a topic explainer with no PR (use /faceless-explainer), captions on existing footage (use /embedded-captions), or a short unnarrated motion graphic (use /motion-graphics). If the intent is unclear, route through /hyperframes first.

1345AbhishekKumar/starter-supabase

Port an existing Remotion (React) composition to HyperFrames HTML. Use ONLY when the user explicitly asks to port/convert/migrate/translate a Remotion source. Do NOT use: (a) authoring a new HyperFrames composition; (b) Remotion mentioned in passing; (c) Remotion code shared as reference only; (d) "same video as my Remotion one" without explicit migrate request — treat as fresh build. Doubt → `/general-video`. One-way, Remotion-only: no reverse export (HyperFrames→Remotion or any framework), no non-Remotion source (After Effects, Framer Motion, plain React/CSS) → out of scope, re-create via `/general-video`. Flags unsupported patterns (useState, useEffect, async calculateMetadata, third-party React libs, `@remotion/lambda`) and recommends runtime interop over lossy translation. Unsure whether to port vs. build fresh, or only a passing Remotion mention? → /hyperframes.

1345AbhishekKumar/starter-supabase

Capture a general website/URL and turn it into a HyperFrames video (site tour, showcase, or social clip from the site's own visuals). Uses headless Chrome screenshots + brand assets. Use when intent is general — portfolio/blog/landing-page showcase or social clip from the site. NOT for: product/SaaS launch or promo (→ /product-launch-video, even from a URL); topic explainer with no site (→ /faceless-explainer); GitHub PR (→ /pr-to-video); adding captions to existing video (→ /embedded-captions); short unnarrated page-highlight motion graphic (→ /motion-graphics). Unclear launch-vs-general-site? Ask one question or start at /hyperframes.

相關技能