Communitygithub.com

1345AbhishekKumar/starter-nextjs

為 HyperFrames 創作提供音訊與媒體資產,由統一的音訊引擎驅動,支援多供應商 TTS、背景音樂、音效、轉錄、字幕及去背功能。

starter-nextjs 是什麼?

HyperFrames Media 是一個為 HyperFrames 組合型創作提供完整音訊與媒體資產的技能,其核心是單一的 `scripts/audio.mjs` 引擎。該技能支援多供應商的文字轉語音(包括 HeyGen、ElevenLabs 及本機 Kokoro)、背景音樂與音效(預設透過 HeyGen 音訊庫檢索,無認證時則使用本機 Lyria / MusicGen 生成或內建音效庫)、Whisper 語音轉錄、去背及字幕撰寫。此技能設計用於 Claude Code、Coder、Cursor 及 Codex 等 AI 代理平台,開發者可透過統一的 `audio_request.json` 請求格式來驅動所有音訊生成,最終產出結構化的 `audio_meta.json` 與資產檔案。技能的關鍵優勢在於其優雅的降級機制:根據是否有 HeyGen 憑證,自動在雲端與本機引擎之間切換,並透過詳盡的參考文件提供供應商鏈、路由規則與不可協商的開發指引,確保工作流程的一致性與可靠性。

相容平台~Claude Code~Codex CLI~Cursor
npx skills add https://github.com/1345AbhishekKumar/starter-nextjs/tree/HEAD/.agents/skills/hyperframes-media

在你喜歡的 AI 中提問

開啟一個已預先載入此 Agent Skill 的新對話。

說明文件

HyperFrames 媒體資產

為創作所需的音訊與媒體資產——配音(TTS)、背景音樂 + 音效、轉錄、字幕、去背——然後在 HTML 中消費這些資料並製作動畫。若要將資產放入組合中,請參閱 hyperframes-core。

音訊引擎——TTS · BGM · SFX 的單一來源

工作流程不應自行處理音訊或使用供應商的複製品。只有一個引擎——scripts/audio.mjs——它接受中性的 audio_request.json 並寫出 audio_meta.json(以及在 assets/voice|bgm|sfx 下的資產):

# <MEDIA_DIR> = 此技能的路徑
node <MEDIA_DIR>/scripts/audio.mjs --request ./audio_request.json --hyperframes . --out ./audio_meta.json

所有三項功能的降級取決於單一切換開關——是否有 HeyGen 憑證(從 $HEYGEN_API_KEY / $HYPERFRAMES_API_KEY / ~/.heygen 解析,不是 CLI 參數):

功能HeyGen 憑證存在憑證不存在
TTSHeyGen Starfish REST(原生單詞時間戳)→ ElevenLabs → Kokoro(鏈結 transcribe 以取得單詞)
BGMHeyGen 音樂檢索Lyria → MusicGen 本機生成(分離執行)
SFXHeyGen 音效檢索(min_score 0.4)內建 21 檔案庫(assets/sfx/)
  • 請求(audio_request.json):{ provider?, lang?, speed?, lines: [{ id, text, sfx?: [names] }], bgm: { mode?, query?, prompt? } }。id 將每一行與呼叫者的模型(影格編號、場景 ID 等)連接起來。bgm.mode = retrieve | generate | none;省略則自動(有憑證時檢索,否則生成)。明確的 retrieve 是嚴格的——它會跳過而非啟動分離的生成(適用於沒有 wait-bgm 步驟的呼叫者)。
  • 輸出(audio_meta.json,以 ID 為鍵):{ tts_provider, voice_id, bgm, bgm_pending, …, voices: [{ id, path, duration_s, words }], sfx: [{ id, name, file, source, offset_s, duration_s, volume }], total_duration_s }。
  • --only tts,bgm,sfx 執行子集並合併到現有的 --out 中(例如,提前執行 TTS+BGM,等線索出現後再執行 SFX)。
  • BGM 生成以分離方式啟動(bgm_pending: true)——在組合前執行 scripts/wait-bgm.mjs。
  • scripts/heygen-tts.mjs 是一個基於相同程式碼的一次性 CLI(一個文本 → wav + 單詞),當你只需要 HeyGen TTS 而無需請求檔案時使用。

完整的選項列表和 audio_meta.json 的結構位於 scripts/audio.mjs 的標頭註解中。下面的參考資料涵蓋了每個功能背後的供應商細節和邊緣情況。

前置檢查——在任何音訊生成前顯示登入狀態

在生成語音或 BGM 之前,始終執行此步驟——無論是在完整工作流程中還是單次「為我生成 BGM/配音」請求中。 沒有 HeyGen 憑證不應被當作默默回退到本機引擎的理由:首先建議登入,讓使用者決定。執行共享的前置檢查並逐字傳遞其輸出——不要自行設計「缺少金鑰」的提示,也不要主動將金鑰寫入每個儲存庫的 .env:

npx hyperframes auth status
  • 已登入 → 它會列印帳戶資訊;繼續進行。
  • 未登入(exit 1 是預期的——「未登入」是正常狀態,不是失敗)→ 它會列印註冊優先的指引。建議登入:npx hyperframes auth login 是瀏覽器 OAuth——它會登入並建立帳戶(始終可透過此儲存庫的 CLI 使用)。要使用現有的 HeyGen API 金鑰(來自 app.heygen.com/settings/api),請執行 npx hyperframes auth login --api-key——它會儲存到共享的 ~/.heygen(無需每個儲存庫的 .env)。輸出還會列出語音/BGM 會回退到的本機引擎以及缺少依賴時的 pip 提示。逐字傳遞此輸出——不要用自己的話重新解釋。 然後停止並等待使用者選擇——登入,或說「go」/「local」以繼續離線進行——在生成任何內容之前。 這是一個真正的決策點,不是一個隨意的備註:不要將其摺疊到另一個問題中,也不要自行越過它。(例外:在自主/非互動模式中,記錄狀態並繼續離線進行。)
  • npx hyperframes auth status --json 會返回 { configured, recommended_action, offline_engines } 以進行確定性分支。
  • 如果 CLI 無法執行(不在 PATH 上且 npx 無法獲取)→ 仍然建議登入(npx hyperframes auth login)並停止等待使用者選擇——不要把「沒有憑證」當作本機生成的默許綠燈。

憑證解析、完整金鑰優先順序以及本機依賴清單在 references/requirements.md 中。

供應商鏈(引擎背後的細節)

TTS——第一個可用的供應商勝出(引擎,或 npx hyperframes tts "..."):

順序供應商偵測時機單詞時間戳
1HeyGen (Starfish)$HEYGEN_API_KEY / hyperframes auth login是,原生——傳遞 --words narration.words.json 以捕獲
2ElevenLabs$ELEVENLABS_API_KEY 已設定否——後續鏈結 transcribe
3Kokoro-82M(本機,54 種語音)始終(無需金鑰)否——後續鏈結 transcribe

已發布的 hyperframes tts CLI 通常是僅本機的建置(其 --help 顯示「Kokoro-82M」,沒有 --provider/--words),即使設定了 $HEYGEN_API_KEY,它也會默默地回退到 Kokoro。這就是為什麼引擎的 HeyGen 路徑是自包含的 scripts/heygen-tts.mjs(REST),而不是 CLI;CLI 僅用於 Kokoro 路徑。請參閱 references/tts.md。

BGM 與 SFX——預設從 HeyGen 音訊庫檢索(/v3/audio/sounds),使用與 HeyGen TTS 相同的憑證,無憑證時則使用上述開關的回退:

資產HeyGen type存放位置回退(無憑證)
BGMmusicassets/bgm/track.mp3(檢索)· track.wav(生成)Lyria / MusicGen 生成
SFXsound_effects(min_score 0.4)assets/sfx/<slug>.mp3內建 21 檔案庫(assets/sfx/* + manifest.json)

請參閱 references/bgm.md 和 references/sfx.md。

路由

任務閱讀
音訊引擎——請求/輸出結構、--only、切換開關scripts/audio.mjs(標頭註解)
npx hyperframes tts / heygen-tts.mjs——供應商、語音、單詞references/tts.md
BGM——HeyGen 檢索 + 本機 Lyria / MusicGen 生成references/bgm.md
SFX——HeyGen 檢索(min_score 0.4)+ 內建本機庫references/sfx.md
npx hyperframes transcribe——Whisper、模型規則、輸出格式references/transcribe.md
npx hyperframes remove-background——透明裁剪references/remove-background.md
TTS → 轉錄 → 字幕(無錄製配音)references/tts-to-captions.md
字幕撰寫——樣式偵測、佈局、單詞分組、結束references/captions/authoring.md
轉錄處理——輸入格式、品質閘門、清理、APIreferences/captions/transcript-handling.md
字幕動畫——卡拉 OK、標記效果、音訊反應式references/captions/motion.md
模型快取、系統依賴、故障排除references/requirements.md

不可協商的規則

  • 一個引擎,沒有複製品。 透過 scripts/audio.mjs(或一次性 HeyGen TTS 的 heygen-tts.mjs)產生音訊。不要在工作流程中重新實作 TTS/BGM/SFX——請撰寫 audio_request.json 適配器並呼叫引擎。
  • 「HeyGen 可用」= 可解析的憑證,不是 CLI。 整個切換開關以 heygenCredential() 為依據;已發布的 hyperframes tts 可能僅支援 Kokoro,並且根本沒有 hyperframes bgm / hyperframes sfx 命令。
  • 語音 ID 是供應商特定的。 am_michael 僅適用於 Kokoro;HeyGen UUID 不適用於 Kokoro。如果你傳遞 --voice,也請固定 --provider,以避免在使用者環境變更時發生無聲的供應商漂移。
  • 始終將 --model 傳遞給 transcribe。 CLI 預設值 small.en 會默默地翻譯非英語音訊。請參閱 references/transcribe.md →「語言規則」。
  • HeyGen 返回單詞時間戳;ElevenLabs / Kokoro 則否。 引擎會自動為後兩者鏈結 transcribe;單獨使用時,請對 HeyGen 傳遞 --words,或對音訊檔案執行 transcribe。
  • 字幕消耗扁平的單詞陣列格式,包含 { id, text, start, end }。請參閱 references/transcribe.md →「輸出格式」。
  • remove-background --background-output 是挖洞,不是修補。 對於「不含人物的場景」,需要不同的工具。請參閱 references/remove-background.md →「何時不是正確的工具」。
  • BGM/SFX 預設為 HeyGen 檢索;無憑證時的回退是生成(BGM)或內建庫(SFX)。 /audio/sounds 根據文字查詢排名——具體命名音效(glass shatter,而不是 dramatic sound);未匹配則跳過,絕不會阻擋渲染。SFX 在語音 + BGM 下的音量約為 0.35。請參閱 references/sfx.md / references/bgm.md。
  • 將工作流程字幕 HTML 視為生成的輸出。 對於基於預設的影片,可重複使用的皮膚原始碼位於 .hyperframes/caption-skin.html,而工作流程腳本會寫入 compositions/captions.html;不要編輯生成的 compositions/captions.html 來修復皮膚。請透過工作流程的 captions.mjs 重建,或在使用時使用該工作流程的顯式覆寫機制。

Individual skills in this repo

This repo contains 13 individual skills — each has its own dedicated page.

1345AbhishekKumar/starter-nextjs

Add captions to a talking-head video. ONE catalog (CATALOG.md) of 32 visual identities behind two engines: column-flow (captions composited INTO the scene — matte occlusion + mix-blend; cream/ink/editorial/keynote/documentary/loud/neon/glitch/chrome/velocity) and themed constitutions (anchor/ordnance/terminal/neonsign/stardust/stomp/scoreboard/transit/vhs/arcade/dossier/laser/thunder/hologram/biolume/aurora/spectrum/papercut/popup/chalkboard/graffiti/brush/inkwater/ransom/lastpage/nightcity — e.g. a glyph-decode climax, a neon sign WRITTEN stroke by stroke, or the quiet `anchor` rail default). Route by identity, never by mode. Trigger on "captions/subtitles", "embed/cinematic captions", "VFX captions", "炸/特效/酷炫字幕", a named identity, or top-tier motion-graphics asks. Embedding every word is wrong for most talking-head content — `anchor` is the verbatim default. Pipeline: transcription → hyperframes remove-background matting → HTML render → ffmpeg overlay. Requires hyperframes and a single-subject clip.

1345AbhishekKumar/starter-nextjs

The fallback workflow for authoring custom HyperFrames video compositions at any length or format — longer or multi-scene pieces, brand / sizzle reels, montages, title cards, static loops, and freeform compositions. Input- and length-agnostic. If a specialized workflow clearly fits the input — a marketed product, a website, a topic explainer, a GitHub PR, existing footage, a short motion graphic, or a Remotion port — prefer it (see /hyperframes); use this only as the general fallback when none fit.

1345AbhishekKumar/starter-nextjs

All animation knowledge for HyperFrames — atomic motion rules, multi-phase scene blueprints, scene transitions, broader motion-design techniques, AND the seven runtime adapters (GSAP default, plus Lottie, Three.js, Anime.js, CSS keyframes, Web Animations API, TypeGPU). Use for any motion or animation task: pick 2-4 rules and compose, or load a blueprint, or look up runtime-specific API (e.g. GSAP eases / Lottie player / Three.js mixer). HyperFrames-native: single paused timeline, seek-safe, deterministic.

1345AbhishekKumar/starter-nextjs

HyperFrames CLI dev loop. Use when running npx hyperframes init, add, catalog, capture, lint, validate, inspect, layout, snapshot, preview, play, render, publish, lambda, doctor, browser, info, upgrade, skills, compositions, docs, benchmark, telemetry, transcribe, tts, or remove-background, or when troubleshooting the HyperFrames build/render environment. Entry point for AWS Lambda cloud rendering (`hyperframes lambda deploy / render / progress / destroy / policies`).

1345AbhishekKumar/starter-nextjs

The HyperFrames composition contract — build one renderable project. Use for composition structure, the `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, deterministic-render rules, and validation. Read before writing composition HTML.

1345AbhishekKumar/starter-nextjs

Non-animation creative direction for HyperFrames videos. Use for design spec (frame.md / design.md) handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns, and brand / style decisions. For atomic motion patterns and scene blueprints, use `hyperframes-animation`.

1345AbhishekKumar/starter-nextjs

Install and wire registry blocks and components into HyperFrames compositions. Use when running hyperframes add, installing a block or component, wiring an installed item into index.html, or working with hyperframes.json. Covers the add command, install locations, block sub-composition wiring, component snippet merging, registry discovery, and authoring a new block or component to contribute upstream (idea → scaffold → validate → PR).

1345AbhishekKumar/starter-nextjs

Use when the user wants a short, design-led motion graphic where motion is the message: kinetic typography, stat or number count-up, chart/data-viz hit, logo sting, brand lockup, lower-third, callout, social overlay, animated headline/tweet/news item, motion poster, or quick captured-page highlight. Usually under 10s and up to ~30s, with no narration arc, voice-over, or live-action subject. Can render to MP4 or transparent overlay. Not for longer, multi-scene, narrated, or brand-reel pieces (use general-video), narrated website videos (website-to-video), topic explainers (faceless-explainer), product promos (product-launch-video), PR videos (pr-to-video), or captions on existing footage (embedded-captions). When unsure whether it's a quick motion-first piece or a longer / narrated treatment, see /hyperframes.

1345AbhishekKumar/starter-nextjs

Use when the user has a music track (an audio file, or a video to pull audio from) and wants a beat-synced HyperFrames video, calm to hard-hitting. The music drives everything: one analyzer reads it once, the orchestrator lays out the frames and fills a per-frame plan, and one sub-agent builds each frame. Typography and templates are the floor — a complete video needs zero assets — but any images or videos the user supplies are cut into the frames on the same beat grid (beat-cut / ken-burns). The genre (lyric video, slideshow, kinetic promo) falls out of the per-frame choices; the pipeline never branches on it.

1345AbhishekKumar/starter-nextjs

turn a product or marketing URL, pasted script, or brief into a product launch video, including SaaS promos, feature reveals, app launches, company promos, and product marketing videos. Use this skill when the user wants to market, launch, promote, or reveal a product. Do not use it for general non-launch website tours, non-product topic explainers, GitHub pull requests, captioning existing footage, or short unnarrated motion graphics. If the intent is unclear, route through /hyperframes first. This is the new shot-sequence architecture: every visual frame is authored as a time-coded shot sequence picked from a menu of golden blueprints, so frames develop over their full duration instead of freezing after entrance.

1345AbhishekKumar/starter-nextjs

turn a GitHub pull request (a PR URL like github.com/<owner>/<repo>/pull/<N>, an <owner>/<repo>#<N> ref, or 'this PR' in a checked-out repo) into a code-change explainer video, up to ~3 min (sweet spot 30-90s) — changelog, feature reveal, fix, or refactor walkthrough, rendered from the diff / commits / files. The input is a CODE CHANGE read via the gh CLI; there is no website capture. Use this skill for a GitHub PR. Do not use it for a product launch/promo (use /product-launch-video), a tour of a real website (use /website-to-video), a topic explainer with no PR (use /faceless-explainer), captions on existing footage (use /embedded-captions), or a short unnarrated motion graphic (use /motion-graphics). If the intent is unclear, route through /hyperframes first.

1345AbhishekKumar/starter-nextjs

Port an existing Remotion (React) composition to HyperFrames HTML. Use ONLY when the user explicitly asks to port/convert/migrate/translate a Remotion source. Do NOT use: (a) authoring a new HyperFrames composition; (b) Remotion mentioned in passing; (c) Remotion code shared as reference only; (d) "same video as my Remotion one" without explicit migrate request — treat as fresh build. Doubt → `/general-video`. One-way, Remotion-only: no reverse export (HyperFrames→Remotion or any framework), no non-Remotion source (After Effects, Framer Motion, plain React/CSS) → out of scope, re-create via `/general-video`. Flags unsupported patterns (useState, useEffect, async calculateMetadata, third-party React libs, `@remotion/lambda`) and recommends runtime interop over lossy translation. Unsure whether to port vs. build fresh, or only a passing Remotion mention? → /hyperframes.

1345AbhishekKumar/starter-nextjs

Capture a general website/URL and turn it into a HyperFrames video (site tour, showcase, or social clip from the site's own visuals). Uses headless Chrome screenshots + brand assets. Use when intent is general — portfolio/blog/landing-page showcase or social clip from the site. NOT for: product/SaaS launch or promo (→ /product-launch-video, even from a URL); topic explainer with no site (→ /faceless-explainer); GitHub PR (→ /pr-to-video); adding captions to existing video (→ /embedded-captions); short unnarrated page-highlight motion graphic (→ /motion-graphics). Unclear launch-vs-general-site? Ask one question or start at /hyperframes.

相關技能