Communitygithub.com

starter-nextjs

将音乐曲目转换为节拍同步的 HyperFrames 视频。分析音频一次,编排帧布局,填充每帧计划,并构建每帧为合成体。支持从平静到强烈等多种风格,无需任何素材即可通过字体和模板生成完整视频。

starter-nextjs 是什么?

该技能将音乐轨道转化为节拍同步的 HyperFrames 视频,由单一分析器驱动。它允许用户提供音频文件(或从中提取音频的视频),并可选地提供图像或视频素材。视频风格可涵盖抒情视频、幻灯片或动态宣传片,具体取决于每帧的选择,而非预定义类型。工作流程包括七个步骤:设置和背景音乐放置、音乐分析(生成 audiomap.json)、帧骨架创建、用户审批计划、通过子代理构建每帧、组装最终 index.html,以及验证和渲染 MP4。支持 HyperFrames 作为平台,并强调信任单一分析器(analyze-beatgrid.py)以实现节拍检测。该技能适用于需要自动化、节拍同步视频制作的开发者,适合宣传片、音乐视频或内容创作,无需手动编辑或外部设计工具。

兼容平台✓Claude Code~Codex CLI~Cursor
npx skills add https://github.com/1345AbhishekKumar/starter-nextjs/tree/HEAD/.agents/skills/music-to-video

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

music-to-video —— 一个基于音乐、节拍同步的视频工作流

使用此技能将音乐曲目转换为节拍同步的 HyperFrames 视频。您分析曲目一次,布局帧,填充每帧计划,并将每帧构建为合成体。输入是音乐曲目加上用户可选的图像或视频——没有旁白,也没有网站捕获。字体和模板是基础(完整视频不需要任何素材);用户提供的任何媒体都会在相同的节拍网格上裁剪进去。

您是编排者。在 videos/<project>/ 中工作。按顺序执行步骤,并在继续之前通过每个关卡。两个步骤需要用户:步骤 3(计划批准)和步骤 6(渲染批准)。除步骤 4 外,您需要自己完成每一步,在该步骤中,您为每帧分配一个子代理。将设计和运动规则保留在此文件之外——它们位于 references/ 和 frame-worker 子代理中。

SKILL_DIR = 此技能目录。PROJECT_DIR = videos/<project-name>/。

工作流:步骤 0 设置 → hyperframes.json + assets/bgm.mp3;步骤 1 分析 → audiomap.json;步骤 2 骨架 → STORYBOARD.md(帧,组 TBD);步骤 3 计划 → 完整 STORYBOARD.md + frame.md;步骤 4 构建 → compositions/frames/NN-*.html;步骤 5 组装 → index.html;步骤 6 渲染 → renders/video.mp4。

塑造一切的两个理念

  • 一个分析器,并且您相信它。 analyze-beatgrid.py 是唯一的节拍分析器——绝不要用其他工具或通过耳朵重新测量节拍。其能量/密度/滚动/起始/静音始终可靠。其 bpm 和 beats_sec 仅在音乐真正有节奏时才可靠;在平静的音乐上,网格是跟踪器强加的节拍器,因此根据乐句和能量来定速,绝不要硬切到它。判断您处于哪种情况是每帧的 pacing(步骤 2)。
  • 一帧 = 一个文件;组位于内部。 步骤 2 将曲目切割成帧,每帧变成一个合成文件 compositions/frames/NN-<frame_id>.html,由一个 frame-worker 构建。一帧可以细分为组(每个组是一个模板或运动原语组合)。额外密度放在_组内部_,因此帧计数跟踪不同的处理方式,而不是节拍——快速曲目不会增加子代理的数量。

---\n

步骤 0:设置、背景音乐和输入

目标:确定音乐源,创建 HyperFrames 项目,并记录用户提供的任何媒体。

音乐是主线——在一切之前确定一首曲目。此技能针对快节奏、高能量背景音乐进行了调优:强大的节拍网格驱动剪辑(平静曲目也可以,但按乐句而不是节拍定速)。如果用户给了您音频——音乐文件,或要从中提取音频的视频——请使用它。如果没有,生成一个:根据用户的描述选择情绪(例如“驱动力合成波”、“陷阱节拍”、“欢快企业风”),并通过 /hyperframes-media 生成曲目(references/bgm.md —— 有凭证时使用 HeyGen 检索,否则使用本地 Lyria / MusicGen;ElevenLabs 或其他生成器也可用)。在生成之前,运行 npx hyperframes auth status 并逐字输出其结果(不要解释或重写它)——它显示背景音乐是来自 HeyGen 还是本地 MusicGen,如果未登录,则显示如何登录。如果未登录,则停止并等待用户选择——登录,或在离线状态下继续使用本地 MusicGen——然后再生成曲目;不要将密钥写入每个仓库的 .env 文件中。(在自主模式下,记下状态并继续离线)。有关规范指南,请参阅 /hyperframes-media → Preflight。无论哪种方式,曲目都会放在 assets/bgm.mp3 中。暂存用户提供的任何图像或视频,以便帧可以在节拍网格上编织它们;否则,字体会承载整个视频。

仅当 hyperframes.json 缺失时才初始化。根据简要说明以短横线命名法命名 <project>,例如 midnight-drive-loop——绝不要使用时间戳。init 检查已安装的技能与 GitHub 上的最新版本,并更新全局集合(如果任何技能过时)。

npx hyperframes init "videos/<project>" --non-interactive --example=blank
mkdir -p "$PROJECT_DIR/assets" "$PROJECT_DIR/renders"
cp "<user-music>" "$PROJECT_DIR/assets/bgm.mp3"   # 如果需要,先从视频中提取
# 仅当用户提供了图像/视频时:
node <SKILL_DIR>/scripts/stage-assets.mjs --from <dir> --hyperframes "$PROJECT_DIR" --into public

品牌(字体+调色板)在步骤 3 中选择,而不是在这里。不要预先选择流派或曲目类型——资产只是一个可选成分,流派从每帧选择中浮现。

关卡: hyperframes.json + assets/bgm.mp3 存在;宽高比/时长/帧率以及(如果有)资产清单已记录。


步骤 1:分析音乐

目标:生成整个视频构建所依赖的唯一规范时间分析。

analyze-beatgrid.py 是唯一的节拍分析器——绝不要用其他工具或通过耳朵重新测量节拍。它读取曲目一次并将 audiomap.json 写入:能量阶段(水平/密度/感觉)、起始 + onset_rate、滚动、静音、hard_stops、key_moments、乐句、节奏/网格,以及 audio.duration_sec。它是确定性的——相同的文件总是给出相同的地图。大多数字段在任何音乐上都是可靠的;bpm 和 beats_sec 仅在音乐真正有节奏时才可靠,判断这一点是您在步骤 2 中做出的决定。

先决条件:Python 3 具有可用的 librosa、numpy 和 soundfile。如果导入失败,请在运行分析器之前将它们安装到活动的 Python 环境中:

python3 -m pip install librosa numpy soundfile
python3 <SKILL_DIR>/scripts/analyze-beatgrid.py "$PROJECT_DIR/assets/bgm.mp3" \
  -o "$PROJECT_DIR/audiomap.json" --print

关卡: audiomap.json 存在;audio.duration_sec 已知。


步骤 2:帧骨架(仅结构)

目标:读取音乐并布局帧——STORYBOARD.md 的骨架。

阅读 references/frame-skeleton.md。自己将 audiomap.json 转为 STORYBOARD.md 的骨架——没有中间 JSON。在真实的音乐变化处(hard_stops、SURGE / DROP key_moments、滚动的边缘、没有起始的段落、大的能量跳跃)将曲目切割成帧,将每个边界对齐到 audiomap 锚点。对于每帧,设置 span_sec、pacing(步骤 1 信任调用的判定结果——当网格真实时为 beat_cut,当它是强加于平静音乐的节拍器时为 phrase_flow)、mood,以及一行 feel(步骤 3 将模板与之匹配的普通音乐情况)。这里只进行分类和布局:将每帧的 ### Groups 留为 TBD (Step 3),并将前置元数据 style 留空——没有模板、文案、颜色或字体。预期约 1-6 帧。

关卡: 帧平铺整个曲目(第一个在 0,最后一个在 duration_s);每帧带有 span_sec + pacing + mood + feel;每个 ### Groups 是 TBD;任何地方都没有内容。


步骤 3:填充计划(用户关卡)

目标:将骨架变为已批准、完整的 STORYBOARD.md。

阅读 references/planning.md、storyboard-format.md、template-catalog.md、motion-primitive-catalog.md 和 montage.md(仅当用户提供了资产时)。就地编辑同一文件,执行两项操作:

  1. 选择品牌。 从 ../hyperframes-creative/frame-presets/ 中使用 ../hyperframes-creative/references/design-spec.md 中的表格选择一个预设(匹配曲目的情绪;只有其字体和颜色重要——模板拥有构图)。将其未经修改地复制到 frame.md 中,并从中填充前置元数据 style(字体 + 最多 4-6 种颜色的调色板)。
  2. 填充每帧。 决定其组并为每个组提供一种处理方式:从目录中匹配的模板(带有绑定参数和真实的 audiomap 锚点)、从原语目录中自由组合,或遵守 pacing 的资产处理方式。编写文案。您拥有 WHAT(模板/原语 + 内容 + 锚点);frame-worker 拥有 HOW——绝不要将毫秒级补间写入故事板。
node <SKILL_DIR>/scripts/validate-plan.mjs --storyboard "$PROJECT_DIR/STORYBOARD.md" \
  --audiomap "$PROJECT_DIR/audiomap.json" --templates <SKILL_DIR>/references/templates

修复每个 ✗(硬错误:时长不匹配、帧未平铺曲目、缺少 src);警告是尽力而为。然后向用户显示逐帧摘要并迭代,直到他们批准。

关卡: frame.md 是预设的逐字副本;validate-plan.mjs 以 0 退出;用户批准了计划。


步骤 4:从计划构建帧

目标:将每帧构建为一个自包含的合成文件。

创建 compositions/frames/。阅读 sub-agents/frame-worker.md 和 ../hyperframes-core/references/subagent-dispatch.md。为每帧分配一个 frame-worker,尽可能并行(否则分批进行)。每个 worker 只获得一帧和以下上下文:

PROJECT_DIR: <绝对路径>
frame_id: <NN-frame_id>              # = 帧文件名主干,例如 02-f2;合成 ID
您的块:`## Frame N — <frame_id>` 块在 PROJECT_DIR/STORYBOARD.md 中
audiomap: PROJECT_DIR/audiomap.json
frame.md: PROJECT_DIR/frame.md
材料:对于每组,<SKILL_DIR>/references/templates/<id>/index.html(模板)和
          <SKILL_DIR>/references/motion-primitives/<id>/(自由组合);暂存的 assets/(资产组)
合同:../hyperframes-core/references/sub-compositions.md + determinism-rules.md
画布:<w>×<h>   定速:<beat_cut|phrase_flow>
写入到:PROJECT_DIR/compositions/frames/<frame_id>.html

Worker 复制引用的材料,将每个锚点转换为帧局部秒(local_t = track_t − span_sec[0]),用 0ms 剪辑门控其组,并写入一个可安全查找的帧文件。Worker 从不运行 hyperframes CLI——这些命令操作的是组装后的项目,该项目尚不存在,因此它们会报告错误的文件。Worker 只需写入合同并停止;您在组装后验证(步骤 6)。当每个 worker 返回时,您可以确认其文件已写入磁盘。

关卡: 每帧都有其 compositions/frames/NN-*.html 在磁盘上。


步骤 5:组装

目标:将构建的帧 + 背景音乐连接到可播放的 index.html 中。

assemble-index.mjs 是确定性的——没有子代理,没有判断。它以其累积的 data-start 引用每个帧文件,将 assets/bgm.mp3 挂载到轨道 11,并硬切帧到帧(帧平铺整个曲目,没有间隙,因此没有过渡注入器)。

node <SKILL_DIR>/scripts/assemble-index.mjs --storyboard "$PROJECT_DIR/STORYBOARD.md" \
  --hyperframes "$PROJECT_DIR" --audiomap "$PROJECT_DIR/audiomap.json"

修复它报告的任何 ✗——缺少或空白的帧文件意味着该 worker 写入了部分文件;重新分配它(步骤 4)并重新组装。

关卡: index.html 存在;总时长 == audiomap.audio.duration_sec。


步骤 6:验证并渲染

目标:验证组装的视频,获得用户批准,并渲染最终的 MP4。

在组装后的项目上运行 CLI——这是正确的单元(每帧 workers 无法运行它)。lint 检查结构,validate 运行无头 Chrome(捕获 JS 错误和丢失的资产),inspect 快照帧。

( cd "$PROJECT_DIR" && npx hyperframes lint . && npx hyperframes validate . && npx hyperframes inspect . )

在 t=0、每个帧开始、最强的 DROP / SURGE、每个 hard_stops[].t 以及最后一帧处检查。如果失败,自行进行最便宜的安全修复:编辑有问题的 compositions/frames/NN-*.html。绝不要改变时长或音频时间线来隐藏同步问题。一旦关卡通过,暂停让用户审查,然后仅在批准后渲染:

( cd "$PROJECT_DIR" && npx hyperframes render . --skill=music-to-video -q draft -o renders/video.mp4 --fps 30 )

关卡: lint / validate / inspect 通过;用户批准;renders/video.mp4 存在并带有音频,时长 == audiomap.audio.duration_sec。最终回复说明 MP4 路径和时长。


恢复表

您拥有从何处继续
仅 assets/bgm.mp3步骤 1
audiomap.json步骤 2
STORYBOARD.md(骨架)步骤 3
STORYBOARD.md(完整)步骤 4
所有帧文件步骤 5
index.html步骤 6

快速参考

格式: 默认横向 1920x1080;纵向 1080x1920;方形 1080x1080。在故事板前置元数据中设置一次画布(canvas: { w, h, fps })。

脚本 位于 scripts/:analyze-beatgrid.py(唯一分析器)、validate-plan.mjs(计划检查)、assemble-index.mjs(索引组装)、stage-assets.mjs(暂存用户媒体)、lib/storyboard.mjs(供应商解析器)。其他一切均为 hyperframes CLI。

读取何时
references/frame-skeleton.md步骤 2:读取音乐,布局帧,设置定速
references/planning.md · storyboard-format.md步骤 3:选择品牌,填充每帧,编写计划
references/template-catalog.md步骤 3:为每组选择模板
references/motion-primitive-catalog.md步骤 3:为自由组合组选择运动原语
references/montage.md步骤 3:资产剪辑(节拍剪切、Ken Burns)
sub-agents/frame-worker.md步骤 4:构建每帧
../hyperframes-core/references/subagent-dispatch.md步骤 4:分派 frame-worker
../hyperframes-creative/references/design-spec.md步骤 3:查找预设并选择品牌

Individual skills in this repo

This repo contains 13 individual skills — each has its own dedicated page.

1345AbhishekKumar/starter-nextjs

Add captions to a talking-head video. ONE catalog (CATALOG.md) of 32 visual identities behind two engines: column-flow (captions composited INTO the scene — matte occlusion + mix-blend; cream/ink/editorial/keynote/documentary/loud/neon/glitch/chrome/velocity) and themed constitutions (anchor/ordnance/terminal/neonsign/stardust/stomp/scoreboard/transit/vhs/arcade/dossier/laser/thunder/hologram/biolume/aurora/spectrum/papercut/popup/chalkboard/graffiti/brush/inkwater/ransom/lastpage/nightcity — e.g. a glyph-decode climax, a neon sign WRITTEN stroke by stroke, or the quiet `anchor` rail default). Route by identity, never by mode. Trigger on "captions/subtitles", "embed/cinematic captions", "VFX captions", "炸/特效/酷炫字幕", a named identity, or top-tier motion-graphics asks. Embedding every word is wrong for most talking-head content — `anchor` is the verbatim default. Pipeline: transcription → hyperframes remove-background matting → HTML render → ffmpeg overlay. Requires hyperframes and a single-subject clip.

1345AbhishekKumar/starter-nextjs

The fallback workflow for authoring custom HyperFrames video compositions at any length or format — longer or multi-scene pieces, brand / sizzle reels, montages, title cards, static loops, and freeform compositions. Input- and length-agnostic. If a specialized workflow clearly fits the input — a marketed product, a website, a topic explainer, a GitHub PR, existing footage, a short motion graphic, or a Remotion port — prefer it (see /hyperframes); use this only as the general fallback when none fit.

1345AbhishekKumar/starter-nextjs

All animation knowledge for HyperFrames — atomic motion rules, multi-phase scene blueprints, scene transitions, broader motion-design techniques, AND the seven runtime adapters (GSAP default, plus Lottie, Three.js, Anime.js, CSS keyframes, Web Animations API, TypeGPU). Use for any motion or animation task: pick 2-4 rules and compose, or load a blueprint, or look up runtime-specific API (e.g. GSAP eases / Lottie player / Three.js mixer). HyperFrames-native: single paused timeline, seek-safe, deterministic.

1345AbhishekKumar/starter-nextjs

HyperFrames CLI dev loop. Use when running npx hyperframes init, add, catalog, capture, lint, validate, inspect, layout, snapshot, preview, play, render, publish, lambda, doctor, browser, info, upgrade, skills, compositions, docs, benchmark, telemetry, transcribe, tts, or remove-background, or when troubleshooting the HyperFrames build/render environment. Entry point for AWS Lambda cloud rendering (`hyperframes lambda deploy / render / progress / destroy / policies`).

1345AbhishekKumar/starter-nextjs

The HyperFrames composition contract — build one renderable project. Use for composition structure, the `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, deterministic-render rules, and validation. Read before writing composition HTML.

1345AbhishekKumar/starter-nextjs

Non-animation creative direction for HyperFrames videos. Use for design spec (frame.md / design.md) handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns, and brand / style decisions. For atomic motion patterns and scene blueprints, use `hyperframes-animation`.

1345AbhishekKumar/starter-nextjs

Audio and media assets for HyperFrames compositions, produced by one shared audio engine (`scripts/audio.mjs`) — multi-provider TTS (HeyGen / ElevenLabs / Kokoro local), background music + sound effects (HeyGen audio-library retrieval by default, with local Lyria / MusicGen BGM generation and a bundled SFX library as the no-credential fallback), Whisper transcription, background removal, and caption authoring. Use for voiceover / TTS, BGM, SFX / sound effects, transcription, captions / subtitles / lyrics / karaoke / per-word styling, voice + provider selection, and music-mood prompting.

1345AbhishekKumar/starter-nextjs

Install and wire registry blocks and components into HyperFrames compositions. Use when running hyperframes add, installing a block or component, wiring an installed item into index.html, or working with hyperframes.json. Covers the add command, install locations, block sub-composition wiring, component snippet merging, registry discovery, and authoring a new block or component to contribute upstream (idea → scaffold → validate → PR).

1345AbhishekKumar/starter-nextjs

Use when the user wants a short, design-led motion graphic where motion is the message: kinetic typography, stat or number count-up, chart/data-viz hit, logo sting, brand lockup, lower-third, callout, social overlay, animated headline/tweet/news item, motion poster, or quick captured-page highlight. Usually under 10s and up to ~30s, with no narration arc, voice-over, or live-action subject. Can render to MP4 or transparent overlay. Not for longer, multi-scene, narrated, or brand-reel pieces (use general-video), narrated website videos (website-to-video), topic explainers (faceless-explainer), product promos (product-launch-video), PR videos (pr-to-video), or captions on existing footage (embedded-captions). When unsure whether it's a quick motion-first piece or a longer / narrated treatment, see /hyperframes.

1345AbhishekKumar/starter-nextjs

turn a product or marketing URL, pasted script, or brief into a product launch video, including SaaS promos, feature reveals, app launches, company promos, and product marketing videos. Use this skill when the user wants to market, launch, promote, or reveal a product. Do not use it for general non-launch website tours, non-product topic explainers, GitHub pull requests, captioning existing footage, or short unnarrated motion graphics. If the intent is unclear, route through /hyperframes first. This is the new shot-sequence architecture: every visual frame is authored as a time-coded shot sequence picked from a menu of golden blueprints, so frames develop over their full duration instead of freezing after entrance.

1345AbhishekKumar/starter-nextjs

turn a GitHub pull request (a PR URL like github.com/<owner>/<repo>/pull/<N>, an <owner>/<repo>#<N> ref, or 'this PR' in a checked-out repo) into a code-change explainer video, up to ~3 min (sweet spot 30-90s) — changelog, feature reveal, fix, or refactor walkthrough, rendered from the diff / commits / files. The input is a CODE CHANGE read via the gh CLI; there is no website capture. Use this skill for a GitHub PR. Do not use it for a product launch/promo (use /product-launch-video), a tour of a real website (use /website-to-video), a topic explainer with no PR (use /faceless-explainer), captions on existing footage (use /embedded-captions), or a short unnarrated motion graphic (use /motion-graphics). If the intent is unclear, route through /hyperframes first.

1345AbhishekKumar/starter-nextjs

Port an existing Remotion (React) composition to HyperFrames HTML. Use ONLY when the user explicitly asks to port/convert/migrate/translate a Remotion source. Do NOT use: (a) authoring a new HyperFrames composition; (b) Remotion mentioned in passing; (c) Remotion code shared as reference only; (d) "same video as my Remotion one" without explicit migrate request — treat as fresh build. Doubt → `/general-video`. One-way, Remotion-only: no reverse export (HyperFrames→Remotion or any framework), no non-Remotion source (After Effects, Framer Motion, plain React/CSS) → out of scope, re-create via `/general-video`. Flags unsupported patterns (useState, useEffect, async calculateMetadata, third-party React libs, `@remotion/lambda`) and recommends runtime interop over lossy translation. Unsure whether to port vs. build fresh, or only a passing Remotion mention? → /hyperframes.

1345AbhishekKumar/starter-nextjs

Capture a general website/URL and turn it into a HyperFrames video (site tour, showcase, or social clip from the site's own visuals). Uses headless Chrome screenshots + brand assets. Use when intent is general — portfolio/blog/landing-page showcase or social clip from the site. NOT for: product/SaaS launch or promo (→ /product-launch-video, even from a URL); topic explainer with no site (→ /faceless-explainer); GitHub PR (→ /pr-to-video); adding captions to existing video (→ /embedded-captions); short unnarrated page-highlight motion graphic (→ /motion-graphics). Unclear launch-vs-general-site? Ask one question or start at /hyperframes.

相关技能