Communitygithub.com

samyakiasonline-ctrl/hyperframe

将一首音乐曲目(音频文件、从中提取音频的视频,或根据情绪简报生成的曲目)转换为节拍同步的视频——歌词视频、幻灯片或动态宣传片。音乐驱动所有节奏;用户提供的任何图片/视频都会被剪辑到相同的节拍网格上,并且一个完整的视频不需要任何素材。对于带有旁白的片段,请使用匹配输入的工作流程(参见 /hyperframes)。如果不确定,请参考 /hyperframes。

hyperframe 是什么?

此技能将音乐曲目转换为节拍同步视频,专注于歌词视频、幻灯片和动态宣传片。它支持多种输入:音频文件、视频(提取音频)或根据情绪简报生成的曲目。核心工作流程涉及分析音乐节拍、创建故事板、构建每个帧作为独立的HTML文件,最后组装成可播放的HTML并渲染为MP4视频。整个过程由音乐驱动,用户提供的任何图片或视频都会被自动剪辑到相同的节拍网格上,使得即使没有用户素材也能生成完整的视频。此技能针对Claude Code、Cursor和Codex等AI代理平台进行了优化,完全自动化的步骤和用户审批检查点确保了高质量输出。

兼容平台✓Claude Code~Codex CLI~Cursor
npx skills add https://github.com/samyakiasonline-ctrl/hyperframe/tree/HEAD/.claude/skills/music-to-video

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

插件安装: 在运行设置或更新命令之前,请遵循插件执行规则(当此技能位于HyperFrames插件内部时)。独立安装保留以下更新说明。

首先,保持此技能最新——请在运行前与用户确认: npx hyperframes skills update music-to-video。如果所有内容都是最新的,则快速执行无操作;否则,在依赖它们之前刷新此技能及其依赖的核心领域技能。

music-to-video — 一个基于音乐、节拍同步的视频工作流

使用此技能将音乐曲目转换为节拍同步的HyperFrames视频。您将分析曲目一次,布局帧,填充每帧计划,并将每帧构建为合成。输入是音乐曲目加上可选的用户图片或视频——没有旁白,也没有网站捕获。排版和模板是基础(一个完整的视频不需要任何素材);用户提供的任何媒体都会被剪辑到相同的节拍网格上。

您是编排者。在 videos/<project>/ 中工作。按顺序执行步骤,并在继续之前通过每个关卡。两个步骤需要用户参与:步骤3(计划审批)和步骤6(渲染审批)——这两个是检查点关卡,根据 ../hyperframes/references/brief-contract.md(在步骤0之前阅读它):在自主模式下,将步骤3摘要作为通知发布并继续;步骤6渲染审批仍然需要询问,作为保留的问题。除了步骤4,您自己执行每个步骤,其中您需要为每个帧分派一个子代理。将设计和运动规则保留在此文件之外——它们存在于 references/ 和 frame-worker 子代理中。

SKILL_DIR = 此技能目录。PROJECT_DIR = videos/<project-name>/。

工作流:步骤0 设置 → hyperframes.json + assets/bgm.mp3;步骤1 分析 → audiomap.json;步骤2 骨架 → STORYBOARD.md(帧,组 TBD);步骤3 计划 → 完整的 STORYBOARD.md + frame.md;步骤4 构建 → compositions/frames/NN-*.html;步骤5 组装 → index.html;步骤6 渲染 → renders/video.mp4。

两个塑造一切的理念

  • 一个分析器,并且您信任它。 analyze-beatgrid.py 是唯一的节拍分析器——切勿使用其他工具或凭耳朵重新测量节拍。其能量/密度/滚动/起始/静音始终可靠。其 bpm 和 beats_sec 仅在音乐真正有节律性时才可靠;在平静的音乐上,网格是节拍追踪器强加的节拍器,因此应根据乐句和能量来安排节奏,切勿硬切。判断您处于哪种情况是每个帧的 pacing(步骤2)。
  • 一个帧 = 一个文件;组位于内部。 步骤2将曲目切割为帧,每个帧成为一个合成文件 compositions/frames/NN-<frame_id>.html,由一个 frame-worker 构建。一个帧可以细分为组(每个组是一个模板或运动原语组合)。额外的密度放在组_内部_,因此帧数量跟踪不同的处理方式,而不是节拍——快节奏的曲目不会增加子代理的数量。

步骤0:设置、BGM和输入

目标:确定音乐源,创建HyperFrames项目,并记录任何用户提供的媒体。

简报从意图层开始。 打开规则,按顺序:(1) BRIEF.md 存在 → 读取它,不询问它已回答的任何问题——其 flow/storyboard 推导出模式(简报合同 § 1)。(2) 没有 BRIEF.md 但项目存在 → 从磁盘内容恢复;绝不重新询问。(3) 直接到达这里的全新创建请求 → 读取 /hyperframes 并运行其意图层(references/intent-interview.md):它确认此路由的必要条件(音乐源,目标 → 宽高比 — ../hyperframes/references/routes/music-to-video.md),并宣布哪些内容延迟处理——品牌和流派按设计在步骤3选择。在 init 之后立即写入 BRIEF.md(绝不在此之前写入——init 拒绝非空目录),并记录基于偏好的答案(brief-format.md)。编辑请求跳过所有这些。

音乐是主干——在任何其他事情之前确定一首曲目。此技能针对快速、高能量的BGM进行了优化:强节拍网格驱动剪辑(平静的曲目也可以,但按乐句而非节拍来安排节奏)。如果用户提供了音频——音频文件或要从中提取音频的视频——请使用它。否则,从请求中选择情绪并通过 /media-use(audio/references/bgm.md)生成曲目。在第一个经过身份验证的提供商操作之前,运行 npx hyperframes auth status 并按原样输出其输出。如果已登出,应用一个分支:

  • 协作模式: 等待登录或明确选择继续离线使用本地提供商。
  • 自主模式: 声明状态并通过可用的本地提供商继续。

如果没有离线提供商可以满足所需的音乐能力,请显示阻塞。切勿将密钥写入每个仓库的 .env 文件中。认证所有权和离线回退位于 /media-use references/setup-providers.md § 提供商中。生成的曲目将位于 assets/bgm.mp3。暂存提供的图片或视频,以便帧可以在节拍网格上使用它们;否则,排版将承载视频。

歌词视频: 对于与歌词同步的歌词,通过 /media-use 转录音轨来获取单词/行的时间,或者向用户请求歌词文本并将行放置在节拍网格上。

仅在 hyperframes.json 缺失时初始化。从简报中命名 <project>,使用kebab-case,例如 midnight-drive-loop——绝不要使用时间戳。init 检查已安装的技能与GitHub上的最新版本,如果任何技能过时则更新全局集合。

npx hyperframes init "videos/<project>" --non-interactive --example=blank --skill=music-to-video
mkdir -p "$PROJECT_DIR/assets" "$PROJECT_DIR/renders"
cp "<user-music>" "$PROJECT_DIR/assets/bgm.mp3"   # 如果需要,先从视频中提取
# 仅当用户提供了图片/视频时:
node <SKILL_DIR>/scripts/stage-assets.mjs --from <dir> --hyperframes "$PROJECT_DIR" --into public

品牌(字体+调色板)在步骤3选择,而不是在这里。不要预先选择流派或曲目类型——素材只是一个可选成分,流派从每帧选择中浮现。

关卡: hyperframes.json + assets/bgm.mp3 存在;宽高比/长度/fps和(如果有的话)素材清单已记录。


步骤1:分析音乐

目标:生成整个视频所依据的一个规范时间分析。

analyze-beatgrid.py 是唯一的节拍分析器——切勿使用其他工具或凭耳朵重新测量节拍。它读取曲目一次并写入 audiomap.json:能量阶段(级别/密度/感觉)、起始点 + onset_rate、滚动、静音、hard_stops、key_moments、乐句、节奏/网格和 audio.duration_sec。它是确定性的——相同的文件总是产生相同的映射。大多数字段在任何音乐上都可靠;bpm 和 beats_sec 仅在音乐真正有节律性时才可靠,判断这一点是您在步骤2中做出的决定。

先决条件:Python 3 并安装了 librosa、numpy 和 soundfile。如果导入失败,请在运行分析器之前将它们安装到活动的Python环境中:

python3 -m pip install librosa numpy soundfile
python3 <SKILL_DIR>/scripts/analyze-beatgrid.py "$PROJECT_DIR/assets/bgm.mp3" \
  -o "$PROJECT_DIR/audiomap.json" --print

关卡: audiomap.json 存在;audio.duration_sec 已知。


步骤2:帧骨架(仅结构)

目标:读取音乐并布局帧——STORYBOARD.md 的骨架。

阅读 references/frame-skeleton.md。自行将 audiomap.json 转换为 STORYBOARD.md 的骨架——没有中间JSON。在真实的音乐变化处(hard_stops、SURGE / DROP key_moments、滚动的边缘、没有起始点的片段、大的能量跳跃)将曲目切割为帧,将每个边界对齐到 audiomap 锚点。为每个帧设置 span_sec、pacing(来自步骤1信任调用的裁决——当网格为实时为 beat_cut,当它是强加于平静音乐的节拍器时为 phrase_flow)、mood 和一行 feel(步骤3会将模板与之匹配的简单音乐情况)。仅在此处分类和布局:将每个帧的 ### Groups 保留为 TBD (Step 3),并将前言的 style 留空——没有模板、文案、颜色或字体。预计大约1–6个帧。

关卡: 帧平铺了整个曲目(第一个在0处,最后一个在 duration_s 处);每个帧携带 span_sec + pacing + mood + feel;每个 ### Groups 都是 TBD;没有任何内容。


步骤3:填写计划(用户关卡)

目标:将骨架转换为已批准的完整 STORYBOARD.md。

阅读 references/planning.md、storyboard-format.md、template-catalog.md、motion-primitive-catalog.md 和 montage.md(仅当用户提供了素材时)。在原地编辑同一个文件,做两件事:

  1. 选择品牌。 从 ../hyperframes-creative/frame-presets/ 中选择一个预设,使用 ../hyperframes-creative/references/design-spec.md 中的表格(匹配曲目的情绪;仅其字体和颜色重要——模板拥有合成)。将其未经修改复制到 frame.md 中,并从中填充前言的 style(字体 + ≤4–6 色板调色板)。
  2. 填充每个帧。 决定其组并为每个组分配一个处理方式:来自目录的匹配模板(带有绑定参数和真实的audiomap锚点)、来自原语目录的自由组合、或遵循 pacing 的素材处理。在您自由组合一个命名外观之前,搜索实时目录:对于用户要求的每个外观、效果、处理或过渡——"CRT扫描线"、"故障"、"胶片颗粒"、"闪光扫掠"——运行 npx hyperframes catalog --query "<外观,用纯英文>" --json 并读取顶部结果。template-catalog.md 和 motion-primitive-catalog.md 仅列出此技能自己的本地材料;搜索排名整个托管注册表(约400个块和组件)并且不需要安装任何东西——没有项目、没有先前的 add、没有账户。仅在对某个外观的搜索没有返回任何合适的结果时,再自由组合一个外观。编写文案。您拥有WHAT(模板/原语 + 内容 + 锚点);frame-worker 拥有HOW——切勿将毫秒级的补间写入故事板。
node <SKILL_DIR>/scripts/validate-plan.mjs --storyboard "$PROJECT_DIR/STORYBOARD.md" \
  --audiomap "$PROJECT_DIR/audiomap.json" --templates <SKILL_DIR>/references/templates

修复每个 ✗(硬错误:持续时间不匹配、帧未平铺曲目、缺少 src);警告是尽力而为。然后在聊天中作为提案呈现逐帧摘要(../hyperframes/references/review-loop.md § 1),并根据用户的回复进行迭代,直到他们批准;对于 storyboard: yes,还要将其编写为 storyboard.html(../hyperframes-creative/references/storyboard-recipe.md § 3),以便他们可以打开。在自主模式下,这是一个检查点关卡:将摘要作为通知发布并继续(validate-plan.mjs 通过是一个质量关卡,并且仍然会阻塞)。

关卡: frame.md 是原始预设副本;validate-plan.mjs 退出代码为0;用户批准了计划(自主模式:摘要已作为通知发布)。


步骤4:根据计划构建帧

目标:将每个帧构建为一个独立的合成文件。

创建 compositions/frames/。阅读 sub-agents/frame-worker.md 和 ../hyperframes/references/subagent-dispatch.md。为每个帧分派一个 frame-worker,尽可能并行(否则按波次)。每个worker仅需处理一个帧以及此上下文:

PROJECT_DIR: <绝对路径>
frame_id: <NN-frame_id>              # = 帧文件主干,例如 02-f2;合成ID
您的块:PROJECT_DIR/STORYBOARD.md 中的 `## Frame N — <frame_id>` 块
audiomap: PROJECT_DIR/audiomap.json
frame.md: PROJECT_DIR/frame.md
材料:对于每个组,<SKILL_DIR>/references/templates/<id>/index.html(模板)和
           <SKILL_DIR>/references/motion-primitives/<id>/(自由);已暂存的 assets/(素材组)
合同:../hyperframes-core/references/sub-compositions.md + determinism-rules.md
画布:<宽>×<高>   节奏:<beat_cut|phrase_flow>
写入位置:PROJECT_DIR/compositions/frames/<frame_id>.html

worker 分叉引用的材料,将每个锚点转换为帧本地秒数(local_t = track_t − span_sec[0]),用0毫秒剪辑来封闭其组,并写入一个支持搜索的帧文件。worker 绝不运行 hyperframes CLI——这些命令操作已组装的成品,而它尚不存在,因此它们会报告错误的文件。worker 仅根据合同写入并停止;您在组装后验证(步骤6)。随着每个worker返回,您可以确认其文件已落地到磁盘。

关卡: 每个帧都有其 compositions/frames/NN-*.html 在磁盘上。


步骤5:组装

目标:将构建的帧 + BGM 连接到可播放的 index.html。

assemble-index.mjs 是确定性的——没有子代理,没有判断。它在其累积的 data-start 处引用每个帧文件,将 assets/bgm.mp3 挂载到轨道11上,并硬切帧 → 帧(帧平铺整个曲目,没有间隙,因此没有过渡注入器)。

node <SKILL_DIR>/scripts/assemble-index.mjs --storyboard "$PROJECT_DIR/STORYBOARD.md" \
  --hyperframes "$PROJECT_DIR" --audiomap "$PROJECT_DIR/audiomap.json"

修复其报告的任何 ✗——缺少或空白的帧文件意味着该worker写入了部分文件;重新分派它(步骤4)并重新组装。

关卡: index.html 存在;总持续时间 == audiomap.audio.duration_sec。


步骤6:验证和渲染

目标:验证已组装的视频,获得用户批准,并渲染最终的MP4。

在已组装的成品上运行CLI——这是正确的单元(每帧worker无法运行它)。check 运行一些...

…(截断以供翻译)

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

samyakiasonline-ctrl/hyperframe

Add captions or subtitles to an existing single-subject talking-head video without editing the footage. Use for plain verbatim captions, cinematic captions embedded behind the subject, VFX captions, “炸/特效/酷炫字幕,” or a named identity from the 35-style catalog. Route by visual identity, not by backend engine. The quiet `anchor` rail is the default; embed every word only when the user explicitly wants a fully cinematic treatment. The workflow runs locally end to end, including transcription and subject matting; split multi-shot footage before applying it.

samyakiasonline-ctrl/hyperframe

Turn arbitrary text — an article, notes, a topic, a brief — into a faceless explainer video: there is no site or footage to capture, so the visuals are invented per scene (typography, abstract graphics, diagrams, data-viz). Use for topic explainers, concept breakdowns, how-tos, listicles. Not a video built from a website (/product-launch-video — promo or tour). Unclear → /hyperframes.

samyakiasonline-ctrl/hyperframe

Import Figma content into a HyperFrames composition — rendered assets, brand tokens, components, storyboard sections → reconstructed motion (frames read as states, not slides) (REST/CLI), connector-assisted motion when available, and shaders from a connector or native export. Use when the user pastes a figma.com link or asks to bring a Figma design, frame, logo, brand, or animation into a video/composition.

samyakiasonline-ctrl/hyperframe

Author or edit a custom HyperFrames composition when no specialized workflow fits, or when BRIEF.md sets flow: companion. Use for longer or multi-scene pieces, brand and sizzle reels, montages, static loops, static title cards, footage remixes, and freeform builds. Use motion-graphics instead for a short unnarrated motion-first unit, including an animated title. Route fresh creation through hyperframes before using this skill.

samyakiasonline-ctrl/hyperframe

All animation knowledge for HyperFrames — atomic motion rules, multi-phase scene blueprints, scene transitions, broader motion-design techniques, AND the seven runtime adapters (GSAP default, plus Lottie, Three.js, Anime.js, CSS keyframes, Web Animations API, TypeGPU). Use for any motion or animation task: pick 2-4 rules and compose, or load a blueprint, or look up runtime-specific API (e.g. GSAP eases / Lottie player / Three.js mixer). Also covers auditing an existing composition's choreography (animation map) and 24 named text-animation effects. HyperFrames-native: single paused timeline, seek-safe, deterministic.

samyakiasonline-ctrl/hyperframe

Use when audio already placed in a HyperFrames composition needs to be mixed: fade-in/fade-out, crossfade, track gain or volume, volume automation, ducking, a music bed that fights a voiceover (voiceover carve), effects on a track (EQ, compressor, limiter, gate, saturation, delay, reverb, chorus, phaser, bitcrush), automation envelopes drawn on a track's volume or any effect parameter, or one submix bus carrying a chain, a fader and an automation clock for several tracks at once (`<hf-audio-group>`). Don't use for sourcing or generating audio — finding BGM, SFX, or making a voiceover is `/media-use`. Don't use for clip timing or track layout, which is `/hyperframes-core`.

samyakiasonline-ctrl/hyperframe

Use the HyperFrames CLI development loop: init, add, catalog, capture, lint, check, snapshot, compare, grade-compare, preview, play, present, beats, keyframes, single or batch render, publish, cloud, cloudrun, feedback, lambda, doctor, browser, info, upgrade, skills, compositions, timeline, history, clean, docs, benchmark, telemetry, transcribe, auth, tts, and remove-background. Also use when diagnosing build or render failures. validate, inspect, and layout are deprecated aliases; use check. Covers local, HeyGen-hosted cloud, AWS Lambda, and Google Cloud Run rendering.

samyakiasonline-ctrl/hyperframe

The HyperFrames composition contract — build one renderable project. Use for composition structure, the `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, deterministic-render rules, and validation. Read before writing composition HTML.

samyakiasonline-ctrl/hyperframe

Non-animation creative direction for HyperFrames videos. Use for design spec (frame.md / design.md) handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns, and brand / style decisions. For atomic motion patterns and scene blueprints, use `hyperframes-animation`.

samyakiasonline-ctrl/hyperframe

Use when a HyperFrames composition needs a punch-in, punch-out, zoom, reframe, Ken Burns treatment, camera move, visual match/whip handoff, or other seek-safe 2D/3D keyframes; also for GSAP, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, text trails, 3D depth, or `hyperframes keyframes` diagnostics. Don't use for broad scene strategy, brand design, media sourcing, captions, or general video planning.

samyakiasonline-ctrl/hyperframe

Search, install, and wire registry blocks and components into HyperFrames compositions. Use BEFORE hand-building any named visual — whenever a brief, a user, or a storyboard names a look, effect, treatment, or transition such as CRT scanlines, glitch, chromatic aberration, film grain, a shimmer sweep, a chart, a code or terminal window, a map, or a confetti burst — because roughly 400 hosted items already cover many of them and the search ranks all of them with nothing installed, no project, and no account. Also use when running hyperframes add or hyperframes catalog, installing one item or every block matching a tag, wiring an installed item into index.html, or working with hyperframes.json. Covers discovery, install locations, block sub-composition wiring, component snippet merging, and authoring a new block or component to contribute upstream (idea → scaffold → validate → PR).

samyakiasonline-ctrl/hyperframe

Mandatory entry point: read this first for any request to make, create, edit, animate, or render a video, animation, or motion graphic, including a promo, explainer, captioned clip, title card, overlay, slideshow or interactive deck, Remotion port, or any HyperFrames HTML composition. Also use it to inspect, diagnose, validate, preview, publish, or batch-render an existing HyperFrames project. Inputs may be a website URL, GitHub PR, Figma design or URL, text or brief, existing footage, or music. It resumes project state, captures intent when applicable, selects and installs the owning workflow, and routes domain capabilities. HyperFrames is the default output framework unless the user explicitly chooses another framework for the deliverable or asks only to record a browser session.

samyakiasonline-ctrl/hyperframe

Use when working with a person on a HyperFrames project in Studio: first, whether their message asks for a change at all (questions, loose ideas and "don't change anything" get an answer and a plan, not an edit); for a new film, the plan, storyboard and build order that the HyperFrames launch films follow; and how the timeline should be laid out so it reads well (one caption track, one element kind per track, every scene a sub-composition) and where captions and key content may sit (safe zones). Don't use for how to perform an individual edit (split, trim, retime, volume, copy, swap): that is `creator-editing-recipes.md` in `/hyperframes-core`.

samyakiasonline-ctrl/hyperframe

Agent Media OS, the single skill for every media need in a HyperFrames project. Resolve BGM, SFX, image, icon, brand logo, voice, color grade, or LUT into a frozen local file or paste-ready block + ledger record (one verb, `resolve`); generate via TTS / music / image models when the catalog misses; produce voiceover, transcription, captions, and background removal through one shared audio engine; operate on media (cut / reframe / transform); and reuse assets across projects. Also use for vague feedback that real footage looks dark, flat, boring, should feel retro/camcorder/print/ASCII, needs privacy, or needs a media reveal.

samyakiasonline-ctrl/hyperframe

A short, design-led motion graphic where motion is the message — kinetic typography, stat count-up, chart/data-viz hit, logo sting / brand lockup, lower-third / callout / social overlay, animated map (highlight regions, connect places, zoom to a location), animated tweet / news-article / headline, webpage / UI animation (scroll, cursor, callouts), or fusing a real image's geometry into a chart. Usually under 10s (up to ~30s), no narration or live-action subject; renders to MP4 or transparent overlay. Longer / narrated / multi-scene → /general-video. Unclear → /hyperframes.

samyakiasonline-ctrl/hyperframe

Turn a product or marketing URL, pasted script, or brief into a product launch / promo video — SaaS promos, feature reveals, product demos, app and company launches. Use when the user wants to market, launch, promote, or reveal a product; the default for any commercial URL. Site tours / showcases of a website route here too — the brief carries the show-it-as-is intent. Unclear → /hyperframes.

samyakiasonline-ctrl/hyperframe

Turn a GitHub pull request (a PR URL, owner/repo#N, or 'this PR' in a checked-out repo) into a code-change explainer video — changelog, feature reveal, fix, or refactor walkthrough built from the diff, commits, and files: the input is a code change, not a website. Not a product promo (/product-launch-video) or a no-PR topic explainer (/faceless-explainer). Unclear → /hyperframes.

samyakiasonline-ctrl/hyperframe

Port an existing Remotion (React) composition''s source to HyperFrames HTML. Use ONLY on an explicit ask to port/convert/migrate/translate a Remotion source — one-way, Remotion-only. A passing Remotion mention, reference-only code, or "make something like my Remotion video" is a fresh build (/general-video). Unclear → /hyperframes.

samyakiasonline-ctrl/hyperframe

Author a HyperFrames slideshow — a presentation, pitch deck, or interactive deck with discrete slides, fragment reveals, branching, hotspot navigation, and built-in presenter mode with speaker notes; also converts an existing page into a deck. Output is a navigable deck, not a rendered MP4. If the user didn't explicitly ask for a slideshow, confirm before authoring. Unclear → /hyperframes.

samyakiasonline-ctrl/hyperframe

Package an existing talking-head / interview / podcast video with timed, designed GRAPHIC OVERLAY cards — kinetic titles, lower-thirds, data callouts, quotes, side panels, picture-in-picture — synced to the transcript, on a 16:9 / 9:16 / 4:5 canvas of your choice; the clip plays untouched underneath. Trigger on "graphic overlays", "on-screen graphics", "package / dress up my video". Not plain subtitles (/embedded-captions). Unclear → /hyperframes.

相关技能