Communitygithub.com

AlessandroFare/fluxychat

将音乐曲目(音频文件、从中提取音频的视频,或根据情绪简报生成的曲目)转换为节拍同步的视频——包括歌词视频、幻灯片或动态宣传片。音乐驱动所有节奏;用户提供的任何图像/视频都会按照相同的节拍网格进行剪辑,且完整视频无需任何素材。

fluxychat 是什么?

该技能(music-to-video)是HyperFrames工作流的一部分,用于将单个音乐曲目转换为完全节拍同步的视频。它可以处理多种输入源:用户提供的音频文件、视频(从中提取音频),或通过/ media-use生成的基于情绪的音乐曲目。该工作流在`videos/<project>/`目录下运行,遵循七个严格顺序的步骤:设置(Step 0)、音频分析(Step 1)、故事板骨架(Step 2)、计划批准(Step 3)、帧构建(通过每个帧的单个子代理进行分发,Step 4)、组装(Step 5)和渲染批准(Step 6)。该技能在自主模式下运行,使用`analyze-beatgrid.py`作为其唯一的节拍分析器,确保所有切割和过渡都与检测到的节拍网格一致。它支持Claude Code和可能支持Cursor/Codex等环境,但强烈设计为无需用户干预即可运行——唯一的用户交互点是Step 3(计划批准)和Step 6(渲染批准),在自主模式下这些会被作为通知而非阻塞性障碍处理。该技能特别适用于快速、高能量的BGM曲目,其中明显的节拍网格驱动视频编辑;对于较平静的曲目,则通过乐句和能量水平进行节奏控制。完整视频可以实现无需任何外部素材,因为排版和模板提供了基础内容;任何用户提供的图像或视频资产都会按照相同的节拍网格进行剪辑以增强内容。歌词视频支持通过/ media-use进行音频转录以进行歌词同步,或使用用户提供的歌词文本在节拍上放置歌词。在技能执行之前,它会自动通过`npx hyperframes skills update music-to-video`检查更新,确保在运行工作流时使用最新版本。

兼容平台✓Claude Code~Codex CLI~Cursor
npx skills add https://github.com/AlessandroFare/fluxychat/tree/HEAD/.agents/skills/music-to-video

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

首先,保持这项技能更新 — 静默运行,无需提问: npx hyperframes skills update music-to-video。当一切为最新时,这是一个快速的无操作;否则,它会刷新此技能及其依赖的核心领域技能,然后才能依赖它们。

music-to-video — 一个基于音乐、节拍同步的视频工作流程

使用此技能将音乐曲目转换为节拍同步的 HyperFrames 视频。你分析一次曲目,布局帧,填写每帧计划,并将每帧构建为合成。输入是音乐曲目加上可选的用户图像或视频 — 没有旁白,也没有网站捕获。排版和模板是基础(完整视频无需任何素材);用户提供的任何媒体都会按照相同的节拍网格切入。

你是编排者。在 videos/<project>/ 中工作。按顺序执行步骤,并通过每个关卡后再进入下一步。有两个步骤需要用户参与:步骤 3(计划审批)和步骤 6(渲染审批)— 两者都是每 ../hyperframes-core/references/brief-contract.md(在步骤 0 之前阅读)的检查点关卡:在自主模式下,发布摘要作为提醒并继续,而不是等待。除了步骤 4(你为每帧分派一个子代理),其他步骤都自己完成。将设计和运动规则保留在此文件之外 — 它们位于 references/ 和 frame-worker 子代理中。

SKILL_DIR = 此技能目录。PROJECT_DIR = videos/<project-name>/。

工作流程:步骤 0 设置 → hyperframes.json + assets/bgm.mp3;步骤 1 分析 → audiomap.json;步骤 2 骨架 → STORYBOARD.md(帧,组为 TBD);步骤 3 计划 → 完整的 STORYBOARD.md + frame.md;步骤 4 构建 → compositions/frames/NN-*.html;步骤 5 组装 → index.html;步骤 6 渲染 → renders/video.mp4。

两个塑造一切的观念

  • 一个分析器,并且你信任它。 analyze-beatgrid.py 是唯一的节拍分析器 — 切勿使用其他工具或凭耳朵重新测量节拍。其能量/密度/滚奏/起音/静音始终可靠。其 bpm 和 beats_sec 仅当音乐真正有节奏时才可靠;对于平静音乐,网格是跟踪器施加的节拍器,因此根据乐句和能量来安排节奏,切勿强烈切割到网格。判断你处于哪种情况是每帧的 pacing(步骤 2)。
  • 一帧 = 一个文件;组位于内部。 步骤 2 将曲目切割为帧,每帧成为一个合成文件 compositions/frames/NN-<frame_id>.html,由一个 frame-worker 构建。一帧可以细分为组(每个组是一个模板或一个运动原语组合)。额外的密度进入_组内部_,因此帧数追踪不同的处理方式,而非节拍 — 快节奏曲目不会增加子代理的数量。

步骤 0:设置、BGM 和输入

目标:确定音乐源,创建 HyperFrames 项目,并记录任何用户提供的媒体。

简要从意图层开始。 开启规则,按顺序:(1) BRIEF.md 存在 → 读取它,并且不要询问它已回答的任何问题 — 其 flow/storyboard 派生模式(简要合同 § 1)。(2) 没有 BRIEF.md 但项目存在 → 从磁盘上的内容继续;切勿重新询问。(3) 一个直接到达此处的全新创建请求 → 读取 /hyperframes 并运行其意图层(references/intent-interview.md):它确认此路线的必须项(音乐源,目的地 → 宽高比 — ../hyperframes/references/routes/music-to-video.md)并宣布哪些内容被推迟 — 品牌和流派在步骤 3 设计时选择。初始化后立即写入 BRIEF.md(之前切勿 — init 拒绝非空目录)并记录基于偏好的答案(brief-format.md)。编辑请求跳过所有这些。

音乐是核心 — 在任何其他事情之前确立一首曲目。此技能针对快速、高能的 BGM 进行了优化:强大的节拍网格驱动切换(平静曲目也可以,但根据乐句而非节拍来安排节奏)。如果用户提供了音频 — 一个音乐文件或一个要提取音频的视频 — 则使用它。否则,从请求中选择情绪并通过 /media-use(references/bgm.md)生成一首曲目。在第一个经过身份验证的提供者操作之前,运行 npx hyperframes auth status 并逐字输出其结果。如果已退出,则应用一个分支:

  • 协作: 等待登录或明确选择继续使用本地提供者脱机。
  • 自主: 说明状态并通过可用的本地提供者继续。

如果没有离线提供者能够满足所需的音乐能力,则提出障碍。切勿将密钥写入每个仓库的 .env。所有权和离线回退位于 /media-use references/setup-providers.md § 提供者。生成的曲目位于 assets/bgm.mp3。将提供的图像或视频暂存,以便帧可以在节拍网格上使用它们;否则,排版承载视频。

歌词视频: 对于与声乐同步的歌词,通过 /media-use 转录音轨以获取单词/行的时间,或要求用户提供歌词文本,并将行放置在节拍网格上。

仅在 hyperframes.json 缺失时初始化。从简要中以 kebab-case 命名 <project>,例如 midnight-drive-loop — 切勿使用时间戳。init 检查已安装的技能与 GitHub 上的最新版本,并更新全局集(如果有任何技能过时)。

npx hyperframes init "videos/<project>" --non-interactive --example=blank --skill=music-to-video
mkdir -p "$PROJECT_DIR/assets" "$PROJECT_DIR/renders"
cp "<user-music>" "$PROJECT_DIR/assets/bgm.mp3"   # 如果需要,先从视频中提取
# 仅当用户提供了图像/视频时:
node <SKILL_DIR>/scripts/stage-assets.mjs --from <dir> --hyperframes "$PROJECT_DIR" --into public

品牌(字体+调色板)在步骤 3 选择,而不是这里。不要预先选择流派或曲目类型 — 资产只是一个可选成分,流派从每帧选择中涌现。

关卡: hyperframes.json + assets/bgm.mp3 存在;记录了宽高比/长度/帧率以及(如果有)资产清单。


步骤 1:分析音乐

目标:生成整个视频所依赖的唯一规范时序分析。

analyze-beatgrid.py 是唯一的节拍分析器 — 切勿使用其他工具或凭耳朵重新测量节拍。它读取一次曲目并写入 audiomap.json:能量相位(水平/密度/感觉)、起音 + onset_rate、滚奏、静音、hard_stops、key_moments、乐句、节奏/网格以及 audio.duration_sec。它是确定性的 — 同一文件始终给出相同的映射。大多数字段在任何音乐上都是可靠的;仅在音乐真正有节奏时 bpm 和 beats_sec 才可靠,判断这一点是你在步骤 2 做出的调用。

先决条件:需要可用的 Python 3,带有 librosa、numpy 和 soundfile。如果导入失败,请在运行分析器之前将它们安装到活动的 Python 环境中:

python3 -m pip install librosa numpy soundfile
python3 <SKILL_DIR>/scripts/analyze-beatgrid.py "$PROJECT_DIR/assets/bgm.mp3" \
  -o "$PROJECT_DIR/audiomap.json" --print

关卡: audiomap.json 存在;已知 audio.duration_sec。


步骤 2:帧骨架(仅结构)

目标:读取音乐并布局帧 — STORYBOARD.md 的骨架。

阅读 references/frame-skeleton.md。自己将 audiomap.json 转换为 STORYBOARD.md 的骨架 — 没有中间的 JSON。在真实的音乐变化处(hard_stops、SURGE / DROP key_moments、滚奏的边缘、没有起音的一段、大的能量跳跃)切割曲目为帧,将每个边界对齐到 audiomap 锚点。为每帧设置 span_sec、pacing(来自步骤 1 信任调用的判断 — 当网格真实时为 beat_cut,当它是施加于平静音乐上的节拍器时为 phrase_flow)、mood 和一行 feel(步骤 3 将模板与之匹配的普通音乐状况)。仅在此处分类和布局:将每帧的 ### Groups 保留为 TBD (Step 3),并将前置元数据 style 留空 — 没有模板、文案、颜色或字体。预期 ~1–6 帧。

关卡: 帧覆盖曲目(第一个在 0,最后一个在 duration_s);每个携带 span_sec + pacing + mood + feel;每个 ### Groups 是 TBD;任何地方都没有内容。


步骤 3:填写计划(用户关卡)

目标:将骨架转换为经过批准的完整 STORYBOARD.md。

阅读 references/planning.md、storyboard-format.md、template-catalog.md、motion-primitive-catalog.md 和 montage.md(仅当用户提供了资产时)。在原地编辑同一文件,执行两件事:

  1. 选择品牌。 从 ../hyperframes-creative/frame-presets/ 中选择一个预设,使用 ../hyperframes-creative/references/design-spec.md 中的表格(匹配曲目的情绪;仅其字体和颜色重要 — 模板拥有合成)。将其未修改地复制到 frame.md 中,并从中填充前置元数据 style(字体 + 一个 ≤4–6 色板调色板)。
  2. 填充每帧。 决定其组并为每个组赋予一种处理方式:从目录中匹配的模板(带有绑定参数和真实的 audiomap 锚点)、来自原语目录的自由组合或遵守 pacing 的资产处理。编写文案。你负责什么(模板/原语 + 内容 + 锚点);frame-worker 负责如何 — 切勿将毫秒级补间写入 storyboard。
node <SKILL_DIR>/scripts/validate-plan.mjs --storyboard "$PROJECT_DIR/STORYBOARD.md" \
  --audiomap "$PROJECT_DIR/audiomap.json" --templates <SKILL_DIR>/references/templates

修复每个 ✗(硬错误:持续时间不匹配、帧未覆盖曲目、缺少 src);警告是尽力而为。然后向用户显示逐帧摘要,并迭代直到他们批准。在自主模式下,这是一个检查点关卡:发布摘要作为提醒并继续(validate-plan.mjs 通过是质量关卡,仍然会阻止)。

关卡: frame.md 是逐字预设副本;validate-plan.mjs 退出码为 0;用户批准了计划(自主:摘要已作为提醒发布)。


步骤 4:从计划构建帧

目标:将每帧构建为自包含的合成文件。

创建 compositions/frames/。阅读 sub-agents/frame-worker.md 和 ../hyperframes-core/references/subagent-dispatch.md。分派每帧一个 frame-worker,尽可能并行(否则按波次)。每个 worker 精确获取一帧和此上下文:

PROJECT_DIR: <绝对路径>
frame_id: <NN-frame_id>              # = 帧文件名主干,例如 02-f2;合成 ID
你的块:PROJECT_DIR/STORYBOARD.md 中的 `## Frame N — <frame_id>` 块
audiomap: PROJECT_DIR/audiomap.json
frame.md: PROJECT_DIR/frame.md
材料:对于每个组,<SKILL_DIR>/references/templates/<id>/index.html(模板)和
           <SKILL_DIR>/references/motion-primitives/<id>/(自由);暂存的 assets/(资产组)
合同:../hyperframes-core/references/sub-compositions.md + determinism-rules.md
画布:<w>×<h>   节奏:<beat_cut|phrase_flow>
写入:PROJECT_DIR/compositions/frames/<frame_id>.html

Worker 分叉引用的材料,将每个锚点转换为帧本地秒数(local_t = track_t − span_sec[0]),以 0ms 切割门控其组,并写入一个查找安全的帧文件。Worker 从不运行 hyperframes CLI — 那些命令操作已组装的项目,该项目尚不存在,因此它们会报告错误的文件。Worker 仅写入合同并停止;你在组装后验证(步骤 6)。当每个 worker 返回时,你可以确认其文件已落在磁盘上。

关卡: 每帧都有其 compositions/frames/NN-*.html 在磁盘上。


步骤 5:组装

目标:将构建的帧 + BGM 连接到可播放的 index.html。

assemble-index.mjs 是确定性的 — 没有子代理,没有判断。它在累积的 data-start 引用每个帧文件,将 assets/bgm.mp3 挂载到轨道 11,并硬切割帧 → 帧(帧覆盖曲目,没有间隙,因此没有过渡注入器)。

node <SKILL_DIR>/scripts/assemble-index.mjs --storyboard "$PROJECT_DIR/STORYBOARD.md" \
  --hyperframes "$PROJECT_DIR" --audiomap "$PROJECT_DIR/audiomap.json"

修复它报告的任何 ✗ — 缺失或空白帧文件意味着该 worker 写了部分文件;重新分派它(步骤 4)并重新组装。

关卡: index.html 存在;总持续时间 == audiomap.audio.duration_sec。


步骤 6:验证并渲染

目标:验证组装的视频,获得用户批准,并渲染最终的 MP4。

在组装的项目上运行 CLI — 这是正确的单元(每帧 worker 无法运行它)。check 在一个传递中运行结构检查和无头浏览器运行时、布局、运动和对比度关卡;--snapshots 还发出审查帧。

( cd "$PROJECT_DIR" && npx hyperframes check . --snapshots )

在 t=0、每帧开始、最强的 DROP / SURGE、每个 hard_stops[].t 以及最后一帧检查。失败时,自己进行最便宜的安全修复:编辑有问题的 compositions/frames/NN-*.html。切勿更改持续时间或音频时序以隐藏同步问题。一旦关卡通过,暂停等待用户审查,然后仅在批准时渲染(自主模式:问一个保留的问题 — “先预览,还是渲染?” — 然后交付 MP4 和联系表):

( cd "$PROJECT_DIR" && npx hyperframes render . --skill=music-to-video -q draft -o renders/video.mp4 --fps 30 )

关卡: check 通过并且审查了快照;用户批准(自主:检查通过,交付包括联系表);renders/video.mp4 存在,带有音频,持续时间 == audiomap.audio.duration_sec。最终回复说明 MP4 路径和持续时间。


恢复表

| 你已有 | 从中继续… | | -- …

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

AlessandroFare/fluxychat

Add captions or subtitles to an existing single-subject talking-head video without editing the footage. Use for plain verbatim captions, cinematic captions embedded behind the subject, VFX captions, “炸/特效/酷炫字幕,” or a named identity from the 35-style catalog. Route by visual identity, not by backend engine. The quiet `anchor` rail is the default; embed every word only when the user explicitly wants a fully cinematic treatment. The workflow runs locally end to end, including transcription and subject matting; split multi-shot footage before applying it.\n

AlessandroFare/fluxychat

Use when audio already placed in a HyperFrames composition needs to be mixed: a music bed that fights a voiceover (voiceover carve), effects on a track (EQ, compressor, limiter, gate, saturation, delay, reverb, chorus, phaser, bitcrush), or automation envelopes drawn on a track's volume or any effect parameter. Don't use for sourcing or generating audio — finding BGM, SFX, or making a voiceover is `/media-use`. Don't use for clip timing or track layout, which is `/hyperframes-core`.\n

AlessandroFare/fluxychat

Use the HyperFrames CLI development loop: init, add, catalog, capture, lint, check, snapshot, compare, grade-compare, preview, play, present, beats, keyframes, single or batch render, publish, cloud, cloudrun, feedback, lambda, doctor, browser, info, upgrade, skills, compositions, docs, benchmark, telemetry, transcribe, auth, tts, and remove-background. Also use when diagnosing build or render failures. validate, inspect, and layout are deprecated aliases; use check. Covers local, HeyGen-hosted cloud, AWS Lambda, and Google Cloud Run rendering.\n

AlessandroFare/fluxychat

Install, discover, and wire registry blocks and components into HyperFrames compositions. Use when running hyperframes add or hyperframes catalog, installing one item or every block matching a tag, wiring an installed item into index.html, or working with hyperframes.json. Covers discovery, install locations, block sub-composition wiring, component snippet merging, and authoring a new block or component to contribute upstream (idea → scaffold → validate → PR).

AlessandroFare/fluxychat

Mandatory entry point: read this first for any request to make, create, edit, animate, or render a video, animation, or motion graphic, including a promo, explainer, captioned clip, title card, overlay, slideshow or interactive deck, Remotion port, or any HyperFrames HTML composition. Also use it to inspect, diagnose, validate, preview, publish, or batch-render an existing HyperFrames project. Inputs may be a website URL, GitHub PR, Figma design or URL, text or brief, existing footage, or music. It resumes project state, captures intent when applicable, selects and installs the owning workflow, and routes domain capabilities. HyperFrames is the default output framework unless the user explicitly chooses another framework for the deliverable or asks only to record a browser session.\n

AlessandroFare/fluxychat

Overlay doctrine for the embedded-captions workflow — the caption MODEL (drop / rail / embed) and the rule that captions are an OVERLAY composited on top of the film, never a reserved bottom band you shift content up to avoid. Load when adding captions/subtitles to a talking-head or launch video, when deciding whether a phrase should be dropped, ride the verbatim rail, or be promoted to a scarce embedded climax, when laying out a composition that will carry captions (do NOT reserve a keep-out band), or when centering a composition on the true frame center under captions. Quotes the rail+embed model from embedded-captions and constraint #13 (captions overlay, keep-out band retired) from the product-launch-video scene agent. Applies ON TOP of embedded-captions.

AlessandroFare/fluxychat

Turn a weekly changelog .md into a finished branded changelog video (square 1080, ~45-60s, Annie VO, animated brand background, mock-UI visualizations, lowkey captions). Use when the user provides a changelog/digest markdown and wants the weekly video, or says "changelog video". Self-contained — fonts, background, lexicon, and scripts ship in this skill.

AlessandroFare/fluxychat

Audit and fix animation performance issues including layout thrashing, compositor properties, scroll-linked motion, and blur effects. Use when animations stutter, transitions jank, or reviewing CSS/JS animation performance.

AlessandroFare/fluxychat

Author or edit a custom HyperFrames composition when no specialized workflow fits, or when BRIEF.md sets flow: companion. Use for longer or multi-scene pieces, brand and sizzle reels, montages, static loops, static title cards, footage remixes, and freeform builds. Use motion-graphics instead for a short unnarrated motion-first unit, including an animated title. Route fresh creation through hyperframes before using this skill.

AlessandroFare/fluxychat

All animation knowledge for HyperFrames — atomic motion rules, multi-phase scene blueprints, scene transitions, broader motion-design techniques, AND the seven runtime adapters (GSAP default, plus Lottie, Three.js, Anime.js, CSS keyframes, Web Animations API, TypeGPU). Use for any motion or animation task: pick 2-4 rules and compose, or load a blueprint, or look up runtime-specific API (e.g. GSAP eases / Lottie player / Three.js mixer). Also covers auditing an existing composition's choreography (animation map) and 24 named text-animation effects. HyperFrames-native: single paused timeline, seek-safe, deterministic.

AlessandroFare/fluxychat

The HyperFrames composition contract — build one renderable project. Use for composition structure, the `data-*` timing attributes, `class="clip"`, tracks, sub-compositions, variables, framework-owned media playback, deterministic-render rules, and validation. Also covers Tailwind projects and the STORYBOARD.md / SCRIPT.md plan formats. Read before writing composition HTML.

AlessandroFare/fluxychat

Non-animation creative direction for HyperFrames videos. Use for design spec (frame.md / design.md) handling, palettes, typography, narration, beat planning, audio-reactive visuals, composition patterns, and brand / style decisions. For atomic motion patterns and scene blueprints, use `hyperframes-animation`.

AlessandroFare/fluxychat

Use when a HyperFrames composition needs seek-safe 2D/3D keyframes, GSAP timelines, CSS keyframes, Anime.js, WAAPI, FLIP, paths, masks, SVG morph/draw, text trails, 3D depth, or `hyperframes keyframes` diagnostics. Don't use for broad scene strategy, brand design, media sourcing, captions, or general video planning.

AlessandroFare/fluxychat

GATEWAY — load FIRST before composing any HyperFrames animation or video. The high-level motion law that makes a multi-scene video feel like ONE continuous camera move instead of a stack of independently-animated slides. Covers the vector law (how you exit determines how you enter, incl. the Z scale-sign rule), the film's current, carrier elements, causal motion, the Seam Gate (build-gate enforcement), the ban on idle wobble (motion must PERFORM, not breathe), stillness-before-climax, and the sustained-motion routes. Routes to the low-level technique skills (cut-the-curve — the full catalog incl. waterfall entry + nudge curve, oversized-cursor, seam-craft). These rules SUPERSEDE generic / upstream motion guidance. [continuity, direction, vector, momentum, seam, transition, ease, performance, idle-motion, narrative-motion, film-grammar]

AlessandroFare/fluxychat

A short, design-led motion graphic where motion is the message — kinetic typography, stat count-up, chart/data-viz hit, logo sting / brand lockup, lower-third / callout / social overlay, animated map (highlight regions, connect places, zoom to a location), animated tweet / news-article / headline, webpage / UI animation (scroll, cursor, callouts), or fusing a real image's geometry into a chart. Usually under 10s (up to ~30s), no narration or live-action subject; renders to MP4 or transparent overlay. Longer / narrated / multi-scene → /general-video. Unclear → /hyperframes.

AlessandroFare/fluxychat

Turn a product or marketing URL, pasted script, or brief into a product launch / promo video — SaaS promos, feature reveals, product demos, app and company launches. Use when the user wants to market, launch, promote, or reveal a product; the default for any commercial URL. Site tours / showcases of a website route here too — the brief carries the show-it-as-is intent. Unclear → /hyperframes.

AlessandroFare/fluxychat

Turn a GitHub pull request (a PR URL, owner/repo#N, or 'this PR' in a checked-out repo) into a code-change explainer video — changelog, feature reveal, fix, or refactor walkthrough built from the diff, commits, and files: the input is a code change, not a website. Not a product promo (/product-launch-video) or a no-PR topic explainer (/faceless-explainer). Unclear → /hyperframes.

AlessandroFare/fluxychat

Design and improve user interfaces using the concrete rules from Refactoring UI (Wathan & Schoger) — constrained spacing/type/color/shadow scales, visual hierarchy through weight and color rather than size, and depth through emulated light. Use when building or styling any UI (web, app, dashboard, landing page, component), when picking font sizes, spacing, colors, shadows or border radius, when designing a color palette or design tokens, and whenever someone says a UI "looks off", "looks amateur", "feels cluttered/plain/unfinished", or asks to "make this look better".

AlessandroFare/fluxychat

Port an existing Remotion (React) composition''s source to HyperFrames HTML. Use ONLY on an explicit ask to port/convert/migrate/translate a Remotion source — one-way, Remotion-only. A passing Remotion mention, reference-only code, or "make something like my Remotion video" is a fresh build (/general-video). Unclear → /hyperframes.

AlessandroFare/fluxychat

Use before UI-related work to select the smallest useful UI Skills context through the ui-skills CLI.

相关技能