Communitygithub.com

视频转精灵 V2 (仅限 Grok Build)

利用 Grok Build 的原生视频工具,将 2D 静态角色图像转换为密集动画精灵序列。

视频转精灵 V2 (仅限 Grok Build) 是什么?

本技能(Video2dsprite)专注于将 2D 静态角色图像通过 Grok Build 的图像到视频工具(imagine_image_to_video)生成原地循环动画,并利用 ffmpeg、Pillow 和 numpy 脚本进行抽帧、色度抠图(品红色背景)和归一化采样,最终输出密集精灵条、网格或循环 GIF。它适用于需要更平滑奔跑/行走循环或密集中间姿态的场景,但其局限性包括像素软化、角色身份漂移和色度残留,因此更适合实验性动作密集化而非生产级像素贴片。该技能仅支持 Grok Build 运行时,在其他 Agent 环境中会拒绝执行并推荐替代技能 `generate2dsprite`。工作流程包括:规划输出目录、生成带品红色背景的基础静态图、调用视频工具添加运动约束、本地脚本处理视频帧、质量检查以及最后报告输出路径。

兼容平台✓Claude Code✓Codex CLI~Cursor
npx skills add https://github.com/evalix098/tiktoklive/tree/HEAD/.grok/skills/video2dsprite

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

视频转精灵 V2 (仅限 Grok Build) 是做什么的?

利用 Grok Build 的原生视频工具,将 基础 2D 角色图像 转换为 密集动画精灵。

App-builder / Grok 环境

项目值
技能目录.grok/skills/video2dsprite/
脚本python3 .grok/skills/video2dsprite/scripts/video2dsprite.py …
视频工具imagine_image_to_video — 对单个 file_path 进行动画处理(确认文件存在)
检查用 read_file 检查静态帧/帧文件;报告视频路径
依赖ffmpeg + Pillow + numpy(已在 app-builder 镜像中预装)
输出目录assets/sprites/video2dsprite/<name>/ ,位于 /workspace 下
默认对于生产级核心角色,优先使用 generate2dsprite;此技能是更密集动作的实现路径
基础静态图 → imagine_image_to_video (原地运动) → 提取帧 → 色度抠图 → 采样/归一化 → 精灵条 / 网格 / GIF

平台限制(先阅读)

运行时是否支持?
Grok Build (xAI)是 — 需要图像生成器 + 图像转视频工具
Codex / Claude / 其他 Agent否 — 它们缺少 Grok 视频工具。告知用户此技能仅适用于 Grok Build,并推荐 $generate2dsprite 替代方案

基于能力进行限制,而非确切的工具名称。 图像生成可能以 imagine_text_to_image / imagine_image_to_image 形式出现;视频生成以 imagine_image_to_video 或(旧版)generate_video 形式出现。请使用你工具列表中可用的配对。仅当完全没有图像转视频工具时才停止并解释。不要使用代码绘制的帧来伪造运动。

此技能是一个可选的密集动作路径。它不能替代 $generate2dsprite:

何时使用 $generate2dsprite…何时使用 $video2dsprite…
需要清晰的像素贴片、固定网格、身份关键的核心角色用户需要更密集的中间姿态 / 更平滑的循环效果
攻击/施法身体贴片、道具包、引擎图集尝试基于视频的奔跑/行走/待机动作
大多数游戏精灵的生产默认选择用户明确要求视频 → 帧 → 精灵

视频会软化像素、漂移角色身份,并产生色度残留。务必进行质量检查。对于生产级核心角色,除非用户明确想要视频效果,否则优先使用 $generate2dsprite。

参数

根据用户请求推断:

  • subject:角色/生物描述,或现有静态图的路径
  • action:run | walk | idle | attack | 自定义动作短语
  • view:通常为 side(横版卷轴)。topdown 较困难 — 需警告并保持相机锁定
  • duration:6(默认)或 10 秒
  • frame_counts:需要导出的密集帧集,默认为 8,16,24,48
  • cell_size:输出精灵单元格大小,默认为 128
  • anchor:feet(默认,用于侧向移动) | center
  • bg:纯色 #FF00FF(色度抠图必需)
  • name:输出标识名
  • out_dir:工作文件夹(默认为 ./sprites/video2dsprite/<name>/ 或项目相对路径)

Agent 规则

  1. 仅限 Grok。 在非 Grok 运行时上拒绝执行,并附带简短说明 + 推荐 $generate2dsprite 替代方案。
  2. 静态图 → 视频,不能单独使用文本转视频。 先通过 imagine_text_to_image(根据提示或参考 file_path)生成一张清晰的静态图作为第一帧。然后使用该静态图的 file_path 调用 imagine_image_to_video。
  3. 原地运动。 提示要求角色朝向固定方向原地奔跑/行走。无相机平移、无背景滚动、无场景变化。角色大致保持在画面中心。
  4. 纯色品红色背景 应用于基础图并保留在视频提示中(#FF00FF / 纯品红色)。这是泛洪填充色度抠图的必需条件。
  5. 不要用 PIL/Canvas 捏造美术资源。 基础美术来自 imagine_text_to_image 或用户/本地的静态图。脚本仅进行后处理。
  6. 除非用户要求集成,否则不要将实验性输出放入游戏。
  7. 对于游戏使用,优先采用一个移动循环。 对于预览,可以从整个 6 秒多周期剪辑中进行密集采样;对于引擎贴片,可选地在视觉质量检查后对一个周期(12–16 帧)进行重新采样。
  8. 完成后报告视频、清理后的帧、精灵条和预览 GIF 的绝对路径。

工作流程

1. 规划

选择最小可用的运行方式:

  • 侧视奔跑/行走循环 → 使用本技能
  • 多动作核心角色工具包 → 仍然每个动作使用 $generate2dsprite;仅当用户要求时才使用视频生成移动动画
  • FX / 抛射物 / 道具包 → 使用 $generate2dsprite,不用视频

创建目录结构:

<out_dir>/
  base/
  video/
  frames-raw/
  frames-clean/
  sprite/          # 默认 8 帧集 + 更密集的 x16/x24/x48 帧集
  prompt-used.txt
  pipeline-meta.json
  README.txt

2. 构建基础静态图

选项:

  • A. 现有精灵: 读取图像,如有需要合成到纯 #FF00FF 背景上,然后将该沙盒路径传递给 imagine_image_to_image / imagine_image_to_video
  • B. 新角色: 使用 imagine_text_to_image 生成纯品红色背景、全身、侧视图、居中的图像
  • C. 匹配参考: 使用参考 file_path 调用 imagine_image_to_image,将其迁移到品红色背景上并保持角色身份

基础图要求:

  • 全身可见,品红色边缘充足
  • 侧视图用于奔跑/行走(侧面或 3/4 侧),脚部接近底部三分之一处
  • 当存在参考时,艺术风格与项目其余部分一致
  • 无文字、UI、水印或第二个角色

复制返回的 file_path 并保存为 <out_dir>/base/<name>-base.png;保留基础图的 file_path 用于视频调用。

将确切的图像提示写入 prompt-used.txt。

3. 使用 imagine_image_to_video 制作动画

调用 Grok 的 imagine_image_to_video:

  • 源路径:<基础静态图 file_path>(由 T2I / I2I 返回的沙盒路径)
  • duration:6(默认)或 10
  • resolution_name:480p,除非用户要求 720p
  • prompt:一个简短的现在时态镜头描述(参见 references/prompt-rules.md)

提示中强制包含的运动约束:

  • 角色原地奔跑/行走(类似跑步机效果)
  • 相机锁定 — 无平移、缩放或环绕
  • 背景保持纯色品红色
  • 角色的身份、服装、调色板在整个镜头中保持稳定
  • 仅包含一个连续动作

将返回的视频 file_path 复制到 <out_dir>/video/<name>-<duration>s.mp4。

如果视频工具不可用,则停止(平台限制)。

4. 提取 + 色度抠图 + 采样(本地脚本)

运行处理器(ffmpeg + Pillow + numpy):

python3 .grok/skills/video2dsprite/scripts/video2dsprite.py process \
  --video <out_dir>/video/<name>-6s.mp4 \
  --out-dir <out_dir> \
  --name <name> \
  --frame-counts 8,16,24,48 \
  --cell-size 128 \
  --body-height 100 \
  --foot-y 118 \
  --fps 0

注意:

  • --fps 0 = 提取每个解码帧(使用源帧率)
  • 从角落进行品红色泛洪填充 + 去溢色
  • 对 --frame-counts 中的每个数量进行均匀采样
  • 脚部归一化的单元格、水平精灵条、网格、每个帧数的循环 GIF

可选:仅从已有的清理帧中重新采样更密集的帧集:

python3 .grok/skills/video2dsprite/scripts/video2dsprite.py sample \
  --clean-dir <out_dir>/frames-clean \
  --out-dir <out_dir> \
  --frame-counts 16,24,48 \
  --cell-size 128

5. 质量检查

视觉检查:

  • 预览 GIF 循环无明显跳动
  • 品红色消失(无纯粉色区域);允许边缘残留或重新抠图
  • 脚部保持在稳定基线(无因裁剪不当引起的跳动)
  • 角色身份大致稳定(脸部/衣服不会每帧变化)
  • 动作是原地进行的(未滑出画面)
  • 用于游戏时:选择一个帧数(通常为 16 或 24)或裁剪出一个真实循环

如果角色身份严重漂移或像素过于模糊,则回退到 $generate2dsprite 生成生产级贴片,并将视频帧集仅作为动作参考。

6. 交付

仅报告路径(除非用户要求集成到游戏中):

  • 视频:video/*.mp4
  • 密集精灵:sprite/x16|x24|x48/
  • 精灵条 / 网格 / GIF:sprite/run-strip-N.png,run-grid-N.png,run-preview-N.gif
  • 元数据:pipeline-meta.json

除非用户要求,否则不要修改游戏代码。

默认值

  • 时长:6s
  • 动作:侧向原地奔跑,面向右侧
  • 导出帧数:8, 16, 24, 48
  • 单元格:128²,身体高度约 100,脚在 y≈118 处
  • 背景:#FF00FF
  • 优先使用单个资源的 imagine_image_to_video,而不是多参考(如果需要多参考,先用 imagine_text_to_image 合成)

权衡(向用户说明一次)

优点: 更密集的中间姿态 → 通常比 4–8 个离散生成姿态更平滑。
缺点: 像素更软、角色身份漂移、色度残留、6 秒多周期剪辑不是一个完美的单循环、资源更重。
经验法则: 8→16→24 通常能获得平滑度的提升;48 往往是边际效益递减;145 个原始帧用于采样,而不是全部用于运行时。

资源

与其他技能的关系

  • $generate2dsprite — 主要贴片生成流程(Codex + Grok,在存在图像生成工具时)
  • $generate2dmap — 地图;此处不使用
  • $video2dsprite — Grok Build 独占的动作密集化路径
  • $game-asset-core / $game-animation-frames — 循环/翻转测试/运动规律和引擎就绪默认值;仍然使用本技能的脚本在沙盒中执行(品红色基础图 + 色度抠图),而不是使用自由形式的背景颜色

Individual skills in this repo

This repo contains 17 individual skills — each has its own dedicated page.

evalix098/tiktoklive

Add user accounts and sign-in to this TanStack Start app. Use when the app needs authentication, sign-in, user accounts, protected routes, or per-user data. Triggers on "auth", "login", "log in", "sign in", "sign up", "account", "users", "authentication", "protected", "who is logged in", "current user", "per-user".

evalix098/tiktoklive

Build browser games and interactive/canvas/3D experiences in this TanStack Start + React app. Use for any game, simulation, or WebGL/Canvas experience — 2D or 3D, single-player. Covers the game loop & timing, 3D orientation/camera conventions, collision, performance, assets, audio, save, game feel, and per-genre playbooks. For WASD / vehicle / flight **input signs and inverted A/D**, open the **`controls`** skill — do not rely on this file or racing-kart alone. Triggers on "game", "minecraft", "fps", "platformer", "racing", "tetris", "snake", "shooter", "3d", "three.js", "canvas", "voxel", "physics".

evalix098/tiktoklive

Player-facing input signs for browser games: WASD, vehicles, flight, FPS mouse-look, and the #1 failure mode (inverted A/D). Mandatory control self-tests and a tiny test interface so you can verify A turns left before shipping. Load for ANY game with movement, steering, flying, driving, tanks, boats, mechs, drones, planes, karts, third-person follow cams — not only racing. Triggers on "controls", "WASD", "inverted", "steer", "flight", "airplane", "kart", "vehicle", "yaw", "roll", "pitch", "mouse look", "A/D".

evalix098/tiktoklive

Design and build polished, non-generic UI for this TanStack Start + React + Tailwind v4 + shadcn/Radix app. Use whenever you create or restyle any interface surface — pages, landing pages, dashboards, forms, modals, nav, and game overlays (start screens, HUD, menus). Covers design tokens, layout, typography, color, spacing, motion, and the anti-"AI-slop" rules that keep output from looking generic. Triggers on "design", "UI", "make it look good", "polish", "landing page", "theme", "style", "redesign", "ugly", "clean up".

evalix098/tiktoklive

Deep guide for game ANIMATION assets: motion cycles, action keyframes, effect sequences, and animation sprite sheets — built around a video-first pipeline. In this app-builder sandbox, execute via the video2dsprite / generate2dsprite skills (magenta + scripts), not ad-hoc ffmpeg. Use whenever generating anything that moves: walk/run cycles, attacks, idles, FX, flags, fire, animation sheets. Complements game-asset-core.

evalix098/tiktoklive

Core discipline for ANY game-asset generation with Imagine tools: the engine-ready defaults users don't state, spec checklists, style anchoring, read-back verification, honest defect flagging. Use whenever generating any game art (sprites, sheets, animations, tiles, UI, FX) — then ALSO load the matching specialist skill: game-animation-frames for anything that moves, game-tilesets for tiles/terrain, game-character-consistency for recurring characters, game-ui-icons for UI and icons.

evalix098/tiktoklive

Deep guide for CHARACTER IDENTITY across images: turnarounds (front/side/ back), state and damage variants, palette swaps, equipment changes, and same-character-in-context sets. Use whenever generating character turnarounds, character sheets, variants of an existing sprite, or any same-subject multi-image set. Complements game-asset-core.

evalix098/tiktoklive

Deep guide for game TILE assets: seamless tileable textures, terrain transition tilesets, autotiles, and ground/platform tiles. Use whenever generating tileable textures, tilesets, terrain transitions, or seamless patterns. Complements game-asset-core.

evalix098/tiktoklive

Deep guide for game UI assets: buttons with interaction states, panels, bars, wordmark logos, and icon sets. Use whenever generating game UI elements, HUD assets, inventory icons, icon sets, buttons, or title logos. Complements game-asset-core.

evalix098/tiktoklive

Generate production-oriented 2D game maps with `imagine_text_to_image`: RPG/top-down maps, side-scroller parallax stages, tilemaps, layered raster maps, prop packs, collision zones, and walkable areas. Use when building browser games that need real map art (not pure code-drawn tiles), layered props, or map collision metadata. Triggers on "map", "level", "stage", "tilemap", "overworld", "dungeon", "side scroller background", "prop pack", "2D map".

evalix098/tiktoklive

Generate and postprocess 2D game sprites and animation sheets: pixel-art characters, NPCs, creatures, spells, projectiles, impacts, props, summons, and transparent PNG/GIF exports. Use when building browser games that need real sprite sheets (not code-drawn placeholders), matching a map art style, or producing magenta-background sheets for chroma-key cleanup. Triggers on "sprite", "sprite sheet", "animation sheet", "pixel art character", "walk cycle", "attack animation", "projectile sprite", "2D game asset".

evalix098/tiktoklive

How to use the Imagine tools in Grok Build: imagine_text_to_image, imagine_image_to_image, imagine_reference_to_image, imagine_text_to_video, imagine_image_to_video, imagine_reference_to_video, and render_file for chat previews. When to build a visual with code instead of generating it, prompt-craft, reference-first handling of real people, factual grounding, and asset-consistency. Load this whenever generating or editing an image or video is on the table. Tool-usage-driven, not triggered by a user merely mentioning images.

evalix098/tiktoklive

Peer-to-peer realtime multiplayer over WebRTC data channels: every user of the deployed app connects directly to every other user (full mesh), the server only brokers the handshake at /api/rtc. Lowest possible latency, zero per-message server cost. Use for 2-8 player co-op/casual realtime: shared cursors, drawing, party games, casual action. Triggers: p2p, peer to peer, webrtc, low latency multiplayer, direct connection.

evalix098/tiktoklive

Use Neon Postgres (the database) in this TanStack Start app. Use when the app needs to store or query data, persist state, or keep per-user data. Triggers on "database", "Postgres", "Neon", "save data", "store data", "persist", "tables", "SQL", "query", "migrations".

evalix098/tiktoklive

Share-link previews and app identity for apps on *.grok.me: the injector-owned og:image card, the SVG favicon, and PWA icons for installable apps. Use when scaffolding, renaming, or restyling the app — and for share / unfurl / OG / Twitter card questions. A custom 1200×630 card from the app's own art is the default — games of every kind (DOM board/word games included), whimsical apps, creative tools, and brand-forward pages; only plain utilities keep the placeholder. Always run the brand-asset pass as a `task` subagent and never wait for it. Triggers on "share", "rename", "app name", "OG", "Open Graph", "twitter card", "unfurl", "og:image", "og:type", "x:game:image", "x-banner", "link preview", "social card", "thumbnail", "preview image", "favicon", "app icon", "PWA", "manifest", "installable", "home screen", "SEO", "meta description".

evalix098/tiktoklive

Official Three.js API and TSL (Three.js Shading Language) reference for LLM code generation. Load when writing or debugging three.js / WebGL / WebGPU / custom materials / shaders / GLTF / advanced three APIs beyond basic game loop/controls. Prefer building-games for game correctness (loop, WASD, camera, orientation); use this skill for full API/TSL depth. Triggers on "three.js", "threejs", "WebGPU", "TSL", "NodeMaterial", "shader", "GLTF", "MeshStandard", "OrbitControls", "WebGLRenderer".

evalix098/tiktoklive

Call the xAI API (Grok) from this app's server code using the injected XAI_API_KEY: chat/LLM features, image and video generation (Imagine), and voice (text-to-speech). Use when the app needs any "AI" / "assistant" / "chatbot" / "Grok" functionality, runtime image/video generation, or speech. Triggers on "AI", "LLM", "chatbot", "assistant", "Grok", "xAI", "generate text", "summarize", "generate image", "AI video", "voice", "text to speech", "TTS", "OpenAI" (use xAI instead).

相关技能