Communitygithub.com

Remotion Controls Skill

Professional skill for code-driven video animation using Remotion. Precisely control 2D animations with React/TypeScript code, natively support transparent background (Alpha channel) rendering, and output WebM/MOV videos with alpha channel. Suitable for template animation assets, UI animations, text layout animations, and 2D character animations. Provides API service, template library, quality control, and self-learning evolution capabilities.

What is Remotion Controls Skill?

Remotion Controls Skill is a dedicated AI agent skill for programmatic video animation generation using the Remotion framework, built on React and TypeScript. Its primary advantage is the ability to precisely control every frame through code, with native support for transparent background rendering, outputting videos with alpha channels in formats like WebM (VP9) and MOV (ProRes 4444). The skill is designed for developers and video editors who need to create high-quality, controllable 2D animations for various use cases, including UI animations, text layouts, character animations, and template assets for video editing software like CapCut (Jianying). It offers a comprehensive suite of capabilities including an HTTP API for service orchestration, a library of pre-built animation templates, automated quality control checks (frame rate, resolution, alpha channel integrity), and a self-learning mechanism that records rendering parameters and scores to optimize future outputs. The skill distinguishes itself from AI-powered tools like ComfyUI or 3D tools like Blender by focusing on deterministic, code-controlled 2D animation with perfect alpha transparency, making it ideal for layering animations onto different backgrounds. It can be integrated into larger video editing workflows, such as `ai-video-editor`, by receiving animation instructions and outputting transparent video files that are then used as animation layers. Users can quickly start by installing dependencies, rendering transparent videos via CLI or API, and serving the API, with detailed documentation on the correct workflow to preserve alpha channels (rendering to PNG sequences then encoding with `ffmpeg`). The skill also provides a rich set of base components for characters, text animations, transitions, and impact effects, all located under `remotion-project/src/components/`.

Works with✓Claude Code~Codex CLI~Cursor
npx skills add beimeibeile/Remotion-controls-skill

Ask in your favorite AI

Open a new chat with this agent skill pre-loaded.

Documentation

What does Remotion Controls Skill do?

Professional skill for code-driven video animation. Remotion is a React-based video creation framework that gives you precise control over every frame via code, with native support for transparent background rendering.

Core Capabilities

CapabilityDescription
Transparent Background RenderingNatively supports Alpha channel; outputs WebM (VP9) / MOV (ProRes 4444) with transparent video
Precise Code ControlUse React/TypeScript to exactly control position, scale, rotation, opacity, and easing curves
API ServiceHTTP API to receive animation commands and output video files
Template LibraryPre-built animation templates (character animation, UI animation, text animation, transition effects)
Quality ControlAutomatic detection of frame rate, resolution, bitrate, and Alpha channel integrity
Self-Learning EvolutionRecords rendering parameters and effect scores to optimize preset parameters

Positioning Differences from ComfyUI / Blender

SkillCore StrengthUse Cases
ComfyUI-controlsAI generation (image/video/audio), creativity uncontrollableCharacter images, scene images, special effects video generation
Blender-controls3D animation, physics simulation, high-quality rendering3D character animation, product demos, physical effects
Remotion-controls2D code animation, precisely controllable, transparent backgroundTemplate animation assets, UI animations, text layout, 2D character animation

Quick Start

1. Install Dependencies

cd remotion-project
npm install

2. Render Transparent Background Video

# CLI method
npx remotion render src/index.ts MyComp out/video.webm --transparent

# API method
python remotion_controls.py render --composition MyComp --output out/video.webm --transparent

3. Start API Service

python remotion_controls.py serve --port 8765

Transparent Background Rendering Instructions (Verified Process)

Remotion natively supports transparent backgrounds, but the CLI's --transparent encoding loses the Alpha channel (verified failed for WebM VP8/VP9, MOV ProRes).

Correct verified process (PIT-024):

  1. Render PNG Sequence: npx remotion render src/index.ts Composition out/frames --sequence (outputs PNG sequence with Alpha channel)
  2. Synthesize ProRes 4444 with ffmpeg:
    ffmpeg -y -framerate 30 -i out/frames/%04d.png \
      -c:v prores_ks -profile:v 4 -pix_fmt yuva444p12le \
      out/animation.mov
    
  3. Verify Alpha Channel: python verify_animation.py out/animation.mov

CapCut Integration Notes (PIT-024):

  • Using add_media_safe to import a ProRes 4444 mov auto-transcodes it to H.264 mp4, losing Alpha (66MB → 0.07MB)
  • Must import manually: copy the mov directly into the draft materials directory → create a VideoMaterial → register in materials.videos → VideoSegment(material=anim_material) → add_segment

Animation Asset Quality Standards

See full standard in QUALITY_STANDARD.md.

Hard requirements for qualified animation assets:

  • Alpha channel preserved (yuva444p12le)
  • ProRes 4444 encoding (profile=4)
  • Resolution / frame rate / duration matching configuration
  • Content coverage ≥ 50% (10-point sampling)
  • File size reasonable (≤15MB/sec)

Automated verification:

python verify_animation.py <animation file> --duration 20 --width 1080 --height 1920

Base Animation Component Library

Located in remotion-project/src/components/, all components support transparent background.

Character & Motion Components

ComponentDescription
CharacterSpriteCharacter sprite: multi-pose switching, position/scale/rotation/opacity keyframe animation
MotionPathMotion path: linear/bezier/elastic/bounce motion
FlashEffectFlash white effect: quick fade-in, slow fade-out
ShakeEffectShake effect: random shake with amplitude decay
FadeEffectFade in/out: controllable fade-in/fade-out duration
PopEffectPop effect: elastic scale bounce
SlideEffectSlide effect: ease-out slide in four directions

Text Animation Components (TextAnimations.tsx)

ComponentDescriptionUse Cases
TextRevealCharacter-by-character reveal: typewriter/fade-in/slide-up/slide-leftNarration, dialogue, title entry
TextPopPop text: scale/rotate/bounceKeyword emphasis, beat sync
TextWaveWave text: character-by-character sine wave floatFun style, background music rhythm
TextGlitchGlitch style: RGB separation + shakeTech vibe, glitch art, transitions
TextGradientGradient shimmer: color flow + light sweepTitles, brand names, premium feel
KineticTextKinetic typography: multi-line combined animationOpening titles, end credits

Transition Effect Components (TransitionEffects.tsx)

ComponentDescriptionUse Cases
FadeTransitionFade in/out transition (in/out/inOut)General transitions, emotional mood shifts
SlideTransitionSlide transition (up/down/left/right)Scene changes, content replacement
ZoomTransitionZoom transition (push in/pull out)Highlight emphasis, lens push
WipeTransitionWipe transition (up/down/left/right/diagonal)Creative transitions, stylized look
BlurTransitionBlur transition (blur then clear)Dreamy transitions, memory flashbacks
GlitchTransitionGlitch transition (RGB separation + scan lines)Tech vibe, glitch art

Impact Effect Components (ImpactEffects.tsx)

ComponentDescriptionUse Cases
PunchImpactPunch impact (shockwave + stars + white flash)Fighting, hitting, collision
ImpactFlashImpact flash (quick white flash)Heavy hit, explosion, beat sync
ParticleBurstParticle burst (stars/sparks/debris)Celebration, explosion, impact
ShockwaveShockwave (multi-ring expansion)Energy release, explosion, tremor
SpeedLinesSpeed lines (radial)High-speed impact, sprint, combo hit

Usage example:

import { PunchImpact, ParticleBurst, FadeTransition } from "./components";

// Fight scene
<PunchImpact startFrame={90} x={540} y={800} scale={1.2} />
<ParticleBurst startFrame={90} x={540} y={800} count={16} particleType="star" />
<FadeTransition direction="in" duration={15} startFrame={0} color="#1a1a2e" />

Usage example:

import { CharacterSprite, FlashEffect, TextReveal, TextPop } from "./components";

<CharacterSprite
  poses={[
    { name: "idle", image: "/doubao_normal.png", frameRange: [0, 90],
      position: [{frame: 0, x: 0, y: 0}, {frame: 90, x: 100, y: 0}] },
    { name: "falling", image: "/doubao_falling.png", frameRange: [90, 150] },
  ]}
  width={250} height={250}
/>
<FlashEffect startFrame={90} duration={10} />
<TextReveal text="Got hit!" fontSize={56} startFrame={95} revealType="typewriter" />
<TextPop text="Ouch!" fontSize={72} color="#ff6b6b" startFrame={110} popType="bounce" />

Project Structure

remotion-controls-skill/
├── SKILL.md                    # This file
├── QUALITY_STANDARD.md         # Animation asset quality standards
├── verify_animation.py         # Automated verification tool
├── remotion_controls.py        # Main control script (CLI/API)
├── .env.example                # Environment variable example
├── .gitignore
├── capabilities/
│   ├── cap_api_wrapper/        # Remotion API wrapper
│   ├── cap_animation_renderer/ # Animation renderer
│   ├── cap_template_library/   # Animation template library
│   ├── cap_quality_control/    # Quality control
│   └── cap_self_evolution/     # Self-learning evolution
├── remotion-project/           # Remotion project
│   ├── src/
│   │   ├── components/         # Base animation component library
│   │   │   ├── CharacterSprite.tsx
│   │   │   ├── MotionPath.tsx
│   │   │   ├── Effects.tsx
│   │   │   └── index.ts
│   │   ├── AnimationTemplate.tsx  # JSON-driven animation template
│   │   ├── Root.tsx
│   │   └── index.ts
│   ├── public/                 # Static assets (character/prop images)
│   └── package.json
└── out/                        # Render output

Capability Modules

cap_api_wrapper — Remotion API Wrapper

  • render_media(): Render video (supports transparent background)
  • render_still(): Render a single frame image
  • select_composition(): Get list of available compositions
  • get_props_schema(): Get Props definition for a Composition

cap_animation_renderer — Animation Renderer

  • Input animation command JSON → Output transparent background video
  • Supports keyframe animation (position/scale/rotation/opacity)
  • Supports easing curves (ease-in/ease-out/ease-in-out/linear/spring)
  • Supports multi-layer compositing

cap_template_library — Animation Template Library

  • Character animation templates (Doubao getting hit, expression switching, etc.)
  • UI animation templates (button click, page transition, etc.)
  • Text animation templates (typewriter, fade in/out, bounce, etc.)
  • Transition effect templates (fade in/out, slide, zoom, etc.)

cap_quality_control — Quality Control

  • Frame rate detection (24/30/60fps)
  • Resolution detection (1080p/4K)
  • Alpha channel integrity detection
  • Video duration vs. expected comparison
  • File size reasonability check

cap_self_evolution — Self-Learning Evolution

  • Records rendering parameters and effect scores
  • Optimizes preset parameters
  • Generates best practice documentation
  • Template effect leaderboard

Environment Variables

REMOTION_PROJECT_PATH=./remotion-project
REMOTION_OUTPUT_DIR=./output
REMOTION_DEFAULT_FPS=30
REMOTION_DEFAULT_WIDTH=1080
REMOTION_DEFAULT_HEIGHT=1920
REMOTION_API_PORT=8765

Integration with ai-video-editor

Remotion-controls-skill serves as the animation asset creation tool for ai-video-editor:

  1. ai-video-editor's director engine outputs animation instructions
  2. Remotion-controls-skill receives instructions and renders transparent background video
  3. The transparent background video serves as an animation asset layer for CapCut templates
  4. After users replace the background, the animation perfectly overlays

Notes

  • Remotion requires Node.js 18+ environment
  • For transparent background video, WebM format is recommended (better CapCut compatibility)
  • Rendering long videos is time-consuming; segment rendering and compositing is recommended
  • Avoid using CSS filters in animations (some filters do not support transparent backgrounds)

Related Skills