cutlab — AI Video Region Editing
Edits a masked region of every frame of a video using the Wan2.1-VACE-1.3B diffusion model. Describe the desired result in a text prompt; the model inpaints the masked area while keeping the rest unchanged. Audio is preserved.
The skill directory (where this SKILL.md lives) is referred to as $SKILL_DIR below.
GPU required. Needs a CUDA GPU with ≥ 8 GB free VRAM. Pause the LLM context before running to free VRAM.
When to Use
Use this skill when the user wants to:
- Replace the background of a talking-head video with a different scene
- Edit a rectangular region of a video (sky, sign, screen, etc.)
- Change environment details while preserving the subject
Setup (first run only)
cd "$SKILL_DIR" && uv sync
Wan2.1-VACE-1.3B model weights are downloaded from HuggingFace on first run (~3 GB). rembg model weights are also downloaded on first background-mode run.
Agent Workflow
1. Ask the user
Before I edit the video(s), I need to know:
💬 Prompt (required)
Describe what the edited region should look like, e.g.:
"modern office with floor-to-ceiling windows and city view"
🎭 Mask mode
- background — auto-detect and replace the background [default]
- region — edit a rectangular area (x1,y1,x2,y2 as fractions 0.0–1.0)
📐 If mode=region: mask_region
Format: "x1,y1,x2,y2" e.g. "0.0,0.0,1.0,0.3" for the top 30% of the frame
⚙️ Strength (0.0–1.0, default 0.85)
How strongly to apply the edit. Lower = more faithful to original.
📁 Input / output directories (default: ./input and ./output)
Wait for user response before proceeding.
2. Edit config.json
Write or update $SKILL_DIR/config.json based on the user's choices.
3. Run
cd "$SKILL_DIR" && uv run python scripts/vace.py --config config.json
4. Report results
Tell the user the output file paths, mask mode used, and prompt applied.
Config Reference
| Key | Values | Default | Description |
|---|---|---|---|
input_dir | path | ./input | Folder containing input videos |
output_dir | path | ./output | Destination folder |
prompt | string | (required) | Description of desired edited region |
negative_prompt | string | "worst quality, blurry, distorted" | What to avoid |
mask | background, region | background | Masking strategy |
mask_region | "x1,y1,x2,y2" | (required if region) | Rectangle as fractions |
strength | float (0, 1] | 0.85 | Inpainting strength |
num_inference_steps | integer | 30 | Diffusion steps |
guidance_scale | float | 5.0 | Prompt adherence |
batch_size | integer | 8 | Frames processed per inference batch (lower for less VRAM) |
max_proc_dim | integer | 384 | Max frame dimension before inpaint (lower for less VRAM) |
model | HuggingFace ID | Wan-AI/Wan2.1-VACE-1.3B-diffusers | Model to use |
Common Invocations
# Replace background automatically
cd "$SKILL_DIR" && uv run python scripts/vace.py \
--prompt "serene mountain lake at sunset"
# Edit a rectangular region (top third of frame = sky replacement)
cd "$SKILL_DIR" && uv run python scripts/vace.py \
--mask region --mask-region "0.0,0.0,1.0,0.33" \
--prompt "clear blue sky with scattered clouds"
# Lighter edit (preserve more original detail)
cd "$SKILL_DIR" && uv run python scripts/vace.py \
--prompt "cosy home office" --strength 0.6
# Custom directories
cd "$SKILL_DIR" && uv run python scripts/vace.py \
--input /path/to/videos --output /path/to/edited \
--prompt "futuristic cityscape"
Output
Each input video produces one .mp4 in output_dir with the same filename.
Original audio is muxed back in.
Error Handling
- Insufficient VRAM → prints required vs available GB, tips to free VRAM, exits
- No CUDA GPU → clear error message, exits
- No videos in input_dir → clean message, exits
- Invalid mask_region format → clear error with expected format, exits
- Individual video errors → logged; other videos continue processing