video-captioner — Animated Caption Pipeline

Transcribes speech with Whisper and burns animated word-by-word captions into videos. No colour grading — for visual enhancement use the video-enhancer skill first.

Pipeline

input video → Whisper transcription → pycaps subtitle render → output video with captions

Caption styles

Three options:

Default (no flag) — pycaps built-in minimalist style: plain white text, no effects.
Futuristic (bundled) — --css scripts/futuristic.css — alternating gold/magenta captions with a glow effect and monospace font. Good for tech, gaming, or high-energy content.
Custom — the user can supply any CSS file path via --css /path/to/custom.css.

If the user has not mentioned a caption style, ask whether they want the default look, the futuristic style, or a custom CSS file.

Static captions

Use --caption to add predefined text at specific timestamps instead of transcribing audio:

--caption "START-END: TEXT"

Format:

Times use M:SS or MM:SS format (e.g., 0:05 = 5 seconds, 1:30 = 90 seconds)
Multiple captions: use --caption multiple times
If both static captions and transcription are provided, static captions take precedence

Example:

uv run python scripts/caption_service.py \
  --input ./input --output ./output \
  --caption "0:01-0:05: Hello world" \
  --caption "0:10-0:20: This is a demo" \
  --css scripts/futuristic.css

Caption styling

Styling is managed via JSON config files. When using static captions, styling is applied automatically.

Default styling

When no --style-config is specified, config.default.json is used:

Background: white (#FFFFFF)
Text color: blue (#0066FF)
Font: auto — uses an available brand font tagged caption first, then falls back to Courier New
Padding: 10px 15px
Margin: 0

Using a style config

Switch styles by passing a config file:

# Use dark-mode style
uv run python scripts/caption_service.py \
  --input ./input --output ./output \
  --caption "0:01-0:05: Hello" \
  --style-config config.dark-mode.json

Included configs:

config.default.json — white bg, blue text, Courier New (default)
config.dark-mode.json — dark bg, white text, larger font

Creating custom style configs

Copy config.default.json and modify:

{
  "caption_bg_color": "#000000",
  "caption_color": "#FFFFFF",
  "caption_font": "auto",
  "caption_font_tag": "caption",
  "caption_padding": "20px 30px",
  "caption_margin": "10px",
  "caption_font_size": 48
}

Then use it:

uv run python scripts/caption_service.py \
  --input ./input --output ./output \
  --caption "0:01-0:05: Text" \
  --style-config my-custom-style.json

Overriding config with CLI flags

CLI flags take precedence over config file values:

uv run python scripts/caption_service.py \
  --input ./input --output ./output \
  --caption "0:01-0:05: Hello" \
  --style-config config.dark-mode.json \
  --caption-color "#00FF00"  # Override config's text color

Style config reference

Key	Type	Description
`caption_bg_color`	string	Background color (hex or CSS color name)
`caption_color`	string	Text color (hex or CSS color name)
`caption_font`	string	`auto`, font family, or path to `.ttf`, `.otf`, `.woff`, `.woff2` file
`caption_font_tag`	string or null	Brand font tag to prefer from `brand-manager` (`caption`, `body`, `heading`, etc.)
`caption_padding`	string	CSS padding (e.g., `"10px 15px"`)
`caption_margin`	string	CSS margin (e.g., `"0"`)
`caption_font_size`	integer or null	Font size in pixels (null = auto-scale)

Available fonts

System fonts can be specified by name:

DejaVu Sans / DejaVu Sans Bold / DejaVu Serif
Liberation Sans / Liberation Mono`
Courier New
Ubuntu Mono
Any TTF file path: /path/to/font.ttf

Or choose from brand fonts in skills/brand-manager/brand-assets/asset-manifest.json. When caption_font is auto, the skill reads the brand manifest (or CLAW_BRAND_ASSETS_DIR if set) and uses the first available font tagged by caption_font_tag; if no tagged font exists, it tries brand fonts tagged caption, body, heading, or bold, then falls back to Courier New. If the resolved brand font is a file path, the generated CSS includes an @font-face rule so pycaps can render it.

How to run

Install dependencies (first run only):

cd "$SKILL_DIR" && uv sync

Note: the very first run will be slow (potentially several minutes) because pycaps downloads the Whisper speech recognition model. Warn the user about this before starting.

Then process videos:

cd "$SKILL_DIR" && uv run python scripts/caption_service.py \
  --input <path/to/input> \
  --output <path/to/output> \
  [--css scripts/futuristic.css]

Common invocations

# Default minimalist captions (Whisper transcription)
uv run python scripts/caption_service.py --input ./input --output ./output

# Futuristic style
uv run python scripts/caption_service.py \
  --input ./input --output ./output \
  --css scripts/futuristic.css

# Custom CSS
uv run python scripts/caption_service.py \
  --input ./input --output ./output \
  --css /path/to/my.css

# Static captions with default styling (white bg, blue text, Courier New)
uv run python scripts/caption_service.py \
  --input ./input --output ./output \
  --caption "0:01-0:05: Subscribe for more" \
  --caption "0:10-0:15: Follow us on Instagram"

# Static captions with dark-mode style
uv run python scripts/caption_service.py \
  --input ./input --output ./output \
  --caption "0:01-0:05: Hello world" \
  --caption "0:10-0:20: This is a demo" \
  --style-config config.dark-mode.json

# Static captions with custom style config
uv run python scripts/caption_service.py \
  --input ./input --output ./output \
  --caption "0:01-0:05: Custom text" \
  --style-config my-custom-style.json

# Static captions with CLI flag overrides
uv run python scripts/caption_service.py \
  --input ./input --output ./output \
  --caption "0:01-0:05: Hello" \
  --style-config config.dark-mode.json \
  --caption-color "#00FF00"

# Watch mode (polls for new videos every 10s)
uv run python scripts/caption_service.py --input ./input --output ./output --watch

After running

Report back to the user:

How many videos were processed successfully and how many failed.
The full path to the output directory.
If any videos failed, name them explicitly.

Edge cases

If ffmpeg is not installed: tell the user to install it (sudo pacman -S ffmpeg on Arch/CachyOS, brew install ffmpeg on macOS, sudo apt install ffmpeg on Ubuntu)
If uv is not installed: direct to https://docs.astral.sh/uv/getting-started/installation/
If the input directory is empty: report clearly rather than silently exiting
The first run downloads the Whisper model (~140 MB for tiny, ~1.5 GB for large) — this is one-time only

Remote Transcription

This skill now supports optional remote transcription while keeping pycaps rendering local.

Default behavior is still fully local: Whisper transcription inside pycaps + local render
Remote mode only replaces the transcription step
Caption rendering, templates, and CSS handling stay local

Supported providers

huggingface
replicate

Config / CLI keys

Key	Default	Notes
`transcription_provider`	`null`	`null` keeps local Whisper-in-pycaps flow
`remote_model`	`null`	Optional provider-specific transcription model override
`hf_token_env`	`HF_TOKEN`	HuggingFace auth env var name
`replicate_api_key_env`	`REPLICATE_API_TOKEN`	Replicate auth env var name
`remote_timeout_seconds`	`300`	Remote call timeout

Examples

# HuggingFace remote transcription + local pycaps rendering
export HF_TOKEN=hf_your_token
uv run python scripts/caption_service.py --input ./input --output ./output --transcription-provider huggingface

# Replicate remote transcription + local pycaps rendering
export REPLICATE_API_TOKEN=r8_your_token
uv run python scripts/caption_service.py --input ./input --output ./output --transcription-provider replicate --remote-model <replicate-model-slug>

Notes

There is no silent fallback to local mode if remote auth/config is missing
Existing local CLI usage remains valid
If you do not set --transcription-provider, pycaps continues using its built-in Whisper flow

FilippTrigub/abra

What is abra?

Ask in your favorite AI

Documentation