Lightning Emperor: CG-into-plate compositing
A workflow plus tools for putting CG behind or around live-action actors. The tools exist three ways, all running the same code:
the MCP server (lightning-emperor-mcp), the CLI (lightning-emperor ...) and the Blender add-on. Prefer MCP tools when
they are connected; otherwise use the CLI. This file is the judgment: what to measure, what goes wrong, how to check your own work.
Rules that prevent the expensive mistakes
- Measure the plate before deciding anything. Lens, sensor, exposure, white balance, screen colour and strength, noise, where the light comes from. Ten seconds of measurement beats an hour of guessing.
- Get absolute scale right first, and get it two ways. CG scale against the actors depends directly on camera-to-subject distance. Measure it from the pixel size of something known (a head is about 0.24 m crown to chin) and from the lens focus distance in the camera file. If they disagree, stop and resolve it. A mis-measured distance makes the whole CG set the wrong size in every frame. Never add a fudge factor to hide a scale error.
- Find the key light from the actors, not from a brief or set photo. Compare the two sides of a face in linear light. Set photos show what lights exist; the plate shows which dominates.
- Composite in linear light and say what the plate is. Display-referred video: linearise, mix, re-encode. Do not assume log because an image looks flat.
- Look at the image at full size and in crops. Thumbnails hide spill, holes and fringes. Always open the preview a tool writes before trusting a matte.
- A fix can be worse than the bug. After each fix re-look at the whole frame and re-measure what you were protecting.
- You cannot watch video. Measure motion with numbers (step sizes, jitter) and say plainly what only the user's eyes can judge.
- Never overwrite originals. Check input sizes/durations before a long render; keep source files untouched.
- Be straight about limits. When the user says something looks wrong they are almost always right: investigate, report numbers, say what you got wrong.
Workflow
0. Intake. Probe every file. Read camera metadata if the format allows (read_camera_metadata / lightning-emperor metadata): focal length (did it zoom?), focus distance, ISO, white balance.
Ask only what changes the work: where the CG sits relative to the actors, camera height, deliverable format.
1. Camera. solve_camera_distance from a known size in the plate and the lens focus; require agree: true. Then plan_camera (pose, overscan size, pixels per metre).
Render with overscan (about 1.25) so tracked motion never reveals the render edge.
2. Lighting. Find the key side from the faces. plan_lights gives starting positions and colours (azimuth 0 = from camera, +90 = camera-right, kelvin colours).
Calibrate CG exposure to a measurement in the plate (skin patch, backdrop), not by eye. Blinking practicals: practical_states gives the frames to render (on/off).
3. Key. key_frame and open preview_on_grey.png. The key self-calibrates to the screen, but read references/keying.md for what to check (holes in dark clothing, spill colour casts, marks left as islands).
4. Track (only if the camera moves and the backdrop is replaced). track_markers follows a marker constellation with a homography per frame. Read references/tracking.md: frames with few markers are noisy, a flat backdrop is a plane (a homography is the right model), no markers in view means you are holding or interpolating.
5. Shadows. shadow_offset computes where an actor's shadow falls on a backdrop from the light geometry and the actor-to-wall gap. See references/shadows-and-grain.md.
6. Grain. measure_grain from the flat screen area; apply to the CG so it does not look pasted-on. Measure at the final resolution.
7. Composite. composite_frame on a single frame first. Check wall size against the actors, shadow side, edges, black levels, then scale up. Render long jobs in chunks and verify frame counts.
8. Hand off. When tracking cannot be trusted over a stretch, give an artist layered passes and an honest per-frame quality flag, not a pretend-finished comp.
Tool map
| Need | MCP tool | CLI | Add-on |
|---|---|---|---|
| Lens/exposure from camera file | read_camera_metadata | metadata | Plate panel |
| Line an export up with the camera file | align_clip | align | |
| Subject distance | solve_camera_distance | distance | Subject + camera |
| Camera pose, render size | plan_camera, build_shot_scene | Create shot camera | |
| Light rig | plan_lights | Lights | |
| Practical on/off frames | practical_states | Blink | |
| CG states render | render_states | Render CG | |
| Matte + despill | key_frame | key | Plate tools |
| Marker tracking | track_markers | track | Plate tools |
| Shadow offset | shadow_offset | shadow-offset | |
| Grain | measure_grain | grain | Plate tools |
| One-frame comp | composite_frame | comp |
Tool errors come back as {"error": "..."} with the reason; fix the input and retry.
Blender runs as headless jobs. Never try to open a socket that executes code inside a running Blender window.
References (read at the stage they apply)
references/failure-modes.md- what goes wrong, why, and the fix. Skim first.references/scale-and-distance.md- the camera-distance mistake and the rules that prevent it.references/lighting.md- finding the real key light, building and calibrating the rig.references/compositing.md- matte, four kinds of spill, pedestal, softness, light wrap, checks.references/camera-metadata.md- what camera files tell you, and aligning an export to them.references/review.md- how to use critic agents without fooling yourself.references/keying.md- matte, despill, edge handling, and what real footage taught us.references/tracking.md- marker tracking, stabilising, gaps.references/shadows-and-grain.md- shadow geometry, grain measurement and matching.references/blender-notes.md- Blender 4.2-5.x scripting notes for headless work.