Multicam podcast editor
You are replacing a human podcast editor. Inputs are usually: one close-up camera per host, one wide camera, one microphone file per host (the cameras' own audio is used only for syncing), optionally a reference episode cut by a human, the list of news/topics with links, and the show's branding assets. The goal: an episode that can be watched like the reference — picture never ahead of or behind the voices, sound only from the mics, cuts follow the conversation, news plaques appear when the topic is discussed.
Everything runs through one CLI: python scripts/pe.py <command> episode.json. The scripts do the deterministic
work (sync, mixing, rendering); you do the editorial work (choosing crops, teaser quotes, plaque texts and timing).
0. Setup (once per machine)
ffmpeg/ffprobein PATH (or envFFMPEG/FFPROBEpointing to binaries). Checkffmpeg -filters | grep scale_vton macOS.pip install -r requirements.txtin a venv (seereferences/setup.mdfor Whisper models, offline/blocked-network options and fonts). Optional:scripts/fetch_fonts.shdownloads Oswald + Montserrat (OFL, Latin+Cyrillic).- Keep the venv, converted Whisper model and brand assets in a stable folder — temp folders get wiped.
1. Gather and write the config
Create episode.json (copy assets/examples/episode.example.json) and brand.json (assets/examples/brand.example.json).
Ask the user for anything you cannot infer: which mic belongs to whom, the news links, the reference episode,
branding (intro sting, subscribe animation, colours, fonts). Schema: references/config.md.
Run pe.py probe episode.json — durations, resolution, HDR. Camera file names change every recording; identify roles
by looking at frames, not by names.
2. Sync, speakers, audio, transcript
pe.py sync episode.json # camera_offsets + mic_offsets on timeline T (T = reference mic time)
pe.py activity episode.json # who speaks when, from mic levels (expect >15 dB separation)
pe.py audio episode.json # declip, HPF, level-match, duck idle mics -8 dB, compress, limit, -16 LUFS
pe.py transcribe episode.json # words.json + transcript.txt
Read the sync report. Healthy: z > 50, consistency [0.0], |drift| < 30 ppm. If one camera disagrees between mics
or drift is large, stop and investigate (wrong audio stream, recording split in two files, variable sample rate).
If there is a clap, verify it visually: pe.py frames episode.json --at <clap T> — hands meet on the frame of the sound.
3. Look at the cameras and choose crops
pe.py frames episode.json --at 120 700 1500 writes grid frames (100-px grid in 4K units). Read them to:
- map cameras to speakers (
speakers[].camera,wide.camera); - size the close-ups. Match the reference episode's face size (e.g. glasses width ~260 px in 1080p) and keep the face
at ~0.49 width / ~0.32 height of the frame; never crop tighter than 2x (4K -> 1080p keeps full detail up to 2x).
Write
crop: {"scale": [3072,1728], "rect": [w,h,x,y]}(16:9 rect inside the 3072x1728-scaled frame) and check withpe.py frames episode.json --at 120 700 --crops. Cameras move between sessions — re-check every episode.
4. Edit: main range, cameras, plaques, teaser
Read transcript.txt, then use pe.py words episode.json 10-30 1780-1800 ... for exact word timings + speaker.
main.start: ~0.4 s before the greeting;main.end: ~1.5 s after the last goodbye;close_wide_at: start of the goodbye so both hosts wave on the wide shot.pe.py intermediates episode.json(synced 1080p per camera; 10–35 min for a 30-min episode) andpe.py edl episode.json. Shot statistics to aim for (from a human edit): mean ~7–9 s, wide 30–40 %.- Plaques: one per news item at the moment the host starts reading it (usually "next news…"), 8.5–10 s.
Facts only from the linked source; if a link fails (paywall/403), find the same story elsewhere and cite that outlet.
Hosts' own theories/opinions get a distinct tag (e.g. "THEORY 1 • hosts' take") and text starting with "Version:" —
never present speculation as news. For scandals about real people use only officially confirmed facts.
Details and copy rules:
references/editorial.md. - Teaser (~40 s, 8–10 clips): the punchiest, self-contained lines, alternate speakers, end on a laugh/callback.
Avoid accusatory lines about real people. Write caption groups by hand (
captions), one token per spoken word, check every clip withpe.py words. Put light leaks on same-speaker cuts (light_leaks_after, indices must exist).
pe.py plaques episode.json
pe.py teaser-captions episode.json
pe.py teaser-motion episode.json
Look at work/plaques/preview_*.png and a few teaser frames before rendering.
5. Render and check
pe.py assemble episode.json --test # ~2 min: teaser -> intro -> first minute; look at it (contact sheet)
pe.py assemble episode.json # full render; ~0.5–1.2x real time
pe.py qa episode.json # loudness, A/V offset vs every camera, plaque frames, ending, chapters.txt
Accept when: A/V within ±1 frame (close-up cameras usually ±20 ms; a far wide camera may read ~30 ms from room
acoustics), loudness at target, every plaque visible at the right moment, ending on the wide shot. Look at
work/qa/*.jpg. Tell the user honestly what was verified by measurement and what they should watch themselves.
6. Deliver
The output file, chapters.txt (paste into the YouTube description; first line must be 00:00), and a short report:
what was done, plaque list with times, teaser lines, anything uncertain (failed links, low-confidence sync).
New versions go to new files (v2, v3); never delete the user's earlier versions or the reference episode unasked.
When the user gives feedback
Typical requests and where they live: "more dynamic teaser" → teaser.zoom_rate; "frame us tighter" → speaker
crops (re-run intermediates); "warmer colour like the reference" → build a LUT with pe.py lut
(references/color.md); "higher quality" → encode.crf 14→12 / preset slow. Record approved settings so the next
episode reuses them.
Reference files
references/config.md— every field of episode.json / brand.jsonreferences/editorial.md— plaques, teaser, camera rules, how to study a reference episode (pe.py reference)references/color.md— HDR iPhone → SDR, matching a reference look with a LUTreferences/setup.md— install, Whisper offline, fonts, performancereferences/troubleshooting.md— known pitfalls and fixes