video-making

Video making

Turn a folder of raw clips into a narrated video (wide 16:9 and tall 9:16) that the user reviews and fixes in his Shot Picker app. You cannot watch video. You work from small proxies, contact sheets you look at with vision_analyze, and transcripts. The scripts in scripts/ do all cutting, timing and rendering: never hand-write ffmpeg commands or compute timeline times yourself.

When to Use

• "Make a video from this footage", "edit my run / trip / build video", "use the video skill".
• A project folder under /workspace/ with raw clips, where the user wants a plan, a draft, then a final.
• Don't use for: editing one finished video file, GIFs, or music-only jobs.

Rules that never bend

1. Footage is read-only. Never write, rename or delete anything in the footage folder.
2. No invented facts. Every number or claim in narration and captions (distance, pace, weeks, dates, what an AI said) must come from the user's material or his own words. If you cannot point to the source, ask him or leave it out.
3. Plan before work. No narration audio, music download or render until the user has approved PLAN.md.
4. His edits win. Once a *.shotpicker.json exists the user may have edited it in Shot Picker. Never overwrite it. build_project.py writes a new version (-v2, -v3) each time. Render from the file he names, re-read from disk every time.
5. Say what needs his eyes. Every shot you are unsure about gets a "review" reason (see references/picking-moments.md). Never present a guess as checked.
6. Money and keys. Fish Audio free model only. Pass key file paths, never print a key.

Setup (once per session)

SK=/root/.hermes/skills/media/video-making/scripts      # this skill's scripts folder
P="/workspace/<project folder>"                         # quote it: folder names have spaces

If $SK does not exist: git clone https://github.com/semputs/eromes-video-skill /workspace/eromes-video-skill and use SK=/workspace/eromes-video-skill/video-making/scripts. Run every script as python3 $SK/<script> "$P" ... through terminal. Each script prints its own usage when run with no arguments.

Sandbox facts that shape everything: /workspace is slow to read, a foreground terminal call dies at about 420 s, and there is no GPU. The scripts copy what they need to temp storage and stop themselves before the cap. Exit code 3 means "not finished, run the same command again". (1 or 2 is a real error: read it.) For a full-quality render use terminal with background=true, notify=true.

Project folder layout (create missing folders; the user drops files into the first two):

Path	What
footage/	raw clips from camera and phone. Read-only. (If absent, clips sit in the project root.)
journey/	screenshots, exports, photos the user gives you
brief.md	what the video is, in the user's words
_work/	everything you generate: proxies, sheets, transcripts, stills, log
vo/, music/	narration audio (vo/<id>.wav) and music (+ music/CREDITS.md)
plan.json, PLAN.md, REVIEW.md, SHOTS_NEEDED.md	the plan and what the user must check or supply
<name>-vN.shotpicker.json	the edit. Opens in Shot Picker.
out/	rendered videos

Procedure

Do the steps in order. Do not skip a "Done when".

1. Brief. Read brief.md. If it is missing or thin, ask the user at most five questions (story in one sentence, length, wide/tall, narration voice and language, music mood, must-have moments) and write it. If STYLE_PROFILE.md exists in the project folder or in /workspace/youtube-style/, read it: it is how his videos sound and look. Done when: brief.md states story, target length, formats, voice, language.

2. Inventory. python3 $SK/probe.py "$P". Read the table. Fix any wrong kind (cam / screen) in _work/inventory.json. Note the printed read speed: it decides steps 3 and 10. Done when: every clip is listed with a sane length.

3. Proxies. Read speed 15 MB/s or more: python3 $SK/proxies.py "$P" (repeat while it exits 3). Slower: python3 $SK/proxies.py "$P" --bat, ask the user to double-click MAKE_PROXIES.bat in the project folder, then python3 $SK/proxies.py "$P" --check. Done when: --check says all proxies are ready.

4. Look and listen. python3 $SK/sheets.py "$P" then python3 $SK/transcribe.py "$P" (repeat while it exits 3). Look at every sheet in _work/sheets/ with vision_analyze (each is 800x450 with up to 12 frames, time in seconds printed on each frame, exact times in _work/sheets/index.json). Read _work/transcript.md. For a stretch that looks promising, zoom in: python3 $SK/sheets.py "$P" --only <clip name> --step 1. Write _work/log.md: per clip, one line per usable moment: clip, from-to seconds, what is visible, what is said, why it could be used, score 1-3. Follow references/picking-moments.md. Done when: every clip has at least one log line or the note "nothing usable", and no sheet is unviewed.

5. Clock sync (only when footage comes from two devices, e.g. action cam + phone screen recording). Follow references/clock-sync.md. Short version: find a camera frame whose real time is known (the camera filming the phone clock), then python3 $SK/sync.py "$P" --date YYYY-MM-DD --anchor "CLIP.MP4@seconds=HH:MM:SS" --screen-anchor "SCREEN.mp4@seconds=HH:MM:SS". Done when: _work/wallclock.json exists and every CHECK: line is either fixed or copied into your review notes.

6. Missing material. Decide what the story needs that the footage lacks. Fetch what you can yourself (a page of his site with browser_navigate + screenshot, a shared sheet as CSV). Write the rest to SHOTS_NEEDED.md as a numbered list the user can act on ("Screenshot the Gemini chat where the 18-week plan was first proposed, whole message visible") and tell him. Turn each still into a clip: python3 $SK/still2clip.py "<image>" --out "$P/_work/stills/<name>.mp4" --dur 6 (--scroll for tall chat screenshots, --zoom for a slow push-in). Look at every image before using it. Done when: each story beat has footage, a still, or a line in SHOTS_NEEDED.md.

7. Plan. Write plan.json (format: python3 $SK/build_project.py with no arguments prints it; see also references/picking-moments.md for structure, pacing and narration rules). Then python3 $SK/build_project.py "$P". Fix every error it prints and run again; read every WARNING. Send the user: the length, the section list, the full narration script, and the contents of REVIEW.md. Point him to PLAN.md. Stop and wait for approval or changes. Done when: the user has said the plan is OK (or you have applied his changes and he approved).

8. Narration and music (after approval only). Narration: one audio file per narration line at vo/<id>.wav, made with the local-tts-voice-cloning skill (Fish Audio reference). Voice: the one named in brief.md; if none is named, ask. QC every line with local-speech-transcription; re-take failures; list lines that still sound wrong in your review notes. Music: free-to-use tracks only (use web-asset-download if a site hides the file). Save to music/, and record title, author, URL and licence in music/CREDITS.md. Add it to plan.json under "music". Run build_project.py again: real audio lengths replace the estimates and may raise new timing errors. Done when: build_project.py prints no errors and no "no audio yet" warning.

9. Draft. python3 $SK/render.py "$P/<name>-vN.shotpicker.json" --draft --format both. Check your own work before showing it: python3 $SK/sheets.py --file "<draft.mp4>" --out "$P/_work/qc/<name>" --step 3, look at every sheet (wide and tall): captions readable and not cut off, subject inside the tall crop (fix with "x9"), phone overlay not covering faces or captions, no black frames. Fix, rebuild, re-render. Then tell the user: the two draft paths (Windows form, as printed), the project file to open in Shot Picker, and REVIEW.md items with their times in the video. Done when: the user has both drafts and the review list.

10. Changes. If he edited in Shot Picker: re-render that file (render.py re-reads it). If he asks for changes in chat and has not edited the file: change plan.json, rebuild (new version), render. If he asks in chat and has edited the file: copy his file to the next version number and edit that JSON directly (same fields as plan.json shots, see the Shot Picker README "For agents"). Done when: he approves a draft.

11. Final. Read speed 15 MB/s or more: python3 $SK/render.py "<project file>" --format both with background=true, notify=true. Slower: python3 $SK/render.py "<project file>" --write-bat and ask him to double-click RENDER_FINAL.bat. Check the finals with sheets.py --file as in step 9. For FilmCraft: he presses Ctrl+Shift+E in Shot Picker (File ▸ Export timeline for FilmCraft). Tell him plainly that the FilmCraft timeline carries the cuts, narration and music, but not captions, subtitles or the phone overlay: those exist only in render.py output. Deliver with the videos: a description draft in his style, chapter times (section starts from PLAN.md), and the music credits. Done when: wide and tall finals exist in out/ and you have looked at sheets of both.

Quick Reference

python3 $SK/probe.py "$P"                                   list clips, measure read speed
python3 $SK/proxies.py "$P" [--bat|--check]                 360p proxies (all later steps use them)
python3 $SK/sheets.py "$P" [--only NAME] [--step N]         contact sheets for vision_analyze
python3 $SK/sheets.py --file V.mp4 --out DIR --step 3       sheets of any video (QC a render)
python3 $SK/transcribe.py "$P" [--lang en] [--no-whisper]   speech + loud moments -> _work/transcript.md
python3 $SK/sync.py "$P" --date D --anchor "CLIP@s=HH:MM:SS" [--screen-anchor ...]
python3 $SK/sync.py "$P" --where "CLIP@s"                   same moment in the screen recording
python3 $SK/still2clip.py IMG --out "$P/_work/stills/x.mp4" [--dur 6] [--scroll|--zoom]
python3 $SK/build_project.py "$P"                           plan.json -> -vN.shotpicker.json, PLAN.md, REVIEW.md
python3 $SK/render.py FILE.shotpicker.json [--draft] [--format 16:9|9:16|both] [--write-bat]

Pitfalls

• Times are seconds inside the source clip, not times in the finished video. The frame label on a sheet is the clip time. build_project.py works out video times; read them from PLAN.md.
• A sheet shows one frame every few seconds. Before using a moment, zoom in with --step 1 and set in/out on what you actually see. Add 0.5 s of handle on each side of speech.
• "pip": "auto" puts the phone screen recording of the same real moment in a corner. It needs step 5. If sync is unverified the builder flags the shot for review by itself: leave that flag in.
• Tall version is a crop of the wide one. "x9" moves the crop (0 left, 0.5 centre, 1 right). Stills keep the whole picture automatically.
• Narration must fit. A line starts on the shot that names it and may run over following shots. The builder refuses overlapping lines and tells you how many seconds to add. Do not stack two lines on short shots.
• Japanese in captions needs a font with those glyphs. render.py picks one by itself in the sandbox. If a draft sheet still shows boxes, pass --font "<name>" with a name from fc-list :lang=ja (IPAGothic works here). The Windows render uses Arial, which falls back for Japanese.
• Whisper invents text over silence and mishears Malay and names. Treat transcripts as hints for where speech is, and confirm wording against what the user says.
• Shot Picker's own Export video and the FilmCraft timeline do not include captions, subtitles or the phone overlay. Only render.py draws those.
• After a timeout look for leftover ffmpeg/python3 processes and kill them by PID before retrying.

Verification

• python3 $SK/build_project.py "$P" exits 0 and REVIEW.md lists every doubt you have.
• The draft's printed length is inside the target in brief.md.
• You have looked at contact sheets of the rendered wide and tall videos, not only of the source clips.
• Every narration fact has a source you can name (file, screenshot, or the user's message).