Communitygithub.com

video-voice-and-sound

Everything heard in a film, made without being able to hear it. Generate and verify the narrator's voice, choose a voice with the person, score the film so the music follows the mood, place sound effects, duck music under the voice, and hit a loudness target. Use once the script is written, and whenever lines, voice, music or mix change.

What is video-voice-and-sound?

video-voice-and-sound is a Claude Code agent skill that everything heard in a film, made without being able to hear it. Generate and verify the narrator's voice, choose a voice with the person, score the film so the music follows the mood, place sound effects, duck music under the voice, and hit a loudness target. Use once the script is written, and whenever lines, voice, music or mix change.

Works with~Claude Code~Codex CLI~Cursor
npx skills add https://github.com/shubhamnshukla/video-agent-kit/tree/main/skills/video-voice-and-sound

Ask in your favorite AI

Open a new chat with this agent skill pre-loaded.

Documentation

Voice and sound

You cannot hear. Decide from measurements, verify by transcription, and tell the person what to listen for.

1. The voice

Generate one file per line with scripts/vo.py lines.json assets/vo-raw <voice_id>. It asks ElevenLabs for word timings and passes the lines before and after, so the intonation runs on from line to line. Those word timings are the film's clock: every cut and every card is placed on a word.

Then scripts/vo_finish.py assets/vo-raw assets/vo:

  • levels every line to −17 LUFS (generated takes vary by 10 dB or more);
  • with --tighten <prefix>, shortens pauses over 0.34 s in fast sections. Lines written as fragments ("Kabir. Designer. ...") come back with gaps of half a second and more, and the film feels slow for it.

Verify every line. Join the files, transcribe them (npx hyperframes transcribe file.wav --json) and read the text against the script. Generated speech drops and swaps small words: "couldn't" came back as "could". Rewrite such a line ("could not") and generate it again. Report anything the transcriber and the script disagree on, even when you think the transcriber is the one that is wrong.

Pace. After finishing, the film's speech should run 2.4 to 2.8 words a second for an energetic piece. A calm, slow narrator makes a slow film whatever the cutting; this is the most common reason a first draft is called boring.

Redo single lines with --only id,id; the timing file is merged.

2. Choosing a narrator with the person

Voice is taste, and it is theirs. Offer samples early, before the film is built on one voice, because a new voice changes every line's length and every cue.

  • Search the voice library by accent, gender and use (its API lists shared voices with descriptions).
  • Generate the same passage with five to eight voices: vo.py lines.json samples A,B,C --sample 5. Use a passage with the names and the brand name in it.
  • Give them the files with one line each: the library's own description and the measured pace. You cannot rank them by ear; say so.

Names. A voice may mispronounce names from another language. Make an A/B of the name lines: as written, and respelt for that reader (Kabeer, Urjun). Let them pick. Listen-checking the brand name is the person's job; ask for it explicitly.

3. Music that follows the mood

One track from start to end flattens a story. Map the acts to moods first, then find a track for each:

Part of the filmMoodWhat to look for
Opening, meeting peoplelight, hopefulquiet piano or soft pulse, little percussion
The problemtense, heavya dark bed that builds slowly; add a heartbeat or a low drone
The line that sums it upnear silenceone low hit, then almost nothing under the words
The product arrivesliftan upbeat track whose opening is restrained and which kicks later
Ending and end cardreliefthe same track's softer section

Choosing without hearing. For each candidate measure the level over time (scripts/levels.py track 2) and its tempo. You are looking for shape: a steady quiet stretch for the opening; a slow rise for tension; a clear jump of 5 dB or more where a track kicks. Place the kick on the moment the product's answer appears. Find the beat grid (tempo and the time of the first downbeat) for any section cut to the beat.

Editing a track. Cut only on bar lines, with a 40 ms crossfade, and cover the join with a cut in the picture. To end a track early, jump from a bar line in the loud section to a bar line where the soft section begins. After the edit, measure across the join to confirm the level changes where you meant it to.

Changing tracks. Change on a hard cut in the picture, with a hit on the cut. Fade the outgoing track over 0.3 s.

4. Sound effects

Small sounds make cards and cuts feel placed. Keep a kit: three or four short pops, a tap, sent and received tones, pen on paper, a soft whoosh, a low boom, a dark riser, a heartbeat.

  • Measure where each effect's peak is (levels.py file 0.1). A riser may peak 1.9 s after its file begins; a hit may have 0.25 s of silence first. Start the file so the peak lands on the picture.
  • Put the sound about 20 ms after the picture, never before it.
  • Alternate two or three different pops in a run of cards.
  • A heartbeat under a tense section: quiet at first (0.3), rising (0.4, then 0.6), and left alone for a second when the music stops.

5. The mix

Clips in the root composition: music on its own clips with data-volume="1", voice at 1.25 after levelling, effects between 0.35 and 2.0 depending on the file.

scripts/mix.py index.html mix.json writes the volume lanes that duck music under the voice. Choose the levels from the track's measured level: a bed should sit 11 to 13 dB under the voice.

Bed, measuredUnder the voiceIn the gaps
quiet piano, −25 dBFS0.91.5
dark bed, −21 dBFS0.550.95
upbeat track, restrained opening, −17 dBFS0.340.62
the same track at full arrangement, −11 dBFS0.190.4
its soft section, −15 dBFS0.30.75

Run mix.py again whenever a voice clip moves.

Target. Measure the rendered file: ffmpeg -i film.mp4 -af ebur128=peak=true -f null -. Aim for about −16 LUFS integrated and a true peak at or under −1 dBFS.

6. What to tell the person

Every hand-over says, in plain words: nothing was heard; the voice lines were transcribed back and match (or which did not); the loudness figures; and the specific things to listen for, such as how the brand name is said, whether the music is too loud under the voice in a named stretch, and whether a music change feels clean.

7. Licences and keys

Music and effects come from a library the person has an account with, or from sources whose licence allows the use. Record titles and the account in CREDITS.md, and remind them to confirm the plan covers the place the film will be shown. Keys are read from the environment or a file outside the project, never written into it.

Individual skills in this repo

This repo contains 3 individual skills — each has its own dedicated page.

Related Skills