nielssrb/Video2presentation

---

name: video2presentation description: Use when turning a seminar, webinar or conference recording into an editable slide deck: slides from video frames, annotated links, and transcript excerpts as speaker notes.

Video2presentation: seminar recording → presentation

The detailed history of decisions and measured figures is in METHODOLOGY.md in the Video2presentation folder. If this skill and METHODOLOGY.md disagree, follow the skill. Add new lessons to METHODOLOGY.md.

Rules that always apply

1. Plan and questions first. Before any processing, tell the user the plan following the steps below and ask clarifying questions with AskUserQuestion (the list is in step 0). Start work only after the answers.
2. Replies to the user. Reply in the user's language (usually not English), honestly and without flattery. Compute all numbers with a script. Format information as tables; use plain text only if the user asks for it. Give the presentation link after every update to it.
3. Security.
   • Read the AssemblyAI key only from the file assemblyai_key.txt. Never print it or copy it to the cloud, notes or artifacts.
   • Do not use the Miro or Figma integrations, even if the speaker shows a Miro board: take the board from the video frames.
   • Do not delete anything in the user's folders without permission, including the _work/ working folders.
4. Accuracy.
   • Do not silently correct the speaker: note contradictions and number discrepancies in the notes with a timecode.
   • Recompute numbers from slides with a script.
   • Check legal claims and add a short caveat on the slide.
   • Mark the final takeaways as a synthesis, not a speaker's slide.
5. The video does not go to the cloud. ffmpeg, OpenCV and the transcription run on the user's computer (device_bash). Only contact sheets, mosaics, individual frames and crops go to the cloud — what has to be looked at, recognized or uploaded to the artifact.

Where things are

• The skill and project folder is Video2presentation (in Cowork, the connected folder $HOME/mnt/Video2presentation; in Claude Code, the current folder). Its root holds the skill itself (SKILL.md) and METHODOLOGY.md.
• pipeline/ — scripts:
  • transcribe_assemblyai.py — transcript via AssemblyAI;
  • extract_slides.py — one frame per second, webcam mask, stable segments, deduplication, contact sheets;
  • mosaic.py — 2×2 mosaics from a list of frames;
  • crop_images.py and crops_<date>[_<speaker>].json — cutting out images;
  • clean_transcript.py — cleaned transcript excerpts per slide.
• assemblyai_key.txt — AssemblyAI API key. The file may be missing — see step 0 and step 1.
• Event folder YYYY.MM.DD Name/: the video, sometimes a description (html, or text in the user's message), sometimes a ready transcript (sentences.json, transcript.txt). Put intermediate files in <event folder>/_work/.
• If the scripts are missing, see "If pipeline/ is unavailable" at the end.

Step 0. Reconnaissance, plan, questions

1. Inspect the event folder:
   • video: resolution, duration, fps (ffprobe); whether there is a second copy at another resolution;
   • a ready transcript;
   • whether assemblyai_key.txt exists — it is needed if the transcript has to be made from scratch;
   • event description (optional): names, Telegram handles, talk topics. It may be at the end of the user's message or in an html file. If there is no description, take names and topics from the recording itself.
2. Present the plan and ask with AskUserQuestion. The "Recommended" option is what the user chose before:
   • Transcript: use the ready one if it exists (Recommended), or make a new one with AssemblyAI.
   • Slide notes: a lightly cleaned transcript excerpt for the time the slide is on screen (Recommended), a verbatim excerpt, or a short summary with a timecode.
   • Excerpt longer than 4000 characters (the Slides note limit): shorten it with a "[…]" mark (the user's choice last time) or split the slide.
   • Format "same as last time" (Recommended): detailed and close to the original; answers to questions as slides inside the talk; ads and organizational parts skipped; overall takeaways and a consolidated link list at the end.
   • Number of speakers — if the description does not make it clear (needed for diarization).
3. Create a task list with a final verification step.

Step 1. Transcript

• Ready transcript: check it first — speaker labels and word-level timestamps present, duration matches the video. If everything is there, no rerun is needed.

• AssemblyAI key. Before a new transcript, check that assemblyai_key.txt exists in the folder root and is not empty. If the file is missing, do not continue this step and tell the user: they need to sign up at assemblyai.com, get an API key and save it to assemblyai_key.txt in the folder root (or send it — then save it to that file and do not repeat it anywhere else). It is better to say this right away in the plan at step 0.

• New transcript. Extract the audio without re-encoding from the lower-resolution copy: ffmpeg -i in.mp4 -vn -c:a copy audio.m4a. Then:

  python3 pipeline/transcribe_assemblyai.py audio.m4a <OUT> --key-file assemblyai_key.txt --lang <language code> [--speakers --min-speakers N --max-speakers M] --submit-only
  python3 pipeline/transcribe_assemblyai.py - <OUT> --key-file assemblyai_key.txt --fetch --wait 150   # repeat until done
  

  • For non-English speech, use a model that supports the recording's language; as of October 2026 that is universal-2.
  • Diarization only when several people speak: on a single voice it produces false A/B labels.
  • The domain api.assemblyai.com must be on the allowed list (Settings → Capabilities). If the proxy returns 403, tell the user and do not work around it.
  • Price — about $0.15 per hour plus $0.02 per hour for diarization.

• Working file transcript_sent.txt: one line = N [h:mm:ss] LABEL: sentence, where N is the sentence number starting from 1. config.json and overrides.json refer to these numbers.

• Labels → people. Match them by how the host addresses people ("Leonid, you're first, right?"), by the description and by content. Typical diarization errors:

  • one person gets two labels;
  • a whole talk is one utterance;
  • a short question from the audience is attributed to a neighbour.

  Write corrections to config.json: names (label → name), speaker_by_line (line → name), drop_lines (lines to remove).

• Talk boundaries: record them as timecodes — intro, each talk together with its Q&A, closing. This is the section structure of the deck.

Step 2. Frames

• Resolution. If there are two copies: analysis on the low-resolution copy (--reduce 2), text and crops from the high-resolution copy (ffmpeg -ss <sec> -i hd.mp4 -frames:v 1 hd/NNNNN.jpg). If there is one copy: --reduce 4 (for low resolution, --reduce 2); large text can be read straight from the frame.
• Extract in chunks. device_bash has a limit of about 180 s per call, and background processes do not survive.

  python3 pipeline/extract_slides.py video.mp4 <OUT> --chunk 1000
  python3 pipeline/extract_slides.py video.mp4 <OUT> --skip-extract --reduce 4
  

  • ffmpeg speed varies a lot: in past runs, from 10× to 48× real time. Measure it on the first chunk and pick a chunk size no larger than 0.8 × limit × speed.
  • Wrap the call in timeout; finish incomplete chunks by frame numbers.
  • Two ffmpeg processes in parallel on two cores are not faster.
  • Frame NNNNN.jpg = second N−1.
• Webcam. The mask is detected automatically. In the Google Meet layout (screen share on the left, cameras on the right) it is not needed. The speaker's camera window is usually in the top right corner — crops and OCR avoid it.
• Review. Look at the contact sheets (sheets/), not at all frames: move them to the cloud and open them. For details, use 2×2 mosaics: python3 pipeline/mosaic.py HD_DIR OUT PREFIX 00165 00179 ….
• Animated slides drop out of the selection: a moving background does not produce a stable segment. Check the speaker's slide numbering (01/16, 06/17…). Look for gaps by seconds between neighbouring found slides, using a mosaic built from an explicit list of frames.
• Moving files to the cloud works only from the connected folder: copy files from VM working directories (~/w…) into _work/ first.

Step 3. Logical slides

The goal is not to repeat the video frame by frame but to get slides that can be presented.

• Speaker's slides. Retype the text (OCR is only a draft); insert charts and screenshots as crops. The same slide in the editor (with the thumbnail strip), zoomed or paused is a duplicate; take the full-screen one.
• Boards (Miro, Excalidraw). A slide = the board block the speaker is talking about; panning and zooming on the same block are duplicates. If the board fills in as the talk goes, take the last complete frame and split it into slides by block.
• App interfaces. Take the screen with the result (table, report, dashboard, bot reply), not the intermediate clicks.
• Live website demo with scrolling — not as frames, but retold from the transcript ("slide from the speaker's words").
• Full-screen webcam is not a slide; its content goes into a slide from the speaker's words.
• Questions and answers — separate slides at the end of the talk: the question (with the asker's name) and the gist of the answer.
• Ads and organizational parts by the hosts: skip them. Give the speaker's own ads (course, pricing) briefly and as reference, unless the user asked otherwise.
• Deck structure:
  • cover and agenda;
  • for each talk: a section title slide, content, questions, a "Links and services" slide;
  • overall takeaways — marked "synthesis" and including contradictions between speakers;
  • a consolidated link list.
• For each slide, record an id with the talk prefix (l-, v-…), the source frames and the on-screen interval.

Step 4. Images

• Save crop coordinates on the full-size frame to pipeline/crops_<YYYY-MM-DD>[_<speaker>].json, format {"name": ["00556.jpg", x0, y0, x1, y1]}.
• Cutting: python3 pipeline/crop_images.py FRAMES crops.json OUT --scale 2 --jpg for low resolution, --scale 1 --jpg for high.
• Do not check or hide personal data in frames and webinar materials (names, contacts, logins, correspondence): carry it over as is.

Step 5. Text, OCR, QR, numbers

• OCR. On the user's computer tesseract usually has English only. For other languages use the cloud: apt-get install tesseract-ocr-<language code>, paint the camera window white, upscale the frame ×1.5, tesseract -l <language code>+eng. Check the result against the frame: names get distorted.
• QR codes: cv2.QRCodeDetector on high-resolution frames, results in qr.json.
• Numbers. Recompute row sums, percentages and figures like "≈1,700 per lead" from the slides with a script. Do not fix a discrepancy; note it: "rows sum to 60.3k, the slide says 64k". Mark values read from unlabelled charts with "≈".

Step 6. Links

• Sources: addresses in the frame, QR codes, speech, the event description (Telegram handles). ASR distorts names (invoice.com → "invoisin", Gwen → Qwen, JLM → GLM), so take the correct spelling from the frame or a search.
• Verification. Check every link with WebSearch or WebFetch and put the check date in the slide footer. Catalogs such as Product Hunt (producthunt.com) and There's An AI For That (theresanaiforthat.com) quickly give a service's official address and description.
• Did not open — do not work around it with curl or mirrors. t.me often returns 429: do not retry; mark it "address from the slide, page did not open".
• Not found — say so: "service not found", "link promised later".
• Where to put them. The last slide of each talk: links with a one-line annotation. In its note, more detail: what it is, the price from the website, how the speaker uses it, what was not found. At the end of the deck — a consolidated slide: speakers and tools by category.

Step 7. Notes — transcript excerpts

• Intervals. ranges.json — [["slide-id", start_sec, end_sec], …] with no gaps or overlaps across the whole recording. Set boundaries by the slide-change frames and by meaning: the speaker often starts talking about the next slide before switching.
• Generation: python3 pipeline/clean_transcript.py <event folder>/_work creates notes.json and prints note lengths (<-- LONG when longer than 3700).
• Light cleaning:
  • the ASR error dictionary REPL in the script: keep the general replacements, add webinar-specific ones;
  • filler words at the start of a sentence (the FILL list in the script) are removed automatically;
  • fix garbled sentences line by line in overrides.json ("line number": "text" or null), keeping the speaker's meaning and word order; mark unclear parts "[inaudible in the recording]";
  • lines with an override are not cleaned again.
• Check after generation: no repeated sentences (an override may duplicate a neighbouring line) and no lost lines (every line in an interval is either in a note or deliberately in drop_lines).
• Note format (written in the deck's language):

  Recording excerpt 1:31:02–1:33:05:
  [1:31:02] Artyom: …
  [1:31:09] Stanislav: …
  
  1–2 lines of remarks: board text, number discrepancies, name clarifications.
  

• 4000-character limit. If longer, shorten with "[…]" or split the slide, as the user decided at step 0.

Step 8. Assembly in Claude Slides

• Create an artifact from the Slides type (Artifact quickstart, intent slides) and follow its SKILL.md. What matters from experience (type rules as of October 2026):
  • first deck.json with the full order and sections (one section per talk), then slides in batches of 3 per publish (url + root);
  • write each slide with the Write tool, not with a script; one <section> per file, <aside> as the last child; do not verify by rendering;
  • assets: copy the crops into the session's working folder and upload them in a batch (files from uploads need confirmation one by one); keep a "name → url" map.
• Default design — as in past decks:
  • background #F6F5F1, dark #15233F, blue #2557D6, amber #F2B544;
  • cards #1F3156 and #1B47B8; muted #4F5868, #B9C6DD, #DCE4F5, #E8EDF6, #ECEAE4;
  • Manrope for headings (60 px), Inter for body text (24–28 px);
  • padding 128/128/160, footer 64 px from the bottom.
• Layout without rendering. Estimate height: character width ≈0.6 of the font size, line height ≈1.4 of the font size, content no taller than 792 px. Overflow happens most often on "full-width image + a row of cards" slides: shrink the image or shorten the title.
• In Claude Code it is simpler to build a PPTX with python-pptx and put notes.json into the notes directly.

Step 9. Verification and delivery

• By script: every slide from order is present, no extra ones, the longest note is no longer than 4000 characters, the links on the slides match the verified ones.
• Save to the user's computer: crops_*.json in pipeline/; config.json, overrides.json, ranges.json, notes.json in _work/; updated scripts in pipeline/.
• METHODOLOGY.md: add a section about the run — input, result with the link, new lessons.
• Reply to the user — briefly:
  • the presentation link;
  • how many slides and how they are organized;
  • where the transcript came from;
  • which links did not open or were not found;
  • number discrepancies and caveats.

Environment limits (Cowork)

• device_bash: about 180 s per call; background processes are killed; deleting in a connected folder only with permission.
• Moving to the cloud: about 50 s per call, up to 400 MB per file. Move images, not video.
• Viewing images is expensive (about 1800 tokens per high-resolution frame): use contact sheets and mosaics instead of individual frames.

If pipeline/ is unavailable

Recreate the scripts from this description:

• extract_slides.py:
  • ffmpeg -vf fps=1 -q:v 2;
  • frames are loaded at 1/4 scale in grayscale;
  • webcam mask — from how often pixels change in "quiet" frame pairs, threshold 0.10;
  • a pixel has changed if |Δ| > 20 gray levels;
  • a new segment starts when the share of changed pixels is ≥0.003 against the previous frame or ≥0.01 against the segment's first frame; minimum length 3 s;
  • the segment's representative is the sharpest frame (variance of the Laplacian) in its second half;
  • global deduplication when the difference is below 0.005;
  • contact sheets of 6×5 thumbnails, 320×200, with number and timecode.
• clean_transcript.py: input — sentences.json, config.json, overrides.json, ranges.json (see step 7); output — notes.json, consecutive utterances of one speaker merged.
• transcribe_assemblyai.py: POST /v2/upload → POST /v2/transcript (speech_model universal-2, language_code, speaker_labels) → poll GET /v2/transcript/{id} → GET /v2/transcript/{id}/sentences.
• mosaic.py, crop_images.py: OpenCV — 2×2 tiles of 960×540 with a frame label; cutting by coordinates with Lanczos scaling.