.claude/skills/video-visual-production/references/runbook.md

# Runbook: make a video beat, step by step

Read this top to bottom once, then follow the steps in order. Every command is run from the **repo root** (`ecosystem-generic/`) in Git Bash or PowerShell. Use forward slashes in paths. Do not skip the checks: they are what tell you the step worked, because you cannot watch the video or hear it.

**What you are making.** A "beat" is one section of the script (B1 is "The city he left", about 100 s). The output is a 1080p MP4 with the voice-over, on-screen evidence and graphics tied to spoken words, small sound effects, and a quiet bed. The worked example is beat B1 of Moser Baer v7 Hinglish. Its beat sheet is `research/the-fall-of-moser-baer/video/beat-sheet-B1.json`. When unsure, open it and copy its shape.

**The one idea.** You never draw anything. You write a **beat sheet** (a JSON file: one scene per sentence, each scene says its type, its words, its text, which spoken word triggers each build, which sounds). Scripts then check it, fetch assets, compute frame times, render, mix and check the result. If you want a different video, you change the beat sheet, never the output.

## 0. Before you start (once per machine)

1. `node -v` must print v22 or higher. `ffmpeg -version` must work and `ffmpeg -encoders` must list `libx264`.
2. Install the one dependency: `cd .claude/skills/video-visual-production/render && npm install` then go back to the repo root. If `node_modules/@napi-rs` already exists there, skip this.
3. Chrome or Edge must be installed (the tool that captures article passages uses it).
4. Heavy output goes to `C:/video-build/<slug>/<beat>/` (outside OneDrive). Do not change this. Tracked files go in `research/<topic>/video/`.

## 1. Collect the inputs for the beat

You need three things. If any is missing, stop and fix it first.

| Input | Where it is for Moser Baer v7 Hinglish | Test |
|---|---|---|
| Voice-over WAV for the beat | `research/the-fall-of-moser-baer/script/audio/gemini-v7-hinglish/beats/B<n>-<slug>.wav` | `ffprobe -v error -show_entries format=duration -of default=nw=1:nk=1 <file>` prints seconds |
| Aligned word times (every script word with a start and end second) | `research/the-fall-of-moser-baer/video/timing/words.json` | `node .claude/skills/video-visual-production/scripts/edit.js words 0 20 --sheet <any beat sheet>` prints words with times. **Only B1 and B2 are aligned today.** |
| Fact ledger rows for what the beat shows | `research/the-fall-of-moser-baer/script/SCRIPT-v7.md`, section 5 | each row has a status of `[V]`, `[V-search]`, `derived`, `analysis` or `[CHECK]` |

**If the beat is not aligned (tested on B0, 90% of words measured, about 5 minutes):**
1. Transcribe with the `groq-transcribe` skill, with a Latin-script prompt taken from a **different** beat (a prompt from the same beat makes the match circular): `node "<home>/.claude/skills/groq-transcribe/scripts/transcribe.mjs" <beat.wav> C:/video-build/<slug>/<beat>/align/<beat>.groq.json --lang hi --prompt-file <prompt.txt>`. This sends the beat's audio to Groq's hosted Whisper (the user installed and asked for this skill; if the user has not, ask first). Never print the key.
2. Build `script.txt` from the words that were sent to the voice (`video/timing/reference-words.json`): one line per paragraph, `[P1] word word ...`; take paragraph breaks from `para_end`. A 10-line node script does it: for the beat's words join the `text` fields and start a new line after each word with `para_end: true`.
3. Align with the `script-audio-align` skill: `node "<home>/.claude/skills/script-audio-align/scripts/align.mjs" script.txt <beat>.groq.json <outDir>`. It prints `Aligned N/M script words (X%)`. Below about 85% something is wrong (wrong audio, wrong text, or a bad transcript): stop and tell the user.
4. Convert: `node .claude/skills/video-visual-production/scripts/words-from-alignment.js --beat B0 --alignment <outDir>/alignment.json --reference research/the-fall-of-moser-baer/video/timing/reference-words.json --out research/the-fall-of-moser-baer/video/timing/words-B0.json`. Use that file as `words` in the beat sheet with `"time_offset_s": 0` (it is local to the beat). The older `words.json` (B1, B2) uses a global timeline and needs the offset the scaffold writes.
5. Interpolated words are estimates; the lint lists every cue that sits on one. Before trusting a cue on an interpolated word, look at its neighbours with `edit.js words`.
Shortcut: `node .claude/skills/video-visual-production/scripts/align-beat.js --beat B3` does steps 1 to 4 in one command (and `brief-beat.js --beat B3` writes the brief). It tries whisper-large-v3-turbo on the whole beat, then whisper-large-v3, then the beat cut into about 30 s chunks at silences, and keeps the best; it stops under 85% matched. Measured so far: B0 90.2%, B2 90.6%, B4 91.9%, B5 90.5% on the first attempt; B3 (a 100 s beat with an English quote and many numbers) was 63.5% and 77.1% on the whole-beat attempts and 87.0% chunked, because Whisper dropped a stretch of 85 words in the long pass. The B3 WAV also has odd timestamps (ffmpeg cannot seek into it with an input-side `-ss`), so cutting is done by decoding first.

The older local alternative is the Whisper base chain in `research/the-fall-of-moser-baer/video/PRODUCTION-PLAN.md` section 1 (6 GB RAM, one job at a time); it produced B1 and B2.

**Honesty rule for facts.** Only ledger rows with status `V`, `V-1source` (from `[V]` with one source), `derived` or `analysis` may be drawn. A row marked `[CHECK]` or `[V-search]` must not appear on screen. If a beat needs one, tell the user and leave that visual out (use a plain typographic scene).

## 2. Scaffold the beat sheet

```
node .claude/skills/video-visual-production/scripts/new-beat.js --beat B4 --title "The treadmill" --slug moser-baer-v7-hinglish --audio research/the-fall-of-moser-baer/script/audio/gemini-v7-hinglish/beats/B4-the-treadmill.wav --words research/the-fall-of-moser-baer/video/timing/words.json --dir research/the-fall-of-moser-baer/video
```

Expected: `N scenes scaffolded -> .../beat-sheet-B4.json`. This makes one plain typographic scene per sentence so the file is valid from the start. It refuses to overwrite an existing sheet. Also creates `ledger-B4.json` (empty). `words.json` lays all beats end to end on one timeline, so the scaffold writes `"time_offset_s"` (where this beat starts there) into the sheet and every tool subtracts it; do not change it. A beat sheet with scene `span`s that look huge (a scene 100 s long) means this offset is missing.

## 3. Author the scenes (the creative step)

Work **one paragraph at a time**. For each sentence, read what the voice says (`edit.js words <from> <to> --sheet <sheet>` prints the word indexes and times), then pick the scene type that makes the viewer **see what the sentence claims**. Decision table:

| What the sentence does | Scene type | Honesty class |
|---|---|---|
| States a year or date as the point | `date` (or a number on a `timeline`) | T or D |
| Introduces a person for the first time | `cast` (name card with a role; tag the person where they arrive and where they return) | M |
| Quotes or paraphrases a published source | `doc` (captured passage, highlight the quoted words as they are spoken) | E |
| Lists steps in time (jobs, events) | `timeline` | D |
| A picture of a real thing that is free to use | `photo` | E |
| Moves from one place to another | `route` | M |
| Names an idea that will come back (a pivot, a loop) | `ladder` or a new diagram scene, reuse it every time the idea returns | M |
| Joining or buying | `merge` | M |
| A limit or ceiling | `ceiling` | M |
| Speaks to the viewer ("if you have ever...") | `crowd` | M |
| A plain statement worth reading | `slam` | T |
| Anything where you cannot find an honest picture | `slam` with the key phrase. **Never invent a picture.** | T |

Rules you must not break (the lint enforces most of them):
- Every scene has a `class`: E evidence, D data from the ledger, M diagram, T typography.
- Every number on screen is a number the voice says in that scene (or recalls one said earlier in the beat, marked `recall_from`). Every numeral the voice says is shown.
- Every evidence scene (`doc`, `photo`) has an on-screen source line and an asset record.
- No broadcaster news video, no company logo, no photo of a private person, no stock footage, no AI-generated photo of a real place. Article screenshots and typeset text are allowed (the user decided this on 2026-11-01); one short passage per scene.
- A typeset card must never be styled like the newspaper's own page.
- Do not show anything the voice-over does not say, except a label that names what is being shown.
- Words on screen stay at least `max(1.2 s, 0.35 s per word)`. If a scene is too short for its text, move the cue to an earlier word, shorten the text, or merge with the neighbour scene.
- Pace: a typical scene is 4 to 7 s. Do not leave 8 s without something changing.

How to fill a scene: copy the matching example from `references/scene-catalogue.md`, change the props, and set **cues** to word indexes. A cue is "this build starts on this spoken word". Find the word with `edit.js words`. Put numbers and names on the word that says them.

**Spans must tile the beat:** scene 1 starts at word 0, each next scene starts at the previous scene's last word plus 1, the last scene ends at the beat's last word. A scene may start a few frames late with `"start_shift_frames": 12` (used when a text needs more hold time).

**Sounds:** add `sfx` entries only where a visual event happens (a number appears: `tick`; a stamp or date: `stamp`; a highlight starts: `swipe`; a place changes: `whoosh`; a named idea lands: `pivot`). Aim for 5 to 8 a minute; hard limit 12. Each needs `role` and `event`. No sound where nothing happens on screen.

**Assets:** for a `doc` scene add a capture request in `assets`: the article URL, the first words and last words of the passage (`anchor`), and the exact phrases to highlight (`highlights`). The phrases must appear in the page text exactly as written (straight or curly quotes do not matter). To find the exact wording: `curl -s -A "Mozilla/5.0" <url> > /tmp/page.html` then search it, or run the capture and read the error. For a `photo` use a Wikimedia Commons file: `node research/video-production/probes/probe-commons.js "search words"` lists files with licences; only CC0, public domain, CC BY, CC BY-SA, GODL are accepted. Add the credit line to the scene.

## 4. Lint

```
node .claude/skills/video-visual-production/scripts/beatsheet-lint.js --sheet research/the-fall-of-moser-baer/video/beat-sheet-B4.json
```

Output starts with `Hard gates failing: N`. **N must be 0.** Read each `- **H8** ...` line: it names the scene and the problem. Fix the beat sheet (never the lint rules) and run again. Warnings (`W`) are allowed, but read them and fix the cheap ones. See `references/troubleshooting.md` for what each rule id means. Run it with `--damage` once at the end: it must print `all 15 mutations caught`.

## 5. Build a preview, then the real thing

```
node .claude/skills/video-visual-production/scripts/build-beat.js --sheet research/the-fall-of-moser-baer/video/beat-sheet-B4.json --preview
```
Preview is 960x540, skips the check and takes about a third of the time. Use it to catch broken scenes. Then:
```
node .claude/skills/video-visual-production/scripts/build-beat.js --sheet research/the-fall-of-moser-baer/video/beat-sheet-B4.json
```
It prints `== <stage>` for each stage and ends with `BUILD PASSED (...)` or `BUILD FAILED: <stage> failed`. A full beat takes about 4 minutes (about 2.5 of them rendering). Run it in the background if your tool times out. On `BUILD FAILED`, read the lines above it: the first error is the cause.

To look at one scene without any video: `node .claude/skills/video-visual-production/scripts/preview-scene.js --sheet <sheet> --scene B4.03` prints the path of a PNG. Open it and look.

## 6. Look at what you built (do not skip)

1. Open `research/the-fall-of-moser-baer/video/STATUS-video-B4.md`. Every row should say `pass` (rows marked `warn` are allowed). The verdict line must say `HARD GATES PASS, BAR NOT YET MET`. If it says `HARD GATES FAILING`, fix the rows marked `**FAIL**`.
2. Open the contact sheet `C:/video-build/<slug>/<beat>/contact-sheet.png` with the image viewer tool and check, scene by scene: text is not cut off, nothing overlaps, the right phrase is highlighted, the scene matches what the voice says in that scene (read `REVIEW-B4.md`, which lists the words of each scene next to what is on screen).
3. Open `spectrogram.png` only if the status shows an audio failure.
4. You cannot hear or watch it. **Do not say it is good.** Say what was measured, what you saw in stills, and what only the user can judge (motion feel, sound taste, whether the bed suits the story).

## 7. Hand over

Tell the user, in plain words and without hype: the path of `B4.mp4`, the contact sheet and `REVIEW-B4.md`; which hard gates pass; the warnings that matter (interpolated cues, any typeset fallback, anything left plain because no honest visual existed); and that the review sheet lists scene ids so they can send notes like `B4.06 at 40 s: the highlight comes late`. Then follow `references/edit-guide.md` for every note.

## 8. Leave the project in order

- If you added or changed a script or skill: add a row to `.claude/skills/video-script-production/references/tools.md` and a line to `history.md` (a standing rule of this project).
- Update the memory note `project_video-production-phase` with the state (which beats are built).
- **Do not commit, push or deploy unless the user asks.** Do not spawn subagents unless asked.

## Quick reference

| Task | Command |
|---|---|
| Find the word index for a cue | `node .claude/skills/video-visual-production/scripts/edit.js words 100 140 --sheet <sheet>` |
| Lint | `node .../scripts/beatsheet-lint.js --sheet <sheet>` |
| Preview build | `node .../scripts/build-beat.js --sheet <sheet> --preview` |
| Full build | `node .../scripts/build-beat.js --sheet <sheet>` |
| See one scene | `node .../scripts/preview-scene.js --sheet <sheet> --scene B4.03` |
| Change something | `node .../scripts/edit.js set|cue|level|sfx-remove|undo ...` (see `edit-guide.md`) |
| Re-fetch assets | add `--refresh-assets` to the build |
| Run the whole B1 proof | `node .../scripts/smoke-b1.js` |

## Stitching all beats into one video (after every beat passes its own check)

1. Every beat B0 to B13 must have `STATUS-video-B#.md` saying `HARD GATES PASS` (`build-many.js --beats ...` rebuilds failing ones; unchanged scenes come from the cache).
2. `node .claude/skills/video-visual-production/scripts/stitch.js` (all 14 beats; `--beats B2,B3,B4` for a test into `FINAL-test/`). About 30 to 40 minutes the first time on this machine (the video pieces encode once; a rerun reuses unchanged pieces and cached score).
3. The joins and the music are set in `research/<topic>/video/score-spec.json`: `transitions` (`"B2>B3": "dip"`, `dissolve`, `cut`), `global` (`under_voice_db`, `duck_db`, `crossfade_s`, `outro_s`) and one entry per beat (`root`, `chords`, `bright`, `sub_db`, `air_db`, `pulse`, `arc`, `level_db`). Change a line and rerun; only the join pieces and the changed beat's score are rebuilt.
4. Outputs in `C:/video-build/<slug>/FINAL/`: `final.mp4`, `final-mix.wav`, `final-score-only.wav`, `final.srt`, `chapters.txt`, `seams.png`, `stitch-plan.json`; the report is `research/<topic>/video/STITCH-REPORT.md`. It measures frames, drift, loudness, true peak and undeclared black. It cannot say the music suits the video or that a join feels right: those are for the user.
5. Never fix a stitch problem by editing a beat's output; fix the beat sheet (`edit.js`), rebuild the beat, rerun `stitch.js`.