README-SERIES-ANALYSIS.md

# Series analysis

Takes a series idea (`series/<id>.json`), finds YouTube videos with the same or adjacent packaging, fetches video and channel metadata, and ranks which videos are worth learning from. Own database (`data/series-analysis.db`, override with `SERIES_ANALYSIS_DB`), keyed by series id, so it never touches the ecosystem databases. It only reads your existing research DB (`researchDb` in the spec) to add candidates.

```
npm run series:all                                   # plan, search, mine, enrich, rank, report for why-x-died
npm run series -- <command> --series=why-x-died      # run one stage
```

| Command | What it does | Useful options |
|---|---|---|
| `plan` | Builds search queries in six tiers: exact, exact-hinglish, adjacent-packaging, generic, adjacent-topic, reference-channel | `--max-queries=N`, `--llm` (adds LLM-proposed queries) |
| `search` | Runs pending queries, stores videos and which query/position found them | `--limit=N`, `--results=N`, `--retry` |
| `mine` | Adds on-series videos already in your research DB | `--research-db=PATH`, `--min-views=N` |
| `enrich` | Full video metadata (views, likes, exact date, duration, description), channel metadata, and each channel's recent-uploads baseline | `--videos=N`, `--channels=N`, `--basic` (faster, no likes/exact date), `--retry` |
| `judge` | Optional LLM fit check on the shortlist (uses `src/llm.js`) | `--top=N` |
| `rank` | Scores everything, picks the learn-from set | `--top=N` |
| `transcripts` | Saves transcripts of the learn-from set to `transcripts/series-analysis/<id>/` in the VIDEO / PERFORMANCE / CHANNEL / TRANSCRIPT layout | `--top=N` |
| `cohorts` | Picks the study cohorts (new vs established, same rule, set in the spec's `cohorts` block) and saves a transcript for each into `transcripts/series-analysis/<id>/cohort-<name>/`. Transcripts must pass capture checks (coverage, words/min, subscribe-spam, caption loops) or the next candidate is used. These folders feed the `script-cohort-analysis` skill (`--metric performance`). | `--size=N` |
| `report` | Writes `reports/series-analysis/<id>-<run>.md` and `.json` | |

Every stage is resumable: it skips work already stored. Add `--verbose` to see the shared YouTube helpers' logging.

## How the ranking works

`learn_score = 0.35 fit + 0.35 performance + 0.30 learnability` (weights in the spec).

- **fit**: failure vocabulary in the title, structure (`Why X died`, `The fall of X`, `rise and fall`, Hinglish forms), a named company, and how many queries surfaced it. Averaged with the LLM judgement if `judge` was run. Below `minFitToRank` a video is excluded.
- **performance**: how far the video beats its own channel's typical upload (median of recent uploads older than 30 days, log-scaled), views per day with age floored at 30 days (percentile within the on-series pool), and like ratio. Missing parts are re-weighted, not zeroed.
- **learnability**: length in the 8 to 35 minute sweet spot, channel size you could plausibly match (10k to 1.5M), recency, and a heavy penalty for formats whose views come from elsewhere (podcasts, interviews, fiction clips, shorts, reaction, news).
- **selection**: greedy top N with at most 2 per channel and 2 per company.

The report also lists breakouts (small channels at 3x+ their norm), reference giants (3M+ subscribers), and a contrast set (same packaging, weak result).

## Known limits

- Fit is read from titles only; thumbnails and scripts are not seen yet. A title like "Why the World's Largest Megaproject Collapsed" scores as on-series even though it is not a company. `judge` (or the later skill pass) is what catches that.
- The company list in the spec is a starting point; edit it freely.
- Views are lifetime totals. Channel baselines come from at most the latest `channelBaselineVideos` uploads.
- Full video info costs about 5 s per video, so the default 150 videos take roughly 12 minutes.

## Viewer

`npm run series:viewer` serves http://localhost:3100 (set `SERIES_VIEWER_PORT` to change it).

- `/` lists the series in `series/*.json`.
- `/s/<series>` has a Videos tab (search, sort, filters, click a channel to filter) and a Channels tab.
- `/s/<series>/v/<videoId>` shows the thumbnail, stats, description, transcript and teardown. With no teardown, the "Generate teardown" button fetches the transcript if missing, runs a Sonnet analyst through the `claude` CLI (needs it on PATH), validates the quotes, and shows the result. It takes about 3 minutes. Output goes to `transcripts/series-analysis/<series>/teardown/single/<videoId>/`.