Show exactly which episodes the LLM would run on (0.32.0)

/pipeline gains a 'What the LLM would run' panel: for ad detection it lists
the actual seed episodes detect-ads would read (the campaign set-cover, the
fewest covering every unread campaign) with per-episode cost, and states
that the other pending episodes are caught free by fingerprinting once
these are confirmed. seeds --count persists that set to llm_ad_seeds so the
read-only web can show it. Updates docs/PLAN.md with the ad-pipeline
maturation and current status.

CHANGELOG.md

@@ -7,6 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.32.0] - 2026-07-24
+
+### Added
+
+- **`/pipeline` now shows exactly which episodes the LLM would run on** — a "What the LLM would
+  run" panel making the cost of turning on a key transparent up front. For ad detection it lists
+  the actual **seed episodes** `detect-ads` would read — the campaign set-cover, the fewest that
+  cover every unread ad campaign — each with the campaigns it covers and its estimated \$, and
+  states plainly that the *other* N pending episodes are caught for free by fingerprinting once
+  these are confirmed (so the model never reads them). Topic-extraction and comparison queues are
+  shown with a one-line note on what each reads (metadata vs full transcripts).
+- **`seeds --count` persists the campaign set-cover** to a new `llm_ad_seeds` table, so the
+  read-only web can display it — `select_seed_episodes` writes (via `ensure_schema`), so the web
+  can't recompute it on a page load.
+
+### Docs
+
+- `docs/PLAN.md` updated with an "Ad-stripping pipeline — matured (0.18–0.32)" section and a
+  current-status banner; the remaining work is not code — it's the `ANTHROPIC_API_KEY`.
+
 ## [0.31.0] - 2026-07-24
 
 ### Added

docs/PLAN.md

@@ -2,6 +2,11 @@
 
 Milestones. Each one ships something usable and gets a CHANGELOG version.
 
+> **Status (2026-07-24, v0.32.0, deployed):** M0–M4 done; the ad-stripping pipeline is mature and
+> fully autonomous (`hark pipeline`), with a `/pipeline` UI dashboard. The one thing gating the LLM
+> tiers (`detect-ads`, `extract`, `compare`) is an `ANTHROPIC_API_KEY` — everything downstream is
+> built, budget-gated, and dormant. See **"Ad-stripping pipeline — matured (0.18–0.32)"** below.
+
 ## M0 — scaffold + ingest (done, 0.1.0)
 
 - Project scaffold: uv/pyproject, src layout, pytest.
@@ -580,6 +585,47 @@ of this page"); this pass upgrades it rather than adding a new page.
   commands use) are still a deliberate follow-up — explicitly deferred again on request,
   not forgotten.
 
+## Ad-stripping pipeline — matured (0.18–0.32)
+
+The ad-stripping feature grew from "chapters + a manual LLM path" into a layered, autonomous,
+observable pipeline. Highlights, newest last:
+
+- **Layered tier taxonomy (adscrub).** Ad spans carry a `source`; hark treats them by role:
+  - **evidence** (`GROUND_TRUTH_SOURCES` = `llm`, `chapter`) — the only tiers trusted to seed the
+    fingerprint library or the text-repeat library.
+  - **inference** (`repeat`, `fpmatch`, `dai`, `recur`) — cheap, keyless, never seed the library.
+  - `FP_LIBRARY_SOURCES` = evidence + `dai` (DAI boundaries are structural, trustworthy enough to
+    fingerprint against); `CUT_SOURCES` = `chapter`/`llm`/`repeat`/`fpmatch` (what actually gets
+    cut). `recur` is discovery-only and deliberately NOT a pipeline stage.
+- **Acoustic fingerprinting (Chromaprint).** `fingerprint.py` matches our own corpus against a
+  library of CONFIRMED ad recordings — ~89% duration recall on Casefile, generalises cross-show,
+  zero tokens/transcript. Split into a bounded **index** step (fpcalc-once-and-cache) and a cheap
+  **match** step. A **streaming index** (`--stream`, 0.30/0.17.0) fingerprints un-downloaded
+  episodes by fetch-and-discard, so coverage reaches the whole ~27.8k corpus without storing audio.
+- **DAI probe** (`dai.py`): two independently-targeted fetches; where they diverge is provably
+  server-inserted. Escalates to a larger window to find a long ad's end (0.30). `dai-probe` skips
+  platforms proven non-DAI so probe budget lands where insertion actually happens.
+- **The pipeline is autonomous, tested Python** (`orchestrator.py`, `hark pipeline`, 0.26.0),
+  replacing a 1,755-char shell string. Per-stage cadence (every cycle / ~30 min) + gates; single
+  SQLite writer; crash-isolated stages. Per-stage **heartbeat** and **self log-rotation** (0.26.1).
+- **Two independent LLM budget pools (0.27.0):** `HARK_LLM_ADS_BUDGET` (detect-ads) and
+  `HARK_LLM_COMPARISONS_BUDGET` (extract, compare). Each dollars/day, default 0 = off; a key alone
+  never spends. `detect-ads` reads the **campaign set-cover** — the fewest episodes covering every
+  unread ad campaign — not all pending, so the LLM cost is bounded and the fingerprint tier
+  amplifies each confirmed campaign for free.
+- **Robustness:** dead-audio (404/410) quarantine so an expired URL can't head-of-line-block the
+  transcription queue (0.28.0); anomalous cuts (>35%) are HELD and the original is served (0.29.0);
+  exit-code hygiene (nothing-to-do ≠ failure); opt-in ntfy alerting on cycle/stage errors (0.29.0);
+  an inference-drift signal (`verify-inference`).
+- **`/pipeline` UI dashboard (0.31.0–0.32.0):** every stage's cadence/gate/last-run/status, ad-tier
+  roles, work queues, today's spend — and a **"what the LLM would run"** panel showing the exact
+  seed episodes `detect-ads` would read (campaign-deduped) with per-episode cost, so funding the
+  key holds no surprises.
+
+**What's left is not code:** provide an `ANTHROPIC_API_KEY` (+ fund whichever budget pools you
+want) and the LLM tiers auto-enable; optionally set `HARK_NTFY_URL` for alerts. The transcription
+and fingerprint backfills over the ~27.8k corpus are slow-but-flowing, bounded per cycle.
+
 ## Seed shows (feeds.txt)
 
 Start with well-known subject-per-episode shows across two genres, e.g.: Casefile,

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "hark"
-version = "0.31.0"
+version = "0.32.0"
 description = "Cross-podcast topic index and discovery service"
 readme = "README.md"
 requires-python = ">=3.12"

src/hark/__init__.py

@@ -1,3 +1,3 @@
 """hark — cross-podcast topic index and discovery service."""
 
-__version__ = "0.31.0"
+__version__ = "0.32.0"

src/hark/cli.py

@@ -709,11 +709,30 @@ def cmd_seeds(args: argparse.Namespace) -> int:
 
     if args.count:
         # Free visibility into the bootstrap gap: how many ad campaigns still lack a
-        # ground-truth confirmation, and how few episodes would retire them. No key, no reading,
-        # no writes — safe to run every pipeline cycle so the number never goes silently unseen.
+        # ground-truth confirmation, and how few episodes would retire them.
         seeds = (ad_fingerprint.select_seed_episodes(
             conn, episode_ids=sorted(e["id"] for e in all_pending)) if all_pending else [])
         campaigns = sum(covers for _, covers in seeds)
+        # Persist the seed set (episode + campaigns covered + estimated $) so the read-only web
+        # /pipeline can show EXACTLY which episodes detect-ads would read — and that it is these
+        # few, campaign-deduplicated, not the thousands merely "pending". select_seed_episodes
+        # writes via ensure_schema, so the web can't recompute this itself.
+        from datetime import datetime, timezone
+        now_iso = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+        by_id = {e["id"]: e for e in all_pending}
+        conn.execute(
+            "CREATE TABLE IF NOT EXISTS llm_ad_seeds (episode_id INTEGER PRIMARY KEY, title TEXT, "
+            "campaigns INTEGER, est_dollars REAL, unread_pending INTEGER, updated_at TEXT)")
+        conn.execute("DELETE FROM llm_ad_seeds")
+        for eid, covers in seeds:
+            ep = by_id.get(eid)
+            conn.execute(
+                "INSERT OR REPLACE INTO llm_ad_seeds "
+                "(episode_id, title, campaigns, est_dollars, unread_pending, updated_at) "
+                "VALUES (?,?,?,?,?,?)",
+                (eid, (ep["title"] if ep else None), covers,
+                 _estimate_episode_dollars(ep) if ep else 0.0, len(all_pending), now_iso))
+        conn.commit()
         print(f"{campaigns} unread ad campaign(s); {len(seeds)} seed episode(s) would confirm "
               f"them — run `hark detect-ads` (needs a key + ads budget), or `hark seeds --out "
               f"FILE` to read them in a session")

src/hark/queries.py

@@ -186,6 +186,15 @@ def pipeline_status(conn: sqlite3.Connection) -> dict:
             "SELECT COUNT(*) FROM episodes WHERE cut_held_at IS NOT NULL").fetchone()[0]
     except sqlite3.OperationalError:
         pass
+    # Exactly which episodes detect-ads would read (campaign set-cover), persisted by the
+    # `seeds --count` stage — so the read-only web can show it (select_seed_episodes writes).
+    out["ad_seeds"] = []
+    try:
+        out["ad_seeds"] = conn.execute(
+            "SELECT episode_id, title, campaigns, est_dollars, unread_pending FROM llm_ad_seeds "
+            "ORDER BY campaigns DESC, episode_id").fetchall()
+    except sqlite3.OperationalError:
+        pass
     return out
 
 

src/hark/views.py

@@ -337,6 +337,9 @@ class App:
                 "SELECT COUNT(*) FROM episodes WHERE cut_path IS NULL AND cut_held_at IS NULL "
                 "AND EXISTS (SELECT 1 FROM ad_segments WHERE episode_id=episodes.id)"
             ).fetchone()[0]
+            extract_pending = conn.execute(
+                "SELECT COUNT(*) FROM episodes e JOIN shows s ON s.id = e.show_id "
+                "WHERE e.extracted_at IS NULL AND s.topic_index_enabled = 1").fetchone()[0]
             compare_pending = claims.count_pending_topics(conn)
         finally:
             conn.close()
@@ -403,6 +406,40 @@ class App:
         tiers_table = ("<table><tr><th>tier</th><th class='num'>spans</th><th>role</th></tr>"
                        + tier_rows + "</table>")
 
+        # --- What the LLM would actually run on (so funding the key holds no surprises) ---
+        ad_seeds = st["ad_seeds"]
+        unread_pending = ad_seeds[0]["unread_pending"] if ad_seeds else 0
+        seed_est = sum((s["est_dollars"] or 0.0) for s in ad_seeds)
+        if ad_seeds:
+            seed_rows = "".join(
+                f"<tr><td><a href='/episode/{s['episode_id']}'>{esc(s['title'])}</a></td>"
+                f"<td class='num'>{s['campaigns']}</td>"
+                f"<td class='num dim'>${(s['est_dollars'] or 0.0):.3f}</td></tr>" for s in ad_seeds)
+            free_left = max(0, unread_pending - len(ad_seeds))
+            ads_llm = (
+                '<div class="status active"><p><strong>Ad detection</strong> (ads pool): the model '
+                f'would read only <span class="pending">{plural(len(ad_seeds), "seed episode")}</span> '
+                '— the fewest that cover <em>every</em> unread ad campaign — for an estimated '
+                f'<span class="pending">${seed_est:.2f}</span>. The other {free_left} pending '
+                'episode(s) are caught for FREE by fingerprinting once these are confirmed, so the '
+                'model never reads them.</p></div>'
+                '<table><tr><th>episode detect-ads would read next</th>'
+                '<th class="num">campaigns</th><th class="num">est.&nbsp;$</th></tr>'
+                f'{seed_rows}</table>')
+        else:
+            ads_llm = ('<div class="status"><p><strong>Ad detection</strong> (ads pool): no unread ad '
+                       'campaigns to read right now — the free tiers have it covered, or too little '
+                       'audio is fingerprinted yet to find any.</p></div>')
+        other_llm = (
+            '<div class="status"><p><strong>Topic extraction</strong> (comparisons pool): '
+            f'{plural(extract_pending, "episode")} pending — the model reads each episode\'s title '
+            'and description (cheap, metadata only).</p>'
+            '<p><strong>Cross-show comparison</strong> (comparisons pool): '
+            f'{plural(compare_pending, "topic")} ready — the model reads the full transcripts of the '
+            '2+ shows on a shared topic (the priciest LLM call).</p></div>')
+        llm_section = ("<h2>What the LLM would run (once a key + budget is set)</h2>"
+                       + ads_llm + other_llm)
+
         queues = (
             "<h2>Work queues</h2><table>"
             f"<tr><td>transcription</td><td class='num'>{transcribe_pending}</td></tr>"
@@ -412,7 +449,7 @@ class App:
             "</table>")
 
         body = (
-            "<h1>Pipeline</h1>" + cards + budgets
+            "<h1>Pipeline</h1>" + cards + budgets + llm_section
             + "<h2>Stages</h2>" + stage_table + legend
             + "<h2>Ad-detection tiers</h2>" + tiers_table
             + queues

tests/test_cli.py

@@ -1248,6 +1248,21 @@ def test_seeds_emits_only_the_selected_episodes(tmp_path, capsys, monkeypatch):
         assert f"[{i}] " in out, "every segment should be offered to the reader"
 
 
+def test_seeds_count_persists_the_llm_seed_set_for_the_ui(tmp_path, monkeypatch):
+    """`seeds --count` persists the campaign set-cover to llm_ad_seeds so the read-only web can
+    show EXACTLY which episodes detect-ads would read (and that it's these few, not all pending)."""
+    path = _seeds_corpus(tmp_path)
+    monkeypatch.setattr(cli.ad_fingerprint, "select_seed_episodes", lambda conn, **kw: [(1, 3)])
+    assert cli.main(["--db", str(path), "seeds", "--count"]) == 0
+    conn = db.connect(path)
+    rows = conn.execute(
+        "SELECT episode_id, title, campaigns, unread_pending FROM llm_ad_seeds").fetchall()
+    assert len(rows) == 1
+    assert rows[0]["episode_id"] == 1 and rows[0]["campaigns"] == 3
+    assert rows[0]["title"] == "Ep One"
+    assert rows[0]["unread_pending"] == 1     # 1 detect-pending episode, deduped to this 1 seed
+
+
 def test_seeds_omits_regions_a_cheaper_tier_already_covered(tmp_path, capsys, monkeypatch):
     """Same reasoning as the API path: nobody should be paid to re-read covered regions."""
     path = _seeds_corpus(tmp_path)

tests/test_web.py

@@ -125,6 +125,35 @@ def test_pipeline_page_lists_every_stage(server):
         assert meta["name"] in body, meta["name"]
     assert "never run" in body                       # no pipeline_runs yet in this fixture
     assert "No ad spans found yet" in body           # no ad_segments yet — guarded, not a 500
+    assert "What the LLM would run" in body          # the funding-transparency section is present
+
+
+def test_pipeline_page_shows_which_episodes_the_llm_would_read(tmp_path):
+    conn = db.connect(tmp_path / "hark.db")
+    conn.execute("INSERT INTO shows (query, title, feed_url) VALUES ('q', 'Show A', 'http://x')")
+    conn.execute("INSERT INTO episodes (show_id, guid, title, audio_url, transcript_path) "
+                 "VALUES (1, 'g1', 'Seed Ep', 'http://a/1.mp3', 't.json')")
+    # what the seeds --count stage persists: 1 seed covering 4 campaigns, out of 30 unread pending
+    conn.execute("CREATE TABLE llm_ad_seeds (episode_id INTEGER PRIMARY KEY, title TEXT, "
+                 "campaigns INTEGER, est_dollars REAL, unread_pending INTEGER, updated_at TEXT)")
+    conn.execute("INSERT INTO llm_ad_seeds VALUES (1, 'Seed Ep', 4, 0.12, 30, '2026-01-01T00:00:00Z')")
+    conn.commit()
+    conn.close()
+    srv = web.make_server(tmp_path / "hark.db", tmp_path / "auth.db",
+                          bind="127.0.0.1:0", admin_token="t")
+    thread = threading.Thread(target=srv.serve_forever, daemon=True)
+    thread.start()
+    try:
+        resp, _ = request(srv, "POST", "/login", body={"username": "admin", "password": "t"})
+        cookie = resp.getheader("Set-Cookie").split(";")[0]
+        resp, body = request(srv, "GET", "/pipeline", cookie=cookie)
+        assert resp.status == 200
+        assert "Seed Ep" in body                       # the actual episode detect-ads would read
+        assert "1 seed episode" in body                # campaign-deduped, not "30"
+        assert "caught for FREE" in body               # the other 29 are free via fingerprinting
+        assert "$0.12" in body                         # estimated spend, shown before you fund it
+    finally:
+        srv.shutdown()
 
 
 def test_login_with_admin_token_and_browse(server):

uv.lock

@@ -242,7 +242,7 @@ wheels = [
 
 [[package]]
 name = "hark"
-version = "0.31.0"
+version = "0.32.0"
 source = { editable = "." }
 dependencies = [
     { name = "adscrub" },