fix: cap fetch_all_assets at 5000 items; filter non-dict page entries

Fetching up to MAX_PAGES*page_size (1M) assets before the 3000-item
diversity pool cap was applied could exhaust memory on large Immich
libraries. Early-exit once 5000 items are collected — the pool cap
of 3000 makes anything beyond that wasteful. Also filter null/non-dict
items from page responses at fetch time.

CHANGELOG.md

@@ -7,31 +7,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
-## [0.4.9] - 2026-06-14
-
-### Changed
-
-- **`FRIGATE_SCORE_CEILING` is now dynamic by default**: previously defaulted to `0` (disabled). Now unset (default) enables a self-calibrating novelty gate — below-cap candidates are skipped if their pre-upload Frigate score exceeds the most-redundant tracked file's score. This catches conditions already covered by manually-added Frigate images that winnow cannot track. Set `FRIGATE_SCORE_CEILING=0` to disable entirely; set a positive value (e.g. `0.85`) for a fixed hard ceiling.
-- **Quality replacement branches consolidated**: the Frigate-score and blur-score replacement paths in the upload loop shared identical structure. Merged into a single code path parameterised by score source and comparison direction.
-- **`MIN_FACE_COUNT` default raised from `0` to `3`**: people with fewer than 3 tagged photos produce degenerate training sets; skipping them by default avoids noisy runs.
-- **`STRATEGY=adaptive`** is the new primary name for embedding-based diversity selection; `auto` remains a silent alias for backwards compatibility.
-- **`MERGE_DUPLICATE_PEOPLE` and `TRACE_CROP_SIZE`** added to the README env var table (were in the codebase but undocumented).
-- **CUDA version corrected** in the image tags table (was 13.3, actual base image is 12.8.1).
-
-## [0.4.8] - 2026-06-14
-
-### Changed
-
-- **Tracker write-through cache**: `upload_tracker` now keeps an in-memory copy of each JSON file keyed by its resolved path. All reads after the first hit the cache instead of disk; writes go to both disk and cache atomically. Cuts per-person disk I/O in the upload loop from ~90 reads to ~1, with no API or behaviour changes.
-
-## [0.4.7] - 2026-06-14
-
-### Changed
-
-- **`_dedup_embeddings` pre-allocated buffer**: replaced the grow-on-keep `np.vstack` pattern with a pre-allocated `(Q, D)` buffer filled row-by-row. Eliminates O(K²) copy work and the GC pressure from K intermediate heap allocations while keeping identical arithmetic for the similarity checks.
-- **`_kmedoids` cost computation vectorized**: the Python-level `sum(dist_matrix[i, medoids[labels[i]]] for i in range(n))` generator (called once per swap evaluation) is replaced with `dist_matrix[np.arange(n), np.array(medoids)[labels]].sum()` — a single numpy fancy-index + reduction, ~20–50× faster in the swap loop.
-- **`_reconcile_frigate_mappings` single-write batch**: previously called `record_frigate_file` once per uploaded file, each doing a full JSON load + save (O(L) disk round-trips per person). Now builds the full `{frigate_filename: asset_id}` mapping dict and writes it in one `record_frigate_files_batch` call (O(1) disk round-trip).
-
 ## [0.4.6] - 2026-06-14
 
 ### Fixed

README.md

@@ -43,34 +43,26 @@ Immich library
    • Objects → SigLIP (Vision Transformer)         → 768-dim vector
       │
       ▼
-5. Near-duplicate removal — greedy cosine-distance pass drops burst shots
+5. Diversity selection
-   and near-identical photos before clustering runs; the highest-quality
-   image from each near-duplicate group is kept
-      │
-      ▼
-6. Diversity selection
    • K-Medoids clustering → one representative per natural group
    • Farthest Point Sampling → fill remaining slots with maximally spread picks
-   • Hard example weighting — low-confidence detections get a distance boost
+   • Hard example weighting — unusual angles and low-confidence detections
-     so unusual angles and harder looks are preferred over easy frontals
+     are biased toward selection, since those are where models tend to fail
-   • Adaptive mode: stops when the next candidate is too similar to those already
+   • Auto mode: stops when similarity to the existing set exceeds a threshold
-     selected (distance threshold = 20 % of median pairwise distance for
+     (20 % of median pairwise distance for faces, 10 % for objects)
-     faces, 10 % for objects)
       │
       ▼
-7. Download full-resolution originals from Immich
+6. Download full-resolution originals from Immich
       │
       ▼
-8. Crop and process
+7. Crop and process
    • Face mode: EXIF-corrected, landmark-aligned 112×112 crop (ArcFace format)
    • Object mode: YOLOv9c detection → one crop per matched instance
       │
       ▼
-9. Deliver
+8. Deliver
    • Face mode: upload crops to Frigate's face registration API
-     ↳ below MAX_AUTO_IMAGES — upload, unless the novelty gate
+     ↳ below MAX_AUTO_IMAGES — upload freely
-       (FRIGATE_SCORE_CEILING) determines the candidate is already
-       covered by the current training set
      ↳ at cap + QUALITY_REPLACEMENT=true — with Frigate scoring active,
        swap the most redundant tracked image (highest pre-upload recognize
        score) if the candidate is more novel (lower score); falling back to
@@ -80,7 +72,7 @@ Immich library
    • Object mode: save crops to disk → place into your Frigate data directory
 ```
 
-Uploaded and rejected asset IDs are persisted across runs. The same image is never processed twice; rejected assets are permanently skipped unless `RETRY_REJECTED=true`.
+Uploaded and rejected asset IDs are persisted across runs. The same image is never processed twice; Frigate rejections are permanently skipped unless `RETRY_REJECTED=true`.
 
 ---
 
@@ -98,7 +90,7 @@ Uploaded and rejected asset IDs are persisted across runs. The same image is nev
 
 | Tag | Arch | Acceleration |
 | :-- | :-- | :-- |
-| `:latest` | amd64 + arm64 | NVIDIA CUDA 12.8 (amd64) · requires [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) |
+| `:latest` | amd64 + arm64 | NVIDIA CUDA 13.3 (amd64) · requires [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) |
 | `:rocm` | amd64 | AMD ROCm · pass `/dev/kfd` + `/dev/dri` |
 | `:intel` | amd64 | Intel Arc / iGPU via OpenVINO · pass `/dev/dri`, set `OPENVINO_DEVICE=GPU` |
 | `:cpu` | amd64 + arm64 | CPU only · ~2 GB smaller · no GPU required |
@@ -181,10 +173,10 @@ In scheduled mode the process (and loaded models) stays resident between runs. T
 | Variable | Default | Description |
 | :--- | :--- | :--- |
 | `TRAINING_MODE` | `face` | `face` — upload crops to Frigate; `object` — save crops to disk |
-| `STRATEGY` | `adaptive` | `adaptive` — embedding-based diversity selection, stops when candidates become redundant; `standard` — fixed 30 images; `broad` — fixed 100 images |
+| `STRATEGY` | `auto` | `auto` (embedding-based adaptive), `standard` (30 images), `broad` (100 images) |
 | `LIMIT` | *(unset)* | Exact image count — overrides `STRATEGY` |
 | `OBJECT_CLASS` | `dog` | Target class for object mode (any YOLO class: `dog`, `cat`, `car`, etc.) |
-| `AUTO_MODE` | *(auto)* | Skip interactive prompts and process all people unattended — auto-detected when no TTY is present (Docker, cron); set `true` to force in a terminal |
+| `AUTO_MODE` | *(auto)* | Force non-interactive mode in a terminal; auto-detected otherwise |
 | `VERBOSE` | `false` | Enable DEBUG-level console output (log file is always DEBUG) |
 
 ### People Filtering
@@ -193,31 +185,23 @@ In scheduled mode the process (and loaded models) stays resident between runs. T
 | :--- | :--- | :--- |
 | `ONLY_PEOPLE` | *(unset)* | Comma-separated whitelist — process only these people |
 | `SKIP_PEOPLE` | *(unset)* | Comma-separated list — skip these people |
-| `MIN_FACE_COUNT` | `3` | Skip people with fewer than N tagged assets in Immich |
+| `MIN_FACE_COUNT` | `0` | Skip people with fewer than N tagged assets in Immich |
-| `MERGE_DUPLICATE_PEOPLE` | `false` | When Immich has duplicate entries for the same person (same face split across multiple names), merge their asset pools before processing. Without this, each duplicate group emits a warning and is skipped |
 | `YEARS_FILTER` | `10` | Ignore images older than N years |
 
 ### Image Quality
 
 | Variable | Default | Description |
 | :--- | :--- | :--- |
-| `MAX_AUTO_IMAGES` | `80` | Maximum training images per person in Frigate |
-| `QUALITY_REPLACEMENT` | `true` | When at cap, swap a weaker tracked image for a better candidate. With Frigate scoring active, targets the most redundant image (highest pre-upload recognize score); otherwise uses blur score. Never touches manually added Frigate files. Set `false` to skip people at cap |
-
-#### Advanced Tuning *(calibrated — do not adjust)*
-
-These defaults are tuned for Frigate's ArcFace requirements. winnow will warn on launch if any are set. Image quality issues caused by non-default values will not be investigated.
-
-| Variable | Default | Description |
-| :--- | :--- | :--- |
-| `ENABLE_FRIGATE_SCORES` | `true` | Call Frigate's recognize endpoint pre-upload to store diversity scores used for quality replacement. Adds ~200 ms per upload. Disabling also disables the below-cap novelty gate |
-| `FRIGATE_SCORE_CEILING` | *(unset)* | Below-cap novelty gate. Unset: dynamic — skips candidates whose Frigate score exceeds the most-redundant tracked file's score, auto-calibrates each run. `0`: disable entirely. Positive value (e.g. `0.85`): fixed hard ceiling |
 | `MIN_FACE_WIDTH` | `90` | Minimum face crop width in pixels |
 | `FACE_MARGIN` | `0.15` | Padding around bounding box crop (fraction of face size) |
 | `ENABLE_FACE_ALIGNMENT` | `true` | Align to ArcFace 112×112 format using facial landmarks |
 | `USE_FULL_RESOLUTION` | `true` | Download full-resolution originals rather than preview thumbnails |
 | `MIN_CONFIDENCE` | `0.7` | Minimum Immich face detection confidence |
 | `BLUR_THRESHOLD` | `120.0` | Laplacian variance threshold — lower accepts more blur |
+| `MAX_AUTO_IMAGES` | `80` | Maximum training images per person in Frigate |
+| `QUALITY_REPLACEMENT` | `true` | When at cap, swap a weaker tracked image for a better candidate. With Frigate scoring active, targets the most redundant image (highest pre-upload recognize score); otherwise uses blur score. Never touches manually added Frigate files. Set `false` to skip people at cap |
+| `FRIGATE_SCORE_CEILING` | `0.0` | Skip uploads whose pre-upload Frigate recognize score exceeds this value — they are already well-covered. `0` disables; requires at least one prior run to have scores |
+| `ENABLE_FRIGATE_SCORES` | `true` | Call Frigate's recognize endpoint pre-upload to store diversity scores used for quality replacement. Adds ~200 ms per upload. Disable to use blur-score replacement only |
 
 ### GPU & Models
 
@@ -241,9 +225,8 @@ These defaults are tuned for Frigate's ArcFace requirements. winnow will warn on
 | Variable | Default | Description |
 | :--- | :--- | :--- |
 | `DRY_RUN` | `false` | Preview selection without downloading or uploading |
-| `RETRY_REJECTED` | `false` | Re-attempt all previously rejected assets (low-confidence skips, Frigate rejections, and other permanent exclusions) |
+| `RETRY_REJECTED` | `false` | Re-attempt assets previously rejected by Frigate |
-| `RESET_PERSON` | *(unset)* | Set to a person's name to clear their upload history and delete their winnow-managed Frigate training files so the next run starts fresh. Set to `*` to reset all tracked people at once. Manually added Frigate files are never touched |
+| `RESET_PERSON` | *(unset)* | Clear upload history for one person and delete their winnow-managed Frigate training files so the next run starts fresh. Manually added Frigate files are never touched |
-| `TRACE_CROP_SIZE` | *(unset)* | Debug: print all tracked crops whose width or height matches this pixel value, then exit |
 
 ### Scheduling
 
@@ -264,7 +247,7 @@ uv run winnow
 
 Requires Python 3.13+ and [uv](https://astral.sh/uv). An NVIDIA, AMD, or Intel GPU is recommended — CPU mode works but embedding computation is slower.
 
-When run with a terminal attached, winnow starts an interactive session: select which people to process and choose a strategy (adaptive, standard, broad, or a custom count) per person. Without a TTY — Docker, cron, or `AUTO_MODE=true` — it processes all people unattended using the configured defaults.
+When run with a terminal attached, winnow starts an interactive session: select which people to process and choose a strategy (auto, standard, broad, or a custom count) per person. Without a TTY — Docker, cron, or `AUTO_MODE=true` — it processes all people automatically using the configured defaults.
 
 ---
 

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "winnow"
-version = "0.4.9"
+version = "0.4.6"
 description = "Selects diverse, high-quality photos from Immich as training data for Frigate face recognition and object classification."
 license = "AGPL-3.0-or-later"
 requires-python = ">=3.13"

tests/test_config.py

@@ -18,7 +18,7 @@ def test_config_loads_defaults(monkeypatch):
     assert cfg.OUTPUT_DIR == "./frigate_train"
     assert cfg.YEARS_FILTER == 10
     assert cfg.MIN_FACE_WIDTH == 90
-    assert cfg.MIN_FACE_COUNT == 3
+    assert cfg.MIN_FACE_COUNT == 0
     assert cfg.BLUR_THRESHOLD == 120.0
     assert cfg.MIN_CONFIDENCE == 0.7
     assert cfg.MAX_AUTO_IMAGES == 80

uv.lock

@@ -2348,7 +2348,7 @@ wheels = [
 
 [[package]]
 name = "winnow"
-version = "0.4.8"
+version = "0.4.4"
 source = { editable = "." }
 dependencies = [
     { name = "croniter" },

winnow/cli.py

@@ -130,18 +130,6 @@ def _handle_duplicate_people(people: list[dict]) -> list[dict]:
     return people
 
 
-_UNSUPPORTED_VARS = [
-    "ENABLE_FRIGATE_SCORES",
-    "FRIGATE_SCORE_CEILING",
-    "MIN_FACE_WIDTH",
-    "FACE_MARGIN",
-    "ENABLE_FACE_ALIGNMENT",
-    "USE_FULL_RESOLUTION",
-    "MIN_CONFIDENCE",
-    "BLUR_THRESHOLD",
-]
-
-
 def main() -> None:
     """Entry point for winnow CLI."""
     try:
@@ -157,17 +145,6 @@ def main() -> None:
     [dim]Immich -> Frigate Training Data Curator[/dim]
         """)
 
-        set_unsupported = [v for v in _UNSUPPORTED_VARS if os.environ.get(v)]
-        if set_unsupported:
-            console.print(
-                f"[bold yellow]⚠  Advanced tuning vars set: "
-                f"{', '.join(set_unsupported)}[/bold yellow]"
-            )
-            console.print(
-                "[dim]  These defaults are calibrated for Frigate's ArcFace requirements. "
-                "Image quality issues caused by non-default values will not be investigated.[/dim]\n"
-            )
-
         ConfigManager.get().interactive_setup()
 
         try:

winnow/config.py

@@ -31,11 +31,11 @@ class _Config:
     MIN_CONFIDENCE: float = 0.7
     MAX_AUTO_IMAGES: int = 80
     QUALITY_REPLACEMENT: bool = True
-    FRIGATE_SCORE_CEILING: float | None = None
+    FRIGATE_SCORE_CEILING: float = 0.0
     ENABLE_FRIGATE_SCORES: bool = True
 
     # People filtering
-    MIN_FACE_COUNT: int = 3
+    MIN_FACE_COUNT: int = 0
     MERGE_DUPLICATE_PEOPLE: bool = False
 
     # Output quality
@@ -60,14 +60,13 @@ class _Config:
         self.OUTPUT_DIR = os.getenv("OUTPUT_DIR", "./frigate_train")
         self.YEARS_FILTER = int(os.getenv("YEARS_FILTER", "10"))
         self.MIN_FACE_WIDTH = int(os.getenv("MIN_FACE_WIDTH", "90"))
-        self.MIN_FACE_COUNT = int(os.getenv("MIN_FACE_COUNT", "3"))
+        self.MIN_FACE_COUNT = int(os.getenv("MIN_FACE_COUNT", "0"))
         self.MERGE_DUPLICATE_PEOPLE = os.getenv("MERGE_DUPLICATE_PEOPLE", "false").lower() in ("true", "1", "yes")
         self.BLUR_THRESHOLD = float(os.getenv("BLUR_THRESHOLD", "120.0"))
         self.MIN_CONFIDENCE = float(os.getenv("MIN_CONFIDENCE", "0.7"))
         self.MAX_AUTO_IMAGES = int(os.getenv("MAX_AUTO_IMAGES", "80"))
         self.QUALITY_REPLACEMENT = os.getenv("QUALITY_REPLACEMENT", "true").lower() in ("true", "1", "yes")
-        _ceiling_env = os.getenv("FRIGATE_SCORE_CEILING", "").strip()
+        self.FRIGATE_SCORE_CEILING = float(os.getenv("FRIGATE_SCORE_CEILING", "0.0"))
-        self.FRIGATE_SCORE_CEILING = float(_ceiling_env) if _ceiling_env else None
         self.ENABLE_FRIGATE_SCORES = os.getenv("ENABLE_FRIGATE_SCORES", "true").lower() in ("true", "1", "yes")
         self.FACE_MARGIN = float(os.getenv("FACE_MARGIN", "0.15"))
         self.USE_FULL_RESOLUTION = os.getenv("USE_FULL_RESOLUTION", "true").lower() in ("true", "1", "yes")

winnow/diversity.py

@@ -337,19 +337,16 @@ def _dedup_embeddings(
     order = sorted(range(len(candidates)), key=lambda i: quality_scores[i], reverse=True)
 
     kept_indices = []
-    # Pre-allocate a max-size buffer and fill row-by-row — eliminates the O(K²)
+    kept_stack: np.ndarray | None = None  # rebuilt only when a new item is kept (not every iteration)
-    # copy overhead from vstack-on-keep while keeping identical arithmetic.
-    kept_buf = np.empty((len(order), emb_normed.shape[1]), dtype=emb_normed.dtype)
-    n_kept = 0
 
     for i in order:
-        if n_kept > 0:
+        if kept_stack is not None:
-            sims = emb_normed[i] @ kept_buf[:n_kept].T
+            sims = emb_normed[i] @ kept_stack.T
             if np.any(sims > 1 - _DEDUP_THRESHOLD):
                 continue
-        kept_buf[n_kept] = emb_normed[i]
-        n_kept += 1
         kept_indices.append(i)
+        row = emb_normed[i : i + 1]
+        kept_stack = row if kept_stack is None else np.vstack([kept_stack, row])
 
     dropped = len(embeddings) - len(kept_indices)
     if dropped:
@@ -393,7 +390,7 @@ def _kmedoids(dist_matrix: np.ndarray, k: int, max_iter: int = 50) -> tuple[list
     # Iterative swap step
     medoids = list(medoids)
     labels = np.argmin(dist_matrix[:, medoids], axis=1)
-    cost = dist_matrix[np.arange(n), np.array(medoids)[labels]].sum()
+    cost = sum(dist_matrix[i, medoids[labels[i]]] for i in range(n))
 
     for _ in range(max_iter):
         improved = False
@@ -408,7 +405,7 @@ def _kmedoids(dist_matrix: np.ndarray, k: int, max_iter: int = 50) -> tuple[list
                 new_medoids = medoids.copy()
                 new_medoids[m_idx] = cand
                 new_labels = np.argmin(dist_matrix[:, new_medoids], axis=1)
-                new_cost = dist_matrix[np.arange(n), np.array(new_medoids)[new_labels]].sum()
+                new_cost = sum(dist_matrix[i, new_medoids[new_labels[i]]] for i in range(n))
                 if new_cost < cost:
                     medoids = new_medoids
                     labels = new_labels

winnow/executor.py

@@ -31,7 +31,7 @@ from .upload_tracker import (
     has_frigate_scores,
     mark_rejected,
     mark_uploaded,
-    record_frigate_files_batch,
+    record_frigate_file,
     remove_frigate_file,
 )
 
@@ -100,12 +100,10 @@ def _reconcile_frigate_mappings(
             except (ValueError, IndexError):
                 return 0.0
 
-        mappings = {
+        for (fname, asset_id), frigate_file in zip(uploaded, sorted(new_files, key=_ts)):
-            frigate_file: asset_id
+            if asset_id:
-            for (_, asset_id), frigate_file in zip(uploaded, sorted(new_files, key=_ts))
+                record_frigate_file(person_name, frigate_file, asset_id)
-            if asset_id
+        logger.debug(f"{person_name}: batch-mapped {target} Frigate file(s)")
-        }
-        record_frigate_files_batch(person_name, mappings)
     elif len(new_files) > target:
         logger.info(
             f"{person_name}: {len(new_files)} new Frigate files for {target} uploads"
@@ -231,7 +229,6 @@ def execute_jobs(jobs: list[dict]) -> None:
                                 f"[yellow]Skipped {asset['id']}"
                                 f" (detection confidence {conf:.2f} < {Config.MIN_CONFIDENCE})[/yellow]"
                             )
-                            mark_rejected(asset["id"], person_name=name)
                             progress.advance(job_task)
                             progress.advance(overall_task)
                             continue
@@ -483,28 +480,14 @@ def upload_to_frigate(jobs: list[dict]) -> None:
                         if _result is not None and (_result[0] or "").casefold() == name.casefold():
                             pre_fscore = _result[1]
 
-                # Below-cap novelty gate: skip candidates already covered by the Frigate model,
+                # Ceiling check: skip if the existing training set already covers this
-                # including conditions learned from manually-added images winnow can't track.
+                # face condition well. Applies below cap only — at cap, replacement logic
-                # pre_fscore is None on the first run (pre_run_count == 0 skips recognize_face
+                # drives the decision.
-                # above), so this block never fires on the first run without an extra guard.
+                if not at_cap and Config.FRIGATE_SCORE_CEILING > 0 and pre_run_count > 0:
-                if not at_cap and pre_fscore is not None:
+                    if pre_fscore is not None and pre_fscore > Config.FRIGATE_SCORE_CEILING:
-                    _ceiling = Config.FRIGATE_SCORE_CEILING
-                    if _ceiling is None:
-                        # Dynamic default: bar = most-redundant tracked file's Frigate score.
-                        # Falls back to uploading freely when no tracked scores exist yet.
-                        _bar = get_most_redundant_mapped_file(name)
-                        _skip = _bar is not None and pre_fscore > _bar[2]
-                        _bar_str = f"most redundant tracked {_bar[2]:.2f}" if _bar else ""
-                    elif _ceiling == 0.0:
-                        _skip = False  # explicitly disabled
-                        _bar_str = ""
-                    else:
-                        _skip = pre_fscore > _ceiling
-                        _bar_str = f"ceiling {_ceiling:.2f}"
-                    if _skip:
                         progress.console.print(
                             f"    [dim]⏭  {fname}: Frigate score {pre_fscore:.2f}"
-                            f" > {_bar_str}, already covered[/dim]"
+                            f" > ceiling {Config.FRIGATE_SCORE_CEILING:.2f}, already covered[/dim]"
                         )
                         progress.advance(upload_task)
                         continue
@@ -518,50 +501,70 @@ def upload_to_frigate(jobs: list[dict]) -> None:
                     using_fscore = person_has_fscores and Config.ENABLE_FRIGATE_SCORES
                     if using_fscore:
                         candidate_score = pre_fscore
-                        get_target = get_most_redundant_mapped_file
+                        if candidate_score is None:
-                        score_label, better_note = "frigate", " (more novel)"
+                            progress.console.print(
-                        no_score_msg = "Frigate recognize unavailable, skipping replacement"
+                                f"    [dim]⏭  {fname}: Frigate recognize unavailable, skipping replacement[/dim]"
+                            )
+                            progress.advance(upload_task)
+                            continue
+                        # Low score = more novel than the most redundant mapped file = replace
+                        target = get_most_redundant_mapped_file(name, exclude=failed_deletes)
+                        if target is None or candidate_score >= target[2]:
+                            target_score_str = f"{target[2]:.3f}" if target is not None else "N/A"
+                            progress.console.print(
+                                f"    [dim]⏭  {fname}: frigate {candidate_score:.3f} ≥ most redundant"
+                                f" {target_score_str}, not more novel[/dim]"
+                            )
+                            progress.advance(upload_task)
+                            continue
+                        target_frigate_file, _target_asset_id, target_score = target
+                        progress.console.print(
+                            f"    🔄 {fname}: frigate {candidate_score:.3f} < {target_score:.3f},"
+                            f" replacing {target_frigate_file} (more novel)"
+                        )
+                        if delete_frigate_person_files(name, [target_frigate_file]):
+                            remove_frigate_file(name, target_frigate_file)
+                            person_has_fscores = has_frigate_scores(name)
+                            effective_count -= 1
+                            # clear any blur-mode slot floor — Frigate uses a different score metric
+                            min_quality_score_for_slot = None
+                        else:
+                            logger.warning(f"Failed to delete {target_frigate_file} for {name}, skipping replacement")
+                            failed_deletes.add(target_frigate_file)
+                            progress.advance(upload_task)
+                            continue
                     else:
                         candidate_score = score_map.get(fname)
-                        get_target = get_lowest_quality_mapped_file
+                        if candidate_score is None:
-                        score_label, better_note = "blur", ""
+                            progress.console.print(
-                        no_score_msg = "no quality score, skipping replacement"
+                                f"    [dim]⏭  {fname}: no quality score, skipping replacement[/dim]"
-
+                            )
-                    if candidate_score is None:
+                            progress.advance(upload_task)
-                        progress.console.print(f"    [dim]⏭  {fname}: {no_score_msg}[/dim]")
+                            continue
-                        progress.advance(upload_task)
+                        target = get_lowest_quality_mapped_file(name, exclude=failed_deletes)
-                        continue
+                        if target is None or candidate_score <= target[2]:
-
+                            target_score_str = f"{target[2]:.3f}" if target is not None else "N/A"
-                    target = get_target(name, exclude=failed_deletes)
+                            progress.console.print(
-                    not_better = target is None or (
+                                f"    [dim]⏭  {fname}: blur {candidate_score:.3f} ≤ worst"
-                        candidate_score >= target[2] if using_fscore else candidate_score <= target[2]
+                                f" {target_score_str}, skipping[/dim]"
-                    )
+                            )
-                    if not_better:
+                            progress.advance(upload_task)
-                        target_str = f"{target[2]:.3f}" if target is not None else "N/A"
+                            continue
-                        op = "<" if using_fscore else ">"
+                        target_frigate_file, _target_asset_id, target_score = target
                         progress.console.print(
-                            f"    [dim]⏭  {fname}: {score_label} {candidate_score:.3f}"
+                            f"    🔄 {fname}: blur {candidate_score:.3f} > {target_score:.3f},"
-                            f" not {op} {target_str}, skipping[/dim]"
+                            f" replacing {target_frigate_file}"
                         )
-                        progress.advance(upload_task)
+                        if delete_frigate_person_files(name, [target_frigate_file]):
-                        continue
+                            remove_frigate_file(name, target_frigate_file)
-
+                            person_has_fscores = has_frigate_scores(name)
-                    target_frigate_file, _target_asset_id, target_score = target
+                            effective_count -= 1
-                    op = "<" if using_fscore else ">"
+                            min_quality_score_for_slot = score_map.get(fname)
-                    progress.console.print(
+                        else:
-                        f"    🔄 {fname}: {score_label} {candidate_score:.3f} {op} {target_score:.3f},"
+                            logger.warning(f"Failed to delete {target_frigate_file} for {name}, skipping replacement")
-                        f" replacing {target_frigate_file}{better_note}"
+                            failed_deletes.add(target_frigate_file)
-                    )
+                            progress.advance(upload_task)
-                    if delete_frigate_person_files(name, [target_frigate_file]):
+                            continue
-                        remove_frigate_file(name, target_frigate_file)
-                        person_has_fscores = has_frigate_scores(name)
-                        effective_count -= 1
-                        min_quality_score_for_slot = None if using_fscore else candidate_score
-                    else:
-                        logger.warning(f"Failed to delete {target_frigate_file} for {name}, skipping replacement")
-                        failed_deletes.add(target_frigate_file)
-                        progress.advance(upload_task)
-                        continue
 
                 for attempt in range(1, max_retries + 1):
                     try:

winnow/jobs.py

@@ -20,7 +20,7 @@ logger = logging.getLogger(__name__)
 
 # Strategy presets: (limit, mode_name)
 STRATEGY_PRESETS = {
-    "1": ("auto", "Adaptive Diversity"),
+    "1": ("auto", "Auto Diversity"),
     "2": (30, "Standard (30)"),
     "3": (100, "Broad (100)"),
 }
@@ -31,7 +31,7 @@ def _get_strategy_choice(has_embedding: bool, entity_type: str) -> tuple[int | s
     model_name = "InsightFace" if entity_type == "face" else "SigLIP"
 
     if has_embedding:
-        rprint("  [bold]1.[/bold] Adaptive Diversity [green][Recommended][/green]")
+        rprint("  [bold]1.[/bold] Auto (Objective Diversity) [green][Recommended][/green]")
         rprint("     [dim]• Dynamically selects images until redundancy starts[/dim]")
         rprint("  [bold]2.[/bold] Standard (30 images)")
         rprint("  [bold]3.[/bold] Broad (100 images)")
@@ -77,8 +77,7 @@ def _resolve_strategy(strategy: str, has_embedding: bool) -> tuple[int | str, st
         return int(custom_limit), "smart"
 
     strategy_map = {
-        "adaptive": ("auto", "smart"),
+        "auto": ("auto", "smart"),
-        "auto": ("auto", "smart"),  # legacy alias for adaptive
         "standard": (30, "smart"),
         "broad": (100, "smart"),
     }

winnow/upload_tracker.py

@@ -39,11 +39,6 @@ logger = logging.getLogger(__name__)
 UPLOAD_TRACKER_FILE = "frigate_uploaded_ids.json"
 REJECT_TRACKER_FILE = "frigate_rejected_ids.json"
 
-# Write-through in-memory cache keyed by the resolved file path.
-# Reduces per-call JSON reads from O(calls) to O(1) after the first load.
-# Keyed by full path so tests with isolated tmp dirs never share entries.
-_cache: dict[str, dict] = {}
-
 
 def _tracker_path(filename: str) -> Path:
     try:
@@ -55,23 +50,18 @@ def _tracker_path(filename: str) -> Path:
 
 def _load(filename: str) -> dict:
     path = _tracker_path(filename)
-    key = str(path)
+    if not path.exists():
-    if key in _cache:
+        return {}
-        return _cache[key]
+    try:
-    data: dict = {}
+        with open(path) as f:
-    if path.exists():
+            return json.load(f)
-        try:
+    except (json.JSONDecodeError, OSError) as e:
-            with open(path) as f:
+        logger.warning(f"Could not load tracker {filename}: {e}")
-                data = json.load(f)
+        return {}
-        except (json.JSONDecodeError, OSError) as e:
-            logger.warning(f"Could not load tracker {filename}: {e}")
-    _cache[key] = data
-    return data
 
 
 def _save(filename: str, data: dict) -> None:
     path = _tracker_path(filename)
-    _cache[str(path)] = data  # keep cache consistent with what we write
     path.parent.mkdir(parents=True, exist_ok=True)
     with open(path, "w") as f:
         json.dump(data, f, indent=2)
@@ -171,19 +161,6 @@ def record_frigate_file(person_name: str, frigate_filename: str, asset_id: str)
     logger.debug(f"Mapped Frigate file {frigate_filename} → {asset_id} ({person_name})")
 
 
-def record_frigate_files_batch(person_name: str, mappings: dict[str, str]) -> None:
-    """Record multiple Frigate filename → asset_id mappings in a single load/save."""
-    if not mappings:
-        return
-    data = _load(UPLOAD_TRACKER_FILE)
-    by_person = data.setdefault("by_person", {})
-    entry = _migrate_entry(by_person.get(person_name, {}))
-    entry["frigate_files"].update(mappings)
-    by_person[person_name] = entry
-    _save(UPLOAD_TRACKER_FILE, data)
-    logger.debug(f"Batch-mapped {len(mappings)} Frigate file(s) for {person_name}")
-
-
 def remove_frigate_file(person_name: str, frigate_filename: str) -> None:
     """Remove a Frigate filename from the mapping after it has been deleted.