fix: v0.5.21 — diversity cap, fetch rejection, tracker isolation, merge dedup, env parsing

- diversity: cap k-medoids seed count at target so _cluster_aware_selection
  never returns more images than requested (violated MAX_AUTO_IMAGES when
  remaining capacity was 1-4 slots); add early return for limit=0 to
  prevent k-medoids from running with a zero budget; slice return to target
  as a final guard
- executor: mark_rejected() when fetch_full_image returns None so assets
  that can't be fetched (both original and preview) aren't retried every run
- executor: wrap mark_uploaded() in its own try/except so a SQLite disk-full
  error after a successful HTTP 200 doesn't retry the Frigate POST (duplicate
  upload) — the upload succeeded; only the tracker write failed
- cli: apply skip_ids deduplication to the re-fetched people list after a
  partial merge (some groups succeed, some fail) so unmerged duplicates
  don't produce two jobs for the same Frigate folder
- jobs: strip whitespace from SKIP_PEOPLE/ONLY_PEOPLE elements on split
  so "Alice, Bob" (space after comma) correctly matches "Bob"
This commit is contained in:
2026-06-15 03:56:57 +00:00
parent 95fb39ed50
commit 0602c4ac04
6 changed files with 60 additions and 13 deletions
+14
View File
@@ -7,6 +7,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.5.21] - 2026-06-15
### Fixed
- **`_cluster_aware_selection` now respects the requested limit**: the initial K-Medoids seed count `k` was never capped at `target`, so when the remaining capacity was 1–4 slots the function returned 5+ images instead of the requested count, silently violating `MAX_AUTO_IMAGES`. `k` is now `min(..., target)` and the returned list is sliced to `target` as a final guard. A new early-return for `limit == 0` prevents k-medoids from running at all and returning medoids for a zero-budget request.
- **`USE_FULL_RESOLUTION=True` path now marks assets rejected on persistent fetch failure**: when both the original and preview fallback in `fetch_full_image()` fail, the asset was silently re-selected and re-attempted on every future run. The full-res path now calls `mark_rejected()` on a `None` return, matching the behavior added in v0.5.20 for the thumbnail path.
- **`mark_uploaded` tracker failure no longer causes a duplicate Frigate upload**: `mark_uploaded()` was called inside the upload retry `try/except` block. A SQLite error (e.g. disk-full) after a successful HTTP 200 response would propagate to the retry handler, which would retry the POST and upload the same file twice. `mark_uploaded()` is now wrapped in its own `try/except`; a tracker write failure is logged and the upload loop breaks normally so Frigate never receives a duplicate.
- **`_handle_duplicate_people` deduplicates failed merges when some succeed**: when `MERGE_DUPLICATE_PEOPLE=true` and a mix of merges succeed and fail, `get_people()` was returned directly. The re-fetched list still contained the un-merged duplicate pairs, creating two jobs for the same Frigate folder. The re-fetched list is now filtered using the same skip-id logic applied in the all-fail path.
- **`SKIP_PEOPLE`/`ONLY_PEOPLE` now strip whitespace from each element**: `"Alice, Bob".split(",")` produces `[' Bob']` (with a leading space), which never matched person names from Immich. Both env vars now use a list comprehension with `.strip()` on each element, so space-padded comma-separated values work as expected.
## [0.5.20] - 2026-06-15
### Fixed
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "winnow"
version = "0.5.20"
version = "0.5.21"
description = "Selects diverse, high-quality photos from Immich as training data for Frigate face recognition."
license = "AGPL-3.0-or-later"
requires-python = ">=3.13"
+11 -1
View File
@@ -125,7 +125,17 @@ def _handle_duplicate_people(people: list[dict]) -> list[dict]:
if merged_any:
rprint(" [dim]Re-fetching people after merge...[/dim]")
return get_people()
fresh = get_people()
# Filter out the smaller duplicate from any group whose merge failed — those
# IDs still exist in Immich and would produce two jobs for the same folder.
# IDs from groups that merged successfully are already gone from Immich, so
# this filter is a no-op for them.
skip_ids = {
p["id"]
for ps in duplicates.values()
for p in sorted(ps, key=lambda x: x.get("assetCount", 0), reverse=True)[1:]
}
return [p for p in fresh if p.get("id") not in skip_ids]
# All merges failed — fall back to local deduplication (keep largest per name) so
# downstream job creation never runs two jobs for the same Frigate folder.
+12 -2
View File
@@ -494,8 +494,13 @@ def _cluster_aware_selection(
auto_threshold = _compute_adaptive_threshold(emb_normed) if limit == "auto" else 0.0
target = Config.MAX_AUTO_IMAGES if limit == "auto" else limit
# Short-circuit: nothing to select
if limit != "auto" and target <= 0:
return []
# --- Stage 1: K-Medoids clustering ---
k = min(max(5, target // 4), max(1, n // 3), n) # e.g., 1-20 clusters
# Cap k at target so we never seed more cluster representatives than requested.
k = min(max(5, target // 4), max(1, n // 3), n, target) # e.g., 1-20 clusters
logger.debug("Clustering %s embeddings into %s groups (K-Medoids)...", n, k)
# Compute full cosine distance matrix
@@ -547,7 +552,12 @@ def _cluster_aware_selection(
hard_count = sum(1 for c in selected_conf if c < 0.85)
logger.info("Selection complete: %s images (%s hard examples with confidence < 0.85).", len(selected), hard_count)
return [candidates[i] for i in selected]
# Slice to target: the while loop enforces this for non-auto mode, but
# guard here too in case the medoid seed already exceeded target (small target).
result = [candidates[i] for i in selected]
if limit != "auto":
result = result[:target]
return result
# =============================================================================
+20 -7
View File
@@ -149,6 +149,10 @@ def execute_jobs(jobs: list[dict]) -> None:
# Use full-resolution for final output when configured
if use_full_res:
img = fetch_full_image(asset["id"])
if img is None:
# Both original and preview fallback failed — mark rejected
# so this asset isn't retried on every future run.
mark_rejected(asset["id"], person_name=name)
else:
resp = requests.get(
f"{Config.IMMICH_URL}/api/assets/{asset['id']}/thumbnail?size=preview&format=JPEG",
@@ -498,13 +502,22 @@ def upload_to_frigate(jobs: list[dict]) -> None:
asset_id = asset_map.get(fname)
if asset_id:
mark_uploaded(
asset_id,
person_name=name,
score=score_map.get(fname),
crop_dims=dims_map.get(fname),
frigate_score=pre_fscore,
)
try:
mark_uploaded(
asset_id,
person_name=name,
score=score_map.get(fname),
crop_dims=dims_map.get(fname),
frigate_score=pre_fscore,
)
except Exception as tracker_exc:
# Upload to Frigate succeeded — don't retry on tracker
# failure or we'd upload a duplicate to Frigate.
logger.error(
"Tracker write failed for %s — upload succeeded"
" but asset may be re-selected next run: %s",
fname, tracker_exc,
)
if pre_fscore is not None:
person_has_fscores = True
actually_uploaded.append((fname, asset_id))
+2 -2
View File
@@ -229,8 +229,8 @@ def auto_configure(people: list[dict]) -> list[dict]:
return []
strategy = os.environ.get("STRATEGY", "auto")
skip = os.environ.get("SKIP_PEOPLE", "").split(",") if os.environ.get("SKIP_PEOPLE") else []
only = os.environ.get("ONLY_PEOPLE", "").split(",") if os.environ.get("ONLY_PEOPLE") else []
skip = [s.strip() for s in os.environ.get("SKIP_PEOPLE", "").split(",") if s.strip()]
only = [s.strip() for s in os.environ.get("ONLY_PEOPLE", "").split(",") if s.strip()]
if only:
valid_people = [p for p in valid_people if p["name"] in only]