docs: annotate known limitations and Frigate API improvement hooks

Adds inline LIMITATION / TODO(frigate-api) comments at each specific
code site rather than a separate doc that would drift from the code.

frigate_api.py — recognize_face:
  Mean-embedding limitation: score reflects the arithmetic mean of all
  training embeddings. A bimodal set (frontals + profiles) has a mean
  between clusters, making both ends look more novel than they are.
  Fixable if Frigate exposes per-file embeddings for nearest-neighbour
  comparison.

frigate_api.py — get_all_frigate_person_files:
  "train" key exclusion is a hardcoded string. If Frigate adds other
  special top-level keys in /api/faces they'll be silently treated as
  person names. Needs a typed schema when Frigate documents the contract.

executor.py — recognize_face call site:
  Async rebuild: each deletion triggers a background model rebuild in
  Frigate. Subsequent recognize calls in the same run return None
  (rebuild in progress), degrading quality replacement for later
  candidates. Fixable with a rebuild-complete signal from Frigate.

executor.py — effective_count / manual file handling:
  Manually-added files are invisible to diversity decisions. Winnow
  observes their effect only indirectly via the Frigate score, not by
  measuring their embedding distribution. Per-file embeddings from
  Frigate would allow direct diversity measurement against the full set.

executor.py — Frigate version assumption:
  All face training endpoints are v0.16+. No version check at startup;
  failures on older versions are opaque 404s.

cache.py — MODEL_VERSIONS:
  Version string is a hardcoded constant. Manual model file replacement
  (custom weights, InsightFace update) won't invalidate cached embeddings.
  Needs file-checksum-derived versioning or a CLEAR_EMBEDDING_CACHE flag.

diversity.py — thumbnail-resolution embeddings:
  Diversity selection runs InsightFace on preview thumbnails; the actual
  training crop comes from full-resolution originals. Negligible in
  practice but degrades if Immich preview quality is low.
This commit is contained in:
2026-06-14 18:41:37 +00:00
parent 4147dbec1c
commit 1d44df6e96
4 changed files with 49 additions and 1 deletions
+21
View File
@@ -311,6 +311,11 @@ def upload_to_frigate(jobs: list[dict]) -> None:
rprint("[yellow]⚠️ FRIGATE_URL not set, skipping upload.[/yellow]")
return
# TODO: verify Frigate version at startup (GET /api/version) and warn if
# below v0.16. The face training API (/api/faces/{name}/register,
# /api/faces/{name}/delete, /api/faces/recognize) was introduced in v0.16;
# older versions return 404s that surface as opaque upload failures.
rprint("\n[bold cyan]📤 Uploading to Frigate[/bold cyan]")
rprint(f" Target: [dim]{frigate_url}[/dim]")
@@ -380,6 +385,14 @@ def upload_to_frigate(jobs: list[dict]) -> None:
# manually-added Frigate files don't consume winnow's managed quota.
# Replacement targets also come exclusively from the tracker, so manually
# added files are never selected for deletion — only winnow-uploaded ones.
# LIMITATION — manual files are invisible to diversity decisions: winnow
# can observe their effect on the Frigate score (indirectly, via recognize)
# but cannot measure their embedding distribution directly. If a user has
# 20 manually-added frontals and winnow has room for 20 more, winnow may
# add more frontals because it can't see that frontals are already covered.
# TODO(frigate-api): if Frigate exposes per-file embeddings, compute
# diversity against the full training set (tracked + manual) rather than
# relying solely on the Frigate score as a proxy signal.
_snapshot = (
all_frigate_files.get(name, []) if all_frigate_files is not None
else get_frigate_person_files(name)
@@ -447,6 +460,14 @@ def upload_to_frigate(jobs: list[dict]) -> None:
# Frigate rebuilds its model asynchronously after any delete (clear + background
# thread), so the first recognize call after a deletion returns None — our code
# handles this conservatively by skipping that candidate until the next run.
# LIMITATION — async rebuild during multi-replacement runs: each deletion in a
# single run triggers a background model rebuild in Frigate. Subsequent recognize
# calls in the same run may get None (rebuild in progress), causing later
# candidates to fall back to blur-score replacement or be skipped entirely.
# The more replacements that happen in one run, the worse the scoring gets.
# TODO(frigate-api): if Frigate exposes a model generation counter or a
# rebuild-complete signal, poll it between recognize calls during replacement
# sequences rather than accepting stale/None scores.
pre_fscore: float | None = None
if Config.ENABLE_FRIGATE_SCORES and pre_run_count > 0:
if not at_cap or person_has_fscores: