docs: annotate known limitations and Frigate API improvement hooks

Adds inline LIMITATION / TODO(frigate-api) comments at each specific
code site rather than a separate doc that would drift from the code.

frigate_api.py — recognize_face:
  Mean-embedding limitation: score reflects the arithmetic mean of all
  training embeddings. A bimodal set (frontals + profiles) has a mean
  between clusters, making both ends look more novel than they are.
  Fixable if Frigate exposes per-file embeddings for nearest-neighbour
  comparison.

frigate_api.py — get_all_frigate_person_files:
  "train" key exclusion is a hardcoded string. If Frigate adds other
  special top-level keys in /api/faces they'll be silently treated as
  person names. Needs a typed schema when Frigate documents the contract.

executor.py — recognize_face call site:
  Async rebuild: each deletion triggers a background model rebuild in
  Frigate. Subsequent recognize calls in the same run return None
  (rebuild in progress), degrading quality replacement for later
  candidates. Fixable with a rebuild-complete signal from Frigate.

executor.py — effective_count / manual file handling:
  Manually-added files are invisible to diversity decisions. Winnow
  observes their effect only indirectly via the Frigate score, not by
  measuring their embedding distribution. Per-file embeddings from
  Frigate would allow direct diversity measurement against the full set.

executor.py — Frigate version assumption:
  All face training endpoints are v0.16+. No version check at startup;
  failures on older versions are opaque 404s.

cache.py — MODEL_VERSIONS:
  Version string is a hardcoded constant. Manual model file replacement
  (custom weights, InsightFace update) won't invalidate cached embeddings.
  Needs file-checksum-derived versioning or a CLEAR_EMBEDDING_CACHE flag.

diversity.py — thumbnail-resolution embeddings:
  Diversity selection runs InsightFace on preview thumbnails; the actual
  training crop comes from full-resolution originals. Negligible in
  practice but degrades if Immich preview quality is low.
This commit is contained in:
2026-06-14 18:41:37 +00:00
parent 4147dbec1c
commit 1d44df6e96
4 changed files with 49 additions and 1 deletions
+14
View File
@@ -33,6 +33,10 @@ def get_all_frigate_person_files() -> dict[str, list[str]] | None:
return None
# Response: {person_name: [file, ...], "train": [...], ...}
# "train" is a flat pending list, not a person — skip it.
# TODO(frigate-api): "train" is the only known special key as of Frigate v0.16.
# If Frigate adds other top-level non-person keys, they'll be silently treated
# as person names here. Switch to an allowlist or a typed schema when Frigate
# documents its response contract.
return {
name: files
for name, files in data.items()
@@ -75,6 +79,16 @@ def recognize_face(file_path: str) -> tuple[str | None, float] | None:
Returns None if FRIGATE_URL is unset, the API is unreachable, no face is
detected, or face recognition is not enabled in Frigate.
LIMITATION — mean embedding comparison: the score reflects similarity to
the arithmetic mean of all training embeddings, not to individual ones.
A bimodal training set (e.g. frontals + profiles) has a mean that sits
between both clusters, making candidates from either cluster look more
novel than they are. Winnow could add redundant frontals while the score
suggests novelty, because the mean is pulled toward profiles.
TODO(frigate-api): if Frigate exposes per-file embeddings via the API,
replace mean-comparison with nearest-neighbour distance across individual
training embeddings for accurate coverage detection.
"""
frigate_url = os.environ.get("FRIGATE_URL", "").rstrip("/")
if not frigate_url: