docs: annotate known limitations and Frigate API improvement hooks
Adds inline LIMITATION / TODO(frigate-api) comments at each specific code site rather than a separate doc that would drift from the code. frigate_api.py — recognize_face: Mean-embedding limitation: score reflects the arithmetic mean of all training embeddings. A bimodal set (frontals + profiles) has a mean between clusters, making both ends look more novel than they are. Fixable if Frigate exposes per-file embeddings for nearest-neighbour comparison. frigate_api.py — get_all_frigate_person_files: "train" key exclusion is a hardcoded string. If Frigate adds other special top-level keys in /api/faces they'll be silently treated as person names. Needs a typed schema when Frigate documents the contract. executor.py — recognize_face call site: Async rebuild: each deletion triggers a background model rebuild in Frigate. Subsequent recognize calls in the same run return None (rebuild in progress), degrading quality replacement for later candidates. Fixable with a rebuild-complete signal from Frigate. executor.py — effective_count / manual file handling: Manually-added files are invisible to diversity decisions. Winnow observes their effect only indirectly via the Frigate score, not by measuring their embedding distribution. Per-file embeddings from Frigate would allow direct diversity measurement against the full set. executor.py — Frigate version assumption: All face training endpoints are v0.16+. No version check at startup; failures on older versions are opaque 404s. cache.py — MODEL_VERSIONS: Version string is a hardcoded constant. Manual model file replacement (custom weights, InsightFace update) won't invalidate cached embeddings. Needs file-checksum-derived versioning or a CLEAR_EMBEDDING_CACHE flag. diversity.py — thumbnail-resolution embeddings: Diversity selection runs InsightFace on preview thumbnails; the actual training crop comes from full-resolution originals. Negligible in practice but degrades if Immich preview quality is low.
This commit is contained in:
@@ -33,6 +33,10 @@ def get_all_frigate_person_files() -> dict[str, list[str]] | None:
|
||||
return None
|
||||
# Response: {person_name: [file, ...], "train": [...], ...}
|
||||
# "train" is a flat pending list, not a person — skip it.
|
||||
# TODO(frigate-api): "train" is the only known special key as of Frigate v0.16.
|
||||
# If Frigate adds other top-level non-person keys, they'll be silently treated
|
||||
# as person names here. Switch to an allowlist or a typed schema when Frigate
|
||||
# documents its response contract.
|
||||
return {
|
||||
name: files
|
||||
for name, files in data.items()
|
||||
@@ -75,6 +79,16 @@ def recognize_face(file_path: str) -> tuple[str | None, float] | None:
|
||||
|
||||
Returns None if FRIGATE_URL is unset, the API is unreachable, no face is
|
||||
detected, or face recognition is not enabled in Frigate.
|
||||
|
||||
LIMITATION — mean embedding comparison: the score reflects similarity to
|
||||
the arithmetic mean of all training embeddings, not to individual ones.
|
||||
A bimodal training set (e.g. frontals + profiles) has a mean that sits
|
||||
between both clusters, making candidates from either cluster look more
|
||||
novel than they are. Winnow could add redundant frontals while the score
|
||||
suggests novelty, because the mean is pulled toward profiles.
|
||||
TODO(frigate-api): if Frigate exposes per-file embeddings via the API,
|
||||
replace mean-comparison with nearest-neighbour distance across individual
|
||||
training embeddings for accurate coverage detection.
|
||||
"""
|
||||
frigate_url = os.environ.get("FRIGATE_URL", "").rstrip("/")
|
||||
if not frigate_url:
|
||||
|
||||
Reference in New Issue
Block a user