docs: annotate known limitations and Frigate API improvement hooks
Adds inline LIMITATION / TODO(frigate-api) comments at each specific code site rather than a separate doc that would drift from the code. frigate_api.py — recognize_face: Mean-embedding limitation: score reflects the arithmetic mean of all training embeddings. A bimodal set (frontals + profiles) has a mean between clusters, making both ends look more novel than they are. Fixable if Frigate exposes per-file embeddings for nearest-neighbour comparison. frigate_api.py — get_all_frigate_person_files: "train" key exclusion is a hardcoded string. If Frigate adds other special top-level keys in /api/faces they'll be silently treated as person names. Needs a typed schema when Frigate documents the contract. executor.py — recognize_face call site: Async rebuild: each deletion triggers a background model rebuild in Frigate. Subsequent recognize calls in the same run return None (rebuild in progress), degrading quality replacement for later candidates. Fixable with a rebuild-complete signal from Frigate. executor.py — effective_count / manual file handling: Manually-added files are invisible to diversity decisions. Winnow observes their effect only indirectly via the Frigate score, not by measuring their embedding distribution. Per-file embeddings from Frigate would allow direct diversity measurement against the full set. executor.py — Frigate version assumption: All face training endpoints are v0.16+. No version check at startup; failures on older versions are opaque 404s. cache.py — MODEL_VERSIONS: Version string is a hardcoded constant. Manual model file replacement (custom weights, InsightFace update) won't invalidate cached embeddings. Needs file-checksum-derived versioning or a CLEAR_EMBEDDING_CACHE flag. diversity.py — thumbnail-resolution embeddings: Diversity selection runs InsightFace on preview thumbnails; the actual training crop comes from full-resolution originals. Negligible in practice but degrades if Immich preview quality is low.
This commit is contained in:
@@ -311,6 +311,11 @@ def upload_to_frigate(jobs: list[dict]) -> None:
|
||||
rprint("[yellow]⚠️ FRIGATE_URL not set, skipping upload.[/yellow]")
|
||||
return
|
||||
|
||||
# TODO: verify Frigate version at startup (GET /api/version) and warn if
|
||||
# below v0.16. The face training API (/api/faces/{name}/register,
|
||||
# /api/faces/{name}/delete, /api/faces/recognize) was introduced in v0.16;
|
||||
# older versions return 404s that surface as opaque upload failures.
|
||||
|
||||
rprint("\n[bold cyan]📤 Uploading to Frigate[/bold cyan]")
|
||||
rprint(f" Target: [dim]{frigate_url}[/dim]")
|
||||
|
||||
@@ -380,6 +385,14 @@ def upload_to_frigate(jobs: list[dict]) -> None:
|
||||
# manually-added Frigate files don't consume winnow's managed quota.
|
||||
# Replacement targets also come exclusively from the tracker, so manually
|
||||
# added files are never selected for deletion — only winnow-uploaded ones.
|
||||
# LIMITATION — manual files are invisible to diversity decisions: winnow
|
||||
# can observe their effect on the Frigate score (indirectly, via recognize)
|
||||
# but cannot measure their embedding distribution directly. If a user has
|
||||
# 20 manually-added frontals and winnow has room for 20 more, winnow may
|
||||
# add more frontals because it can't see that frontals are already covered.
|
||||
# TODO(frigate-api): if Frigate exposes per-file embeddings, compute
|
||||
# diversity against the full training set (tracked + manual) rather than
|
||||
# relying solely on the Frigate score as a proxy signal.
|
||||
_snapshot = (
|
||||
all_frigate_files.get(name, []) if all_frigate_files is not None
|
||||
else get_frigate_person_files(name)
|
||||
@@ -447,6 +460,14 @@ def upload_to_frigate(jobs: list[dict]) -> None:
|
||||
# Frigate rebuilds its model asynchronously after any delete (clear + background
|
||||
# thread), so the first recognize call after a deletion returns None — our code
|
||||
# handles this conservatively by skipping that candidate until the next run.
|
||||
# LIMITATION — async rebuild during multi-replacement runs: each deletion in a
|
||||
# single run triggers a background model rebuild in Frigate. Subsequent recognize
|
||||
# calls in the same run may get None (rebuild in progress), causing later
|
||||
# candidates to fall back to blur-score replacement or be skipped entirely.
|
||||
# The more replacements that happen in one run, the worse the scoring gets.
|
||||
# TODO(frigate-api): if Frigate exposes a model generation counter or a
|
||||
# rebuild-complete signal, poll it between recognize calls during replacement
|
||||
# sequences rather than accepting stale/None scores.
|
||||
pre_fscore: float | None = None
|
||||
if Config.ENABLE_FRIGATE_SCORES and pre_run_count > 0:
|
||||
if not at_cap or person_has_fscores:
|
||||
|
||||
Reference in New Issue
Block a user