- Config: remove _ConfigAccessor and ConfigManager; use __getattr__ for lazy
loading on single _Config class; re-register self as _instance in __getattr__
so reset() always clears the correct object (item 1)
- upload_tracker: replace hand-rolled JSON store with sqlite3; auto-migrates
existing JSON on first run; remove dead record_frigate_file function;
connection re-opens when CACHE_DIR changes for test isolation (items 2, 8)
- diversity: move ThreadPoolExecutor import to module level; inject optional
fetch_fn parameter for testability (items 3, 6)
- pyproject: consolidate 4 variant files into extras (gpu/rocm/intel/cpu);
update Dockerfile to use --extra flag; delete variant pyproject/lock files;
uv.lock needs regen with `uv lock` after this change (item 4)
- jobs: extract _build_job helper to separate business logic from terminal I/O;
auto_configure delegates dedup/selection to _build_job (item 5)
- logging: convert f-string log calls to % interpolation throughout all winnow/
modules (item 7)
- reconcile: new module with reconcile_frigate_mappings and
enrich_asset_with_face_data extracted from executor.py (item 9)
- scheduler: print next scheduled run time after startup and after each run;
fix f-string logger.error call (item 10)
cache.py — model fingerprint auto-invalidation:
Replace hardcoded "buffalo_l_v1" version string with a fingerprint
derived from buffalo_l .onnx file sizes and mtimes. EmbeddingCache now
computes this at init time; stale embeddings from replaced or updated
model files are automatically invalidated. Falls back to the static
string before the model is downloaded.
Note: existing caches built against the old key will miss on the first
run after upgrade and recompute cleanly.
frigate_api.py — Frigate version check:
Add get_frigate_version() (GET /api/version). Called at the start of
upload_to_frigate(); warns if below v0.16 where the face training API
endpoints don't exist.
immich_api.py + cli.py — Immich version check:
Add get_immich_version() (GET /api/server/version). Called at startup
before get_people(); warns if below v1.106 where the face data and
merge APIs winnow depends on aren't guaranteed present.
Remaining TODO(frigate-api) annotations are left in place — they require
Frigate to expose per-file embeddings or a rebuild-complete signal before
they can be addressed.
Adds inline LIMITATION / TODO(frigate-api) comments at each specific
code site rather than a separate doc that would drift from the code.
frigate_api.py — recognize_face:
Mean-embedding limitation: score reflects the arithmetic mean of all
training embeddings. A bimodal set (frontals + profiles) has a mean
between clusters, making both ends look more novel than they are.
Fixable if Frigate exposes per-file embeddings for nearest-neighbour
comparison.
frigate_api.py — get_all_frigate_person_files:
"train" key exclusion is a hardcoded string. If Frigate adds other
special top-level keys in /api/faces they'll be silently treated as
person names. Needs a typed schema when Frigate documents the contract.
executor.py — recognize_face call site:
Async rebuild: each deletion triggers a background model rebuild in
Frigate. Subsequent recognize calls in the same run return None
(rebuild in progress), degrading quality replacement for later
candidates. Fixable with a rebuild-complete signal from Frigate.
executor.py — effective_count / manual file handling:
Manually-added files are invisible to diversity decisions. Winnow
observes their effect only indirectly via the Frigate score, not by
measuring their embedding distribution. Per-file embeddings from
Frigate would allow direct diversity measurement against the full set.
executor.py — Frigate version assumption:
All face training endpoints are v0.16+. No version check at startup;
failures on older versions are opaque 404s.
cache.py — MODEL_VERSIONS:
Version string is a hardcoded constant. Manual model file replacement
(custom weights, InsightFace update) won't invalidate cached embeddings.
Needs file-checksum-derived versioning or a CLEAR_EMBEDDING_CACHE flag.
diversity.py — thumbnail-resolution embeddings:
Diversity selection runs InsightFace on preview thumbnails; the actual
training crop comes from full-resolution originals. Negligible in
practice but degrades if Immich preview quality is low.
- upload_tracker: extract _pick_mapped_file() private helper; get_lowest_quality_mapped_file
and get_most_redundant_mapped_file are now one-liners over the same body
- upload_tracker: remove_frigate_file now also prunes the corresponding frigate_scores entry,
preventing unbounded accumulation of orphaned score entries across replacement cycles
- frigate_api: get_frigate_face_counts delegates to get_all_frigate_person_files, eliminating
the duplicated "name != 'train' and isinstance(files, list)" filter body
- executor: cache has_frigate_scores(name) as person_has_fscores before the per-file loop;
refresh it after each remove_frigate_file call and after each scored upload, eliminating
two redundant disk reads per at-cap file iteration
- executor: casefold() both sides of the recognize_face person-name comparison so a Frigate
casing normalization or manual-registration casing mismatch does not silently suppress scoring
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Finalizes the 0.4.0 release:
- Version bumped to 0.4.0 in pyproject.toml
- CHANGELOG.md: add [0.4.0] section covering Frigate pre-upload scoring,
quality replacement inversion, bootstrap fix, FRIGATE_SCORE_CEILING,
ENABLE_FRIGATE_SCORES, removal of post-upload quality gate, and all
doc/default corrections
- README.md: step 8 updated for dual-mode replacement, FRIGATE_SCORE_CEILING
and ENABLE_FRIGATE_SCORES added to env var table, MIN_FACE_WIDTH and
BLUR_THRESHOLD defaults corrected (50→90, 100→120)
- .env.example: FRIGATE_SCORE_THRESHOLD replaced with FRIGATE_SCORE_CEILING;
QUALITY_REPLACEMENT line added; comments updated to match current semantics
- winnow/executor.py: bootstrap fix — recognize now called for all below-cap
uploads when ENABLE_FRIGATE_SCORES=true (was gated on CEILING > 0)
- winnow/upload_tracker.py: frigate_scores schema comment corrected to
pre-upload; get_most_redundant_mapped_file() added
- winnow/frigate_api.py: recognize_face returns (face_name, score)|None tuple
so wrong-person scores never drive replacement or ceiling decisions
- winnow/config.py: FRIGATE_SCORE_THRESHOLD renamed to FRIGATE_SCORE_CEILING;
ENABLE_FRIGATE_SCORES added
- tests/test_upload_tracker.py: 4 new tests for get_most_redundant_mapped_file
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fetch all Frigate training files once before the upload loop instead of once
per person — for N people this reduces GET /api/faces calls from N to 1.
Falls back to per-person calls if the pre-fetch fails.
Check InsightFace detection confidence immediately after face enrichment,
before fetching the full-resolution image. Assets that fail MIN_CONFIDENCE
are skipped without downloading, saving potentially large image downloads.
Collapse the gate removal tracker writes from 3×N file ops into 2 total
via remove_and_reclassify_batch: one write to the uploaded tracker (remove
file mappings + remove from flat set) and one write to the rejected tracker.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After each successful upload, call POST /api/faces/recognize to get
Frigate's own confidence score (0-1) for the uploaded crop. Store it
in the tracker as frigate_scores alongside the existing blur score.
When quality replacement activates and frigate_scores are present,
use them for the replacement comparison instead of blur scores — an
image Frigate recognizes poorly is a worse training image than one it
recognizes well, regardless of sharpness. Falls back to blur scores
on first run before any frigate_scores are populated.
Also surfaces frigate_score in TRACE_CROP_SIZE output.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
C1/C4: Cap blur-score computation at 1440 px before calling assess_quality
so scores are always on the same Laplacian scale as the embedding path
(which operates on Immich preview thumbnails). Also converts the image to
RGB before scoring and stores 0.0 on assess_quality failure so files
uploaded without a score remain eligible for future quality replacement
instead of occupying a slot permanently.
C2: Fall back to the tracker's mapped-filename set as the pre-upload
baseline when the Frigate GET /api/faces endpoint is unreachable at upload
start. Previously, uploads that succeeded during a partial API outage were
never mapped in frigate_files, leaving get_tracked_frigate_file_count
permanently under-counting those files and allowing Frigate to exceed
MAX_AUTO_IMAGES over time.
C3: Track min_quality_score_for_slot when a quality-replacement delete
succeeds but the subsequent upload fails. This ensures the freed slot can
only be filled by a candidate that beats the deleted file's score, not just
the next file in iteration order (which could be lower quality than what
was deleted).
Add get_tracked_frigate_filenames() to upload_tracker and expand tracker
tests to cover the new function and exclude-parameter behaviour.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a person is at MAX_AUTO_IMAGES, winnow now replaces the
lowest-quality mapped training image in Frigate if a higher-confidence
candidate is available, keeping the training set always optimised.
Only files winnow uploaded (tracked via frigate_files mapping) are ever
replaced — manually added Frigate training images are never touched.
A concurrent-upload race condition is detected per-file: if N>1 new
files appear after one upload, the mapping is skipped rather than
guessed, logging at INFO level. The per-file snapshot approach is
retained over a batch approach because wrong mappings (which a batch
approach risks on race) are worse than no mapping.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Frigate /api/faces response has person names as top-level keys with
lists of filenames — {person: [file, ...], "train": [...]}. The old
code incorrectly looked inside data["train"] as if it were a dict of
persons, causing 'list object has no attribute items' on every run.
Immich get_people() now checks for 401 before raise_for_status() and
logs a clear "API key invalid or expired" message instead of the raw
requests exception string.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Store Immich face confidence scores per asset in frigate_uploaded_ids.json
- Add frigate_api.py: query GET /api/faces to count trained images per person
- Record Frigate training count as frigate_count in tracker for offline fallback
- MAX_AUTO_IMAGES cap now uses live Frigate count → cached frigate_count → local uploaded count
- Startup summary shows last known Frigate training count per person
- Migrate by_person entries from flat list to {asset_ids, scores, frigate_count} dict
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
flan
