FRIGATE_SCORE_CEILING now defaults to dynamic mode (unset): below-cap
candidates are skipped if their pre-upload Frigate score exceeds the
most-redundant tracked file's score. This catches conditions already
covered by manually-added Frigate images that winnow cannot track —
the embedding-based diversity selection has no visibility into those.
Set FRIGATE_SCORE_CEILING=0 to disable; a positive value (e.g. 0.85)
still acts as a fixed hard ceiling. First-run safety is unchanged
(pre_run_count==0 prevents recognize_face from being called).
The two quality replacement branches (Frigate-score and blur-score)
shared identical structure and are merged into a single code path
parameterised by score source and comparison direction.
Also raises MIN_FACE_COUNT default from 0 to 3 and updates the
config test to match.
All _load() calls after the first return the cached dict instead of
re-reading disk. _save() updates both disk and cache atomically.
Drops per-person tracker reads from ~90 to ~1 in the upload loop.
Keyed by resolved file path so test isolation (unique tmp_path dirs)
is preserved with no fixture changes needed.
- _dedup_embeddings: pre-allocated (Q,D) buffer replaces vstack-on-keep,
dropping O(K²×D) copy overhead down to O(K×D) fill work
- _kmedoids: swap cost sum replaced with numpy fancy-index reduction,
~20-50x faster per swap evaluation
- _reconcile_frigate_mappings: O(L) load/save pairs collapsed to one
batch write via record_frigate_files_batch
Fetching up to MAX_PAGES*page_size (1M) assets before the 3000-item
diversity pool cap was applied could exhaust memory on large Immich
libraries. Early-exit once 5000 items are collected — the pool cap
of 3000 makes anything beyond that wasteful. Also filter null/non-dict
items from page responses at fetch time.
- _dedup_embeddings: rebuild kept_stack only on keep (was every iteration → O(N²))
- _dedup_embeddings: fix quality_score sort key to use explicit None check (falsy-zero)
- _select_by_embedding: add post-dedup pool < limit guard with warning
- executor: use full resp.text for 'face' keyword check; only truncate display snippet
- _safe_person_dir: avoid false "//" prefix when output_dir resolves to filesystem root
Previously reset_person wiped the local tracker but left existing Frigate
training files as orphans, causing the next run to upload a full new batch
on top of them. Now deletes all winnow-managed files from Frigate first so
the next run starts truly clean. Manually-added Frigate files are never
touched.
Also fixes a spurious warning when FRIGATE_URL is unset: the deletion step
is now skipped at info level rather than logging a misleading error. Moves
the deferred import to top-level and eliminates a double disk read.
Bumps to 0.4.1. Also fixes ruff lint violations in executor.py (import
sort, line length) and promotes the "winnow only touches files it uploaded"
callout to the README intro.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Finalizes the 0.4.0 release:
- Version bumped to 0.4.0 in pyproject.toml
- CHANGELOG.md: add [0.4.0] section covering Frigate pre-upload scoring,
quality replacement inversion, bootstrap fix, FRIGATE_SCORE_CEILING,
ENABLE_FRIGATE_SCORES, removal of post-upload quality gate, and all
doc/default corrections
- README.md: step 8 updated for dual-mode replacement, FRIGATE_SCORE_CEILING
and ENABLE_FRIGATE_SCORES added to env var table, MIN_FACE_WIDTH and
BLUR_THRESHOLD defaults corrected (50→90, 100→120)
- .env.example: FRIGATE_SCORE_THRESHOLD replaced with FRIGATE_SCORE_CEILING;
QUALITY_REPLACEMENT line added; comments updated to match current semantics
- winnow/executor.py: bootstrap fix — recognize now called for all below-cap
uploads when ENABLE_FRIGATE_SCORES=true (was gated on CEILING > 0)
- winnow/upload_tracker.py: frigate_scores schema comment corrected to
pre-upload; get_most_redundant_mapped_file() added
- winnow/frigate_api.py: recognize_face returns (face_name, score)|None tuple
so wrong-person scores never drive replacement or ceiling decisions
- winnow/config.py: FRIGATE_SCORE_THRESHOLD renamed to FRIGATE_SCORE_CEILING;
ENABLE_FRIGATE_SCORES added
- tests/test_upload_tracker.py: 4 new tests for get_most_redundant_mapped_file
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Store (width, height) of each face crop in the tracker at upload time
alongside the existing blur score. Expose TRACE_CROP_SIZE=<px> to look
up which Immich asset produced a crop with that pixel dimension, making
it straightforward to trace unexpected or low-quality images visible in
Frigate back to their source.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a person is at MAX_AUTO_IMAGES, winnow now replaces the
lowest-quality mapped training image in Frigate if a higher-confidence
candidate is available, keeping the training set always optimised.
Only files winnow uploaded (tracked via frigate_files mapping) are ever
replaced — manually added Frigate training images are never touched.
A concurrent-upload race condition is detected per-file: if N>1 new
files appear after one upload, the mapping is skipped rather than
guessed, logging at INFO level. The per-file snapshot approach is
retained over a batch approach because wrong mappings (which a batch
approach risks on race) are worse than no mapping.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New image variants:
- :rocm — InsightFace via ROCmExecutionProvider, SigLIP via PyTorch ROCm 6.3
- :intel — InsightFace via OpenVINOExecutionProvider (onnxruntime-openvino);
Intel GPU compute runtime auto-installed from Intel graphics repo;
OPENVINO_DEVICE=GPU opts into Arc/iGPU inference (default: CPU)
Also adds:
- pyproject-rocm.toml + uv-rocm.lock, pyproject-intel.toml + uv-intel.lock
- compose.yml device passthrough snippets for AMD and Intel
- CI: build-rocm and build-intel jobs in docker-publish.yml; all four
variants built and tagged in release.yml
- README reworked: cleaner structure, GPU variant quick-start examples,
OPENVINO_DEVICE env var documented
- CHANGELOG entry and version bump to 0.2.12
Fix: IntPrompt in dict literal was eagerly evaluated in the no-embedding
fallback path of _get_strategy_choice, prompting users for a custom count
regardless of which strategy they picked.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
insightface 1.0.1 added a hard dep on the CPU onnxruntime package.
Combined with an incorrect override-dependencies entry in 0.2.10 that
forced onnxruntime (no platform marker) unconditionally, both packages
were installed into the venv on x86_64 Linux — the CPU package landed
last and overwrote onnxruntime-gpu, removing CUDAExecutionProvider
from the provider list.
Fix: declare the two packages as conflicting in uv's resolver so only
the correct one is installed per environment.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
onnxruntime-gpu has no arm64 wheels (manylinux_2_27_x86_64 /
manylinux_2_28_x86_64 only). uv sync --frozen failed on the arm64
image with exit code 2. Gated onnxruntime-gpu behind the x86_64
marker; arm64 and non-Linux use the CPU onnxruntime package. Added
required-environments so the lockfile is solved for both platforms.
Removed onnxruntime-gpu from override-dependencies (it had no marker
support and blocked arm64 resolution).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
curl|gpg --dearmor was silently dropping the deadsnakes key (gpg exits 0
on bad input), leaving apt unable to find python3.13. Reverted to
add-apt-repository with GNUPGHOME=$(mktemp -d) to isolate gpg from any
pre-existing agent socket. Removed --platform=\$BUILDPLATFORM from the
arm64 base so the image contains real arm64 binaries. License updated
from MIT to AGPL-3.0-or-later.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Ubuntu 26.04 ships Python 3.14, not 3.13. Reverted arm64 to ubuntu:24.04.
Both architectures now add the deadsnakes PPA by fetching the GPG key via
curl and piping through gpg --dearmor — no gpg-agent, safe under QEMU.
Removes the per-arch conditional and ARG TARGETARCH dependency in RUN commands.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Docker automatic platform ARGs are only in scope for FROM instructions.
$TARGETARCH in RUN commands was always empty, so the deadsnakes PPA
conditional never ran and python3.13 could not be found on the Ubuntu
22.04 CUDA base.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All platforms now use Python 3.13. amd64 installs via deadsnakes PPA on
the Ubuntu 22.04 CUDA base; arm64 gets Python 3.13 natively from Ubuntu
26.04. Verified cp313 wheels exist for onnxruntime-gpu 1.26.0 and
torch 2.12.0+cu126. uv.lock regenerated under CPython 3.13.5.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Store Immich face confidence scores per asset in frigate_uploaded_ids.json
- Add frigate_api.py: query GET /api/faces to count trained images per person
- Record Frigate training count as frigate_count in tracker for offline fallback
- MAX_AUTO_IMAGES cap now uses live Frigate count → cached frigate_count → local uploaded count
- Startup summary shows last known Frigate training count per person
- Migrate by_person entries from flat list to {asset_ids, scores, frigate_count} dict
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- entrypoint.sh: replace `uv run` with `/app/.venv/bin/python` to skip
uv's sync check which was re-downloading ruff and rebuilding the package
on every container startup
- compose.yml, .env.example: fix INSIGHTFACE_HOME /models → /models/.insightface;
InsightFace appends models/ to root, so /models produced /models/models/buffalo_l
- README.md: add Immich and Frigate badges from upstream
- CHANGELOG.md, pyproject.toml, uv.lock: bump to 0.2.1
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- pyproject.toml: version 0.1.0 → 0.2.0
- CHANGELOG.md: full [0.2.0] entry covering all changes since the fork —
headless operation, Docker/scheduling, object mode, people filtering,
quality controls, CI/CD, tests, docs, and all bug fixes
- uv.lock: regenerated after version bump
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add docs/setup.md, docs/troubleshooting.md, docs/faq.md as user-facing wiki
- Add .env.example with all env vars and inline comments
- Fix compose.override.yml: rename service if-curator → winnow, update volume paths
- Fix CHANGELOG.md: rename if-curator → winnow in release notes
- Link docs from README
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
flan
