Smaller default cap is more conservative for new installs and better
reflects the minimum viable training set for Frigate face recognition.
Users who need more can set MAX_AUTO_IMAGES explicitly.
* revert: replace SQLite tracker with JSON backend (v0.6.0)
The SQLite migration (v0.5.0) spawned 21 bug-fix releases in two days:
data-loss risk in the migration layer, schema PK conflicts on per-person
tracking, tracker isolation races under concurrent runs, and a disk-full
error that triggered duplicate Frigate uploads. The complexity cost
outweighs the benefit.
Restored the pre-SQL JSON tracker (frigate_uploaded_ids.json /
frigate_rejected_ids.json in DATA_DIR). Public API is identical — all
callers in executor.py, jobs.py, cli.py, and reconcile.py work unchanged.
Existing JSON files are read automatically; frigate_tracker.db can be
deleted once verified.
* fix: narrow corrupt-thumbnail exception to UnidentifiedImageError; restore IMMICH_URL empty-string fallback
* docs: rewrite v0.6.0 changelog, strip v0.5.x entries, fix README SQLite references
* chore: remove dead get_frigate_filename_for_asset (orphaned since FRIGATE_SCORE_THRESHOLD removal in v0.4.0)
* fix: sort imports in executor.py (ruff I001)
* refactor: rename CACHE_DIR to DATA_DIR, default path .if_cache → data
CACHE_DIR held both the embedding cache and the SQLite tracker DB, making
the name misleading. DATA_DIR is more accurate.
- Config reads DATA_DIR first; falls back to CACHE_DIR with a deprecation
warning so existing setups don't break on upgrade
- Default local path: data (was .if_cache)
- Docker default path: /app/data (was /app/.if_cache)
- Internal references (embeddings.py, upload_tracker.py) updated to DATA_DIR
- compose.yml, .env.example, README, wiki, and changelog updated
- Version bumped to 0.5.1
* chore: update lockfile
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* refactor: rename CACHE_DIR to DATA_DIR, default path .if_cache → data
CACHE_DIR held both the embedding cache and the SQLite tracker DB, making
the name misleading. DATA_DIR is more accurate.
- Config reads DATA_DIR first; falls back to CACHE_DIR with a deprecation
warning so existing setups don't break on upgrade
- Default local path: data (was .if_cache)
- Docker default path: /app/data (was /app/.if_cache)
- Internal references (embeddings.py, upload_tracker.py) updated to DATA_DIR
- compose.yml, .env.example, README, wiki, and changelog updated
- Version bumped to 0.5.1
* chore: update lockfile
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
- release.yml: replace per-variant lockfile generation loop with a
single 'uv lock'; add idempotent release creation (skip if tag
already has a release so re-triggered runs don't 422)
- docker-publish.yml: remove stale uv-cpu/rocm/intel.lock entries
from paths-ignore (those files no longer exist)
- README: CACHE_DIR description now names winnow_tracker.db; tracker
description mentions SQLite
Tests cover the core ML pipeline algorithms in diversity.py — previously
untested. No network or model dependencies; all pure-function or
numpy-only paths:
- Face bbox and confidence extraction from Immich metadata, including
person_id filtering and missing-data edge cases
- Face crop scaling: verifies bbox coordinates are correctly scaled when
the thumbnail dimensions differ from the metadata image dimensions
- Near-duplicate dedup: removal below cosine threshold, quality-score
preference between duplicates, zero-quality-score treated as zero not
missing (falsy bug guard)
- K-Medoids: correct medoid count, distinctness, valid index range, and
full-N edge case
- Adaptive threshold: positive output, floor at 0.05 for identical
embeddings, single-point, scales with embedding spread
- Time-spread fallback: exact count, all-under-limit passthrough,
auto→30 default, first/last inclusion
- Cluster-aware selection: exact limit, subset invariant, auto-stop on
tight cluster, hard-example confidence weighting accepted
- immich_api.py: remove FaceData.embedding field and numpy import — Immich face
embeddings were never consumed after being fetched; bbox/confidence is all that's used
- embeddings.py: remove immich_embedding param from get_embedding() — never passed by
any caller; simplify docstring accordingly
- image_processing.py: wrap insightface_app.get() in warnings filter to suppress the
scikit-image FutureWarning about estimate being deprecated (already suppressed in
embeddings.py for the diversity path, was leaking from the crop-alignment path)
- README.md: remove two stale "object mode" / "object classification" fragments
winnow is a face recognition training tool. Object mode required manual
file placement with no Frigate API, pulled in torch/torchvision/transformers/
ultralytics (~2 GB), and was architecturally misaligned with the project goal.
Removed:
- process_object_mode (YOLO inference), get_yolo_model
- SigLIP model stack (get_siglip_model, get_object_embedding, batch variant)
- entity_type branching throughout diversity, embeddings, jobs, executor
- TRAINING_MODE and OBJECT_CLASS env vars
- torch, torchvision, transformers, ultralytics dependencies
- pytorch index entries from pyproject.toml
- Object mode from README (Modes section, env var table, How It Works)
- How It Works step 9: "upload freely" → note novelty gate may skip below-cap candidates
- Persistence note: "Frigate rejections" → "rejected assets" (covers confidence skips too)
- RETRY_REJECTED description: explicitly covers all rejection types, not just Frigate
- Image Quality section: split into user-adjustable controls and calibrated image
processing defaults with a support disclaimer to deter blind tuning
FRIGATE_SCORE_CEILING now defaults to dynamic mode (unset): below-cap
candidates are skipped if their pre-upload Frigate score exceeds the
most-redundant tracked file's score. This catches conditions already
covered by manually-added Frigate images that winnow cannot track —
the embedding-based diversity selection has no visibility into those.
Set FRIGATE_SCORE_CEILING=0 to disable; a positive value (e.g. 0.85)
still acts as a fixed hard ceiling. First-run safety is unchanged
(pre_run_count==0 prevents recognize_face from being called).
The two quality replacement branches (Frigate-score and blur-score)
shared identical structure and are merged into a single code path
parameterised by score source and comparison direction.
Also raises MIN_FACE_COUNT default from 0 to 3 and updates the
config test to match.
AUTO_MODE (batch/unattended) and STRATEGY=auto (diversity algorithm) shared
the same word for unrelated concepts. Renaming the strategy value to
'adaptive' eliminates the ambiguity. The old value is kept as a silent alias
so existing configs continue to work.
Interactive menu updated from "Auto (Objective Diversity)" to "Adaptive
Diversity". README AUTO_MODE description clarified to emphasise unattended
batch processing, not selection strategy.
- `:latest` image tag listed CUDA 13.3 — actual base is 12.8.1
- MERGE_DUPLICATE_PEOPLE existed in config but was absent from env var table
- TRACE_CROP_SIZE existed in CLI but was absent from env var table
- RESET_PERSON description now mentions `*` wildcard for bulk reset of all people
- Step 5 (near-duplicate removal) was added in v0.4.4 but never
documented; added between embedding and diversity selection
- Auto-stop was described as 'stops when similarity exceeds threshold'
which is backwards; it stops when the next candidate's distance to
already-selected images falls below the threshold (too similar, not
enough new information)
- Renumbered steps 5-8 to 6-9
- Tightened hard-example weighting description to match the code
Previously reset_person wiped the local tracker but left existing Frigate
training files as orphans, causing the next run to upload a full new batch
on top of them. Now deletes all winnow-managed files from Frigate first so
the next run starts truly clean. Manually-added Frigate files are never
touched.
Also fixes a spurious warning when FRIGATE_URL is unset: the deletion step
is now skipped at info level rather than logging a misleading error. Moves
the deferred import to top-level and eliminates a double disk read.
Bumps to 0.4.1. Also fixes ruff lint violations in executor.py (import
sort, line length) and promotes the "winnow only touches files it uploaded"
callout to the README intro.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Finalizes the 0.4.0 release:
- Version bumped to 0.4.0 in pyproject.toml
- CHANGELOG.md: add [0.4.0] section covering Frigate pre-upload scoring,
quality replacement inversion, bootstrap fix, FRIGATE_SCORE_CEILING,
ENABLE_FRIGATE_SCORES, removal of post-upload quality gate, and all
doc/default corrections
- README.md: step 8 updated for dual-mode replacement, FRIGATE_SCORE_CEILING
and ENABLE_FRIGATE_SCORES added to env var table, MIN_FACE_WIDTH and
BLUR_THRESHOLD defaults corrected (50→90, 100→120)
- .env.example: FRIGATE_SCORE_THRESHOLD replaced with FRIGATE_SCORE_CEILING;
QUALITY_REPLACEMENT line added; comments updated to match current semantics
- winnow/executor.py: bootstrap fix — recognize now called for all below-cap
uploads when ENABLE_FRIGATE_SCORES=true (was gated on CEILING > 0)
- winnow/upload_tracker.py: frigate_scores schema comment corrected to
pre-upload; get_most_redundant_mapped_file() added
- winnow/frigate_api.py: recognize_face returns (face_name, score)|None tuple
so wrong-person scores never drive replacement or ceiling decisions
- winnow/config.py: FRIGATE_SCORE_THRESHOLD renamed to FRIGATE_SCORE_CEILING;
ENABLE_FRIGATE_SCORES added
- tests/test_upload_tracker.py: 4 new tests for get_most_redundant_mapped_file
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Pipeline rewritten as 8 steps: separates thumbnail pass (quality filter +
embeddings) from full-res download and crop; adds quality replacement
logic at the upload step
- Persistence note updated to cover rejected IDs and RETRY_REJECTED
- Auto mode stopping threshold documented (20%/10% of median pairwise distance)
- Add OUTPUT_DIR env var (was undocumented)
- Local Install section notes interactive vs auto mode behaviour
- QUALITY_REPLACEMENT description tightened
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- update-lockfile.yml: rename workflow to "Update lockfiles"
- release.yml: add explicit uv python install 3.13 for consistency with other workflows
- docker-publish.yml: rename cpu cache scope from linux/amd64-cpu to cpu (now multi-arch)
- README: add GitHub release version badge and License badge
- README: shorten QUALITY_REPLACEMENT table cell
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Dockerfile: ARG VERSION + OCI labels (title, description, source, licenses, version)
- release.yml: pass VERSION build-arg to all four image builds
- docker-publish.yml: extend paths-ignore to cover community files
- .github/ISSUE_TEMPLATE/config.yml: disable blank issues, link to Discussions and wiki
- README.md: add Getting Help section
- pyproject.toml: expand description; add System Administrators audience classifier
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a person is at MAX_AUTO_IMAGES, winnow now replaces the
lowest-quality mapped training image in Frigate if a higher-confidence
candidate is available, keeping the training set always optimised.
Only files winnow uploaded (tracked via frigate_files mapping) are ever
replaced — manually added Frigate training images are never touched.
A concurrent-upload race condition is detected per-file: if N>1 new
files appear after one upload, the mapping is skipped rather than
guessed, logging at INFO level. The per-file snapshot approach is
retained over a batch approach because wrong mappings (which a batch
approach risks on race) are worse than no mapping.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New image variants:
- :rocm — InsightFace via ROCmExecutionProvider, SigLIP via PyTorch ROCm 6.3
- :intel — InsightFace via OpenVINOExecutionProvider (onnxruntime-openvino);
Intel GPU compute runtime auto-installed from Intel graphics repo;
OPENVINO_DEVICE=GPU opts into Arc/iGPU inference (default: CPU)
Also adds:
- pyproject-rocm.toml + uv-rocm.lock, pyproject-intel.toml + uv-intel.lock
- compose.yml device passthrough snippets for AMD and Intel
- CI: build-rocm and build-intel jobs in docker-publish.yml; all four
variants built and tagged in release.yml
- README reworked: cleaner structure, GPU variant quick-start examples,
OPENVINO_DEVICE env var documented
- CHANGELOG entry and version bump to 0.2.12
Fix: IntPrompt in dict literal was eagerly evaluated in the no-embedding
fallback path of _get_strategy_choice, prompting users for a custom count
regardless of which strategy they picked.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- .env.example: add VERBOSE, remove active AUTO_MODE=true (now TTY-detected),
comment out FORCE_CPU/ENABLE_CACHE default values, update CRON_SCHEDULE
section to document all three modes
- README: clarify that log file is always DEBUG regardless of VERBOSE
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The log file already captures DEBUG unconditionally. This env var wires
the same to the Rich console handler for troubleshooting without needing
to read the log file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three container lifetime modes via CRON_SCHEDULE:
unset — run once on startup, exit
empty string — sleep infinity; use docker exec -it winnow winnow
cron expression — run on startup, then on schedule
This replaces the need for a separate MANUAL_MODE env var. The empty
string is a natural "I want the container alive but unscheduled" signal.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The primary use case is headless Docker, so auto mode is now the default
whenever stdin has no TTY. Interactive mode activates when a terminal is
present (docker run -it, local shell). AUTO_MODE=true remains as an
explicit override for scripting with a pseudo-TTY.
Removes AUTO_MODE=true, stdin_open, tty, and FORCE_CPU=false from
compose.yml — none are needed for headless operation. Updates README
and the interactive-mode hint in the CLI.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces a VARIANT=gpu|cpu build arg to the Dockerfile. The cpu
variant uses ubuntu:22.04 (no CUDA base), installs torch+cpu and
onnxruntime (no GPU deps) via a separate pyproject-cpu.toml / uv-cpu.lock,
and is published as :cpu (dev-cpu on the dev branch) via a new
build-cpu CI job. Saves ~2 GB over the default GPU image.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Dockerfile: ENV INSIGHTFACE_HOME=/models → /models/.insightface to
match compose.yml and .env.example; the old value caused InsightFace
to store models at /models/models/buffalo_l (double-appended subdir)
- entrypoint.sh: use /app/.venv/bin/winnow (installed entry point)
instead of python -m winnow.cli
- config.py: ENABLE_CACHE default false → true; embedding cache is
always beneficial in practice; users can opt out with ENABLE_CACHE=false
- compose.yml: comment out CRON_SCHEDULE so scheduling is opt-in;
flip ENABLE_CACHE to commented opt-out to reflect new default
- README.md: update ENABLE_CACHE default documentation to true
- tests/test_config.py: update default assertion to match
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Title first, then badges on one consistent line. Removed redundant
Release and Lint badges. Fixed docs links to point to the wiki.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds context that winnow is especially useful for people who aren't
around enough for Frigate's live detections to supply adequate training
data on their own.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
flan
