winnow

Note: winnow's approach to training Frigate face recognition is not an officially documented workflow — results may vary.

Early Development — Use With Caution winnow is functional but still maturing. Features that modify your Frigate training data — quality replacement, stale mapping cleanup — can remove images from your dataset and are not yet battle-tested at scale. Review the logs after each run and keep backups of your Frigate face training directory until you are confident in the results.

Docs: Setup · Troubleshooting · FAQ

winnow pulls photos from your Immich library, selects the most diverse and highest-quality subset using AI embeddings, and delivers them as training data for Frigate's face recognition and object classification models.

Frigate's face recognition is only as good as its training data — and the key quality metric is diversity, not volume. A hundred photos from the same week teach the model one lighting condition. What you need is a spread: different years, different angles, different lighting, different contexts. Your photo library already has that data. winnow finds and delivers the right subset automatically.

winnow only touches files it uploaded. Faces added to Frigate manually through its UI are never deleted, replaced, or modified — not by quality replacement, not by RESET_PERSON, not by stale cleanup. If you have a curated training set you want to keep, it is safe.

---

How It Works

Immich library
      │
      ▼
1. Fetch all assets tagged with this person
      │
      ▼
2. Filter by recency (YEARS_FILTER) and skip already-uploaded
   and rejected assets (persistent tracker in CACHE_DIR)
      │
      ▼
3. Quality filter — download preview thumbnails and reject:
   • Blurry images (Laplacian variance)
   • Grayscale / infrared (channel similarity)
   • Over- or underexposed
   • Low detection confidence
   • Face crops below minimum pixel size
      │
      ▼
4. Compute embeddings from the same preview thumbnails
   • Faces   → InsightFace (ArcFace / Buffalo_L)  → 512-dim vector
   • Objects → SigLIP (Vision Transformer)         → 768-dim vector
      │
      ▼
5. Near-duplicate removal — greedy cosine-distance pass drops burst shots
   and near-identical photos before clustering runs; the highest-quality
   image from each near-duplicate group is kept
      │
      ▼
6. Diversity selection
   • K-Medoids clustering → one representative per natural group
   • Farthest Point Sampling → fill remaining slots with maximally spread picks
   • Hard example weighting — low-confidence detections get a distance boost
     so unusual angles and harder looks are preferred over easy frontals
   • Auto mode: stops when the next candidate is too similar to those already
     selected (distance threshold = 20 % of median pairwise distance for
     faces, 10 % for objects)
      │
      ▼
7. Download full-resolution originals from Immich
      │
      ▼
8. Crop and process
   • Face mode: EXIF-corrected, landmark-aligned 112×112 crop (ArcFace format)
   • Object mode: YOLOv9c detection → one crop per matched instance
      │
      ▼
9. Deliver
   • Face mode: upload crops to Frigate's face registration API
     ↳ below MAX_AUTO_IMAGES — upload freely
     ↳ at cap + QUALITY_REPLACEMENT=true — with Frigate scoring active,
       swap the most redundant tracked image (highest pre-upload recognize
       score) if the candidate is more novel (lower score); falling back to
       blur-score comparison when no Frigate scores are available; manually
       added files are never touched
     ↳ at cap + QUALITY_REPLACEMENT=false — skip this person
   • Object mode: save crops to disk → place into your Frigate data directory

Uploaded and rejected asset IDs are persisted across runs. The same image is never processed twice; Frigate rejections are permanently skipped unless RETRY_REJECTED=true.

---

Modes

Face mode (default) — extracts face crops using Immich's bounding box metadata, applies EXIF orientation correction, and aligns them to ArcFace's standard 112×112 format using 5-point facial landmarks. Crops are uploaded directly to Frigate's face registration API.

Object mode — runs each full-resolution image through YOLOv9c to detect instances of a target class (dog, cat, car, etc.), crops each detection, and saves it to the output directory. Frigate has no API for uploading object training data; place the crops into your Frigate data directory manually.

---

Running in Docker

Image Tags

Tag	Arch	Acceleration
:latest	amd64 + arm64	NVIDIA CUDA 12.8 (amd64) · requires NVIDIA Container Toolkit
:rocm	amd64	AMD ROCm · pass /dev/kfd + /dev/dri
:intel	amd64	Intel Arc / iGPU via OpenVINO · pass /dev/dri, set OPENVINO_DEVICE=GPU
:cpu	amd64 + arm64	CPU only · ~2 GB smaller · no GPU required

Quick Start

NVIDIA:

services:
  winnow:
    image: ghcr.io/sudolulo/winnow:latest
    environment:
      - IMMICH_URL=http://192.168.1.10:2283
      - API_KEY=your-immich-api-key
      - FRIGATE_URL=http://192.168.1.10:5000
      - CRON_SCHEDULE=0 3 * * 0
    volumes:
      - /path/to/models:/models
      - /path/to/cache:/app/.if_cache
      - /path/to/output:/app/frigate_train
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

AMD (:rocm): use image: ghcr.io/sudolulo/winnow:rocm and replace the deploy: block with:

    devices:
      - /dev/kfd
      - /dev/dri
    group_add:
      - video
      - render

Intel (:intel): use image: ghcr.io/sudolulo/winnow:intel and replace the deploy: block with:

    devices:
      - /dev/dri
    group_add:
      - render
    environment:
      - OPENVINO_DEVICE=GPU   # omit to run OpenVINO inference on CPU (default)

CPU (:cpu): use image: ghcr.io/sudolulo/winnow:cpu, remove the deploy: block, and add mem_limit: 2g to prevent OOM on large libraries.

See compose.yml for the full annotated example with all options.

Scheduling

CRON_SCHEDULE controls container lifetime:

CRON_SCHEDULE value	Behaviour
(unset)	Run once on startup, then exit
(empty string)	Stay alive, run nothing — trigger manually with docker exec -it winnow winnow
Cron expression	Run on startup, then repeat on schedule

In scheduled mode the process (and loaded models) stays resident between runs. The first run after a fresh install downloads the embedding models (~1–2 GB); subsequent runs use the cached models from the mounted volume.

---

Environment Variables

Connection

Variable	Default	Description
IMMICH_URL	(required)	Full URL to your Immich instance
API_KEY	(required)	Immich API key
FRIGATE_URL	(unset)	Frigate URL — required for face upload; omit to skip

Mode & Strategy

Variable	Default	Description
TRAINING_MODE	face	face — upload crops to Frigate; object — save crops to disk
STRATEGY	auto	auto (embedding-based adaptive), standard (30 images), broad (100 images)
LIMIT	(unset)	Exact image count — overrides STRATEGY
OBJECT_CLASS	dog	Target class for object mode (any YOLO class: dog, cat, car, etc.)
AUTO_MODE	(auto)	Force non-interactive mode in a terminal; auto-detected otherwise
VERBOSE	false	Enable DEBUG-level console output (log file is always DEBUG)

People Filtering

Variable	Default	Description
ONLY_PEOPLE	(unset)	Comma-separated whitelist — process only these people
SKIP_PEOPLE	(unset)	Comma-separated list — skip these people
MIN_FACE_COUNT	3	Skip people with fewer than N tagged assets in Immich
MERGE_DUPLICATE_PEOPLE	false	When Immich has duplicate entries for the same person (same face split across multiple names), merge their asset pools before processing. Without this, each duplicate group emits a warning and is skipped
YEARS_FILTER	10	Ignore images older than N years

Image Quality

Variable	Default	Description
MIN_FACE_WIDTH	90	Minimum face crop width in pixels
FACE_MARGIN	0.15	Padding around bounding box crop (fraction of face size)
ENABLE_FACE_ALIGNMENT	true	Align to ArcFace 112×112 format using facial landmarks
USE_FULL_RESOLUTION	true	Download full-resolution originals rather than preview thumbnails
MIN_CONFIDENCE	0.7	Minimum Immich face detection confidence
BLUR_THRESHOLD	120.0	Laplacian variance threshold — lower accepts more blur
MAX_AUTO_IMAGES	80	Maximum training images per person in Frigate
QUALITY_REPLACEMENT	true	When at cap, swap a weaker tracked image for a better candidate. With Frigate scoring active, targets the most redundant image (highest pre-upload recognize score); otherwise uses blur score. Never touches manually added Frigate files. Set false to skip people at cap
FRIGATE_SCORE_CEILING	0.0	Skip uploads whose pre-upload Frigate recognize score exceeds this value — they are already well-covered. 0 disables; requires at least one prior run to have scores
ENABLE_FRIGATE_SCORES	true	Call Frigate's recognize endpoint pre-upload to store diversity scores used for quality replacement. Adds ~200 ms per upload. Disable to use blur-score replacement only

GPU & Models

Variable	Default	Description
FORCE_CPU	false	Disable GPU — fall back to CPU for all inference
OPENVINO_DEVICE	CPU	Intel variant only: set GPU to use Arc or iGPU; default runs on CPU
ENABLE_CACHE	true	Cache computed embeddings to disk (speeds up re-runs on the same library)
CACHE_DIR	.if_cache	Path for embedding cache and upload tracker files
HF_HOME	(system)	HuggingFace model cache path (SigLIP)
INSIGHTFACE_HOME	(system)	InsightFace model cache path (Buffalo_L)

Output

Variable	Default	Description
OUTPUT_DIR	./frigate_train	Directory for object-mode crops and the winnow.log file. In Docker, set this via the volume mount instead.

Tracker Overrides (one-shot — remove after use)

Variable	Default	Description
DRY_RUN	false	Preview selection without downloading or uploading
RETRY_REJECTED	false	Re-attempt assets previously rejected by Frigate
RESET_PERSON	(unset)	Set to a person's name to clear their upload history and delete their winnow-managed Frigate training files so the next run starts fresh. Set to * to reset all tracked people at once. Manually added Frigate files are never touched
TRACE_CROP_SIZE	(unset)	Debug: print all tracked crops whose width or height matches this pixel value, then exit

Scheduling

Variable	Default	Description
CRON_SCHEDULE	(unset)	Unset = run once and exit; empty = stay alive; cron expression = scheduled

---

Local Install

git clone https://github.com/sudolulo/winnow.git
cd winnow
uv sync
uv run winnow

Requires Python 3.13+ and uv. An NVIDIA, AMD, or Intel GPU is recommended — CPU mode works but embedding computation is slower.

When run with a terminal attached, winnow starts an interactive session: select which people to process and choose a strategy (auto, standard, broad, or a custom count) per person. Without a TTY — Docker, cron, or AUTO_MODE=true — it processes all people automatically using the configured defaults.

---

Requirements

• Immich v1.106+
• Frigate v0.16+ (face mode only — object mode has no Frigate dependency)
• GPU recommended: NVIDIA (CUDA), AMD (ROCm), or Intel (Arc / iGPU via OpenVINO)
• Python 3.13+

---

Getting Help

• GitHub Discussions — questions, setup help, and general discussion
• Wiki — setup guide, troubleshooting, and FAQ
• Issues — bugs and feature requests only

---

Attribution

Based on if_curator by Sebastian, licensed MIT.