diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..9c708d4 --- /dev/null +++ b/.env.example @@ -0,0 +1,45 @@ +# ── Required ────────────────────────────────────────────────────────────────── +IMMICH_URL=http://192.168.1.10:2283 +API_KEY=your-immich-api-key +FRIGATE_URL=http://192.168.1.10:5000 + +# ── Mode & Strategy ─────────────────────────────────────────────────────────── +AUTO_MODE=true +# TRAINING_MODE: face = upload to Frigate face recognition API +# object = save crops to output dir for manual Frigate placement +TRAINING_MODE=face +# STRATEGY: auto = objective diversity (recommended), standard = 30 imgs, broad = 100 imgs +STRATEGY=auto +# LIMIT=50 # Custom image count; overrides STRATEGY preset +# OBJECT_CLASS=dog # Object label for object mode (e.g. dog, cat, car) + +# ── People Filtering ────────────────────────────────────────────────────────── +# ONLY_PEOPLE=John,Jane # Comma-separated; process only these people +# SKIP_PEOPLE=Unknown # Comma-separated; skip these people +# MIN_FACE_COUNT=5 # Skip people with fewer than N assets in Immich +# YEARS_FILTER=10 # Only include images from the last N years (default: 10) + +# ── Image Quality ───────────────────────────────────────────────────────────── +# MIN_FACE_WIDTH=50 # Minimum face width in pixels (default: 50) +# FACE_MARGIN=0.15 # Padding around face crop as fraction (default: 0.15) +# ENABLE_FACE_ALIGNMENT=true # Align face before cropping (default: true) +# USE_FULL_RESOLUTION=true # Use full-res images vs thumbnails (default: true) +# MIN_CONFIDENCE=0.7 # Minimum face detection confidence (default: 0.7) +# BLUR_THRESHOLD=100.0 # Laplacian blur threshold; lower = accept more blur (default: 100.0) +# MAX_AUTO_IMAGES=80 # Hard cap on auto-diversity selection (default: 80) + +# ── Caching & Models ────────────────────────────────────────────────────────── +FORCE_CPU=false +ENABLE_CACHE=true +CACHE_DIR=/app/.if_cache +HF_HOME=/models/huggingface +INSIGHTFACE_HOME=/models + +# ── Tracker overrides (one-shot — remove after use) ─────────────────────────── +# DRY_RUN=true # Preview selection without downloading/uploading +# RETRY_REJECTED=true # Re-attempt previously rejected images +# RESET_PERSON=John # Clear uploaded+rejected history for one person + +# ── Scheduling ──────────────────────────────────────────────────────────────── +# Cron expression (unset = run once and exit) +CRON_SCHEDULE=0 3 * * 0 # Every Sunday at 3 AM diff --git a/CHANGELOG.md b/CHANGELOG.md index 60c36b0..66c586d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [0.1.0] - 2026-03-03 ### Added -- Initial release of `if-curator` — Immich to Frigate training set curator +- Initial release of `winnow` — Immich to Frigate training set curator - **Face recognition prep**: InsightFace (ArcFace/Buffalo_L) embeddings on face crops for Frigate face recognition training - **Object/state classification prep**: SigLIP (Vision Transformer) embeddings with YOLOv9c object detection - **Smart diversity selection**: K-Medoids clustering + Farthest Point Sampling (FPS) with hard-example weighting diff --git a/README.md b/README.md index 8c0dd0c..d1a3522 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,7 @@ [![Publish Docker Image](https://github.com/sudolulo/winnow/actions/workflows/docker-publish.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/docker-publish.yml) [![Release](https://github.com/sudolulo/winnow/actions/workflows/release.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/release.yml) [![Lint](https://github.com/sudolulo/winnow/actions/workflows/lint.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/lint.yml) [![Test](https://github.com/sudolulo/winnow/actions/workflows/test.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/test.yml) +**Docs:** [Setup Guide](docs/setup.md) · [Troubleshooting](docs/troubleshooting.md) · [FAQ](docs/faq.md) + # winnow `winnow` pulls photos of people and objects from your [Immich](https://immich.app) library, selects the most diverse and highest-quality subset using AI embeddings, and delivers them as training data for [Frigate](https://frigate.video)'s face recognition and object classification models. diff --git a/compose.override.yml b/compose.override.yml index 0c3ebf9..85b35d0 100644 --- a/compose.override.yml +++ b/compose.override.yml @@ -1,8 +1,8 @@ services: - if-curator: + winnow: build: . network_mode: host volumes: - - /code/if-curator/models:/models - - /code/if-curator/embeddings:/app/.if_cache - - /code/if-curator/output:/app/output + - /code/winnow/models:/models + - /code/winnow/embeddings:/app/.if_cache + - /code/winnow/output:/app/frigate_train diff --git a/docs/faq.md b/docs/faq.md new file mode 100644 index 0000000..bd1d4af --- /dev/null +++ b/docs/faq.md @@ -0,0 +1,63 @@ +# FAQ + +## Does winnow modify my Immich library? + +No. winnow only reads from Immich (assets, people, face bounding boxes). It never writes back to Immich or deletes anything. + +--- + +## How many images should I upload to Frigate? + +The `auto` strategy decides this for you — it keeps selecting until adding more images would be redundant. In practice this is usually 20–60 per person. You can cap it with `MAX_AUTO_IMAGES` (default 80). + +Quality and diversity matter far more than volume. 30 well-spread images outperform 200 from the same week. + +--- + +## What's the difference between face mode and object mode? + +- **Face mode**: Extracts and aligns face crops, uploads them directly to Frigate's face training API. This is for teaching Frigate to recognize specific people. +- **Object mode**: Runs YOLO detection on full images and saves crops of a target class (dog, cat, car, etc.) to disk. Frigate has no API for object training data, so you place them manually. + +--- + +## Can I run it without Frigate? + +Yes — in object mode, `FRIGATE_URL` is not used and crops are saved to the output volume. In face mode you need Frigate to receive the uploads, but you can use `DRY_RUN=true` to preview selection without uploading. + +--- + +## How does auto-diversity mode work? + +winnow computes a vector embedding for each candidate image (what the face/object actually looks like — angle, lighting, expression). It then clusters those embeddings and picks representatives that are maximally spread across the embedding space. It stops when the next-most-different image is already close to something already selected. See the README for the full pipeline. + +--- + +## Does it support multiple people in one run? + +Yes. By default it processes every named person in your Immich library. Use `ONLY_PEOPLE` to whitelist specific names or `SKIP_PEOPLE` to exclude them. + +--- + +## What GPU is needed? + +Any NVIDIA GPU with CUDA 12.x support. The models (InsightFace Buffalo_L + SigLIP) fit comfortably in 4 GB VRAM. CPU mode works but is significantly slower. + +ARM builds (linux/arm64) use CPU-only — CUDA is not available on ARM. + +--- + +## Does it work on Unraid / Proxmox / bare Docker? + +Yes — the `compose.yml` uses standard Docker volume mounts. The TrueNAS paths in the example (`/mnt//...`) are just an example; replace them with whatever paths suit your setup. + +--- + +## How do I update winnow? + +```bash +docker compose pull +docker compose up -d +``` + +The `latest` tag on GHCR tracks the `main` branch. Pinning to a version tag (e.g. `ghcr.io/sudolulo/winnow:v0.2.0`) is recommended for stability. diff --git a/docs/setup.md b/docs/setup.md new file mode 100644 index 0000000..80316ca --- /dev/null +++ b/docs/setup.md @@ -0,0 +1,88 @@ +# Setup Guide + +## Prerequisites + +- [Immich](https://immich.app) v1.106+ with face recognition enabled and people tagged +- [Frigate](https://frigate.video) v0.16+ (face mode only) +- Docker with the NVIDIA container toolkit (optional but strongly recommended) + +--- + +## 1. Get your Immich API key + +1. Open Immich → **Account Settings** → **API Keys** +2. Click **New API Key**, give it a name (e.g. `winnow`), copy the key + +--- + +## 2. Get your Frigate URL + +This is the base URL of your Frigate instance, e.g. `http://192.168.1.10:5000`. Only needed for face mode — omit it entirely if you're using object mode. + +--- + +## 3. Deploy with Docker Compose + +Copy [`compose.yml`](../compose.yml) and [`.env.example`](../.env.example) to a directory on your host: + +```bash +mkdir winnow && cd winnow +curl -O https://raw.githubusercontent.com/sudolulo/winnow/main/compose.yml +curl -O https://raw.githubusercontent.com/sudolulo/winnow/main/.env.example +cp .env.example .env +``` + +Edit `.env` with your values: + +```bash +IMMICH_URL=http://192.168.1.10:2283 +API_KEY=your-immich-api-key +FRIGATE_URL=http://192.168.1.10:5000 +``` + +Edit the volume paths in `compose.yml` to match your storage layout (replace `/mnt/` with your actual path). + +Start it: + +```bash +docker compose up -d +``` + +Logs: + +```bash +docker compose logs -f winnow +``` + +--- + +## 4. First run + +On the first run, winnow downloads the embedding models (~1–2 GB) from HuggingFace and InsightFace. This happens once — subsequent runs use the cached models from your mounted volume and start immediately. + +--- + +## 5. Scheduling + +Set `CRON_SCHEDULE` in your `.env` to keep winnow running on a schedule: + +``` +CRON_SCHEDULE=0 3 * * 0 # Every Sunday at 3 AM +``` + +Without `CRON_SCHEDULE`, the container runs once and exits. + +--- + +## TrueNAS Scale + +The included `compose.yml` uses TrueNAS-style volume paths. Replace `` with your pool name: + +```yaml +volumes: + - /mnt/tank/winnow/models:/models + - /mnt/tank/winnow/embeddings:/app/.if_cache + - /mnt/tank/winnow/output:/app/frigate_train +``` + +GPU passthrough on TrueNAS requires the NVIDIA app to be installed from the TrueNAS catalog and the `deploy.resources.reservations.devices` block in `compose.yml` (already included). diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md new file mode 100644 index 0000000..4e391f4 --- /dev/null +++ b/docs/troubleshooting.md @@ -0,0 +1,77 @@ +# Troubleshooting + +## Container exits immediately + +Check logs: +```bash +docker compose logs winnow +``` + +Common causes: +- **Missing required env var** — `IMMICH_URL` or `API_KEY` not set +- **Cannot reach Immich** — check the URL and that Immich is running; use `http://` not `https://` unless you have TLS set up + +--- + +## "No people found" / nothing processed + +- Make sure Immich has completed face recognition and you have named people in your library +- `YEARS_FILTER` defaults to 10 years — increase it if your tagged photos are older +- `MIN_FACE_COUNT` skips people with few photos — lower or remove it + +--- + +## Frigate upload fails + +- Confirm `FRIGATE_URL` is reachable from inside the container: `docker exec winnow curl $FRIGATE_URL/api/stats` +- Check Frigate v0.16+ — older versions don't have the face training API +- Set `DRY_RUN=true` to verify selection without uploading + +--- + +## Models fail to download + +winnow downloads InsightFace and HuggingFace (SigLIP) models on first run. + +- Ensure the container has internet access +- Confirm the model volume is mounted and writable +- If behind a proxy, set `HTTP_PROXY` / `HTTPS_PROXY` env vars + +--- + +## Running on CPU (no GPU) + +Set `FORCE_CPU=true`. Everything works but embedding computation is slower — expect several minutes per person instead of seconds. + +If you have a GPU but it's not being used: +- Confirm the NVIDIA container toolkit is installed: `docker run --rm --gpus all nvidia/cuda:12.9.2-base-ubuntu22.04 nvidia-smi` +- Confirm the `deploy.resources.reservations.devices` block is present in `compose.yml` + +--- + +## Same images uploaded every run + +The upload tracker is stored in `CACHE_DIR` (`/app/.if_cache` by default). If this volume isn't persisted between runs, the tracker resets and images are re-uploaded. + +Make sure `/app/.if_cache` is mounted to a persistent host path. + +--- + +## Re-uploading a specific person + +To clear the upload history for one person and start fresh: + +```env +RESET_PERSON=John +``` + +Remove this after one run — it clears the history and then processes normally. + +--- + +## Image quality issues + +- **Too blurry**: Lower `BLUR_THRESHOLD` (default 100) — e.g. `50` accepts more blur +- **Face too small**: Lower `MIN_FACE_WIDTH` (default 50px) +- **Low confidence detections included**: Raise `MIN_CONFIDENCE` (default 0.7) +- **Rejected images being re-tried**: Set `RETRY_REJECTED=true` for one run