diff --git a/README.md b/README.md index 0a30db1..9f2c891 100644 --- a/README.md +++ b/README.md @@ -1,191 +1,85 @@ -
+# if-curator-headless -# ๐Ÿ–ผ๏ธ if-curator -### Immich to Frigate Curator +Headless fork of [if-curator](https://github.com/ds-sebastian/if_curator) with automatic Frigate face training upload and Docker support. -[![Python](https://img.shields.io/badge/Python-3.12%2B-blue?style=for-the-badge&logo=python&logoColor=white)](https://python.org) -[![Immich](https://img.shields.io/badge/Immich-v1.106%2B-violet?style=for-the-badge)](https://immich.app) -[![Frigate](https://img.shields.io/badge/Frigate-Ready-green?style=for-the-badge)](https://frigate.video) +## What Changed -*A specialized tool to extract **high-quality, diverse** training images from your Immich library for Frigate's Face Recognition (ArcFace) and Object/State Classification models.* +Only `cli.py` was modified. Three functions were added, one was changed. All other modules (`config.py`, `immich_api.py`, `embeddings.py`, `diversity.py`, `image_processing.py`, `quality.py`, `cache.py`, `logging.py`) are identical to upstream. -
+### New functions in `cli.py` -> [!WARNING] -> **Regarding Object Classification** -> -> Frigate **does not support** uploading custom images for object classification training via the UI or API. -> This tool currently prepares the dataset (crops and categorizes images) for training external models (like YOLO) manually. +**`auto_configure(people)`** โ€” Non-interactive replacement for `interactive_configure()`. Fetches all named people from Immich, filters by env vars, and iterates through each without prompts. ---- +**`_resolve_strategy(strategy, has_embedding)`** โ€” Non-interactive replacement for `_get_strategy_choice()`. Maps `STRATEGY` env var to `(limit, selection_mode)` without prompting. -## โšก Why This Tool? +**`upload_to_frigate(jobs)`** โ€” POSTs face crops to Frigate's `/api/faces/train/{name}/classify` API. Skipped if `FRIGATE_URL` is not set. -> **"Diversity matters far more than volume."** โ€” *Frigate Developer Tips* +### Modified function: `main()` -Training AI models on "bulk" data is often harmful. If you feed the model 50 images from the same 10-second video clip, it learns to recognize the *lighting and background*, not the actual *face* or *object*. +- Added `AUTO_MODE` env var check to branch between `auto_configure()` and `interactive_configure()` +- Confirmation prompt skipped when `AUTO_MODE=true` +- `upload_to_frigate(jobs)` called after `execute_jobs()` -`if-curator` solves this using **AI-powered diversity selection** and **quality filtering**: - -| Mode | Embedding Model | Algorithm | -| :--- | :--- | :--- | -| **๐Ÿ‘ค Face** | InsightFace (ArcFace) | K-Medoids Clustering + FPS + Hard Example Weighting | -| **๐Ÿถ Object** | SigLIP (Vision Transformer) | K-Medoids Clustering + FPS | - -The pipeline uses **K-Medoids clustering** to guarantee coverage of every distinct "look", then fills the remaining budget with **Farthest Point Sampling (FPS)** biased toward **hard examples** (unusual angles, partial occlusions). An **adaptive threshold** stops selection automatically when adding more images becomes redundant. - ---- - -## โœจ Features - -### ๐ŸŽฏ Smart Selection -- **Auto Diversity [Recommended]**: Clusters images by visual similarity, selects representatives from each cluster, then fills with maximally-diverse picks until redundancy starts (capped at 80) -- **Standard (30 images)**: Balanced set using Smart Diversity -- **Broad (100 images)**: Extensive set using Smart Diversity -- **Custom Count**: You choose the limit - -### ๏ฟฝ Quality Filtering -Bad training data hurts ArcFace models. Images are automatically rejected if they are: -- **Blurry** โ€” Laplacian variance below threshold -- **Grayscale / IR** โ€” ArcFace is trained on color images only -- **Over/Underexposed** โ€” Washed-out or too dark to use -- **Low confidence** โ€” Partial or occluded face detections -- **Too small** โ€” Faces under 100px (configurable) lack features - -### ๐Ÿ‘ค Face Recognition Prep -- Uses **InsightFace** (ArcFace/Buffalo_L) embeddings on **face crops** (not full images โ€” avoids wrong-face in group photos) -- **Hard example prioritization** โ€” unusual angles, sunglasses, and low-confidence detections are biased for selection -- **Face alignment** via InsightFace landmarks (standard 112ร—112 ArcFace input) -- Downloads **full-resolution** originals for final crops (falls back to JPEG preview for HEIC/RAW) -- Configurable crop margin (default 15%) - -### ๐Ÿ“ฆ Object/State Classification Prep -- Uses **SigLIP** (Vision Transformer) embeddings for semantic diversity -- **YOLOv9c** to detect and crop specific objects (dogs, cars, etc.) -- Captures variation in poses, lighting, and backgrounds - -### โšก Performance -- **Concurrent thumbnail downloads** (8 parallel workers) -- **Batch-capable** SigLIP embeddings for GPU efficiency -- Optional **disk-based embedding cache** for faster re-runs -- **Multi-person batch mode** โ€” process multiple people in one session - -### ๐Ÿ“‹ Preview Before Download -After selection, a summary table shows what will be processed: -``` - ๐Ÿ“‹ Training Job Preview -โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“ -โ”ƒ Person โ”ƒ Mode โ”ƒ Images โ”ƒ Date Range โ”ƒ -โ”กโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ -โ”‚ Sebastian โ”‚ face โ”‚ 80 โ”‚ 2021-04-03 โ†’ 2026-02-18 โ”‚ -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ -``` - ---- - -## ๐Ÿš€ Installation - -### Prerequisites -- **Python 3.12+** -- **[uv](https://astral.sh/uv/)** (highly recommended) -- **Immich Server** (v1.106+) - -### Setup - -```bash -git clone -cd if-curator -uv sync -``` - -### ๐ŸŽ๏ธ GPU Support (Recommended) -For faster embedding computation, install with GPU extras: - -```bash -uv sync --extra gpu -``` -*Automatically detects CUDA (NVIDIA), ROCm (AMD), or MPS (macOS).* - ---- - -## ๐Ÿ’ป Usage - -```bash -uv run if-curator -``` - -### Interactive Flow -The tool will guide you through: -1. **Select Person** โ€” Choose from your Immich people (supports multi-person batch) -2. **Training Mode** โ€” Face (Recognition) or Object (Classification) -3. **Strategy** โ€” Auto, Standard, Broad, or Custom -4. **Preview** โ€” Review the selection summary before downloading -5. **Execute** โ€” Downloads and processes images with progress tracking - -```text -Using InsightFace (face embeddings) for diversity analysis... -Quality filtering removed 76 images. -Adaptive threshold: 0.1721 (median_dist=0.8605, fraction=0.2) -Clustering 223 embeddings into 20 groups (K-Medoids)... -Selected 20 cluster medoids as initial picks. -Selection complete: 80 images (0 hard examples with confidence < 0.85). -``` - ---- - -## ๐Ÿ› ๏ธ Configuration - -The tool prompts for your Immich URL and API Key on the first run and saves them to `.immich_config.json`. - -### Environment Variables +## New Environment Variables | Variable | Default | Description | | :--- | :--- | :--- | -| `IMMICH_URL` | โ€” | Full URL to Immich (e.g. `http://192.168.1.10:2283`) | -| `API_KEY` | โ€” | Your Immich API Key | -| `FORCE_CPU` | `false` | Disable GPU acceleration | -| `MIN_FACE_WIDTH` | `100` | Minimum face crop size (pixels) | -| `BLUR_THRESHOLD` | `100.0` | Laplacian variance threshold for blur detection | -| `MIN_CONFIDENCE` | `0.7` | Minimum Immich detection confidence | -| `MAX_AUTO_IMAGES` | `80` | Safety cap for auto-diversity mode | -| `FACE_MARGIN` | `0.15` | Crop margin around face (fraction) | -| `USE_FULL_RESOLUTION` | `true` | Download originals for final crops | -| `ENABLE_FACE_ALIGNMENT` | `true` | Align faces to ArcFace 112ร—112 format | -| `ENABLE_CACHE` | `false` | Cache embeddings to disk for faster re-runs | -| `CACHE_DIR` | `.if_cache` | Directory for embedding cache | +| `AUTO_MODE` | `false` | Enable headless mode | +| `FRIGATE_URL` | *(empty)* | Frigate URL for auto-upload (skipped if empty) | +| `TRAINING_MODE` | `face` | `face` or `object` | +| `STRATEGY` | `auto` | `auto`, `standard`, or `broad` | +| `SKIP_PEOPLE` | *(empty)* | Comma-separated people to skip | +| `ONLY_PEOPLE` | *(empty)* | Comma-separated people to process (whitelist) | +| `MIN_FACE_COUNT` | `3` | Skip people with fewer assets | +| `OBJECT_CLASS` | `dog` | Object class (only with `TRAINING_MODE=object`) | ---- +All original if-curator variables (`IMMICH_URL`, `API_KEY`, `FORCE_CPU`, `OUTPUT_DIR`, etc.) still work. See the [upstream README](https://github.com/ds-sebastian/if_curator) for those. -## ๐Ÿง  Technical Details +## Docker -### Models -- **InsightFace (Buffalo_L)** โ€” Face detection and embedding (ArcFace, 512-d) -- **SigLIP** โ€” Visual embeddings via `transformers` (google/siglip-base-patch16-224, 768-d) -- **YOLOv9c** โ€” Object detection for cropping +Upstream has no Dockerfile. This fork adds Docker support with NVIDIA GPU acceleration and TrueNAS-compatible user mapping. -### Algorithms -- **K-Medoids Clustering** โ€” Groups embeddings into k clusters using cosine distance, selecting actual data points (medoids) as cluster centers. Guarantees one representative from every distinct "look" -- **Farthest Point Sampling** โ€” After medoid selection, fills remaining budget by iteratively selecting the most distant point from the current set -- **Hard Example Weighting** โ€” Candidates with detection confidence < 0.85 get a 1.2โ€“1.5ร— distance boost, biasing selection toward challenging images (unusual angles, occlusions) -- **Adaptive Auto-Threshold** โ€” Computed as 20% of the median pairwise cosine distance; stops when the next-best image is too similar -- **Quality Filtering** โ€” Blur (Laplacian), grayscale/IR (channel comparison), exposure (histogram), confidence (Immich metadata) -- **Face Crop Embedding** โ€” Extracts the target person's face (using Immich bbox) before embedding, preventing wrong-face selection in group photos +### Dockerfile -### Architecture -``` -Immich API โ”€โ–บ Fetch Assets by Person โ”€โ–บ Time Filter - โ”‚ - Concurrent Thumbnail Download (8 workers) - โ”‚ - Quality Filtering (blur, IR, exposure...) - โ”‚ - Face Crop Extraction (bbox from Immich metadata) - โ”‚ - Compute Embeddings (InsightFace / SigLIP) - โ”‚ - K-Medoids Clustering โ†’ FPS + Hard Example Weighting - โ”‚ - Preview Summary Table - โ”‚ - Download Full-Res โ”€โ–บ Face Alignment โ”€โ–บ Save -``` +```dockerfile +FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04 + +ENV DEBIAN_FRONTEND=noninteractive + +RUN apt-get update && apt-get install -y --no-install-recommends \ + software-properties-common \ + && add-apt-repository ppa:deadsnakes/ppa -y \ + && apt-get update && apt-get install -y --no-install-recommends \ + python3.12 \ + python3.12-venv \ + python3.12-dev \ + libgl1 \ + libglib2.0-0 \ + libxext6 \ + git \ + curl \ + g++ \ + && rm -rf /var/lib/apt/lists/* \ + && ln -sf /usr/bin/python3.12 /usr/bin/python \ + && ln -sf /usr/bin/python3.12 /usr/bin/python3 + +RUN curl -LsSf https://astral.sh/uv/install.sh | sh +ENV PATH="/root/.local/bin:${PATH}" + +WORKDIR /app + +RUN git clone --depth 1 https://github.com/sudolulo/if_curator_headless.git . \ + && uv sync --extra gpu \ + && uv cache clean + +RUN groupadd -g 568 apps \ + && useradd -u 568 -g apps -m -s /bin/bash appuser \ + && chown -R appuser:apps /app + +USER appuser + +ENV FORCE_CPU=false \ + HF_HOME=/models/huggingface \ + INSIGHTFACE_HOME=/models/insightface + +ENTRYPOINT ["uv", "run", "if-curator"]