flan/winnow
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
flan
2026-06-10 15:48:08 -04:00
committed by GitHub
parent 9c1fbc1da2
commit a38122a5fe
1 changed files with 67 additions and 173 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+67 -173
View File
@@ -1,191 +1,85 @@
<div align="center">
# if-curator-headless
# 🖼️ if-curator
### Immich to Frigate Curator
Headless fork of [if-curator](https://github.com/ds-sebastian/if_curator) with automatic Frigate face training upload and Docker support.
[![Python](https://img.shields.io/badge/Python-3.12%2B-blue?style=for-the-badge&logo=python&logoColor=white)](https://python.org)
[![Immich](https://img.shields.io/badge/Immich-v1.106%2B-violet?style=for-the-badge)](https://immich.app)
[![Frigate](https://img.shields.io/badge/Frigate-Ready-green?style=for-the-badge)](https://frigate.video)
## What Changed
*A specialized tool to extract **high-quality, diverse** training images from your Immich library for Frigate's Face Recognition (ArcFace) and Object/State Classification models.*
Only `cli.py` was modified. Three functions were added, one was changed. All other modules (`config.py`, `immich_api.py`, `embeddings.py`, `diversity.py`, `image_processing.py`, `quality.py`, `cache.py`, `logging.py`) are identical to upstream.
</div>
### New functions in `cli.py`
> [!WARNING]
> **Regarding Object Classification**
>
> Frigate **does not support** uploading custom images for object classification training via the UI or API.
> This tool currently prepares the dataset (crops and categorizes images) for training external models (like YOLO) manually.
**`auto_configure(people)`** — Non-interactive replacement for `interactive_configure()`. Fetches all named people from Immich, filters by env vars, and iterates through each without prompts.
---
**`_resolve_strategy(strategy, has_embedding)`** — Non-interactive replacement for `_get_strategy_choice()`. Maps `STRATEGY` env var to `(limit, selection_mode)` without prompting.
## ⚡ Why This Tool?
**`upload_to_frigate(jobs)`** — POSTs face crops to Frigate's `/api/faces/train/{name}/classify` API. Skipped if `FRIGATE_URL` is not set.
> **"Diversity matters far more than volume."** — *Frigate Developer Tips*
### Modified function: `main()`
Training AI models on "bulk" data is often harmful. If you feed the model 50 images from the same 10-second video clip, it learns to recognize the *lighting and background*, not the actual *face* or *object*.
- Added `AUTO_MODE` env var check to branch between `auto_configure()` and `interactive_configure()`
- Confirmation prompt skipped when `AUTO_MODE=true`
- `upload_to_frigate(jobs)` called after `execute_jobs()`
`if-curator` solves this using **AI-powered diversity selection** and **quality filtering**:
| Mode | Embedding Model | Algorithm |
| :--- | :--- | :--- |
| **👤 Face** | InsightFace (ArcFace) | K-Medoids Clustering + FPS + Hard Example Weighting |
| **🐶 Object** | SigLIP (Vision Transformer) | K-Medoids Clustering + FPS |
The pipeline uses **K-Medoids clustering** to guarantee coverage of every distinct "look", then fills the remaining budget with **Farthest Point Sampling (FPS)** biased toward **hard examples** (unusual angles, partial occlusions). An **adaptive threshold** stops selection automatically when adding more images becomes redundant.
---
## ✨ Features
### 🎯 Smart Selection
- **Auto Diversity [Recommended]**: Clusters images by visual similarity, selects representatives from each cluster, then fills with maximally-diverse picks until redundancy starts (capped at 80)
- **Standard (30 images)**: Balanced set using Smart Diversity
- **Broad (100 images)**: Extensive set using Smart Diversity
- **Custom Count**: You choose the limit
### � Quality Filtering
Bad training data hurts ArcFace models. Images are automatically rejected if they are:
- **Blurry** — Laplacian variance below threshold
- **Grayscale / IR** — ArcFace is trained on color images only
- **Over/Underexposed** — Washed-out or too dark to use
- **Low confidence** — Partial or occluded face detections
- **Too small** — Faces under 100px (configurable) lack features
### 👤 Face Recognition Prep
- Uses **InsightFace** (ArcFace/Buffalo_L) embeddings on **face crops** (not full images — avoids wrong-face in group photos)
- **Hard example prioritization** — unusual angles, sunglasses, and low-confidence detections are biased for selection
- **Face alignment** via InsightFace landmarks (standard 112×112 ArcFace input)
- Downloads **full-resolution** originals for final crops (falls back to JPEG preview for HEIC/RAW)
- Configurable crop margin (default 15%)
### 📦 Object/State Classification Prep
- Uses **SigLIP** (Vision Transformer) embeddings for semantic diversity
- **YOLOv9c** to detect and crop specific objects (dogs, cars, etc.)
- Captures variation in poses, lighting, and backgrounds
### ⚡ Performance
- **Concurrent thumbnail downloads** (8 parallel workers)
- **Batch-capable** SigLIP embeddings for GPU efficiency
- Optional **disk-based embedding cache** for faster re-runs
- **Multi-person batch mode** — process multiple people in one session
### 📋 Preview Before Download
After selection, a summary table shows what will be processed:
```
📋 Training Job Preview
┏━━━━━━━━━━━┳━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Person ┃ Mode ┃ Images ┃ Date Range ┃
┡━━━━━━━━━━━╇━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Sebastian │ face │ 80 │ 2021-04-03 → 2026-02-18 │
└───────────┴──────┴────────┴─────────────────────────┘
```
---
## 🚀 Installation
### Prerequisites
- **Python 3.12+**
- **[uv](https://astral.sh/uv/)** (highly recommended)
- **Immich Server** (v1.106+)
### Setup
```bash
git clone <repository_url>
cd if-curator
uv sync
```
### 🏎️ GPU Support (Recommended)
For faster embedding computation, install with GPU extras:
```bash
uv sync --extra gpu
```
*Automatically detects CUDA (NVIDIA), ROCm (AMD), or MPS (macOS).*
---
## 💻 Usage
```bash
uv run if-curator
```
### Interactive Flow
The tool will guide you through:
1. **Select Person** — Choose from your Immich people (supports multi-person batch)
2. **Training Mode** — Face (Recognition) or Object (Classification)
3. **Strategy** — Auto, Standard, Broad, or Custom
4. **Preview** — Review the selection summary before downloading
5. **Execute** — Downloads and processes images with progress tracking
```text
Using InsightFace (face embeddings) for diversity analysis...
Quality filtering removed 76 images.
Adaptive threshold: 0.1721 (median_dist=0.8605, fraction=0.2)
Clustering 223 embeddings into 20 groups (K-Medoids)...
Selected 20 cluster medoids as initial picks.
Selection complete: 80 images (0 hard examples with confidence < 0.85).
```
---
## 🛠️ Configuration
The tool prompts for your Immich URL and API Key on the first run and saves them to `.immich_config.json`.
### Environment Variables
## New Environment Variables
| Variable | Default | Description |
| :--- | :--- | :--- |
| `IMMICH_URL` | — | Full URL to Immich (e.g. `http://192.168.1.10:2283`) |
| `API_KEY` | — | Your Immich API Key |
| `FORCE_CPU` | `false` | Disable GPU acceleration |
| `MIN_FACE_WIDTH` | `100` | Minimum face crop size (pixels) |
| `BLUR_THRESHOLD` | `100.0` | Laplacian variance threshold for blur detection |
| `MIN_CONFIDENCE` | `0.7` | Minimum Immich detection confidence |
| `MAX_AUTO_IMAGES` | `80` | Safety cap for auto-diversity mode |
| `FACE_MARGIN` | `0.15` | Crop margin around face (fraction) |
| `USE_FULL_RESOLUTION` | `true` | Download originals for final crops |
| `ENABLE_FACE_ALIGNMENT` | `true` | Align faces to ArcFace 112×112 format |
| `ENABLE_CACHE` | `false` | Cache embeddings to disk for faster re-runs |
| `CACHE_DIR` | `.if_cache` | Directory for embedding cache |
| `AUTO_MODE` | `false` | Enable headless mode |
| `FRIGATE_URL` | *(empty)* | Frigate URL for auto-upload (skipped if empty) |
| `TRAINING_MODE` | `face` | `face` or `object` |
| `STRATEGY` | `auto` | `auto`, `standard`, or `broad` |
| `SKIP_PEOPLE` | *(empty)* | Comma-separated people to skip |
| `ONLY_PEOPLE` | *(empty)* | Comma-separated people to process (whitelist) |
| `MIN_FACE_COUNT` | `3` | Skip people with fewer assets |
| `OBJECT_CLASS` | `dog` | Object class (only with `TRAINING_MODE=object`) |
---
All original if-curator variables (`IMMICH_URL`, `API_KEY`, `FORCE_CPU`, `OUTPUT_DIR`, etc.) still work. See the [upstream README](https://github.com/ds-sebastian/if_curator) for those.
## 🧠 Technical Details
## Docker
### Models
- **InsightFace (Buffalo_L)** — Face detection and embedding (ArcFace, 512-d)
- **SigLIP** — Visual embeddings via `transformers` (google/siglip-base-patch16-224, 768-d)
- **YOLOv9c** — Object detection for cropping
Upstream has no Dockerfile. This fork adds Docker support with NVIDIA GPU acceleration and TrueNAS-compatible user mapping.
### Algorithms
- **K-Medoids Clustering** — Groups embeddings into k clusters using cosine distance, selecting actual data points (medoids) as cluster centers. Guarantees one representative from every distinct "look"
- **Farthest Point Sampling** — After medoid selection, fills remaining budget by iteratively selecting the most distant point from the current set
- **Hard Example Weighting** — Candidates with detection confidence < 0.85 get a 1.2–1.5× distance boost, biasing selection toward challenging images (unusual angles, occlusions)
- **Adaptive Auto-Threshold** — Computed as 20% of the median pairwise cosine distance; stops when the next-best image is too similar
- **Quality Filtering** — Blur (Laplacian), grayscale/IR (channel comparison), exposure (histogram), confidence (Immich metadata)
- **Face Crop Embedding** — Extracts the target person's face (using Immich bbox) before embedding, preventing wrong-face selection in group photos
### Dockerfile
### Architecture
```
Immich API ─► Fetch Assets by Person ─► Time Filter
│
Concurrent Thumbnail Download (8 workers)
│
Quality Filtering (blur, IR, exposure...)
│
Face Crop Extraction (bbox from Immich metadata)
│
Compute Embeddings (InsightFace / SigLIP)
│
K-Medoids Clustering → FPS + Hard Example Weighting
│
Preview Summary Table
│
Download Full-Res ─► Face Alignment ─► Save
```
```dockerfile
FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update &amp;&amp; apt-get install -y --no-install-recommends \
software-properties-common \
&amp;&amp; add-apt-repository ppa:deadsnakes/ppa -y \
&amp;&amp; apt-get update &amp;&amp; apt-get install -y --no-install-recommends \
python3.12 \
python3.12-venv \
python3.12-dev \
libgl1 \
libglib2.0-0 \
libxext6 \
git \
curl \
g++ \
&amp;&amp; rm -rf /var/lib/apt/lists/* \
&amp;&amp; ln -sf /usr/bin/python3.12 /usr/bin/python \
&amp;&amp; ln -sf /usr/bin/python3.12 /usr/bin/python3
RUN curl -LsSf https://astral.sh/uv/install.sh | sh
ENV PATH=&quot;/root/.local/bin:${PATH}&quot;
WORKDIR /app
RUN git clone --depth 1 https://github.com/sudolulo/if_curator_headless.git . \
&amp;&amp; uv sync --extra gpu \
&amp;&amp; uv cache clean
RUN groupadd -g 568 apps \
&amp;&amp; useradd -u 568 -g apps -m -s /bin/bash appuser \
&amp;&amp; chown -R appuser:apps /app
USER appuser
ENV FORCE_CPU=false \
HF_HOME=/models/huggingface \
INSIGHTFACE_HOME=/models/insightface
ENTRYPOINT [&quot;uv&quot;, &quot;run&quot;, &quot;if-curator&quot;]