# ๐ผ๏ธ if-curator
### Immich to Frigate Curator
[](https://python.org)
[](https://immich.app)
[](https://frigate.video)
*A specialized tool to extract **high-quality, diverse** training images from your Immich library for Frigate's Face Recognition (ArcFace) and Object/State Classification models.*
> [!WARNING]
> **Regarding Object Classification**
>
> Frigate **does not support** uploading custom images for object classification training via the UI or API.
> This tool currently prepares the dataset (crops and categorizes images) for training external models (like YOLO) manually.
---
## โก Why This Tool?
> **"Diversity matters far more than volume."** โ *Frigate Developer Tips*
Training AI models on "bulk" data is often harmful. If you feed the model 50 images from the same 10-second video clip, it learns to recognize the *lighting and background*, not the actual *face* or *object*.
`if-curator` solves this using **AI-powered diversity selection** and **quality filtering**:
| Mode | Embedding Model | Algorithm |
| :--- | :--- | :--- |
| **๐ค Face** | InsightFace (ArcFace) | K-Medoids Clustering + FPS + Hard Example Weighting |
| **๐ถ Object** | SigLIP (Vision Transformer) | K-Medoids Clustering + FPS |
The pipeline uses **K-Medoids clustering** to guarantee coverage of every distinct "look", then fills the remaining budget with **Farthest Point Sampling (FPS)** biased toward **hard examples** (unusual angles, partial occlusions). An **adaptive threshold** stops selection automatically when adding more images becomes redundant.
---
## โจ Features
### ๐ฏ Smart Selection
- **Auto Diversity [Recommended]**: Clusters images by visual similarity, selects representatives from each cluster, then fills with maximally-diverse picks until redundancy starts (capped at 80)
- **Standard (30 images)**: Balanced set using Smart Diversity
- **Broad (100 images)**: Extensive set using Smart Diversity
- **Custom Count**: You choose the limit
### ๏ฟฝ Quality Filtering
Bad training data hurts ArcFace models. Images are automatically rejected if they are:
- **Blurry** โ Laplacian variance below threshold
- **Grayscale / IR** โ ArcFace is trained on color images only
- **Over/Underexposed** โ Washed-out or too dark to use
- **Low confidence** โ Partial or occluded face detections
- **Too small** โ Faces under 100px (configurable) lack features
### ๐ค Face Recognition Prep
- Uses **InsightFace** (ArcFace/Buffalo_L) embeddings on **face crops** (not full images โ avoids wrong-face in group photos)
- **Hard example prioritization** โ unusual angles, sunglasses, and low-confidence detections are biased for selection
- **Face alignment** via InsightFace landmarks (standard 112ร112 ArcFace input)
- Downloads **full-resolution** originals for final crops (falls back to JPEG preview for HEIC/RAW)
- Configurable crop margin (default 15%)
### ๐ฆ Object/State Classification Prep
- Uses **SigLIP** (Vision Transformer) embeddings for semantic diversity
- **YOLOv9c** to detect and crop specific objects (dogs, cars, etc.)
- Captures variation in poses, lighting, and backgrounds
### โก Performance
- **Concurrent thumbnail downloads** (8 parallel workers)
- **Batch-capable** SigLIP embeddings for GPU efficiency
- Optional **disk-based embedding cache** for faster re-runs
- **Multi-person batch mode** โ process multiple people in one session
### ๐ Preview Before Download
After selection, a summary table shows what will be processed:
```
๐ Training Job Preview
โโโโโโโโโโโโโณโโโโโโโณโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Person โ Mode โ Images โ Date Range โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Sebastian โ face โ 80 โ 2021-04-03 โ 2026-02-18 โ
โโโโโโโโโโโโโดโโโโโโโดโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโ
```
---
## ๐ Installation
### Prerequisites
- **Python 3.12+**
- **[uv](https://astral.sh/uv/)** (highly recommended)
- **Immich Server** (v1.106+)
### Setup
```bash
git clone
cd if-curator
uv sync
```
### ๐๏ธ GPU Support (Recommended)
For faster embedding computation, install with GPU extras:
```bash
uv sync --extra gpu
```
*Automatically detects CUDA (NVIDIA), ROCm (AMD), or MPS (macOS).*
---
## ๐ป Usage
```bash
uv run if-curator
```
### Interactive Flow
The tool will guide you through:
1. **Select Person** โ Choose from your Immich people (supports multi-person batch)
2. **Training Mode** โ Face (Recognition) or Object (Classification)
3. **Strategy** โ Auto, Standard, Broad, or Custom
4. **Preview** โ Review the selection summary before downloading
5. **Execute** โ Downloads and processes images with progress tracking
```text
Using InsightFace (face embeddings) for diversity analysis...
Quality filtering removed 76 images.
Adaptive threshold: 0.1721 (median_dist=0.8605, fraction=0.2)
Clustering 223 embeddings into 20 groups (K-Medoids)...
Selected 20 cluster medoids as initial picks.
Selection complete: 80 images (0 hard examples with confidence < 0.85).
```
---
## ๐ ๏ธ Configuration
The tool prompts for your Immich URL and API Key on the first run and saves them to `.immich_config.json`.
### Environment Variables
| Variable | Default | Description |
| :--- | :--- | :--- |
| `IMMICH_URL` | โ | Full URL to Immich (e.g. `http://192.168.1.10:2283`) |
| `API_KEY` | โ | Your Immich API Key |
| `FORCE_CPU` | `false` | Disable GPU acceleration |
| `MIN_FACE_WIDTH` | `100` | Minimum face crop size (pixels) |
| `BLUR_THRESHOLD` | `100.0` | Laplacian variance threshold for blur detection |
| `MIN_CONFIDENCE` | `0.7` | Minimum Immich detection confidence |
| `MAX_AUTO_IMAGES` | `80` | Safety cap for auto-diversity mode |
| `FACE_MARGIN` | `0.15` | Crop margin around face (fraction) |
| `USE_FULL_RESOLUTION` | `true` | Download originals for final crops |
| `ENABLE_FACE_ALIGNMENT` | `true` | Align faces to ArcFace 112ร112 format |
| `ENABLE_CACHE` | `false` | Cache embeddings to disk for faster re-runs |
| `CACHE_DIR` | `.if_cache` | Directory for embedding cache |
---
## ๐ง Technical Details
### Models
- **InsightFace (Buffalo_L)** โ Face detection and embedding (ArcFace, 512-d)
- **SigLIP** โ Visual embeddings via `transformers` (google/siglip-base-patch16-224, 768-d)
- **YOLOv9c** โ Object detection for cropping
### Algorithms
- **K-Medoids Clustering** โ Groups embeddings into k clusters using cosine distance, selecting actual data points (medoids) as cluster centers. Guarantees one representative from every distinct "look"
- **Farthest Point Sampling** โ After medoid selection, fills remaining budget by iteratively selecting the most distant point from the current set
- **Hard Example Weighting** โ Candidates with detection confidence < 0.85 get a 1.2โ1.5ร distance boost, biasing selection toward challenging images (unusual angles, occlusions)
- **Adaptive Auto-Threshold** โ Computed as 20% of the median pairwise cosine distance; stops when the next-best image is too similar
- **Quality Filtering** โ Blur (Laplacian), grayscale/IR (channel comparison), exposure (histogram), confidence (Immich metadata)
- **Face Crop Embedding** โ Extracts the target person's face (using Immich bbox) before embedding, preventing wrong-face selection in group photos
### Architecture
```
Immich API โโบ Fetch Assets by Person โโบ Time Filter
โ
Concurrent Thumbnail Download (8 workers)
โ
Quality Filtering (blur, IR, exposure...)
โ
Face Crop Extraction (bbox from Immich metadata)
โ
Compute Embeddings (InsightFace / SigLIP)
โ
K-Medoids Clustering โ FPS + Hard Example Weighting
โ
Preview Summary Table
โ
Download Full-Res โโบ Face Alignment โโบ Save
```