Update README.md
This commit is contained in:
@@ -1,191 +1,85 @@
|
|||||||
<div align="center">
|
# if-curator-headless
|
||||||
|
|
||||||
# 🖼️ if-curator
|
Headless fork of [if-curator](https://github.com/ds-sebastian/if_curator) with automatic Frigate face training upload and Docker support.
|
||||||
### Immich to Frigate Curator
|
|
||||||
|
|
||||||
[](https://python.org)
|
## What Changed
|
||||||
[](https://immich.app)
|
|
||||||
[](https://frigate.video)
|
|
||||||
|
|
||||||
*A specialized tool to extract **high-quality, diverse** training images from your Immich library for Frigate's Face Recognition (ArcFace) and Object/State Classification models.*
|
Only `cli.py` was modified. Three functions were added, one was changed. All other modules (`config.py`, `immich_api.py`, `embeddings.py`, `diversity.py`, `image_processing.py`, `quality.py`, `cache.py`, `logging.py`) are identical to upstream.
|
||||||
|
|
||||||
</div>
|
### New functions in `cli.py`
|
||||||
|
|
||||||
> [!WARNING]
|
**`auto_configure(people)`** — Non-interactive replacement for `interactive_configure()`. Fetches all named people from Immich, filters by env vars, and iterates through each without prompts.
|
||||||
> **Regarding Object Classification**
|
|
||||||
>
|
|
||||||
> Frigate **does not support** uploading custom images for object classification training via the UI or API.
|
|
||||||
> This tool currently prepares the dataset (crops and categorizes images) for training external models (like YOLO) manually.
|
|
||||||
|
|
||||||
---
|
**`_resolve_strategy(strategy, has_embedding)`** — Non-interactive replacement for `_get_strategy_choice()`. Maps `STRATEGY` env var to `(limit, selection_mode)` without prompting.
|
||||||
|
|
||||||
## ⚡ Why This Tool?
|
**`upload_to_frigate(jobs)`** — POSTs face crops to Frigate's `/api/faces/train/{name}/classify` API. Skipped if `FRIGATE_URL` is not set.
|
||||||
|
|
||||||
> **"Diversity matters far more than volume."** — *Frigate Developer Tips*
|
### Modified function: `main()`
|
||||||
|
|
||||||
Training AI models on "bulk" data is often harmful. If you feed the model 50 images from the same 10-second video clip, it learns to recognize the *lighting and background*, not the actual *face* or *object*.
|
- Added `AUTO_MODE` env var check to branch between `auto_configure()` and `interactive_configure()`
|
||||||
|
- Confirmation prompt skipped when `AUTO_MODE=true`
|
||||||
|
- `upload_to_frigate(jobs)` called after `execute_jobs()`
|
||||||
|
|
||||||
`if-curator` solves this using **AI-powered diversity selection** and **quality filtering**:
|
## New Environment Variables
|
||||||
|
|
||||||
| Mode | Embedding Model | Algorithm |
|
|
||||||
| :--- | :--- | :--- |
|
|
||||||
| **👤 Face** | InsightFace (ArcFace) | K-Medoids Clustering + FPS + Hard Example Weighting |
|
|
||||||
| **🐶 Object** | SigLIP (Vision Transformer) | K-Medoids Clustering + FPS |
|
|
||||||
|
|
||||||
The pipeline uses **K-Medoids clustering** to guarantee coverage of every distinct "look", then fills the remaining budget with **Farthest Point Sampling (FPS)** biased toward **hard examples** (unusual angles, partial occlusions). An **adaptive threshold** stops selection automatically when adding more images becomes redundant.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ✨ Features
|
|
||||||
|
|
||||||
### 🎯 Smart Selection
|
|
||||||
- **Auto Diversity [Recommended]**: Clusters images by visual similarity, selects representatives from each cluster, then fills with maximally-diverse picks until redundancy starts (capped at 80)
|
|
||||||
- **Standard (30 images)**: Balanced set using Smart Diversity
|
|
||||||
- **Broad (100 images)**: Extensive set using Smart Diversity
|
|
||||||
- **Custom Count**: You choose the limit
|
|
||||||
|
|
||||||
### � Quality Filtering
|
|
||||||
Bad training data hurts ArcFace models. Images are automatically rejected if they are:
|
|
||||||
- **Blurry** — Laplacian variance below threshold
|
|
||||||
- **Grayscale / IR** — ArcFace is trained on color images only
|
|
||||||
- **Over/Underexposed** — Washed-out or too dark to use
|
|
||||||
- **Low confidence** — Partial or occluded face detections
|
|
||||||
- **Too small** — Faces under 100px (configurable) lack features
|
|
||||||
|
|
||||||
### 👤 Face Recognition Prep
|
|
||||||
- Uses **InsightFace** (ArcFace/Buffalo_L) embeddings on **face crops** (not full images — avoids wrong-face in group photos)
|
|
||||||
- **Hard example prioritization** — unusual angles, sunglasses, and low-confidence detections are biased for selection
|
|
||||||
- **Face alignment** via InsightFace landmarks (standard 112×112 ArcFace input)
|
|
||||||
- Downloads **full-resolution** originals for final crops (falls back to JPEG preview for HEIC/RAW)
|
|
||||||
- Configurable crop margin (default 15%)
|
|
||||||
|
|
||||||
### 📦 Object/State Classification Prep
|
|
||||||
- Uses **SigLIP** (Vision Transformer) embeddings for semantic diversity
|
|
||||||
- **YOLOv9c** to detect and crop specific objects (dogs, cars, etc.)
|
|
||||||
- Captures variation in poses, lighting, and backgrounds
|
|
||||||
|
|
||||||
### ⚡ Performance
|
|
||||||
- **Concurrent thumbnail downloads** (8 parallel workers)
|
|
||||||
- **Batch-capable** SigLIP embeddings for GPU efficiency
|
|
||||||
- Optional **disk-based embedding cache** for faster re-runs
|
|
||||||
- **Multi-person batch mode** — process multiple people in one session
|
|
||||||
|
|
||||||
### 📋 Preview Before Download
|
|
||||||
After selection, a summary table shows what will be processed:
|
|
||||||
```
|
|
||||||
📋 Training Job Preview
|
|
||||||
┏━━━━━━━━━━━┳━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┓
|
|
||||||
┃ Person ┃ Mode ┃ Images ┃ Date Range ┃
|
|
||||||
┡━━━━━━━━━━━╇━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━┩
|
|
||||||
│ Sebastian │ face │ 80 │ 2021-04-03 → 2026-02-18 │
|
|
||||||
└───────────┴──────┴────────┴─────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 Installation
|
|
||||||
|
|
||||||
### Prerequisites
|
|
||||||
- **Python 3.12+**
|
|
||||||
- **[uv](https://astral.sh/uv/)** (highly recommended)
|
|
||||||
- **Immich Server** (v1.106+)
|
|
||||||
|
|
||||||
### Setup
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git clone <repository_url>
|
|
||||||
cd if-curator
|
|
||||||
uv sync
|
|
||||||
```
|
|
||||||
|
|
||||||
### 🏎️ GPU Support (Recommended)
|
|
||||||
For faster embedding computation, install with GPU extras:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
uv sync --extra gpu
|
|
||||||
```
|
|
||||||
*Automatically detects CUDA (NVIDIA), ROCm (AMD), or MPS (macOS).*
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 💻 Usage
|
|
||||||
|
|
||||||
```bash
|
|
||||||
uv run if-curator
|
|
||||||
```
|
|
||||||
|
|
||||||
### Interactive Flow
|
|
||||||
The tool will guide you through:
|
|
||||||
1. **Select Person** — Choose from your Immich people (supports multi-person batch)
|
|
||||||
2. **Training Mode** — Face (Recognition) or Object (Classification)
|
|
||||||
3. **Strategy** — Auto, Standard, Broad, or Custom
|
|
||||||
4. **Preview** — Review the selection summary before downloading
|
|
||||||
5. **Execute** — Downloads and processes images with progress tracking
|
|
||||||
|
|
||||||
```text
|
|
||||||
Using InsightFace (face embeddings) for diversity analysis...
|
|
||||||
Quality filtering removed 76 images.
|
|
||||||
Adaptive threshold: 0.1721 (median_dist=0.8605, fraction=0.2)
|
|
||||||
Clustering 223 embeddings into 20 groups (K-Medoids)...
|
|
||||||
Selected 20 cluster medoids as initial picks.
|
|
||||||
Selection complete: 80 images (0 hard examples with confidence < 0.85).
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🛠️ Configuration
|
|
||||||
|
|
||||||
The tool prompts for your Immich URL and API Key on the first run and saves them to `.immich_config.json`.
|
|
||||||
|
|
||||||
### Environment Variables
|
|
||||||
|
|
||||||
| Variable | Default | Description |
|
| Variable | Default | Description |
|
||||||
| :--- | :--- | :--- |
|
| :--- | :--- | :--- |
|
||||||
| `IMMICH_URL` | — | Full URL to Immich (e.g. `http://192.168.1.10:2283`) |
|
| `AUTO_MODE` | `false` | Enable headless mode |
|
||||||
| `API_KEY` | — | Your Immich API Key |
|
| `FRIGATE_URL` | *(empty)* | Frigate URL for auto-upload (skipped if empty) |
|
||||||
| `FORCE_CPU` | `false` | Disable GPU acceleration |
|
| `TRAINING_MODE` | `face` | `face` or `object` |
|
||||||
| `MIN_FACE_WIDTH` | `100` | Minimum face crop size (pixels) |
|
| `STRATEGY` | `auto` | `auto`, `standard`, or `broad` |
|
||||||
| `BLUR_THRESHOLD` | `100.0` | Laplacian variance threshold for blur detection |
|
| `SKIP_PEOPLE` | *(empty)* | Comma-separated people to skip |
|
||||||
| `MIN_CONFIDENCE` | `0.7` | Minimum Immich detection confidence |
|
| `ONLY_PEOPLE` | *(empty)* | Comma-separated people to process (whitelist) |
|
||||||
| `MAX_AUTO_IMAGES` | `80` | Safety cap for auto-diversity mode |
|
| `MIN_FACE_COUNT` | `3` | Skip people with fewer assets |
|
||||||
| `FACE_MARGIN` | `0.15` | Crop margin around face (fraction) |
|
| `OBJECT_CLASS` | `dog` | Object class (only with `TRAINING_MODE=object`) |
|
||||||
| `USE_FULL_RESOLUTION` | `true` | Download originals for final crops |
|
|
||||||
| `ENABLE_FACE_ALIGNMENT` | `true` | Align faces to ArcFace 112×112 format |
|
|
||||||
| `ENABLE_CACHE` | `false` | Cache embeddings to disk for faster re-runs |
|
|
||||||
| `CACHE_DIR` | `.if_cache` | Directory for embedding cache |
|
|
||||||
|
|
||||||
---
|
All original if-curator variables (`IMMICH_URL`, `API_KEY`, `FORCE_CPU`, `OUTPUT_DIR`, etc.) still work. See the [upstream README](https://github.com/ds-sebastian/if_curator) for those.
|
||||||
|
|
||||||
## 🧠 Technical Details
|
## Docker
|
||||||
|
|
||||||
### Models
|
Upstream has no Dockerfile. This fork adds Docker support with NVIDIA GPU acceleration and TrueNAS-compatible user mapping.
|
||||||
- **InsightFace (Buffalo_L)** — Face detection and embedding (ArcFace, 512-d)
|
|
||||||
- **SigLIP** — Visual embeddings via `transformers` (google/siglip-base-patch16-224, 768-d)
|
|
||||||
- **YOLOv9c** — Object detection for cropping
|
|
||||||
|
|
||||||
### Algorithms
|
### Dockerfile
|
||||||
- **K-Medoids Clustering** — Groups embeddings into k clusters using cosine distance, selecting actual data points (medoids) as cluster centers. Guarantees one representative from every distinct "look"
|
|
||||||
- **Farthest Point Sampling** — After medoid selection, fills remaining budget by iteratively selecting the most distant point from the current set
|
|
||||||
- **Hard Example Weighting** — Candidates with detection confidence < 0.85 get a 1.2–1.5× distance boost, biasing selection toward challenging images (unusual angles, occlusions)
|
|
||||||
- **Adaptive Auto-Threshold** — Computed as 20% of the median pairwise cosine distance; stops when the next-best image is too similar
|
|
||||||
- **Quality Filtering** — Blur (Laplacian), grayscale/IR (channel comparison), exposure (histogram), confidence (Immich metadata)
|
|
||||||
- **Face Crop Embedding** — Extracts the target person's face (using Immich bbox) before embedding, preventing wrong-face selection in group photos
|
|
||||||
|
|
||||||
### Architecture
|
```dockerfile
|
||||||
```
|
FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04
|
||||||
Immich API ─► Fetch Assets by Person ─► Time Filter
|
|
||||||
│
|
ENV DEBIAN_FRONTEND=noninteractive
|
||||||
Concurrent Thumbnail Download (8 workers)
|
|
||||||
│
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
Quality Filtering (blur, IR, exposure...)
|
software-properties-common \
|
||||||
│
|
&& add-apt-repository ppa:deadsnakes/ppa -y \
|
||||||
Face Crop Extraction (bbox from Immich metadata)
|
&& apt-get update && apt-get install -y --no-install-recommends \
|
||||||
│
|
python3.12 \
|
||||||
Compute Embeddings (InsightFace / SigLIP)
|
python3.12-venv \
|
||||||
│
|
python3.12-dev \
|
||||||
K-Medoids Clustering → FPS + Hard Example Weighting
|
libgl1 \
|
||||||
│
|
libglib2.0-0 \
|
||||||
Preview Summary Table
|
libxext6 \
|
||||||
│
|
git \
|
||||||
Download Full-Res ─► Face Alignment ─► Save
|
curl \
|
||||||
```
|
g++ \
|
||||||
|
&& rm -rf /var/lib/apt/lists/* \
|
||||||
|
&& ln -sf /usr/bin/python3.12 /usr/bin/python \
|
||||||
|
&& ln -sf /usr/bin/python3.12 /usr/bin/python3
|
||||||
|
|
||||||
|
RUN curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||||
|
ENV PATH="/root/.local/bin:${PATH}"
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
RUN git clone --depth 1 https://github.com/sudolulo/if_curator_headless.git . \
|
||||||
|
&& uv sync --extra gpu \
|
||||||
|
&& uv cache clean
|
||||||
|
|
||||||
|
RUN groupadd -g 568 apps \
|
||||||
|
&& useradd -u 568 -g apps -m -s /bin/bash appuser \
|
||||||
|
&& chown -R appuser:apps /app
|
||||||
|
|
||||||
|
USER appuser
|
||||||
|
|
||||||
|
ENV FORCE_CPU=false \
|
||||||
|
HF_HOME=/models/huggingface \
|
||||||
|
INSIGHTFACE_HOME=/models/insightface
|
||||||
|
|
||||||
|
ENTRYPOINT ["uv", "run", "if-curator"]
|
||||||
|
|||||||
Reference in New Issue
Block a user