flan/winnow
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'dev'

Browse Source
This commit is contained in:
flan
2026-06-14 18:05:00 +00:00
parent d12abbc543 a237983777
commit ea99ad10e3
20 changed files with 159 additions and 2155 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+18
View File
@@ -7,6 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.4.11] - 2026-06-14
### Removed
- **Object mode pipeline fully removed**: YOLO object detection, SigLIP image classification, `TRAINING_MODE`, and `OBJECT_CLASS` env vars are gone. Frigate has no training API for objects; the ~2 GB model stack (torch, torchvision, transformers, ultralytics) was dead weight.
- **Dead Immich embedding path removed**: `FaceData.embedding` field and the `immich_embedding` parameter to `get_embedding()` were never consumed by any caller. Both removed along with the NumPy import in `immich_api.py` that existed solely for that path.
- **Dead `mode` config key removed**: `"mode": "face"` was written into job config dicts in `jobs.py` but never read after object mode removal.
### Fixed
- **InsightFace `FutureWarning` suppressed in crop-alignment path**: the `insightface_app.get()` call in `image_processing.py` now wraps the same `warnings.catch_warnings()` suppressor already present in `embeddings.py`, preventing scikit-image deprecation noise in logs.
### Changed
- **Variant pyproject files synced to current state**: `pyproject-rocm.toml`, `pyproject-cpu.toml`, `pyproject-intel.toml` were at v0.2.13 and still listed torch/transformers/ultralytics. Updated to v0.4.11 and cleaned to face-only deps. Note: corresponding lockfiles (uv-rocm.lock, uv-cpu.lock, uv-intel.lock) need regeneration in their respective platform environments.
- **`MERGE_DUPLICATE_PEOPLE` documented**: README and wiki now explain the default warn-and-skip behaviour vs. setting `true` for a permanent Immich merge, with irreversibility callout.
- **Wiki fully updated**: all five wiki pages rewritten to remove object mode references, correct model size (~300 MB InsightFace vs former ~1–2 GB HuggingFace+InsightFace), fix default values (`MAX_AUTO_IMAGES` 80→20, `MIN_FACE_COUNT` 0→3), add `MERGE_DUPLICATE_PEOPLE` coverage, and update GPU verification commands for current ONNX provider API.
## [0.4.10] - 2026-06-14
### Changed
Dockerfile
+3 -3
View File
@@ -67,7 +67,7 @@ FROM base-${TARGETARCH}-${VARIANT} AS runtime
ARG VARIANT=gpu
ARG VERSION=dev
LABEL org.opencontainers.image.title="winnow" \
org.opencontainers.image.description="Selects diverse, high-quality photos from Immich as training data for Frigate face recognition and object classification." \
org.opencontainers.image.description="Selects diverse, high-quality photos from Immich as training data for Frigate face recognition." \
org.opencontainers.image.source="https://github.com/sudolulo/winnow" \
org.opencontainers.image.licenses="AGPL-3.0-or-later" \
org.opencontainers.image.version="${VERSION}"
@@ -116,7 +116,7 @@ https://repositories.intel.com/graphics/ubuntu jammy flex" \
fi
RUN groupadd -g 568 apps && useradd -u 568 -g apps -m -s /bin/bash appuser \
&& mkdir -p /models/.insightface /models/huggingface \
&& mkdir -p /models/.insightface \
&& chown -R appuser:apps /app /models
WORKDIR /app
@@ -124,7 +124,7 @@ USER appuser
# PYTHONPATH=/app makes the winnow package importable from the entry point script.
# uv sync builds the wheel before winnow/ is COPY'd, so site-packages has only
# the dist-info. Explicitly adding /app lets Python find winnow/__init__.py there.
ENV HF_HOME=/models/huggingface INSIGHTFACE_HOME=/models/.insightface PYTHONPATH=/app
ENV INSIGHTFACE_HOME=/models/.insightface PYTHONPATH=/app
HEALTHCHECK CMD test -f /app/entrypoint.sh || exit 1
ENTRYPOINT ["tini", "--", "/app/entrypoint.sh"]
README.md
+19 -31
View File
@@ -9,7 +9,7 @@
**Docs:** [Setup](https://github.com/sudolulo/winnow/wiki/Setup) · [Troubleshooting](https://github.com/sudolulo/winnow/wiki/Troubleshooting) · [FAQ](https://github.com/sudolulo/winnow/wiki/FAQ)
`winnow` pulls photos from your [Immich](https://immich.app) library, selects the most diverse and highest-quality subset using AI embeddings, and delivers them as training data for [Frigate](https://frigate.video)'s face recognition and object classification models.
`winnow` pulls photos from your [Immich](https://immich.app) library, selects the most diverse and highest-quality subset using AI embeddings, and delivers them as training data for [Frigate](https://frigate.video)'s face recognition.
The best Frigate training data is images you curate manually — photos taken specifically for recognition, in controlled conditions, uploaded directly through Frigate's UI. For people you can do that for, do it. winnow is for everyone else: people in your library you want Frigate to recognise but don't have dedicated training photos for. It mines your existing Immich library for the most diverse spread of real-world appearances and fills the gap.
@@ -39,8 +39,7 @@ Immich library
│
▼
4. Compute embeddings from the same preview thumbnails
• Faces → InsightFace (ArcFace / Buffalo_L) → 512-dim vector
• Objects → SigLIP (Vision Transformer) → 768-dim vector
• InsightFace (ArcFace / Buffalo_L) → 512-dim vector
│
▼
5. Near-duplicate removal — greedy cosine-distance pass drops burst shots
@@ -61,35 +60,25 @@ Immich library
7. Download full-resolution originals from Immich
│
▼
8. Crop and process
• Face mode: EXIF-corrected, landmark-aligned 112×112 crop (ArcFace format)
• Object mode: YOLOv9c detection → one crop per matched instance
8. Crop and process — EXIF-corrected, landmark-aligned 112×112 crop (ArcFace format)
│
▼
9. Deliver
• Face mode: upload crops to Frigate's face registration API
↳ below MAX_AUTO_IMAGES — upload, unless the novelty gate
(FRIGATE_SCORE_CEILING) determines the candidate is already
covered by the current training set
↳ at cap + QUALITY_REPLACEMENT=true — with Frigate scoring active,
swap the most redundant tracked image (highest pre-upload recognize
score) if the candidate is more novel (lower score); falling back to
blur-score comparison when no Frigate scores are available; manually
added files are never touched
↳ at cap + QUALITY_REPLACEMENT=false — skip this person
• Object mode: save crops to disk → place into your Frigate data directory
9. Deliver — upload crops to Frigate's face registration API
↳ below MAX_AUTO_IMAGES — upload, unless the novelty gate
(FRIGATE_SCORE_CEILING) determines the candidate is already
covered by the current training set
↳ at cap + QUALITY_REPLACEMENT=true — with Frigate scoring active,
swap the most redundant tracked image (highest pre-upload recognize
score) if the candidate is more novel (lower score); falling back to
blur-score comparison when no Frigate scores are available; manually
added files are never touched
↳ at cap + QUALITY_REPLACEMENT=false — skip this person
```
Uploaded and rejected asset IDs are persisted across runs. The same image is never processed twice; rejected assets are permanently skipped unless `RETRY_REJECTED=true`.
---
## Modes
**Face mode** (default) — extracts face crops using Immich's bounding box metadata, applies EXIF orientation correction, and aligns them to ArcFace's standard 112×112 format using 5-point facial landmarks. Crops are uploaded directly to Frigate's face registration API.
**Object mode** — runs each full-resolution image through YOLOv9c to detect instances of a target class (dog, cat, car, etc.), crops each detection, and saves it to the output directory. Frigate has no API for uploading object training data; place the crops into your Frigate data directory manually.
---
## Running in Docker
@@ -116,7 +105,7 @@ services:
- FRIGATE_URL=http://192.168.1.10:5000
- CRON_SCHEDULE=0 3 * * 0
volumes:
- /path/to/models:/models
- /path/to/models:/models # INSIGHTFACE_HOME — persists Buffalo_L model (~300 MB)
- /path/to/cache:/app/.if_cache
- /path/to/output:/app/frigate_train
deploy:
@@ -162,7 +151,7 @@ See [compose.yml](compose.yml) for the full annotated example with all options.
| *(empty string)* | Stay alive, run nothing — trigger manually with `docker exec -it winnow winnow` |
| Cron expression | Run on startup, then repeat on schedule |
In scheduled mode the process (and loaded models) stays resident between runs. The first run after a fresh install downloads the embedding models (~1–2 GB); subsequent runs use the cached models from the mounted volume.
In scheduled mode the process (and loaded models) stays resident between runs. The first run after a fresh install downloads InsightFace Buffalo_L (~300 MB); subsequent runs use the cached model from the mounted volume.
---
@@ -180,10 +169,8 @@ In scheduled mode the process (and loaded models) stays resident between runs. T
| Variable | Default | Description |
| :--- | :--- | :--- |
| `TRAINING_MODE` | `face` | `face` — upload crops to Frigate; `object` — save crops to disk |
| `STRATEGY` | `adaptive` | `adaptive` — embedding-based diversity selection, stops when candidates become redundant; `standard` — fixed 30 images; `broad` — fixed 100 images |
| `LIMIT` | *(unset)* | Exact image count — overrides `STRATEGY` |
| `OBJECT_CLASS` | `dog` | Target class for object mode (any YOLO class: `dog`, `cat`, `car`, etc.) |
| `AUTO_MODE` | *(auto)* | Skip interactive prompts and process all people unattended — auto-detected when no TTY is present (Docker, cron); set `true` to force in a terminal |
| `VERBOSE` | `false` | Enable DEBUG-level console output (log file is always DEBUG) |
@@ -197,6 +184,8 @@ In scheduled mode the process (and loaded models) stays resident between runs. T
| `MERGE_DUPLICATE_PEOPLE` | `false` | When Immich has duplicate entries for the same person (same face split across multiple names), merge their asset pools before processing. Without this, each duplicate group emits a warning and is skipped |
| `YEARS_FILTER` | `10` | Ignore images older than N years |
> **Duplicate people detection** — winnow warns at startup if the same name appears on multiple Immich person records (a common side-effect of Immich's face clustering creating separate pools for the same individual). By default (`false`) it logs the duplicates, keeps only the person with the most assets, and skips the rest — no data is changed. Set `MERGE_DUPLICATE_PEOPLE=true` to permanently merge each duplicate group inside Immich (the person with the most assets absorbs the others). **This modifies Immich and cannot be undone.** Only enable it once you've verified the duplicates are actually the same person.
### Image Quality
| Variable | Default | Description |
@@ -227,14 +216,13 @@ These defaults are tuned for Frigate's ArcFace requirements. winnow will warn on
| `OPENVINO_DEVICE` | `CPU` | Intel variant only: set `GPU` to use Arc or iGPU; default runs on CPU |
| `ENABLE_CACHE` | `true` | Cache computed embeddings to disk (speeds up re-runs on the same library) |
| `CACHE_DIR` | `.if_cache` | Path for embedding cache and upload tracker files |
| `HF_HOME` | *(system)* | HuggingFace model cache path (SigLIP) |
| `INSIGHTFACE_HOME` | *(system)* | InsightFace model cache path (Buffalo_L) |
### Output
| Variable | Default | Description |
| :--- | :--- | :--- |
| `OUTPUT_DIR` | `./frigate_train` | Directory for object-mode crops and the `winnow.log` file. In Docker, set this via the volume mount instead. |
| `OUTPUT_DIR` | `./frigate_train` | Directory where face crops are staged before upload and where `winnow.log` is written. In Docker, set this via the volume mount instead. |
### Tracker Overrides *(one-shot — remove after use)*
@@ -271,7 +259,7 @@ When run with a terminal attached, winnow starts an interactive session: select
## Requirements
- **Immich** v1.106+
- **Frigate** v0.16+ (face mode only — object mode has no Frigate dependency)
- **Frigate** v0.16+
- **GPU** recommended: NVIDIA (CUDA), AMD (ROCm), or Intel (Arc / iGPU via OpenVINO)
- **Python** 3.13+
compose.yml
+1 -6
View File
@@ -13,13 +13,9 @@ services:
# Set AUTO_MODE=true to force auto mode in an interactive terminal.
# To run interactively: docker exec -it winnow winnow
# - VERBOSE=true # Enable DEBUG-level console output
# TRAINING_MODE: face = upload to Frigate face recognition API
# object = save crops to output dir for manual Frigate placement
- TRAINING_MODE=face
# STRATEGY: auto = objective diversity (recommended), standard = 30 imgs, broad = 100 imgs
- STRATEGY=auto
# - LIMIT=50 # Custom image count; overrides STRATEGY preset
# - OBJECT_CLASS=dog # Object label for object mode (e.g. dog, cat, car)
# ── People Filtering ──────────────────────────────────────────────────
# - ONLY_PEOPLE=John,Jane # Comma-separated; process only these people
@@ -35,14 +31,13 @@ services:
# - USE_FULL_RESOLUTION=true # Use full-res images vs thumbnails (default: true)
# - MIN_CONFIDENCE=0.7 # Minimum face detection confidence (default: 0.7)
# - BLUR_THRESHOLD=100.0 # Laplacian blur threshold; lower = accept more blur (default: 100.0)
# - MAX_AUTO_IMAGES=80 # Hard cap on auto-diversity selection (default: 80)
# - MAX_AUTO_IMAGES=80 # Hard cap on auto-diversity selection (default: 20)
# ── Caching & Models ──────────────────────────────────────────────────
# - FORCE_CPU=true # Disable GPU, fall back to CPU
# - OPENVINO_DEVICE=GPU # Intel variant only: use Arc/iGPU instead of CPU (default: CPU)
# - ENABLE_CACHE=false # Disable embedding cache (default: true)
- CACHE_DIR=/app/.if_cache
- HF_HOME=/models/huggingface
- INSIGHTFACE_HOME=/models/.insightface
# ── Tracker overrides (one-shot, remove after use) ────────────────────
pyproject-cpu.toml
+3 -22
View File
@@ -1,7 +1,7 @@
[project]
name = "winnow"
version = "0.2.13"
description = "Immich to Frigate training sets"
version = "0.4.11"
description = "Selects diverse, high-quality photos from Immich as training data for Frigate face recognition."
license = "AGPL-3.0-or-later"
requires-python = ">=3.13"
authors = [{ name = "Holden Salomon", email = "holden@arch.fyi" }]
@@ -23,10 +23,6 @@ dependencies = [
"python-dotenv>=1.2.1",
"requests>=2.32.5",
"rich>=14.2.0",
"torch>=2.12.0",
"torchvision>=0.27.0",
"transformers>=5.12.0",
"ultralytics>=8.4.66",
]
[project.scripts]
@@ -34,25 +30,13 @@ winnow = "winnow.cli:main"
[project.urls]
Repository = "https://github.com/sudolulo/winnow"
Changelog = "https://github.com/sudolulo/winnow/blob/main/CHANGELOG.md"
[tool.uv]
required-environments = [
"sys_platform == 'linux' and platform_machine == 'x86_64'",
]
[tool.uv.sources]
torch = [
{ index = "pytorch-cpu", marker = "sys_platform == 'linux' and platform_machine == 'x86_64'" },
]
torchvision = [
{ index = "pytorch-cpu", marker = "sys_platform == 'linux' and platform_machine == 'x86_64'" },
]
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[dependency-groups]
dev = [
"pytest>=8.0",
@@ -81,9 +65,6 @@ numpy = "numpy"
onnxruntime = "onnxruntime"
requests = "requests"
rich = "rich"
torch = "torch"
transformers = "transformers"
ultralytics = "ultralytics"
[tool.pytest.ini_options]
testpaths = ["tests"]
pyproject-intel.toml
+3 -22
View File
@@ -1,7 +1,7 @@
[project]
name = "winnow"
version = "0.2.13"
description = "Immich to Frigate training sets"
version = "0.4.11"
description = "Selects diverse, high-quality photos from Immich as training data for Frigate face recognition."
license = "AGPL-3.0-or-later"
requires-python = ">=3.13"
authors = [{ name = "Holden Salomon", email = "holden@arch.fyi" }]
@@ -23,10 +23,6 @@ dependencies = [
"python-dotenv>=1.2.1",
"requests>=2.32.5",
"rich>=14.2.0",
"torch>=2.12.0",
"torchvision>=0.27.0",
"transformers>=5.12.0",
"ultralytics>=8.4.66",
]
[project.scripts]
@@ -34,6 +30,7 @@ winnow = "winnow.cli:main"
[project.urls]
Repository = "https://github.com/sudolulo/winnow"
Changelog = "https://github.com/sudolulo/winnow/blob/main/CHANGELOG.md"
[tool.uv]
conflicts = [
@@ -47,19 +44,6 @@ required-environments = [
"sys_platform == 'linux' and platform_machine == 'x86_64'",
]
[tool.uv.sources]
torch = [
{ index = "pytorch-cpu", marker = "sys_platform == 'linux' and platform_machine == 'x86_64'" },
]
torchvision = [
{ index = "pytorch-cpu", marker = "sys_platform == 'linux' and platform_machine == 'x86_64'" },
]
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[dependency-groups]
dev = [
"pytest>=8.0",
@@ -88,9 +72,6 @@ numpy = "numpy"
onnxruntime-openvino = "onnxruntime"
requests = "requests"
rich = "rich"
torch = "torch"
transformers = "transformers"
ultralytics = "ultralytics"
[tool.deptry.per_rule_ignores]
DEP002 = ["onnxruntime-openvino"]
pyproject-rocm.toml
+3 -21
View File
@@ -1,7 +1,7 @@
[project]
name = "winnow"
version = "0.2.13"
description = "Immich to Frigate training sets"
version = "0.4.11"
description = "Selects diverse, high-quality photos from Immich as training data for Frigate face recognition."
license = "AGPL-3.0-or-later"
requires-python = ">=3.13"
authors = [{ name = "Holden Salomon", email = "holden@arch.fyi" }]
@@ -23,10 +23,6 @@ dependencies = [
"python-dotenv>=1.2.1",
"requests>=2.32.5",
"rich>=14.2.0",
"torch>=2.5.0",
"torchvision>=0.20.0",
"transformers>=5.12.0",
"ultralytics>=8.4.66",
]
[project.scripts]
@@ -34,6 +30,7 @@ winnow = "winnow.cli:main"
[project.urls]
Repository = "https://github.com/sudolulo/winnow"
Changelog = "https://github.com/sudolulo/winnow/blob/main/CHANGELOG.md"
[tool.uv]
index-strategy = "unsafe-best-match"
@@ -48,18 +45,6 @@ required-environments = [
"sys_platform == 'linux' and platform_machine == 'x86_64'",
]
[tool.uv.sources]
torch = [
{ index = "pytorch-rocm63", marker = "sys_platform == 'linux' and platform_machine == 'x86_64'" },
]
torchvision = [
{ index = "pytorch-rocm63", marker = "sys_platform == 'linux' and platform_machine == 'x86_64'" },
]
[[tool.uv.index]]
name = "pytorch-rocm63"
url = "https://download.pytorch.org/whl/rocm6.3"
[dependency-groups]
dev = [
"pytest>=8.0",
@@ -88,9 +73,6 @@ numpy = "numpy"
onnxruntime-rocm = "onnxruntime"
requests = "requests"
rich = "rich"
torch = "torch"
transformers = "transformers"
ultralytics = "ultralytics"
[tool.deptry.per_rule_ignores]
DEP002 = ["onnxruntime-rocm"]
pyproject.toml
+2 -30
View File
@@ -1,7 +1,7 @@
[project]
name = "winnow"
version = "0.4.10"
description = "Selects diverse, high-quality photos from Immich as training data for Frigate face recognition and object classification."
version = "0.4.11"
description = "Selects diverse, high-quality photos from Immich as training data for Frigate face recognition."
license = "AGPL-3.0-or-later"
requires-python = ">=3.13"
authors = [{ name = "Holden Salomon", email = "holden@arch.fyi" }]
@@ -27,10 +27,6 @@ dependencies = [
"python-dotenv>=1.2.1",
"requests>=2.32.5",
"rich>=14.2.0",
"torch>=2.12.0",
"torchvision>=0.27.0",
"transformers>=5.12.0",
"ultralytics>=8.4.66",
]
[project.scripts]
@@ -53,27 +49,6 @@ required-environments = [
"sys_platform == 'linux' and platform_machine == 'aarch64'",
]
[tool.uv.sources]
torch = [
{ index = "pytorch-cu126", marker = "sys_platform == 'linux' and platform_machine == 'x86_64'" },
{ index = "pytorch-cpu", marker = "sys_platform == 'linux' and platform_machine == 'aarch64'" },
{ index = "pytorch-cpu", marker = "sys_platform != 'linux'" },
]
torchvision = [
{ index = "pytorch-cu126", marker = "sys_platform == 'linux' and platform_machine == 'x86_64'" },
{ index = "pytorch-cpu", marker = "sys_platform == 'linux' and platform_machine == 'aarch64'" },
{ index = "pytorch-cpu", marker = "sys_platform != 'linux'" },
]
[[tool.uv.index]]
name = "pytorch-cu126"
url = "https://download.pytorch.org/whl/cu126"
explicit = true
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[dependency-groups]
dev = [
@@ -104,9 +79,6 @@ nvidia-cudnn-cu12 = "nvidia.cudnn"
onnxruntime-gpu = "onnxruntime"
requests = "requests"
rich = "rich"
torch = "torch"
transformers = "transformers"
ultralytics = "ultralytics"
[tool.deptry.per_rule_ignores]
DEP002 = ["onnxruntime-gpu", "nvidia-cudnn-cu12"]
scheduler.py
-4
View File
@@ -16,7 +16,6 @@ except ImportError:
from winnow.cli import main
SCHEDULE = os.environ["CRON_SCHEDULE"]
MODELS_DIR = os.environ.get("HF_HOME", "/models/huggingface")
INSIGHTFACE_HOME = os.environ.get("INSIGHTFACE_HOME", "/models/.insightface")
logger = logging.getLogger(__name__)
@@ -24,11 +23,8 @@ logger = logging.getLogger(__name__)
def check_models() -> None:
buffalo = Path(INSIGHTFACE_HOME) / "models" / "buffalo_l"
hf_hub = Path(MODELS_DIR) / "hub"
if not buffalo.exists():
print(" InsightFace Buffalo_L not found — will download on first run", flush=True)
if not (hf_hub.exists() and any(hf_hub.iterdir())):
print(" HuggingFace models not found — will download on first run", flush=True)
NOW = time.time()
scripts/benchmark.py
+2 -66
View File
@@ -2,8 +2,8 @@
"""
winnow inference benchmark: GPU vs CPU throughput.
Measures InsightFace (face mode) and SigLIP (object mode) latency and
throughput. Run with FORCE_CPU=true for CPU-only baseline.
Measures InsightFace (ArcFace) latency and throughput.
Run with FORCE_CPU=true for CPU-only baseline.
Usage inside container:
# GPU mode:
@@ -47,11 +47,6 @@ def make_face_image(size: int = 640) -> Image.Image:
return img
def make_random_image(width: int = 224, height: int = 224) -> Image.Image:
rng = np.random.default_rng(42)
return Image.fromarray(rng.integers(0, 256, (height, width, 3), dtype=np.uint8), "RGB")
def _stats(times_s: list[float]) -> dict:
arr = np.array(times_s) * 1000 # ms
return {
@@ -119,61 +114,6 @@ def bench_insightface(n_warmup: int = 5, n_runs: int = 30) -> None:
print(f" 320×320 median : {s2['median_ms']:.1f} ms ({s2['ips']:.1f} img/s)")
def bench_siglip(
n_warmup: int = 3,
n_runs: int = 20,
batch_sizes: tuple = (1, 4, 8, 16, 32),
) -> None:
import torch
import winnow.embeddings as emb_mod
emb_mod._siglip_model = None
emb_mod._siglip_processor = None
emb_mod._siglip_loaded = False
print(" Loading model...")
t_load = time.perf_counter()
model, processor = emb_mod.get_siglip_model()
load_s = time.perf_counter() - t_load
if model is None:
print(" SKIP: SigLIP failed to load")
return
device = next(model.parameters()).device
print(f" Model load time : {load_s:.2f} s (device: {device})")
print(f" {'Batch':>5} {'ms/batch':>10} {'ms/img':>8} {'img/s':>8} {'p95/img':>9}")
for bs in batch_sizes:
imgs = [make_random_image(224, 224) for _ in range(bs)]
inputs = processor(images=imgs, return_tensors="pt")
inputs = {k: v.to(device) for k, v in inputs.items()}
# Warmup
for _ in range(n_warmup):
with torch.no_grad():
model(**inputs)
if str(device) != "cpu":
torch.cuda.synchronize()
times: list[float] = []
for _ in range(n_runs):
if str(device) != "cpu":
torch.cuda.synchronize()
t0 = time.perf_counter()
with torch.no_grad():
model(**inputs)
if str(device) != "cpu":
torch.cuda.synchronize()
times.append(time.perf_counter() - t0)
s = _stats(times)
print(
f" {bs:>5} {s['median_ms']:>10.1f} {s['median_ms']/bs:>8.2f}"
f" {bs * 1000 / s['median_ms']:>8.1f} {s['p95_ms']/bs:>9.2f}"
)
def main() -> None:
print("=" * 56)
print(" winnow inference benchmark")
@@ -185,10 +125,6 @@ def main() -> None:
bench_insightface()
print()
print("── SigLIP google/siglip-base-patch16-224 (objects) ───")
bench_siglip()
print()
if __name__ == "__main__":
# Add winnow to path when run directly inside container
uv.lock
Generated
+2 -1537
View File
File diff suppressed because it is too large Load Diff
winnow/__init__.py
+1 -2
View File
@@ -1,8 +1,7 @@
"""Immich to Frigate training set curator.
AI-powered tool to extract high-quality, diverse training images from your
Immich library for Frigate's Face Recognition (ArcFace) and Object/State
Classification models.
Immich library for Frigate's face recognition (ArcFace/Buffalo_L).
"""
from importlib.metadata import PackageNotFoundError, version
winnow/cache.py
-1
View File
@@ -15,7 +15,6 @@ logger = logging.getLogger(__name__)
# Model versions — bump these when the upstream model changes
MODEL_VERSIONS = {
"insightface": "buffalo_l_v1",
"siglip": "siglip-base-patch16-224_v1",
"immich": "immich_buffalo_l_v1",
}
winnow/diversity.py
+32 -55
View File
@@ -5,7 +5,7 @@ Selection pipeline:
1. Concurrent thumbnail download
2. Quality filtering (blur, IR, exposure, confidence, face size)
3. Face crop extraction (embed person's face, not full image)
4. Embedding computation (InsightFace or SigLIP)
4. Embedding computation (InsightFace)
5. Cluster-aware selection (K-Medoids + FPS with hard example weighting)
"""
@@ -28,7 +28,6 @@ def select_diverse_assets(
limit: int | str,
entity_name: str,
selection_mode: str = "smart",
entity_type: str = "face",
person_id: str | None = None,
progress_callback=None,
) -> list:
@@ -38,9 +37,8 @@ def select_diverse_assets(
Args:
assets: List of asset dicts from Immich API
limit: Number to select, or "auto" for dynamic selection
entity_name: Name of the person/object for logging
entity_name: Name of the person for logging
selection_mode: 'smart' (embedding-based) or 'time' (time spread)
entity_type: 'face' or 'object' - determines embedding model
progress_callback: Optional callback(current, total) for progress
Returns:
@@ -53,14 +51,13 @@ def select_diverse_assets(
# Sort by creation time
assets = sorted(assets, key=lambda x: x.get("fileCreatedAt", ""))
if selection_mode != "smart" or not is_embedding_available(entity_type):
if selection_mode != "smart" or not is_embedding_available():
if selection_mode == "smart":
model_name = "InsightFace" if entity_type == "face" else "SigLIP"
logger.warning(f"{model_name} unavailable. Falling back to time spread.")
logger.warning("InsightFace unavailable. Falling back to time spread.")
return _select_time_spread(assets, limit)
try:
return _select_by_embedding(assets, limit, entity_type, person_id, progress_callback)
return _select_by_embedding(assets, limit, person_id, progress_callback)
except Exception as e:
logger.error(f"Smart Diversity failed: {e}. Falling back to time spread.")
return _select_time_spread(assets, limit)
@@ -179,7 +176,6 @@ def _crop_face_from_thumbnail(
def _select_by_embedding(
assets: list,
limit: int | str,
entity_type: str,
person_id: str | None = None,
progress_callback=None,
) -> list:
@@ -188,7 +184,7 @@ def _select_by_embedding(
Pipeline:
1. Concurrent thumbnail download
2. Quality filtering
3. Face crop extraction (face mode only)
3. Face crop extraction
4. Embedding computation
5. Cluster-aware selection with hard example weighting
"""
@@ -243,28 +239,25 @@ def _select_by_embedding(
confidence = _get_face_confidence(asset, person_id=person_id)
if entity_type == "face":
face_bbox = _get_face_bbox(asset, person_id=person_id)
quality = assess_quality(
img,
face_bbox=face_bbox,
confidence=confidence,
blur_threshold=Config.BLUR_THRESHOLD,
min_face_px=Config.MIN_FACE_WIDTH,
min_confidence=Config.MIN_CONFIDENCE,
)
if not quality.passed:
quality_filtered += 1
logger.debug(f"Quality filtered {asset['id']}: {quality.reason}")
continue
face_bbox = _get_face_bbox(asset, person_id=person_id)
quality = assess_quality(
img,
face_bbox=face_bbox,
confidence=confidence,
blur_threshold=Config.BLUR_THRESHOLD,
min_face_px=Config.MIN_FACE_WIDTH,
min_confidence=Config.MIN_CONFIDENCE,
)
if not quality.passed:
quality_filtered += 1
logger.debug(f"Quality filtered {asset['id']}: {quality.reason}")
continue
asset["quality_score"] = quality.blur_score
face_crop = _crop_face_from_thumbnail(img, asset, person_id=person_id)
embed_img = face_crop if face_crop is not None else img
else:
embed_img = img
asset["quality_score"] = quality.blur_score
face_crop = _crop_face_from_thumbnail(img, asset, person_id=person_id)
embed_img = face_crop if face_crop is not None else img
emb = get_embedding(embed_img, entity_type, asset_id=asset["id"])
emb = get_embedding(embed_img, asset_id=asset["id"])
if emb is not None:
embeddings.append(emb)
valid_candidates.append(asset)
@@ -300,7 +293,6 @@ def _select_by_embedding(
embeddings,
valid_candidates,
limit,
entity_type=entity_type,
confidence_scores=confidence_scores,
)
@@ -429,11 +421,11 @@ def _kmedoids(dist_matrix: np.ndarray, k: int, max_iter: int = 50) -> tuple[list
# =============================================================================
def _compute_adaptive_threshold(emb_normed: np.ndarray, entity_type: str) -> float:
def _compute_adaptive_threshold(emb_normed: np.ndarray) -> float:
"""Compute adaptive FPS stop threshold based on actual embedding distribution.
Instead of a hardcoded threshold, samples pairwise distances and sets
the threshold as a fraction of the median pairwise distance.
the threshold as 20% of the median pairwise distance.
"""
n = len(emb_normed)
sample_size = min(200, n)
@@ -441,22 +433,14 @@ def _compute_adaptive_threshold(emb_normed: np.ndarray, entity_type: str) -> flo
indices = rng.choice(n, sample_size, replace=False) if n > sample_size else np.arange(n)
sample = emb_normed[indices]
# Compute pairwise cosine distances for the sample
pairwise = 1 - sample @ sample.T
upper_tri = pairwise[np.triu_indices(len(sample), k=1)]
if len(upper_tri) == 0:
return 0.05
median_dist = float(np.median(upper_tri))
threshold = max(0.05, median_dist * 0.20)
# Faces: 20% of median (tighter — want fewer, more distinct images)
# Objects: 10% of median (wider — want more diversity)
fraction = 0.20 if entity_type == "face" else 0.10
threshold = max(0.05, median_dist * fraction)
logger.debug(
f"Adaptive threshold: {threshold:.4f} "
f"(median_dist={median_dist:.4f}, fraction={fraction}, type={entity_type})"
)
logger.debug(f"Adaptive threshold: {threshold:.4f} (median_dist={median_dist:.4f})")
return threshold
@@ -464,7 +448,6 @@ def _cluster_aware_selection(
embeddings: list,
candidates: list,
limit: int | str,
entity_type: str = "face",
confidence_scores: list | None = None,
) -> list:
"""Two-stage selection: K-Medoids clustering → FPS with hard example weighting.
@@ -484,13 +467,13 @@ def _cluster_aware_selection(
# Build confidence weight array for hard example boosting
conf_array = np.ones(n)
if confidence_scores and entity_type == "face":
if confidence_scores:
for i, c in enumerate(confidence_scores):
if c is not None:
conf_array[i] = c
# Compute adaptive threshold for auto mode
auto_threshold = _compute_adaptive_threshold(emb_normed, entity_type) if limit == "auto" else 0.0
auto_threshold = _compute_adaptive_threshold(emb_normed) if limit == "auto" else 0.0
target = Config.MAX_AUTO_IMAGES if limit == "auto" else limit
# --- Stage 1: K-Medoids clustering ---
@@ -542,15 +525,9 @@ def _cluster_aware_selection(
min_dists = np.minimum(min_dists, dists_to_new)
min_dists[best_idx] = -np.inf
# Log hard example stats
if entity_type == "face":
selected_conf = [conf_array[i] for i in selected if conf_array[i] < 1.0]
hard_count = sum(1 for c in selected_conf if c < 0.85)
logger.info(
f"Selection complete: {len(selected)} images " f"({hard_count} hard examples with confidence < 0.85)."
)
else:
logger.info(f"Selection complete: {len(selected)} diverse images.")
selected_conf = [conf_array[i] for i in selected if conf_array[i] < 1.0]
hard_count = sum(1 for c in selected_conf if c < 0.85)
logger.info(f"Selection complete: {len(selected)} images ({hard_count} hard examples with confidence < 0.85).")
return [candidates[i] for i in selected]
winnow/embeddings.py
+18 -176
View File
@@ -1,8 +1,7 @@
"""
Unified embedding interface for faces and objects.
Embedding interface for face diversity selection.
- Faces: InsightFace (ArcFace/Buffalo_L) — or reuse from Immich
- Objects: SigLIP (Vision Transformer via transformers)
- Caching: Disk-based cache avoids recomputation on reruns
"""
@@ -42,12 +41,9 @@ def _suppress_output():
os.close(saved_err)
# Lazy-loaded singletons
# Lazy-loaded singleton
_insightface_app = None
_insightface_loaded = False
_siglip_model = None
_siglip_processor = None
_siglip_loaded = False
def _is_force_cpu() -> bool:
@@ -202,169 +198,33 @@ def get_face_embedding(img_pil: Image.Image) -> np.ndarray | None:
# =============================================================================
# SigLIP (Objects)
# =============================================================================
def get_siglip_model():
"""Singleton for SigLIP model and processor with GPU auto-detection."""
global _siglip_model, _siglip_processor, _siglip_loaded
if _siglip_loaded:
return _siglip_model, _siglip_processor
_siglip_loaded = True
try:
import warnings
import torch
from transformers import AutoImageProcessor, SiglipVisionModel
model_name = "google/siglip-base-patch16-224"
# Disk cache check — path derived from model_name using HuggingFace's slug convention
hf_home = os.environ.get("HF_HOME", os.path.join(os.path.expanduser("~"), ".cache", "huggingface"))
cache_slug = "models--" + model_name.replace("/", "--")
model_cache = Path(hf_home) / "hub" / cache_slug
if model_cache.exists() and any(model_cache.iterdir()):
logger.info(f"SigLIP {model_name}: found in model cache")
else:
logger.info(f"SigLIP {model_name}: not cached — downloading now (~380 MB)")
logger.info(f"SigLIP {model_name}: loading into memory...")
t0 = time.time()
with warnings.catch_warnings():
warnings.filterwarnings("ignore", category=FutureWarning)
warnings.filterwarnings("ignore", message=".*use_fast.*")
_siglip_processor = AutoImageProcessor.from_pretrained(model_name, use_fast=True)
_siglip_model = SiglipVisionModel.from_pretrained(model_name)
_siglip_model.eval()
# Move to GPU if available (ROCm builds expose torch.cuda.is_available() == True)
if not _is_force_cpu():
if torch.cuda.is_available():
_siglip_model = _siglip_model.cuda()
device_name = "CUDA GPU"
elif hasattr(torch, "xpu") and torch.xpu.is_available():
_siglip_model = _siglip_model.to("xpu")
device_name = "Intel XPU"
elif hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
_siglip_model = _siglip_model.to("mps")
device_name = "Apple MPS"
else:
device_name = "CPU"
else:
device_name = "CPU (FORCE_CPU)"
logger.info(f"SigLIP {model_name}: ready on {device_name} ({time.time() - t0:.1f}s)")
return _siglip_model, _siglip_processor
except ImportError as e:
logger.error(f"transformers/torch not installed: {e}")
return None, None
except Exception as e:
logger.error(f"Failed to load SigLIP: {e}")
return None, None
def get_object_embedding(img_pil: Image.Image) -> np.ndarray | None:
"""Get 768-dim SigLIP embedding for an image."""
model, processor = get_siglip_model()
if model is None:
return None
try:
import torch
inputs = processor(images=img_pil, return_tensors="pt")
device = next(model.parameters()).device
inputs = {k: v.to(device) for k, v in inputs.items()}
with torch.no_grad():
outputs = model(**inputs)
return outputs.pooler_output.squeeze().cpu().numpy()
except Exception as e:
logger.error(f"Error getting object embedding: {e}")
return None
def get_object_embeddings_batch(images: list[Image.Image]) -> list[np.ndarray | None]:
"""Get SigLIP embeddings for a batch of images (GPU-efficient)."""
model, processor = get_siglip_model()
if model is None:
return [None] * len(images)
try:
import torch
inputs = processor(images=images, return_tensors="pt", padding=True)
device = next(model.parameters()).device
inputs = {k: v.to(device) for k, v in inputs.items()}
with torch.no_grad():
outputs = model(**inputs)
embeddings = outputs.pooler_output.cpu().numpy()
return [embeddings[i] for i in range(len(embeddings))]
except Exception as e:
logger.error(f"Error in batch embedding: {e}")
# Fall back to individual computation
return [get_object_embedding(img) for img in images]
# =============================================================================
# Unified Interface with Caching
# Embedding Interface with Caching
# =============================================================================
def get_embedding(
img_pil: Image.Image,
entity_type: str = "face",
asset_id: str | None = None,
immich_embedding: np.ndarray | None = None,
) -> np.ndarray | None:
"""Get embedding for an image based on entity type.
"""Get embedding for a face image.
Priority:
1. Pre-fetched Immich embedding (if provided)
2. Disk cache (if enabled and asset_id provided)
3. Local model computation (InsightFace or SigLIP)
Args:
img_pil: The image to embed
entity_type: 'face' or 'object'
asset_id: Optional asset ID for cache lookup
immich_embedding: Optional pre-fetched embedding from Immich API
Checks disk cache first (if enabled and asset_id provided),
then falls back to local InsightFace computation.
"""
from .config import Config
use_cache = Config.ENABLE_CACHE and asset_id is not None
cache = get_cache(Config.CACHE_DIR) if use_cache else None
# Use a single consistent cache key per model so lookups and stores always match.
# "immich" was previously used as the face key on the lookup path but "insightface"
# on the store path — meaning the cache was never hit for locally-computed embeddings.
cache_key = "insightface" if entity_type == "face" else "siglip"
# 1. Use Immich embedding if provided
if immich_embedding is not None:
if cache:
cache.put(asset_id, immich_embedding, cache_key)
return immich_embedding
# 2. Check disk cache
if cache:
cached = cache.get(asset_id, cache_key)
cached = cache.get(asset_id, "insightface")
if cached is not None:
return cached
# 3. Compute locally
if entity_type == "face":
emb = get_face_embedding(img_pil)
else:
emb = get_object_embedding(img_pil)
emb = get_face_embedding(img_pil)
if emb is not None and cache:
cache.put(asset_id, emb, cache_key)
cache.put(asset_id, emb, "insightface")
return emb
@@ -378,36 +238,18 @@ def _is_module_available(module_name: str) -> bool:
return False
def is_embedding_available(entity_type: str = "face", *, load: bool = False) -> bool:
"""Check if embedding model is available for the given entity type.
def is_embedding_available(*, load: bool = False) -> bool:
"""Check if InsightFace is available.
By default this performs a lightweight import-check only (no model loading).
Pass ``load=True`` to actually load the model (expensive, hundreds of MB).
Args:
entity_type: 'face' or 'object'
load: If True, fully load the model to verify. If False (default),
only check that the required packages are importable.
Pass ``load=True`` to actually load the model (expensive, ~300 MB).
"""
if load:
if entity_type == "face":
return get_insightface_app() is not None
model, _ = get_siglip_model()
return model is not None
# Lightweight check: just verify the packages are importable
if entity_type == "face":
return _is_module_available("insightface") and _is_module_available("onnxruntime")
return _is_module_available("transformers") and _is_module_available("torch")
def load_embedding_model(entity_type: str = "face") -> bool:
"""Explicitly load the embedding model for the given entity type.
Returns True if the model loaded successfully.
"""
if entity_type == "face":
return get_insightface_app() is not None
model, _ = get_siglip_model()
return model is not None
return _is_module_available("insightface") and _is_module_available("onnxruntime")
def load_embedding_model() -> bool:
"""Explicitly load InsightFace. Returns True if the model loaded successfully."""
return get_insightface_app() is not None
winnow/executor.py
+30 -59
View File
@@ -19,7 +19,7 @@ from .frigate_api import (
get_frigate_person_files,
recognize_face,
)
from .image_processing import process_face_mode, process_full_mode, process_object_mode
from .image_processing import process_face_mode
from .immich_api import fetch_face_data, fetch_full_image
from .log_config import console
from .quality import assess_quality
@@ -173,11 +173,11 @@ def execute_jobs(jobs: list[dict]) -> None:
use_full_res = Config.USE_FULL_RESOLUTION
# Load InsightFace app for landmark-based crop alignment (face mode only).
# Load InsightFace app for landmark-based crop alignment.
# The model is already resident from the diversity/embedding phase, so this
# is just a singleton lookup — no load cost.
insightface_app = None
if any(j["config"].get("mode", "face") == "face" for j in jobs) and Config.ENABLE_FACE_ALIGNMENT:
if Config.ENABLE_FACE_ALIGNMENT:
try:
from .embeddings import get_insightface_app
@@ -196,8 +196,8 @@ def execute_jobs(jobs: list[dict]) -> None:
overall_task = progress.add_task("[green]Overall Progress", total=grand_total)
for job in jobs:
person, assets, config = job["person"], job["assets"], job["config"]
name, mode = person["name"], config.get("mode", "face")
person, assets = job["person"], job["assets"]
name = person["name"]
job_task = progress.add_task(f"Processing {name}...", total=len(assets))
try:
@@ -206,8 +206,7 @@ def execute_jobs(jobs: list[dict]) -> None:
logger.error(str(e))
continue
# Face crops are transient (uploaded then discarded); wipe before each run.
# Object crops are the deliverable; preserve them across runs.
if mode == "face" and os.path.isdir(person_dir):
if os.path.isdir(person_dir):
shutil.rmtree(person_dir)
os.makedirs(person_dir, exist_ok=True)
@@ -219,22 +218,21 @@ def execute_jobs(jobs: list[dict]) -> None:
count = 0
for asset in assets:
try:
# For face mode, enrich the asset with face bounding box data
# from the Immich faces API (not included in search/metadata results)
if mode == "face":
asset = _enrich_asset_with_face_data(asset, person)
# Skip download if detection confidence already disqualifies
# the asset — avoids fetching a large image we'll discard.
conf = asset.get("face_confidence")
if conf is not None and conf < Config.MIN_CONFIDENCE:
progress.console.print(
f"[yellow]Skipped {asset['id']}"
f" (detection confidence {conf:.2f} < {Config.MIN_CONFIDENCE})[/yellow]"
)
mark_rejected(asset["id"], person_name=name)
progress.advance(job_task)
progress.advance(overall_task)
continue
# Enrich the asset with face bounding box data from the Immich
# faces API (not included in search/metadata results).
asset = _enrich_asset_with_face_data(asset, person)
# Skip download if detection confidence already disqualifies
# the asset — avoids fetching a large image we'll discard.
conf = asset.get("face_confidence")
if conf is not None and conf < Config.MIN_CONFIDENCE:
progress.console.print(
f"[yellow]Skipped {asset['id']}"
f" (detection confidence {conf:.2f} < {Config.MIN_CONFIDENCE})[/yellow]"
)
mark_rejected(asset["id"], person_name=name)
progress.advance(job_task)
progress.advance(overall_task)
continue
# Use full-resolution for final output when configured
if use_full_res:
@@ -250,26 +248,21 @@ def execute_jobs(jobs: list[dict]) -> None:
if img is None:
progress.console.print(f"[red]Failed download {asset['id']}[/red]")
else:
saved = (
process_face_mode(img, asset, person, person_dir, count, insightface_app=insightface_app)
if mode == "face"
else process_object_mode(img, config, person_dir, count)
if mode == "object"
else process_full_mode(img, person_dir, count)
saved = process_face_mode(
img, asset, person, person_dir, count, insightface_app=insightface_app
)
if saved:
# Record which asset produced which output file
filename = f"{count}.jpg"
asset_map[filename] = asset["id"]
score_map[filename] = asset.get("quality_score")
if mode == "face" and isinstance(saved, tuple):
if isinstance(saved, tuple):
dims_map[filename] = saved
# Time-spread path: compute blur score from the downloaded
# image. Cap at 1440px so the scale matches the preview
# thumbnails the embedding path uses for scoring — Laplacian
# variance grows with resolution, making full-res and
# thumbnail scores incomparable if left uncapped.
if mode == "face" and score_map[filename] is None:
if score_map[filename] is None:
try:
score_img = img.convert("RGB") if img.mode != "RGB" else img
if score_img.width > 1440 or score_img.height > 1440:
@@ -279,12 +272,6 @@ def execute_jobs(jobs: list[dict]) -> None:
except Exception as exc:
logger.debug(f"Quality score fallback for {asset['id']}: {exc}")
score_map[filename] = 0.0 # unknown quality — treat as lowest
# Also record object-mode variant filenames
if mode == "object":
for f in sorted(os.listdir(person_dir)):
if f.startswith(f"{count}_") and f not in asset_map:
asset_map[f] = asset["id"]
score_map[f] = asset.get("face_confidence")
count += 1
else:
@@ -312,29 +299,13 @@ def execute_jobs(jobs: list[dict]) -> None:
def upload_to_frigate(jobs: list[dict]) -> None:
"""Upload processed face crops to Frigate via API with detailed logging.
Only runs for face-mode jobs. Object-mode crops are saved to the output
directory as the deliverable and must be copied to Frigate manually.
After each successful upload, records the Immich asset ID in the
upload tracker so it is skipped on future runs.
"""
face_jobs = [j for j in jobs if j["config"].get("mode", "face") == "face"]
if not face_jobs:
rprint("[dim]No face-mode jobs to upload.[/dim]")
if not jobs:
rprint("[dim]No jobs to upload.[/dim]")
return
# Notify user about object-mode jobs that were skipped
object_jobs = [j for j in jobs if j["config"].get("mode") == "object"]
for job in object_jobs:
name = job["person"]["name"]
try:
person_dir = _safe_person_dir(Config.OUTPUT_DIR, name)
except ValueError as e:
logger.error(str(e))
continue
rprint(f" [dim]📁 {name} (object): crops saved to {person_dir} — copy to Frigate manually[/dim]")
frigate_url = os.environ.get("FRIGATE_URL", "")
if not frigate_url:
rprint("[yellow]⚠️ FRIGATE_URL not set, skipping upload.[/yellow]")
@@ -347,7 +318,7 @@ def upload_to_frigate(jobs: list[dict]) -> None:
# from the asset_map stored on each job during execute_jobs()
filename_to_asset_id: dict[str, dict[str, str]] = {}
total_files = 0
for job in face_jobs:
for job in jobs:
name = job["person"]["name"]
asset_map = job.get("asset_map", {})
filename_to_asset_id[name] = asset_map
@@ -357,7 +328,7 @@ def upload_to_frigate(jobs: list[dict]) -> None:
rprint(" [yellow]No images found to upload.[/yellow]")
return
rprint(f" People: [bold]{len(face_jobs)}[/bold], Total images: [bold]{total_files}[/bold]")
rprint(f" People: [bold]{len(jobs)}[/bold], Total images: [bold]{total_files}[/bold]")
uploaded, failed = 0, 0
max_retries = 2
@@ -375,7 +346,7 @@ def upload_to_frigate(jobs: list[dict]) -> None:
) as progress:
upload_task = progress.add_task("[green]Uploading to Frigate", total=total_files)
for job in face_jobs:
for job in jobs:
name = job["person"]["name"]
# URL-encode the name for the API (handles spaces, special chars)
encoded_name = quote(name, safe="")
winnow/image_processing.py
+5 -61
View File
@@ -1,7 +1,8 @@
"""Image processing functions for cropping faces and objects."""
"""Image processing functions for cropping faces."""
import logging
import os
import warnings
import numpy as np
from PIL import Image
@@ -10,9 +11,6 @@ from .config import Config
logger = logging.getLogger(__name__)
# Lazy singleton
_yolo_model = None
def _save_jpeg(img: Image.Image, path: str) -> None:
if img.mode != "RGB":
@@ -20,17 +18,6 @@ def _save_jpeg(img: Image.Image, path: str) -> None:
img.save(path, format="JPEG")
def get_yolo_model():
"""Singleton for YOLO model."""
global _yolo_model
if _yolo_model is None:
from ultralytics import YOLO
logger.info("Loading YOLOv9c model...")
_yolo_model = YOLO("yolov9c.pt")
return _yolo_model
def align_face(img: Image.Image, landmarks: list[list[float]] | np.ndarray) -> Image.Image | None:
"""Align face using 5-point landmarks to standard ArcFace input format (112x112).
@@ -127,7 +114,9 @@ def process_face_mode(
min(img_h, y2 + pad_y),
)
search_crop = img.crop(search_box)
detected = insightface_app.get(np.asarray(search_crop))
with warnings.catch_warnings():
warnings.filterwarnings("ignore", message=".*estimate.*is deprecated", category=FutureWarning)
detected = insightface_app.get(np.asarray(search_crop))
if detected:
cx, cy = search_crop.width / 2, search_crop.height / 2
best = min(
@@ -170,49 +159,4 @@ def process_face_mode(
return face_crop.size
def process_object_mode(
img: Image.Image,
config: dict,
output_dir: str,
count: int,
) -> bool:
"""Detect and crop objects using YOLO."""
try:
model = get_yolo_model()
target_class = config.get("object_class", "dog")
import torch
if os.getenv("FORCE_CPU", "").lower() in ("true", "1", "yes"):
device = "cpu"
elif hasattr(torch, "xpu") and torch.xpu.is_available():
device = "xpu"
else:
device = None # YOLO auto-selects (CUDA/ROCm/CPU)
results = model(img, verbose=False, device=device)
found = False
class_idx = 0 # Sequential counter per target class (Issue #10)
for box in (box for r in results for box in r.boxes):
cls_id = int(box.cls[0])
conf = float(box.conf[0])
if 0 <= cls_id < len(model.names) and model.names[cls_id] == target_class and conf > 0.5:
x1, y1, x2, y2 = box.xyxy[0].tolist()
_save_jpeg(
img.crop((x1, y1, x2, y2)),
os.path.join(output_dir, f"{count}_{class_idx}.jpg"),
)
class_idx += 1
found = True
return found
except Exception as e:
logger.error(f"YOLO processing failed: {e}")
return False
def process_full_mode(img: Image.Image, output_dir: str, count: int) -> bool:
"""Save full image."""
_save_jpeg(img, os.path.join(output_dir, f"{count}.jpg"))
return True
winnow/immich_api.py
+2 -11
View File
@@ -5,7 +5,6 @@ from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from io import BytesIO
import numpy as np
import requests
from PIL import Image, ImageOps
@@ -21,7 +20,6 @@ _MAX_ASSETS_PER_PERSON = 5000 # Stop fetching after this many — diversity poo
class FaceData:
"""Pre-computed face data from Immich."""
embedding: np.ndarray | None
bbox: tuple[float, float, float, float] # (x1, y1, x2, y2)
confidence: float | None
image_width: int
@@ -110,7 +108,7 @@ def fetch_all_assets(person: dict) -> list[dict]:
def fetch_face_data(asset_id: str, person_id: str | None = None) -> FaceData | None:
"""Fetch pre-computed face data (embedding, bbox, confidence) from Immich.
"""Fetch pre-computed face data (bbox, confidence) from Immich.
Queries GET /api/faces?id={asset_id} to retrieve face detection results
that Immich already computed using InsightFace Buffalo_L.
@@ -120,7 +118,7 @@ def fetch_face_data(asset_id: str, person_id: str | None = None) -> FaceData | N
person_id: Optional person ID to match the specific face
Returns:
FaceData with embedding, bbox, and confidence, or None if unavailable
FaceData with bbox and confidence, or None if unavailable
"""
try:
resp = requests.get(
@@ -148,12 +146,6 @@ def fetch_face_data(asset_id: str, person_id: str | None = None) -> FaceData | N
if face is None:
face = faces[0] # Fall back to first/largest face
# Extract embedding if available
embedding = None
if "embedding" in face:
embedding = np.array(face["embedding"], dtype=np.float32)
# Extract bounding box
bbox = (
face.get("boundingBoxX1", 0),
face.get("boundingBoxY1", 0),
@@ -163,7 +155,6 @@ def fetch_face_data(asset_id: str, person_id: str | None = None) -> FaceData | N
score = face.get("score")
return FaceData(
embedding=embedding,
bbox=bbox,
confidence=score if score is not None else face.get("confidence"),
image_width=face.get("imageWidth", 0),
winnow/jobs.py
+15 -44
View File
@@ -26,10 +26,8 @@ STRATEGY_PRESETS = {
}
def _get_strategy_choice(has_embedding: bool, entity_type: str) -> tuple[int | str, str]:
def _get_strategy_choice(has_embedding: bool) -> tuple[int | str, str]:
"""Prompt user for training strategy and return (limit, selection_mode)."""
model_name = "InsightFace" if entity_type == "face" else "SigLIP"
if has_embedding:
rprint(" [bold]1.[/bold] Adaptive Diversity [green][Recommended][/green]")
rprint(" [dim]• Dynamically selects images until redundancy starts[/dim]")
@@ -51,7 +49,7 @@ def _get_strategy_choice(has_embedding: bool, entity_type: str) -> tuple[int | s
return 30, "smart"
# Fallback when embedding model not available
rprint(f" [yellow]Note: {model_name} not available. Using Time Spread.[/yellow]")
rprint(" [yellow]Note: InsightFace not available. Using Time Spread.[/yellow]")
rprint(" [bold]1.[/bold] Standard (30 images) [green][Recommended][/green]")
rprint(" [bold]2.[/bold] Broad (100 images)")
rprint(" [bold]3.[/bold] Custom Count")
@@ -86,15 +84,13 @@ def _resolve_strategy(strategy: str, has_embedding: bool) -> tuple[int | str, st
def _perform_selection(
assets: list, limit: int | str, name: str, selection_mode: str, entity_type: str, person_id: str | None = None
assets: list, limit: int | str, name: str, selection_mode: str, person_id: str | None = None
) -> list:
"""Run diversity selection with progress display."""
if selection_mode == "smart":
model_display = "InsightFace (face embeddings)" if entity_type == "face" else "SigLIP (visual embeddings)"
rprint(f"\n[cyan]Using {model_display} for diversity analysis...[/cyan]")
rprint("\n[cyan]Using InsightFace (face embeddings) for diversity analysis...[/cyan]")
# Pre-load model explicitly (separate from availability check)
load_embedding_model(entity_type)
load_embedding_model()
with Progress(
SpinnerColumn(),
@@ -109,7 +105,6 @@ def _perform_selection(
limit,
name,
selection_mode=selection_mode,
entity_type=entity_type,
person_id=person_id,
progress_callback=lambda c, t: progress.update(task, completed=c, total=t),
)
@@ -120,9 +115,7 @@ def _perform_selection(
rprint(f"\n[cyan]Using time-spread selection for {limit} images...[/cyan]")
with console.status(f"[bold]Selecting {limit} images evenly distributed over time...[/bold]"):
selected = select_diverse_assets(
assets, limit, name, selection_mode="time", entity_type=entity_type, person_id=person_id
)
selected = select_diverse_assets(assets, limit, name, selection_mode="time", person_id=person_id)
rprint(f" [green]Selected {len(selected)} images using time spread.[/green]")
return selected
@@ -132,22 +125,12 @@ def _configure_person(person: dict, people: list[dict]) -> dict | None:
name = person["name"]
console.print(f"\nSelected: [bold green]{name}[/bold green]")
# Select training mode
rprint("\n[bold cyan]Training Mode:[/bold cyan]")
rprint(" [bold]1.[/bold] Face (Frigate Face Recognition)")
rprint(" [bold]2.[/bold] Object (Frigate Object Classification)")
mode_choice = Prompt.ask("Choice", choices=["1", "2"], default="1")
entity_type = "face" if mode_choice == "1" else "object"
config = {"name": name, "mode": entity_type, "quality_replacement": Config.QUALITY_REPLACEMENT}
if entity_type == "object":
config["object_class"] = Prompt.ask("Enter Object Class (e.g. dog, cat, car)", default="dog")
config = {"name": name, "quality_replacement": Config.QUALITY_REPLACEMENT}
# Fetch and filter assets
years = IntPrompt.ask("Filter images older than (years)", default=Config.YEARS_FILTER)
console.print(f"Scanning for {name} ({entity_type})...")
console.print(f"Scanning for {name}...")
with console.status("[bold green]Fetching assets...[/bold green]"):
all_assets = fetch_all_assets(person)
recent_assets = filter_recent_assets(all_assets, years=years)
@@ -171,17 +154,15 @@ def _configure_person(person: dict, people: list[dict]) -> dict | None:
return None
# Strategy selection
has_embedding = is_embedding_available(entity_type)
has_embedding = is_embedding_available()
rprint(f"\n[bold cyan]Select Training Strategy for {name}:[/bold cyan]")
limit, selection_mode = _get_strategy_choice(has_embedding, entity_type)
limit, selection_mode = _get_strategy_choice(has_embedding)
if selection_mode == "skip":
return None
# Perform selection
selected_assets = _perform_selection(
recent_assets, limit, name, selection_mode, entity_type, person_id=person["id"]
)
selected_assets = _perform_selection(recent_assets, limit, name, selection_mode, person_id=person["id"])
rprint(f" [green]Queued {len(selected_assets)} images for {name}.[/green]")
return {"person": person, "assets": selected_assets, "limit": len(selected_assets), "config": config}
@@ -231,7 +212,6 @@ def auto_configure(people: list[dict]) -> list[dict]:
rprint("[red]No people found with names in Immich.[/red]")
return []
mode = os.environ.get("TRAINING_MODE", "face")
strategy = os.environ.get("STRATEGY", "auto")
skip = os.environ.get("SKIP_PEOPLE", "").split(",") if os.environ.get("SKIP_PEOPLE") else []
only = os.environ.get("ONLY_PEOPLE", "").split(",") if os.environ.get("ONLY_PEOPLE") else []
@@ -260,11 +240,7 @@ def auto_configure(people: list[dict]) -> list[dict]:
jobs = []
for person in valid_people:
name = person["name"]
entity_type = mode
config = {"name": name, "mode": entity_type}
if entity_type == "object":
config["object_class"] = os.environ.get("OBJECT_CLASS", "dog")
config = {"name": name}
all_assets = fetch_all_assets(person)
recent_assets = filter_recent_assets(all_assets, years=Config.YEARS_FILTER)
@@ -307,7 +283,7 @@ def auto_configure(people: list[dict]) -> list[dict]:
config["quality_replacement"] = quality_replacement_only or Config.QUALITY_REPLACEMENT
has_embedding = is_embedding_available(entity_type)
has_embedding = is_embedding_available()
limit, selection_mode = _resolve_strategy(strategy, has_embedding)
# Cap selection to remaining capacity (no cap when replacement-only — executor
@@ -323,9 +299,7 @@ def auto_configure(people: list[dict]) -> list[dict]:
if selection_mode == "skip":
continue
selected_assets = _perform_selection(
recent_assets, limit, name, selection_mode, entity_type, person_id=person["id"]
)
selected_assets = _perform_selection(recent_assets, limit, name, selection_mode, person_id=person["id"])
if auto_cap is not None:
selected_assets = selected_assets[:auto_cap]
@@ -340,20 +314,17 @@ def _show_preview(jobs: list[dict]) -> None:
"""Show a summary table of all queued jobs before execution."""
table = Table(title="📋 Training Job Preview", show_header=True, header_style="bold cyan")
table.add_column("Person", style="bold")
table.add_column("Mode", style="dim")
table.add_column("Images", justify="right")
table.add_column("Date Range", style="dim")
for job in jobs:
name = job["person"]["name"]
mode = job["config"].get("mode", "face")
count = str(job["limit"])
# Date range
dates = sorted(a.get("fileCreatedAt", "")[:10] for a in job["assets"] if a.get("fileCreatedAt"))
date_range = f"{dates[0]} → {dates[-1]}" if len(dates) >= 2 else (dates[0] if dates else "—")
table.add_row(name, mode, count, date_range)
table.add_row(name, count, date_range)
console.print()
console.print(table)
winnow/log_config.py
-4
View File
@@ -13,12 +13,9 @@ console = Console()
NOISY_LOGGERS = (
"urllib3",
"PIL",
"ultralytics",
"insightface",
"onnxruntime",
"matplotlib",
"transformers",
"torch",
)
@@ -51,7 +48,6 @@ def setup_logging(verbose: bool = False) -> logging.Logger:
# Suppress Python warnings from ML libraries
warnings.filterwarnings("ignore", category=UserWarning, module="onnxruntime")
warnings.filterwarnings("ignore", category=FutureWarning, module="transformers")
return root