Compare commits

..
10 Commits
Author SHA1 Message Date
flanandClaude Sonnet 4.6 0e416176c6 docs: clarify use case includes people who have never been to your property
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 02:14:26 +00:00
flanandClaude Sonnet 4.6 e856cba6d7 docs: explain use case for infrequent visitors
Adds context that winnow is especially useful for people who aren't
around enough for Frigate's live detections to supply adequate training
data on their own.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 02:14:16 +00:00
flanandClaude Sonnet 4.6 e4d5f603d8 docs: remove TrueNAS references, make volume paths platform agnostic
- compose.yml: generic /path/to/winnow/... placeholder paths
- docs/setup.md: replace TrueNAS section with generic GPU passthrough guide
- docs/faq.md: remove platform-specific path example
- README.md: soften crop quality disclaimer, remove TrueNAS compose note
- CHANGELOG.md: scrub TrueNAS mentions from release notes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 02:13:29 +00:00
flanandClaude Sonnet 4.6 0a67699db4 docs: add early stage disclaimer and manual review note
Advises users to review Frigate uploads after a run and remove bad
crops manually. Links to GitHub Issues for feedback.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 02:12:12 +00:00
flanandClaude Sonnet 4.6 f9b9cea84f docs: add lint fix to 0.2.1 changelog
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 02:11:35 +00:00
flanandClaude Sonnet 4.6 4b6bce9b74 fix: resolve ruff lint errors in scheduler and tests
- scheduler.py: sort import block (I001)
- tests/test_config.py: remove unused os, pytest imports (F401)
- tests/test_upload_tracker.py: remove unused json, os, tempfile imports (F401)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 02:11:16 +00:00
flanandClaude Sonnet 4.6 c3bf81d490 fix: direct venv python in entrypoint, fix INSIGHTFACE_HOME path, add badges
- entrypoint.sh: replace `uv run` with `/app/.venv/bin/python` to skip
  uv's sync check which was re-downloading ruff and rebuilding the package
  on every container startup
- compose.yml, .env.example: fix INSIGHTFACE_HOME /models → /models/.insightface;
  InsightFace appends models/ to root, so /models produced /models/models/buffalo_l
- README.md: add Immich and Frigate badges from upstream
- CHANGELOG.md, pyproject.toml, uv.lock: bump to 0.2.1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 02:10:01 +00:00
flanandClaude Sonnet 4.6 4529a63082 fix: add WORKDIR /app to runtime stage in Dockerfile
Runtime stage was missing WORKDIR, so uv run started from / and
couldn't find the .venv or pyproject.toml, causing "No module named
'winnow'" on container startup.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 02:01:32 +00:00
flanandClaude Sonnet 4.6 ec9d2ecd0f chore: remove compose.override.yml from repo, add to .gitignore
Local dev override with machine-specific paths and build context;
not useful to other users.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 02:01:32 +00:00
flan 1df7c99c2e Update README.md 2026-06-11 21:47:10 -04:00
15 changed files with 57 additions and 42 deletions
+1 -1
View File
@@ -33,7 +33,7 @@ FORCE_CPU=false
ENABLE_CACHE=true ENABLE_CACHE=true
CACHE_DIR=/app/.if_cache CACHE_DIR=/app/.if_cache
HF_HOME=/models/huggingface HF_HOME=/models/huggingface
INSIGHTFACE_HOME=/models INSIGHTFACE_HOME=/models/.insightface
# ── Tracker overrides (one-shot — remove after use) ─────────────────────────── # ── Tracker overrides (one-shot — remove after use) ───────────────────────────
# DRY_RUN=true # Preview selection without downloading/uploading # DRY_RUN=true # Preview selection without downloading/uploading
+1
View File
@@ -236,3 +236,4 @@ __marimo__/
# Streamlit # Streamlit
.streamlit/secrets.toml .streamlit/secrets.toml
compose.override.yml
+11 -3
View File
@@ -7,6 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
## [0.2.1] - 2026-06-12
### Fixed
- **Container startup reinstalling packages**: `entrypoint.sh` used `uv run`, which performs a sync check on every startup and re-downloaded `ruff` and rebuilt the package each time. Replaced with direct `.venv/bin/python` calls to skip the sync entirely.
- **InsightFace double `models/` path**: `INSIGHTFACE_HOME=/models` caused InsightFace to download Buffalo_L to `/models/models/buffalo_l` (InsightFace always appends `models/` to the root). Updated default in `compose.yml` and `.env.example` to `/models/.insightface`.
- **Lint errors in CI**: unused imports in `tests/test_config.py` and `tests/test_upload_tracker.py`, unsorted imports in `scheduler.py` — all would have failed the ruff CI check.
## [0.2.0] - 2026-06-12 ## [0.2.0] - 2026-06-12
First release of winnow. Forked from [if-curator](https://github.com/ds-sebastian/if_curator) by Sebastian and rewritten for headless Docker deployment. First release of winnow. Forked from [if-curator](https://github.com/ds-sebastian/if_curator) by Sebastian and rewritten for headless Docker deployment.
@@ -21,14 +29,14 @@ First release of winnow. Forked from [if-curator](https://github.com/ds-sebastia
**Docker and scheduling** **Docker and scheduling**
- `Dockerfile` — multi-stage build (CUDA 12.9 on amd64, plain Ubuntu on arm64); runtime stage excludes build tools (g++, python3.12-dev, curl, gnupg) - `Dockerfile` — multi-stage build (CUDA 12.9 on amd64, plain Ubuntu on arm64); runtime stage excludes build tools (g++, python3.12-dev, curl, gnupg)
- `compose.yml` — fully annotated with inline comments grouped by concern; TrueNAS volume paths in the example - `compose.yml` — fully annotated with inline comments grouped by concern
- `entrypoint.sh` — runs the tool once on startup, then hands off to the scheduler if `CRON_SCHEDULE` is set - `entrypoint.sh` — runs the tool once on startup, then hands off to the scheduler if `CRON_SCHEDULE` is set
- `scheduler.py` — in-process cron scheduler that keeps the container (and loaded models) alive between runs - `scheduler.py` — in-process cron scheduler that keeps the container (and loaded models) alive between runs
- `CRON_SCHEDULE` env var — standard cron expression for recurring runs; unset exits after first run - `CRON_SCHEDULE` env var — standard cron expression for recurring runs; unset exits after first run
- `.dockerignore` — keeps `.venv`, `__pycache__`, test files, and logs out of the image context - `.dockerignore` — keeps `.venv`, `__pycache__`, test files, and logs out of the image context
- Multi-arch image: `linux/amd64` and `linux/arm64` built and merged into a single manifest on GHCR - Multi-arch image: `linux/amd64` and `linux/arm64` built and merged into a single manifest on GHCR
- `tini` as PID 1 init process for correct signal handling - `tini` as PID 1 init process for correct signal handling
- Non-root container user (`appuser`, uid 568) matching TrueNAS default app UID - Non-root container user (`appuser`, uid 568)
- `HEALTHCHECK` in Dockerfile - `HEALTHCHECK` in Dockerfile
**Object mode** **Object mode**
@@ -73,7 +81,7 @@ First release of winnow. Forked from [if-curator](https://github.com/ds-sebastia
- 24 unit tests across four modules: `test_config`, `test_immich_api`, `test_jobs`, `test_upload_tracker` - 24 unit tests across four modules: `test_config`, `test_immich_api`, `test_jobs`, `test_upload_tracker`
**Documentation** **Documentation**
- `docs/setup.md` — step-by-step install guide with TrueNAS and bare-Docker sections - `docs/setup.md` — step-by-step install and GPU passthrough guide
- `docs/troubleshooting.md` — common failure modes with fixes - `docs/troubleshooting.md` — common failure modes with fixes
- `docs/faq.md` — answers to questions new users will ask - `docs/faq.md` — answers to questions new users will ask
- `.env.example` — copy-paste starting point with every env var and inline comments - `.env.example` — copy-paste starting point with every env var and inline comments
+1
View File
@@ -68,6 +68,7 @@ RUN groupadd -g 568 apps && useradd -u 568 -g apps -m -s /bin/bash appuser \
&& mkdir -p /models/.insightface /models/huggingface \ && mkdir -p /models/.insightface /models/huggingface \
&& chown -R appuser:apps /app /models && chown -R appuser:apps /app /models
WORKDIR /app
USER appuser USER appuser
ENV HF_HOME=/models/huggingface INSIGHTFACE_HOME=/models ENV HF_HOME=/models/huggingface INSIGHTFACE_HOME=/models
+13 -2
View File
@@ -1,4 +1,5 @@
[![Publish Docker Image](https://github.com/sudolulo/winnow/actions/workflows/docker-publish.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/docker-publish.yml) [![Release](https://github.com/sudolulo/winnow/actions/workflows/release.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/release.yml) [![Lint](https://github.com/sudolulo/winnow/actions/workflows/lint.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/lint.yml) [![Test](https://github.com/sudolulo/winnow/actions/workflows/test.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/test.yml) [![Publish Docker Image](https://github.com/sudolulo/winnow/actions/workflows/docker-publish.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/docker-publish.yml) [![Release](https://github.com/sudolulo/winnow/actions/workflows/release.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/release.yml) [![Lint](https://github.com/sudolulo/winnow/actions/workflows/lint.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/lint.yml) [![Test](https://github.com/sudolulo/winnow/actions/workflows/test.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/test.yml)
[![Immich](https://img.shields.io/badge/Immich-v1.106%2B-violet?style=for-the-badge)](https://immich.app) [![Frigate](https://img.shields.io/badge/Frigate-Ready-green?style=for-the-badge)](https://frigate.video)
**Docs:** [Setup Guide](docs/setup.md) · [Troubleshooting](docs/troubleshooting.md) · [FAQ](docs/faq.md) **Docs:** [Setup Guide](docs/setup.md) · [Troubleshooting](docs/troubleshooting.md) · [FAQ](docs/faq.md)
@@ -16,6 +17,8 @@ Frigate's face recognition model (ArcFace) and object classifier are only as goo
If you upload 100 photos from the same week, the model learns the lighting in your living room and the jacket you wore that month. It struggles the moment anything changes. What you actually want is a spread: different years, different lighting conditions, different angles, different contexts. If you upload 100 photos from the same week, the model learns the lighting in your living room and the jacket you wore that month. It struggles the moment anything changes. What you actually want is a spread: different years, different lighting conditions, different angles, different contexts.
This is especially true for people who have never been to your property, or who visit rarely — family members, friends, anyone Frigate has never seen in person. Live detections alone will never build a reliable model for these people. Your photo library already has the data; winnow finds and delivers the right subset of it.
Finding that spread manually across a library of thousands of photos is not practical. `winnow` does it automatically. Finding that spread manually across a library of thousands of photos is not practical. `winnow` does it automatically.
--- ---
@@ -68,6 +71,14 @@ Uploaded asset IDs are recorded so the same image is never uploaded twice, even
--- ---
## Note on Crop Quality
winnow works well, but no automated pipeline is perfect. Occasionally a bad crop will slip through quality filtering — a partial face, someone in the background, a blurry frame. After a run it's worth a quick review in Frigate's face management UI to remove anything that doesn't belong.
Issues and feedback welcome via [GitHub Issues](https://github.com/sudolulo/winnow/issues).
---
## Modes ## Modes
### Face Mode (default) ### Face Mode (default)
@@ -126,7 +137,7 @@ services:
capabilities: [gpu] capabilities: [gpu]
``` ```
See [compose.yml](compose.yml) for the full annotated example including TrueNAS volume paths. See [compose.yml](compose.yml) for the full annotated example.
### Scheduling Behaviour ### Scheduling Behaviour
@@ -227,4 +238,4 @@ Requires Python 3.12+ and [uv](https://astral.sh/uv/). An NVIDIA GPU is strongly
## Attribution ## Attribution
Based on [winnow](https://github.com/ds-sebastian/if_curator) by Sebastian, licensed MIT. Based on [if_curator](https://github.com/ds-sebastian/if_curator) by Sebastian, licensed MIT.
-8
View File
@@ -1,8 +0,0 @@
services:
winnow:
build: .
network_mode: host
volumes:
- /code/winnow/models:/models
- /code/winnow/embeddings:/app/.if_cache
- /code/winnow/output:/app/frigate_train
+5 -5
View File
@@ -38,7 +38,7 @@ services:
- ENABLE_CACHE=true - ENABLE_CACHE=true
- CACHE_DIR=/app/.if_cache - CACHE_DIR=/app/.if_cache
- HF_HOME=/models/huggingface - HF_HOME=/models/huggingface
- INSIGHTFACE_HOME=/models - INSIGHTFACE_HOME=/models/.insightface
# ── Tracker overrides (one-shot, remove after use) ──────────────────── # ── Tracker overrides (one-shot, remove after use) ────────────────────
# - DRY_RUN=true # Preview selection without downloading/uploading # - DRY_RUN=true # Preview selection without downloading/uploading
@@ -52,10 +52,10 @@ services:
# - CRON_SCHEDULE=0 3 1 * * # - CRON_SCHEDULE=0 3 1 * *
# - CRON_SCHEDULE=*/30 * * * * # - CRON_SCHEDULE=*/30 * * * *
volumes: volumes:
# Replace <pool> with your TrueNAS pool name, e.g. /mnt/tank/winnow/... # Replace with absolute paths on your host, e.g. /opt/winnow/models
- /mnt/<pool>/winnow/models:/models - /path/to/winnow/models:/models
- /mnt/<pool>/winnow/embeddings:/app/.if_cache - /path/to/winnow/cache:/app/.if_cache
- /mnt/<pool>/winnow/output:/app/frigate_train - /path/to/winnow/output:/app/frigate_train
stdin_open: true stdin_open: true
tty: true tty: true
restart: unless-stopped restart: unless-stopped
+1 -1
View File
@@ -49,7 +49,7 @@ ARM builds (linux/arm64) use CPU-only — CUDA is not available on ARM.
## Does it work on Unraid / Proxmox / bare Docker? ## Does it work on Unraid / Proxmox / bare Docker?
Yes — the `compose.yml` uses standard Docker volume mounts. The TrueNAS paths in the example (`/mnt/<pool>/...`) are just an example; replace them with whatever paths suit your setup. Yes — the `compose.yml` uses standard Docker volume mounts. Replace the example paths with whatever absolute paths suit your setup.
--- ---
+16 -9
View File
@@ -40,7 +40,16 @@ API_KEY=your-immich-api-key
FRIGATE_URL=http://192.168.1.10:5000 FRIGATE_URL=http://192.168.1.10:5000
``` ```
Edit the volume paths in `compose.yml` to match your storage layout (replace `/mnt/<pool>` with your actual path). Edit the volume paths in `compose.yml` to point to directories on your host where models, cache, and output crops should be stored:
```yaml
volumes:
- /your/path/to/models:/models
- /your/path/to/cache:/app/.if_cache
- /your/path/to/output:/app/frigate_train
```
These directories will be created automatically by Docker if they don't exist.
Start it: Start it:
@@ -74,15 +83,13 @@ Without `CRON_SCHEDULE`, the container runs once and exits.
--- ---
## TrueNAS Scale ## GPU passthrough
The included `compose.yml` uses TrueNAS-style volume paths. Replace `<pool>` with your pool name: To enable GPU acceleration, include the `deploy` block in `compose.yml` (already present in the example) and ensure the NVIDIA container toolkit is installed on your host:
```yaml ```bash
volumes: # Verify GPU is accessible to Docker
- /mnt/tank/winnow/models:/models docker run --rm --gpus all nvidia/cuda:12.9.2-base-ubuntu22.04 nvidia-smi
- /mnt/tank/winnow/embeddings:/app/.if_cache
- /mnt/tank/winnow/output:/app/frigate_train
``` ```
GPU passthrough on TrueNAS requires the NVIDIA app to be installed from the TrueNAS catalog and the `deploy.resources.reservations.devices` block in `compose.yml` (already included). CPU mode works without any GPU setup — set `FORCE_CPU=true` to disable GPU explicitly.
+2 -2
View File
@@ -4,13 +4,13 @@ export PYTHONUNBUFFERED=1
# 1. Run the job immediately on startup # 1. Run the job immediately on startup
echo "▶ Running on startup..." echo "▶ Running on startup..."
uv run python -m winnow.cli /app/.venv/bin/python -m winnow.cli
# 2. If a schedule exists, start the scheduler # 2. If a schedule exists, start the scheduler
if [ -n "${CRON_SCHEDULE:-}" ]; then if [ -n "${CRON_SCHEDULE:-}" ]; then
echo "▶ CRON_SCHEDULE set to: $CRON_SCHEDULE" echo "▶ CRON_SCHEDULE set to: $CRON_SCHEDULE"
echo "▶ Switching to scheduled mode..." echo "▶ Switching to scheduled mode..."
exec uv run python3 /app/scheduler.py exec /app/.venv/bin/python /app/scheduler.py
else else
echo "▶ No schedule set, exiting." echo "▶ No schedule set, exiting."
fi fi
+1 -1
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "winnow" name = "winnow"
version = "0.2.0" version = "0.2.1"
description = "Immich to Frigate training sets" description = "Immich to Frigate training sets"
license = "MIT" license = "MIT"
requires-python = ">=3.12" requires-python = ">=3.12"
+4 -4
View File
@@ -1,9 +1,9 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
import os
import sys
import subprocess
import time
import logging import logging
import os
import subprocess
import sys
import time
from pathlib import Path from pathlib import Path
try: try:
-2
View File
@@ -1,8 +1,6 @@
"""Smoke tests for configuration loading.""" """Smoke tests for configuration loading."""
import os
import pytest
def test_config_loads_defaults(monkeypatch): def test_config_loads_defaults(monkeypatch):
-3
View File
@@ -1,8 +1,5 @@
"""Tests for upload tracker — mark, filter, reset, and summary logic.""" """Tests for upload tracker — mark, filter, reset, and summary logic."""
import json
import os
import tempfile
import pytest import pytest
Generated
+1 -1
View File
@@ -2289,7 +2289,7 @@ wheels = [
[[package]] [[package]]
name = "winnow" name = "winnow"
version = "0.2.0" version = "0.2.1"
source = { editable = "." } source = { editable = "." }
dependencies = [ dependencies = [
{ name = "croniter" }, { name = "croniter" },