Compare commits

...
3 Commits
Author SHA1 Message Date
flanandClaude Sonnet 4.6 7b4722d3f2 release: v0.2.3 — security fixes and CI improvements
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 03:53:16 +00:00
flanandClaude Sonnet 4.6 444932c369 fix: remove API key from plain-text config file, add workflow permissions
- config.py: stop writing API_KEY to .immich_config.json; direct users
  to .env instead. Resolves CodeQL py/clear-text-storage-sensitive-data.
- test.yml, lint.yml: add permissions: contents: read to satisfy
  actions/missing-workflow-permissions scanner.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 03:51:55 +00:00
flanandClaude Sonnet 4.6 8f2a6d3163 docs: move to GitHub wiki (https://github.com/sudolulo/winnow/wiki)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 03:48:53 +00:00
9 changed files with 25 additions and 242 deletions
+2
View File
@@ -9,6 +9,8 @@ on:
jobs:
lint:
runs-on: ubuntu-latest
permissions:
contents: read
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
steps:
+2
View File
@@ -9,6 +9,8 @@ on:
jobs:
test:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout code
uses: actions/checkout@v4
+12
View File
@@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.2.3] - 2026-06-12
### Fixed
- **Clear-text API key storage**: `config.py` no longer writes `API_KEY` to `.immich_config.json`. The key must come from an environment variable or `.env` file. Interactive mode now prints a tip directing users to `.env`. Resolves CodeQL `py/clear-text-storage-sensitive-data`.
### Changed
- **CI workflow permissions**: `test.yml` and `lint.yml` now declare `permissions: contents: read`, following least-privilege principle and resolving `actions/missing-workflow-permissions` scanner alerts.
- **CI lockfile race condition**: removed the `verify-lockfile` pre-job from `docker-publish.yml` and `release.yml`. The `update-lockfile.yml` bot maintains the lockfile; the verify step raced against it on the same push event and caused false failures. `release.yml` now runs `uv lock` inline so tag-triggered builds are always self-consistent.
- **Docs moved to wiki**: `docs/` folder removed from the repository. Setup, Troubleshooting, and FAQ pages are now at the [GitHub wiki](https://github.com/sudolulo/winnow/wiki).
## [0.2.2] - 2026-06-12
### Added
-63
View File
@@ -1,63 +0,0 @@
# FAQ
## Does winnow modify my Immich library?
No. winnow only reads from Immich (assets, people, face bounding boxes). It never writes back to Immich or deletes anything.
---
## How many images should I upload to Frigate?
The `auto` strategy decides this for you — it keeps selecting until adding more images would be redundant. In practice this is usually 20–60 per person. You can cap it with `MAX_AUTO_IMAGES` (default 80).
Quality and diversity matter far more than volume. 30 well-spread images outperform 200 from the same week.
---
## What's the difference between face mode and object mode?
- **Face mode**: Extracts and aligns face crops, uploads them directly to Frigate's face training API. This is for teaching Frigate to recognize specific people.
- **Object mode**: Runs YOLO detection on full images and saves crops of a target class (dog, cat, car, etc.) to disk. Frigate has no API for object training data, so you place them manually.
---
## Can I run it without Frigate?
Yes — in object mode, `FRIGATE_URL` is not used and crops are saved to the output volume. In face mode you need Frigate to receive the uploads, but you can use `DRY_RUN=true` to preview selection without uploading.
---
## How does auto-diversity mode work?
winnow computes a vector embedding for each candidate image (what the face/object actually looks like — angle, lighting, expression). It then clusters those embeddings and picks representatives that are maximally spread across the embedding space. It stops when the next-most-different image is already close to something already selected. See the README for the full pipeline.
---
## Does it support multiple people in one run?
Yes. By default it processes every named person in your Immich library. Use `ONLY_PEOPLE` to whitelist specific names or `SKIP_PEOPLE` to exclude them.
---
## What GPU is needed?
Any NVIDIA GPU with CUDA 12.x support. The models (InsightFace Buffalo_L + SigLIP) fit comfortably in 4 GB VRAM. CPU mode works but is significantly slower.
ARM builds (linux/arm64) use CPU-only — CUDA is not available on ARM.
---
## Does it work on Unraid / Proxmox / bare Docker?
Yes — the `compose.yml` uses standard Docker volume mounts. Replace the example paths with whatever absolute paths suit your setup.
---
## How do I update winnow?
```bash
docker compose pull
docker compose up -d
```
The `latest` tag on GHCR tracks the `main` branch. Pinning to a version tag (e.g. `ghcr.io/sudolulo/winnow:v0.2.0`) is recommended for stability.
-95
View File
@@ -1,95 +0,0 @@
# Setup Guide
## Prerequisites
- [Immich](https://immich.app) v1.106+ with face recognition enabled and people tagged
- [Frigate](https://frigate.video) v0.16+ (face mode only)
- Docker with the NVIDIA container toolkit (optional but strongly recommended)
---
## 1. Get your Immich API key
1. Open Immich → **Account Settings** → **API Keys**
2. Click **New API Key**, give it a name (e.g. `winnow`), copy the key
---
## 2. Get your Frigate URL
This is the base URL of your Frigate instance, e.g. `http://192.168.1.10:5000`. Only needed for face mode — omit it entirely if you're using object mode.
---
## 3. Deploy with Docker Compose
Copy [`compose.yml`](../compose.yml) and [`.env.example`](../.env.example) to a directory on your host:
```bash
mkdir winnow && cd winnow
curl -O https://raw.githubusercontent.com/sudolulo/winnow/main/compose.yml
curl -O https://raw.githubusercontent.com/sudolulo/winnow/main/.env.example
cp .env.example .env
```
Edit `.env` with your values:
```bash
IMMICH_URL=http://192.168.1.10:2283
API_KEY=your-immich-api-key
FRIGATE_URL=http://192.168.1.10:5000
```
Edit the volume paths in `compose.yml` to point to directories on your host where models, cache, and output crops should be stored:
```yaml
volumes:
- /your/path/to/models:/models
- /your/path/to/cache:/app/.if_cache
- /your/path/to/output:/app/frigate_train
```
These directories will be created automatically by Docker if they don't exist.
Start it:
```bash
docker compose up -d
```
Logs:
```bash
docker compose logs -f winnow
```
---
## 4. First run
On the first run, winnow downloads the embedding models (~1–2 GB) from HuggingFace and InsightFace. This happens once — subsequent runs use the cached models from your mounted volume and start immediately.
---
## 5. Scheduling
Set `CRON_SCHEDULE` in your `.env` to keep winnow running on a schedule:
```
CRON_SCHEDULE=0 3 * * 0 # Every Sunday at 3 AM
```
Without `CRON_SCHEDULE`, the container runs once and exits.
---
## GPU passthrough
To enable GPU acceleration, include the `deploy` block in `compose.yml` (already present in the example) and ensure the NVIDIA container toolkit is installed on your host:
```bash
# Verify GPU is accessible to Docker
docker run --rm --gpus all nvidia/cuda:12.9.2-base-ubuntu22.04 nvidia-smi
```
CPU mode works without any GPU setup — set `FORCE_CPU=true` to disable GPU explicitly.
-77
View File
@@ -1,77 +0,0 @@
# Troubleshooting
## Container exits immediately
Check logs:
```bash
docker compose logs winnow
```
Common causes:
- **Missing required env var** — `IMMICH_URL` or `API_KEY` not set
- **Cannot reach Immich** — check the URL and that Immich is running; use `http://` not `https://` unless you have TLS set up
---
## "No people found" / nothing processed
- Make sure Immich has completed face recognition and you have named people in your library
- `YEARS_FILTER` defaults to 10 years — increase it if your tagged photos are older
- `MIN_FACE_COUNT` skips people with few photos — lower or remove it
---
## Frigate upload fails
- Confirm `FRIGATE_URL` is reachable from inside the container: `docker exec winnow curl $FRIGATE_URL/api/stats`
- Check Frigate v0.16+ — older versions don't have the face training API
- Set `DRY_RUN=true` to verify selection without uploading
---
## Models fail to download
winnow downloads InsightFace and HuggingFace (SigLIP) models on first run.
- Ensure the container has internet access
- Confirm the model volume is mounted and writable
- If behind a proxy, set `HTTP_PROXY` / `HTTPS_PROXY` env vars
---
## Running on CPU (no GPU)
Set `FORCE_CPU=true`. Everything works but embedding computation is slower — expect several minutes per person instead of seconds.
If you have a GPU but it's not being used:
- Confirm the NVIDIA container toolkit is installed: `docker run --rm --gpus all nvidia/cuda:12.9.2-base-ubuntu22.04 nvidia-smi`
- Confirm the `deploy.resources.reservations.devices` block is present in `compose.yml`
---
## Same images uploaded every run
The upload tracker is stored in `CACHE_DIR` (`/app/.if_cache` by default). If this volume isn't persisted between runs, the tracker resets and images are re-uploaded.
Make sure `/app/.if_cache` is mounted to a persistent host path.
---
## Re-uploading a specific person
To clear the upload history for one person and start fresh:
```env
RESET_PERSON=John
```
Remove this after one run — it clears the history and then processes normally.
---
## Image quality issues
- **Too blurry**: Lower `BLUR_THRESHOLD` (default 100) — e.g. `50` accepts more blur
- **Face too small**: Lower `MIN_FACE_WIDTH` (default 50px)
- **Low confidence detections included**: Raise `MIN_CONFIDENCE` (default 0.7)
- **Rejected images being re-tried**: Set `RETRY_REJECTED=true` for one run
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "winnow"
version = "0.2.2"
version = "0.2.3"
description = "Immich to Frigate training sets"
license = "MIT"
requires-python = ">=3.12"
Generated
+1 -1
View File
@@ -2289,7 +2289,7 @@ wheels = [
[[package]]
name = "winnow"
version = "0.2.2"
version = "0.2.3"
source = { editable = "." }
dependencies = [
{ name = "croniter" },
+7 -5
View File
@@ -67,12 +67,11 @@ class _Config:
self.ENABLE_CACHE = os.getenv("ENABLE_CACHE", "false").lower() in ("true", "1", "yes")
self.CACHE_DIR = os.getenv("CACHE_DIR", ".if_cache")
# Fall back to config file for missing values
# Fall back to config file for non-sensitive values (API_KEY not stored here)
if CONFIG_FILE.exists():
try:
data = json.loads(CONFIG_FILE.read_text())
self.IMMICH_URL = self.IMMICH_URL or data.get("IMMICH_URL")
self.API_KEY = self.API_KEY or data.get("API_KEY")
if not os.getenv("OUTPUT_DIR"):
self.OUTPUT_DIR = data.get("OUTPUT_DIR", self.OUTPUT_DIR)
except (json.JSONDecodeError, OSError) as e:
@@ -84,13 +83,16 @@ class _Config:
cls._instance = None
def save(self) -> None:
"""Persist configuration to file."""
"""Persist non-sensitive configuration to file.
API_KEY is intentionally excluded — store it in .env or as an
environment variable instead of a plain-text config file.
"""
try:
CONFIG_FILE.write_text(
json.dumps(
{
"IMMICH_URL": self.IMMICH_URL,
"API_KEY": self.API_KEY,
"OUTPUT_DIR": self.OUTPUT_DIR,
},
indent=2,
@@ -113,8 +115,8 @@ class _Config:
if not self.API_KEY:
console.print("[yellow]Immich API Key not found.[/yellow]")
console.print("[dim]Tip: set API_KEY in your .env file to avoid re-entering it.[/dim]")
self.API_KEY = Prompt.ask("Enter Immich API Key", password=True)
self.save()
def validate(self) -> None:
"""Raise ValueError if required config is missing."""