flan/winnow
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: full wiki audit and update for 0.4.1 

Troubleshooting.md:
- Fix BLUR_THRESHOLD default: 100 → 120.0
- Fix blur direction: 'too blurry → lower' was inverted; split into two
  clear cases (too many blurry selected vs too many rejected as blurry)
- Fix MIN_FACE_WIDTH default: 50px → 90px
- Update RESET_PERSON description: now deletes Frigate files before reset

FAQ.md:
- Update quality replacement description to reflect 0.4.0 behavior:
  Frigate-score-based replacement (most redundant → most novel) with
  blur-score fallback; old description said 'lowest-scoring' only

Setup.md:
- Add full environment variable reference with correct defaults
  (previously had no env var table at all)
- Covers all vars including ENABLE_FRIGATE_SCORES and FRIGATE_SCORE_CEILING
  added in 0.4.0

Home.md:
- Add 'winnow only touches files it uploaded' callout

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
flan
2026-06-13 20:37:29 +00:00
co-authored by Claude Sonnet 4.6
parent 82244e09ae
commit fa00ef5e1e
4 changed files with 92 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
FAQ.md
+7 -2
@@ -8,9 +8,14 @@ No. winnow only reads from Immich (assets, people, face bounding boxes). It neve
## How many images should I upload to Frigate? ## How many images should I upload to Frigate?
The `auto` strategy decides this for you — it keeps selecting until adding more images would be redundant. In practice this is usually 20–60 per person. You can cap it with `MAX_AUTO_IMAGES` (default 80). The `auto` strategy decides this for you — it keeps selecting until adding more images would be redundant. In practice this is usually 20–60 per person. You can cap it with `MAX_AUTO_IMAGES` (default `80`).
When winnow reaches the cap on subsequent runs, `QUALITY_REPLACEMENT=true` (default) lets it swap out the lowest-scoring existing image if a better candidate is found — so the training set improves over time rather than stalling. Only images winnow uploaded are ever replaced; anything added manually through Frigate's UI is never touched. When winnow reaches the cap on subsequent runs, `QUALITY_REPLACEMENT=true` (default) lets it improve the training set rather than stalling:
- **With Frigate scores** (default, after the first run): targets the most *redundant* existing file — the one Frigate already recognizes most confidently — and replaces it only if the new candidate is more *novel* (scores lower). This keeps the set maximally diverse.
- **Without Frigate scores** (first run, or `ENABLE_FRIGATE_SCORES=false`): falls back to blur scores — replaces the blurriest existing file if the candidate is sharper.
Only files winnow uploaded are ever replaced. Anything added manually through Frigate's UI is never touched.
Quality and diversity matter far more than volume. 30 well-spread images outperform 200 from the same week. Quality and diversity matter far more than volume. 30 well-spread images outperform 200 from the same week.
Home.md
+2
@@ -4,6 +4,8 @@ winnow pulls photos from [Immich](https://immich.app), selects diverse high-qual
It runs fully headless in Docker, is configured entirely through environment variables, and can run on a schedule. It runs fully headless in Docker, is configured entirely through environment variables, and can run on a schedule.
> **winnow only touches files it uploaded.** Faces added to Frigate manually through its UI are never deleted, replaced, or modified — not by quality replacement, not by `RESET_PERSON`, not by stale cleanup. If you have a curated training set, it is safe.
## Pages ## Pages
- [[Setup]] — installation, Docker Compose configuration, GPU passthrough, environment variables - [[Setup]] — installation, Docker Compose configuration, GPU passthrough, environment variables
Setup.md
+78
@@ -163,3 +163,81 @@ Set `VERBOSE=true` in your `.env` to enable DEBUG-level output on the console. T
# log file is written to the output volume (frigate_train mount) # log file is written to the output volume (frigate_train mount)
docker exec winnow cat /app/frigate_train/winnow.log docker exec winnow cat /app/frigate_train/winnow.log
``` ```
---
## Environment Variables
### Connection
| Variable | Default | Description |
| :--- | :--- | :--- |
| `IMMICH_URL` | *(required)* | Full URL to your Immich instance |
| `API_KEY` | *(required)* | Immich API key |
| `FRIGATE_URL` | *(unset)* | Frigate base URL — required for face upload; omit for object mode only |
### Mode & Strategy
| Variable | Default | Description |
| :--- | :--- | :--- |
| `TRAINING_MODE` | `face` | `face` — upload crops to Frigate; `object` — save crops to disk |
| `STRATEGY` | `auto` | `auto` (embedding-based adaptive), `standard` (30 images), `broad` (100 images) |
| `LIMIT` | *(unset)* | Exact image count — overrides `STRATEGY` |
| `OBJECT_CLASS` | `dog` | Target class for object mode (any YOLO class: `dog`, `cat`, `car`, etc.) |
| `AUTO_MODE` | *(auto)* | Force non-interactive mode in a terminal; auto-detected otherwise |
| `VERBOSE` | `false` | Enable DEBUG-level console output (log file is always DEBUG) |
### People Filtering
| Variable | Default | Description |
| :--- | :--- | :--- |
| `ONLY_PEOPLE` | *(unset)* | Comma-separated whitelist — process only these people |
| `SKIP_PEOPLE` | *(unset)* | Comma-separated list — skip these people |
| `MIN_FACE_COUNT` | `0` | Skip people with fewer than N tagged assets in Immich |
| `YEARS_FILTER` | `10` | Ignore images older than N years |
### Image Quality
| Variable | Default | Description |
| :--- | :--- | :--- |
| `MIN_FACE_WIDTH` | `90` | Minimum face crop width in pixels |
| `FACE_MARGIN` | `0.15` | Padding around bounding box crop (fraction of face size) |
| `ENABLE_FACE_ALIGNMENT` | `true` | Align to ArcFace 112×112 format using facial landmarks |
| `USE_FULL_RESOLUTION` | `true` | Download full-resolution originals rather than preview thumbnails |
| `MIN_CONFIDENCE` | `0.7` | Minimum Immich face detection confidence |
| `BLUR_THRESHOLD` | `120.0` | Laplacian variance threshold — higher rejects more blur |
| `MAX_AUTO_IMAGES` | `80` | Maximum training images per person in Frigate |
| `QUALITY_REPLACEMENT` | `true` | When at cap, swap a weaker tracked image for a better candidate. With Frigate scoring active, targets the most redundant image (highest recognize score); otherwise uses blur score. Never touches manually added Frigate files. Set `false` to skip people at cap |
| `FRIGATE_SCORE_CEILING` | `0.0` | Skip uploads whose pre-upload Frigate recognize score exceeds this value — they are already well-covered. `0` disables; requires at least one prior run to have scores |
| `ENABLE_FRIGATE_SCORES` | `true` | Call Frigate's recognize endpoint pre-upload to store diversity scores used for quality replacement. Adds ~200 ms per upload. Disable to use blur-score replacement only |
### GPU & Models
| Variable | Default | Description |
| :--- | :--- | :--- |
| `FORCE_CPU` | `false` | Disable GPU — fall back to CPU for all inference |
| `OPENVINO_DEVICE` | `CPU` | Intel variant only: set `GPU` to use Arc or iGPU |
| `ENABLE_CACHE` | `true` | Cache computed embeddings to disk (speeds up re-runs on the same library) |
| `CACHE_DIR` | `.if_cache` | Path for embedding cache and upload tracker files |
| `HF_HOME` | *(system)* | HuggingFace model cache path (SigLIP) |
| `INSIGHTFACE_HOME` | *(system)* | InsightFace model cache path (Buffalo_L) |
### Output
| Variable | Default | Description |
| :--- | :--- | :--- |
| `OUTPUT_DIR` | `./frigate_train` | Directory for object-mode crops and the `winnow.log` file. In Docker, set this via the volume mount instead |
### Tracker Overrides *(one-shot — remove after use)*
| Variable | Default | Description |
| :--- | :--- | :--- |
| `DRY_RUN` | `false` | Preview selection without downloading or uploading |
| `RETRY_REJECTED` | `false` | Re-attempt assets previously rejected by Frigate |
| `RESET_PERSON` | *(unset)* | Clear upload history for one person and delete their winnow-managed Frigate training files so the next run starts fresh. Manually added Frigate files are never touched |
### Scheduling
| Variable | Default | Description |
| :--- | :--- | :--- |
| `CRON_SCHEDULE` | *(unset)* | Unset = run once and exit; empty string = stay alive; cron expression = scheduled |
Troubleshooting.md
+5 -4
@@ -155,13 +155,14 @@ To clear the upload history for one person and start fresh:
RESET_PERSON=John RESET_PERSON=John
``` ```
Remove this after one run — it clears the history and then processes normally. Remove this after one run. It deletes winnow-managed Frigate training files for that person, wipes their upload history, then processes normally. Manually-added Frigate files are never touched.
--- ---
## Image quality issues ## Image quality issues
- **Too blurry**: Lower `BLUR_THRESHOLD` (default 100) — e.g. `50` accepts more blur - **Too many blurry images selected**: Raise `BLUR_THRESHOLD` (default `120.0`) — higher values require sharper images; e.g. `200` is stricter
- **Face too small**: Lower `MIN_FACE_WIDTH` (default 50px) - **Too many images rejected as blurry**: Lower `BLUR_THRESHOLD` — e.g. `80` accepts more blur
- **Low confidence detections included**: Raise `MIN_CONFIDENCE` (default 0.7) - **Face too small**: Lower `MIN_FACE_WIDTH` (default `90px`)
- **Low confidence detections included**: Raise `MIN_CONFIDENCE` (default `0.7`)
- **Rejected images being re-tried**: Set `RETRY_REJECTED=true` for one run - **Rejected images being re-tried**: Set `RETRY_REJECTED=true` for one run