From 32e42353840175f99b63bb9bd41a2711139a3d4f Mon Sep 17 00:00:00 2001 From: Holden Date: Fri, 12 Jun 2026 16:51:53 +0000 Subject: [PATCH 1/3] docs: add 0.2.10 changelog Co-Authored-By: Claude Sonnet 4.6 --- CHANGELOG.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 849622a..a558717 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.2.10] - 2026-06-12 + +### Added + +- **`VERBOSE` env var**: set `VERBOSE=true` to enable DEBUG-level console output. The log file always captures DEBUG; this flag controls what appears on the terminal. Useful when diagnosing issues without a full shell into the container. +- **`:cpu` Docker image tag**: a separate CPU-only image (`ghcr.io/sudolulo/winnow:cpu`) is now built and pushed alongside `:latest`. Uses `onnxruntime` instead of `onnxruntime-gpu`; ~2 GB smaller. Suitable for systems without an NVIDIA GPU. +- **Empty `CRON_SCHEDULE` keeps container alive**: setting `CRON_SCHEDULE=` (empty string) starts the container without running immediately and without exiting — useful for `docker exec` ad-hoc runs on a long-lived container. Previously, an empty value was treated the same as unset (run once, then exit). + +### Changed + +- **TTY auto-detection replaces `AUTO_MODE`**: winnow now detects whether a TTY is attached (`sys.stdin.isatty()`) and switches between interactive and auto mode automatically. `AUTO_MODE=true` becomes an explicit override for forcing auto mode in a terminal session. No config change needed for normal Docker deployments. +- **Logging levels audited**: internal algorithmic detail (clustering steps, asset fetch progress, per-page pagination) demoted from INFO to DEBUG. INFO now reflects meaningful pipeline milestones only (model ready, selection complete, quality filtered). Reduces noise in production logs without losing information. +- **Model load logging improved**: SigLIP and InsightFace loading now reports cache hit/miss, download size estimate, device used, and load time. + +### Fixed + +- **GPU OOM crash loop (production)**: `CUDAExecutionProvider` was silently absent even with a GPU attached, causing InsightFace to run on CPU and exhaust RAM processing large person libraries. Root cause: CUDA/cuDNN libraries in nvidia pip packages were invisible to onnxruntime. Fixed by running `ldconfig` over all `nvidia-*/lib/` directories in the venv at image build time. +- **ldconfig path now Python-version-agnostic**: the `find` command used to register nvidia pip libraries hardcoded `python3.13`; replaced with `python3.*` glob so the path survives a Python upgrade without silently producing an empty ldconfig config. +- **`PYTHONPATH=/app` added to Dockerfile**: the entry point script sets `sys.path[0]` to the script directory, not `/app`. Since `uv sync` runs before `COPY winnow/`, the wheel has only dist-info in site-packages. `PYTHONPATH=/app` makes the `winnow` package importable without reverting to `python -m`. +- **CPU fallback retrying broken GPU provider**: InsightFace CPU fallback omitted `providers=["CPUExecutionProvider"]`, causing onnxruntime to retry `CUDAExecutionProvider` on every inference call. Now explicitly sets the CPU provider and suppresses C-extension noise via fd-level redirect. +- **`_suppress_output` stderr loss on fd exhaustion**: if the first `os.dup2` in the finally block raised `OSError`, the second call was skipped, permanently redirecting stderr to `/dev/null` for the process lifetime. Wrapped in nested `try/finally` so both restores are always attempted. +- **Frigate `/api/faces` response parsing**: the response is `{person_name: [files], "train": [...]}` — `"train"` is a flat pending list, not a person. Previous code called `.items()` on the `"train"` value (a list), crashing with `AttributeError`. Now skips the `"train"` key explicitly. +- **Immich 401 detection**: a stale or invalid API key now logs a clear error message (`Immich API key is invalid or expired (401 Unauthorized)`) instead of raising an unhandled exception. +- **Falsy-zero detection confidence**: `face.get("score") or face.get("confidence")` treated a valid `score=0.0` as falsy, falling through to the `confidence` field (often `None`). Replaced with an explicit `None` check. Affected both quality filtering and hard-example weighting in diversity selection. +- **Face crop using wrong person's image dimensions**: in multi-person assets, `_crop_face_from_thumbnail`'s scale-factor loop matched the first person with any face regardless of `person_id`, producing incorrectly scaled bounding box coordinates for the target person. Loop now applies the same `person_id` filter as `_get_face_bbox`. +- **Adaptive stopping bypassed for partially-trained people**: in auto mode with `already_uploaded > 0`, `limit` was converted from `"auto"` to an integer, disabling the FPS adaptive threshold and early-stop check. Now keeps `limit="auto"` through selection and trims the result to the remaining capacity afterward. +- **Embedding cache key mismatch**: HuggingFace cache path check hardcoded the model slug string; replaced with a derivation from `model_name` using `"models--" + model_name.replace("/", "--")` so the check stays correct if the model name changes. +- **Scheduler: sleep until next run**: the loop slept a fixed 60 seconds regardless of schedule interval, causing runs to fire up to 59 seconds late and waking the process unnecessarily on long schedules (e.g. weekly). Now sleeps exactly until `next_run`. +- **Scheduler swallowing `SystemExit`**: `except BaseException` in the run wrapper was replaced with `except Exception` (with `KeyboardInterrupt` re-raised above), so `sys.exit()` calls propagate correctly. +- **Log handler leak**: `setup_logging` now closes and removes existing handlers before adding new ones, preventing file handle accumulation across repeated calls. +- **`RETRY_REJECTED` silently applied in interactive mode**: the env var was applied unconditionally even in interactive sessions. Now used only as the default for the interactive prompt so users can override it per-run. +- **`compose.yml` comment inverted**: a comment stated `-it` forces non-interactive mode; corrected to reflect that `-it` allocates a TTY (interactive mode). + +### Security + +- API key is no longer stored in any config file. All authentication uses environment variables or `.env` only. + ## [0.2.9] - 2026-06-12 ### Fixed From 1ed8d7e25e940374f91b099cd2a3387aba81bc74 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 12 Jun 2026 16:56:13 +0000 Subject: [PATCH 2/3] chore: update uv.lock --- uv.lock | 66 ++++++++++++++++++++++++++++----------------------------- 1 file changed, 33 insertions(+), 33 deletions(-) diff --git a/uv.lock b/uv.lock index da952c8..a323309 100644 --- a/uv.lock +++ b/uv.lock @@ -1861,16 +1861,15 @@ version = "2.12.0" source = { registry = "https://download.pytorch.org/whl/cpu" } resolution-markers = [ "platform_machine != 's390x' and sys_platform == 'darwin'", - "platform_machine == 's390x' and sys_platform == 'darwin'", ] dependencies = [ - { name = "filelock", marker = "sys_platform == 'darwin'" }, - { name = "fsspec", marker = "sys_platform == 'darwin'" }, - { name = "jinja2", marker = "sys_platform == 'darwin'" }, - { name = "networkx", marker = "sys_platform == 'darwin'" }, - { name = "setuptools", marker = "sys_platform == 'darwin'" }, - { name = "sympy", marker = "sys_platform == 'darwin'" }, - { name = "typing-extensions", marker = "sys_platform == 'darwin'" }, + { name = "filelock", marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, + { name = "fsspec", marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, + { name = "jinja2", marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, + { name = "networkx", marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, + { name = "setuptools", marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, + { name = "sympy", marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, + { name = "typing-extensions", marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, ] wheels = [ { url = "https://download-r2.pytorch.org/whl/cpu/torch-2.12.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:90dd587a5f61bfe1307148b581e2084fc5bc4a06e2b90a20e9a36b81087ff16b", upload-time = "2026-05-12T16:20:17Z" }, @@ -1925,15 +1924,16 @@ resolution-markers = [ "platform_machine == 's390x' and sys_platform == 'win32'", "platform_machine != 's390x' and sys_platform != 'darwin' and sys_platform != 'linux' and sys_platform != 'win32'", "platform_machine == 's390x' and sys_platform != 'darwin' and sys_platform != 'linux' and sys_platform != 'win32'", + "platform_machine == 's390x' and sys_platform == 'darwin'", ] dependencies = [ - { name = "filelock", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, - { name = "fsspec", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, - { name = "jinja2", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, - { name = "networkx", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, - { name = "setuptools", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, - { name = "sympy", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, - { name = "typing-extensions", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "filelock", marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "fsspec", marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "jinja2", marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "networkx", marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "setuptools", marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "sympy", marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "typing-extensions", marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, ] wheels = [ { url = "https://download-r2.pytorch.org/whl/cpu/torch-2.12.0%2Bcpu-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:68b7ddd4db4603a03e106e74c7098c8d8c8943d33c1e5ada009ca4cd885759c3", upload-time = "2026-05-12T23:17:12Z" }, @@ -1983,12 +1983,11 @@ version = "0.27.0" source = { registry = "https://download.pytorch.org/whl/cpu" } resolution-markers = [ "platform_machine != 's390x' and sys_platform == 'darwin'", - "platform_machine == 's390x' and sys_platform == 'darwin'", ] dependencies = [ - { name = "numpy", marker = "sys_platform == 'darwin'" }, - { name = "pillow", marker = "sys_platform == 'darwin'" }, - { name = "torch", version = "2.12.0", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "sys_platform == 'darwin'" }, + { name = "numpy", marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, + { name = "pillow", marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, + { name = "torch", version = "2.12.0", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, ] wheels = [ { url = "https://download-r2.pytorch.org/whl/cpu/torchvision-0.27.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:41d6dae73e1af09fa82ded597ae57f2a2314285acde54b25890a8f8e51b999d7", upload-time = "2026-05-12T16:20:37Z" }, @@ -2031,11 +2030,12 @@ resolution-markers = [ "platform_machine == 's390x' and sys_platform == 'win32'", "platform_machine != 's390x' and sys_platform != 'darwin' and sys_platform != 'linux' and sys_platform != 'win32'", "platform_machine == 's390x' and sys_platform != 'darwin' and sys_platform != 'linux' and sys_platform != 'win32'", + "platform_machine == 's390x' and sys_platform == 'darwin'", ] dependencies = [ - { name = "numpy", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, - { name = "pillow", marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, - { name = "torch", version = "2.12.0+cpu", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "numpy", marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "pillow", marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "torch", version = "2.12.0+cpu", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, ] wheels = [ { url = "https://download-r2.pytorch.org/whl/cpu/torchvision-0.27.0%2Bcpu-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:69093b64b2762c43df17be2db2be163029963d90bc3f1801500fdeb723e54833", upload-time = "2026-05-12T16:20:36Z" }, @@ -2153,13 +2153,13 @@ dependencies = [ { name = "pyyaml" }, { name = "requests" }, { name = "scipy" }, - { name = "torch", version = "2.12.0", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "sys_platform == 'darwin'" }, + { name = "torch", version = "2.12.0", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, { name = "torch", version = "2.12.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux'" }, - { name = "torch", version = "2.12.0+cpu", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "torch", version = "2.12.0+cpu", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, { name = "torch", version = "2.12.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, - { name = "torchvision", version = "0.27.0", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "sys_platform == 'darwin'" }, + { name = "torchvision", version = "0.27.0", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, { name = "torchvision", version = "0.27.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux'" }, - { name = "torchvision", version = "0.27.0+cpu", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "torchvision", version = "0.27.0+cpu", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, { name = "torchvision", version = "0.27.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, { name = "ultralytics-thop" }, ] @@ -2174,9 +2174,9 @@ version = "2.0.20" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "numpy" }, - { name = "torch", version = "2.12.0", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "sys_platform == 'darwin'" }, + { name = "torch", version = "2.12.0", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, { name = "torch", version = "2.12.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux'" }, - { name = "torch", version = "2.12.0+cpu", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "torch", version = "2.12.0+cpu", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, { name = "torch", version = "2.12.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/98/c6/d25cb53e141242f74950744179c21b8798bb09b9e7161465ecda4f577ddf/ultralytics_thop-2.0.20.tar.gz", hash = "sha256:f3595e0d8c6fd0b9f62fc2cd9be921755e2649a05c34f1fabaea0bff7295d641", size = 34682, upload-time = "2026-06-06T11:42:42.184Z" } @@ -2195,7 +2195,7 @@ wheels = [ [[package]] name = "winnow" -version = "0.2.9" +version = "0.2.10" source = { editable = "." } dependencies = [ { name = "croniter" }, @@ -2210,13 +2210,13 @@ dependencies = [ { name = "python-dotenv" }, { name = "requests" }, { name = "rich" }, - { name = "torch", version = "2.12.0", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "sys_platform == 'darwin'" }, + { name = "torch", version = "2.12.0", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, { name = "torch", version = "2.12.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux'" }, - { name = "torch", version = "2.12.0+cpu", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "torch", version = "2.12.0+cpu", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, { name = "torch", version = "2.12.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, - { name = "torchvision", version = "0.27.0", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "sys_platform == 'darwin'" }, + { name = "torchvision", version = "0.27.0", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "platform_machine != 's390x' and sys_platform == 'darwin'" }, { name = "torchvision", version = "0.27.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'aarch64' and platform_machine != 'x86_64' and sys_platform == 'linux'" }, - { name = "torchvision", version = "0.27.0+cpu", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "(platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, + { name = "torchvision", version = "0.27.0+cpu", source = { registry = "https://download.pytorch.org/whl/cpu" }, marker = "(platform_machine == 's390x' and sys_platform == 'darwin') or (platform_machine == 'aarch64' and sys_platform == 'linux') or (sys_platform != 'darwin' and sys_platform != 'linux')" }, { name = "torchvision", version = "0.27.0+cu126", source = { registry = "https://download.pytorch.org/whl/cu126" }, marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, { name = "transformers" }, { name = "ultralytics" }, From 567e568c47e5dc2dc39b1b09b1abfbb5b410bd17 Mon Sep 17 00:00:00 2001 From: Holden Date: Fri, 12 Jun 2026 17:00:11 +0000 Subject: [PATCH 3/3] docs: correct CPU embedding speed estimate Co-Authored-By: Claude Sonnet 4.6 --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a7ee3b6..aa5166e 100644 --- a/README.md +++ b/README.md @@ -239,7 +239,7 @@ uv sync uv run winnow ``` -Requires Python 3.13+ and [uv](https://astral.sh/uv/). An NVIDIA GPU is strongly recommended — CPU mode works but embedding computation is significantly slower. +Requires Python 3.13+ and [uv](https://astral.sh/uv/). An NVIDIA GPU is strongly recommended — CPU mode works but embedding computation is slower (typically tens of seconds per person vs under a second on GPU). ---