Compare commits

..
53 Commits
Author SHA1 Message Date
flan 558eb8c321 Land README CI badge on main 2026-08-04 02:51:28 +00:00
flan 336cc394c7 Add Tests status badge to README 2026-08-04 02:45:45 +00:00
flan f650e65c0e Merge pull request 'Land README badge-wall removal on main' (#5) from dev into main 2026-08-03 19:48:06 -04:00
flan e48b9e04b3 Merge pull request 'Remove README badge wall; add NOTICE' (#4) from chore/readme-cleanup into dev 2026-08-03 19:48:02 -04:00
flan f6eee63ffe Remove README badge wall; add NOTICE 2026-08-03 23:43:00 +00:00
flan 7358a2e63a Merge pull request 'Land donation links and sponsor badges on main' (#3) from dev into main 2026-08-03 19:17:59 +00:00
flan 7831c6a967 Merge pull request 'Add donation links (GitHub Sponsors, Ko-fi)' (#2) from feat/donation-links into dev 2026-08-03 19:17:45 +00:00
flan 28217c3f00 Add sponsor badges to README 2026-08-03 19:10:11 +00:00
flan 206aa629f6 Add donation links (GitHub Sponsors, Ko-fi) 2026-08-03 18:45:51 +00:00
flan 348895294a Merge pull request 'Fix Frigate faces crash, tracker races, and batch write loss' (#1) from fix/repo-review-2026-07-19 into main 2026-07-19 16:53:58 -04:00
flan 01fa307031 Fix Frigate faces crash, tracker races, and batch write loss
- frigate_api._get_faces_data now validates the /api/faces response is a
  dict before returning it, so a malformed body no longer crashes
  data.items() in get_all_frigate_person_files (and the equivalent
  data.get() in get_frigate_person_files).
- upload_tracker now takes an exclusive flock on a DATA_DIR lock file for
  every tracker load-mutate-save cycle, so a scheduled run and a manual
  docker exec against the same DATA_DIR can no longer race a
  read-modify-write and silently drop the loser's marks.
- begin_batch()/flush_batch() now flush to disk every 10 marks instead of
  deferring the whole per-person upload loop, bounding how many uploaded
  marks a crash mid-batch can lose.
2026-07-19 20:46:09 +00:00
flan 59f894742f gitignore data/ — it holds face embeddings keyed by real names
data/ is the DATA_DIR the README tells you to mount, and it was not ignored: it holds 1,593
face-embedding .npy files plus upload trackers keyed by person name. A single 'git add -A'
would have published them. Never committed to date, so history is clean.
2026-07-13 17:55:07 +00:00
flan eba50f8a4a Merge pull request #48 from sudolulo/dependabot/docker/nvidia/cuda-12.9.2-cudnn-runtime-ubuntu24.04
chore(deps): bump nvidia/cuda from 12.8.1-cudnn-runtime-ubuntu24.04 to 12.9.2-cudnn-runtime-ubuntu24.04
2026-06-26 18:14:53 -04:00
flan 95d5e91a67 Merge pull request #49 from sudolulo/dependabot/uv/python-deps-c7af4feaef
chore(deps): bump the python-deps group with 3 updates
2026-06-26 18:14:22 -04:00
dependabot[bot] 9e904d9f34 chore(deps): bump the python-deps group with 3 updates
Bumps the python-deps group with 3 updates: [numpy](https://github.com/numpy/numpy), [pytest](https://github.com/pytest-dev/pytest) and [ruff](https://github.com/astral-sh/ruff).


Updates `numpy` from 2.4.6 to 2.5.0
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](https://github.com/numpy/numpy/compare/v2.4.6...v2.5.0)

Updates `pytest` from 9.1.0 to 9.1.1
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.1.0...9.1.1)

Updates `ruff` from 0.15.18 to 0.15.20
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.18...0.15.20)

---
updated-dependencies:
- dependency-name: numpy
  dependency-version: 2.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-deps
- dependency-name: pytest
  dependency-version: 9.1.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: python-deps
- dependency-name: ruff
  dependency-version: 0.15.20
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: python-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-25 18:55:02 +00:00
dependabot[bot] 18305b5b06 chore(deps): bump nvidia/cuda
Bumps nvidia/cuda from 12.8.1-cudnn-runtime-ubuntu24.04 to 12.9.2-cudnn-runtime-ubuntu24.04.

---
updated-dependencies:
- dependency-name: nvidia/cuda
  dependency-version: 12.9.2-cudnn-runtime-ubuntu24.04
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-25 18:53:34 +00:00
flan 9b68f62f69 Merge pull request #44 from sudolulo/dependabot/github_actions/actions/checkout-7.0.0
chore(deps): bump actions/checkout from 6.0.3 to 7.0.0
2026-06-20 15:06:54 -04:00
flan b5d03a2695 Merge branch 'main' into dependabot/github_actions/actions/checkout-7.0.0 2026-06-20 15:05:37 -04:00
flan fce46d409e Merge pull request #45 from sudolulo/dependabot/uv/python-deps-ae41630b22
chore(deps): bump the python-deps group across 1 directory with 5 updates
2026-06-20 15:05:13 -04:00
dependabot[bot] 5044343a99 chore(deps): bump the python-deps group across 1 directory with 5 updates
Bumps the python-deps group with 5 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [onnxruntime-gpu](https://github.com/microsoft/onnxruntime) | `1.26.0` | `1.27.0` |
| [nvidia-cudnn-cu12](https://developer.nvidia.com/cuda-zone) | `9.23.1.3` | `9.23.2.1` |
| [onnxruntime](https://github.com/microsoft/onnxruntime) | `1.26.0` | `1.27.0` |
| [pytest](https://github.com/pytest-dev/pytest) | `9.0.3` | `9.1.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.15.17` | `0.15.18` |



Updates `onnxruntime-gpu` from 1.26.0 to 1.27.0
- [Release notes](https://github.com/microsoft/onnxruntime/releases)
- [Changelog](https://github.com/microsoft/onnxruntime/blob/main/docs/ReleaseManagement.md)
- [Commits](https://github.com/microsoft/onnxruntime/commits)

Updates `nvidia-cudnn-cu12` from 9.23.1.3 to 9.23.2.1

Updates `onnxruntime` from 1.26.0 to 1.27.0
- [Release notes](https://github.com/microsoft/onnxruntime/releases)
- [Changelog](https://github.com/microsoft/onnxruntime/blob/main/docs/ReleaseManagement.md)
- [Commits](https://github.com/microsoft/onnxruntime/commits)

Updates `pytest` from 9.0.3 to 9.1.0
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.3...9.1.0)

Updates `ruff` from 0.15.17 to 0.15.18
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.17...0.15.18)

---
updated-dependencies:
- dependency-name: nvidia-cudnn-cu12
  dependency-version: 9.23.2.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-deps
- dependency-name: onnxruntime
  dependency-version: 1.27.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-deps
- dependency-name: onnxruntime-gpu
  dependency-version: 1.27.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-deps
- dependency-name: pytest
  dependency-version: 9.1.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: python-deps
- dependency-name: ruff
  dependency-version: 0.15.18
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: python-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-18 20:42:03 +00:00
dependabot[bot] 2a673fcd9d chore(deps): bump actions/checkout from 6.0.3 to 7.0.0
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.3 to 7.0.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/df4cb1c069e1874edd31b4311f1884172cec0e10...9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-18 20:40:32 +00:00
flan 04151057c9 Merge pull request #46 from sudolulo/dev
release: v0.6.6
2026-06-18 16:38:28 -04:00
flan 3ea6b9d566 Merge pull request #47 from sudolulo/chore/simplify-lockfile-ci
chore: replace lockfile auto-update with uv lock --check
2026-06-18 16:36:10 -04:00
flan e281ac01e1 chore: add pre-commit hook to auto-update lockfile on pyproject.toml changes 2026-06-18 20:33:58 +00:00
flan 0ac02c3108 chore: update lockfile for v0.6.6 2026-06-18 20:31:36 +00:00
flan db30449b09 chore: replace lockfile auto-update with uv lock --check 2026-06-18 20:29:43 +00:00
flan 1e4425bfd7 release: v0.6.6 2026-06-18 20:23:58 +00:00
flan 2be9f400dd Merge pull request #42 from sudolulo/fix/frigate-500-permanent-rejection
fix: Frigate 500 permanent rejection + 13 correctness bugs
2026-06-17 15:17:20 -04:00
flan f3b5bd9334 fix: update MAX_AUTO_IMAGES default assertion to 5 2026-06-17 19:16:43 +00:00
flan 0906342edf fix: 10 correctness bugs from full codebase audit
- immich_api: .get("items") or [] handles {"items": null} without crashing len()
- immich_api: catch TypeError alongside ValueError in filter_recent_assets for
  timezone-naive fileCreatedAt comparisons
- scheduler: catch SystemExit in addition to KeyboardInterrupt so cli.main()
  cannot kill the long-running scheduler process
- scheduler: reseed croniter from wall-clock time after each run so overrunning
  jobs don't schedule an immediate back-to-back rerun
- config: reject negative YEARS_FILTER values with a warning, reset to default 10
- frigate_api: return True (not False) for empty filenames list — callers cannot
  distinguish no-op from network failure on False
- jobs: warn on unrecognised STRATEGY value instead of silently falling back
- jobs: casefold ONLY_PEOPLE / SKIP_PEOPLE matching so "john doe" matches "John Doe"
- executor: <= → < so a same-score candidate can fill a freed replacement slot
- embeddings: set _insightface_loaded=True on GPU+CPU double-failure to prevent
  N re-init attempts (one per asset) when InsightFace is broken for a whole run
2026-06-17 19:12:58 +00:00
flan 5e0a871314 fix: 2 low findings from audit — consistent 500 match source, accurate return type
Use full_body for the 500 'could not process' permanent-rejection check,
consistent with the 400 'face' check on the line above. error_detail is
truncated to 100 chars via the fallback path, which could silently miss
the phrase in a long response body.

Remove | None from process_face_mode return type — every code path returns
tuple[int,int] or str; None is unreachable. Update docstring to match.
2026-06-17 17:47:31 +00:00
flan 86caffc8d7 fix: 3 audit findings — truthy skip-reason bug, 500 match consistency, CHANGELOG note
1. if saved: → if isinstance(saved, tuple): so string skip-reasons from
   process_face_mode no longer register as successes and create phantom
   asset_map entries with no JPEG on disk. Dead reason/fallback code in
   the else branch now correctly handles str and None returns.

2. "could not process" permanent-rejection check now uses error_detail
   (json message field, falling back to body[:100]) instead of full_body,
   keeping the match consistent with what is displayed to the user.

3. CHANGELOG [Unreleased] breaking-change note for MAX_AUTO_IMAGES 20→5
   so upgrading users know to set the env var if they want the old cap.
2026-06-17 17:31:20 +00:00
flan f6e494071e fix: lower MAX_AUTO_IMAGES default from 20 to 5
Smaller default cap is more conservative for new installs and better
reflects the minimum viable training set for Frigate face recognition.
Users who need more can set MAX_AUTO_IMAGES explicitly.
2026-06-17 17:19:09 +00:00
flan b69f776378 fix: surface skip reasons and suppress norm_crop FutureWarning
process_face_mode now returns a descriptive string instead of None for
filtered-out faces ("face too small 45x38px, min 90px", "no face
metadata"), so the executor can print a useful reason rather than the
generic "no usable face data".

Also suppresses the InsightFace norm_crop FutureWarning about deprecated
estimate usage, which was noisy at INFO level on every aligned crop.
2026-06-17 17:18:43 +00:00
flan c4910d82ef fix: treat Frigate 500 'Could not process' as permanent rejection
Frigate returns HTTP 500 with 'Could not process' when its face detector
cannot find or embed a face in the uploaded crop — this will never succeed
on retry. Previously these were silently logged at DEBUG and retried on
every future run.

- Surface 500 error details inline (same display path as 400)
- Mark 500 + 'could not process' as a permanent rejection so the asset
  is skipped on future runs instead of retried indefinitely
2026-06-17 17:16:04 +00:00
flan 5b8b3ab736 Merge pull request #41 from sudolulo/dev
fix: lockfile workflow opens PR on dev instead of direct push
2026-06-16 23:23:44 -04:00
flan dc0c431e6b chore: update lockfile 2026-06-17 03:12:20 +00:00
flan 3296940806 fix: lockfile workflow opens PR on dev instead of direct push
dev is a protected branch requiring PRs. The previous direct push caused
the lockfile update CI job to fail with 'protected branch hook declined'.
When the triggering branch is dev, the workflow now creates a side branch
and opens a PR; all other branches continue to push directly.
2026-06-17 03:12:00 +00:00
flan fdcb4efac3 Merge pull request #40 from sudolulo/dev
release: v0.6.5
2026-06-16 23:08:08 -04:00
flan c1f04be15b release: v0.6.5 2026-06-17 03:06:27 +00:00
flan 001dd2c575 fix: reinstall onnxruntime-gpu after uv sync to guarantee GPU binary wins
insightface depends on onnxruntime (CPU) as a direct dependency. During
uv sync --extra gpu, both onnxruntime (CPU, 24.6 MB binary) and
onnxruntime-gpu (GPU, 24.7 MB binary) are installed in parallel — both
claim onnxruntime/capi/onnxruntime_pybind11_state.so. The last writer wins,
which is non-deterministic in uv's parallel installer.

On GitHub Actions (no GPU, different scheduler ordering), the CPU binary
consistently wins, leaving onnxruntime-gpu's pybind11_state.so as the CPU
version. CUDAExecutionProvider then silently disappears because the CPU
binary's provider registration code has no CUDA EP.

Fix: after uv sync, reinstall onnxruntime-gpu explicitly using the already-
cached wheel. Since uv pip install is synchronous and runs after the parallel
sync completes, the GPU binary is guaranteed to be on disk when the build
layer commits.
2026-06-17 03:05:33 +00:00
flan edf576bc93 Merge pull request #39 from sudolulo/fix/codebase-audit-r2
fix: codebase audit rounds 2-5 (correctness, cli, jobs)
2026-06-16 23:02:42 -04:00
flan 561a1a3d72 fix: 5 findings from codebase audit round 5
cli.py:
- _smaller_duplicate_ids: walrus operator eliminates double p.get("id")
  per element; truthiness check replaces dead "is not None" guard (all
  persons in by_name are guaranteed to have a truthy id after the
  line-75 gate)
- Extract _excl() helper inside _handle_duplicate_people — replaces 4
  identical [p for p in lst if p.get("id") not in skip_ids] expressions
  across all return paths

jobs.py:
- Extract _valid_people() — shared filter for interactive_configure and
  auto_configure; uses (p.get("name") or "").strip() to match cli.py's
  whitespace-strip gate, preventing whitespace-only Immich names from
  reaching _build_job and creating blank Frigate person labels
- Hoist queued_ids set before the display loop in interactive_configure:
  O(N) set lookup per render instead of O(N×|jobs|) linear scan
2026-06-17 02:28:58 +00:00
flan 614542decd fix: 4 findings from codebase audit round 4
jobs.py:
- Add p.get("id") guard to valid_people filter in both
  interactive_configure and auto_configure — id-less named persons
  passed through by _handle_duplicate_people are now excluded before
  any bare-subscript access in the configure paths
- Fix bare p["id"] → p.get("id") in the queued-marker check at line 214
  (runs unconditionally on all valid_people during menu display, before
  any user selection or fetch_all_assets guard)

cli.py:
- Remove dead-code survivor_id and merge_ids guards: after the by_name
  fix (line 75 requires p.get("id")), all persons in any ordered list
  have ids, so neither guard can ever fire; removing them prevents
  misleading readers about what states are reachable
2026-06-17 02:20:38 +00:00
flan 3fccf9c8f9 fix: 6 findings from codebase audit round 3
cli.py:
- Filter id-less persons from by_name at construction (root fix for all
  bare-subscript crashes downstream — persons with a name but no id are
  excluded from duplicate detection entirely)
- Belt-and-suspenders on warning-path display: p['id'] → p.get('id')
- Extract survivor_id with .get(); skip group if survivor has no id
- Guard merge_ids: skip API call when list is empty after id filtering
- Walrus operator in merge_ids comprehension: p.get("id") called once
  per item instead of twice

executor.py:
- Add cross-reference comment at success-path reset so the for/else
  rollback pairing is explicit for future maintainers
2026-06-17 02:09:16 +00:00
flan eab3d9fe64 fix: 3 correctness bugs from codebase audit round 2
- executor.py: clear min_quality_score_for_slot alongside effective_count
  restore in for/else block; leaving the stale floor from the deleted
  file's score blocked the next candidate from filling the restored slot
- cli.py: guard merge_ids with p.get('id') is not None, consistent with
  the _smaller_duplicate_ids fix; bare p['id'] raised KeyError on any
  person dict missing the id field in the auto-merge path
- immich_api.py: replace bare data['major'/'minor'/'patch'] subscripts
  with .get() in get_immich_version; KeyError was silently swallowed by
  except Exception, causing version-gated flags to disable without warning
2026-06-17 01:54:13 +00:00
flan 5509be150e Merge pull request #38 from sudolulo/fix/codebase-audit-r1
fix: codebase audit r1 — correctness fixes, version banner, GPU dep
2026-06-16 21:51:09 -04:00
flan 0bd2eaf9fb fix: add missing nvidia CUDA pip packages for onnxruntime-gpu 1.26.0
ORT 1.26.0 changed provider loading to gate on the presence of required
nvidia pip packages before attempting to load libonnxruntime_providers_cuda.so.
Without nvidia-cuda-runtime-cu12, nvidia-cufft-cu12, and nvidia-curand-cu12
installed as Python packages, ORT silently skips the CUDA EP plugin entirely
(confirmed via /proc/maps: the .so was never dlopen'd despite existing on disk
and all system CUDA libs being present in ldconfig).

nvidia-nvjitlink-cu12 pulled in as a transitive dependency.
2026-06-17 01:47:49 +00:00
flan b80d26b36b feat: display version in startup banner 2026-06-17 01:18:45 +00:00
flan 068a8e675f fix: 4 correctness bugs from full-codebase audit
- executor: restore effective_count when replacement upload fails all retries
  (delete succeeded but slot was never filled, leaving cap undercount)
- diversity: skip zero-norm embeddings before dedup/FPS selection
  (InsightFace zeros pass dedup with similarity 0 and score distance 1.0,
  getting selected first as maximally diverse)
- cli: exclude None from skip_ids in _smaller_duplicate_ids
  (p.get('id') without None guard lets None into the set, silently
  dropping every other id-less person from the processed list)
- embeddings: select face nearest crop centre instead of largest by area
  (25% margin can pull a bigger neighbouring face into the crop;
  largest-face selection then embeds the wrong person)
2026-06-17 01:12:28 +00:00
flan c36e7bf28e Merge pull request #37 from sudolulo/dev
fix: exclude main from lockfile update workflow trigger
2026-06-16 21:03:35 -04:00
flan 0ffe08bc6f Merge pull request #36 from sudolulo/fix/lockfile-workflow-main-exclusion
fix: exclude main from lockfile update workflow trigger
2026-06-16 21:01:25 -04:00
flan 3423d41535 fix: exclude main from lockfile update trigger
main is protected and only receives merges from dev; pushing directly
to it from CI is blocked by branch protection rules.
2026-06-17 00:56:52 +00:00
30 changed files with 710 additions and 311 deletions
+7
View File
@@ -0,0 +1,7 @@
#!/usr/bin/env bash
set -euo pipefail
if git diff --cached --name-only | grep -q "^pyproject\.toml$"; then
uv lock
git add uv.lock
fi
+2
View File
@@ -0,0 +1,2 @@
github: sudolulo
ko_fi: sudolulo
+4 -4
View File
@@ -54,7 +54,7 @@ jobs:
df -h df -h
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
ref: ${{ inputs.tag || github.ref }} ref: ${{ inputs.tag || github.ref }}
@@ -178,7 +178,7 @@ jobs:
df -h df -h
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
ref: ${{ inputs.tag || github.ref }} ref: ${{ inputs.tag || github.ref }}
@@ -264,7 +264,7 @@ jobs:
df -h df -h
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
ref: ${{ inputs.tag || github.ref }} ref: ${{ inputs.tag || github.ref }}
@@ -347,7 +347,7 @@ jobs:
df -h df -h
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
ref: ${{ inputs.tag || github.ref }} ref: ${{ inputs.tag || github.ref }}
+1 -1
View File
@@ -23,7 +23,7 @@ jobs:
echo "Disk space freed." echo "Disk space freed."
- name: Checkout code - name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Run Ruff - name: Run Ruff
uses: astral-sh/ruff-action@0ce1b0bf8b818ef400413f810f8a11cdbda0034b # v4.0.0 uses: astral-sh/ruff-action@0ce1b0bf8b818ef400413f810f8a11cdbda0034b # v4.0.0
+1 -1
View File
@@ -32,7 +32,7 @@ jobs:
echo "Disk space freed." echo "Disk space freed."
- name: Checkout - name: Checkout
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
fetch-depth: 0 fetch-depth: 0
+4 -1
View File
@@ -13,7 +13,7 @@ jobs:
contents: read contents: read
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Install uv - name: Install uv
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0 uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
@@ -21,6 +21,9 @@ jobs:
- name: Set up Python - name: Set up Python
run: uv python install 3.13 run: uv python install 3.13
- name: Check lockfile is up to date
run: uv lock --check
- name: Install dependencies - name: Install dependencies
run: uv sync --extra cpu run: uv sync --extra cpu
-46
View File
@@ -1,46 +0,0 @@
# .github/workflows/update-lockfile.yml
name: Update lockfile
on:
push:
branches:
- '**'
paths:
- 'pyproject.toml'
workflow_dispatch:
jobs:
update-lockfile:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- name: Install uv
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
- name: Set up Python
run: uv python install 3.13
- name: Regenerate lockfile
run: uv lock
- name: Check for changes
id: diff
run: |
if git diff --quiet uv.lock; then
echo "changed=false" >> "$GITHUB_OUTPUT"
else
echo "changed=true" >> "$GITHUB_OUTPUT"
fi
- name: Commit and push updated lockfile
if: steps.diff.outputs.changed == 'true'
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add uv.lock
git commit -m "chore: update lockfile"
git push
+4
View File
@@ -34,3 +34,7 @@ wheels/
.pytest_cache/ .pytest_cache/
.mypy_cache/ .mypy_cache/
.python-version .python-version
# Runtime data: face embeddings + upload trackers keyed by real person names.
# Never commit — this is the DATA_DIR the README tells you to mount.
data/
+46
View File
@@ -7,6 +7,52 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
### Fixed
- **`get_all_frigate_person_files` crashed on a non-dict `/api/faces` response**: `data.items()` assumed the JSON body was always an object; a malformed response from Frigate crashed the call instead of degrading gracefully like the rest of the API layer. `_get_faces_data` now validates the response shape and returns `None` on a non-dict body, protecting both `get_all_frigate_person_files` and `get_frigate_person_files`.
- **Tracker JSON files had no cross-process locking**: two winnow invocations against the same `DATA_DIR` (e.g. a scheduled run overlapping a manual `docker exec`, which the docs explicitly instruct) could race a read-modify-write on `frigate_uploaded_ids.json`/`frigate_rejected_ids.json` and silently lose the loser's marks. Tracker load-mutate-save cycles now hold an exclusive `flock` on a `DATA_DIR` lock file for the duration of the operation.
- **`begin_batch`/`flush_batch` could lose an entire person's upload marks on a crash**: tracker writes were deferred for the whole per-person upload loop, so a SIGKILL/OOM/host crash mid-batch lost every successful Frigate upload from the tracker even though the files were already live in Frigate, causing duplicate re-uploads on the next run. Batches now flush to disk every 10 marks, bounding the loss to a small, fixed window.
## [0.6.6] - 2026-06-18
### Changed
- **`MAX_AUTO_IMAGES` default lowered from 20 to 5** — existing users who have not set this variable and already have more than 5 winnow-managed images in Frigate will find themselves at cap on the next run. With `QUALITY_REPLACEMENT=true` (the default), winnow will attempt to swap weaker images rather than uploading new ones. Set `MAX_AUTO_IMAGES=20` to restore the previous behaviour.
## [0.6.5] - 2026-06-17
### Added
- **Version displayed in startup banner**: winnow now prints its installed version at launch.
### Fixed
- **GPU image: `CUDAExecutionProvider` missing due to parallel install race**: `insightface` declares `onnxruntime` (CPU) as a dependency, causing `uv sync` to install both `onnxruntime` and `onnxruntime-gpu` in parallel — both packages claim the same `pybind11_state.so` binary. On GitHub Actions the CPU binary consistently won the race, leaving the GPU build without CUDA support at runtime despite all CUDA libraries being present. Fixed by reinstalling `onnxruntime-gpu` sequentially after `uv sync` to guarantee its GPU binary is on disk.
- **GPU extra was missing three required nvidia pip packages**: `onnxruntime-gpu` 1.26.0 gates CUDA EP loading on the Python-importability of `nvidia-cuda-runtime-cu12`, `nvidia-cufft-cu12`, and `nvidia-curand-cu12`. These packages were not declared in the `gpu` extra and were absent on fresh installs, silently disabling GPU inference.
- **`_handle_duplicate_people` raises `KeyError` on id-less person records**: bare `p["id"]` subscripts in the auto-merge loop and `_smaller_duplicate_ids` raised `KeyError` when Immich returned a person dict without an `id` field (e.g. unconfirmed face clusters). Fixed by using `p.get("id")` and filtering `None` from `skip_ids`.
- **`_smaller_duplicate_ids` could include `None` in the skip set**: `p.get("id")` without a `None` guard populated `skip_ids` with `None`, causing `p.get("id") not in skip_ids` to pass for every id-less person, so unnamed face clusters were silently re-included in all return paths.
- **`_handle_duplicate_people` dead code removed**: guards `if not survivor_id` and `if not merge_ids` became unreachable after the id-gate fix; their presence suggested they still ran.
- **`_valid_people` in `jobs.py` used wrong name filter**: whitespace-only names (e.g. `" "`) passed the `p.get("name")` truthiness check and were included in the person list. Fixed using `(p.get("name") or "").strip()` consistent with the cli.py gate.
- **`interactive_configure` queued-marker check was O(N²)**: `[j for j in jobs if j["person"]["id"] == p.get("id")]` ran a full scan over jobs for every person in the display loop. Replaced with a `queued_ids` set hoisted before the loop.
- **`executor.py` slot restore did not clear `min_quality_score_for_slot`**: when a replacement upload failed all retries after a deletion, `effective_count` was restored but the stale quality-score floor from the deleted file remained, blocking the next candidate from filling the slot.
- **`get_immich_version` swallowed `KeyError` on unexpected schema**: bare `data["major"]` / `data["minor"]` / `data["patch"]` subscripts were silently caught by the surrounding `except Exception`, returning `None` without logging. Replaced with `.get()` calls that log a debug warning on unexpected schemas.
- **Face embedding selects nearest face to crop centre, not largest by area**: a 25 % margin on the crop window can pull a larger neighbouring face into the bounding box; selecting the biggest face by area then embeds the wrong person. Centre-proximity is now used instead.
- **Zero-norm face embeddings skipped before diversity selection**: InsightFace occasionally returns a zero vector for low-quality detections; zero embeddings pass deduplication with similarity 0 and score distance 1.0, causing them to be selected first as maximally diverse.
- **`executor.py` slot restore did not clear `min_quality_score_for_slot`**: stale quality floor from the deleted file blocked the next candidate from filling the restored slot in quality-replacement mode.
## [0.6.4] - 2026-06-17 ## [0.6.4] - 2026-06-17
### Fixed ### Fixed
+3
View File
@@ -17,8 +17,11 @@ git clone https://github.com/sudolulo/winnow.git
cd winnow cd winnow
git checkout dev git checkout dev
uv sync uv sync
git config core.hooksPath .githooks
``` ```
The last line activates the project's git hooks. The pre-commit hook automatically runs `uv lock` and stages the result whenever `pyproject.toml` is part of a commit, keeping the lockfile in sync without any extra steps.
## Running Tests and Lint ## Running Tests and Lint
```bash ```bash
+4 -2
View File
@@ -8,7 +8,7 @@
ARG VARIANT=gpu ARG VARIANT=gpu
FROM --platform=$BUILDPLATFORM nvidia/cuda:12.8.1-cudnn-runtime-ubuntu24.04 AS base-amd64-gpu FROM --platform=$BUILDPLATFORM nvidia/cuda:12.9.2-cudnn-runtime-ubuntu24.04 AS base-amd64-gpu
FROM ubuntu:26.04 AS base-amd64-rocm FROM ubuntu:26.04 AS base-amd64-rocm
FROM ubuntu:22.04 AS base-amd64-intel FROM ubuntu:22.04 AS base-amd64-intel
FROM ubuntu:26.04 AS base-amd64-cpu FROM ubuntu:26.04 AS base-amd64-cpu
@@ -50,7 +50,9 @@ RUN if [ "$VARIANT" = "cpu" ]; then \
elif [ "$VARIANT" = "intel" ]; then \ elif [ "$VARIANT" = "intel" ]; then \
uv sync --frozen --no-dev --extra intel; \ uv sync --frozen --no-dev --extra intel; \
elif [ "$VARIANT" = "gpu" ]; then \ elif [ "$VARIANT" = "gpu" ]; then \
uv sync --frozen --no-dev --extra gpu; \ uv sync --frozen --no-dev --extra gpu && \
ORT_GPU_VER=$(.venv/bin/python -c "import importlib.metadata; print(importlib.metadata.version('onnxruntime-gpu'))") && \
uv pip install --python .venv/bin/python --no-deps --reinstall "onnxruntime-gpu==$ORT_GPU_VER"; \
else \ else \
echo "Unknown VARIANT: '$VARIANT'. Must be one of: cpu, rocm, intel, gpu" >&2; \ echo "Unknown VARIANT: '$VARIANT'. Must be one of: cpu, rocm, intel, gpu" >&2; \
exit 1; \ exit 1; \
+3
View File
@@ -0,0 +1,3 @@
winnow
Portions of this project were written with assistance from Claude (Anthropic).
+7 -2
View File
@@ -1,6 +1,6 @@
# winnow # winnow
[![Docker](https://github.com/sudolulo/winnow/actions/workflows/docker-publish.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/docker-publish.yml) [![Test](https://github.com/sudolulo/winnow/actions/workflows/test.yml/badge.svg)](https://github.com/sudolulo/winnow/actions/workflows/test.yml) [![GitHub release](https://img.shields.io/github/v/release/sudolulo/winnow)](https://github.com/sudolulo/winnow/releases/latest) [![License: AGPL v3](https://img.shields.io/badge/License-AGPL_v3-blue.svg)](LICENSE) [![Immich](https://img.shields.io/badge/Immich-v1.106%2B-blueviolet)](https://immich.app) [![Frigate](https://img.shields.io/badge/Frigate-Ready-brightgreen)](https://frigate.video) [![Tests](https://github.com/sudolulo/winnow/actions/workflows/test.yml/badge.svg)](https://github.com/sudolulo/winnow/actions)
> **Note:** winnow's approach to training Frigate face recognition is not an officially documented workflow — results may vary. > **Note:** winnow's approach to training Frigate face recognition is not an officially documented workflow — results may vary.
@@ -187,7 +187,7 @@ In scheduled mode the process (and loaded models) stays resident between runs. T
| Variable | Default | Description | | Variable | Default | Description |
| :--- | :--- | :--- | | :--- | :--- | :--- |
| `MAX_AUTO_IMAGES` | `20` | Maximum training images per person in Frigate | | `MAX_AUTO_IMAGES` | `5` | Maximum training images per person in Frigate |
| `QUALITY_REPLACEMENT` | `true` | When at cap, swap a weaker tracked image for a better candidate. With Frigate scoring active, targets the most redundant image (highest pre-upload recognize score); otherwise uses blur score. Never touches manually added Frigate files. Set `false` to skip people at cap | | `QUALITY_REPLACEMENT` | `true` | When at cap, swap a weaker tracked image for a better candidate. With Frigate scoring active, targets the most redundant image (highest pre-upload recognize score); otherwise uses blur score. Never touches manually added Frigate files. Set `false` to skip people at cap |
#### Advanced Tuning *(calibrated — do not adjust)* #### Advanced Tuning *(calibrated — do not adjust)*
@@ -273,3 +273,8 @@ When run with a terminal attached, winnow starts an interactive session: select
## Attribution ## Attribution
Based on [if_curator](https://github.com/ds-sebastian/if_curator) by Sebastian, licensed MIT. Based on [if_curator](https://github.com/ds-sebastian/if_curator) by Sebastian, licensed MIT.
## Support
If winnow is useful to you, consider supporting development via
[GitHub Sponsors](https://github.com/sponsors/sudolulo) or [Ko-fi](https://ko-fi.com/sudolulo).
+1 -1
View File
@@ -31,7 +31,7 @@ services:
# - USE_FULL_RESOLUTION=true # Use full-res images vs thumbnails (default: true) # - USE_FULL_RESOLUTION=true # Use full-res images vs thumbnails (default: true)
# - MIN_CONFIDENCE=0.7 # Minimum face detection confidence (default: 0.7) # - MIN_CONFIDENCE=0.7 # Minimum face detection confidence (default: 0.7)
# - BLUR_THRESHOLD=100.0 # Laplacian blur threshold; lower = accept more blur (default: 100.0) # - BLUR_THRESHOLD=100.0 # Laplacian blur threshold; lower = accept more blur (default: 100.0)
# - MAX_AUTO_IMAGES=80 # Hard cap on auto-diversity selection (default: 20) # - MAX_AUTO_IMAGES=80 # Hard cap on auto-diversity selection (default: 5)
# ── Caching & Models ────────────────────────────────────────────────── # ── Caching & Models ──────────────────────────────────────────────────
# - FORCE_CPU=true # Disable GPU, fall back to CPU # - FORCE_CPU=true # Disable GPU, fall back to CPU
+10 -7
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "winnow" name = "winnow"
version = "0.6.4" version = "0.6.6"
description = "Selects diverse, high-quality photos from Immich as training data for Frigate face recognition." description = "Selects diverse, high-quality photos from Immich as training data for Frigate face recognition."
license = "AGPL-3.0-or-later" license = "AGPL-3.0-or-later"
requires-python = ">=3.13" requires-python = ">=3.13"
@@ -17,7 +17,7 @@ classifiers = [
dependencies = [ dependencies = [
"croniter>=5.0.2", "croniter>=5.0.2",
"insightface>=0.7.3", "insightface>=0.7.3",
"numpy>=2.2.6", "numpy>=2.5.0",
"opencv-python-headless>=4.12.0.88", "opencv-python-headless>=4.12.0.88",
"pillow>=12.1.0", "pillow>=12.1.0",
"python-dotenv>=1.2.1", "python-dotenv>=1.2.1",
@@ -27,12 +27,15 @@ dependencies = [
[project.optional-dependencies] [project.optional-dependencies]
gpu = [ gpu = [
"onnxruntime-gpu>=1.23.2; sys_platform == 'linux' and platform_machine == 'x86_64'", "onnxruntime-gpu>=1.27.0; sys_platform == 'linux' and platform_machine == 'x86_64'",
"nvidia-cudnn-cu12>=9.0.0; sys_platform == 'linux' and platform_machine == 'x86_64'", "nvidia-cudnn-cu12>=9.23.2.1; sys_platform == 'linux' and platform_machine == 'x86_64'",
"nvidia-cuda-runtime-cu12>=12.0; sys_platform == 'linux' and platform_machine == 'x86_64'",
"nvidia-cufft-cu12>=11.0; sys_platform == 'linux' and platform_machine == 'x86_64'",
"nvidia-curand-cu12>=10.0; sys_platform == 'linux' and platform_machine == 'x86_64'",
] ]
rocm = ["onnxruntime-rocm>=1.16.0; sys_platform == 'linux' and platform_machine == 'x86_64'"] rocm = ["onnxruntime-rocm>=1.16.0; sys_platform == 'linux' and platform_machine == 'x86_64'"]
intel = ["onnxruntime-openvino>=1.20.0; sys_platform == 'linux' and platform_machine == 'x86_64'"] intel = ["onnxruntime-openvino>=1.20.0; sys_platform == 'linux' and platform_machine == 'x86_64'"]
cpu = ["onnxruntime>=1.23.2"] cpu = ["onnxruntime>=1.27.0"]
[project.scripts] [project.scripts]
winnow = "winnow.cli:main" winnow = "winnow.cli:main"
@@ -60,8 +63,8 @@ required-environments = [
[dependency-groups] [dependency-groups]
dev = [ dev = [
"pytest>=8.0", "pytest>=9.1.1",
"ruff>=0.15.17", "ruff>=0.15.20",
] ]
[tool.hatch.build.targets.wheel] [tool.hatch.build.targets.wheel]
+2 -1
View File
@@ -49,11 +49,12 @@ def _run_scheduler() -> None:
try: try:
main() main()
print("winnow run complete", flush=True) print("winnow run complete", flush=True)
except KeyboardInterrupt: except (KeyboardInterrupt, SystemExit):
raise raise
except Exception as e: except Exception as e:
logger.error("winnow run failed: %s", e, exc_info=True) logger.error("winnow run failed: %s", e, exc_info=True)
print(f"winnow run failed: {e}", flush=True) print(f"winnow run failed: {e}", flush=True)
cron = croniter(schedule, time.time())
next_run = cron.get_next(float) next_run = cron.get_next(float)
print(f"Next run: {time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(next_run))}", flush=True) print(f"Next run: {time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(next_run))}", flush=True)
time.sleep(min(60, max(1, next_run - time.time()))) time.sleep(min(60, max(1, next_run - time.time())))
+1 -1
View File
@@ -21,7 +21,7 @@ def test_config_loads_defaults(monkeypatch):
assert cfg.MIN_FACE_COUNT == 3 assert cfg.MIN_FACE_COUNT == 3
assert cfg.BLUR_THRESHOLD == 120.0 assert cfg.BLUR_THRESHOLD == 120.0
assert cfg.MIN_CONFIDENCE == 0.7 assert cfg.MIN_CONFIDENCE == 0.7
assert cfg.MAX_AUTO_IMAGES == 20 assert cfg.MAX_AUTO_IMAGES == 5
assert cfg.QUALITY_REPLACEMENT is True assert cfg.QUALITY_REPLACEMENT is True
assert cfg.FACE_MARGIN == 0.15 assert cfg.FACE_MARGIN == 0.15
assert cfg.USE_FULL_RESOLUTION is True assert cfg.USE_FULL_RESOLUTION is True
+56
View File
@@ -0,0 +1,56 @@
"""Tests for Frigate API helpers (no network required)."""
import pytest
class _FakeResponse:
def __init__(self, json_body, status_code=200):
self._json_body = json_body
self.status_code = status_code
self.ok = 200 <= status_code < 300
def raise_for_status(self):
if not self.ok:
raise Exception(f"HTTP {self.status_code}")
def json(self):
return self._json_body
@pytest.fixture(autouse=True)
def frigate_url(monkeypatch):
monkeypatch.setenv("FRIGATE_URL", "http://frigate.test")
yield
def test_get_all_frigate_person_files_happy_path(monkeypatch):
from winnow import frigate_api
body = {"Alice": ["a-1.webp", "a-2.webp"], "train": ["pending.webp"]}
monkeypatch.setattr(frigate_api.requests, "get", lambda *a, **k: _FakeResponse(body))
result = frigate_api.get_all_frigate_person_files()
assert result == {"Alice": ["a-1.webp", "a-2.webp"]}
def test_get_all_frigate_person_files_rejects_non_dict_body(monkeypatch):
"""A non-dict /api/faces response must not crash data.items() — it should degrade to None."""
from winnow import frigate_api
monkeypatch.setattr(frigate_api.requests, "get", lambda *a, **k: _FakeResponse(["not", "a", "dict"]))
assert frigate_api.get_all_frigate_person_files() is None
def test_get_frigate_person_files_rejects_non_dict_body(monkeypatch):
from winnow import frigate_api
monkeypatch.setattr(frigate_api.requests, "get", lambda *a, **k: _FakeResponse(["not", "a", "dict"]))
assert frigate_api.get_frigate_person_files("Alice") is None
def test_get_frigate_face_counts_rejects_non_dict_body(monkeypatch):
from winnow import frigate_api
monkeypatch.setattr(frigate_api.requests, "get", lambda *a, **k: _FakeResponse("oops"))
assert frigate_api.get_frigate_face_counts() is None
def test_get_all_frigate_person_files_rejects_scalar_body(monkeypatch):
from winnow import frigate_api
monkeypatch.setattr(frigate_api.requests, "get", lambda *a, **k: _FakeResponse(42))
assert frigate_api.get_all_frigate_person_files() is None
+121
View File
@@ -1,5 +1,8 @@
"""Tests for upload tracker — mark, filter, reset, and summary logic.""" """Tests for upload tracker — mark, filter, reset, and summary logic."""
import fcntl
import json
import os
import pytest import pytest
@@ -260,3 +263,121 @@ def test_get_most_redundant_exclude_all_returns_none():
mark_uploaded("asset-a", person_name="Alice", score=0.5, frigate_score=0.70) mark_uploaded("asset-a", person_name="Alice", score=0.5, frigate_score=0.70)
record_frigate_file("Alice", "Alice-a.webp", "asset-a") record_frigate_file("Alice", "Alice-a.webp", "asset-a")
assert get_most_redundant_mapped_file("Alice", exclude={"Alice-a.webp"}) is None assert get_most_redundant_mapped_file("Alice", exclude={"Alice-a.webp"}) is None
# ── cross-process locking ──────────────────────────────────────────────────────
def test_tracker_lock_blocks_concurrent_exclusive_access():
"""While upload_tracker holds the tracker lock, a second (non-blocking) attempt
to exclusively lock the same file must fail — proving the lock is real and
guards a second winnow process from racing a read-modify-write."""
from winnow import upload_tracker as ut
ut._acquire_lock()
try:
lock_path = ut._lock_path()
assert lock_path.exists()
fd = os.open(lock_path, os.O_RDWR)
try:
with pytest.raises(OSError):
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
finally:
os.close(fd)
finally:
ut._release_lock()
# Released — a second exclusive, non-blocking lock now succeeds immediately.
fd = os.open(ut._lock_path(), os.O_RDWR)
try:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
fcntl.flock(fd, fcntl.LOCK_UN)
finally:
os.close(fd)
def test_tracker_lock_reentrant_within_process():
"""Nested acquisitions (e.g. begin_batch() for both tracker files, or a tracker
call made while a batch is open) must not deadlock the process that holds them."""
from winnow import upload_tracker as ut
assert ut._lock_depth == 0
ut._acquire_lock()
ut._acquire_lock()
assert ut._lock_depth == 2
ut._release_lock()
assert ut._lock_depth == 1
ut._release_lock()
assert ut._lock_depth == 0
def test_begin_flush_batch_releases_lock_for_next_caller():
"""begin_batch()/flush_batch() (including a nested mark_uploaded call inside the
batch) must fully release the lock so it doesn't stay held for the rest of the run."""
from winnow import upload_tracker as ut
from winnow.upload_tracker import (
REJECT_TRACKER_FILE,
UPLOAD_TRACKER_FILE,
begin_batch,
flush_batch,
mark_uploaded,
)
begin_batch(UPLOAD_TRACKER_FILE)
begin_batch(REJECT_TRACKER_FILE)
mark_uploaded("a1", person_name="Alice")
flush_batch(UPLOAD_TRACKER_FILE)
flush_batch(REJECT_TRACKER_FILE)
assert ut._lock_depth == 0
fd = os.open(ut._lock_path(), os.O_RDWR)
try:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
fcntl.flock(fd, fcntl.LOCK_UN)
finally:
os.close(fd)
# ── batched writes: incremental flush bounds crash loss ───────────────────────
def test_batch_defers_writes_below_flush_threshold(isolated_cache):
"""Below the flush threshold, writes stay in memory — batching still avoids
a disk write per mark."""
from winnow.upload_tracker import UPLOAD_TRACKER_FILE, begin_batch, flush_batch, mark_uploaded
tracker_path = isolated_cache / UPLOAD_TRACKER_FILE
begin_batch(UPLOAD_TRACKER_FILE)
try:
mark_uploaded("asset-0", person_name="Alice")
assert not tracker_path.exists()
finally:
flush_batch(UPLOAD_TRACKER_FILE)
assert tracker_path.exists()
def test_batch_flushes_incrementally_bounding_crash_loss(isolated_cache):
"""A crash mid-batch (SIGKILL/OOM/host crash) must lose at most a bounded number
of marks, not the whole per-person batch — verified by reading the file straight
off disk before flush_batch() is ever called."""
from winnow.upload_tracker import _BATCH_FLUSH_EVERY, UPLOAD_TRACKER_FILE, begin_batch, flush_batch, mark_uploaded
tracker_path = isolated_cache / UPLOAD_TRACKER_FILE
begin_batch(UPLOAD_TRACKER_FILE)
try:
for i in range(_BATCH_FLUSH_EVERY):
mark_uploaded(f"asset-{i}", person_name="Alice")
# Threshold reached: disk already reflects all marks so far, even though
# flush_batch() has not run yet — simulates surviving a crash here.
on_disk = json.loads(tracker_path.read_text())
ids = on_disk["by_person"]["Alice"]["asset_ids"]
assert len(ids) == _BATCH_FLUSH_EVERY
# One more mark past the threshold stays deferred again until the next
# periodic flush or flush_batch().
mark_uploaded("asset-extra", person_name="Alice")
on_disk = json.loads(tracker_path.read_text())
assert len(on_disk["by_person"]["Alice"]["asset_ids"]) == _BATCH_FLUSH_EVERY
finally:
flush_batch(UPLOAD_TRACKER_FILE)
on_disk = json.loads(tracker_path.read_text())
assert len(on_disk["by_person"]["Alice"]["asset_ids"]) == _BATCH_FLUSH_EVERY + 1
Generated
+138 -99
View File
@@ -269,52 +269,42 @@ wheels = [
[[package]] [[package]]
name = "numpy" name = "numpy"
version = "2.4.6" version = "2.5.0"
source = { registry = "https://pypi.org/simple" } source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d0/ad/fed0499ce6a338d2a03ebae59cd15093910c8875328855781952abf6c2fe/numpy-2.4.6.tar.gz", hash = "sha256:f3a3570c4a2a16746ac2c31a7c7c7b0c186b95ce902e33db6f28094ed7387dda", size = 20735807, upload-time = "2026-05-18T23:37:14.07Z" } sdist = { url = "https://files.pythonhosted.org/packages/e7/05/3d27272d30698dc0ecb7fdfaa41ad70303b444f81722bb99bce1d818638a/numpy-2.5.0.tar.gz", hash = "sha256:5a129578019311b6e56bdd714250f19b518f7dceeeb8d1af5490f4942d3f891c", size = 20652461, upload-time = "2026-06-21T20:57:51.95Z" }
wheels = [ wheels = [
{ url = "https://files.pythonhosted.org/packages/fb/82/bdab26d7438c6791ca31b7c024ca37c1eab8b726ba236129005cd4a06e45/numpy-2.4.6-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:511dbaf848decaaaf4b4ca48032619fb3138710c4bf7da7617765edad1ef96b0", size = 16684648, upload-time = "2026-05-18T23:34:29.41Z" }, { url = "https://files.pythonhosted.org/packages/8a/33/07675aaad7f26ea013d5e884d9a0d784b79c6bd7566c333f5a52fa3c610b/numpy-2.5.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:520e6b8be0a4b65840ac8090d4f51cef4bed66e2b0894d5a520f099adc24a9b2", size = 16784890, upload-time = "2026-06-21T20:56:40.799Z" },
{ url = "https://files.pythonhosted.org/packages/1b/30/a80189bcc7f5e4258b3fbc3968d909d1756f54d023299ecc39ad6fdb9ef8/numpy-2.4.6-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bf162abab1c1a736333192707cef898e735a5ca00f38f27eeedf44b39d9e85eb", size = 14693902, upload-time = "2026-05-18T23:34:33.013Z" }, { url = "https://files.pythonhosted.org/packages/85/4b/953118a730ee3b35e28645e0eb4cf9beec5bdbb954e1ac2f5fcefba6bbc3/numpy-2.5.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:146b81cdd3967fdb6beca8ba25f00c58741d8f3cbd797f55af0fbe0bfec3469c", size = 11754584, upload-time = "2026-06-21T20:56:43.094Z" },
{ url = "https://files.pythonhosted.org/packages/97/12/70b5d0d7c15e1ebb8a6a84a8caa1d19e181d84fb58bb6d70aca29099dec1/numpy-2.4.6-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:043191bfa8eab18c776647b62723ac9dddece59743b13f49b2016094129c2b3f", size = 5198992, upload-time = "2026-05-18T23:34:36.132Z" }, { url = "https://files.pythonhosted.org/packages/44/9b/56dd530c367c74ae17411027cea4135ca57e1e0583bf5594cee18bd83217/numpy-2.5.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:126b88d95e8ff9b00c9e717aa540469f21d6180162f84c0caec51b16215d49cd", size = 5313904, upload-time = "2026-06-21T20:56:45.503Z" },
{ url = "https://files.pythonhosted.org/packages/ba/8c/ebd2a8f8a83541f8d38cc5667e8c2b69cecfd30da6e45693e8158857d44b/numpy-2.4.6-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:6180d8b35af935aed8ece3a85e0a43f87393ae0ac87c8d2c8bd2c993f7270ef3", size = 6546944, upload-time = "2026-05-18T23:34:38.484Z" }, { url = "https://files.pythonhosted.org/packages/ce/b0/bcd672edad27ecca7da1f7bb0ce72cd1706a4f2d79ae94990afc97c13e1c/numpy-2.5.0-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:d4313cef1594c5ce46c31b6e54e918338f63f16ee9322304e8c9114d6d81c8bd", size = 6648504, upload-time = "2026-06-21T20:56:47.567Z" },
{ url = "https://files.pythonhosted.org/packages/bb/c5/7b863a97a91671a0338f4253bd3b5a3d3852f0692dae91711c9f4a10e787/numpy-2.4.6-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:72fbe16c6fac95aedf5937fa873445cec2110be35d8a4e9433d7501fd98dae6b", size = 15669392, upload-time = "2026-05-18T23:34:41.257Z" }, { url = "https://files.pythonhosted.org/packages/80/9e/15cdfcbd30a1544a46c9e487a00df331c4672450216538705a9e51fa6710/numpy-2.5.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:750fb097caf26fa878746d9d119f6f9da12dedcbff1eea966c3e3447647c4a9e", size = 15150086, upload-time = "2026-06-21T20:56:49.352Z" },
{ url = "https://files.pythonhosted.org/packages/a5/9d/3584b9984ca4c047aea75214ce1a4c4c73d849bd71b604264b7f5653f8a8/numpy-2.4.6-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a7830bab239b79cda9c08c2da014761cafb48da6150e1da17ac06283f43b6089", size = 16633220, upload-time = "2026-05-18T23:34:45.075Z" }, { url = "https://files.pythonhosted.org/packages/32/4e/8d7656ccaab3e81e97258b8a9bc5f0c8502513a92fb4ceb0a2cbfebc17bf/numpy-2.5.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3893adc2dc7c0412ba76777db55a049215d99c9aa3113003be8f49f4f1290ab9", size = 16647250, upload-time = "2026-06-21T20:56:51.542Z" },
{ url = "https://files.pythonhosted.org/packages/05/ae/7c67fba23bd98caec7c99261f3a16072ade14813486b0282cb29846de832/numpy-2.4.6-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ef4aea96ce4d3b074422cb4f2f64e216bf9e213004bb58ecfdf50ea02ea8eb9a", size = 17020800, upload-time = "2026-05-18T23:34:49.065Z" }, { url = "https://files.pythonhosted.org/packages/3c/81/97060281b602ed07f21b12f4ec409eac1f75a2f91fbc829ed8b2becf3ad4/numpy-2.5.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:835e454dd99b238cdc5a3f63bce2371296f5ebc53ca1e0f8e6ddbb6d92a29aab", size = 16512864, upload-time = "2026-06-21T20:56:55.401Z" },
{ url = "https://files.pythonhosted.org/packages/d9/5d/3b6725cb31d983c5e66916f5d36f6d7e5521129e4c4404d64f918292a5b6/numpy-2.4.6-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:dfa20cc6ca228e6b155b11da03825975ce66aea520985dbbddf0f2a5a495c605", size = 18357600, upload-time = "2026-05-18T23:34:52.709Z" }, { url = "https://files.pythonhosted.org/packages/33/ab/4496208146911f8d8ddb54f68a972aafa6c8d44babcb2ea03b0e5cc87c9d/numpy-2.5.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6f9836778081a0a3c02a6a21493f3e9f5b311f8d2541934f31f05583dc999ea4", size = 18408407, upload-time = "2026-06-21T20:56:57.75Z" },
{ url = "https://files.pythonhosted.org/packages/f7/da/2ccc6c2fe8898dee01d90c75c5f5f914a23daf99e3e0f59516a08760c8b5/numpy-2.4.6-cp313-cp313-win32.whl", hash = "sha256:56b39e5e0622a09a25bf5baf62f4bcf0cb8a41ae6e2819cf49bbc5a74c083f91", size = 5961134, upload-time = "2026-05-18T23:34:55.618Z" }, { url = "https://files.pythonhosted.org/packages/d4/9f/a4df67c181e4ee8b467aa3332dc2db10fd5c515136831302f3ca48bc0a01/numpy-2.5.0-cp313-cp313-win32.whl", hash = "sha256:0b525be4744b60bb0557ac872d53ef07d085b5f39622bc579c98d3809d05b988", size = 6054431, upload-time = "2026-06-21T20:57:00.016Z" },
{ url = "https://files.pythonhosted.org/packages/b5/cd/9cc4dc876fb065d5c220aae4d5e14826b2715331bb7618ce1fb07a679d99/numpy-2.4.6-cp313-cp313-win_amd64.whl", hash = "sha256:c4fc99836233ea196540b17ab0983aff60ed07941751930f5f4d05bc3b3b7359", size = 12318598, upload-time = "2026-05-18T23:34:58.928Z" }, { url = "https://files.pythonhosted.org/packages/30/53/491e1c47c55b62ccc6a63c1c5b8635c73fc2258dddeb9bda27cae4a0ae96/numpy-2.5.0-cp313-cp313-win_amd64.whl", hash = "sha256:44353e2878930039db472b99dc353d749826e4010bd4d2a7f835e94a97a5c748", size = 12414420, upload-time = "2026-06-21T20:57:01.815Z" },
{ url = "https://files.pythonhosted.org/packages/39/1e/c0bcba1f8694116485fe28fd1be698c278fcda4141c5b0e53a2aed8b12a8/numpy-2.4.6-cp313-cp313-win_arm64.whl", hash = "sha256:a7c711e21628b52034bb5ab8d1bce291f752fcc5e92accc615778acee1ff4778", size = 10222272, upload-time = "2026-05-18T23:35:02.167Z" }, { url = "https://files.pythonhosted.org/packages/eb/4a/25c2906f541e9d9f4c5769764db732e6627be91a13f4724fa10634d77db4/numpy-2.5.0-cp313-cp313-win_arm64.whl", hash = "sha256:48f54b00711f83a5f796b70c518e8c2b3c5848dda03a54911f23eb68519b9b60", size = 10339533, upload-time = "2026-06-21T20:57:03.961Z" },
{ url = "https://files.pythonhosted.org/packages/63/6d/cc5619247c8f4204e507f5883528372e4ac4bb189e579fb859a12e480b1f/numpy-2.4.6-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:112b06a867b235ef466ed3508ddf0238050df9c727cafb5301ac385b899189a1", size = 14821197, upload-time = "2026-05-18T23:35:05.468Z" }, { url = "https://files.pythonhosted.org/packages/86/ad/abc44aaceaf7b17ee1edde2bbb4458da591bc79574cffff50c4bb35f00d1/numpy-2.5.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:f27582c55ba4c750b7c58c8faf021d2cd9324a662b466229db8a417b41368af9", size = 16783807, upload-time = "2026-06-21T20:57:06.253Z" },
{ url = "https://files.pythonhosted.org/packages/00/58/f1c39161c87d9e9bed660f1ed4bafc0e403d5ec9650b6dd77aead07d489b/numpy-2.4.6-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:eaf7fa2de5c0be8ae6ff8e9bea2ccd725e980541244521d8d4b5f3354a27babe", size = 5326287, upload-time = "2026-05-18T23:35:08.693Z" }, { url = "https://files.pythonhosted.org/packages/5d/39/b72e168daf9c00fb20c9fc996d00437ccecdef3102387775d29d7a62576d/numpy-2.5.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:28e7137057d551e4a83c4ae414e3451f50568409db7569aacc7f9811ee06a446", size = 11765215, upload-time = "2026-06-21T20:57:08.547Z" },
{ url = "https://files.pythonhosted.org/packages/af/57/3917ab0fd97f271a8694513581b8a36c655f111c446852c302f04ccdb6fc/numpy-2.4.6-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:7265a2f3d436e54ef9f2b52b5c937e6be778781bd97a590319d7348f1c1ca997", size = 6646763, upload-time = "2026-05-18T23:35:11.459Z" }, { url = "https://files.pythonhosted.org/packages/f7/a0/8400a9c0e3625182347593f5e1f57da9a617a534794805c8df5518154ddc/numpy-2.5.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:e1da54b53e75cd9fcfc23efcc7edab2c6aecf97b6037566d8a0fe804af8ec57c", size = 5324493, upload-time = "2026-06-21T20:57:11.012Z" },
{ url = "https://files.pythonhosted.org/packages/eb/0f/037e64c494b67581ae18193d770adef354c41f3f2c8ebf865602d949bf8f/numpy-2.4.6-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f74a575920ab21fe304421a3fc28793d82e299cae9eccb37084e9fc7f3617c20", size = 15728070, upload-time = "2026-05-18T23:35:14.79Z" }, { url = "https://files.pythonhosted.org/packages/f6/8c/0d104deaa0401c93395a629ec902891618a2eff76d19229139cb5a887bfc/numpy-2.5.0-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:694d8f74e156f7fd01179f1aa8faa2f648ab6ae0f70b6c3fe57a03249aea2303", size = 6645211, upload-time = "2026-06-21T20:57:12.919Z" },
{ url = "https://files.pythonhosted.org/packages/21/a6/5d2bae9c9542eb4df16dc9c46dc79c186e9bad53805dfa5399a6023c6db0/numpy-2.4.6-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ede83e07a75dd06bc501566c1eca2afc0d61677c1472ac9ad93fdee6e638a48d", size = 16681752, upload-time = "2026-05-18T23:35:18.836Z" }, { url = "https://files.pythonhosted.org/packages/6a/d9/4a4a628c812750363786afc3d33492709a5cd64b215469c16b0f6c7bb811/numpy-2.5.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1a7569a7b53c77716f036bb28cb1c91f166a26ec7d9502cd1e4bdfe502fdec22", size = 15166004, upload-time = "2026-06-21T20:57:14.717Z" },
{ url = "https://files.pythonhosted.org/packages/92/14/23d1dfb410ae362cd59ce53e936b1513d545eb40db3949ced632e19a459e/numpy-2.4.6-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:68bb27509ac1b9a3443094260f6326150663b06abe40b73a2f81160623da5b67", size = 17086024, upload-time = "2026-05-18T23:35:22.52Z" }, { url = "https://files.pythonhosted.org/packages/a0/5e/2a902317d7fc4aa93236e80c932662dadfc459b323d758329e01775125e1/numpy-2.5.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:39a0433bd4086ebd462960cf375e19195bb07b53dc1d87dd5fcf47ad78576f03", size = 16650797, upload-time = "2026-06-21T20:57:16.906Z" },
{ url = "https://files.pythonhosted.org/packages/4b/6e/23595a2c642cdf3bc567877064bdd7f91c8b0038a4453cf2daf7248eafe9/numpy-2.4.6-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:a0df0043bdb289bde1f62da130d20df23d58b45429f752bc7a8fc5325a225ecd", size = 18403398, upload-time = "2026-05-18T23:35:26.398Z" }, { url = "https://files.pythonhosted.org/packages/e9/a0/a0090e6329f4ca5992c07847bb579c5259a19953dc57255bb08793142ffb/numpy-2.5.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:929f0c79ac38bcbd7154fe631dc907abfeddbcc5027a896bd1f7767323271e7a", size = 16524647, upload-time = "2026-06-21T20:57:19.165Z" },
{ url = "https://files.pythonhosted.org/packages/8a/90/0ac3bc947217e66dec77e7cbc6a1979d1af70b6461b82f620d3bccd5e4c8/numpy-2.4.6-cp313-cp313t-win32.whl", hash = "sha256:29a287e0cf63ff528da061de6b9f64a4618da591ca1046aafc54062e40ca7eab", size = 6084971, upload-time = "2026-05-18T23:35:29.387Z" }, { url = "https://files.pythonhosted.org/packages/5e/7d/6caf27734c42b65837e7461ed0dbbd6b6fc835060c9714ec59d673bb383a/numpy-2.5.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:cc4f247a47bbf070bfd70be53ccdcf47b800af563535e7bbe172322197c30e21", size = 18411841, upload-time = "2026-06-21T20:57:21.638Z" },
{ url = "https://files.pythonhosted.org/packages/77/71/5673e351671a1d2bd6063b91b44f70c0affea7d1516fa7a6572941ba4aa1/numpy-2.4.6-cp313-cp313t-win_amd64.whl", hash = "sha256:25c692919ac5a01f170a3bfcd62d745b24fd095c353d50812637d6fcab442e75", size = 12458532, upload-time = "2026-05-18T23:35:32.175Z" }, { url = "https://files.pythonhosted.org/packages/13/dc/26edadbd812536769a82c2e9e002234e33feb5da43061d47a044f6d309b7/numpy-2.5.0-cp314-cp314-win32.whl", hash = "sha256:5dc71423499fab3f46f7a7201155ade1669ea101f2f429d332df9e72f8161731", size = 6106361, upload-time = "2026-06-21T20:57:23.844Z" },
{ url = "https://files.pythonhosted.org/packages/3f/88/19d3503c5046e688f049274b27a3ef3d771152fa80d3ba3d01a3dff61abe/numpy-2.4.6-cp313-cp313t-win_arm64.whl", hash = "sha256:1e978ec1e8bd0e0e4de6bb75de9d30cbb74db6b6a2bb727618613703ca0167dd", size = 10291881, upload-time = "2026-05-18T23:35:35.465Z" }, { url = "https://files.pythonhosted.org/packages/f2/9e/4dd1459282229a72d92dece2ae9138e5cac94a72263a7ceb48f37434c925/numpy-2.5.0-cp314-cp314-win_amd64.whl", hash = "sha256:ebb81d9d5443e0309d6c54894c3fbed74ad7da0714352a67b6d773cd189eae73", size = 12551749, upload-time = "2026-06-21T20:57:25.945Z" },
{ url = "https://files.pythonhosted.org/packages/f8/91/3ab2044d05fd16d343c5ac2e69b127f1b2854040dd20b193257c78028bd3/numpy-2.4.6-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:06ca2f61ec4385a07a6977c55ba998a4466c123642b4a32694d3128fce18c079", size = 16683458, upload-time = "2026-05-18T23:35:38.353Z" }, { url = "https://files.pythonhosted.org/packages/05/a7/6bc6384c080b86c7f6c85c5bc5b540b24f4f679cd144791d99574e90d462/numpy-2.5.0-cp314-cp314-win_arm64.whl", hash = "sha256:3b94d0d0deceebfad3e67ae5c0e5eb87371e8f7a0581cd04a779928c2450cf1e", size = 10617072, upload-time = "2026-06-21T20:57:28.175Z" },
{ url = "https://files.pythonhosted.org/packages/8e/62/764ce66fa4147ae6d73071a3abf804ffe606f174618697c571acdf26a7c9/numpy-2.4.6-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:38efbc8de75c7a0fc1ac190162d892787f3f47b57cc291231aafee36b80982b7", size = 14704559, upload-time = "2026-05-18T23:35:42.14Z" }, { url = "https://files.pythonhosted.org/packages/86/6b/4a2b71d66ada5608ae02b63f150dfad520f6940721cb7f029ad270befc0e/numpy-2.5.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:22f3d43e362d650bc39db1f17851302874a148ca95ba6981c1dfb5fa6862f35b", size = 11881067, upload-time = "2026-06-21T20:57:30.104Z" },
{ url = "https://files.pythonhosted.org/packages/60/61/23f27c172f022e04025b7dc2367f4d63c1a398120607ec896228649a6f48/numpy-2.4.6-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:d581b735e177fdcdce6fed8e7e8880a3fb6ee4e3653a3ac6af01c6f4c03effc5", size = 5209716, upload-time = "2026-05-18T23:35:45.377Z" }, { url = "https://files.pythonhosted.org/packages/dc/b2/d365eb40a20efb49d67e9feb90494ed8511282ee1f5fa16006675c65397d/numpy-2.5.0-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:243563efb4cd7528a264567e9fd206c87826457322521d06206a00bfa316c927", size = 5440290, upload-time = "2026-06-21T20:57:32.193Z" },
{ url = "https://files.pythonhosted.org/packages/03/71/21cf70dc6ea3e3acb95fc53a265b2fc248b981f0194ceb5b475271b8809d/numpy-2.4.6-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:0a041d3d761dc3c35cc56ce0351506a02bcbc25f7b169f652435141a17db9096", size = 6543947, upload-time = "2026-05-18T23:35:47.926Z" }, { url = "https://files.pythonhosted.org/packages/fa/5e/e9c03188de5f9b767e46a8fe988bcfd3efad066a4a3fda8b9cb11a93f895/numpy-2.5.0-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:84881d825ca75249b189bbee875fcfe3238aa5c479e6100893cda566e8e86826", size = 6748371, upload-time = "2026-06-21T20:57:33.933Z" },
{ url = "https://files.pythonhosted.org/packages/d5/91/64288395ee1799bd2e0b04a305dce9666da90c961e1f3fe982a05ee1c036/numpy-2.4.6-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:40fdc1ae7125e518ea98e53e69a4ebc27e1fd50510c47b7ea130cf21e5e1d42b", size = 15685197, upload-time = "2026-05-18T23:35:50.863Z" }, { url = "https://files.pythonhosted.org/packages/fd/1d/68c186a38a5027bae2c4ddd5ea681fdaf8b4d30fb7301def6d8ad270390f/numpy-2.5.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:cda12aa4779d42b8771180aba759c96f527d43446d8f380ab59e2b35e8489efd", size = 15214643, upload-time = "2026-06-21T20:57:35.677Z" },
{ url = "https://files.pythonhosted.org/packages/f3/eb/ebffaa97dc55502df69584a8f0dcf07f69a3e0b3e2323670a2722db9aa39/numpy-2.4.6-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a2c306dea656c12c68f51f4cea133cbe78ca7435eb28c735eac1d3ebe73be6e8", size = 16638245, upload-time = "2026-05-18T23:35:54.752Z" }, { url = "https://files.pythonhosted.org/packages/8c/67/73f67b7c7e20635baae9c4c3ead4ae7326a005900297a6110971abd62eb5/numpy-2.5.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1c0121101093d2bd74981b10f8837d78e794a8ff57834eb27179f49e1ba11ac6", size = 16690128, upload-time = "2026-06-21T20:57:38.159Z" },
{ url = "https://files.pythonhosted.org/packages/b8/0b/54f9da33128d7e350fab89c7455902eeae70349ee52bddb448dc4a576f45/numpy-2.4.6-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:33111801a01c12a8a1e3721f0a9232f8cfc8ae2c6b7098167e6f623c6073f402", size = 17036587, upload-time = "2026-05-18T23:35:58.355Z" }, { url = "https://files.pythonhosted.org/packages/eb/05/d4c1fb0c46d02a27d6b2b8b319a78c90937acec8631c1641874670b31e6f/numpy-2.5.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:d371c92cfa09da00022f501ab67fafaea813d752eb30ac44336d45b1e5b0268a", size = 16577902, upload-time = "2026-06-21T20:57:40.447Z" },
{ url = "https://files.pythonhosted.org/packages/b6/f0/fdebc1052db1cc37c64beb22072d67cd6d1c71adca1299f53dec2b5e20d3/numpy-2.4.6-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ae506e6902902557576a26ff33eda8695e7ecb3cb36c3b573a0765dee114ebdb", size = 18363226, upload-time = "2026-05-18T23:36:02.845Z" }, { url = "https://files.pythonhosted.org/packages/9e/1d/771c797d50fa26e4888989cccf1d50ee51f530d4e455ad2692dcb64fa711/numpy-2.5.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:9990713e9c38154c6861e7547f1e3fc7a87e75ff09bab24ef1cc81d81c2835e9", size = 18452814, upload-time = "2026-06-21T20:57:42.875Z" },
{ url = "https://files.pythonhosted.org/packages/aa/b4/298628d98c72b57e57f7165ae6a481a1deaf6f3c28262a6e4c739c275930/numpy-2.4.6-cp314-cp314-win32.whl", hash = "sha256:aaf159caa35993cb1f56fb9b8e4610d35758e7ca005412eb1daa856a78c9c4b1", size = 6010196, upload-time = "2026-05-18T23:36:05.92Z" }, { url = "https://files.pythonhosted.org/packages/e8/46/52fc0d2a68d7643f0f149eeea5a5d8ea2a3507056ac8afa83c9212606e8b/numpy-2.5.0-cp314-cp314t-win32.whl", hash = "sha256:edadfbd4794b1086c0d822f81863e8a68fc129d132fd0bb9e31e955d7fbbbdb7", size = 6253168, upload-time = "2026-06-21T20:57:45.101Z" },
{ url = "https://files.pythonhosted.org/packages/df/ac/46de6dda46478f7942f839e094970be2d4a861e005c4b3bf07c92e291a09/numpy-2.4.6-cp314-cp314-win_amd64.whl", hash = "sha256:b507f5c4c1d508876d1819b6bf9a49d365b96320b5d4993426b33a23ca4b8261", size = 12450334, upload-time = "2026-05-18T23:36:09.107Z" }, { url = "https://files.pythonhosted.org/packages/2a/be/6c8d1118b5f13b2881dc095d5b345de19c6638b8959c17409b6eff84c8aa/numpy-2.5.0-cp314-cp314t-win_amd64.whl", hash = "sha256:f7e5fa4382967ae6548bd2f174219afb908e294b0d5f625af01166edd5f7d9aa", size = 12736286, upload-time = "2026-06-21T20:57:46.935Z" },
{ url = "https://files.pythonhosted.org/packages/78/92/b8b798ac784102c0da830d2257d59358e3d3d90d1e2b3f2575dad976c5cf/numpy-2.4.6-cp314-cp314-win_arm64.whl", hash = "sha256:6f41ae150c4e32db4f3310cdaf64b1593a03dbabe29eec77fc9b50fe64061df6", size = 10495678, upload-time = "2026-05-18T23:36:12.766Z" }, { url = "https://files.pythonhosted.org/packages/fd/6a/d3a169aaf8536cf228d56a09e04bcb713a2fe4410d4e2105b9419b5a9c89/numpy-2.5.0-cp314-cp314t-win_arm64.whl", hash = "sha256:016623417bb330d719d579daf2d6b9a01ddc52e41a9ed61a47f39fde46dcd865", size = 10686451, upload-time = "2026-06-21T20:57:49.313Z" },
{ url = "https://files.pythonhosted.org/packages/30/34/ec28d1aa8115971537c01469ab2011ee96827930f0a124de1000cc2a7ed7/numpy-2.4.6-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ece3d2cfe132e7d51f44a832b303895e6f2d499c5e74dfbdb06ee246147a304a", size = 14823672, upload-time = "2026-05-18T23:36:16.473Z" },
{ url = "https://files.pythonhosted.org/packages/16/bd/f6d1fede4e54e8042a7ff97bb495510f3c220f94bcd9e8b228e87c92cc0d/numpy-2.4.6-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:e3e5193ef5a3dc73bceee50f7fdc2c90dbb76c42df8d8fae3d1067a583df579e", size = 5328731, upload-time = "2026-05-18T23:36:19.767Z" },
{ url = "https://files.pythonhosted.org/packages/f4/f0/e105b9e2fd728a9910103884decd6951d9dd73896b914a98d9a231de02ee/numpy-2.4.6-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:17f9ade344e7d9b464a084d69bcf18fc691cb1db67c62ed80820bf4926d78f0e", size = 6649805, upload-time = "2026-05-18T23:36:22.266Z" },
{ url = "https://files.pythonhosted.org/packages/82/dd/1206a7ca6ab15e3f02069707ca96222e202af681bb73756da7527f3cb837/numpy-2.4.6-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9cd5ffd25db4e7ba6a375693b3fc0fc1791ec636c17db3720da19bde7180ec43", size = 15730496, upload-time = "2026-05-18T23:36:25.713Z" },
{ url = "https://files.pythonhosted.org/packages/51/e7/38d3ea825dcab85a591734decb2f6c67caa7c8367d374df1a1c3842f9b07/numpy-2.4.6-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7d92c3819208a60205a12a245c91ad70cb0a85336659b19b834205573ac8456e", size = 16679616, upload-time = "2026-05-18T23:36:29.652Z" },
{ url = "https://files.pythonhosted.org/packages/93/b7/caabfdf53edf663e0b4eb74d7d405d83baef09eb5e83bcd32d601d72b93e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e85b752a1e912b70eaad4fafbd4d1238007ab221de2009b9a2f5ae7461239895", size = 17085145, upload-time = "2026-05-18T23:36:33.449Z" },
{ url = "https://files.pythonhosted.org/packages/f9/45/68d7c33a6bcf3e5aa3bdbd57a367e6f615286dfd6482f97e8ffeb734306e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:29cb7f67d10b479ff07c17d33e39f78c07f71c40ef30d63c153d340e96cd3fb4", size = 18403813, upload-time = "2026-05-18T23:36:37.369Z" },
{ url = "https://files.pythonhosted.org/packages/9c/50/0753655aa844c99cd9e018aacf76f130f1bd81d881bb74bc0aef5d73a8ba/numpy-2.4.6-cp314-cp314t-win32.whl", hash = "sha256:260a5d70215b61ab4fadf5c7baacd64821842975eea312125ed3c39a6391b063", size = 6156982, upload-time = "2026-05-18T23:36:40.817Z" },
{ url = "https://files.pythonhosted.org/packages/b2/d4/7c67becf668f973cb490cec3e98dfd799d866f9c989a54d355672cfa0db6/numpy-2.4.6-cp314-cp314t-win_amd64.whl", hash = "sha256:81a1cca95ed5bb92aa8b10dd2cdc9a0d3853a50fad926c28b5d7e8ea54389627", size = 12638908, upload-time = "2026-05-18T23:36:43.996Z" },
{ url = "https://files.pythonhosted.org/packages/43/bb/e1c71a4295b1b1d1393d50dbb4f2a36283c6859d9d3892e84f00ec5a91d5/numpy-2.4.6-cp314-cp314t-win_arm64.whl", hash = "sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66", size = 10565867, upload-time = "2026-05-18T23:36:47.114Z" },
] ]
[[package]] [[package]]
@@ -340,17 +330,60 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/52/de/823919be3b9d0ccbf1f784035423c5f18f4267fb0123558d58b813c6ec86/nvidia_cuda_nvrtc_cu12-12.9.86-py3-none-win_amd64.whl", hash = "sha256:72972ebdcf504d69462d3bcd67e7b81edd25d0fb85a2c46d3ea3517666636349", size = 76408187, upload-time = "2025-06-05T20:12:27.819Z" }, { url = "https://files.pythonhosted.org/packages/52/de/823919be3b9d0ccbf1f784035423c5f18f4267fb0123558d58b813c6ec86/nvidia_cuda_nvrtc_cu12-12.9.86-py3-none-win_amd64.whl", hash = "sha256:72972ebdcf504d69462d3bcd67e7b81edd25d0fb85a2c46d3ea3517666636349", size = 76408187, upload-time = "2025-06-05T20:12:27.819Z" },
] ]
[[package]]
name = "nvidia-cuda-runtime-cu12"
version = "12.9.79"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/bc/e0/0279bd94539fda525e0c8538db29b72a5a8495b0c12173113471d28bce78/nvidia_cuda_runtime_cu12-12.9.79-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:83469a846206f2a733db0c42e223589ab62fd2fabac4432d2f8802de4bded0a4", size = 3515012, upload-time = "2025-06-05T20:00:35.519Z" },
{ url = "https://files.pythonhosted.org/packages/bc/46/a92db19b8309581092a3add7e6fceb4c301a3fd233969856a8cbf042cd3c/nvidia_cuda_runtime_cu12-12.9.79-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:25bba2dfb01d48a9b59ca474a1ac43c6ebf7011f1b0b8cc44f54eb6ac48a96c3", size = 3493179, upload-time = "2025-06-05T20:00:53.735Z" },
{ url = "https://files.pythonhosted.org/packages/59/df/e7c3a360be4f7b93cee39271b792669baeb3846c58a4df6dfcf187a7ffab/nvidia_cuda_runtime_cu12-12.9.79-py3-none-win_amd64.whl", hash = "sha256:8e018af8fa02363876860388bd10ccb89eb9ab8fb0aa749aaf58430a9f7c4891", size = 3591604, upload-time = "2025-06-05T20:11:17.036Z" },
]
[[package]] [[package]]
name = "nvidia-cudnn-cu12" name = "nvidia-cudnn-cu12"
version = "9.23.1.3" version = "9.23.2.1"
source = { registry = "https://pypi.org/simple" } source = { registry = "https://pypi.org/simple" }
dependencies = [ dependencies = [
{ name = "nvidia-cublas-cu12", marker = "platform_machine != 's390x'" }, { name = "nvidia-cublas-cu12", marker = "platform_machine != 's390x'" },
] ]
wheels = [ wheels = [
{ url = "https://files.pythonhosted.org/packages/10/30/ecca1c8194c8077c4b57a3d96b56d96f15852551b01b919bae6429d92218/nvidia_cudnn_cu12-9.23.1.3-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:6dbc18f05aab2a323a4ffd43d985410608f7db7db9a8596e189cddbd3e527441", size = 778220760, upload-time = "2026-06-09T19:38:19.281Z" }, { url = "https://files.pythonhosted.org/packages/a4/19/e70f60a82986f65111d30ffd1744ed3478a3f8cd67bcdac38bf00d57d9da/nvidia_cudnn_cu12-9.23.2.1-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:3adf706a93367d400b4e7668fc14834c7551c9122eeb22f8aec6435b18da5b6f", size = 778221490, upload-time = "2026-06-16T03:36:51.086Z" },
{ url = "https://files.pythonhosted.org/packages/a0/7e/e0a5d44bf070a1ff945050abc02ef1cff5ca9c6ab5dc6a16ab6322593a32/nvidia_cudnn_cu12-9.23.1.3-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:272d4815eef8f0dd21ecca768bfa18a618fb76ec31547dd0885cd49e76ebcd1d", size = 721095859, upload-time = "2026-06-09T19:41:00.27Z" }, { url = "https://files.pythonhosted.org/packages/e1/ef/276c358c6ab44efbe68e69977657cb80927e7ab6d45968d042e1083f45e5/nvidia_cudnn_cu12-9.23.2.1-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:a5e706320218dc7d661b0e13402f204eeccd07b18d061b4d60668f80e464dd1e", size = 721096736, upload-time = "2026-06-16T04:39:58.1Z" },
{ url = "https://files.pythonhosted.org/packages/75/ec/62b56fc5e8219a268c6f62c4e9fb1369ebec049512328e650d1a9a28bcc8/nvidia_cudnn_cu12-9.23.1.3-py3-none-win_amd64.whl", hash = "sha256:b874af5bfab5e1010ae88bfead14bf8e9da6b20283582288f1c05f056090a398", size = 689996767, upload-time = "2026-06-09T19:44:25.343Z" }, { url = "https://files.pythonhosted.org/packages/9b/93/b37f3a0fe29b1ae3bbb42c22cc25cb152971bb400a589cade336bdf5f4f3/nvidia_cudnn_cu12-9.23.2.1-py3-none-win_amd64.whl", hash = "sha256:549d6eb120cdd89429997243cd2cad1e864aac3a2f887a93f17836ce72d83873", size = 689997647, upload-time = "2026-06-16T04:47:21.167Z" },
]
[[package]]
name = "nvidia-cufft-cu12"
version = "11.4.1.4"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-nvjitlink-cu12", marker = "platform_machine != 's390x'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/9b/2b/76445b0af890da61b501fde30650a1a4bd910607261b209cccb5235d3daa/nvidia_cufft_cu12-11.4.1.4-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:1a28c9b12260a1aa7a8fd12f5ebd82d027963d635ba82ff39a1acfa7c4c0fbcf", size = 200822453, upload-time = "2025-06-05T20:05:27.889Z" },
{ url = "https://files.pythonhosted.org/packages/95/f4/61e6996dd20481ee834f57a8e9dca28b1869366a135e0d42e2aa8493bdd4/nvidia_cufft_cu12-11.4.1.4-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c67884f2a7d276b4b80eb56a79322a95df592ae5e765cf1243693365ccab4e28", size = 200877592, upload-time = "2025-06-05T20:05:45.862Z" },
{ url = "https://files.pythonhosted.org/packages/20/ee/29955203338515b940bd4f60ffdbc073428f25ef9bfbce44c9a066aedc5c/nvidia_cufft_cu12-11.4.1.4-py3-none-win_amd64.whl", hash = "sha256:8e5bfaac795e93f80611f807d42844e8e27e340e0cde270dcb6c65386d795b80", size = 200067309, upload-time = "2025-06-05T20:13:59.762Z" },
]
[[package]]
name = "nvidia-curand-cu12"
version = "10.3.10.19"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/14/1c/2a45afc614d99558d4a773fa740d8bb5471c8398eeed925fc0fcba020173/nvidia_curand_cu12-10.3.10.19-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:de663377feb1697e1d30ed587b07d5721fdd6d2015c738d7528a6002a6134d37", size = 68292066, upload-time = "2025-05-01T19:39:13.595Z" },
{ url = "https://files.pythonhosted.org/packages/31/44/193a0e171750ca9f8320626e8a1f2381e4077a65e69e2fb9708bd479e34a/nvidia_curand_cu12-10.3.10.19-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:49b274db4780d421bd2ccd362e1415c13887c53c214f0d4b761752b8f9f6aa1e", size = 68295626, upload-time = "2025-05-01T19:39:38.885Z" },
{ url = "https://files.pythonhosted.org/packages/e5/98/1bd66fd09cbe1a5920cb36ba87029d511db7cca93979e635fd431ad3b6c0/nvidia_curand_cu12-10.3.10.19-py3-none-win_amd64.whl", hash = "sha256:e8129e6ac40dc123bd948e33d3e11b4aa617d87a583fa2f21b3210e90c743cde", size = 68774847, upload-time = "2025-05-01T19:48:52.93Z" },
]
[[package]]
name = "nvidia-nvjitlink-cu12"
version = "12.9.86"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/46/0c/c75bbfb967457a0b7670b8ad267bfc4fffdf341c074e0a80db06c24ccfd4/nvidia_nvjitlink_cu12-12.9.86-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:e3f1171dbdc83c5932a45f0f4c99180a70de9bd2718c1ab77d14104f6d7147f9", size = 39748338, upload-time = "2025-06-05T20:10:25.613Z" },
{ url = "https://files.pythonhosted.org/packages/97/bc/2dcba8e70cf3115b400fef54f213bcd6715a3195eba000f8330f11e40c45/nvidia_nvjitlink_cu12-12.9.86-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:994a05ef08ef4b0b299829cde613a424382aff7efb08a7172c1fa616cc3af2ca", size = 39514880, upload-time = "2025-06-05T20:10:04.89Z" },
{ url = "https://files.pythonhosted.org/packages/dd/7e/2eecb277d8a98184d881fb98a738363fd4f14577a4d2d7f8264266e82623/nvidia_nvjitlink_cu12-12.9.86-py3-none-win_amd64.whl", hash = "sha256:cc6fcec260ca843c10e34c936921a1c426b351753587fdd638e8cff7b16bb9db", size = 35584936, upload-time = "2025-06-05T20:16:08.525Z" },
] ]
[[package]] [[package]]
@@ -385,7 +418,7 @@ wheels = [
[[package]] [[package]]
name = "onnxruntime" name = "onnxruntime"
version = "1.26.0" version = "1.27.0"
source = { registry = "https://pypi.org/simple" } source = { registry = "https://pypi.org/simple" }
dependencies = [ dependencies = [
{ name = "flatbuffers" }, { name = "flatbuffers" },
@@ -394,25 +427,25 @@ dependencies = [
{ name = "protobuf" }, { name = "protobuf" },
] ]
wheels = [ wheels = [
{ url = "https://files.pythonhosted.org/packages/cf/a2/c801242685e0ce48a4ca51dfafbb588765e0446397e123be53ba5598f3f5/onnxruntime-1.26.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:ccce19c5f771b8268902f77d9fed9e88f9499465d6780808faa6611a789d33f0", size = 18016563, upload-time = "2026-05-08T19:07:28.081Z" }, { url = "https://files.pythonhosted.org/packages/fb/2b/54208fd03ad410480bc17edf4869376362da8bbf46fe186ddf4cb5cc20fe/onnxruntime-1.27.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:b3e5b58b8c89c2b20e086e890aa9527377e5c240dc3ecc1640d18e07705eeb1c", size = 18432958, upload-time = "2026-06-15T22:42:53.105Z" },
{ url = "https://files.pythonhosted.org/packages/e2/64/0492c0b1db04e29b2630c87cfa36f9d6872b1ca8614b90c5cad58fac7d76/onnxruntime-1.26.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bdbed8cf3b672b66acb032f33a253bc27f42bce6ece48ae3fab4fa483a5e96e0", size = 16052634, upload-time = "2026-05-08T19:07:16.885Z" }, { url = "https://files.pythonhosted.org/packages/ce/88/24fc51fcbb126da6d032372314e47b55c3faad58f2aa78c0e199ccd20b9c/onnxruntime-1.27.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:48b3d87eb560ff6a772240506f3c78d6d27c63cafedd5c775672e1194f968cfd", size = 16438180, upload-time = "2026-06-15T22:42:43.093Z" },
{ url = "https://files.pythonhosted.org/packages/3d/26/4d09ddc755a84fc8d5e192991626b0e0680e8f6c5d58f4f1d05c42bc48cf/onnxruntime-1.26.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c07af6fc6d5557835f2b6ee7a96d8b3235d0c57a8e230efdedaee106a8a3cbc6", size = 18185632, upload-time = "2026-05-08T19:07:38.756Z" }, { url = "https://files.pythonhosted.org/packages/cb/19/14929c3c2fe0b79b41cce24463062bf3afa4cdd3c19dccf00319caa92bff/onnxruntime-1.27.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6872443f236a554921cda6f318c900e2d0c226792cf3534d00e5057c6926e5d2", size = 18658445, upload-time = "2026-06-15T22:43:08.053Z" },
{ url = "https://files.pythonhosted.org/packages/77/89/3e52249aa08fa301e217ecba07b5246a8338fa2b401e109326e3fc5be0f9/onnxruntime-1.26.0-cp313-cp313-win_amd64.whl", hash = "sha256:61bec80655efa460591c2bc655392d57d2650ce85533a6b9b3b7a790d7ea7916", size = 13026751, upload-time = "2026-05-08T19:08:06.2Z" }, { url = "https://files.pythonhosted.org/packages/7f/76/59ed932b0244acd7bbbd6449480053a6d958ea66357f022f932872e19287/onnxruntime-1.27.0-cp313-cp313-win_amd64.whl", hash = "sha256:760021bca514d64a811837820d351a08a41741f16f8b4c26450da708fecf14e6", size = 13357856, upload-time = "2026-06-15T22:43:37.315Z" },
{ url = "https://files.pythonhosted.org/packages/06/b3/c1c8782b14af6797c303de132d6eef26a9fb80dfacd3750ce57911d11c6b/onnxruntime-1.26.0-cp313-cp313-win_arm64.whl", hash = "sha256:a6677545ff451e3539a02746d2f207d8c5baa4a0a818886bb9d6a6eb9511ee89", size = 12796807, upload-time = "2026-05-08T19:07:54.879Z" }, { url = "https://files.pythonhosted.org/packages/79/51/d1ec60ec7b1e2ae2d7340ba52b8a13529140039cd4407ba8dddbbc046582/onnxruntime-1.27.0-cp313-cp313-win_arm64.whl", hash = "sha256:2fdfa9df40a0ded0028ce6f9cd863264237f3970559dea2b81456e9ac4622b94", size = 13104412, upload-time = "2026-06-15T22:43:27.457Z" },
{ url = "https://files.pythonhosted.org/packages/c3/f5/47b0676408abec652c14b84d7173e389837832d850c24f87184277313e8d/onnxruntime-1.26.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5e016edc15d3c19f36807e1c6b10be5b27807688c32720f91b5ae480a95215d0", size = 16057265, upload-time = "2026-05-08T19:07:19.603Z" }, { url = "https://files.pythonhosted.org/packages/5e/7d/e6bb1c6445c94f708c38cd8fbb7bf0264108c33498b9445c93e60fe6d329/onnxruntime-1.27.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:54c0c4e9202c36c4ecdb1f3443f5dfbfd5ee3b54d1362c4b4c6134110e74fb32", size = 16443331, upload-time = "2026-06-15T22:42:45.649Z" },
{ url = "https://files.pythonhosted.org/packages/3b/45/33ab6deeef010ca844c877dd618cebc079590bbe52d2a3678e7223b1b908/onnxruntime-1.26.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f5fc48a91a046a6a5c9b147f83fb41d65d24d24923373b222cdd248f0f4f4aac", size = 18197590, upload-time = "2026-05-08T19:07:41.422Z" }, { url = "https://files.pythonhosted.org/packages/72/1b/b18b31e806eabc41077810199fbbb36fbc2d5f19912416e5ccfbf73053d1/onnxruntime-1.27.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1b215aa662c8f983f7d6dedafe65a9be72c26e5338e0fe98b3e0422c32c85428", size = 18670967, upload-time = "2026-06-15T22:43:10.621Z" },
{ url = "https://files.pythonhosted.org/packages/40/89/17546c1c20f6bfc3ae41c22152378a26edfea918af3129e2139dcd7c99f3/onnxruntime-1.26.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:33a791f31432a3af1a96db5e54818b37aba5e5eefc2e6af5794c10a9118a9993", size = 18019724, upload-time = "2026-05-08T19:07:30.723Z" }, { url = "https://files.pythonhosted.org/packages/3a/37/48ab79c39b58a7c9f6f5aac1fa0ff2b993eb2643393d6ed9e839ddb6f347/onnxruntime-1.27.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:0874edc171f470fc4dd2bbb60bc0989612ed1a8b89b365cda016630a93227f13", size = 18433941, upload-time = "2026-06-15T22:42:58.867Z" },
{ url = "https://files.pythonhosted.org/packages/bb/24/89457a35f6af29538a76647f2c18c3a28277e6c19234c847e7b4b7c19860/onnxruntime-1.26.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e90c00732c4553618103149d93f688e8c3063017938f8983e21a71d9f3b6d22e", size = 16054821, upload-time = "2026-05-08T19:07:22.348Z" }, { url = "https://files.pythonhosted.org/packages/6e/24/d535ca8a09dbf697f853377c8dc0820dbcaae5f334316b400b953afbcba8/onnxruntime-1.27.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5b51c014cf1a4fcd93c29a97eac8071fa27710dae05a4d0380bb60a66d60a62c", size = 16439970, upload-time = "2026-06-15T22:42:48.023Z" },
{ url = "https://files.pythonhosted.org/packages/12/f9/15b2e1815cf570d238e0135529f80d2dce64e8e8818a1489cae83823c5c6/onnxruntime-1.26.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:01498e80ba8988428d08c2d51b1338f89e3de2a93e6ffe555f79c68f26a5c06b", size = 18185815, upload-time = "2026-05-08T19:07:44.179Z" }, { url = "https://files.pythonhosted.org/packages/f9/b1/ea9ee80c0bdaa4efb13f29f8c236f3740f6655e8c092a2d119515a5a652c/onnxruntime-1.27.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:445fb702ea5241ba813a3ce2febe2e9408a64f6ad2eb610924322c536165f7cd", size = 18659240, upload-time = "2026-06-15T22:43:13.165Z" },
{ url = "https://files.pythonhosted.org/packages/d7/65/2e11055faf015e4b07f45b513fa49b391baf2e19d92d77d73ebee13c1004/onnxruntime-1.26.0-cp314-cp314-win_amd64.whl", hash = "sha256:7ead61450d8405167c87dd3a31d8da1d576b490a57dab1aa8b82a7da6825f5aa", size = 13349887, upload-time = "2026-05-08T19:08:08.671Z" }, { url = "https://files.pythonhosted.org/packages/e9/f2/1404507d76a21940e8bf46f414e3d1abd94dc888cb89a30f4a540275846f/onnxruntime-1.27.0-cp314-cp314-win_amd64.whl", hash = "sha256:49e416be0d717338b6d041b99911b716d70c397d277056450724f93bdded3fc2", size = 13685306, upload-time = "2026-06-15T22:43:40.416Z" },
{ url = "https://files.pythonhosted.org/packages/19/e4/0f9d1a5718b1781c610c1e354765a3820597081754277a6a9a2b50705702/onnxruntime-1.26.0-cp314-cp314-win_arm64.whl", hash = "sha256:31d71a53490e46910877d0902b5ad99c69a5955e5c7ea6c82863519410e1ba7c", size = 13140121, upload-time = "2026-05-08T19:07:57.804Z" }, { url = "https://files.pythonhosted.org/packages/10/e5/ca5cf012ccccb806c70e94aadfebca5606acc62b33eb88cec13352d0778f/onnxruntime-1.27.0-cp314-cp314-win_arm64.whl", hash = "sha256:856032937dd3bc7a7c141909c8d7ae4fde3e3f59bddf061ae627b9a051bda95c", size = 13456280, upload-time = "2026-06-15T22:43:29.693Z" },
{ url = "https://files.pythonhosted.org/packages/1c/42/3b8e635f067d06d9f45bede470b8d539d101a4166c272213158dfd08b6ce/onnxruntime-1.26.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d7b6d258fb78fdfcf049795bcfaa74dcb90ae7baa277afd21e6fd28b83f2c496", size = 16057240, upload-time = "2026-05-08T19:07:25.163Z" }, { url = "https://files.pythonhosted.org/packages/67/7b/dca330a8397e9d816c976d7aed4e24a4a2d279bb1e551e3d0221d1389b1d/onnxruntime-1.27.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c6197a02e3f620c4dc13cff51b80672409fc1ffab3aa2593911b19fd322ff48b", size = 16443274, upload-time = "2026-06-15T22:42:50.467Z" },
{ url = "https://files.pythonhosted.org/packages/93/99/f2be40a31b908d96b861ae0ce98582fa376c18a7f816b9d5eb4cd6aa0a4c/onnxruntime-1.26.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4eefd386a45202aefb7a5132b94f32df9d506c9edcc7faf2fc60d65183f4b183", size = 18197382, upload-time = "2026-05-08T19:07:46.965Z" }, { url = "https://files.pythonhosted.org/packages/b7/f6/2bac21f722aa45d876d4a51f26bd0ef30e704068a3cd5021a5a7cd784271/onnxruntime-1.27.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:370d211e1ceeac4cd5f45301655463ac59e27cdc74d9f7aeb2d19ff4b7a76715", size = 18670781, upload-time = "2026-06-15T22:43:17.151Z" },
] ]
[[package]] [[package]]
name = "onnxruntime-gpu" name = "onnxruntime-gpu"
version = "1.26.0" version = "1.27.0"
source = { registry = "https://pypi.org/simple" } source = { registry = "https://pypi.org/simple" }
dependencies = [ dependencies = [
{ name = "flatbuffers", marker = "platform_machine != 's390x'" }, { name = "flatbuffers", marker = "platform_machine != 's390x'" },
@@ -421,12 +454,12 @@ dependencies = [
{ name = "protobuf", marker = "platform_machine != 's390x'" }, { name = "protobuf", marker = "platform_machine != 's390x'" },
] ]
wheels = [ wheels = [
{ url = "https://files.pythonhosted.org/packages/dd/97/fe8979f44b9275654b42f7bb556e30789b71a1b22998c83b540df2b1b774/onnxruntime_gpu-1.26.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cfda2fad535595bfc3e570eb588092717711dcb2957656d814695e0c9ceb1508", size = 276974871, upload-time = "2026-05-08T19:15:58.052Z" }, { url = "https://files.pythonhosted.org/packages/8f/8d/6509743801131f1d3791f638ae31edef7701e93c7a4523d74115247dd7d1/onnxruntime_gpu-1.27.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7e6eb554d35706ee154b583dd32e4167a93aa55d3670f59754f43860fd649b92", size = 220304092, upload-time = "2026-06-18T18:27:31.18Z" },
{ url = "https://files.pythonhosted.org/packages/67/3f/59f1777a394625ecc9a85636de57dc47c25dbb5f888da050f1463955a0ce/onnxruntime_gpu-1.26.0-cp313-cp313-win_amd64.whl", hash = "sha256:6ab9f9c741d2e239b2e321ab0d389c04329d4ab7f11e3b92dd3aa7db1c59dee4", size = 226548083, upload-time = "2026-05-08T19:09:44.408Z" }, { url = "https://files.pythonhosted.org/packages/ac/86/8865182eba89cde187163daacfa7f1eb64958616cd080dfb3482be4c8a36/onnxruntime_gpu-1.27.0-cp313-cp313-win_amd64.whl", hash = "sha256:ac352199fb6ac3e366b45a690bb47c9b1c92b1be0e3958149dd1a30285832b9f", size = 213623636, upload-time = "2026-06-15T22:43:10.548Z" },
{ url = "https://files.pythonhosted.org/packages/89/96/360328e3c463f7ea08e853c4239c397e83363dd0204de71a710dc1a544bd/onnxruntime_gpu-1.26.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bcf6f347cad9f88a9a625c2b352cf9de927528aedb627ebbc089a201f1990b94", size = 276992052, upload-time = "2026-05-08T19:16:09.892Z" }, { url = "https://files.pythonhosted.org/packages/cf/e6/16f5ba0f91bd8db5b2881ef1ec57d532c8f297b31b6a95ceab606f471c0a/onnxruntime_gpu-1.27.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3b5652761f14bc74729ffbc8fdc01767e541e163459f2e10aa65af61d5db64c8", size = 220331411, upload-time = "2026-06-18T18:27:39.738Z" },
{ url = "https://files.pythonhosted.org/packages/fd/c8/aa2dc0e79bba577f37d5448bcb32fea79977e07506684d8138c19a0f1077/onnxruntime_gpu-1.26.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9e6e4fb1ec9ae1cf456534d9115f106ab2a1ae96fa513b4ed0f4795302b4a2c6", size = 276978254, upload-time = "2026-05-08T19:16:22.096Z" }, { url = "https://files.pythonhosted.org/packages/d7/97/5584057e355e5d8de3ae5cdd81e946ad2c9d8253626614eac805750645af/onnxruntime_gpu-1.27.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:209caede76604509fb7451178f56daa5292a933478864aaff87c6c29d1257593", size = 220303692, upload-time = "2026-06-18T18:27:48.183Z" },
{ url = "https://files.pythonhosted.org/packages/41/e7/923298431e669567d7ccc2a4c898b6534a47641a051569fd97165fe6d9b8/onnxruntime_gpu-1.26.0-cp314-cp314-win_amd64.whl", hash = "sha256:3e592439b0183d303c2374517b5b392599a3d50b2dc9de949b9b15731ac921c9", size = 229142768, upload-time = "2026-05-08T19:09:54.589Z" }, { url = "https://files.pythonhosted.org/packages/93/7f/6446c80d38ee2c8ec82365f44f1a842993f497da9ebf46a6b37c2ef4d2b7/onnxruntime_gpu-1.27.0-cp314-cp314-win_amd64.whl", hash = "sha256:986b94d513b052dc06b93c5a424f072f07f40d66be40ce2d6320d76d84fa4919", size = 214270425, upload-time = "2026-06-15T22:43:20.017Z" },
{ url = "https://files.pythonhosted.org/packages/97/91/93ffe5431d154989f5e04864a25a97eea480997d771232bcbbc538188241/onnxruntime_gpu-1.26.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:56dc7b73954ff4bdc71f5b8ab306b6f61be5d007881b6ef423a609e2b9cd088b", size = 276991545, upload-time = "2026-05-08T19:16:33.347Z" }, { url = "https://files.pythonhosted.org/packages/83/06/e1b690df1c70772d76464c5d8b2d11b436ade1cf93a061303cdf9d72b3bf/onnxruntime_gpu-1.27.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:42696d6137d6974325a67c88a65dd41625e49bf076da79c658514b498b297142", size = 220327139, upload-time = "2026-06-18T18:27:59.096Z" },
] ]
[[package]] [[package]]
@@ -608,7 +641,7 @@ wheels = [
[[package]] [[package]]
name = "pytest" name = "pytest"
version = "9.0.3" version = "9.1.1"
source = { registry = "https://pypi.org/simple" } source = { registry = "https://pypi.org/simple" }
dependencies = [ dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32' or (extra == 'extra-6-winnow-cpu' and extra == 'extra-6-winnow-gpu') or (extra == 'extra-6-winnow-cpu' and extra == 'extra-6-winnow-intel') or (extra == 'extra-6-winnow-cpu' and extra == 'extra-6-winnow-rocm') or (extra == 'extra-6-winnow-gpu' and extra == 'extra-6-winnow-intel') or (extra == 'extra-6-winnow-gpu' and extra == 'extra-6-winnow-rocm') or (extra == 'extra-6-winnow-intel' and extra == 'extra-6-winnow-rocm')" }, { name = "colorama", marker = "sys_platform == 'win32' or (extra == 'extra-6-winnow-cpu' and extra == 'extra-6-winnow-gpu') or (extra == 'extra-6-winnow-cpu' and extra == 'extra-6-winnow-intel') or (extra == 'extra-6-winnow-cpu' and extra == 'extra-6-winnow-rocm') or (extra == 'extra-6-winnow-gpu' and extra == 'extra-6-winnow-intel') or (extra == 'extra-6-winnow-gpu' and extra == 'extra-6-winnow-rocm') or (extra == 'extra-6-winnow-intel' and extra == 'extra-6-winnow-rocm')" },
@@ -617,9 +650,9 @@ dependencies = [
{ name = "pluggy" }, { name = "pluggy" },
{ name = "pygments" }, { name = "pygments" },
] ]
sdist = { url = "https://files.pythonhosted.org/packages/7d/0d/549bd94f1a0a402dc8cf64563a117c0f3765662e2e668477624baeec44d5/pytest-9.0.3.tar.gz", hash = "sha256:b86ada508af81d19edeb213c681b1d48246c1a91d304c6c81a427674c17eb91c", size = 1572165, upload-time = "2026-04-07T17:16:18.027Z" } sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" }
wheels = [ wheels = [
{ url = "https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl", hash = "sha256:2c5efc453d45394fdd706ade797c0a81091eccd1d6e4bccfcd476e2b8e0ab5d9", size = 375249, upload-time = "2026-04-07T17:16:16.13Z" }, { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },
] ]
[[package]] [[package]]
@@ -673,27 +706,27 @@ wheels = [
[[package]] [[package]]
name = "ruff" name = "ruff"
version = "0.15.17" version = "0.15.20"
source = { registry = "https://pypi.org/simple" } source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/8c/a9/3abdf488f1bf3d24c699415e454ed554a6350d5d89ce183be1ee0a3361ac/ruff-0.15.17.tar.gz", hash = "sha256:2ec446937fd16c8c4de2674a209cc5af64d9c6f17d21fbf1151054fa0bcf5219", size = 4743346, upload-time = "2026-06-11T17:54:47.663Z" } sdist = { url = "https://files.pythonhosted.org/packages/43/dc/35b341fc554ba02f217fc10da57d1a75168cfbcf75b0ef2202176d4c4f2d/ruff-0.15.20.tar.gz", hash = "sha256:1416eb04349192646b54de98f146c4f59afe37d0decfc02c3cbbf396f3a28566", size = 4755489, upload-time = "2026-06-25T17:20:37.578Z" }
wheels = [ wheels = [
{ url = "https://files.pythonhosted.org/packages/db/4d/e11259f5da07cb6afb2d074c31bf09da9671993f7329d4f15d2fdc458301/ruff-0.15.17-py3-none-linux_armv6l.whl", hash = "sha256:d9feddb927fc68bd295f5eebc587a7e42cfaf9b65f60ca4a2386febff575da8f", size = 10856677, upload-time = "2026-06-11T17:54:49.533Z" }, { url = "https://files.pythonhosted.org/packages/94/d9/2d5014f0253ba541d2061d9fa7193f48e941c8b21bb88a7ff9bbe0bd0596/ruff-0.15.20-py3-none-linux_armv6l.whl", hash = "sha256:00e188c53e499c3c1637f73c91dcf2fb56d576cab76ce1be50a27c4e80e37078", size = 10839665, upload-time = "2026-06-25T17:19:44.702Z" },
{ url = "https://files.pythonhosted.org/packages/29/3e/772d679e1a0dc058e58875bd2c0cb713a0530877b4a76fee3c7966df0d49/ruff-0.15.17-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:25805a226d741c47d274a35ad5c10a7dde175fcddfa511d7cf3da0a21eb3eab7", size = 11223443, upload-time = "2026-06-11T17:55:00.573Z" }, { url = "https://files.pythonhosted.org/packages/c6/d3/ac1798ba64f670698867fcfc591d50e7e421bef137db564858f619a30fcf/ruff-0.15.20-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:9ebd1fd9b9c95fc0bd7b2761aebec1f030013d2e193a2901b224af68fe47251b", size = 11208649, upload-time = "2026-06-25T17:19:48.787Z" },
{ url = "https://files.pythonhosted.org/packages/68/58/bd41f7688b2fd5623012605130ed70e60aa7f2244baa3d5066bdd61530c8/ruff-0.15.17-py3-none-macosx_11_0_arm64.whl", hash = "sha256:f6ad73b14c2d18a3bf8ad7cb6974294d7f613a7898604826058e6ac64918ef4d", size = 10566458, upload-time = "2026-06-11T17:55:07.52Z" }, { url = "https://files.pythonhosted.org/packages/47/47/d3ac899991202095dfcf3d5176be4272642be3cf981a2f1a30f72a2afb95/ruff-0.15.20-py3-none-macosx_11_0_arm64.whl", hash = "sha256:c5b16cdd67ca108185cd36dce98c576350c03b1660a751de725fb049193a0632", size = 10622638, upload-time = "2026-06-25T17:19:51.354Z" },
{ url = "https://files.pythonhosted.org/packages/d8/5b/733371013fcf1ec339e477ece6ab42bfe10bdd9bba8ee88a9516aa56bfc0/ruff-0.15.17-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6ba0c1e4f95bcb3869d0d30cbd5917071ef2e28665abfec970cdab0492c713ed", size = 10914483, upload-time = "2026-06-11T17:55:05.501Z" }, { url = "https://files.pythonhosted.org/packages/33/13/4e043fe30aa94d4ff5213a9881fc296d12960f5971b234a5263fdc225312/ruff-0.15.20-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3413bb3c3d2ca6a8208f1f4809cd2dca3c6de6d0b491c0e70847672bde6e6efd", size = 10984227, upload-time = "2026-06-25T17:19:54.044Z" },
{ url = "https://files.pythonhosted.org/packages/bd/cc/6f24251cc0252f7239391ccb85833f320efad14ebe5b443943f37ced6332/ruff-0.15.17-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:81647960f10bff57d2e51cadd0c3950fe598400c852863a038720ef5b8cca91e", size = 10647497, upload-time = "2026-06-11T17:54:57.733Z" }, { url = "https://files.pythonhosted.org/packages/76/e6/92e7bf40388bc5800073b96564f56264f7e48bfd1a498f5ced6ae6d5a769/ruff-0.15.20-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:bd7ec42b3bb3da066488db093308a69c4ac5ee6d2af333a86ba6e2eb2e7dd44b", size = 10622882, upload-time = "2026-06-25T17:19:57.037Z" },
{ url = "https://files.pythonhosted.org/packages/68/dd/0d10c17ce1a1624d6fc3156309c3f834fdb5dfaad026ec90c85684f3990e/ruff-0.15.17-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0e01a84ddbc8c16c23055ba3924476850f1bbc1917cebbb9376665a63e74260d", size = 11416967, upload-time = "2026-06-11T17:54:51.461Z" }, { url = "https://files.pythonhosted.org/packages/13/7a/43460be3f24495a3aa46d4b16873e2c4941b3b5f0b00cf88c03b7b94b339/ruff-0.15.20-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e1a36ad0eb77fba9aabfb69ede54de6f376d04ac18ebea022847046d340a8267", size = 11474808, upload-time = "2026-06-25T17:20:00.357Z" },
{ url = "https://files.pythonhosted.org/packages/2f/91/556bfb156f6144f355e831c23db00b2fc4120f86b3ce81cc5f7fd2df51f3/ruff-0.15.17-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:84fe9f653152f8f294f9f7e03bf3a453d8b4a27f7a59c78c8666167f2b17b96c", size = 12335770, upload-time = "2026-06-11T17:54:45.793Z" }, { url = "https://files.pythonhosted.org/packages/27/a0/f37077884873221c6b33b4ab49eb18f9f88e54a16a25a5bca59bef46dd66/ruff-0.15.20-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b6df3b1e4610432f0386dba04d853b5f08cbbc903410c6fcc02f620f05aff53c", size = 12293094, upload-time = "2026-06-25T17:20:03.446Z" },
{ url = "https://files.pythonhosted.org/packages/88/82/8b5999aa13355e926f06d9f42a32dcca862f623bf0363785ff89d607dffd/ruff-0.15.17-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:8c0fe88a7676e7a05b73174d4d4a59cb2ac21ff8263583f87a81a6018475a978", size = 11575441, upload-time = "2026-06-11T17:54:32.661Z" }, { url = "https://files.pythonhosted.org/packages/a6/74/165545b60256a9704c21ac0ec4a0d07933b320812f9584836c9f4aca4292/ruff-0.15.20-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e89f198a1ea6ef0d727c1cf16088bc91a6cb0ab947dedc966715691647186eae", size = 11526176, upload-time = "2026-06-25T17:20:06.301Z" },
{ url = "https://files.pythonhosted.org/packages/11/93/f10377bb04109ca0e8cbc483ff1982c54b6d418210041776f93e8cdc7fa9/ruff-0.15.17-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ecfc3c7878fff94633ab0348524e093f9ce3243080416dd7d14f8ba400174719", size = 11557614, upload-time = "2026-06-11T17:54:34.698Z" }, { url = "https://files.pythonhosted.org/packages/86/b1/a976a136d40ade83ce743578399865f57001003a409acadc0ecbb3051082/ruff-0.15.20-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:309809086c2acb67624950a3c8133e80f32d0d3e27106c0cd60ff26657c9f24b", size = 11520767, upload-time = "2026-06-25T17:20:09.191Z" },
{ url = "https://files.pythonhosted.org/packages/c7/a6/eeeae7f7d5493df41649ab3db92f086b2d0a30199e4efdf8e3dd7a033f24/ruff-0.15.17-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:b8461180b22420b1bdc289909410930761629fddf2a5aaf60fae1ab26cedc4c4", size = 11544450, upload-time = "2026-06-11T17:54:39.042Z" }, { url = "https://files.pythonhosted.org/packages/19/0f/f032696cb01c9b54c0263fa393474d7758f1cdc021a01b04e3cbc2500999/ruff-0.15.20-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:2d2374caa2f2c2f9e2b7da0a50802cfb8b79f55a9b5e49379f564544fbf56487", size = 11500132, upload-time = "2026-06-25T17:20:13.602Z" },
{ url = "https://files.pythonhosted.org/packages/32/88/5991ce565129a24dd4a00db1254b3b5db2e53018cbe4018ea5a89738e727/ruff-0.15.17-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:6eccbe50a038b503e7140b441aa9c7fc8c1f36edf23ebef9f4165c2f28f568b7", size = 10892524, upload-time = "2026-06-11T17:55:09.432Z" }, { url = "https://files.pythonhosted.org/packages/4b/f4/51b1a14bc69e8c224b15dab9cce8e99b425e0455d462caa2b3c9be2b6a8e/ruff-0.15.20-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:a1ed17b65293e0c2f22fc387bc13198a5de94bf4429589b0ff6946b0feaf21a3", size = 10943828, upload-time = "2026-06-25T17:20:16.635Z" },
{ url = "https://files.pythonhosted.org/packages/f5/1d/0fdd248313425f55223968af04b0a42125466a8d88d21c1d99c6af0a51e8/ruff-0.15.17-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:382fc0521025f5a8ad447d8bdd523545d0d7646adb718eb1c2dac5065ec27c0f", size = 10659573, upload-time = "2026-06-11T17:54:36.824Z" }, { url = "https://files.pythonhosted.org/packages/71/4b/fe267640783cd02bf6c5cc290b1df1051be2ec294c678b5c15fe19e52343/ruff-0.15.20-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:f701305e66b38ea6c91882490eb73459796808e4c6362a1b765255e0cdcd4053", size = 10645418, upload-time = "2026-06-25T17:20:19.4Z" },
{ url = "https://files.pythonhosted.org/packages/9e/0e/072e8260deb9461062ce9311ced27a8e541229a6ffd483013dd37661e43e/ruff-0.15.17-py3-none-musllinux_1_2_i686.whl", hash = "sha256:456d41fcd1b2777ad63f09a6e7121d43f7b688bbc76a800c10f7f8fb1f912c3f", size = 11127818, upload-time = "2026-06-11T17:55:03.124Z" }, { url = "https://files.pythonhosted.org/packages/b0/c0/a65aa4ec2f5e87a1df32dc3ec1fede434fe3dfd5cbcf3b503cafc676ab54/ruff-0.15.20-py3-none-musllinux_1_2_i686.whl", hash = "sha256:5b9c0c367ad8e5d0d5b5b8537864c469a0a0e55417aadfbeca41fa61333be9f4", size = 11211770, upload-time = "2026-06-25T17:20:22.033Z" },
{ url = "https://files.pythonhosted.org/packages/ab/b4/55060a34163121498014696b5f656db5b8c6963768f227dbf0d76b311073/ruff-0.15.17-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:b1a04bcc94ae6194e9db05d16ad31f298a7194bfbcb08258bbe589cee1d587b8", size = 11655901, upload-time = "2026-06-11T17:54:53.562Z" }, { url = "https://files.pythonhosted.org/packages/5a/a4/0caa331d954ae2723d729d351c989cb4ca8b6077d5c6c2cb6de75e98c041/ruff-0.15.20-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:01cc00dd58f0df339d0e902219dd53990ea99996a0344e5d9cc8d45d5307e460", size = 11618698, upload-time = "2026-06-25T17:20:25.259Z" },
{ url = "https://files.pythonhosted.org/packages/49/71/9b29d6b87cef468d697f43c6a91e3fae4a80185779d7d5a4ef27d173439f/ruff-0.15.17-py3-none-win32.whl", hash = "sha256:596065960ab1ff593f744220c9fe6580eda00a95003cffa9f4048bb5b1bf0392", size = 10925574, upload-time = "2026-06-11T17:54:55.723Z" }, { url = "https://files.pythonhosted.org/packages/10/9b/5f14927848d2fd4aa891fd88d883788c5a7baba561c7874732364045708c/ruff-0.15.20-py3-none-win32.whl", hash = "sha256:ed65ef510e43a137207e0f01cfcf998aeddb1aeeda5c9d35023e910284d7cf21", size = 10857322, upload-time = "2026-06-25T17:20:28.612Z" },
{ url = "https://files.pythonhosted.org/packages/3d/b2/8fc77f3723228836fa5d12497eb71c808f83782e10d058d2b15cfa14640b/ruff-0.15.17-py3-none-win_amd64.whl", hash = "sha256:6769e5fa1710b179b92e0bfa5a51735b35baea9013dadb06d5f44cbcf9547084", size = 12058788, upload-time = "2026-06-11T17:54:41.042Z" }, { url = "https://files.pythonhosted.org/packages/fa/f0/fe47c501f9dea92a26d788ff98bb5d92ed4cb4c88792c5c88af6b697dc8e/ruff-0.15.20-py3-none-win_amd64.whl", hash = "sha256:a525c81c70fb0380344dd1d8745d8cc1c890b7fc94a58d5a07bd8eb9557b8415", size = 11993274, upload-time = "2026-06-25T17:20:31.871Z" },
{ url = "https://files.pythonhosted.org/packages/2d/c7/c53e8dbff9c9dc4b7928773421ae294a5d28fcb8dcda1a089579d3a7e510/ruff-0.15.17-py3-none-win_arm64.whl", hash = "sha256:f3be1fbb34bcdfd146240d8fb92a709d4c2c8191348580a3c044ec60fa0b4456", size = 11355275, upload-time = "2026-06-11T17:54:43.635Z" }, { url = "https://files.pythonhosted.org/packages/d7/2b/9555445e1201d92b3195f45cdb153a0b68f24e0a4273f6e3d5ab46e212bb/ruff-0.15.20-py3-none-win_arm64.whl", hash = "sha256:2f5b2a6d614e8700388806a14996c40fab2c47b819ef57d790a34878858ed9ca", size = 11343498, upload-time = "2026-06-25T17:20:35.03Z" },
] ]
[[package]] [[package]]
@@ -862,7 +895,7 @@ wheels = [
[[package]] [[package]]
name = "winnow" name = "winnow"
version = "0.6.2" version = "0.6.6"
source = { editable = "." } source = { editable = "." }
dependencies = [ dependencies = [
{ name = "croniter" }, { name = "croniter" },
@@ -880,7 +913,10 @@ cpu = [
{ name = "onnxruntime" }, { name = "onnxruntime" },
] ]
gpu = [ gpu = [
{ name = "nvidia-cuda-runtime-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cudnn-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, { name = "nvidia-cudnn-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cufft-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-curand-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "onnxruntime-gpu", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, { name = "onnxruntime-gpu", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
] ]
intel = [ intel = [
@@ -900,10 +936,13 @@ dev = [
requires-dist = [ requires-dist = [
{ name = "croniter", specifier = ">=5.0.2" }, { name = "croniter", specifier = ">=5.0.2" },
{ name = "insightface", specifier = ">=0.7.3" }, { name = "insightface", specifier = ">=0.7.3" },
{ name = "numpy", specifier = ">=2.2.6" }, { name = "numpy", specifier = ">=2.5.0" },
{ name = "nvidia-cudnn-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=9.0.0" }, { name = "nvidia-cuda-runtime-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=12.0" },
{ name = "onnxruntime", marker = "extra == 'cpu'", specifier = ">=1.23.2" }, { name = "nvidia-cudnn-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=9.23.2.1" },
{ name = "onnxruntime-gpu", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=1.23.2" }, { name = "nvidia-cufft-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=11.0" },
{ name = "nvidia-curand-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=10.0" },
{ name = "onnxruntime", marker = "extra == 'cpu'", specifier = ">=1.27.0" },
{ name = "onnxruntime-gpu", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=1.27.0" },
{ name = "onnxruntime-openvino", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'intel'", specifier = ">=1.20.0" }, { name = "onnxruntime-openvino", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'intel'", specifier = ">=1.20.0" },
{ name = "onnxruntime-rocm", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'rocm'", specifier = ">=1.16.0" }, { name = "onnxruntime-rocm", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'rocm'", specifier = ">=1.16.0" },
{ name = "opencv-python-headless", specifier = ">=4.12.0.88" }, { name = "opencv-python-headless", specifier = ">=4.12.0.88" },
@@ -916,6 +955,6 @@ provides-extras = ["gpu", "rocm", "intel", "cpu"]
[package.metadata.requires-dev] [package.metadata.requires-dev]
dev = [ dev = [
{ name = "pytest", specifier = ">=8.0" }, { name = "pytest", specifier = ">=9.1.1" },
{ name = "ruff", specifier = ">=0.15.17" }, { name = "ruff", specifier = ">=0.15.20" },
] ]
+18 -12
View File
@@ -7,6 +7,7 @@ import sys
from rich import print as rprint from rich import print as rprint
from rich.prompt import Confirm from rich.prompt import Confirm
from . import __version__
from .config import Config, _getenv_bool from .config import Config, _getenv_bool
from .executor import execute_jobs, upload_to_frigate from .executor import execute_jobs, upload_to_frigate
from .immich_api import get_immich_version, get_people, merge_people from .immich_api import get_immich_version, get_people, merge_people
@@ -71,7 +72,7 @@ def _handle_duplicate_people(people: list[dict]) -> list[dict]:
by_name: dict[str, list[dict]] = defaultdict(list) by_name: dict[str, list[dict]] = defaultdict(list)
for p in people: for p in people:
name = (p.get("name") or "").strip() name = (p.get("name") or "").strip()
if name: if name and p.get("id"):
by_name[name].append(p) by_name[name].append(p)
duplicates = {name: ps for name, ps in by_name.items() if len(ps) > 1} duplicates = {name: ps for name, ps in by_name.items() if len(ps) > 1}
@@ -81,19 +82,23 @@ def _handle_duplicate_people(people: list[dict]) -> list[dict]:
def _smaller_duplicate_ids(groups: dict) -> set[str]: def _smaller_duplicate_ids(groups: dict) -> set[str]:
"""IDs of all but the largest person in each duplicate group.""" """IDs of all but the largest person in each duplicate group."""
return { return {
p.get("id") pid
for ps in groups.values() for ps in groups.values()
for p in sorted(ps, key=lambda x: x.get("assetCount", 0), reverse=True)[1:] for p in sorted(ps, key=lambda x: x.get("assetCount", 0), reverse=True)[1:]
if (pid := p.get("id"))
} }
skip_ids = _smaller_duplicate_ids(duplicates) skip_ids = _smaller_duplicate_ids(duplicates)
def _excl(lst: list[dict]) -> list[dict]:
return [p for p in lst if p.get("id") not in skip_ids]
if not Config.MERGE_DUPLICATE_PEOPLE: if not Config.MERGE_DUPLICATE_PEOPLE:
rprint("\n[bold yellow]⚠ Duplicate person names detected in Immich:[/bold yellow]") rprint("\n[bold yellow]⚠ Duplicate person names detected in Immich:[/bold yellow]")
for name, ps in sorted(duplicates.items()): for name, ps in sorted(duplicates.items()):
ordered = sorted(ps, key=lambda x: x.get("assetCount", 0), reverse=True) ordered = sorted(ps, key=lambda x: x.get("assetCount", 0), reverse=True)
entries = ", ".join( entries = ", ".join(
f"[dim]{p['id'][:8]}…[/dim] ({p.get('assetCount', 0)} assets)" f"[dim]{(p.get('id') or '?')[:8]}…[/dim] ({p.get('assetCount', 0)} assets)"
for p in ordered for p in ordered
) )
rprint(f" [yellow]{name}[/yellow] → {len(ps)} people: {entries}") rprint(f" [yellow]{name}[/yellow] → {len(ps)} people: {entries}")
@@ -109,20 +114,21 @@ def _handle_duplicate_people(people: list[dict]) -> list[dict]:
) )
# Return deduplicated list — keep only the largest per name so that # Return deduplicated list — keep only the largest per name so that
# downstream job creation never runs two jobs for the same Frigate folder. # downstream job creation never runs two jobs for the same Frigate folder.
return [p for p in people if p.get("id") not in skip_ids] return _excl(people)
# Auto-merge: survivor = largest asset count, rest merge into it inside Immich # Auto-merge: survivor = largest asset count, rest merge into it inside Immich
merged_any = False merged_any = False
for name, ps in sorted(duplicates.items()): for name, ps in sorted(duplicates.items()):
ordered = sorted(ps, key=lambda x: x.get("assetCount", 0), reverse=True) ordered = sorted(ps, key=lambda x: x.get("assetCount", 0), reverse=True)
survivor = ordered[0] survivor = ordered[0]
merge_ids = [p["id"] for p in ordered[1:]] survivor_id = survivor.get("id")
merge_ids = [pid for p in ordered[1:] if (pid := p.get("id")) is not None]
rprint( rprint(
f" [cyan]Merging {name!r} inside Immich:[/cyan] keeping " f" [cyan]Merging {name!r} inside Immich:[/cyan] keeping "
f"[dim]{survivor['id'][:8]}…[/dim] ({survivor.get('assetCount', 0)} assets), " f"[dim]{survivor_id[:8]}…[/dim] ({survivor.get('assetCount', 0)} assets), "
f"absorbing {len(merge_ids)} smaller duplicate(s)..." f"absorbing {len(merge_ids)} smaller duplicate(s)..."
) )
if merge_people(survivor["id"], merge_ids): if merge_people(survivor_id, merge_ids):
rprint(f" [green]✓ Merged {name!r}[/green]") rprint(f" [green]✓ Merged {name!r}[/green]")
merged_any = True merged_any = True
else: else:
@@ -141,12 +147,12 @@ def _handle_duplicate_people(people: list[dict]) -> list[dict]:
" — possible transient error or expired API key;" " — possible transient error or expired API key;"
" proceeding with pre-merge list. Check IMMICH_API_KEY if this recurs." " proceeding with pre-merge list. Check IMMICH_API_KEY if this recurs."
) )
return [p for p in people if p.get("id") not in skip_ids] return _excl(people)
# Filter out the smaller duplicate from any group whose merge failed — those # Filter out the smaller duplicate from any group whose merge failed — those
# IDs still exist in Immich and would produce two jobs for the same folder. # IDs still exist in Immich and would produce two jobs for the same folder.
# IDs from groups that merged successfully are already gone from Immich, so # IDs from groups that merged successfully are already gone from Immich, so
# this filter is a no-op for them. # this filter is a no-op for them.
return [p for p in fresh if p.get("id") not in skip_ids] return _excl(fresh)
# All merges failed — fall back to local deduplication (keep largest per name) so # All merges failed — fall back to local deduplication (keep largest per name) so
# downstream job creation never runs two jobs for the same Frigate folder. # downstream job creation never runs two jobs for the same Frigate folder.
@@ -154,7 +160,7 @@ def _handle_duplicate_people(people: list[dict]) -> list[dict]:
" [yellow]All merges failed — applying local deduplication" " [yellow]All merges failed — applying local deduplication"
" to avoid overwriting output.[/yellow]" " to avoid overwriting output.[/yellow]"
) )
return [p for p in people if p.get("id") not in skip_ids] return _excl(people)
_UNSUPPORTED_VARS = [ _UNSUPPORTED_VARS = [
@@ -179,8 +185,8 @@ def main() -> None:
if trace_size: if trace_size:
_handle_trace_crop(trace_size) _handle_trace_crop(trace_size)
console.print(r""" console.print(f"""
[bold blue]winnow[/bold blue] [bold blue]winnow[/bold blue] [dim]v{__version__}[/dim]
[dim]Immich -> Frigate Training Data Curator[/dim] [dim]Immich -> Frigate Training Data Curator[/dim]
""") """)
+4 -1
View File
@@ -127,12 +127,15 @@ class _Config:
self.API_KEY = os.getenv("API_KEY") self.API_KEY = os.getenv("API_KEY")
self.OUTPUT_DIR = os.getenv("OUTPUT_DIR", "./frigate_train") self.OUTPUT_DIR = os.getenv("OUTPUT_DIR", "./frigate_train")
self.YEARS_FILTER = _getenv_int("YEARS_FILTER", 10) self.YEARS_FILTER = _getenv_int("YEARS_FILTER", 10)
if self.YEARS_FILTER < 0:
logging.warning("YEARS_FILTER=%s is negative — using default 10", self.YEARS_FILTER)
self.YEARS_FILTER = 10
self.MIN_FACE_WIDTH = _getenv_int("MIN_FACE_WIDTH", 90) self.MIN_FACE_WIDTH = _getenv_int("MIN_FACE_WIDTH", 90)
self.MIN_FACE_COUNT = _getenv_int("MIN_FACE_COUNT", 3) self.MIN_FACE_COUNT = _getenv_int("MIN_FACE_COUNT", 3)
self.MERGE_DUPLICATE_PEOPLE = _getenv_bool("MERGE_DUPLICATE_PEOPLE", False) self.MERGE_DUPLICATE_PEOPLE = _getenv_bool("MERGE_DUPLICATE_PEOPLE", False)
self.BLUR_THRESHOLD = _getenv_float("BLUR_THRESHOLD", 120.0) self.BLUR_THRESHOLD = _getenv_float("BLUR_THRESHOLD", 120.0)
self.MIN_CONFIDENCE = _getenv_float("MIN_CONFIDENCE", 0.7) self.MIN_CONFIDENCE = _getenv_float("MIN_CONFIDENCE", 0.7)
self.MAX_AUTO_IMAGES = _getenv_int("MAX_AUTO_IMAGES", 20) self.MAX_AUTO_IMAGES = _getenv_int("MAX_AUTO_IMAGES", 5)
self.QUALITY_REPLACEMENT = _getenv_bool("QUALITY_REPLACEMENT", True) self.QUALITY_REPLACEMENT = _getenv_bool("QUALITY_REPLACEMENT", True)
self.FRIGATE_SCORE_CEILING = _getenv_optional_float("FRIGATE_SCORE_CEILING") self.FRIGATE_SCORE_CEILING = _getenv_optional_float("FRIGATE_SCORE_CEILING")
self.ENABLE_FRIGATE_SCORES = _getenv_bool("ENABLE_FRIGATE_SCORES", True) self.ENABLE_FRIGATE_SCORES = _getenv_bool("ENABLE_FRIGATE_SCORES", True)
+3
View File
@@ -303,6 +303,9 @@ def _select_by_embedding(
emb = get_embedding(embed_img, asset_id=asset["id"]) emb = get_embedding(embed_img, asset_id=asset["id"])
if emb is not None: if emb is not None:
if np.linalg.norm(emb) < 1e-6:
logger.debug("Zero-norm embedding for asset %s, skipping", asset["id"])
continue
embeddings.append(emb) embeddings.append(emb)
valid_candidates.append(asset) valid_candidates.append(asset)
confidence_scores.append(confidence) confidence_scores.append(confidence)
+12 -4
View File
@@ -107,7 +107,6 @@ def get_insightface_app():
global _insightface_app, _insightface_loaded global _insightface_app, _insightface_loaded
if _insightface_loaded: if _insightface_loaded:
return _insightface_app return _insightface_app
_insightface_loaded = True
ctx_id = -1 ctx_id = -1
insightface_home = os.environ.get("INSIGHTFACE_HOME", os.path.expanduser("~/.insightface")) insightface_home = os.environ.get("INSIGHTFACE_HOME", os.path.expanduser("~/.insightface"))
@@ -172,10 +171,12 @@ def get_insightface_app():
_insightface_app.prepare(ctx_id=ctx_id, det_size=(640, 640)) _insightface_app.prepare(ctx_id=ctx_id, det_size=(640, 640))
logger.info("InsightFace Buffalo_L: ready on %s (%.1fs)", device_str, time.time() - t0) logger.info("InsightFace Buffalo_L: ready on %s (%.1fs)", device_str, time.time() - t0)
_insightface_loaded = True
return _insightface_app return _insightface_app
except ImportError: except ImportError:
logger.error("InsightFace not installed!") logger.error("InsightFace not installed!")
_insightface_loaded = True
return None return None
except Exception as e: except Exception as e:
logger.error("Failed to load InsightFace: %s", e) logger.error("Failed to load InsightFace: %s", e)
@@ -193,9 +194,11 @@ def get_insightface_app():
) )
_insightface_app.prepare(ctx_id=-1, det_size=(640, 640)) _insightface_app.prepare(ctx_id=-1, det_size=(640, 640))
logger.info("InsightFace Buffalo_L: ready on CPU (fallback, %.1fs)", time.time() - t0) logger.info("InsightFace Buffalo_L: ready on CPU (fallback, %.1fs)", time.time() - t0)
_insightface_loaded = True
return _insightface_app return _insightface_app
except Exception as ex: except Exception as ex:
logger.error("InsightFace CPU fallback failed: %s", ex) logger.error("InsightFace CPU fallback failed: %s", ex)
_insightface_loaded = True
return None return None
@@ -218,9 +221,14 @@ def get_face_embedding(img_pil: Image.Image) -> np.ndarray | None:
if not faces: if not faces:
return None return None
# Return embedding of largest face # Return embedding of the face nearest the crop centre; a large margin can pull
largest = max(faces, key=lambda f: (f.bbox[2] - f.bbox[0]) * (f.bbox[3] - f.bbox[1])) # a bigger neighbouring face into frame, and max-by-area would pick the wrong person.
return largest.embedding cx, cy = img_pil.width / 2, img_pil.height / 2
nearest = min(
faces,
key=lambda f: ((f.bbox[0] + f.bbox[2]) / 2 - cx) ** 2 + ((f.bbox[1] + f.bbox[3]) / 2 - cy) ** 2,
)
return nearest.embedding
except Exception as e: except Exception as e:
logger.error("Error getting face embedding: %s", e) logger.error("Error getting face embedding: %s", e)
return None return None
+18 -8
View File
@@ -186,11 +186,10 @@ def execute_jobs(jobs: list[dict]) -> None:
saved = process_face_mode( saved = process_face_mode(
img, asset, person, person_dir, count, insightface_app=insightface_app img, asset, person, person_dir, count, insightface_app=insightface_app
) )
if saved: if isinstance(saved, tuple):
filename = f"{count}.jpg" filename = f"{count}.jpg"
asset_map[filename] = asset["id"] asset_map[filename] = asset["id"]
score_map[filename] = asset.get("quality_score") score_map[filename] = asset.get("quality_score")
if isinstance(saved, tuple):
dims_map[filename] = saved dims_map[filename] = saved
# Time-spread path: compute blur score from the downloaded # Time-spread path: compute blur score from the downloaded
# image. Capped at 1440px via blur_score_from_image() so the # image. Capped at 1440px via blur_score_from_image() so the
@@ -202,8 +201,9 @@ def execute_jobs(jobs: list[dict]) -> None:
count += 1 count += 1
else: else:
reason = saved if isinstance(saved, str) else "no usable face data"
progress.console.print( progress.console.print(
f"[yellow]Skipped {asset['id']} (no usable face data)[/yellow]" f"[yellow]Skipped {asset['id']} ({reason})[/yellow]"
) )
except Exception as e: except Exception as e:
logger.error("Failed to process asset %s: %s", asset.get("id", "<unknown>"), e) logger.error("Failed to process asset %s: %s", asset.get("id", "<unknown>"), e)
@@ -384,9 +384,9 @@ def upload_to_frigate(jobs: list[dict]) -> None:
# freed slot isn't filled with something worse than what we removed. # freed slot isn't filled with something worse than what we removed.
if min_quality_score_for_slot is not None: if min_quality_score_for_slot is not None:
file_score = score_map.get(fname) file_score = score_map.get(fname)
if file_score is not None and file_score <= min_quality_score_for_slot: if file_score is not None and file_score < min_quality_score_for_slot:
progress.console.print( progress.console.print(
f" [dim]⏭ {fname}: score {file_score:.3f} ≤ freed slot floor" f" [dim]⏭ {fname}: score {file_score:.3f} < freed slot floor"
f" {min_quality_score_for_slot:.3f}, skipping[/dim]" f" {min_quality_score_for_slot:.3f}, skipping[/dim]"
) )
progress.advance(upload_task) progress.advance(upload_task)
@@ -514,8 +514,7 @@ def upload_to_frigate(jobs: list[dict]) -> None:
uploaded += 1 uploaded += 1
person_uploaded += 1 person_uploaded += 1
effective_count += 1 effective_count += 1
min_quality_score_for_slot = None min_quality_score_for_slot = None # for/else rollback mirrors this pair
asset_id = asset_map.get(fname) asset_id = asset_map.get(fname)
if asset_id: if asset_id:
try: try:
@@ -567,13 +566,14 @@ def upload_to_frigate(jobs: list[dict]) -> None:
error_detail = resp.json().get("message", full_body[:100]) error_detail = resp.json().get("message", full_body[:100])
except Exception: except Exception:
error_detail = full_body[:100] error_detail = full_body[:100]
if resp.status_code == 400: if resp.status_code in (400, 500):
progress.console.print(f" [dim]{error_detail}[/dim]") progress.console.print(f" [dim]{error_detail}[/dim]")
else: else:
logger.debug("%s HTTP %s: %s", fname, resp.status_code, error_detail) logger.debug("%s HTTP %s: %s", fname, resp.status_code, error_detail)
_is_permanent = ( _is_permanent = (
(resp.status_code == 400 and "face" in full_body.lower()) (resp.status_code == 400 and "face" in full_body.lower())
or resp.status_code == 422 or resp.status_code == 422
or (resp.status_code == 500 and "could not process" in full_body.lower())
) )
if _is_permanent: if _is_permanent:
asset_id = asset_map.get(fname) asset_id = asset_map.get(fname)
@@ -608,6 +608,16 @@ def upload_to_frigate(jobs: list[dict]) -> None:
progress.console.print( progress.console.print(
f" [red]✗ {fname}: {type(e).__name__} - {e} (after {max_retries} attempts)[/red]" f" [red]✗ {fname}: {type(e).__name__} - {e} (after {max_retries} attempts)[/red]"
) )
else:
# All retries exhausted without a successful upload.
# Restore the slot freed by the preceding delete so the next
# candidate still sees at_cap=True and must beat the replacement gate.
# Also clear the quality floor — the deleted file's score no longer
# represents any live Frigate file, and leaving it blocks the next
# candidate from filling the restored slot.
if at_cap:
effective_count += 1
min_quality_score_for_slot = None
progress.advance(upload_task) progress.advance(upload_task)
+9 -3
View File
@@ -32,14 +32,18 @@ def get_frigate_version() -> str | None:
def _get_faces_data() -> dict | None: def _get_faces_data() -> dict | None:
"""Fetch raw GET /api/faces response. Returns None if unavailable.""" """Fetch raw GET /api/faces response. Returns None if unavailable or malformed."""
frigate_url = _get_frigate_url() frigate_url = _get_frigate_url()
if not frigate_url: if not frigate_url:
return None return None
try: try:
resp = requests.get(f"{frigate_url}/api/faces", timeout=10) resp = requests.get(f"{frigate_url}/api/faces", timeout=10)
resp.raise_for_status() resp.raise_for_status()
return resp.json() data = resp.json()
if not isinstance(data, dict):
logger.warning("Unexpected response shape from Frigate /api/faces: %s", type(data).__name__)
return None
return data
except Exception as e: except Exception as e:
logger.warning("Could not query Frigate faces API: %s", e) logger.warning("Could not query Frigate faces API: %s", e)
return None return None
@@ -146,8 +150,10 @@ def delete_frigate_person_files(person_name: str, filenames: list[str]) -> bool:
Returns True on success, False if unreachable or the request fails. Returns True on success, False if unreachable or the request fails.
""" """
frigate_url = _get_frigate_url() frigate_url = _get_frigate_url()
if not frigate_url or not filenames: if not frigate_url:
return False return False
if not filenames:
return True
from urllib.parse import quote from urllib.parse import quote
encoded_name = quote(person_name, safe="") encoded_name = quote(person_name, safe="")
try: try:
+7 -4
View File
@@ -48,6 +48,8 @@ def align_face(img: Image.Image, landmarks: list[list[float]] | np.ndarray) -> I
if lm.shape != (5, 2): if lm.shape != (5, 2):
logger.debug("Invalid landmark shape: %s, expected (5, 2)", lm.shape) logger.debug("Invalid landmark shape: %s, expected (5, 2)", lm.shape)
return None return None
with warnings.catch_warnings():
warnings.filterwarnings("ignore", message=".*estimate.*is deprecated", category=FutureWarning)
aligned = norm_crop(img_np, lm) aligned = norm_crop(img_np, lm)
return Image.fromarray(aligned) return Image.fromarray(aligned)
except ImportError: except ImportError:
@@ -66,10 +68,11 @@ def process_face_mode(
count: int, count: int,
min_width: int | None = None, min_width: int | None = None,
insightface_app=None, insightface_app=None,
) -> tuple[int, int] | None: ) -> tuple[int, int] | str:
"""Crop face based on Immich metadata and save to output directory. """Crop face based on Immich metadata and save to output directory.
Returns (width, height) of the saved crop, or None if no crop was saved. Returns (width, height) of the saved crop, or a skip-reason string if the
face was filtered out.
When insightface_app is provided and ENABLE_FACE_ALIGNMENT is True, When insightface_app is provided and ENABLE_FACE_ALIGNMENT is True,
re-detects the face in the Immich bbox region using InsightFace to get re-detects the face in the Immich bbox region using InsightFace to get
precise landmarks for a proper 112x112 aligned crop. Falls back to precise landmarks for a proper 112x112 aligned crop. Falls back to
@@ -89,7 +92,7 @@ def process_face_mode(
if not face_info: if not face_info:
logger.debug("No face info for %s in asset %s", person.get("name"), asset.get("id")) logger.debug("No face info for %s in asset %s", person.get("name"), asset.get("id"))
return None return "no face metadata"
img_w, img_h = img.size img_w, img_h = img.size
meta_w = face_info.get("imageWidth") or 0 meta_w = face_info.get("imageWidth") or 0
@@ -108,7 +111,7 @@ def process_face_mode(
face_w, face_h = x2 - x1, y2 - y1 face_w, face_h = x2 - x1, y2 - y1
if face_w < min_width or face_h < min_width: if face_w < min_width or face_h < min_width:
logger.debug("Face too small (%.1fx%.1f)", face_w, face_h) logger.debug("Face too small (%.1fx%.1f)", face_w, face_h)
return None return f"face too small ({face_w:.0f}x{face_h:.0f}px, min {min_width}px)"
# Re-detect face with InsightFace for landmark-based alignment. # Re-detect face with InsightFace for landmark-based alignment.
# Immich's /api/faces endpoint does not include landmarks, so the # Immich's /api/faces endpoint does not include landmarks, so the
+7 -3
View File
@@ -39,7 +39,11 @@ def get_immich_version() -> tuple[int, int, int] | None:
) )
if resp.ok: if resp.ok:
data = resp.json() data = resp.json()
return (int(data["major"]), int(data["minor"]), int(data["patch"])) major, minor, patch = data.get("major"), data.get("minor"), data.get("patch")
if major is None or minor is None or patch is None:
logger.debug("Unexpected Immich version schema: %s", data)
return None
return (int(major), int(minor), int(patch))
return None return None
except Exception: except Exception:
return None return None
@@ -130,7 +134,7 @@ def fetch_all_assets(person: dict) -> tuple[list[dict], int]:
# Immich ≥2.x returns {"assets": {"items": [...]}}; # Immich ≥2.x returns {"assets": {"items": [...]}};
# earlier versions returned {"assets": [...]} directly. # earlier versions returned {"assets": [...]} directly.
if isinstance(page_assets, dict): if isinstance(page_assets, dict):
page_assets = page_assets.get("items", []) page_assets = page_assets.get("items") or []
page_count = len(page_assets) # raw count for termination check before filtering page_count = len(page_assets) # raw count for termination check before filtering
@@ -299,7 +303,7 @@ def filter_recent_assets(assets: list[dict], years: int | None = None) -> list[d
recent.append(asset) recent.append(asset)
else: else:
skipped += 1 skipped += 1
except ValueError: except (ValueError, TypeError):
bad_timestamp += 1 bad_timestamp += 1
continue continue
+20 -9
View File
@@ -86,7 +86,11 @@ def _resolve_strategy(strategy: str, has_embedding: bool) -> tuple[int | str, st
"standard": (30, "smart"), "standard": (30, "smart"),
"broad": (100, "smart"), "broad": (100, "smart"),
} }
return strategy_map.get(strategy, ("auto", "smart")) result = strategy_map.get(strategy)
if result is None:
logger.warning("Unrecognised STRATEGY=%r — falling back to auto", strategy)
return ("auto", "smart")
return result
def _perform_selection( def _perform_selection(
@@ -192,13 +196,20 @@ def _configure_person(person: dict, people: list[dict]) -> dict | None:
return job return job
def _valid_people(people: list[dict]) -> list[dict]:
return sorted(
[p for p in people if (p.get("name") or "").strip() and p.get("id")],
key=lambda x: x["name"],
)
def interactive_configure(people: list[dict]) -> list[dict]: def interactive_configure(people: list[dict]) -> list[dict]:
"""Interactive phase: select person(s), mode, and configure training strategy. """Interactive phase: select person(s), mode, and configure training strategy.
Supports multi-person batch mode — after configuring one person, Supports multi-person batch mode — after configuring one person,
prompts to add another. prompts to add another.
""" """
valid_people = sorted([p for p in people if p.get("name")], key=lambda x: x["name"]) valid_people = _valid_people(people)
if not valid_people: if not valid_people:
rprint("[red]No people found with names in Immich.[/red]") rprint("[red]No people found with names in Immich.[/red]")
@@ -209,9 +220,9 @@ def interactive_configure(people: list[dict]) -> list[dict]:
while True: while True:
# Select person # Select person
console.print("\n[bold cyan]Select Person to Train:[/bold cyan]") console.print("\n[bold cyan]Select Person to Train:[/bold cyan]")
queued_ids = {j["person"]["id"] for j in jobs}
for idx, p in enumerate(valid_people, 1): for idx, p in enumerate(valid_people, 1):
# Mark already-queued people marker = " [dim](queued)[/dim]" if p.get("id") in queued_ids else ""
marker = " [dim](queued)[/dim]" if any(j["person"]["id"] == p["id"] for j in jobs) else ""
console.print(f" [bold]{idx}.[/bold] {p['name']}{marker}") console.print(f" [bold]{idx}.[/bold] {p['name']}{marker}")
p_choice = IntPrompt.ask("Enter Number", choices=[str(i) for i in range(1, len(valid_people) + 1)]) p_choice = IntPrompt.ask("Enter Number", choices=[str(i) for i in range(1, len(valid_people) + 1)])
@@ -230,20 +241,20 @@ def interactive_configure(people: list[dict]) -> list[dict]:
def auto_configure(people: list[dict]) -> list[dict]: def auto_configure(people: list[dict]) -> list[dict]:
"""Non-interactive: configure jobs for all named people automatically.""" """Non-interactive: configure jobs for all named people automatically."""
valid_people = sorted([p for p in people if p.get("name")], key=lambda x: x["name"]) valid_people = _valid_people(people)
if not valid_people: if not valid_people:
rprint("[red]No people found with names in Immich.[/red]") rprint("[red]No people found with names in Immich.[/red]")
return [] return []
strategy = os.environ.get("STRATEGY", "auto") strategy = os.environ.get("STRATEGY", "auto")
skip = [s.strip() for s in os.environ.get("SKIP_PEOPLE", "").split(",") if s.strip()] skip = {s.strip().casefold() for s in os.environ.get("SKIP_PEOPLE", "").split(",") if s.strip()}
only = [s.strip() for s in os.environ.get("ONLY_PEOPLE", "").split(",") if s.strip()] only = {s.strip().casefold() for s in os.environ.get("ONLY_PEOPLE", "").split(",") if s.strip()}
if only: if only:
valid_people = [p for p in valid_people if p["name"] in only] valid_people = [p for p in valid_people if p["name"].casefold() in only]
if skip: if skip:
valid_people = [p for p in valid_people if p["name"] not in skip] valid_people = [p for p in valid_people if p["name"].casefold() not in skip]
min_face_count = Config.MIN_FACE_COUNT min_face_count = Config.MIN_FACE_COUNT
+100 -4
View File
@@ -27,9 +27,11 @@ frigate_files only contains files winnow uploaded — files added manually throu
Frigate's UI are never mapped here and are never touched by quality replacement. Frigate's UI are never mapped here and are never touched by quality replacement.
""" """
import fcntl
import json import json
import logging import logging
import os import os
from contextlib import contextmanager
from pathlib import Path from pathlib import Path
from .frigate_api import _get_frigate_url, delete_frigate_person_files from .frigate_api import _get_frigate_url, delete_frigate_person_files
@@ -38,6 +40,14 @@ logger = logging.getLogger(__name__)
UPLOAD_TRACKER_FILE = "frigate_uploaded_ids.json" UPLOAD_TRACKER_FILE = "frigate_uploaded_ids.json"
REJECT_TRACKER_FILE = "frigate_rejected_ids.json" REJECT_TRACKER_FILE = "frigate_rejected_ids.json"
LOCK_FILE = ".tracker.lock"
# Number of deferred _save calls a batch accumulates before it is flushed to disk
# early. Bounds how many marks a crash mid-batch (SIGKILL/OOM/host crash) can lose —
# without this, begin_batch()/flush_batch() defer every write for an entire
# per-person upload loop, so a crash could lose every mark from that person's batch
# even though the files are already live in Frigate.
_BATCH_FLUSH_EVERY = 10
# Write-through in-memory cache keyed by the resolved file path. # Write-through in-memory cache keyed by the resolved file path.
# Reduces per-call JSON reads from O(calls) to O(1) after the first load. # Reduces per-call JSON reads from O(calls) to O(1) after the first load.
@@ -45,6 +55,16 @@ REJECT_TRACKER_FILE = "frigate_rejected_ids.json"
_cache: dict[str, dict] = {} _cache: dict[str, dict] = {}
_deferred: set[str] = set() # paths whose disk writes are batched until flush_batch() _deferred: set[str] = set() # paths whose disk writes are batched until flush_batch()
_dirty: set[str] = set() # deferred paths that received at least one _save during the batch _dirty: set[str] = set() # deferred paths that received at least one _save during the batch
_batch_writes: dict[str, int] = {} # deferred _save calls since the last disk write, per path
# Cross-process lock guarding the load-mutate-save cycle below. Two winnow
# invocations against the same DATA_DIR (e.g. a scheduled run overlapping a
# manual `docker exec`, which the docs explicitly instruct) can otherwise race
# a read-modify-write and silently lose whichever one saves first. Reentrant
# within a single process (depth-counted) so begin_batch()/flush_batch() pairs
# and nested tracker calls made while a batch is open don't self-deadlock.
_lock_fd: int | None = None
_lock_depth = 0
def _tracker_path(filename: str) -> Path: def _tracker_path(filename: str) -> Path:
@@ -55,6 +75,49 @@ def _tracker_path(filename: str) -> Path:
return Path(filename) return Path(filename)
def _lock_path() -> Path:
return _tracker_path(LOCK_FILE)
def _acquire_lock() -> None:
"""Acquire the cross-process tracker lock. Reentrant within this process."""
global _lock_fd, _lock_depth
if _lock_depth == 0:
path = _lock_path()
path.parent.mkdir(parents=True, exist_ok=True)
fd = os.open(path, os.O_CREAT | os.O_RDWR)
fcntl.flock(fd, fcntl.LOCK_EX) # blocks until any other process's lock is released
_lock_fd = fd
# Cache entries not part of an in-progress deferred batch may be stale —
# another process could have written to disk since we last read them.
# Drop them so the critical section that follows re-reads from disk.
for key in list(_cache):
if key not in _deferred:
del _cache[key]
_lock_depth += 1
def _release_lock() -> None:
global _lock_fd, _lock_depth
if _lock_depth <= 0:
return
_lock_depth -= 1
if _lock_depth == 0 and _lock_fd is not None:
fcntl.flock(_lock_fd, fcntl.LOCK_UN)
os.close(_lock_fd)
_lock_fd = None
@contextmanager
def _locked():
"""Context manager wrapping a single load-mutate-save cycle in the tracker lock."""
_acquire_lock()
try:
yield
finally:
_release_lock()
def _load(filename: str) -> dict: def _load(filename: str) -> dict:
path = _tracker_path(filename) path = _tracker_path(filename)
key = str(path) key = str(path)
@@ -89,6 +152,15 @@ def _save(filename: str, data: dict) -> None:
if key in _deferred: if key in _deferred:
_cache[key] = data # accumulate in cache; disk write deferred until flush_batch() _cache[key] = data # accumulate in cache; disk write deferred until flush_batch()
_dirty.add(key) _dirty.add(key)
# Flush early every _BATCH_FLUSH_EVERY marks so a crash mid-batch (SIGKILL/OOM/
# host crash) loses at most a bounded number of marks instead of the whole batch.
# Safe without re-acquiring the lock: begin_batch() already holds it for the
# duration of the batch.
_batch_writes[key] = _batch_writes.get(key, 0) + 1
if _batch_writes[key] >= _BATCH_FLUSH_EVERY:
_write_to_disk(path, data)
_dirty.discard(key)
_batch_writes[key] = 0
return return
_write_to_disk(path, data) _write_to_disk(path, data)
_cache[key] = data # update cache only after successful write _cache[key] = data # update cache only after successful write
@@ -96,13 +168,19 @@ def _save(filename: str, data: dict) -> None:
def begin_batch(filename: str) -> None: def begin_batch(filename: str) -> None:
"""Defer tracker disk writes for filename. All _save calls accumulate in the """Defer tracker disk writes for filename. All _save calls accumulate in the
in-memory cache until flush_batch() is called. Use around per-person upload loops in-memory cache until flush_batch() is called (with periodic early flushes —
to reduce N writes to 1. see _BATCH_FLUSH_EVERY). Use around per-person upload loops to reduce N writes
towards 1.
Acquires the cross-process tracker lock, held until the matching flush_batch()
releases it, so a concurrent winnow invocation against the same DATA_DIR can't
interleave a read-modify-write with this batch.
If a previous batch for this file was interrupted before flush_batch() was called If a previous batch for this file was interrupted before flush_batch() was called
(e.g. an exception escaped the upload loop), the leftover cache state is flushed (e.g. an exception escaped the upload loop), the leftover cache state is flushed
to disk here before starting fresh so that partial progress is not silently lost. to disk here before starting fresh so that partial progress is not silently lost.
""" """
_acquire_lock()
path = _tracker_path(filename) path = _tracker_path(filename)
key = str(path) key = str(path)
if key in _deferred and key in _dirty: if key in _deferred and key in _dirty:
@@ -117,16 +195,26 @@ def begin_batch(filename: str) -> None:
_deferred.discard(key) _deferred.discard(key)
_dirty.discard(key) _dirty.discard(key)
_deferred.add(key) _deferred.add(key)
_batch_writes[key] = 0
def flush_batch(filename: str) -> None: def flush_batch(filename: str) -> None:
"""Write the accumulated cache state for filename to disk.""" """Write the accumulated cache state for filename to disk and release the tracker lock
acquired by the matching begin_batch().
The lock release always runs, even if the disk write raises, so a write failure
(e.g. a full disk) can't leave the tracker lock held for the rest of the process.
"""
path = _tracker_path(filename) path = _tracker_path(filename)
key = str(path) key = str(path)
try:
if key in _dirty and key in _cache: if key in _dirty and key in _cache:
_write_to_disk(path, _cache[key]) _write_to_disk(path, _cache[key])
finally:
_deferred.discard(key) _deferred.discard(key)
_dirty.discard(key) _dirty.discard(key)
_batch_writes.pop(key, None)
_release_lock()
def _flat_key(filename: str) -> str: def _flat_key(filename: str) -> str:
@@ -165,6 +253,7 @@ def _mark(
if not person_name: if not person_name:
logger.warning("_mark called with empty person_name for asset %s — asset not recorded", asset_id) logger.warning("_mark called with empty person_name for asset %s — asset not recorded", asset_id)
return return
with _locked():
data = _load(filename) data = _load(filename)
by_person = dict(data.get("by_person", {})) by_person = dict(data.get("by_person", {}))
entry = _migrate_entry(by_person.get(person_name, {})) entry = _migrate_entry(by_person.get(person_name, {}))
@@ -229,6 +318,7 @@ def record_frigate_files_batch(person_name: str, mappings: dict[str, str]) -> No
"""Record multiple Frigate filename → asset_id mappings in a single load/save.""" """Record multiple Frigate filename → asset_id mappings in a single load/save."""
if not mappings: if not mappings:
return return
with _locked():
src = _load(UPLOAD_TRACKER_FILE) src = _load(UPLOAD_TRACKER_FILE)
by_person = dict(src.get("by_person", {})) by_person = dict(src.get("by_person", {}))
entry = _migrate_entry(by_person.get(person_name, {})) entry = _migrate_entry(by_person.get(person_name, {}))
@@ -251,6 +341,7 @@ def remove_frigate_file(person_name: str, frigate_filename: str) -> None:
def remove_frigate_files_batch(person_name: str, frigate_filenames: list[str]) -> None: def remove_frigate_files_batch(person_name: str, frigate_filenames: list[str]) -> None:
"""Remove multiple Frigate filenames in a single load/save.""" """Remove multiple Frigate filenames in a single load/save."""
with _locked():
src = _load(UPLOAD_TRACKER_FILE) src = _load(UPLOAD_TRACKER_FILE)
raw = src.get("by_person", {}).get(person_name) raw = src.get("by_person", {}).get(person_name)
if raw is None: if raw is None:
@@ -378,6 +469,7 @@ def find_by_crop_dimension(size: int) -> list[dict]:
def update_frigate_count(person_name: str, count: int) -> None: def update_frigate_count(person_name: str, count: int) -> None:
"""Record Frigate's authoritative training image count for a person.""" """Record Frigate's authoritative training image count for a person."""
with _locked():
data = _load(UPLOAD_TRACKER_FILE) data = _load(UPLOAD_TRACKER_FILE)
by_person = dict(data.get("by_person", {})) by_person = dict(data.get("by_person", {}))
entry = _migrate_entry(by_person.get(person_name, {})) entry = _migrate_entry(by_person.get(person_name, {}))
@@ -394,6 +486,7 @@ def reset_all_people() -> None:
Preferred over calling reset_person() in a loop when RESET_PERSON=* — that Preferred over calling reset_person() in a loop when RESET_PERSON=* — that
approach is O(P²) because each call rebuilds the flat list from all remaining entries. approach is O(P²) because each call rebuilds the flat list from all remaining entries.
""" """
with _locked():
upload_data = _load(UPLOAD_TRACKER_FILE) upload_data = _load(UPLOAD_TRACKER_FILE)
frigate_url = _get_frigate_url() frigate_url = _get_frigate_url()
if not frigate_url: if not frigate_url:
@@ -407,7 +500,9 @@ def reset_all_people() -> None:
if delete_frigate_person_files(person_name, frigate_filenames): if delete_frigate_person_files(person_name, frigate_filenames):
logger.info(f"Deleted {len(frigate_filenames)} Frigate file(s) for {person_name}") logger.info(f"Deleted {len(frigate_filenames)} Frigate file(s) for {person_name}")
else: else:
logger.warning(f"Could not delete Frigate files for {person_name} — tracker reset proceeding anyway") logger.warning(
f"Could not delete Frigate files for {person_name} — tracker reset proceeding anyway"
)
_save(UPLOAD_TRACKER_FILE, {}) _save(UPLOAD_TRACKER_FILE, {})
_save(REJECT_TRACKER_FILE, {}) _save(REJECT_TRACKER_FILE, {})
logger.info("Reset all tracking data") logger.info("Reset all tracking data")
@@ -421,6 +516,7 @@ def reset_person(person_name: str) -> None:
files (not in frigate_files) are never touched. Proceeds with tracker reset files (not in frigate_files) are never touched. Proceeds with tracker reset
even if Frigate is unreachable. even if Frigate is unreachable.
""" """
with _locked():
upload_data = _load(UPLOAD_TRACKER_FILE) upload_data = _load(UPLOAD_TRACKER_FILE)
entry = _migrate_entry(upload_data.get("by_person", {}).get(person_name, {})) entry = _migrate_entry(upload_data.get("by_person", {}).get(person_name, {}))
frigate_filenames = list(entry.get("frigate_files", {}).keys()) frigate_filenames = list(entry.get("frigate_files", {}).keys())