Compare commits

...
Author SHA1 Message Date
dependabot[bot] 6ec49976c7 chore(deps): bump astral-sh/setup-uv from 8.2.0 to 8.3.2
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.2.0 to 8.3.2.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](https://github.com/astral-sh/setup-uv/compare/fac544c07dec837d0ccb6301d7b5580bf5edae39...11f9893b081a58869d3b5fccaea48c9e9e46f990)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 8.3.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-09 18:54:17 +00:00
flan eba50f8a4a Merge pull request #48 from sudolulo/dependabot/docker/nvidia/cuda-12.9.2-cudnn-runtime-ubuntu24.04
chore(deps): bump nvidia/cuda from 12.8.1-cudnn-runtime-ubuntu24.04 to 12.9.2-cudnn-runtime-ubuntu24.04
2026-06-26 18:14:53 -04:00
flan 95d5e91a67 Merge pull request #49 from sudolulo/dependabot/uv/python-deps-c7af4feaef
chore(deps): bump the python-deps group with 3 updates
2026-06-26 18:14:22 -04:00
dependabot[bot] 9e904d9f34 chore(deps): bump the python-deps group with 3 updates
Bumps the python-deps group with 3 updates: [numpy](https://github.com/numpy/numpy), [pytest](https://github.com/pytest-dev/pytest) and [ruff](https://github.com/astral-sh/ruff).


Updates `numpy` from 2.4.6 to 2.5.0
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](https://github.com/numpy/numpy/compare/v2.4.6...v2.5.0)

Updates `pytest` from 9.1.0 to 9.1.1
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.1.0...9.1.1)

Updates `ruff` from 0.15.18 to 0.15.20
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.18...0.15.20)

---
updated-dependencies:
- dependency-name: numpy
  dependency-version: 2.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-deps
- dependency-name: pytest
  dependency-version: 9.1.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: python-deps
- dependency-name: ruff
  dependency-version: 0.15.20
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: python-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-25 18:55:02 +00:00
dependabot[bot] 18305b5b06 chore(deps): bump nvidia/cuda
Bumps nvidia/cuda from 12.8.1-cudnn-runtime-ubuntu24.04 to 12.9.2-cudnn-runtime-ubuntu24.04.

---
updated-dependencies:
- dependency-name: nvidia/cuda
  dependency-version: 12.9.2-cudnn-runtime-ubuntu24.04
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-25 18:53:34 +00:00
flan 9b68f62f69 Merge pull request #44 from sudolulo/dependabot/github_actions/actions/checkout-7.0.0
chore(deps): bump actions/checkout from 6.0.3 to 7.0.0
2026-06-20 15:06:54 -04:00
flan b5d03a2695 Merge branch 'main' into dependabot/github_actions/actions/checkout-7.0.0 2026-06-20 15:05:37 -04:00
flan fce46d409e Merge pull request #45 from sudolulo/dependabot/uv/python-deps-ae41630b22
chore(deps): bump the python-deps group across 1 directory with 5 updates
2026-06-20 15:05:13 -04:00
dependabot[bot] 5044343a99 chore(deps): bump the python-deps group across 1 directory with 5 updates
Bumps the python-deps group with 5 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [onnxruntime-gpu](https://github.com/microsoft/onnxruntime) | `1.26.0` | `1.27.0` |
| [nvidia-cudnn-cu12](https://developer.nvidia.com/cuda-zone) | `9.23.1.3` | `9.23.2.1` |
| [onnxruntime](https://github.com/microsoft/onnxruntime) | `1.26.0` | `1.27.0` |
| [pytest](https://github.com/pytest-dev/pytest) | `9.0.3` | `9.1.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.15.17` | `0.15.18` |



Updates `onnxruntime-gpu` from 1.26.0 to 1.27.0
- [Release notes](https://github.com/microsoft/onnxruntime/releases)
- [Changelog](https://github.com/microsoft/onnxruntime/blob/main/docs/ReleaseManagement.md)
- [Commits](https://github.com/microsoft/onnxruntime/commits)

Updates `nvidia-cudnn-cu12` from 9.23.1.3 to 9.23.2.1

Updates `onnxruntime` from 1.26.0 to 1.27.0
- [Release notes](https://github.com/microsoft/onnxruntime/releases)
- [Changelog](https://github.com/microsoft/onnxruntime/blob/main/docs/ReleaseManagement.md)
- [Commits](https://github.com/microsoft/onnxruntime/commits)

Updates `pytest` from 9.0.3 to 9.1.0
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.3...9.1.0)

Updates `ruff` from 0.15.17 to 0.15.18
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.17...0.15.18)

---
updated-dependencies:
- dependency-name: nvidia-cudnn-cu12
  dependency-version: 9.23.2.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-deps
- dependency-name: onnxruntime
  dependency-version: 1.27.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-deps
- dependency-name: onnxruntime-gpu
  dependency-version: 1.27.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-deps
- dependency-name: pytest
  dependency-version: 9.1.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: python-deps
- dependency-name: ruff
  dependency-version: 0.15.18
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: python-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-18 20:42:03 +00:00
dependabot[bot] 2a673fcd9d chore(deps): bump actions/checkout from 6.0.3 to 7.0.0
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.3 to 7.0.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/df4cb1c069e1874edd31b4311f1884172cec0e10...9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-18 20:40:32 +00:00
flan 04151057c9 Merge pull request #46 from sudolulo/dev
release: v0.6.6
2026-06-18 16:38:28 -04:00
flan 3ea6b9d566 Merge pull request #47 from sudolulo/chore/simplify-lockfile-ci
chore: replace lockfile auto-update with uv lock --check
2026-06-18 16:36:10 -04:00
flan e281ac01e1 chore: add pre-commit hook to auto-update lockfile on pyproject.toml changes 2026-06-18 20:33:58 +00:00
flan 0ac02c3108 chore: update lockfile for v0.6.6 2026-06-18 20:31:36 +00:00
flan db30449b09 chore: replace lockfile auto-update with uv lock --check 2026-06-18 20:29:43 +00:00
flan 1e4425bfd7 release: v0.6.6 2026-06-18 20:23:58 +00:00
flan 2be9f400dd Merge pull request #42 from sudolulo/fix/frigate-500-permanent-rejection
fix: Frigate 500 permanent rejection + 13 correctness bugs
2026-06-17 15:17:20 -04:00
flan f3b5bd9334 fix: update MAX_AUTO_IMAGES default assertion to 5 2026-06-17 19:16:43 +00:00
flan 0906342edf fix: 10 correctness bugs from full codebase audit
- immich_api: .get("items") or [] handles {"items": null} without crashing len()
- immich_api: catch TypeError alongside ValueError in filter_recent_assets for
  timezone-naive fileCreatedAt comparisons
- scheduler: catch SystemExit in addition to KeyboardInterrupt so cli.main()
  cannot kill the long-running scheduler process
- scheduler: reseed croniter from wall-clock time after each run so overrunning
  jobs don't schedule an immediate back-to-back rerun
- config: reject negative YEARS_FILTER values with a warning, reset to default 10
- frigate_api: return True (not False) for empty filenames list — callers cannot
  distinguish no-op from network failure on False
- jobs: warn on unrecognised STRATEGY value instead of silently falling back
- jobs: casefold ONLY_PEOPLE / SKIP_PEOPLE matching so "john doe" matches "John Doe"
- executor: <= → < so a same-score candidate can fill a freed replacement slot
- embeddings: set _insightface_loaded=True on GPU+CPU double-failure to prevent
  N re-init attempts (one per asset) when InsightFace is broken for a whole run
2026-06-17 19:12:58 +00:00
flan 5e0a871314 fix: 2 low findings from audit — consistent 500 match source, accurate return type
Use full_body for the 500 'could not process' permanent-rejection check,
consistent with the 400 'face' check on the line above. error_detail is
truncated to 100 chars via the fallback path, which could silently miss
the phrase in a long response body.

Remove | None from process_face_mode return type — every code path returns
tuple[int,int] or str; None is unreachable. Update docstring to match.
2026-06-17 17:47:31 +00:00
flan 86caffc8d7 fix: 3 audit findings — truthy skip-reason bug, 500 match consistency, CHANGELOG note
1. if saved: → if isinstance(saved, tuple): so string skip-reasons from
   process_face_mode no longer register as successes and create phantom
   asset_map entries with no JPEG on disk. Dead reason/fallback code in
   the else branch now correctly handles str and None returns.

2. "could not process" permanent-rejection check now uses error_detail
   (json message field, falling back to body[:100]) instead of full_body,
   keeping the match consistent with what is displayed to the user.

3. CHANGELOG [Unreleased] breaking-change note for MAX_AUTO_IMAGES 20→5
   so upgrading users know to set the env var if they want the old cap.
2026-06-17 17:31:20 +00:00
flan f6e494071e fix: lower MAX_AUTO_IMAGES default from 20 to 5
Smaller default cap is more conservative for new installs and better
reflects the minimum viable training set for Frigate face recognition.
Users who need more can set MAX_AUTO_IMAGES explicitly.
2026-06-17 17:19:09 +00:00
flan b69f776378 fix: surface skip reasons and suppress norm_crop FutureWarning
process_face_mode now returns a descriptive string instead of None for
filtered-out faces ("face too small 45x38px, min 90px", "no face
metadata"), so the executor can print a useful reason rather than the
generic "no usable face data".

Also suppresses the InsightFace norm_crop FutureWarning about deprecated
estimate usage, which was noisy at INFO level on every aligned crop.
2026-06-17 17:18:43 +00:00
flan c4910d82ef fix: treat Frigate 500 'Could not process' as permanent rejection
Frigate returns HTTP 500 with 'Could not process' when its face detector
cannot find or embed a face in the uploaded crop — this will never succeed
on retry. Previously these were silently logged at DEBUG and retried on
every future run.

- Surface 500 error details inline (same display path as 400)
- Mark 500 + 'could not process' as a permanent rejection so the asset
  is skipped on future runs instead of retried indefinitely
2026-06-17 17:16:04 +00:00
flan 5b8b3ab736 Merge pull request #41 from sudolulo/dev
fix: lockfile workflow opens PR on dev instead of direct push
2026-06-16 23:23:44 -04:00
flan dc0c431e6b chore: update lockfile 2026-06-17 03:12:20 +00:00
flan 3296940806 fix: lockfile workflow opens PR on dev instead of direct push
dev is a protected branch requiring PRs. The previous direct push caused
the lockfile update CI job to fail with 'protected branch hook declined'.
When the triggering branch is dev, the workflow now creates a side branch
and opens a PR; all other branches continue to push directly.
2026-06-17 03:12:00 +00:00
flan fdcb4efac3 Merge pull request #40 from sudolulo/dev
release: v0.6.5
2026-06-16 23:08:08 -04:00
flan c1f04be15b release: v0.6.5 2026-06-17 03:06:27 +00:00
flan 001dd2c575 fix: reinstall onnxruntime-gpu after uv sync to guarantee GPU binary wins
insightface depends on onnxruntime (CPU) as a direct dependency. During
uv sync --extra gpu, both onnxruntime (CPU, 24.6 MB binary) and
onnxruntime-gpu (GPU, 24.7 MB binary) are installed in parallel — both
claim onnxruntime/capi/onnxruntime_pybind11_state.so. The last writer wins,
which is non-deterministic in uv's parallel installer.

On GitHub Actions (no GPU, different scheduler ordering), the CPU binary
consistently wins, leaving onnxruntime-gpu's pybind11_state.so as the CPU
version. CUDAExecutionProvider then silently disappears because the CPU
binary's provider registration code has no CUDA EP.

Fix: after uv sync, reinstall onnxruntime-gpu explicitly using the already-
cached wheel. Since uv pip install is synchronous and runs after the parallel
sync completes, the GPU binary is guaranteed to be on disk when the build
layer commits.
2026-06-17 03:05:33 +00:00
flan edf576bc93 Merge pull request #39 from sudolulo/fix/codebase-audit-r2
fix: codebase audit rounds 2-5 (correctness, cli, jobs)
2026-06-16 23:02:42 -04:00
flan 561a1a3d72 fix: 5 findings from codebase audit round 5
cli.py:
- _smaller_duplicate_ids: walrus operator eliminates double p.get("id")
  per element; truthiness check replaces dead "is not None" guard (all
  persons in by_name are guaranteed to have a truthy id after the
  line-75 gate)
- Extract _excl() helper inside _handle_duplicate_people — replaces 4
  identical [p for p in lst if p.get("id") not in skip_ids] expressions
  across all return paths

jobs.py:
- Extract _valid_people() — shared filter for interactive_configure and
  auto_configure; uses (p.get("name") or "").strip() to match cli.py's
  whitespace-strip gate, preventing whitespace-only Immich names from
  reaching _build_job and creating blank Frigate person labels
- Hoist queued_ids set before the display loop in interactive_configure:
  O(N) set lookup per render instead of O(N×|jobs|) linear scan
2026-06-17 02:28:58 +00:00
flan 614542decd fix: 4 findings from codebase audit round 4
jobs.py:
- Add p.get("id") guard to valid_people filter in both
  interactive_configure and auto_configure — id-less named persons
  passed through by _handle_duplicate_people are now excluded before
  any bare-subscript access in the configure paths
- Fix bare p["id"] → p.get("id") in the queued-marker check at line 214
  (runs unconditionally on all valid_people during menu display, before
  any user selection or fetch_all_assets guard)

cli.py:
- Remove dead-code survivor_id and merge_ids guards: after the by_name
  fix (line 75 requires p.get("id")), all persons in any ordered list
  have ids, so neither guard can ever fire; removing them prevents
  misleading readers about what states are reachable
2026-06-17 02:20:38 +00:00
flan 3fccf9c8f9 fix: 6 findings from codebase audit round 3
cli.py:
- Filter id-less persons from by_name at construction (root fix for all
  bare-subscript crashes downstream — persons with a name but no id are
  excluded from duplicate detection entirely)
- Belt-and-suspenders on warning-path display: p['id'] → p.get('id')
- Extract survivor_id with .get(); skip group if survivor has no id
- Guard merge_ids: skip API call when list is empty after id filtering
- Walrus operator in merge_ids comprehension: p.get("id") called once
  per item instead of twice

executor.py:
- Add cross-reference comment at success-path reset so the for/else
  rollback pairing is explicit for future maintainers
2026-06-17 02:09:16 +00:00
flan eab3d9fe64 fix: 3 correctness bugs from codebase audit round 2
- executor.py: clear min_quality_score_for_slot alongside effective_count
  restore in for/else block; leaving the stale floor from the deleted
  file's score blocked the next candidate from filling the restored slot
- cli.py: guard merge_ids with p.get('id') is not None, consistent with
  the _smaller_duplicate_ids fix; bare p['id'] raised KeyError on any
  person dict missing the id field in the auto-merge path
- immich_api.py: replace bare data['major'/'minor'/'patch'] subscripts
  with .get() in get_immich_version; KeyError was silently swallowed by
  except Exception, causing version-gated flags to disable without warning
2026-06-17 01:54:13 +00:00
flan 5509be150e Merge pull request #38 from sudolulo/fix/codebase-audit-r1
fix: codebase audit r1 — correctness fixes, version banner, GPU dep
2026-06-16 21:51:09 -04:00
flan 0bd2eaf9fb fix: add missing nvidia CUDA pip packages for onnxruntime-gpu 1.26.0
ORT 1.26.0 changed provider loading to gate on the presence of required
nvidia pip packages before attempting to load libonnxruntime_providers_cuda.so.
Without nvidia-cuda-runtime-cu12, nvidia-cufft-cu12, and nvidia-curand-cu12
installed as Python packages, ORT silently skips the CUDA EP plugin entirely
(confirmed via /proc/maps: the .so was never dlopen'd despite existing on disk
and all system CUDA libs being present in ldconfig).

nvidia-nvjitlink-cu12 pulled in as a transitive dependency.
2026-06-17 01:47:49 +00:00
flan b80d26b36b feat: display version in startup banner 2026-06-17 01:18:45 +00:00
flan 068a8e675f fix: 4 correctness bugs from full-codebase audit
- executor: restore effective_count when replacement upload fails all retries
  (delete succeeded but slot was never filled, leaving cap undercount)
- diversity: skip zero-norm embeddings before dedup/FPS selection
  (InsightFace zeros pass dedup with similarity 0 and score distance 1.0,
  getting selected first as maximally diverse)
- cli: exclude None from skip_ids in _smaller_duplicate_ids
  (p.get('id') without None guard lets None into the set, silently
  dropping every other id-less person from the processed list)
- embeddings: select face nearest crop centre instead of largest by area
  (25% margin can pull a bigger neighbouring face into the crop;
  largest-face selection then embeds the wrong person)
2026-06-17 01:12:28 +00:00
flan c36e7bf28e Merge pull request #37 from sudolulo/dev
fix: exclude main from lockfile update workflow trigger
2026-06-16 21:03:35 -04:00
flan 0ffe08bc6f Merge pull request #36 from sudolulo/fix/lockfile-workflow-main-exclusion
fix: exclude main from lockfile update workflow trigger
2026-06-16 21:01:25 -04:00
flan 3423d41535 fix: exclude main from lockfile update trigger
main is protected and only receives merges from dev; pushing directly
to it from CI is blocked by branch protection rules.
2026-06-17 00:56:52 +00:00
flan 6e29407231 Merge pull request #35 from sudolulo/dev
release: v0.6.4
2026-06-16 20:21:19 -04:00
flan 5dcfde7c36 release: v0.6.4 2026-06-17 00:17:32 +00:00
flan 0914608bc8 fix: address 2 missed p[\"id\"] bare subscripts in cli.py (round 12)
Round 11's replace_all missed two occurrences:
- _smaller_duplicate_ids inner comprehension (line 84): p["id"] →
  p.get("id") so a named person with a missing "id" field does not
  crash skip_ids computation before any return path is reached
- all-merges-failed fallback return (line 157): same fix; the outer
  indentation prevented replace_all from matching this occurrence

The intentional p["id"] in merge_ids (line 119) is kept: that ID is
passed directly to merge_people() where None would be a caller bug,
not a silent data corruption.
2026-06-17 00:10:01 +00:00
flan 2182c87c40 fix: address 2 code review findings (round 11)
- upload_tracker: revert data[flat_key] = [] from round 8; clearing the
  entire shared legacy flat list on a corrupt value wipes all persons'
  IDs, not just the one being reset; since a corrupt non-list value is
  already unreadable by load_uploaded_ids, leaving it in place is safer
  than a mass-wipe; update warning message to note the field is unaffected
  but unreadable so the corruption is still observable
- cli: use p.get("id") instead of p["id"] in both people-list fallback
  returns (_handle_duplicate_people lines 144 and 157) for consistency
  with the success path at line 149; bare subscript crashes on malformed
  unnamed persons that bypass _smaller_duplicate_ids
2026-06-17 00:02:49 +00:00
flan 0236ed2d6b fix: address 3 code review findings (round 10)
- immich_api: use 'or []' instead of .get("people", []) in get_people
  so {"people": null} responses (some Immich versions with zero people
  enrolled) return [] rather than None; .get() default only fires when
  the key is absent, not when its value is null
- embeddings: log OSError from os.dup2 restore at DEBUG rather than
  silently swallowing it; if a C extension (CUDA/onnxruntime) invalidates
  the saved fd, the restore fails silently and stdout stays wired to
  /dev/null — logging makes the event observable without changing the
  swallow-and-continue semantics
- cache: remove MemoryError re-raise from EmbeddingCache.get(); a cache
  read OOM aborted the entire diversity-selection batch for the person
  rather than falling back to a fresh embedding computation, which is
  the more appropriate OOM gate; broadening back to except Exception
  restores the pre-round-5 fallback behavior
2026-06-16 23:51:21 +00:00
flan 34f7985357 docs: document BaseException limitation in _suppress_output finally block
A KeyboardInterrupt raised inside the saved_out cleanup block would
propagate past the saved_err and devnull_fd blocks, leaking those fds.
In CPython this race is not realistically triggerable — KI is delivered
between bytecodes and os.dup2 is a single atomic C syscall — so we
accept the theoretical risk rather than silencing BaseException in a
finally block.
2026-06-16 23:41:38 +00:00
flan 25880ded91 fix: address 2 code review findings (round 9)
- executor: revert person_has_fscores=True back into try/except else
  branch; moving it outside in round 8 was a regression — when the
  tracker write fails on the first-ever upload (no prior frigate_scores
  in tracker), setting the flag True prematurely switches at-cap
  replacement into fscore mode, get_most_redundant_mapped_file returns
  None (no entries), and all replacements are silently skipped;
  the flag must only be set when the score is actually written
- diversity: remove dead face_crop-None guard; any face that passes
  assess_quality (≥90 px MIN_FACE_WIDTH) produces a crop ≥135 px
  (face + 25% margin), which is always above the 30 px crop minimum,
  making the guard unreachable; _crop_face_from_thumbnail also calls
  _get_face_bbox internally, so face_bbox is not None guarantees the
  inner bbox check also passes
2026-06-16 23:40:05 +00:00
flan 461ceb7af4 fix: address 5 code review findings (round 8)
- diversity: fix hard_count regression from round 7 — revert to
  'is not None and < 0.85' so only images that actually receive a
  FPS boost (confirmed low confidence) are counted as hard examples;
  None-confidence images use conf_array=1.0 (no boost) and should
  not appear in the hard-example log count
- diversity: fix _scale_bbox_to_thumbnail to use explicit zero-guard
  for imageWidth/imageHeight (meta_w or 0; scale = img_w/meta_w if
  meta_w else 1.0) — mirrors image_processing.py pattern; prevents
  `or img_w` from silently treating imageWidth=0 as missing and
  returning scale=1.0 without surfacing the zero-metadata case
- embeddings: wrap all three os.close calls in _suppress_output
  finally block with try/except OSError: pass so a failed close
  in one branch cannot abort the outer finally and leak devnull_fd
  or the saved_err/saved_out fds
- upload_tracker: clear corrupt flat-list key (data[flat_key] = [])
  after the isinstance warning instead of leaving the corrupt value
  in place — prevents stale IDs persisting across reset_person calls
  and future load_uploaded_ids() from seeing a non-list value
- executor: move 'if pre_fscore is not None: person_has_fscores = True'
  out of the try/except else branch so it fires even when mark_uploaded
  raises; Frigate scores exist once measured regardless of tracker
  write success, and replacement strategy should reflect that
2026-06-16 23:24:11 +00:00
flan 7282c76b68 fix: address 5 code review findings (round 7)
- diversity: revert conf_array default from 0.5 back to 1.0 (np.ones);
  the 0.5 default caused None-confidence images to receive a 1.7× FPS
  boost and beat high-confidence detections — counter-productive for
  Frigate training data quality
- diversity: fix hard_count to include None-confidence images (count
  images where score is None or < 0.85, not only confirmed < 0.85);
  the previous check systematically undercounted boosted images when the
  Immich faces API omits the score field
- executor: fix garbled comment fragment "Skipped on / skipped when"
  left by a partial edit in round 4; merge into a single coherent sentence
- executor: expand actually_uploaded trade-off comment to document all
  three consequences of a tracker write failure (Frigate duplicate,
  quality-replacement exclusion, cap-slot consumption), not only the
  duplicate risk mentioned previously
- upload_tracker: add person_ids guard to reset_person isinstance check
  so the non-list warning only fires when cleanup would actually have run,
  not on no-op calls where person_ids is empty
2026-06-16 23:09:14 +00:00
flan 692d77ee9f fix: address 4 code review findings (round 6)
- executor: snapshot has_frigate_model = effective_count > 0 before the
  upload loop; use it in the recognize_face gate instead of the live
  effective_count, which is incremented mid-loop and would otherwise
  trigger recognize_face calls against an empty Frigate model on first run
- jobs: restore if already_uploaded > 0 guard before limit = capacity so
  first-run auto-strategy jobs keep limit="auto" and the FPS adaptive
  early-stop can fire instead of always filling MAX_AUTO_IMAGES slots
- cli: retry get_people() once after a post-merge empty response before
  falling back to the pre-merge list; improve warning to name expired API
  key as a possible cause alongside transient network errors
- diversity: hoist hard_weight = np.where(...) above the FPS while loop
  since conf_array is constant; eliminates one O(n) numpy pass per
  selected image
2026-06-16 22:22:51 +00:00
flan 2cb126a589 fix: address 5 code review findings (round 5)
- embeddings: move os.close into try/finally so saved_out/saved_err are
  always closed even when os.dup2 restore raises, preventing fd leak
- cache: replace narrow except tuple with except MemoryError: raise /
  except Exception: return None so struct.error and other np.load failures
  return None without masking OOM
- executor: fix first-run advisory message to check effective_count == 0
  (post-stale-cleanup) instead of pre_run_count; remove now-unused
  pre_run_count variable entirely
- jobs: remove dead "skip" entry from strategy_map (unreachable since the
  early-return at the top of _resolve_strategy fires first)
- upload_tracker: log a warning when reset_person encounters a non-list
  flat_key value instead of silently skipping the cleanup
2026-06-16 22:03:59 +00:00
flan 96099ed6e2 fix: address 8 code review findings (round 4)
- diversity: remove erroneous break outside if-faces in _scale_bbox_to_thumbnail
  (broke people-loop early for first unannotated person, defeating scale fix)
- diversity: increment quality_filtered for face-too-small crop skips so the
  summary log counts them alongside assess_quality failures
- diversity: fix hard-example log count to use original confidence_scores[i]
  instead of synthetic conf_array default (0.5), eliminating false 100%
  hard-example reports for persons with no Immich confidence data
- cache: add EOFError to except tuple in EmbeddingCache.get() so truncated
  .npy files return None instead of crashing the embedding pipeline
- embeddings: wrap each os.dup2 restore in its own try/except OSError in
  _suppress_output finally block so stderr is always restored even if the
  stdout restore raises
- executor: gate recognize_face on effective_count > 0 (post-stale-cleanup)
  instead of pre_run_count > 0 so recognize_face is not called against an
  untrained Frigate model after the user manually deletes all training files
- executor: document actually_uploaded trade-off in comment (appending
  unconditionally on tracker failure risks a Frigate duplicate but prevents
  permanent filename unmapping which breaks quality-replacement scoring)
- jobs: check strategy == "skip" before the has_embedding and custom_limit
  early-returns in _resolve_strategy so STRATEGY=skip is always honoured
2026-06-16 21:43:24 +00:00
flan 4af9da2550 fix: address 10 code review findings (round 3)
- diversity: scale face bbox to thumbnail space before quality check so
  check_face_size uses actual thumbnail pixels, not original-image coords
- diversity: skip asset when face bbox exists but crop guard rejects it,
  preventing InsightFace from picking the wrong person in a group photo
- diversity: add _scale_bbox_to_thumbnail helper (extracted from crop logic)
- diversity: use set for medoid membership test in _kmedoids (O(n) not O(n*k))
- diversity: remove dead np.unique in _select_time_spread (linspace produces
  strictly increasing indices; unique is a no-op and implies wrong semantics)
- embeddings: move os.open/os.dup calls inside try in _suppress_output so
  EMFILE during setup does not leak already-allocated fds
- immich_api: count and log assets with missing/unparseable fileCreatedAt in
  filter_recent_assets instead of silently discarding them
- executor: capture pre_run_count before stale-mapping cleanup so the
  "first run" coaching message doesn't fire after manual file deletion
- cli: use p['id'] (KeyError-safe) instead of p.get('id') in fallback path
  to match all other access sites on the same people list
- cache: narrow except to (OSError, ValueError) in EmbeddingCache.get so
  MemoryError propagates instead of converting OOM to a silent cache miss
2026-06-16 21:17:31 +00:00
flan 8bdce9253a fix: address 10 codebase audit findings — API guards, reconcile, merge fallback, tracker guards
- immich_api: guard resp.json() with isinstance(dict) check in get_people and
  fetch_all_assets so AttributeError doesn't escape on proxy/CDN non-dict responses
- executor: move actually_uploaded.append outside try/else so Frigate filename→asset_id
  mapping is created via reconcile even when the tracker write fails
- cli: fall back to pre-merge people list when re-fetch after merge returns empty
  (transient error) instead of silently dropping all people
- cli: treat ENABLE_FRIGATE_SCORES=false / BLUR_THRESHOLD=0 as not-set in
  the unsupported-vars warning (falsy string check replaces raw truthiness)
- upload_tracker: guard set(data[flat_key]) with isinstance(list) check in
  reset_person so a corrupted non-iterable legacy field doesn't crash mid-reset
- upload_tracker: guard dims[0]/dims[1] in find_by_crop_dimension with a
  length check so a truncated crop_dims entry doesn't raise IndexError
- cache: wrap os.remove() in clear() with try/except OSError to handle
  TOCTOU race with concurrent put() calls
- diversity: default conf_array to 0.5 (was 1.0) for faces with missing
  confidence so they receive a moderate diversity boost instead of being
  treated as high-confidence
- diversity: sort assets in the fast path (len <= limit) so return order is
  consistent with the sorted-by-fileCreatedAt path
2026-06-16 20:51:14 +00:00
flan 34fccf8839 fix: address 10 full-codebase audit findings + lint
Correctness:
- jobs: cap auto-diversity limit for brand-new people (was never capped,
  could exceed MAX_AUTO_IMAGES on first run)
- image_processing: separate None/0 guard for imageWidth/imageHeight so
  missing field is explicit rather than silently aliased to img_w
- upload_tracker (_mark, update_frigate_count): copy-before-mutate so
  exceptions between cache access and _save don't corrupt in-process state
- jobs: reject LIMIT=0 on no-embedding path (was silently empty run)
- jobs: add STRATEGY=skip to strategy_map so env var is honoured
- embeddings: convert to RGB before cvtColor so RGBA/grayscale thumbnails
  don't raise cv2.error and silently drop from diversity selection
- config: use falsy guard for OUTPUT_DIR so blank env var falls through
  to config file value
- reconcile: _ts() returns float("inf") on parse failure so unrecognised
  filenames sort last instead of collapsing to 0.0 and corrupting FIFO mapping
- diversity: remove dead selected_set (never read; -np.inf sentinel already
  prevents re-selection)

Lint (ruff):
- executor: sort upload_tracker import block (I001)
- executor: replace lambda is_better_than with operator.lt/gt (E731 x2)
- executor, upload_tracker: wrap long logger.warning calls (E501 x4)
2026-06-16 18:40:13 +00:00
flan 7a268d1ea2 chore: sync dev with main (v0.6.3) 2026-06-16 18:18:42 +00:00
flan 44cbedaf91 Merge branch 'main' of github.com:sudolulo/winnow 2026-06-16 18:16:27 +00:00
flan 3c2ce80282 Merge branch 'main' of github.com:sudolulo/winnow into dev 2026-06-16 18:16:14 +00:00
flan 3c0ef47fdc release: v0.6.3 2026-06-16 18:13:42 +00:00
flan cf7660595d chore: update lockfile 2026-06-16 18:13:42 +00:00
flan 14f759e960 fix: address 3 quality review findings — record_frigate_files_batch cache mutation, tracker_ok flag, LIMIT guard
- record_frigate_files_batch: copy-before-mutate so a write failure
  doesn't leave cache ahead of disk (same fix as remove_frigate_files_batch)
- executor: replace tracker_ok boolean with try/else
- jobs: collapse duplicate custom_limit is not None checks into one guard

Bump version to 0.6.3.
2026-06-16 18:13:36 +00:00
flan e8cb390fe4 fix: address 2 quality review findings — begin_batch dirty guard, LIMIT<=0 warning 2026-06-16 18:04:38 +00:00
flan 54b52b0a73 fix: address 3 quality review findings — batch reject tracker, skip flush when clean, hoist frigate url check 2026-06-16 17:55:34 +00:00
flan b622e58f1b fix: address 3 quality review findings — flush_batch finally guard, _laplacian_var helper, has_frigate_scores no-copy 2026-06-16 17:09:00 +00:00
flan f3622b8d41 fix: address 4 quality review findings — flush_batch order, batch finally guard, cache copy, LIMIT=0 fallthrough 2026-06-16 16:59:43 +00:00
flan 817fa17e41 fix: address 3 quality review findings — tracker_ok gate, LIMIT=0 warning, cache write log level 2026-06-16 16:41:32 +00:00
flan 8846a4f1df fix: address 3 post-fix audit findings — begin_batch flush guard, misleading debug log, shared asset_id score deletion 2026-06-16 16:19:54 +00:00
flan a6bae5da05 fix: address 10 audit findings — import bug, fscore stale flag, cache mutation, batch safety, falsy guards 2026-06-16 16:16:28 +00:00
flan 4cdd4657d6 fix: v0.6.2 — structural tracker refactor, batch writes, multi-instance prep
- Drop flat list as primary storage; derive uploaded/rejected IDs from by_person
  (single source of truth). Legacy flat lists in existing files still read for
  backward compat. Removes dual-representation sync hazard.
- Add begin_batch/flush_batch: per-person upload loop now does 1 os.replace
  instead of N (one per mark_uploaded call). Benefit on slow storage.
- reset_all_people(): RESET_PERSON=* is now O(1) disk writes instead of O(P^2).
- blur_score_from_image inlines cv2.Laplacian directly, removing assess_quality
  call overhead and decoupling from the full quality pipeline.
2026-06-16 15:44:41 +00:00
flan 2de0c02c4e Merge pull request #34 from sudolulo/dev
release: v0.6.0 — revert SQLite tracker to JSON backend
2026-06-15 11:53:23 -04:00
28 changed files with 915 additions and 512 deletions
+7
View File
@@ -0,0 +1,7 @@
#!/usr/bin/env bash
set -euo pipefail
if git diff --cached --name-only | grep -q "^pyproject\.toml$"; then
uv lock
git add uv.lock
fi
+4 -4
View File
@@ -54,7 +54,7 @@ jobs:
df -h
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
ref: ${{ inputs.tag || github.ref }}
@@ -178,7 +178,7 @@ jobs:
df -h
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
ref: ${{ inputs.tag || github.ref }}
@@ -264,7 +264,7 @@ jobs:
df -h
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
ref: ${{ inputs.tag || github.ref }}
@@ -347,7 +347,7 @@ jobs:
df -h
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
ref: ${{ inputs.tag || github.ref }}
+1 -1
View File
@@ -23,7 +23,7 @@ jobs:
echo "Disk space freed."
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Run Ruff
uses: astral-sh/ruff-action@0ce1b0bf8b818ef400413f810f8a11cdbda0034b # v4.0.0
+2 -2
View File
@@ -32,12 +32,12 @@ jobs:
echo "Disk space freed."
- name: Checkout
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 0
- name: Install uv
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
- name: Set up Python
run: uv python install 3.13
+5 -2
View File
@@ -13,14 +13,17 @@ jobs:
contents: read
steps:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Install uv
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
- name: Set up Python
run: uv python install 3.13
- name: Check lockfile is up to date
run: uv lock --check
- name: Install dependencies
run: uv sync --extra cpu
-46
View File
@@ -1,46 +0,0 @@
# .github/workflows/update-lockfile.yml
name: Update lockfile
on:
push:
branches:
- '**'
paths:
- 'pyproject.toml'
workflow_dispatch:
jobs:
update-lockfile:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- name: Install uv
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
- name: Set up Python
run: uv python install 3.13
- name: Regenerate lockfile
run: uv lock
- name: Check for changes
id: diff
run: |
if git diff --quiet uv.lock; then
echo "changed=false" >> "$GITHUB_OUTPUT"
else
echo "changed=true" >> "$GITHUB_OUTPUT"
fi
- name: Commit and push updated lockfile
if: steps.diff.outputs.changed == 'true'
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add uv.lock
git commit -m "chore: update lockfile"
git push
+100
View File
@@ -7,6 +7,106 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.6.6] - 2026-06-18
### Changed
- **`MAX_AUTO_IMAGES` default lowered from 20 to 5** — existing users who have not set this variable and already have more than 5 winnow-managed images in Frigate will find themselves at cap on the next run. With `QUALITY_REPLACEMENT=true` (the default), winnow will attempt to swap weaker images rather than uploading new ones. Set `MAX_AUTO_IMAGES=20` to restore the previous behaviour.
## [0.6.5] - 2026-06-17
### Added
- **Version displayed in startup banner**: winnow now prints its installed version at launch.
### Fixed
- **GPU image: `CUDAExecutionProvider` missing due to parallel install race**: `insightface` declares `onnxruntime` (CPU) as a dependency, causing `uv sync` to install both `onnxruntime` and `onnxruntime-gpu` in parallel — both packages claim the same `pybind11_state.so` binary. On GitHub Actions the CPU binary consistently won the race, leaving the GPU build without CUDA support at runtime despite all CUDA libraries being present. Fixed by reinstalling `onnxruntime-gpu` sequentially after `uv sync` to guarantee its GPU binary is on disk.
- **GPU extra was missing three required nvidia pip packages**: `onnxruntime-gpu` 1.26.0 gates CUDA EP loading on the Python-importability of `nvidia-cuda-runtime-cu12`, `nvidia-cufft-cu12`, and `nvidia-curand-cu12`. These packages were not declared in the `gpu` extra and were absent on fresh installs, silently disabling GPU inference.
- **`_handle_duplicate_people` raises `KeyError` on id-less person records**: bare `p["id"]` subscripts in the auto-merge loop and `_smaller_duplicate_ids` raised `KeyError` when Immich returned a person dict without an `id` field (e.g. unconfirmed face clusters). Fixed by using `p.get("id")` and filtering `None` from `skip_ids`.
- **`_smaller_duplicate_ids` could include `None` in the skip set**: `p.get("id")` without a `None` guard populated `skip_ids` with `None`, causing `p.get("id") not in skip_ids` to pass for every id-less person, so unnamed face clusters were silently re-included in all return paths.
- **`_handle_duplicate_people` dead code removed**: guards `if not survivor_id` and `if not merge_ids` became unreachable after the id-gate fix; their presence suggested they still ran.
- **`_valid_people` in `jobs.py` used wrong name filter**: whitespace-only names (e.g. `" "`) passed the `p.get("name")` truthiness check and were included in the person list. Fixed using `(p.get("name") or "").strip()` consistent with the cli.py gate.
- **`interactive_configure` queued-marker check was O(N²)**: `[j for j in jobs if j["person"]["id"] == p.get("id")]` ran a full scan over jobs for every person in the display loop. Replaced with a `queued_ids` set hoisted before the loop.
- **`executor.py` slot restore did not clear `min_quality_score_for_slot`**: when a replacement upload failed all retries after a deletion, `effective_count` was restored but the stale quality-score floor from the deleted file remained, blocking the next candidate from filling the slot.
- **`get_immich_version` swallowed `KeyError` on unexpected schema**: bare `data["major"]` / `data["minor"]` / `data["patch"]` subscripts were silently caught by the surrounding `except Exception`, returning `None` without logging. Replaced with `.get()` calls that log a debug warning on unexpected schemas.
- **Face embedding selects nearest face to crop centre, not largest by area**: a 25 % margin on the crop window can pull a larger neighbouring face into the bounding box; selecting the biggest face by area then embeds the wrong person. Centre-proximity is now used instead.
- **Zero-norm face embeddings skipped before diversity selection**: InsightFace occasionally returns a zero vector for low-quality detections; zero embeddings pass deduplication with similarity 0 and score distance 1.0, causing them to be selected first as maximally diverse.
- **`executor.py` slot restore did not clear `min_quality_score_for_slot`**: stale quality floor from the deleted file blocked the next candidate from filling the restored slot in quality-replacement mode.
## [0.6.4] - 2026-06-17
### Fixed
- **Face bbox scaled to thumbnail space before quality filtering**: `assess_quality` now receives coordinates in thumbnail-pixel space rather than detection-image space. Previously, a face detected on a full-resolution image (e.g. 4000 px wide) was compared against `MIN_FACE_WIDTH` using its original pixel dimensions, causing faces that appear small on the thumbnail to pass the quality filter — and faces that appear large to be incorrectly rejected.
- **`conf_array` default restored to 1.0 for faces with missing confidence**: the default was incorrectly set to 0.5, causing images with no `score` field in the Immich faces API response to receive a 1.7× FPS diversity boost and be selected ahead of genuinely high-confidence detections. The default is now 1.0 (no boost), treating missing confidence as neutral.
- **`hard_weight` computed once outside FPS loop**: `conf_array` is constant after initialisation; moving the `np.where` call outside the `while` loop eliminates one O(n) numpy pass per selected image.
- **`has_frigate_model` snapshot prevents mid-batch `recognize_face` calls on first run**: `effective_count` is incremented inside the upload loop, so using it as the `recognize_face` gate would incorrectly trigger scoring after the first upload on a first run. A boolean snapshot is now taken before the loop.
- **`person_has_fscores` only set when tracker write succeeds**: the flag was moved outside the `try/except else` block, causing at-cap replacement to switch into Frigate-score mode even when the score was never written to the tracker — `get_most_redundant_mapped_file` then returned `None` and all replacement candidates were silently skipped. The flag is now set only in the `else` branch.
- **`STRATEGY=skip` honoured before embedding and limit checks**: the strategy was silently converted to `auto` when InsightFace was available, because two early-returns in `_resolve_strategy` ran before the `strategy_map` lookup.
- **`limit="auto"` preserved on first run**: switching to `limit = capacity` unconditionally caused the FPS adaptive early-stop to never fire on a person's first upload run. `limit="auto"` is now kept when `already_uploaded == 0`.
- **`EmbeddingCache.get` falls back gracefully on all load errors**: a `MemoryError` during `np.load` of a cached embedding was re-raised, crashing the entire diversity-selection batch for that person. Cache-read failures of any kind now return `None` so the embedding is recomputed fresh.
- **`get_people` returns `[]` when Immich sends `{"people": null}`**: `.get("people", [])` only uses the default when the key is absent, not when its value is `null`. Changed to `data.get("people") or []` so null-valued responses are handled the same as missing keys.
- **`get_people` and `fetch_all_assets` guard against non-dict responses**: a proxy or CDN returning a JSON array (or other non-dict body) previously caused an `AttributeError` from `.get()`. Both functions now check `isinstance(data, dict)` and return an empty result with an error log.
- **`filter_recent_assets` counts and logs assets with missing or unparseable timestamps** instead of silently dropping them.
- **`_suppress_output` fd cleanup restructured**: the context manager now initialises `devnull_fd`, `saved_out`, and `saved_err` to `None` before the `try` block, so the `finally` can close only the descriptors that were successfully opened. Each `os.close` is wrapped in its own `try/except OSError` so a failed close cannot prevent subsequent descriptors from being released. `OSError` from `os.dup2` restore is logged at DEBUG rather than silently swallowed.
- **`blur_score_from_image` copies the image before thumbnail resize**: `Image.thumbnail` modifies the image in-place. When the caller's image was already in RGB mode (no convert copy), the resize would have mutated the caller's object. A copy is now made when `score_img is img`.
- **`imageWidth`/`imageHeight` zero-value treated as missing** in `image_processing.py`: the old `or img_w` fallback silently set `scale = 1.0` for a zero-valued dimension (correct) but also for `None` (also correct) with no distinction. The explicit `scale = img_w / meta_w if meta_w else 1.0` form matches the pattern used in the new `_scale_bbox_to_thumbnail` helper and makes the fallback intent clear.
- **`_mark` and `update_frigate_count` copy before mutate**: both functions now create a shallow copy of the top-level tracker dict before assigning into `by_person`, so a failed `_save` cannot leave the in-memory cache ahead of the on-disk file.
- **`reset_person` flat-list guard only warns when cleanup would have run**: the `isinstance(data[flat_key], list)` check previously emitted a warning even when `person_ids` was empty (a no-op call). The warning is now gated behind `person_ids and`, matching the guard on the cleanup branch.
- **`_handle_duplicate_people` uses `p.get("id")` consistently**: all four return-path filter comprehensions and the `_smaller_duplicate_ids` set comprehension now use `.get("id")` instead of bare `p["id"]`, preventing a `KeyError` if the Immich API returns a person record without an `id` field.
- **`K-Medoids` non-medoid membership test is O(1)**: `non_medoids` now filters against `set(medoids)` instead of the list, eliminating an O(k) scan per candidate on each outer iteration.
## [0.6.3] - 2026-06-16
### Fixed
- **`record_frigate_files_batch` no longer mutates the tracker cache before write**: the function shared the same cache-corruption-on-write-failure bug that was fixed in `remove_frigate_files_batch` in v0.6.1 — `data.setdefault("by_person", {})` mutated the cached dict in-place, so a disk-full or permission error left the in-memory cache ahead of the on-disk file. Now uses the same copy-before-mutate pattern (shallow copies of the top-level dict and `by_person` sub-dict) so a failed write leaves cache and disk in sync.
- **`tracker_ok` boolean flag replaced with try/else**: the intermediate boolean was a misleading placeholder — the `True` initial value suggested success before the operation ran. The control flow is now expressed directly with a try/except/else block.
- **`LIMIT` env var guard simplified**: the two adjacent `if custom_limit is not None` checks in `_resolve_strategy` are collapsed into a single `if custom_limit is not None:` with nested branches, removing redundant evaluation.
## [0.6.2] - 2026-06-16
### Changed
- **Flat `uploaded_asset_ids` / `rejected_asset_ids` lists dropped as primary storage**: asset IDs are now derived on read from `by_person` entries, which are the single source of truth. The legacy flat lists in existing tracker files are still read (union) so no assets become re-eligible after upgrading. New writes no longer maintain the flat lists. This removes the dual-representation sync hazard and paves the way for multi-instance support (per-instance `by_person` keying in a future release).
- **Tracker writes batched per person**: `mark_uploaded` calls inside the per-person upload loop are now accumulated in memory (`begin_batch`) and flushed in a single `os.replace` write at the end of each person's loop (`flush_batch`), reducing N tracker writes per person to 1. Benefits users on slow storage (NAS, SD card, spinning disks).
- **`RESET_PERSON=*` is now O(1) disk writes**: replaced the per-person `reset_person` loop with `reset_all_people()`, which makes one Frigate API call per person for file deletion and then clears both tracker files in two writes. Previously it was O(P²) iterations and 2P writes.
- **`blur_score_from_image` inlines Laplacian computation**: replaced the `assess_quality()` call (which ran grayscale, exposure, and confidence checks whose results were discarded) with a direct `cv2.Laplacian` computation. The function is now self-contained and does not silently inherit future costs added to the full quality pipeline.
## [0.6.1] - 2026-06-16
### Fixed
+3
View File
@@ -17,8 +17,11 @@ git clone https://github.com/sudolulo/winnow.git
cd winnow
git checkout dev
uv sync
git config core.hooksPath .githooks
```
The last line activates the project's git hooks. The pre-commit hook automatically runs `uv lock` and stages the result whenever `pyproject.toml` is part of a commit, keeping the lockfile in sync without any extra steps.
## Running Tests and Lint
```bash
+4 -2
View File
@@ -8,7 +8,7 @@
ARG VARIANT=gpu
FROM --platform=$BUILDPLATFORM nvidia/cuda:12.8.1-cudnn-runtime-ubuntu24.04 AS base-amd64-gpu
FROM --platform=$BUILDPLATFORM nvidia/cuda:12.9.2-cudnn-runtime-ubuntu24.04 AS base-amd64-gpu
FROM ubuntu:26.04 AS base-amd64-rocm
FROM ubuntu:22.04 AS base-amd64-intel
FROM ubuntu:26.04 AS base-amd64-cpu
@@ -50,7 +50,9 @@ RUN if [ "$VARIANT" = "cpu" ]; then \
elif [ "$VARIANT" = "intel" ]; then \
uv sync --frozen --no-dev --extra intel; \
elif [ "$VARIANT" = "gpu" ]; then \
uv sync --frozen --no-dev --extra gpu; \
uv sync --frozen --no-dev --extra gpu && \
ORT_GPU_VER=$(.venv/bin/python -c "import importlib.metadata; print(importlib.metadata.version('onnxruntime-gpu'))") && \
uv pip install --python .venv/bin/python --no-deps --reinstall "onnxruntime-gpu==$ORT_GPU_VER"; \
else \
echo "Unknown VARIANT: '$VARIANT'. Must be one of: cpu, rocm, intel, gpu" >&2; \
exit 1; \
+1 -1
View File
@@ -187,7 +187,7 @@ In scheduled mode the process (and loaded models) stays resident between runs. T
| Variable | Default | Description |
| :--- | :--- | :--- |
| `MAX_AUTO_IMAGES` | `20` | Maximum training images per person in Frigate |
| `MAX_AUTO_IMAGES` | `5` | Maximum training images per person in Frigate |
| `QUALITY_REPLACEMENT` | `true` | When at cap, swap a weaker tracked image for a better candidate. With Frigate scoring active, targets the most redundant image (highest pre-upload recognize score); otherwise uses blur score. Never touches manually added Frigate files. Set `false` to skip people at cap |
#### Advanced Tuning *(calibrated — do not adjust)*
+1 -1
View File
@@ -31,7 +31,7 @@ services:
# - USE_FULL_RESOLUTION=true # Use full-res images vs thumbnails (default: true)
# - MIN_CONFIDENCE=0.7 # Minimum face detection confidence (default: 0.7)
# - BLUR_THRESHOLD=100.0 # Laplacian blur threshold; lower = accept more blur (default: 100.0)
# - MAX_AUTO_IMAGES=80 # Hard cap on auto-diversity selection (default: 20)
# - MAX_AUTO_IMAGES=80 # Hard cap on auto-diversity selection (default: 5)
# ── Caching & Models ──────────────────────────────────────────────────
# - FORCE_CPU=true # Disable GPU, fall back to CPU
+10 -7
View File
@@ -1,6 +1,6 @@
[project]
name = "winnow"
version = "0.6.1"
version = "0.6.6"
description = "Selects diverse, high-quality photos from Immich as training data for Frigate face recognition."
license = "AGPL-3.0-or-later"
requires-python = ">=3.13"
@@ -17,7 +17,7 @@ classifiers = [
dependencies = [
"croniter>=5.0.2",
"insightface>=0.7.3",
"numpy>=2.2.6",
"numpy>=2.5.0",
"opencv-python-headless>=4.12.0.88",
"pillow>=12.1.0",
"python-dotenv>=1.2.1",
@@ -27,12 +27,15 @@ dependencies = [
[project.optional-dependencies]
gpu = [
"onnxruntime-gpu>=1.23.2; sys_platform == 'linux' and platform_machine == 'x86_64'",
"nvidia-cudnn-cu12>=9.0.0; sys_platform == 'linux' and platform_machine == 'x86_64'",
"onnxruntime-gpu>=1.27.0; sys_platform == 'linux' and platform_machine == 'x86_64'",
"nvidia-cudnn-cu12>=9.23.2.1; sys_platform == 'linux' and platform_machine == 'x86_64'",
"nvidia-cuda-runtime-cu12>=12.0; sys_platform == 'linux' and platform_machine == 'x86_64'",
"nvidia-cufft-cu12>=11.0; sys_platform == 'linux' and platform_machine == 'x86_64'",
"nvidia-curand-cu12>=10.0; sys_platform == 'linux' and platform_machine == 'x86_64'",
]
rocm = ["onnxruntime-rocm>=1.16.0; sys_platform == 'linux' and platform_machine == 'x86_64'"]
intel = ["onnxruntime-openvino>=1.20.0; sys_platform == 'linux' and platform_machine == 'x86_64'"]
cpu = ["onnxruntime>=1.23.2"]
cpu = ["onnxruntime>=1.27.0"]
[project.scripts]
winnow = "winnow.cli:main"
@@ -60,8 +63,8 @@ required-environments = [
[dependency-groups]
dev = [
"pytest>=8.0",
"ruff>=0.15.17",
"pytest>=9.1.1",
"ruff>=0.15.20",
]
[tool.hatch.build.targets.wheel]
+2 -1
View File
@@ -49,11 +49,12 @@ def _run_scheduler() -> None:
try:
main()
print("winnow run complete", flush=True)
except KeyboardInterrupt:
except (KeyboardInterrupt, SystemExit):
raise
except Exception as e:
logger.error("winnow run failed: %s", e, exc_info=True)
print(f"winnow run failed: {e}", flush=True)
cron = croniter(schedule, time.time())
next_run = cron.get_next(float)
print(f"Next run: {time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(next_run))}", flush=True)
time.sleep(min(60, max(1, next_run - time.time())))
+1 -1
View File
@@ -21,7 +21,7 @@ def test_config_loads_defaults(monkeypatch):
assert cfg.MIN_FACE_COUNT == 3
assert cfg.BLUR_THRESHOLD == 120.0
assert cfg.MIN_CONFIDENCE == 0.7
assert cfg.MAX_AUTO_IMAGES == 20
assert cfg.MAX_AUTO_IMAGES == 5
assert cfg.QUALITY_REPLACEMENT is True
assert cfg.FACE_MARGIN == 0.15
assert cfg.USE_FULL_RESOLUTION is True
Generated
+138 -99
View File
@@ -269,52 +269,42 @@ wheels = [
[[package]]
name = "numpy"
version = "2.4.6"
version = "2.5.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d0/ad/fed0499ce6a338d2a03ebae59cd15093910c8875328855781952abf6c2fe/numpy-2.4.6.tar.gz", hash = "sha256:f3a3570c4a2a16746ac2c31a7c7c7b0c186b95ce902e33db6f28094ed7387dda", size = 20735807, upload-time = "2026-05-18T23:37:14.07Z" }
sdist = { url = "https://files.pythonhosted.org/packages/e7/05/3d27272d30698dc0ecb7fdfaa41ad70303b444f81722bb99bce1d818638a/numpy-2.5.0.tar.gz", hash = "sha256:5a129578019311b6e56bdd714250f19b518f7dceeeb8d1af5490f4942d3f891c", size = 20652461, upload-time = "2026-06-21T20:57:51.95Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/fb/82/bdab26d7438c6791ca31b7c024ca37c1eab8b726ba236129005cd4a06e45/numpy-2.4.6-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:511dbaf848decaaaf4b4ca48032619fb3138710c4bf7da7617765edad1ef96b0", size = 16684648, upload-time = "2026-05-18T23:34:29.41Z" },
{ url = "https://files.pythonhosted.org/packages/1b/30/a80189bcc7f5e4258b3fbc3968d909d1756f54d023299ecc39ad6fdb9ef8/numpy-2.4.6-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bf162abab1c1a736333192707cef898e735a5ca00f38f27eeedf44b39d9e85eb", size = 14693902, upload-time = "2026-05-18T23:34:33.013Z" },
{ url = "https://files.pythonhosted.org/packages/97/12/70b5d0d7c15e1ebb8a6a84a8caa1d19e181d84fb58bb6d70aca29099dec1/numpy-2.4.6-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:043191bfa8eab18c776647b62723ac9dddece59743b13f49b2016094129c2b3f", size = 5198992, upload-time = "2026-05-18T23:34:36.132Z" },
{ url = "https://files.pythonhosted.org/packages/ba/8c/ebd2a8f8a83541f8d38cc5667e8c2b69cecfd30da6e45693e8158857d44b/numpy-2.4.6-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:6180d8b35af935aed8ece3a85e0a43f87393ae0ac87c8d2c8bd2c993f7270ef3", size = 6546944, upload-time = "2026-05-18T23:34:38.484Z" },
{ url = "https://files.pythonhosted.org/packages/bb/c5/7b863a97a91671a0338f4253bd3b5a3d3852f0692dae91711c9f4a10e787/numpy-2.4.6-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:72fbe16c6fac95aedf5937fa873445cec2110be35d8a4e9433d7501fd98dae6b", size = 15669392, upload-time = "2026-05-18T23:34:41.257Z" },
{ url = "https://files.pythonhosted.org/packages/a5/9d/3584b9984ca4c047aea75214ce1a4c4c73d849bd71b604264b7f5653f8a8/numpy-2.4.6-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a7830bab239b79cda9c08c2da014761cafb48da6150e1da17ac06283f43b6089", size = 16633220, upload-time = "2026-05-18T23:34:45.075Z" },
{ url = "https://files.pythonhosted.org/packages/05/ae/7c67fba23bd98caec7c99261f3a16072ade14813486b0282cb29846de832/numpy-2.4.6-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ef4aea96ce4d3b074422cb4f2f64e216bf9e213004bb58ecfdf50ea02ea8eb9a", size = 17020800, upload-time = "2026-05-18T23:34:49.065Z" },
{ url = "https://files.pythonhosted.org/packages/d9/5d/3b6725cb31d983c5e66916f5d36f6d7e5521129e4c4404d64f918292a5b6/numpy-2.4.6-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:dfa20cc6ca228e6b155b11da03825975ce66aea520985dbbddf0f2a5a495c605", size = 18357600, upload-time = "2026-05-18T23:34:52.709Z" },
{ url = "https://files.pythonhosted.org/packages/f7/da/2ccc6c2fe8898dee01d90c75c5f5f914a23daf99e3e0f59516a08760c8b5/numpy-2.4.6-cp313-cp313-win32.whl", hash = "sha256:56b39e5e0622a09a25bf5baf62f4bcf0cb8a41ae6e2819cf49bbc5a74c083f91", size = 5961134, upload-time = "2026-05-18T23:34:55.618Z" },
{ url = "https://files.pythonhosted.org/packages/b5/cd/9cc4dc876fb065d5c220aae4d5e14826b2715331bb7618ce1fb07a679d99/numpy-2.4.6-cp313-cp313-win_amd64.whl", hash = "sha256:c4fc99836233ea196540b17ab0983aff60ed07941751930f5f4d05bc3b3b7359", size = 12318598, upload-time = "2026-05-18T23:34:58.928Z" },
{ url = "https://files.pythonhosted.org/packages/39/1e/c0bcba1f8694116485fe28fd1be698c278fcda4141c5b0e53a2aed8b12a8/numpy-2.4.6-cp313-cp313-win_arm64.whl", hash = "sha256:a7c711e21628b52034bb5ab8d1bce291f752fcc5e92accc615778acee1ff4778", size = 10222272, upload-time = "2026-05-18T23:35:02.167Z" },
{ url = "https://files.pythonhosted.org/packages/63/6d/cc5619247c8f4204e507f5883528372e4ac4bb189e579fb859a12e480b1f/numpy-2.4.6-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:112b06a867b235ef466ed3508ddf0238050df9c727cafb5301ac385b899189a1", size = 14821197, upload-time = "2026-05-18T23:35:05.468Z" },
{ url = "https://files.pythonhosted.org/packages/00/58/f1c39161c87d9e9bed660f1ed4bafc0e403d5ec9650b6dd77aead07d489b/numpy-2.4.6-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:eaf7fa2de5c0be8ae6ff8e9bea2ccd725e980541244521d8d4b5f3354a27babe", size = 5326287, upload-time = "2026-05-18T23:35:08.693Z" },
{ url = "https://files.pythonhosted.org/packages/af/57/3917ab0fd97f271a8694513581b8a36c655f111c446852c302f04ccdb6fc/numpy-2.4.6-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:7265a2f3d436e54ef9f2b52b5c937e6be778781bd97a590319d7348f1c1ca997", size = 6646763, upload-time = "2026-05-18T23:35:11.459Z" },
{ url = "https://files.pythonhosted.org/packages/eb/0f/037e64c494b67581ae18193d770adef354c41f3f2c8ebf865602d949bf8f/numpy-2.4.6-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f74a575920ab21fe304421a3fc28793d82e299cae9eccb37084e9fc7f3617c20", size = 15728070, upload-time = "2026-05-18T23:35:14.79Z" },
{ url = "https://files.pythonhosted.org/packages/21/a6/5d2bae9c9542eb4df16dc9c46dc79c186e9bad53805dfa5399a6023c6db0/numpy-2.4.6-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ede83e07a75dd06bc501566c1eca2afc0d61677c1472ac9ad93fdee6e638a48d", size = 16681752, upload-time = "2026-05-18T23:35:18.836Z" },
{ url = "https://files.pythonhosted.org/packages/92/14/23d1dfb410ae362cd59ce53e936b1513d545eb40db3949ced632e19a459e/numpy-2.4.6-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:68bb27509ac1b9a3443094260f6326150663b06abe40b73a2f81160623da5b67", size = 17086024, upload-time = "2026-05-18T23:35:22.52Z" },
{ url = "https://files.pythonhosted.org/packages/4b/6e/23595a2c642cdf3bc567877064bdd7f91c8b0038a4453cf2daf7248eafe9/numpy-2.4.6-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:a0df0043bdb289bde1f62da130d20df23d58b45429f752bc7a8fc5325a225ecd", size = 18403398, upload-time = "2026-05-18T23:35:26.398Z" },
{ url = "https://files.pythonhosted.org/packages/8a/90/0ac3bc947217e66dec77e7cbc6a1979d1af70b6461b82f620d3bccd5e4c8/numpy-2.4.6-cp313-cp313t-win32.whl", hash = "sha256:29a287e0cf63ff528da061de6b9f64a4618da591ca1046aafc54062e40ca7eab", size = 6084971, upload-time = "2026-05-18T23:35:29.387Z" },
{ url = "https://files.pythonhosted.org/packages/77/71/5673e351671a1d2bd6063b91b44f70c0affea7d1516fa7a6572941ba4aa1/numpy-2.4.6-cp313-cp313t-win_amd64.whl", hash = "sha256:25c692919ac5a01f170a3bfcd62d745b24fd095c353d50812637d6fcab442e75", size = 12458532, upload-time = "2026-05-18T23:35:32.175Z" },
{ url = "https://files.pythonhosted.org/packages/3f/88/19d3503c5046e688f049274b27a3ef3d771152fa80d3ba3d01a3dff61abe/numpy-2.4.6-cp313-cp313t-win_arm64.whl", hash = "sha256:1e978ec1e8bd0e0e4de6bb75de9d30cbb74db6b6a2bb727618613703ca0167dd", size = 10291881, upload-time = "2026-05-18T23:35:35.465Z" },
{ url = "https://files.pythonhosted.org/packages/f8/91/3ab2044d05fd16d343c5ac2e69b127f1b2854040dd20b193257c78028bd3/numpy-2.4.6-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:06ca2f61ec4385a07a6977c55ba998a4466c123642b4a32694d3128fce18c079", size = 16683458, upload-time = "2026-05-18T23:35:38.353Z" },
{ url = "https://files.pythonhosted.org/packages/8e/62/764ce66fa4147ae6d73071a3abf804ffe606f174618697c571acdf26a7c9/numpy-2.4.6-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:38efbc8de75c7a0fc1ac190162d892787f3f47b57cc291231aafee36b80982b7", size = 14704559, upload-time = "2026-05-18T23:35:42.14Z" },
{ url = "https://files.pythonhosted.org/packages/60/61/23f27c172f022e04025b7dc2367f4d63c1a398120607ec896228649a6f48/numpy-2.4.6-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:d581b735e177fdcdce6fed8e7e8880a3fb6ee4e3653a3ac6af01c6f4c03effc5", size = 5209716, upload-time = "2026-05-18T23:35:45.377Z" },
{ url = "https://files.pythonhosted.org/packages/03/71/21cf70dc6ea3e3acb95fc53a265b2fc248b981f0194ceb5b475271b8809d/numpy-2.4.6-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:0a041d3d761dc3c35cc56ce0351506a02bcbc25f7b169f652435141a17db9096", size = 6543947, upload-time = "2026-05-18T23:35:47.926Z" },
{ url = "https://files.pythonhosted.org/packages/d5/91/64288395ee1799bd2e0b04a305dce9666da90c961e1f3fe982a05ee1c036/numpy-2.4.6-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:40fdc1ae7125e518ea98e53e69a4ebc27e1fd50510c47b7ea130cf21e5e1d42b", size = 15685197, upload-time = "2026-05-18T23:35:50.863Z" },
{ url = "https://files.pythonhosted.org/packages/f3/eb/ebffaa97dc55502df69584a8f0dcf07f69a3e0b3e2323670a2722db9aa39/numpy-2.4.6-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a2c306dea656c12c68f51f4cea133cbe78ca7435eb28c735eac1d3ebe73be6e8", size = 16638245, upload-time = "2026-05-18T23:35:54.752Z" },
{ url = "https://files.pythonhosted.org/packages/b8/0b/54f9da33128d7e350fab89c7455902eeae70349ee52bddb448dc4a576f45/numpy-2.4.6-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:33111801a01c12a8a1e3721f0a9232f8cfc8ae2c6b7098167e6f623c6073f402", size = 17036587, upload-time = "2026-05-18T23:35:58.355Z" },
{ url = "https://files.pythonhosted.org/packages/b6/f0/fdebc1052db1cc37c64beb22072d67cd6d1c71adca1299f53dec2b5e20d3/numpy-2.4.6-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ae506e6902902557576a26ff33eda8695e7ecb3cb36c3b573a0765dee114ebdb", size = 18363226, upload-time = "2026-05-18T23:36:02.845Z" },
{ url = "https://files.pythonhosted.org/packages/aa/b4/298628d98c72b57e57f7165ae6a481a1deaf6f3c28262a6e4c739c275930/numpy-2.4.6-cp314-cp314-win32.whl", hash = "sha256:aaf159caa35993cb1f56fb9b8e4610d35758e7ca005412eb1daa856a78c9c4b1", size = 6010196, upload-time = "2026-05-18T23:36:05.92Z" },
{ url = "https://files.pythonhosted.org/packages/df/ac/46de6dda46478f7942f839e094970be2d4a861e005c4b3bf07c92e291a09/numpy-2.4.6-cp314-cp314-win_amd64.whl", hash = "sha256:b507f5c4c1d508876d1819b6bf9a49d365b96320b5d4993426b33a23ca4b8261", size = 12450334, upload-time = "2026-05-18T23:36:09.107Z" },
{ url = "https://files.pythonhosted.org/packages/78/92/b8b798ac784102c0da830d2257d59358e3d3d90d1e2b3f2575dad976c5cf/numpy-2.4.6-cp314-cp314-win_arm64.whl", hash = "sha256:6f41ae150c4e32db4f3310cdaf64b1593a03dbabe29eec77fc9b50fe64061df6", size = 10495678, upload-time = "2026-05-18T23:36:12.766Z" },
{ url = "https://files.pythonhosted.org/packages/30/34/ec28d1aa8115971537c01469ab2011ee96827930f0a124de1000cc2a7ed7/numpy-2.4.6-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ece3d2cfe132e7d51f44a832b303895e6f2d499c5e74dfbdb06ee246147a304a", size = 14823672, upload-time = "2026-05-18T23:36:16.473Z" },
{ url = "https://files.pythonhosted.org/packages/16/bd/f6d1fede4e54e8042a7ff97bb495510f3c220f94bcd9e8b228e87c92cc0d/numpy-2.4.6-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:e3e5193ef5a3dc73bceee50f7fdc2c90dbb76c42df8d8fae3d1067a583df579e", size = 5328731, upload-time = "2026-05-18T23:36:19.767Z" },
{ url = "https://files.pythonhosted.org/packages/f4/f0/e105b9e2fd728a9910103884decd6951d9dd73896b914a98d9a231de02ee/numpy-2.4.6-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:17f9ade344e7d9b464a084d69bcf18fc691cb1db67c62ed80820bf4926d78f0e", size = 6649805, upload-time = "2026-05-18T23:36:22.266Z" },
{ url = "https://files.pythonhosted.org/packages/82/dd/1206a7ca6ab15e3f02069707ca96222e202af681bb73756da7527f3cb837/numpy-2.4.6-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9cd5ffd25db4e7ba6a375693b3fc0fc1791ec636c17db3720da19bde7180ec43", size = 15730496, upload-time = "2026-05-18T23:36:25.713Z" },
{ url = "https://files.pythonhosted.org/packages/51/e7/38d3ea825dcab85a591734decb2f6c67caa7c8367d374df1a1c3842f9b07/numpy-2.4.6-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7d92c3819208a60205a12a245c91ad70cb0a85336659b19b834205573ac8456e", size = 16679616, upload-time = "2026-05-18T23:36:29.652Z" },
{ url = "https://files.pythonhosted.org/packages/93/b7/caabfdf53edf663e0b4eb74d7d405d83baef09eb5e83bcd32d601d72b93e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e85b752a1e912b70eaad4fafbd4d1238007ab221de2009b9a2f5ae7461239895", size = 17085145, upload-time = "2026-05-18T23:36:33.449Z" },
{ url = "https://files.pythonhosted.org/packages/f9/45/68d7c33a6bcf3e5aa3bdbd57a367e6f615286dfd6482f97e8ffeb734306e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:29cb7f67d10b479ff07c17d33e39f78c07f71c40ef30d63c153d340e96cd3fb4", size = 18403813, upload-time = "2026-05-18T23:36:37.369Z" },
{ url = "https://files.pythonhosted.org/packages/9c/50/0753655aa844c99cd9e018aacf76f130f1bd81d881bb74bc0aef5d73a8ba/numpy-2.4.6-cp314-cp314t-win32.whl", hash = "sha256:260a5d70215b61ab4fadf5c7baacd64821842975eea312125ed3c39a6391b063", size = 6156982, upload-time = "2026-05-18T23:36:40.817Z" },
{ url = "https://files.pythonhosted.org/packages/b2/d4/7c67becf668f973cb490cec3e98dfd799d866f9c989a54d355672cfa0db6/numpy-2.4.6-cp314-cp314t-win_amd64.whl", hash = "sha256:81a1cca95ed5bb92aa8b10dd2cdc9a0d3853a50fad926c28b5d7e8ea54389627", size = 12638908, upload-time = "2026-05-18T23:36:43.996Z" },
{ url = "https://files.pythonhosted.org/packages/43/bb/e1c71a4295b1b1d1393d50dbb4f2a36283c6859d9d3892e84f00ec5a91d5/numpy-2.4.6-cp314-cp314t-win_arm64.whl", hash = "sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66", size = 10565867, upload-time = "2026-05-18T23:36:47.114Z" },
{ url = "https://files.pythonhosted.org/packages/8a/33/07675aaad7f26ea013d5e884d9a0d784b79c6bd7566c333f5a52fa3c610b/numpy-2.5.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:520e6b8be0a4b65840ac8090d4f51cef4bed66e2b0894d5a520f099adc24a9b2", size = 16784890, upload-time = "2026-06-21T20:56:40.799Z" },
{ url = "https://files.pythonhosted.org/packages/85/4b/953118a730ee3b35e28645e0eb4cf9beec5bdbb954e1ac2f5fcefba6bbc3/numpy-2.5.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:146b81cdd3967fdb6beca8ba25f00c58741d8f3cbd797f55af0fbe0bfec3469c", size = 11754584, upload-time = "2026-06-21T20:56:43.094Z" },
{ url = "https://files.pythonhosted.org/packages/44/9b/56dd530c367c74ae17411027cea4135ca57e1e0583bf5594cee18bd83217/numpy-2.5.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:126b88d95e8ff9b00c9e717aa540469f21d6180162f84c0caec51b16215d49cd", size = 5313904, upload-time = "2026-06-21T20:56:45.503Z" },
{ url = "https://files.pythonhosted.org/packages/ce/b0/bcd672edad27ecca7da1f7bb0ce72cd1706a4f2d79ae94990afc97c13e1c/numpy-2.5.0-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:d4313cef1594c5ce46c31b6e54e918338f63f16ee9322304e8c9114d6d81c8bd", size = 6648504, upload-time = "2026-06-21T20:56:47.567Z" },
{ url = "https://files.pythonhosted.org/packages/80/9e/15cdfcbd30a1544a46c9e487a00df331c4672450216538705a9e51fa6710/numpy-2.5.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:750fb097caf26fa878746d9d119f6f9da12dedcbff1eea966c3e3447647c4a9e", size = 15150086, upload-time = "2026-06-21T20:56:49.352Z" },
{ url = "https://files.pythonhosted.org/packages/32/4e/8d7656ccaab3e81e97258b8a9bc5f0c8502513a92fb4ceb0a2cbfebc17bf/numpy-2.5.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3893adc2dc7c0412ba76777db55a049215d99c9aa3113003be8f49f4f1290ab9", size = 16647250, upload-time = "2026-06-21T20:56:51.542Z" },
{ url = "https://files.pythonhosted.org/packages/3c/81/97060281b602ed07f21b12f4ec409eac1f75a2f91fbc829ed8b2becf3ad4/numpy-2.5.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:835e454dd99b238cdc5a3f63bce2371296f5ebc53ca1e0f8e6ddbb6d92a29aab", size = 16512864, upload-time = "2026-06-21T20:56:55.401Z" },
{ url = "https://files.pythonhosted.org/packages/33/ab/4496208146911f8d8ddb54f68a972aafa6c8d44babcb2ea03b0e5cc87c9d/numpy-2.5.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6f9836778081a0a3c02a6a21493f3e9f5b311f8d2541934f31f05583dc999ea4", size = 18408407, upload-time = "2026-06-21T20:56:57.75Z" },
{ url = "https://files.pythonhosted.org/packages/d4/9f/a4df67c181e4ee8b467aa3332dc2db10fd5c515136831302f3ca48bc0a01/numpy-2.5.0-cp313-cp313-win32.whl", hash = "sha256:0b525be4744b60bb0557ac872d53ef07d085b5f39622bc579c98d3809d05b988", size = 6054431, upload-time = "2026-06-21T20:57:00.016Z" },
{ url = "https://files.pythonhosted.org/packages/30/53/491e1c47c55b62ccc6a63c1c5b8635c73fc2258dddeb9bda27cae4a0ae96/numpy-2.5.0-cp313-cp313-win_amd64.whl", hash = "sha256:44353e2878930039db472b99dc353d749826e4010bd4d2a7f835e94a97a5c748", size = 12414420, upload-time = "2026-06-21T20:57:01.815Z" },
{ url = "https://files.pythonhosted.org/packages/eb/4a/25c2906f541e9d9f4c5769764db732e6627be91a13f4724fa10634d77db4/numpy-2.5.0-cp313-cp313-win_arm64.whl", hash = "sha256:48f54b00711f83a5f796b70c518e8c2b3c5848dda03a54911f23eb68519b9b60", size = 10339533, upload-time = "2026-06-21T20:57:03.961Z" },
{ url = "https://files.pythonhosted.org/packages/86/ad/abc44aaceaf7b17ee1edde2bbb4458da591bc79574cffff50c4bb35f00d1/numpy-2.5.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:f27582c55ba4c750b7c58c8faf021d2cd9324a662b466229db8a417b41368af9", size = 16783807, upload-time = "2026-06-21T20:57:06.253Z" },
{ url = "https://files.pythonhosted.org/packages/5d/39/b72e168daf9c00fb20c9fc996d00437ccecdef3102387775d29d7a62576d/numpy-2.5.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:28e7137057d551e4a83c4ae414e3451f50568409db7569aacc7f9811ee06a446", size = 11765215, upload-time = "2026-06-21T20:57:08.547Z" },
{ url = "https://files.pythonhosted.org/packages/f7/a0/8400a9c0e3625182347593f5e1f57da9a617a534794805c8df5518154ddc/numpy-2.5.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:e1da54b53e75cd9fcfc23efcc7edab2c6aecf97b6037566d8a0fe804af8ec57c", size = 5324493, upload-time = "2026-06-21T20:57:11.012Z" },
{ url = "https://files.pythonhosted.org/packages/f6/8c/0d104deaa0401c93395a629ec902891618a2eff76d19229139cb5a887bfc/numpy-2.5.0-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:694d8f74e156f7fd01179f1aa8faa2f648ab6ae0f70b6c3fe57a03249aea2303", size = 6645211, upload-time = "2026-06-21T20:57:12.919Z" },
{ url = "https://files.pythonhosted.org/packages/6a/d9/4a4a628c812750363786afc3d33492709a5cd64b215469c16b0f6c7bb811/numpy-2.5.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1a7569a7b53c77716f036bb28cb1c91f166a26ec7d9502cd1e4bdfe502fdec22", size = 15166004, upload-time = "2026-06-21T20:57:14.717Z" },
{ url = "https://files.pythonhosted.org/packages/a0/5e/2a902317d7fc4aa93236e80c932662dadfc459b323d758329e01775125e1/numpy-2.5.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:39a0433bd4086ebd462960cf375e19195bb07b53dc1d87dd5fcf47ad78576f03", size = 16650797, upload-time = "2026-06-21T20:57:16.906Z" },
{ url = "https://files.pythonhosted.org/packages/e9/a0/a0090e6329f4ca5992c07847bb579c5259a19953dc57255bb08793142ffb/numpy-2.5.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:929f0c79ac38bcbd7154fe631dc907abfeddbcc5027a896bd1f7767323271e7a", size = 16524647, upload-time = "2026-06-21T20:57:19.165Z" },
{ url = "https://files.pythonhosted.org/packages/5e/7d/6caf27734c42b65837e7461ed0dbbd6b6fc835060c9714ec59d673bb383a/numpy-2.5.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:cc4f247a47bbf070bfd70be53ccdcf47b800af563535e7bbe172322197c30e21", size = 18411841, upload-time = "2026-06-21T20:57:21.638Z" },
{ url = "https://files.pythonhosted.org/packages/13/dc/26edadbd812536769a82c2e9e002234e33feb5da43061d47a044f6d309b7/numpy-2.5.0-cp314-cp314-win32.whl", hash = "sha256:5dc71423499fab3f46f7a7201155ade1669ea101f2f429d332df9e72f8161731", size = 6106361, upload-time = "2026-06-21T20:57:23.844Z" },
{ url = "https://files.pythonhosted.org/packages/f2/9e/4dd1459282229a72d92dece2ae9138e5cac94a72263a7ceb48f37434c925/numpy-2.5.0-cp314-cp314-win_amd64.whl", hash = "sha256:ebb81d9d5443e0309d6c54894c3fbed74ad7da0714352a67b6d773cd189eae73", size = 12551749, upload-time = "2026-06-21T20:57:25.945Z" },
{ url = "https://files.pythonhosted.org/packages/05/a7/6bc6384c080b86c7f6c85c5bc5b540b24f4f679cd144791d99574e90d462/numpy-2.5.0-cp314-cp314-win_arm64.whl", hash = "sha256:3b94d0d0deceebfad3e67ae5c0e5eb87371e8f7a0581cd04a779928c2450cf1e", size = 10617072, upload-time = "2026-06-21T20:57:28.175Z" },
{ url = "https://files.pythonhosted.org/packages/86/6b/4a2b71d66ada5608ae02b63f150dfad520f6940721cb7f029ad270befc0e/numpy-2.5.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:22f3d43e362d650bc39db1f17851302874a148ca95ba6981c1dfb5fa6862f35b", size = 11881067, upload-time = "2026-06-21T20:57:30.104Z" },
{ url = "https://files.pythonhosted.org/packages/dc/b2/d365eb40a20efb49d67e9feb90494ed8511282ee1f5fa16006675c65397d/numpy-2.5.0-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:243563efb4cd7528a264567e9fd206c87826457322521d06206a00bfa316c927", size = 5440290, upload-time = "2026-06-21T20:57:32.193Z" },
{ url = "https://files.pythonhosted.org/packages/fa/5e/e9c03188de5f9b767e46a8fe988bcfd3efad066a4a3fda8b9cb11a93f895/numpy-2.5.0-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:84881d825ca75249b189bbee875fcfe3238aa5c479e6100893cda566e8e86826", size = 6748371, upload-time = "2026-06-21T20:57:33.933Z" },
{ url = "https://files.pythonhosted.org/packages/fd/1d/68c186a38a5027bae2c4ddd5ea681fdaf8b4d30fb7301def6d8ad270390f/numpy-2.5.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:cda12aa4779d42b8771180aba759c96f527d43446d8f380ab59e2b35e8489efd", size = 15214643, upload-time = "2026-06-21T20:57:35.677Z" },
{ url = "https://files.pythonhosted.org/packages/8c/67/73f67b7c7e20635baae9c4c3ead4ae7326a005900297a6110971abd62eb5/numpy-2.5.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1c0121101093d2bd74981b10f8837d78e794a8ff57834eb27179f49e1ba11ac6", size = 16690128, upload-time = "2026-06-21T20:57:38.159Z" },
{ url = "https://files.pythonhosted.org/packages/eb/05/d4c1fb0c46d02a27d6b2b8b319a78c90937acec8631c1641874670b31e6f/numpy-2.5.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:d371c92cfa09da00022f501ab67fafaea813d752eb30ac44336d45b1e5b0268a", size = 16577902, upload-time = "2026-06-21T20:57:40.447Z" },
{ url = "https://files.pythonhosted.org/packages/9e/1d/771c797d50fa26e4888989cccf1d50ee51f530d4e455ad2692dcb64fa711/numpy-2.5.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:9990713e9c38154c6861e7547f1e3fc7a87e75ff09bab24ef1cc81d81c2835e9", size = 18452814, upload-time = "2026-06-21T20:57:42.875Z" },
{ url = "https://files.pythonhosted.org/packages/e8/46/52fc0d2a68d7643f0f149eeea5a5d8ea2a3507056ac8afa83c9212606e8b/numpy-2.5.0-cp314-cp314t-win32.whl", hash = "sha256:edadfbd4794b1086c0d822f81863e8a68fc129d132fd0bb9e31e955d7fbbbdb7", size = 6253168, upload-time = "2026-06-21T20:57:45.101Z" },
{ url = "https://files.pythonhosted.org/packages/2a/be/6c8d1118b5f13b2881dc095d5b345de19c6638b8959c17409b6eff84c8aa/numpy-2.5.0-cp314-cp314t-win_amd64.whl", hash = "sha256:f7e5fa4382967ae6548bd2f174219afb908e294b0d5f625af01166edd5f7d9aa", size = 12736286, upload-time = "2026-06-21T20:57:46.935Z" },
{ url = "https://files.pythonhosted.org/packages/fd/6a/d3a169aaf8536cf228d56a09e04bcb713a2fe4410d4e2105b9419b5a9c89/numpy-2.5.0-cp314-cp314t-win_arm64.whl", hash = "sha256:016623417bb330d719d579daf2d6b9a01ddc52e41a9ed61a47f39fde46dcd865", size = 10686451, upload-time = "2026-06-21T20:57:49.313Z" },
]
[[package]]
@@ -340,17 +330,60 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/52/de/823919be3b9d0ccbf1f784035423c5f18f4267fb0123558d58b813c6ec86/nvidia_cuda_nvrtc_cu12-12.9.86-py3-none-win_amd64.whl", hash = "sha256:72972ebdcf504d69462d3bcd67e7b81edd25d0fb85a2c46d3ea3517666636349", size = 76408187, upload-time = "2025-06-05T20:12:27.819Z" },
]
[[package]]
name = "nvidia-cuda-runtime-cu12"
version = "12.9.79"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/bc/e0/0279bd94539fda525e0c8538db29b72a5a8495b0c12173113471d28bce78/nvidia_cuda_runtime_cu12-12.9.79-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:83469a846206f2a733db0c42e223589ab62fd2fabac4432d2f8802de4bded0a4", size = 3515012, upload-time = "2025-06-05T20:00:35.519Z" },
{ url = "https://files.pythonhosted.org/packages/bc/46/a92db19b8309581092a3add7e6fceb4c301a3fd233969856a8cbf042cd3c/nvidia_cuda_runtime_cu12-12.9.79-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:25bba2dfb01d48a9b59ca474a1ac43c6ebf7011f1b0b8cc44f54eb6ac48a96c3", size = 3493179, upload-time = "2025-06-05T20:00:53.735Z" },
{ url = "https://files.pythonhosted.org/packages/59/df/e7c3a360be4f7b93cee39271b792669baeb3846c58a4df6dfcf187a7ffab/nvidia_cuda_runtime_cu12-12.9.79-py3-none-win_amd64.whl", hash = "sha256:8e018af8fa02363876860388bd10ccb89eb9ab8fb0aa749aaf58430a9f7c4891", size = 3591604, upload-time = "2025-06-05T20:11:17.036Z" },
]
[[package]]
name = "nvidia-cudnn-cu12"
version = "9.23.1.3"
version = "9.23.2.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-cublas-cu12", marker = "platform_machine != 's390x'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/10/30/ecca1c8194c8077c4b57a3d96b56d96f15852551b01b919bae6429d92218/nvidia_cudnn_cu12-9.23.1.3-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:6dbc18f05aab2a323a4ffd43d985410608f7db7db9a8596e189cddbd3e527441", size = 778220760, upload-time = "2026-06-09T19:38:19.281Z" },
{ url = "https://files.pythonhosted.org/packages/a0/7e/e0a5d44bf070a1ff945050abc02ef1cff5ca9c6ab5dc6a16ab6322593a32/nvidia_cudnn_cu12-9.23.1.3-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:272d4815eef8f0dd21ecca768bfa18a618fb76ec31547dd0885cd49e76ebcd1d", size = 721095859, upload-time = "2026-06-09T19:41:00.27Z" },
{ url = "https://files.pythonhosted.org/packages/75/ec/62b56fc5e8219a268c6f62c4e9fb1369ebec049512328e650d1a9a28bcc8/nvidia_cudnn_cu12-9.23.1.3-py3-none-win_amd64.whl", hash = "sha256:b874af5bfab5e1010ae88bfead14bf8e9da6b20283582288f1c05f056090a398", size = 689996767, upload-time = "2026-06-09T19:44:25.343Z" },
{ url = "https://files.pythonhosted.org/packages/a4/19/e70f60a82986f65111d30ffd1744ed3478a3f8cd67bcdac38bf00d57d9da/nvidia_cudnn_cu12-9.23.2.1-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:3adf706a93367d400b4e7668fc14834c7551c9122eeb22f8aec6435b18da5b6f", size = 778221490, upload-time = "2026-06-16T03:36:51.086Z" },
{ url = "https://files.pythonhosted.org/packages/e1/ef/276c358c6ab44efbe68e69977657cb80927e7ab6d45968d042e1083f45e5/nvidia_cudnn_cu12-9.23.2.1-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:a5e706320218dc7d661b0e13402f204eeccd07b18d061b4d60668f80e464dd1e", size = 721096736, upload-time = "2026-06-16T04:39:58.1Z" },
{ url = "https://files.pythonhosted.org/packages/9b/93/b37f3a0fe29b1ae3bbb42c22cc25cb152971bb400a589cade336bdf5f4f3/nvidia_cudnn_cu12-9.23.2.1-py3-none-win_amd64.whl", hash = "sha256:549d6eb120cdd89429997243cd2cad1e864aac3a2f887a93f17836ce72d83873", size = 689997647, upload-time = "2026-06-16T04:47:21.167Z" },
]
[[package]]
name = "nvidia-cufft-cu12"
version = "11.4.1.4"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-nvjitlink-cu12", marker = "platform_machine != 's390x'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/9b/2b/76445b0af890da61b501fde30650a1a4bd910607261b209cccb5235d3daa/nvidia_cufft_cu12-11.4.1.4-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:1a28c9b12260a1aa7a8fd12f5ebd82d027963d635ba82ff39a1acfa7c4c0fbcf", size = 200822453, upload-time = "2025-06-05T20:05:27.889Z" },
{ url = "https://files.pythonhosted.org/packages/95/f4/61e6996dd20481ee834f57a8e9dca28b1869366a135e0d42e2aa8493bdd4/nvidia_cufft_cu12-11.4.1.4-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c67884f2a7d276b4b80eb56a79322a95df592ae5e765cf1243693365ccab4e28", size = 200877592, upload-time = "2025-06-05T20:05:45.862Z" },
{ url = "https://files.pythonhosted.org/packages/20/ee/29955203338515b940bd4f60ffdbc073428f25ef9bfbce44c9a066aedc5c/nvidia_cufft_cu12-11.4.1.4-py3-none-win_amd64.whl", hash = "sha256:8e5bfaac795e93f80611f807d42844e8e27e340e0cde270dcb6c65386d795b80", size = 200067309, upload-time = "2025-06-05T20:13:59.762Z" },
]
[[package]]
name = "nvidia-curand-cu12"
version = "10.3.10.19"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/14/1c/2a45afc614d99558d4a773fa740d8bb5471c8398eeed925fc0fcba020173/nvidia_curand_cu12-10.3.10.19-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:de663377feb1697e1d30ed587b07d5721fdd6d2015c738d7528a6002a6134d37", size = 68292066, upload-time = "2025-05-01T19:39:13.595Z" },
{ url = "https://files.pythonhosted.org/packages/31/44/193a0e171750ca9f8320626e8a1f2381e4077a65e69e2fb9708bd479e34a/nvidia_curand_cu12-10.3.10.19-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:49b274db4780d421bd2ccd362e1415c13887c53c214f0d4b761752b8f9f6aa1e", size = 68295626, upload-time = "2025-05-01T19:39:38.885Z" },
{ url = "https://files.pythonhosted.org/packages/e5/98/1bd66fd09cbe1a5920cb36ba87029d511db7cca93979e635fd431ad3b6c0/nvidia_curand_cu12-10.3.10.19-py3-none-win_amd64.whl", hash = "sha256:e8129e6ac40dc123bd948e33d3e11b4aa617d87a583fa2f21b3210e90c743cde", size = 68774847, upload-time = "2025-05-01T19:48:52.93Z" },
]
[[package]]
name = "nvidia-nvjitlink-cu12"
version = "12.9.86"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/46/0c/c75bbfb967457a0b7670b8ad267bfc4fffdf341c074e0a80db06c24ccfd4/nvidia_nvjitlink_cu12-12.9.86-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:e3f1171dbdc83c5932a45f0f4c99180a70de9bd2718c1ab77d14104f6d7147f9", size = 39748338, upload-time = "2025-06-05T20:10:25.613Z" },
{ url = "https://files.pythonhosted.org/packages/97/bc/2dcba8e70cf3115b400fef54f213bcd6715a3195eba000f8330f11e40c45/nvidia_nvjitlink_cu12-12.9.86-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:994a05ef08ef4b0b299829cde613a424382aff7efb08a7172c1fa616cc3af2ca", size = 39514880, upload-time = "2025-06-05T20:10:04.89Z" },
{ url = "https://files.pythonhosted.org/packages/dd/7e/2eecb277d8a98184d881fb98a738363fd4f14577a4d2d7f8264266e82623/nvidia_nvjitlink_cu12-12.9.86-py3-none-win_amd64.whl", hash = "sha256:cc6fcec260ca843c10e34c936921a1c426b351753587fdd638e8cff7b16bb9db", size = 35584936, upload-time = "2025-06-05T20:16:08.525Z" },
]
[[package]]
@@ -385,7 +418,7 @@ wheels = [
[[package]]
name = "onnxruntime"
version = "1.26.0"
version = "1.27.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "flatbuffers" },
@@ -394,25 +427,25 @@ dependencies = [
{ name = "protobuf" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/cf/a2/c801242685e0ce48a4ca51dfafbb588765e0446397e123be53ba5598f3f5/onnxruntime-1.26.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:ccce19c5f771b8268902f77d9fed9e88f9499465d6780808faa6611a789d33f0", size = 18016563, upload-time = "2026-05-08T19:07:28.081Z" },
{ url = "https://files.pythonhosted.org/packages/e2/64/0492c0b1db04e29b2630c87cfa36f9d6872b1ca8614b90c5cad58fac7d76/onnxruntime-1.26.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bdbed8cf3b672b66acb032f33a253bc27f42bce6ece48ae3fab4fa483a5e96e0", size = 16052634, upload-time = "2026-05-08T19:07:16.885Z" },
{ url = "https://files.pythonhosted.org/packages/3d/26/4d09ddc755a84fc8d5e192991626b0e0680e8f6c5d58f4f1d05c42bc48cf/onnxruntime-1.26.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c07af6fc6d5557835f2b6ee7a96d8b3235d0c57a8e230efdedaee106a8a3cbc6", size = 18185632, upload-time = "2026-05-08T19:07:38.756Z" },
{ url = "https://files.pythonhosted.org/packages/77/89/3e52249aa08fa301e217ecba07b5246a8338fa2b401e109326e3fc5be0f9/onnxruntime-1.26.0-cp313-cp313-win_amd64.whl", hash = "sha256:61bec80655efa460591c2bc655392d57d2650ce85533a6b9b3b7a790d7ea7916", size = 13026751, upload-time = "2026-05-08T19:08:06.2Z" },
{ url = "https://files.pythonhosted.org/packages/06/b3/c1c8782b14af6797c303de132d6eef26a9fb80dfacd3750ce57911d11c6b/onnxruntime-1.26.0-cp313-cp313-win_arm64.whl", hash = "sha256:a6677545ff451e3539a02746d2f207d8c5baa4a0a818886bb9d6a6eb9511ee89", size = 12796807, upload-time = "2026-05-08T19:07:54.879Z" },
{ url = "https://files.pythonhosted.org/packages/c3/f5/47b0676408abec652c14b84d7173e389837832d850c24f87184277313e8d/onnxruntime-1.26.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5e016edc15d3c19f36807e1c6b10be5b27807688c32720f91b5ae480a95215d0", size = 16057265, upload-time = "2026-05-08T19:07:19.603Z" },
{ url = "https://files.pythonhosted.org/packages/3b/45/33ab6deeef010ca844c877dd618cebc079590bbe52d2a3678e7223b1b908/onnxruntime-1.26.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f5fc48a91a046a6a5c9b147f83fb41d65d24d24923373b222cdd248f0f4f4aac", size = 18197590, upload-time = "2026-05-08T19:07:41.422Z" },
{ url = "https://files.pythonhosted.org/packages/40/89/17546c1c20f6bfc3ae41c22152378a26edfea918af3129e2139dcd7c99f3/onnxruntime-1.26.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:33a791f31432a3af1a96db5e54818b37aba5e5eefc2e6af5794c10a9118a9993", size = 18019724, upload-time = "2026-05-08T19:07:30.723Z" },
{ url = "https://files.pythonhosted.org/packages/bb/24/89457a35f6af29538a76647f2c18c3a28277e6c19234c847e7b4b7c19860/onnxruntime-1.26.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e90c00732c4553618103149d93f688e8c3063017938f8983e21a71d9f3b6d22e", size = 16054821, upload-time = "2026-05-08T19:07:22.348Z" },
{ url = "https://files.pythonhosted.org/packages/12/f9/15b2e1815cf570d238e0135529f80d2dce64e8e8818a1489cae83823c5c6/onnxruntime-1.26.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:01498e80ba8988428d08c2d51b1338f89e3de2a93e6ffe555f79c68f26a5c06b", size = 18185815, upload-time = "2026-05-08T19:07:44.179Z" },
{ url = "https://files.pythonhosted.org/packages/d7/65/2e11055faf015e4b07f45b513fa49b391baf2e19d92d77d73ebee13c1004/onnxruntime-1.26.0-cp314-cp314-win_amd64.whl", hash = "sha256:7ead61450d8405167c87dd3a31d8da1d576b490a57dab1aa8b82a7da6825f5aa", size = 13349887, upload-time = "2026-05-08T19:08:08.671Z" },
{ url = "https://files.pythonhosted.org/packages/19/e4/0f9d1a5718b1781c610c1e354765a3820597081754277a6a9a2b50705702/onnxruntime-1.26.0-cp314-cp314-win_arm64.whl", hash = "sha256:31d71a53490e46910877d0902b5ad99c69a5955e5c7ea6c82863519410e1ba7c", size = 13140121, upload-time = "2026-05-08T19:07:57.804Z" },
{ url = "https://files.pythonhosted.org/packages/1c/42/3b8e635f067d06d9f45bede470b8d539d101a4166c272213158dfd08b6ce/onnxruntime-1.26.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d7b6d258fb78fdfcf049795bcfaa74dcb90ae7baa277afd21e6fd28b83f2c496", size = 16057240, upload-time = "2026-05-08T19:07:25.163Z" },
{ url = "https://files.pythonhosted.org/packages/93/99/f2be40a31b908d96b861ae0ce98582fa376c18a7f816b9d5eb4cd6aa0a4c/onnxruntime-1.26.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4eefd386a45202aefb7a5132b94f32df9d506c9edcc7faf2fc60d65183f4b183", size = 18197382, upload-time = "2026-05-08T19:07:46.965Z" },
{ url = "https://files.pythonhosted.org/packages/fb/2b/54208fd03ad410480bc17edf4869376362da8bbf46fe186ddf4cb5cc20fe/onnxruntime-1.27.0-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:b3e5b58b8c89c2b20e086e890aa9527377e5c240dc3ecc1640d18e07705eeb1c", size = 18432958, upload-time = "2026-06-15T22:42:53.105Z" },
{ url = "https://files.pythonhosted.org/packages/ce/88/24fc51fcbb126da6d032372314e47b55c3faad58f2aa78c0e199ccd20b9c/onnxruntime-1.27.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:48b3d87eb560ff6a772240506f3c78d6d27c63cafedd5c775672e1194f968cfd", size = 16438180, upload-time = "2026-06-15T22:42:43.093Z" },
{ url = "https://files.pythonhosted.org/packages/cb/19/14929c3c2fe0b79b41cce24463062bf3afa4cdd3c19dccf00319caa92bff/onnxruntime-1.27.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6872443f236a554921cda6f318c900e2d0c226792cf3534d00e5057c6926e5d2", size = 18658445, upload-time = "2026-06-15T22:43:08.053Z" },
{ url = "https://files.pythonhosted.org/packages/7f/76/59ed932b0244acd7bbbd6449480053a6d958ea66357f022f932872e19287/onnxruntime-1.27.0-cp313-cp313-win_amd64.whl", hash = "sha256:760021bca514d64a811837820d351a08a41741f16f8b4c26450da708fecf14e6", size = 13357856, upload-time = "2026-06-15T22:43:37.315Z" },
{ url = "https://files.pythonhosted.org/packages/79/51/d1ec60ec7b1e2ae2d7340ba52b8a13529140039cd4407ba8dddbbc046582/onnxruntime-1.27.0-cp313-cp313-win_arm64.whl", hash = "sha256:2fdfa9df40a0ded0028ce6f9cd863264237f3970559dea2b81456e9ac4622b94", size = 13104412, upload-time = "2026-06-15T22:43:27.457Z" },
{ url = "https://files.pythonhosted.org/packages/5e/7d/e6bb1c6445c94f708c38cd8fbb7bf0264108c33498b9445c93e60fe6d329/onnxruntime-1.27.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:54c0c4e9202c36c4ecdb1f3443f5dfbfd5ee3b54d1362c4b4c6134110e74fb32", size = 16443331, upload-time = "2026-06-15T22:42:45.649Z" },
{ url = "https://files.pythonhosted.org/packages/72/1b/b18b31e806eabc41077810199fbbb36fbc2d5f19912416e5ccfbf73053d1/onnxruntime-1.27.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1b215aa662c8f983f7d6dedafe65a9be72c26e5338e0fe98b3e0422c32c85428", size = 18670967, upload-time = "2026-06-15T22:43:10.621Z" },
{ url = "https://files.pythonhosted.org/packages/3a/37/48ab79c39b58a7c9f6f5aac1fa0ff2b993eb2643393d6ed9e839ddb6f347/onnxruntime-1.27.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:0874edc171f470fc4dd2bbb60bc0989612ed1a8b89b365cda016630a93227f13", size = 18433941, upload-time = "2026-06-15T22:42:58.867Z" },
{ url = "https://files.pythonhosted.org/packages/6e/24/d535ca8a09dbf697f853377c8dc0820dbcaae5f334316b400b953afbcba8/onnxruntime-1.27.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5b51c014cf1a4fcd93c29a97eac8071fa27710dae05a4d0380bb60a66d60a62c", size = 16439970, upload-time = "2026-06-15T22:42:48.023Z" },
{ url = "https://files.pythonhosted.org/packages/f9/b1/ea9ee80c0bdaa4efb13f29f8c236f3740f6655e8c092a2d119515a5a652c/onnxruntime-1.27.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:445fb702ea5241ba813a3ce2febe2e9408a64f6ad2eb610924322c536165f7cd", size = 18659240, upload-time = "2026-06-15T22:43:13.165Z" },
{ url = "https://files.pythonhosted.org/packages/e9/f2/1404507d76a21940e8bf46f414e3d1abd94dc888cb89a30f4a540275846f/onnxruntime-1.27.0-cp314-cp314-win_amd64.whl", hash = "sha256:49e416be0d717338b6d041b99911b716d70c397d277056450724f93bdded3fc2", size = 13685306, upload-time = "2026-06-15T22:43:40.416Z" },
{ url = "https://files.pythonhosted.org/packages/10/e5/ca5cf012ccccb806c70e94aadfebca5606acc62b33eb88cec13352d0778f/onnxruntime-1.27.0-cp314-cp314-win_arm64.whl", hash = "sha256:856032937dd3bc7a7c141909c8d7ae4fde3e3f59bddf061ae627b9a051bda95c", size = 13456280, upload-time = "2026-06-15T22:43:29.693Z" },
{ url = "https://files.pythonhosted.org/packages/67/7b/dca330a8397e9d816c976d7aed4e24a4a2d279bb1e551e3d0221d1389b1d/onnxruntime-1.27.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c6197a02e3f620c4dc13cff51b80672409fc1ffab3aa2593911b19fd322ff48b", size = 16443274, upload-time = "2026-06-15T22:42:50.467Z" },
{ url = "https://files.pythonhosted.org/packages/b7/f6/2bac21f722aa45d876d4a51f26bd0ef30e704068a3cd5021a5a7cd784271/onnxruntime-1.27.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:370d211e1ceeac4cd5f45301655463ac59e27cdc74d9f7aeb2d19ff4b7a76715", size = 18670781, upload-time = "2026-06-15T22:43:17.151Z" },
]
[[package]]
name = "onnxruntime-gpu"
version = "1.26.0"
version = "1.27.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "flatbuffers", marker = "platform_machine != 's390x'" },
@@ -421,12 +454,12 @@ dependencies = [
{ name = "protobuf", marker = "platform_machine != 's390x'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/dd/97/fe8979f44b9275654b42f7bb556e30789b71a1b22998c83b540df2b1b774/onnxruntime_gpu-1.26.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cfda2fad535595bfc3e570eb588092717711dcb2957656d814695e0c9ceb1508", size = 276974871, upload-time = "2026-05-08T19:15:58.052Z" },
{ url = "https://files.pythonhosted.org/packages/67/3f/59f1777a394625ecc9a85636de57dc47c25dbb5f888da050f1463955a0ce/onnxruntime_gpu-1.26.0-cp313-cp313-win_amd64.whl", hash = "sha256:6ab9f9c741d2e239b2e321ab0d389c04329d4ab7f11e3b92dd3aa7db1c59dee4", size = 226548083, upload-time = "2026-05-08T19:09:44.408Z" },
{ url = "https://files.pythonhosted.org/packages/89/96/360328e3c463f7ea08e853c4239c397e83363dd0204de71a710dc1a544bd/onnxruntime_gpu-1.26.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bcf6f347cad9f88a9a625c2b352cf9de927528aedb627ebbc089a201f1990b94", size = 276992052, upload-time = "2026-05-08T19:16:09.892Z" },
{ url = "https://files.pythonhosted.org/packages/fd/c8/aa2dc0e79bba577f37d5448bcb32fea79977e07506684d8138c19a0f1077/onnxruntime_gpu-1.26.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9e6e4fb1ec9ae1cf456534d9115f106ab2a1ae96fa513b4ed0f4795302b4a2c6", size = 276978254, upload-time = "2026-05-08T19:16:22.096Z" },
{ url = "https://files.pythonhosted.org/packages/41/e7/923298431e669567d7ccc2a4c898b6534a47641a051569fd97165fe6d9b8/onnxruntime_gpu-1.26.0-cp314-cp314-win_amd64.whl", hash = "sha256:3e592439b0183d303c2374517b5b392599a3d50b2dc9de949b9b15731ac921c9", size = 229142768, upload-time = "2026-05-08T19:09:54.589Z" },
{ url = "https://files.pythonhosted.org/packages/97/91/93ffe5431d154989f5e04864a25a97eea480997d771232bcbbc538188241/onnxruntime_gpu-1.26.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:56dc7b73954ff4bdc71f5b8ab306b6f61be5d007881b6ef423a609e2b9cd088b", size = 276991545, upload-time = "2026-05-08T19:16:33.347Z" },
{ url = "https://files.pythonhosted.org/packages/8f/8d/6509743801131f1d3791f638ae31edef7701e93c7a4523d74115247dd7d1/onnxruntime_gpu-1.27.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7e6eb554d35706ee154b583dd32e4167a93aa55d3670f59754f43860fd649b92", size = 220304092, upload-time = "2026-06-18T18:27:31.18Z" },
{ url = "https://files.pythonhosted.org/packages/ac/86/8865182eba89cde187163daacfa7f1eb64958616cd080dfb3482be4c8a36/onnxruntime_gpu-1.27.0-cp313-cp313-win_amd64.whl", hash = "sha256:ac352199fb6ac3e366b45a690bb47c9b1c92b1be0e3958149dd1a30285832b9f", size = 213623636, upload-time = "2026-06-15T22:43:10.548Z" },
{ url = "https://files.pythonhosted.org/packages/cf/e6/16f5ba0f91bd8db5b2881ef1ec57d532c8f297b31b6a95ceab606f471c0a/onnxruntime_gpu-1.27.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3b5652761f14bc74729ffbc8fdc01767e541e163459f2e10aa65af61d5db64c8", size = 220331411, upload-time = "2026-06-18T18:27:39.738Z" },
{ url = "https://files.pythonhosted.org/packages/d7/97/5584057e355e5d8de3ae5cdd81e946ad2c9d8253626614eac805750645af/onnxruntime_gpu-1.27.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:209caede76604509fb7451178f56daa5292a933478864aaff87c6c29d1257593", size = 220303692, upload-time = "2026-06-18T18:27:48.183Z" },
{ url = "https://files.pythonhosted.org/packages/93/7f/6446c80d38ee2c8ec82365f44f1a842993f497da9ebf46a6b37c2ef4d2b7/onnxruntime_gpu-1.27.0-cp314-cp314-win_amd64.whl", hash = "sha256:986b94d513b052dc06b93c5a424f072f07f40d66be40ce2d6320d76d84fa4919", size = 214270425, upload-time = "2026-06-15T22:43:20.017Z" },
{ url = "https://files.pythonhosted.org/packages/83/06/e1b690df1c70772d76464c5d8b2d11b436ade1cf93a061303cdf9d72b3bf/onnxruntime_gpu-1.27.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:42696d6137d6974325a67c88a65dd41625e49bf076da79c658514b498b297142", size = 220327139, upload-time = "2026-06-18T18:27:59.096Z" },
]
[[package]]
@@ -608,7 +641,7 @@ wheels = [
[[package]]
name = "pytest"
version = "9.0.3"
version = "9.1.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32' or (extra == 'extra-6-winnow-cpu' and extra == 'extra-6-winnow-gpu') or (extra == 'extra-6-winnow-cpu' and extra == 'extra-6-winnow-intel') or (extra == 'extra-6-winnow-cpu' and extra == 'extra-6-winnow-rocm') or (extra == 'extra-6-winnow-gpu' and extra == 'extra-6-winnow-intel') or (extra == 'extra-6-winnow-gpu' and extra == 'extra-6-winnow-rocm') or (extra == 'extra-6-winnow-intel' and extra == 'extra-6-winnow-rocm')" },
@@ -617,9 +650,9 @@ dependencies = [
{ name = "pluggy" },
{ name = "pygments" },
]
sdist = { url = "https://files.pythonhosted.org/packages/7d/0d/549bd94f1a0a402dc8cf64563a117c0f3765662e2e668477624baeec44d5/pytest-9.0.3.tar.gz", hash = "sha256:b86ada508af81d19edeb213c681b1d48246c1a91d304c6c81a427674c17eb91c", size = 1572165, upload-time = "2026-04-07T17:16:18.027Z" }
sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl", hash = "sha256:2c5efc453d45394fdd706ade797c0a81091eccd1d6e4bccfcd476e2b8e0ab5d9", size = 375249, upload-time = "2026-04-07T17:16:16.13Z" },
{ url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" },
]
[[package]]
@@ -673,27 +706,27 @@ wheels = [
[[package]]
name = "ruff"
version = "0.15.17"
version = "0.15.20"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/8c/a9/3abdf488f1bf3d24c699415e454ed554a6350d5d89ce183be1ee0a3361ac/ruff-0.15.17.tar.gz", hash = "sha256:2ec446937fd16c8c4de2674a209cc5af64d9c6f17d21fbf1151054fa0bcf5219", size = 4743346, upload-time = "2026-06-11T17:54:47.663Z" }
sdist = { url = "https://files.pythonhosted.org/packages/43/dc/35b341fc554ba02f217fc10da57d1a75168cfbcf75b0ef2202176d4c4f2d/ruff-0.15.20.tar.gz", hash = "sha256:1416eb04349192646b54de98f146c4f59afe37d0decfc02c3cbbf396f3a28566", size = 4755489, upload-time = "2026-06-25T17:20:37.578Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/db/4d/e11259f5da07cb6afb2d074c31bf09da9671993f7329d4f15d2fdc458301/ruff-0.15.17-py3-none-linux_armv6l.whl", hash = "sha256:d9feddb927fc68bd295f5eebc587a7e42cfaf9b65f60ca4a2386febff575da8f", size = 10856677, upload-time = "2026-06-11T17:54:49.533Z" },
{ url = "https://files.pythonhosted.org/packages/29/3e/772d679e1a0dc058e58875bd2c0cb713a0530877b4a76fee3c7966df0d49/ruff-0.15.17-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:25805a226d741c47d274a35ad5c10a7dde175fcddfa511d7cf3da0a21eb3eab7", size = 11223443, upload-time = "2026-06-11T17:55:00.573Z" },
{ url = "https://files.pythonhosted.org/packages/68/58/bd41f7688b2fd5623012605130ed70e60aa7f2244baa3d5066bdd61530c8/ruff-0.15.17-py3-none-macosx_11_0_arm64.whl", hash = "sha256:f6ad73b14c2d18a3bf8ad7cb6974294d7f613a7898604826058e6ac64918ef4d", size = 10566458, upload-time = "2026-06-11T17:55:07.52Z" },
{ url = "https://files.pythonhosted.org/packages/d8/5b/733371013fcf1ec339e477ece6ab42bfe10bdd9bba8ee88a9516aa56bfc0/ruff-0.15.17-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6ba0c1e4f95bcb3869d0d30cbd5917071ef2e28665abfec970cdab0492c713ed", size = 10914483, upload-time = "2026-06-11T17:55:05.501Z" },
{ url = "https://files.pythonhosted.org/packages/bd/cc/6f24251cc0252f7239391ccb85833f320efad14ebe5b443943f37ced6332/ruff-0.15.17-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:81647960f10bff57d2e51cadd0c3950fe598400c852863a038720ef5b8cca91e", size = 10647497, upload-time = "2026-06-11T17:54:57.733Z" },
{ url = "https://files.pythonhosted.org/packages/68/dd/0d10c17ce1a1624d6fc3156309c3f834fdb5dfaad026ec90c85684f3990e/ruff-0.15.17-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0e01a84ddbc8c16c23055ba3924476850f1bbc1917cebbb9376665a63e74260d", size = 11416967, upload-time = "2026-06-11T17:54:51.461Z" },
{ url = "https://files.pythonhosted.org/packages/2f/91/556bfb156f6144f355e831c23db00b2fc4120f86b3ce81cc5f7fd2df51f3/ruff-0.15.17-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:84fe9f653152f8f294f9f7e03bf3a453d8b4a27f7a59c78c8666167f2b17b96c", size = 12335770, upload-time = "2026-06-11T17:54:45.793Z" },
{ url = "https://files.pythonhosted.org/packages/88/82/8b5999aa13355e926f06d9f42a32dcca862f623bf0363785ff89d607dffd/ruff-0.15.17-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:8c0fe88a7676e7a05b73174d4d4a59cb2ac21ff8263583f87a81a6018475a978", size = 11575441, upload-time = "2026-06-11T17:54:32.661Z" },
{ url = "https://files.pythonhosted.org/packages/11/93/f10377bb04109ca0e8cbc483ff1982c54b6d418210041776f93e8cdc7fa9/ruff-0.15.17-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ecfc3c7878fff94633ab0348524e093f9ce3243080416dd7d14f8ba400174719", size = 11557614, upload-time = "2026-06-11T17:54:34.698Z" },
{ url = "https://files.pythonhosted.org/packages/c7/a6/eeeae7f7d5493df41649ab3db92f086b2d0a30199e4efdf8e3dd7a033f24/ruff-0.15.17-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:b8461180b22420b1bdc289909410930761629fddf2a5aaf60fae1ab26cedc4c4", size = 11544450, upload-time = "2026-06-11T17:54:39.042Z" },
{ url = "https://files.pythonhosted.org/packages/32/88/5991ce565129a24dd4a00db1254b3b5db2e53018cbe4018ea5a89738e727/ruff-0.15.17-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:6eccbe50a038b503e7140b441aa9c7fc8c1f36edf23ebef9f4165c2f28f568b7", size = 10892524, upload-time = "2026-06-11T17:55:09.432Z" },
{ url = "https://files.pythonhosted.org/packages/f5/1d/0fdd248313425f55223968af04b0a42125466a8d88d21c1d99c6af0a51e8/ruff-0.15.17-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:382fc0521025f5a8ad447d8bdd523545d0d7646adb718eb1c2dac5065ec27c0f", size = 10659573, upload-time = "2026-06-11T17:54:36.824Z" },
{ url = "https://files.pythonhosted.org/packages/9e/0e/072e8260deb9461062ce9311ced27a8e541229a6ffd483013dd37661e43e/ruff-0.15.17-py3-none-musllinux_1_2_i686.whl", hash = "sha256:456d41fcd1b2777ad63f09a6e7121d43f7b688bbc76a800c10f7f8fb1f912c3f", size = 11127818, upload-time = "2026-06-11T17:55:03.124Z" },
{ url = "https://files.pythonhosted.org/packages/ab/b4/55060a34163121498014696b5f656db5b8c6963768f227dbf0d76b311073/ruff-0.15.17-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:b1a04bcc94ae6194e9db05d16ad31f298a7194bfbcb08258bbe589cee1d587b8", size = 11655901, upload-time = "2026-06-11T17:54:53.562Z" },
{ url = "https://files.pythonhosted.org/packages/49/71/9b29d6b87cef468d697f43c6a91e3fae4a80185779d7d5a4ef27d173439f/ruff-0.15.17-py3-none-win32.whl", hash = "sha256:596065960ab1ff593f744220c9fe6580eda00a95003cffa9f4048bb5b1bf0392", size = 10925574, upload-time = "2026-06-11T17:54:55.723Z" },
{ url = "https://files.pythonhosted.org/packages/3d/b2/8fc77f3723228836fa5d12497eb71c808f83782e10d058d2b15cfa14640b/ruff-0.15.17-py3-none-win_amd64.whl", hash = "sha256:6769e5fa1710b179b92e0bfa5a51735b35baea9013dadb06d5f44cbcf9547084", size = 12058788, upload-time = "2026-06-11T17:54:41.042Z" },
{ url = "https://files.pythonhosted.org/packages/2d/c7/c53e8dbff9c9dc4b7928773421ae294a5d28fcb8dcda1a089579d3a7e510/ruff-0.15.17-py3-none-win_arm64.whl", hash = "sha256:f3be1fbb34bcdfd146240d8fb92a709d4c2c8191348580a3c044ec60fa0b4456", size = 11355275, upload-time = "2026-06-11T17:54:43.635Z" },
{ url = "https://files.pythonhosted.org/packages/94/d9/2d5014f0253ba541d2061d9fa7193f48e941c8b21bb88a7ff9bbe0bd0596/ruff-0.15.20-py3-none-linux_armv6l.whl", hash = "sha256:00e188c53e499c3c1637f73c91dcf2fb56d576cab76ce1be50a27c4e80e37078", size = 10839665, upload-time = "2026-06-25T17:19:44.702Z" },
{ url = "https://files.pythonhosted.org/packages/c6/d3/ac1798ba64f670698867fcfc591d50e7e421bef137db564858f619a30fcf/ruff-0.15.20-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:9ebd1fd9b9c95fc0bd7b2761aebec1f030013d2e193a2901b224af68fe47251b", size = 11208649, upload-time = "2026-06-25T17:19:48.787Z" },
{ url = "https://files.pythonhosted.org/packages/47/47/d3ac899991202095dfcf3d5176be4272642be3cf981a2f1a30f72a2afb95/ruff-0.15.20-py3-none-macosx_11_0_arm64.whl", hash = "sha256:c5b16cdd67ca108185cd36dce98c576350c03b1660a751de725fb049193a0632", size = 10622638, upload-time = "2026-06-25T17:19:51.354Z" },
{ url = "https://files.pythonhosted.org/packages/33/13/4e043fe30aa94d4ff5213a9881fc296d12960f5971b234a5263fdc225312/ruff-0.15.20-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3413bb3c3d2ca6a8208f1f4809cd2dca3c6de6d0b491c0e70847672bde6e6efd", size = 10984227, upload-time = "2026-06-25T17:19:54.044Z" },
{ url = "https://files.pythonhosted.org/packages/76/e6/92e7bf40388bc5800073b96564f56264f7e48bfd1a498f5ced6ae6d5a769/ruff-0.15.20-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:bd7ec42b3bb3da066488db093308a69c4ac5ee6d2af333a86ba6e2eb2e7dd44b", size = 10622882, upload-time = "2026-06-25T17:19:57.037Z" },
{ url = "https://files.pythonhosted.org/packages/13/7a/43460be3f24495a3aa46d4b16873e2c4941b3b5f0b00cf88c03b7b94b339/ruff-0.15.20-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e1a36ad0eb77fba9aabfb69ede54de6f376d04ac18ebea022847046d340a8267", size = 11474808, upload-time = "2026-06-25T17:20:00.357Z" },
{ url = "https://files.pythonhosted.org/packages/27/a0/f37077884873221c6b33b4ab49eb18f9f88e54a16a25a5bca59bef46dd66/ruff-0.15.20-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b6df3b1e4610432f0386dba04d853b5f08cbbc903410c6fcc02f620f05aff53c", size = 12293094, upload-time = "2026-06-25T17:20:03.446Z" },
{ url = "https://files.pythonhosted.org/packages/a6/74/165545b60256a9704c21ac0ec4a0d07933b320812f9584836c9f4aca4292/ruff-0.15.20-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e89f198a1ea6ef0d727c1cf16088bc91a6cb0ab947dedc966715691647186eae", size = 11526176, upload-time = "2026-06-25T17:20:06.301Z" },
{ url = "https://files.pythonhosted.org/packages/86/b1/a976a136d40ade83ce743578399865f57001003a409acadc0ecbb3051082/ruff-0.15.20-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:309809086c2acb67624950a3c8133e80f32d0d3e27106c0cd60ff26657c9f24b", size = 11520767, upload-time = "2026-06-25T17:20:09.191Z" },
{ url = "https://files.pythonhosted.org/packages/19/0f/f032696cb01c9b54c0263fa393474d7758f1cdc021a01b04e3cbc2500999/ruff-0.15.20-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:2d2374caa2f2c2f9e2b7da0a50802cfb8b79f55a9b5e49379f564544fbf56487", size = 11500132, upload-time = "2026-06-25T17:20:13.602Z" },
{ url = "https://files.pythonhosted.org/packages/4b/f4/51b1a14bc69e8c224b15dab9cce8e99b425e0455d462caa2b3c9be2b6a8e/ruff-0.15.20-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:a1ed17b65293e0c2f22fc387bc13198a5de94bf4429589b0ff6946b0feaf21a3", size = 10943828, upload-time = "2026-06-25T17:20:16.635Z" },
{ url = "https://files.pythonhosted.org/packages/71/4b/fe267640783cd02bf6c5cc290b1df1051be2ec294c678b5c15fe19e52343/ruff-0.15.20-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:f701305e66b38ea6c91882490eb73459796808e4c6362a1b765255e0cdcd4053", size = 10645418, upload-time = "2026-06-25T17:20:19.4Z" },
{ url = "https://files.pythonhosted.org/packages/b0/c0/a65aa4ec2f5e87a1df32dc3ec1fede434fe3dfd5cbcf3b503cafc676ab54/ruff-0.15.20-py3-none-musllinux_1_2_i686.whl", hash = "sha256:5b9c0c367ad8e5d0d5b5b8537864c469a0a0e55417aadfbeca41fa61333be9f4", size = 11211770, upload-time = "2026-06-25T17:20:22.033Z" },
{ url = "https://files.pythonhosted.org/packages/5a/a4/0caa331d954ae2723d729d351c989cb4ca8b6077d5c6c2cb6de75e98c041/ruff-0.15.20-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:01cc00dd58f0df339d0e902219dd53990ea99996a0344e5d9cc8d45d5307e460", size = 11618698, upload-time = "2026-06-25T17:20:25.259Z" },
{ url = "https://files.pythonhosted.org/packages/10/9b/5f14927848d2fd4aa891fd88d883788c5a7baba561c7874732364045708c/ruff-0.15.20-py3-none-win32.whl", hash = "sha256:ed65ef510e43a137207e0f01cfcf998aeddb1aeeda5c9d35023e910284d7cf21", size = 10857322, upload-time = "2026-06-25T17:20:28.612Z" },
{ url = "https://files.pythonhosted.org/packages/fa/f0/fe47c501f9dea92a26d788ff98bb5d92ed4cb4c88792c5c88af6b697dc8e/ruff-0.15.20-py3-none-win_amd64.whl", hash = "sha256:a525c81c70fb0380344dd1d8745d8cc1c890b7fc94a58d5a07bd8eb9557b8415", size = 11993274, upload-time = "2026-06-25T17:20:31.871Z" },
{ url = "https://files.pythonhosted.org/packages/d7/2b/9555445e1201d92b3195f45cdb153a0b68f24e0a4273f6e3d5ab46e212bb/ruff-0.15.20-py3-none-win_arm64.whl", hash = "sha256:2f5b2a6d614e8700388806a14996c40fab2c47b819ef57d790a34878858ed9ca", size = 11343498, upload-time = "2026-06-25T17:20:35.03Z" },
]
[[package]]
@@ -862,7 +895,7 @@ wheels = [
[[package]]
name = "winnow"
version = "0.6.0"
version = "0.6.6"
source = { editable = "." }
dependencies = [
{ name = "croniter" },
@@ -880,7 +913,10 @@ cpu = [
{ name = "onnxruntime" },
]
gpu = [
{ name = "nvidia-cuda-runtime-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cudnn-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cufft-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-curand-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "onnxruntime-gpu", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" },
]
intel = [
@@ -900,10 +936,13 @@ dev = [
requires-dist = [
{ name = "croniter", specifier = ">=5.0.2" },
{ name = "insightface", specifier = ">=0.7.3" },
{ name = "numpy", specifier = ">=2.2.6" },
{ name = "nvidia-cudnn-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=9.0.0" },
{ name = "onnxruntime", marker = "extra == 'cpu'", specifier = ">=1.23.2" },
{ name = "onnxruntime-gpu", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=1.23.2" },
{ name = "numpy", specifier = ">=2.5.0" },
{ name = "nvidia-cuda-runtime-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=12.0" },
{ name = "nvidia-cudnn-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=9.23.2.1" },
{ name = "nvidia-cufft-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=11.0" },
{ name = "nvidia-curand-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=10.0" },
{ name = "onnxruntime", marker = "extra == 'cpu'", specifier = ">=1.27.0" },
{ name = "onnxruntime-gpu", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'gpu'", specifier = ">=1.27.0" },
{ name = "onnxruntime-openvino", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'intel'", specifier = ">=1.20.0" },
{ name = "onnxruntime-rocm", marker = "platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'rocm'", specifier = ">=1.16.0" },
{ name = "opencv-python-headless", specifier = ">=4.12.0.88" },
@@ -916,6 +955,6 @@ provides-extras = ["gpu", "rocm", "intel", "cpu"]
[package.metadata.requires-dev]
dev = [
{ name = "pytest", specifier = ">=8.0" },
{ name = "ruff", specifier = ">=0.15.17" },
{ name = "pytest", specifier = ">=9.1.1" },
{ name = "ruff", specifier = ">=0.15.20" },
]
+6 -3
View File
@@ -90,7 +90,7 @@ class EmbeddingCache:
np.save(tmp, embedding)
os.replace(tmp, final)
except Exception as e:
logger.debug("Cache write failed for %s: %s", asset_id, e)
logger.warning("Cache write failed for %s: %s", asset_id, e)
try:
os.remove(tmp)
except OSError:
@@ -103,8 +103,11 @@ class EmbeddingCache:
count = 0
for f in os.listdir(self.cache_dir):
if f.endswith(".npy"):
os.remove(os.path.join(self.cache_dir, f))
count += 1
try:
os.remove(os.path.join(self.cache_dir, f))
count += 1
except OSError:
pass
logger.info("Cleared %s cached embeddings.", count)
+34 -16
View File
@@ -7,12 +7,13 @@ import sys
from rich import print as rprint
from rich.prompt import Confirm
from . import __version__
from .config import Config, _getenv_bool
from .executor import execute_jobs, upload_to_frigate
from .immich_api import get_immich_version, get_people, merge_people
from .jobs import _show_preview, auto_configure, interactive_configure
from .log_config import console, setup_logging
from .upload_tracker import find_by_crop_dimension, get_person_summary, reset_person
from .upload_tracker import find_by_crop_dimension, get_person_summary, reset_all_people, reset_person
logger = logging.getLogger(__name__)
@@ -71,7 +72,7 @@ def _handle_duplicate_people(people: list[dict]) -> list[dict]:
by_name: dict[str, list[dict]] = defaultdict(list)
for p in people:
name = (p.get("name") or "").strip()
if name:
if name and p.get("id"):
by_name[name].append(p)
duplicates = {name: ps for name, ps in by_name.items() if len(ps) > 1}
@@ -81,17 +82,23 @@ def _handle_duplicate_people(people: list[dict]) -> list[dict]:
def _smaller_duplicate_ids(groups: dict) -> set[str]:
"""IDs of all but the largest person in each duplicate group."""
return {
p["id"]
pid
for ps in groups.values()
for p in sorted(ps, key=lambda x: x.get("assetCount", 0), reverse=True)[1:]
if (pid := p.get("id"))
}
skip_ids = _smaller_duplicate_ids(duplicates)
def _excl(lst: list[dict]) -> list[dict]:
return [p for p in lst if p.get("id") not in skip_ids]
if not Config.MERGE_DUPLICATE_PEOPLE:
rprint("\n[bold yellow]⚠ Duplicate person names detected in Immich:[/bold yellow]")
for name, ps in sorted(duplicates.items()):
ordered = sorted(ps, key=lambda x: x.get("assetCount", 0), reverse=True)
entries = ", ".join(
f"[dim]{p['id'][:8]}…[/dim] ({p.get('assetCount', 0)} assets)"
f"[dim]{(p.get('id') or '?')[:8]}…[/dim] ({p.get('assetCount', 0)} assets)"
for p in ordered
)
rprint(f" [yellow]{name}[/yellow] → {len(ps)} people: {entries}")
@@ -107,20 +114,21 @@ def _handle_duplicate_people(people: list[dict]) -> list[dict]:
)
# Return deduplicated list — keep only the largest per name so that
# downstream job creation never runs two jobs for the same Frigate folder.
return [p for p in people if p["id"] not in _smaller_duplicate_ids(duplicates)]
return _excl(people)
# Auto-merge: survivor = largest asset count, rest merge into it inside Immich
merged_any = False
for name, ps in sorted(duplicates.items()):
ordered = sorted(ps, key=lambda x: x.get("assetCount", 0), reverse=True)
survivor = ordered[0]
merge_ids = [p["id"] for p in ordered[1:]]
survivor_id = survivor.get("id")
merge_ids = [pid for p in ordered[1:] if (pid := p.get("id")) is not None]
rprint(
f" [cyan]Merging {name!r} inside Immich:[/cyan] keeping "
f"[dim]{survivor['id'][:8]}…[/dim] ({survivor.get('assetCount', 0)} assets), "
f"[dim]{survivor_id[:8]}…[/dim] ({survivor.get('assetCount', 0)} assets), "
f"absorbing {len(merge_ids)} smaller duplicate(s)..."
)
if merge_people(survivor["id"], merge_ids):
if merge_people(survivor_id, merge_ids):
rprint(f" [green]✓ Merged {name!r}[/green]")
merged_any = True
else:
@@ -129,12 +137,22 @@ def _handle_duplicate_people(people: list[dict]) -> list[dict]:
if merged_any:
rprint(" [dim]Re-fetching people after merge...[/dim]")
fresh = get_people()
if not fresh:
# Retry once: get_people() returns [] for both transient failures and
# auth errors (401); a second empty result strongly suggests a real failure.
fresh = get_people()
if not fresh:
logger.warning(
"Re-fetch after merge returned no people (tried twice)"
" — possible transient error or expired API key;"
" proceeding with pre-merge list. Check IMMICH_API_KEY if this recurs."
)
return _excl(people)
# Filter out the smaller duplicate from any group whose merge failed — those
# IDs still exist in Immich and would produce two jobs for the same folder.
# IDs from groups that merged successfully are already gone from Immich, so
# this filter is a no-op for them.
skip_ids = _smaller_duplicate_ids(duplicates)
return [p for p in fresh if p.get("id") not in skip_ids]
return _excl(fresh)
# All merges failed — fall back to local deduplication (keep largest per name) so
# downstream job creation never runs two jobs for the same Frigate folder.
@@ -142,7 +160,7 @@ def _handle_duplicate_people(people: list[dict]) -> list[dict]:
" [yellow]All merges failed — applying local deduplication"
" to avoid overwriting output.[/yellow]"
)
return [p for p in people if p["id"] not in _smaller_duplicate_ids(duplicates)]
return _excl(people)
_UNSUPPORTED_VARS = [
@@ -167,12 +185,13 @@ def main() -> None:
if trace_size:
_handle_trace_crop(trace_size)
console.print(r"""
[bold blue]winnow[/bold blue]
console.print(f"""
[bold blue]winnow[/bold blue] [dim]v{__version__}[/dim]
[dim]Immich -> Frigate Training Data Curator[/dim]
""")
set_unsupported = [v for v in _UNSUPPORTED_VARS if os.environ.get(v)]
_FALSY = {"", "false", "0", "no", "off"}
set_unsupported = [v for v in _UNSUPPORTED_VARS if os.environ.get(v, "").strip().lower() not in _FALSY]
if set_unsupported:
console.print(
f"[bold yellow]⚠ Advanced tuning vars set: "
@@ -207,8 +226,7 @@ def main() -> None:
"and will be reset along with everyone else.[/yellow]"
)
if names:
for name in names:
reset_person(name)
reset_all_people()
rprint(f"[bold yellow]Reset tracking data for all {len(names)} people.[/bold yellow]")
else:
rprint("[dim]No tracking data to reset.[/dim]")
+5 -2
View File
@@ -127,12 +127,15 @@ class _Config:
self.API_KEY = os.getenv("API_KEY")
self.OUTPUT_DIR = os.getenv("OUTPUT_DIR", "./frigate_train")
self.YEARS_FILTER = _getenv_int("YEARS_FILTER", 10)
if self.YEARS_FILTER < 0:
logging.warning("YEARS_FILTER=%s is negative — using default 10", self.YEARS_FILTER)
self.YEARS_FILTER = 10
self.MIN_FACE_WIDTH = _getenv_int("MIN_FACE_WIDTH", 90)
self.MIN_FACE_COUNT = _getenv_int("MIN_FACE_COUNT", 3)
self.MERGE_DUPLICATE_PEOPLE = _getenv_bool("MERGE_DUPLICATE_PEOPLE", False)
self.BLUR_THRESHOLD = _getenv_float("BLUR_THRESHOLD", 120.0)
self.MIN_CONFIDENCE = _getenv_float("MIN_CONFIDENCE", 0.7)
self.MAX_AUTO_IMAGES = _getenv_int("MAX_AUTO_IMAGES", 20)
self.MAX_AUTO_IMAGES = _getenv_int("MAX_AUTO_IMAGES", 5)
self.QUALITY_REPLACEMENT = _getenv_bool("QUALITY_REPLACEMENT", True)
self.FRIGATE_SCORE_CEILING = _getenv_optional_float("FRIGATE_SCORE_CEILING")
self.ENABLE_FRIGATE_SCORES = _getenv_bool("ENABLE_FRIGATE_SCORES", True)
@@ -173,7 +176,7 @@ class _Config:
data = json.loads(config_file.read_text())
if not self.IMMICH_URL:
self.IMMICH_URL = data.get("IMMICH_URL")
if os.getenv("OUTPUT_DIR") is None:
if not os.getenv("OUTPUT_DIR"):
self.OUTPUT_DIR = data.get("OUTPUT_DIR", self.OUTPUT_DIR)
except (json.JSONDecodeError, OSError) as e:
logging.warning("Failed to load config file: %s", e)
+50 -13
View File
@@ -57,9 +57,9 @@ def select_diverse_assets(
Returns:
List of selected assets
"""
# Fast path: fewer assets than limit
# Fast path: fewer assets than limit — sort for consistent ordering with other paths
if limit != "auto" and len(assets) <= limit:
return assets
return sorted(assets, key=lambda x: x.get("fileCreatedAt", ""))
# Sort by creation time
assets = sorted(assets, key=lambda x: x.get("fileCreatedAt", ""))
@@ -181,6 +181,28 @@ def _crop_face_from_thumbnail(
return crop
def _scale_bbox_to_thumbnail(
bbox: tuple[float, float, float, float],
img: Image.Image,
asset: dict,
person_id: str | None = None,
) -> tuple[float, float, float, float]:
"""Scale a face bbox from original detection-image space to thumbnail-pixel space."""
x1, y1, x2, y2 = bbox
img_w, img_h = img.size
for person in asset.get("people", []):
if person_id and person.get("id") != person_id:
continue
faces = person.get("faces", [])
if faces:
meta_w = faces[0].get("imageWidth") or 0
meta_h = faces[0].get("imageHeight") or 0
scale_x = img_w / meta_w if meta_w else 1.0
scale_y = img_h / meta_h if meta_h else 1.0
return (x1 * scale_x, y1 * scale_y, x2 * scale_x, y2 * scale_y)
return bbox
# =============================================================================
# Embedding Collection
# =============================================================================
@@ -258,9 +280,13 @@ def _select_by_embedding(
confidence = _get_face_confidence(asset, person_id=person_id)
face_bbox = _get_face_bbox(asset, person_id=person_id)
thumbnail_bbox = (
_scale_bbox_to_thumbnail(face_bbox, img, asset, person_id)
if face_bbox is not None else None
)
quality = assess_quality(
img,
face_bbox=face_bbox,
face_bbox=thumbnail_bbox,
confidence=confidence,
blur_threshold=Config.BLUR_THRESHOLD,
min_face_px=Config.MIN_FACE_WIDTH,
@@ -277,6 +303,9 @@ def _select_by_embedding(
emb = get_embedding(embed_img, asset_id=asset["id"])
if emb is not None:
if np.linalg.norm(emb) < 1e-6:
logger.debug("Zero-norm embedding for asset %s, skipping", asset["id"])
continue
embeddings.append(emb)
valid_candidates.append(asset)
confidence_scores.append(confidence)
@@ -408,7 +437,8 @@ def _kmedoids(dist_matrix: np.ndarray, k: int, max_iter: int = 50) -> tuple[list
for _ in range(max_iter):
improved = False
# Try swapping each medoid with a random non-medoid
non_medoids = [i for i in range(n) if i not in medoids]
medoid_set = set(medoids)
non_medoids = [i for i in range(n) if i not in medoid_set]
if not non_medoids:
break
@@ -483,7 +513,10 @@ def _cluster_aware_selection(
norms = np.linalg.norm(emb_matrix, axis=1, keepdims=True)
emb_normed = emb_matrix / np.maximum(norms, 1e-8)
# Build confidence weight array for hard example boosting
# Build confidence weight array for hard example boosting.
# Default to 1.0 for faces with no confidence score: treat as high-confidence
# (no boost) rather than hard-example territory. A missing score field should
# not cause these images to beat genuinely high-confidence detections in FPS.
conf_array = np.ones(n)
if confidence_scores:
for i, c in enumerate(confidence_scores):
@@ -508,7 +541,6 @@ def _cluster_aware_selection(
medoid_indices, cluster_labels = _kmedoids(dist_matrix, k)
selected = list(medoid_indices)
selected_set = set(selected)
logger.debug("Selected %s cluster medoids as initial picks.", len(selected))
@@ -522,10 +554,11 @@ def _cluster_aware_selection(
for idx in selected:
min_dists[idx] = -np.inf
# Hard example weighting: boost distance for low-confidence candidates.
# conf_array is constant after this point, so compute once outside the loop.
hard_weight = np.where(conf_array < 0.85, 1.0 + (0.85 - conf_array) * 2.0, 1.0)
while len(selected) < target:
# Hard example weighting: boost distance for low-confidence candidates
# Confidence < 0.85 gets up to 1.5× distance boost
hard_weight = np.where(conf_array < 0.85, 1.0 + (0.85 - conf_array) * 2.0, 1.0)
weighted_dists = min_dists * hard_weight
best_idx = int(np.argmax(weighted_dists))
@@ -541,15 +574,19 @@ def _cluster_aware_selection(
break
selected.append(best_idx)
selected_set.add(best_idx)
# Update min distances
dists_to_new = dist_matrix[best_idx]
min_dists = np.minimum(min_dists, dists_to_new)
min_dists[best_idx] = -np.inf
selected_conf = [conf_array[i] for i in selected if conf_array[i] < 1.0]
hard_count = sum(1 for c in selected_conf if c < 0.85)
hard_count = sum(
1 for i in selected
if confidence_scores
and i < len(confidence_scores)
and confidence_scores[i] is not None
and confidence_scores[i] < 0.85
)
logger.info("Selection complete: %s images (%s hard examples with confidence < 0.85).", len(selected), hard_count)
# Slice to target: the while loop enforces this for non-auto mode, but
@@ -576,4 +613,4 @@ def _select_time_spread(assets: list, limit: int | str) -> list:
return assets
indices = np.linspace(0, len(assets) - 1, limit, dtype=int)
return [assets[i] for i in np.unique(indices)]
return [assets[i] for i in indices]
+47 -13
View File
@@ -26,22 +26,47 @@ logger = logging.getLogger(__name__)
@contextmanager
def _suppress_output():
"""Suppress stdout/stderr at the file-descriptor level, silencing C extension noise."""
devnull_fd = os.open(os.devnull, os.O_WRONLY)
saved_out, saved_err = os.dup(1), os.dup(2)
devnull_fd = None
saved_out = None
saved_err = None
try:
devnull_fd = os.open(os.devnull, os.O_WRONLY)
saved_out = os.dup(1)
saved_err = os.dup(2)
os.dup2(devnull_fd, 1)
os.dup2(devnull_fd, 2)
yield
finally:
try:
os.dup2(saved_out, 1)
finally:
# Each block is a separate sequential statement. A BaseException (e.g.
# KeyboardInterrupt) raised inside block N would propagate past blocks N+1
# and N+2, leaving saved_err or devnull_fd unclosed. In CPython, KI is
# delivered between bytecodes, not mid-syscall; os.dup2 is a single C call
# and completes atomically, so this race is not realistically triggerable.
if saved_out is not None:
try:
os.dup2(saved_out, 1)
except OSError as e:
logger.debug("_suppress_output: failed to restore stdout fd: %s", e)
finally:
try:
os.close(saved_out)
except OSError:
pass
if saved_err is not None:
try:
os.dup2(saved_err, 2)
except OSError as e:
logger.debug("_suppress_output: failed to restore stderr fd: %s", e)
finally:
try:
os.close(saved_err)
except OSError:
pass
if devnull_fd is not None:
try:
os.close(devnull_fd)
os.close(saved_out)
os.close(saved_err)
except OSError:
pass
# Lazy-loaded singleton
@@ -82,7 +107,6 @@ def get_insightface_app():
global _insightface_app, _insightface_loaded
if _insightface_loaded:
return _insightface_app
_insightface_loaded = True
ctx_id = -1
insightface_home = os.environ.get("INSIGHTFACE_HOME", os.path.expanduser("~/.insightface"))
@@ -147,10 +171,12 @@ def get_insightface_app():
_insightface_app.prepare(ctx_id=ctx_id, det_size=(640, 640))
logger.info("InsightFace Buffalo_L: ready on %s (%.1fs)", device_str, time.time() - t0)
_insightface_loaded = True
return _insightface_app
except ImportError:
logger.error("InsightFace not installed!")
_insightface_loaded = True
return None
except Exception as e:
logger.error("Failed to load InsightFace: %s", e)
@@ -168,9 +194,11 @@ def get_insightface_app():
)
_insightface_app.prepare(ctx_id=-1, det_size=(640, 640))
logger.info("InsightFace Buffalo_L: ready on CPU (fallback, %.1fs)", time.time() - t0)
_insightface_loaded = True
return _insightface_app
except Exception as ex:
logger.error("InsightFace CPU fallback failed: %s", ex)
_insightface_loaded = True
return None
@@ -181,8 +209,9 @@ def get_face_embedding(img_pil: Image.Image) -> np.ndarray | None:
return None
try:
# InsightFace expects BGR cv2 image
img_bgr = cv2.cvtColor(np.asarray(img_pil), cv2.COLOR_RGB2BGR)
# InsightFace expects BGR cv2 image; normalise mode first so RGBA/grayscale don't
# raise a channel-count error inside cvtColor.
img_bgr = cv2.cvtColor(np.asarray(img_pil.convert("RGB")), cv2.COLOR_RGB2BGR)
# Suppress scikit-image FutureWarning from InsightFace's face_align.py
with warnings.catch_warnings():
@@ -192,9 +221,14 @@ def get_face_embedding(img_pil: Image.Image) -> np.ndarray | None:
if not faces:
return None
# Return embedding of largest face
largest = max(faces, key=lambda f: (f.bbox[2] - f.bbox[0]) * (f.bbox[3] - f.bbox[1]))
return largest.embedding
# Return embedding of the face nearest the crop centre; a large margin can pull
# a bigger neighbouring face into frame, and max-by-area would pick the wrong person.
cx, cy = img_pil.width / 2, img_pil.height / 2
nearest = min(
faces,
key=lambda f: ((f.bbox[0] + f.bbox[2]) / 2 - cx) ** 2 + ((f.bbox[1] + f.bbox[3]) / 2 - cy) ** 2,
)
return nearest.embedding
except Exception as e:
logger.error("Error getting face embedding: %s", e)
return None
+263 -208
View File
@@ -1,6 +1,7 @@
"""Execution phase: image processing and Frigate upload."""
import logging
import operator
import os
import shutil
from io import BytesIO
@@ -27,6 +28,10 @@ from .log_config import console
from .quality import blur_score_from_image
from .reconcile import enrich_asset_with_face_data, reconcile_frigate_mappings
from .upload_tracker import (
REJECT_TRACKER_FILE,
UPLOAD_TRACKER_FILE,
begin_batch,
flush_batch,
get_lowest_quality_mapped_file,
get_most_redundant_mapped_file,
get_tracked_frigate_file_count,
@@ -181,12 +186,11 @@ def execute_jobs(jobs: list[dict]) -> None:
saved = process_face_mode(
img, asset, person, person_dir, count, insightface_app=insightface_app
)
if saved:
if isinstance(saved, tuple):
filename = f"{count}.jpg"
asset_map[filename] = asset["id"]
score_map[filename] = asset.get("quality_score")
if isinstance(saved, tuple):
dims_map[filename] = saved
dims_map[filename] = saved
# Time-spread path: compute blur score from the downloaded
# image. Capped at 1440px via blur_score_from_image() so the
# scale matches the preview thumbnails the embedding path uses
@@ -197,8 +201,9 @@ def execute_jobs(jobs: list[dict]) -> None:
count += 1
else:
reason = saved if isinstance(saved, str) else "no usable face data"
progress.console.print(
f"[yellow]Skipped {asset['id']} (no usable face data)[/yellow]"
f"[yellow]Skipped {asset['id']} ({reason})[/yellow]"
)
except Exception as e:
logger.error("Failed to process asset %s: %s", asset.get("id", "<unknown>"), e)
@@ -353,242 +358,292 @@ def upload_to_frigate(jobs: list[dict]) -> None:
" (file(s) no longer in Frigate)[/dim]"
)
effective_count = get_tracked_frigate_file_count(name)
pre_run_count = effective_count
quality_replacement = job.get("config", {}).get("quality_replacement", False)
if Config.ENABLE_FRIGATE_SCORES and pre_run_count == 0:
if Config.ENABLE_FRIGATE_SCORES and effective_count == 0:
progress.console.print(
f" [dim]{name}: first run — Frigate diversity scoring will apply from the next run[/dim]"
)
# Snapshot whether Frigate has a model before the upload loop starts.
# effective_count is incremented inside the loop on each successful upload,
# so using the live value would incorrectly trigger recognize_face calls
# mid-batch on the first run (after the first upload sets it to 1).
has_frigate_model = effective_count > 0
actually_uploaded: list[tuple[str, str | None]] = []
failed_deletes: set[str] = set()
min_quality_score_for_slot: float | None = None
person_has_fscores: bool = has_frigate_scores(name)
for fname in person_files:
fpath = os.path.join(person_dir, fname)
begin_batch(UPLOAD_TRACKER_FILE)
begin_batch(REJECT_TRACKER_FILE)
try:
for fname in person_files:
fpath = os.path.join(person_dir, fname)
# If a previous replacement delete succeeded but that upload failed,
# require the next candidate to beat the deleted file's score so the
# freed slot isn't filled with something worse than what we removed.
if min_quality_score_for_slot is not None:
file_score = score_map.get(fname)
if file_score is not None and file_score <= min_quality_score_for_slot:
progress.console.print(
f" [dim]⏭ {fname}: score {file_score:.3f} ≤ freed slot floor"
f" {min_quality_score_for_slot:.3f}, skipping[/dim]"
)
progress.advance(upload_task)
continue
# If a previous replacement delete succeeded but that upload failed,
# require the next candidate to beat the deleted file's score so the
# freed slot isn't filled with something worse than what we removed.
if min_quality_score_for_slot is not None:
file_score = score_map.get(fname)
if file_score is not None and file_score < min_quality_score_for_slot:
progress.console.print(
f" [dim]⏭ {fname}: score {file_score:.3f} < freed slot floor"
f" {min_quality_score_for_slot:.3f}, skipping[/dim]"
)
progress.advance(upload_task)
continue
at_cap = effective_count >= Config.MAX_AUTO_IMAGES
at_cap = effective_count >= Config.MAX_AUTO_IMAGES
# Pre-upload Frigate score — clean measurement (image not yet in training set).
# Called for all below-cap uploads (seeds frigate_scores for future at-cap
# replacement) and for at-cap uploads when scores already exist. Skipped on
# the first run (pre_run_count == 0) since Frigate has no model yet.
# recognize_face returns (face_name, score); we only use the score when the
# best match is for the correct person. Mismatches (or "unknown") are treated
# as None so a wrong-person score never drives a ceiling skip or replacement.
# Frigate rebuilds its model asynchronously after any delete (clear + background
# thread), so the first recognize call after a deletion returns None — our code
# handles this conservatively by skipping that candidate until the next run.
# LIMITATION — async rebuild during multi-replacement runs: each deletion in a
# single run triggers a background model rebuild in Frigate. Subsequent recognize
# calls in the same run may get None (rebuild in progress), causing later
# candidates to fall back to blur-score replacement or be skipped entirely.
# The more replacements that happen in one run, the worse the scoring gets.
# TODO(frigate-api): if Frigate exposes a model generation counter or a
# rebuild-complete signal, poll it between recognize calls during replacement
# sequences rather than accepting stale/None scores.
pre_fscore: float | None = None
if Config.ENABLE_FRIGATE_SCORES and pre_run_count > 0:
if not at_cap or person_has_fscores:
_result = recognize_face(fpath)
if _result is not None and (_result[0] or "").casefold() == name.casefold():
pre_fscore = _result[1]
# Pre-upload Frigate score — clean measurement (image not yet in training set).
# Called for all below-cap uploads (seeds frigate_scores for future at-cap
# replacement) and for at-cap uploads when scores already exist.
# Skipped when has_frigate_model is False (effective_count was 0 before the loop).
# recognize_face returns (face_name, score); we only use the score when the
# best match is for the correct person. Mismatches (or "unknown") are treated
# as None so a wrong-person score never drives a ceiling skip or replacement.
# Frigate rebuilds its model asynchronously after any delete (clear + background
# thread), so the first recognize call after a deletion returns None — our code
# handles this conservatively by skipping that candidate until the next run.
# LIMITATION — async rebuild during multi-replacement runs: each deletion in a
# single run triggers a background model rebuild in Frigate. Subsequent recognize
# calls in the same run may get None (rebuild in progress), causing later
# candidates to fall back to blur-score replacement or be skipped entirely.
# The more replacements that happen in one run, the worse the scoring gets.
# TODO(frigate-api): if Frigate exposes a model generation counter or a
# rebuild-complete signal, poll it between recognize calls during replacement
# sequences rather than accepting stale/None scores.
pre_fscore: float | None = None
if Config.ENABLE_FRIGATE_SCORES and has_frigate_model:
if not at_cap or person_has_fscores:
_result = recognize_face(fpath)
if _result is not None and (_result[0] or "").casefold() == name.casefold():
pre_fscore = _result[1]
# Below-cap novelty gate: skip candidates already covered by the Frigate model,
# including conditions learned from manually-added images winnow can't track.
# pre_fscore is None on the first run (pre_run_count == 0 skips recognize_face
# above), so this block never fires on the first run without an extra guard.
if not at_cap and pre_fscore is not None:
_ceiling = Config.FRIGATE_SCORE_CEILING
if _ceiling is None:
# Dynamic default: bar = most-redundant tracked file's Frigate score.
# Falls back to uploading freely when no tracked scores exist yet.
_bar = get_most_redundant_mapped_file(name)
_skip = _bar is not None and pre_fscore > _bar[2]
_bar_str = f"most redundant tracked {_bar[2]:.2f}" if _bar else ""
elif _ceiling == 0.0:
_skip = False # explicitly disabled
_bar_str = ""
else:
_skip = pre_fscore > _ceiling
_bar_str = f"ceiling {_ceiling:.2f}"
if _skip:
progress.console.print(
f" [dim]⏭ {fname}: Frigate score {pre_fscore:.2f}"
f" > {_bar_str}, already covered[/dim]"
)
progress.advance(upload_task)
continue
# Below-cap novelty gate: skip candidates already covered by the Frigate model,
# including conditions learned from manually-added images winnow can't track.
# pre_fscore is None when effective_count == 0 (no Frigate model yet),
# so this block never fires on the first run without an extra guard.
if not at_cap and pre_fscore is not None:
_ceiling = Config.FRIGATE_SCORE_CEILING
if _ceiling is None:
# Dynamic default: bar = most-redundant tracked file's Frigate score.
# Falls back to uploading freely when no tracked scores exist yet.
_bar = get_most_redundant_mapped_file(name)
_skip = _bar is not None and pre_fscore > _bar[2]
_bar_str = f"most redundant tracked {_bar[2]:.2f}" if _bar else ""
elif _ceiling == 0.0:
_skip = False # explicitly disabled
_bar_str = ""
else:
_skip = pre_fscore > _ceiling
_bar_str = f"ceiling {_ceiling:.2f}"
if _skip:
progress.console.print(
f" [dim]⏭ {fname}: Frigate score {pre_fscore:.2f}"
f" > {_bar_str}, already covered[/dim]"
)
progress.advance(upload_task)
continue
if at_cap:
if not quality_replacement:
progress.console.print(f" [dim]⏭ {fname}: at cap, quality replacement disabled[/dim]")
progress.advance(upload_task)
continue
if at_cap:
if not quality_replacement:
progress.console.print(f" [dim]⏭ {fname}: at cap, quality replacement disabled[/dim]")
progress.advance(upload_task)
continue
using_fscore = person_has_fscores and Config.ENABLE_FRIGATE_SCORES
if using_fscore:
candidate_score = pre_fscore
get_target = get_most_redundant_mapped_file
score_label, better_note = "frigate", " (more novel)"
no_score_msg = "Frigate recognize unavailable, skipping replacement"
is_better_than = lambda c, t: c < t
else:
candidate_score = score_map.get(fname)
get_target = get_lowest_quality_mapped_file
score_label, better_note = "blur", ""
no_score_msg = "no quality score, skipping replacement"
is_better_than = lambda c, t: c > t
using_fscore = person_has_fscores and Config.ENABLE_FRIGATE_SCORES
if using_fscore:
candidate_score = pre_fscore
get_target = get_most_redundant_mapped_file
score_label, better_note = "frigate", " (more novel)"
no_score_msg = "Frigate recognize unavailable, skipping replacement"
is_better_than = operator.lt
else:
candidate_score = score_map.get(fname)
get_target = get_lowest_quality_mapped_file
score_label, better_note = "blur", ""
no_score_msg = "no quality score, skipping replacement"
is_better_than = operator.gt
if candidate_score is None:
progress.console.print(f" [dim]⏭ {fname}: {no_score_msg}[/dim]")
progress.advance(upload_task)
continue
if candidate_score is None:
progress.console.print(f" [dim]⏭ {fname}: {no_score_msg}[/dim]")
progress.advance(upload_task)
continue
target = get_target(name, exclude=failed_deletes)
not_better = target is None or not is_better_than(candidate_score, target[2])
if not_better:
target_str = f"{target[2]:.3f}" if target is not None else "N/A"
target = get_target(name, exclude=failed_deletes)
not_better = target is None or not is_better_than(candidate_score, target[2])
if not_better:
target_str = f"{target[2]:.3f}" if target is not None else "N/A"
cmp_op = "<" if using_fscore else ">"
progress.console.print(
f" [dim]⏭ {fname}: {score_label} {candidate_score:.3f}"
f" not {cmp_op} {target_str}, skipping[/dim]"
)
progress.advance(upload_task)
continue
target_frigate_file, _target_asset_id, target_score = target
cmp_op = "<" if using_fscore else ">"
progress.console.print(
f" [dim]⏭ {fname}: {score_label} {candidate_score:.3f}"
f" not {cmp_op} {target_str}, skipping[/dim]"
f" 🔄 {fname}: {score_label} {candidate_score:.3f} {cmp_op} {target_score:.3f},"
f" replacing {target_frigate_file}{better_note}"
)
progress.advance(upload_task)
continue
target_frigate_file, _target_asset_id, target_score = target
cmp_op = "<" if using_fscore else ">"
progress.console.print(
f" 🔄 {fname}: {score_label} {candidate_score:.3f} {cmp_op} {target_score:.3f},"
f" replacing {target_frigate_file}{better_note}"
)
if delete_frigate_person_files(name, [target_frigate_file]):
remove_frigate_file(name, target_frigate_file)
effective_count -= 1
min_quality_score_for_slot = None if using_fscore else target_score
else:
logger.warning("Failed to delete %s for %s, skipping replacement", target_frigate_file, name)
failed_deletes.add(target_frigate_file)
progress.advance(upload_task)
continue
for attempt in range(1, max_retries + 1):
try:
with open(fpath, "rb") as f:
resp = requests.post(
f"{frigate_url}/api/faces/{encoded_name}/register",
files={"file": (fname, f, "image/jpeg")},
timeout=30,
)
if resp.status_code == 200:
uploaded += 1
person_uploaded += 1
effective_count += 1
min_quality_score_for_slot = None
asset_id = asset_map.get(fname)
if asset_id:
try:
mark_uploaded(
asset_id,
person_name=name,
score=score_map.get(fname),
crop_dims=dims_map.get(fname),
frigate_score=pre_fscore,
)
except Exception as tracker_exc:
# Upload to Frigate succeeded — don't retry on tracker
# failure or we'd upload a duplicate to Frigate.
logger.error(
"Tracker write failed for %s — upload succeeded"
" but asset may be re-selected next run: %s",
fname, tracker_exc,
)
if pre_fscore is not None:
person_has_fscores = True
actually_uploaded.append((fname, asset_id))
break
if delete_frigate_person_files(name, [target_frigate_file]):
remove_frigate_file(name, target_frigate_file)
person_has_fscores = has_frigate_scores(name)
effective_count -= 1
min_quality_score_for_slot = None if using_fscore else target_score
else:
logger.warning(
"Failed to delete %s for %s, skipping replacement",
target_frigate_file, name,
)
failed_deletes.add(target_frigate_file)
progress.advance(upload_task)
continue
for attempt in range(1, max_retries + 1):
try:
with open(fpath, "rb") as f:
resp = requests.post(
f"{frigate_url}/api/faces/{encoded_name}/register",
files={"file": (fname, f, "image/jpeg")},
timeout=30,
)
if resp.status_code == 200:
uploaded += 1
person_uploaded += 1
effective_count += 1
min_quality_score_for_slot = None # for/else rollback mirrors this pair
asset_id = asset_map.get(fname)
if asset_id:
try:
mark_uploaded(
asset_id,
person_name=name,
score=score_map.get(fname),
crop_dims=dims_map.get(fname),
frigate_score=pre_fscore,
)
except Exception as tracker_exc:
# Upload to Frigate succeeded — don't retry on tracker
# failure or we'd upload a duplicate to Frigate.
logger.error(
"Tracker write failed for %s — upload succeeded"
" but asset may be re-selected next run: %s",
fname, tracker_exc,
)
else:
if pre_fscore is not None:
person_has_fscores = True
# Always record for reconcile so the Frigate filename→asset_id
# mapping is created even when the tracker write fails.
# Trade-off: if mark_uploaded failed, asset_id is absent from
# asset_ids and scores. Consequences: (1) re-selected next run
# → Frigate duplicate; (2) excluded from quality-replacement
# candidates (_pick_mapped_file requires a scores entry);
# (3) counted toward MAX_AUTO_IMAGES cap (via frigate_files).
# The alternative — not appending — leaves the file permanently
# unmapped (reconcile never creates the frigate_files entry),
# making (2) and (3) permanent. Frigate duplicate is lesser.
actually_uploaded.append((fname, asset_id))
break
else:
if attempt < max_retries:
logger.warning(
f"Upload attempt {attempt}/{max_retries} for {fname}:"
f" HTTP {resp.status_code}, retrying..."
)
continue
failed += 1
person_failed += 1
progress.console.print(
f" [red]✗ {fname}: HTTP {resp.status_code} (after {max_retries} attempts)[/red]"
)
full_body = resp.text
try:
error_detail = resp.json().get("message", full_body[:100])
except Exception:
error_detail = full_body[:100]
if resp.status_code in (400, 500):
progress.console.print(f" [dim]{error_detail}[/dim]")
else:
logger.debug("%s HTTP %s: %s", fname, resp.status_code, error_detail)
_is_permanent = (
(resp.status_code == 400 and "face" in full_body.lower())
or resp.status_code == 422
or (resp.status_code == 500 and "could not process" in full_body.lower())
)
if _is_permanent:
asset_id = asset_map.get(fname)
if asset_id:
mark_rejected(asset_id, person_name=name)
except (requests.exceptions.ConnectionError, requests.exceptions.Timeout) as exc:
if attempt < max_retries:
logger.warning(
f"Upload attempt {attempt}/{max_retries} for {fname}:"
f" HTTP {resp.status_code}, retrying..."
f" {type(exc).__name__}, retrying..."
)
continue
failed += 1
person_failed += 1
label = (
"Connection refused"
if isinstance(exc, requests.exceptions.ConnectionError)
else "Request timed out (30s)"
)
progress.console.print(
f" [red]✗ {fname}: {label} (after {max_retries} attempts)[/red]"
)
except Exception as e:
if attempt < max_retries:
logger.warning(
f"Upload attempt {attempt}/{max_retries} for {fname}:"
f" {type(e).__name__}, retrying..."
)
continue
failed += 1
person_failed += 1
progress.console.print(
f" [red]✗ {fname}: HTTP {resp.status_code} (after {max_retries} attempts)[/red]"
f" [red]✗ {fname}: {type(e).__name__} - {e} (after {max_retries} attempts)[/red]"
)
full_body = resp.text
try:
error_detail = resp.json().get("message", full_body[:100])
except Exception:
error_detail = full_body[:100]
if resp.status_code == 400:
progress.console.print(f" [dim]{error_detail}[/dim]")
else:
logger.debug("%s HTTP %s: %s", fname, resp.status_code, error_detail)
_is_permanent = (
(resp.status_code == 400 and "face" in full_body.lower())
or resp.status_code == 422
)
if _is_permanent:
asset_id = asset_map.get(fname)
if asset_id:
mark_rejected(asset_id, person_name=name)
except (requests.exceptions.ConnectionError, requests.exceptions.Timeout) as exc:
if attempt < max_retries:
logger.warning(
f"Upload attempt {attempt}/{max_retries} for {fname}:"
f" {type(exc).__name__}, retrying..."
)
continue
failed += 1
person_failed += 1
label = (
"Connection refused"
if isinstance(exc, requests.exceptions.ConnectionError)
else "Request timed out (30s)"
)
progress.console.print(
f" [red]✗ {fname}: {label} (after {max_retries} attempts)[/red]"
)
except Exception as e:
if attempt < max_retries:
logger.warning(
f"Upload attempt {attempt}/{max_retries} for {fname}:"
f" {type(e).__name__}, retrying..."
)
continue
failed += 1
person_failed += 1
progress.console.print(
f" [red]✗ {fname}: {type(e).__name__} - {e} (after {max_retries} attempts)[/red]"
)
else:
# All retries exhausted without a successful upload.
# Restore the slot freed by the preceding delete so the next
# candidate still sees at_cap=True and must beat the replacement gate.
# Also clear the quality floor — the deleted file's score no longer
# represents any live Frigate file, and leaving it blocks the next
# candidate from filling the restored slot.
if at_cap:
effective_count += 1
min_quality_score_for_slot = None
progress.advance(upload_task)
progress.advance(upload_task)
if min_quality_score_for_slot is not None:
logger.warning(
f"{name}: freed replacement slot (floor {min_quality_score_for_slot:.3f})"
" was not filled this run — will be available next run"
)
if min_quality_score_for_slot is not None:
logger.warning(
f"{name}: freed replacement slot (floor {min_quality_score_for_slot:.3f})"
" was not filled this run — will be available next run"
)
finally:
try:
flush_batch(UPLOAD_TRACKER_FILE)
except Exception as _flush_exc:
logger.warning(
"flush_batch failed during cleanup"
" — batch will be recovered on next begin_batch: %s",
_flush_exc,
)
try:
flush_batch(REJECT_TRACKER_FILE)
except Exception as _flush_exc:
logger.warning(
"flush_batch failed during cleanup"
" — batch will be recovered on next begin_batch: %s",
_flush_exc,
)
# Batch-map Frigate filenames to asset IDs now that all uploads are done.
if actually_uploaded and not _skip_reconcile:
+3 -1
View File
@@ -146,8 +146,10 @@ def delete_frigate_person_files(person_name: str, filenames: list[str]) -> bool:
Returns True on success, False if unreachable or the request fails.
"""
frigate_url = _get_frigate_url()
if not frigate_url or not filenames:
if not frigate_url:
return False
if not filenames:
return True
from urllib.parse import quote
encoded_name = quote(person_name, safe="")
try:
+15 -9
View File
@@ -48,7 +48,9 @@ def align_face(img: Image.Image, landmarks: list[list[float]] | np.ndarray) -> I
if lm.shape != (5, 2):
logger.debug("Invalid landmark shape: %s, expected (5, 2)", lm.shape)
return None
aligned = norm_crop(img_np, lm)
with warnings.catch_warnings():
warnings.filterwarnings("ignore", message=".*estimate.*is deprecated", category=FutureWarning)
aligned = norm_crop(img_np, lm)
return Image.fromarray(aligned)
except ImportError:
logger.debug("InsightFace not available for face alignment")
@@ -66,10 +68,11 @@ def process_face_mode(
count: int,
min_width: int | None = None,
insightface_app=None,
) -> tuple[int, int] | None:
) -> tuple[int, int] | str:
"""Crop face based on Immich metadata and save to output directory.
Returns (width, height) of the saved crop, or None if no crop was saved.
Returns (width, height) of the saved crop, or a skip-reason string if the
face was filtered out.
When insightface_app is provided and ENABLE_FACE_ALIGNMENT is True,
re-detects the face in the Immich bbox region using InsightFace to get
precise landmarks for a proper 112x112 aligned crop. Falls back to
@@ -89,14 +92,17 @@ def process_face_mode(
if not face_info:
logger.debug("No face info for %s in asset %s", person.get("name"), asset.get("id"))
return None
return "no face metadata"
img_w, img_h = img.size
meta_w = face_info.get("imageWidth") or img_w
meta_h = face_info.get("imageHeight") or img_h
meta_w = face_info.get("imageWidth") or 0
meta_h = face_info.get("imageHeight") or 0
# Scale bounding box to actual image dimensions
scale_x, scale_y = img_w / meta_w, img_h / meta_h
# Scale bounding box from detection-image space to actual image dimensions.
# Fall back to 1.0 if Immich omits the field — bbox is assumed to already
# be in image space (correct for thumbnails, wrong for full-res).
scale_x = img_w / meta_w if meta_w else 1.0
scale_y = img_h / meta_h if meta_h else 1.0
x1 = face_info["boundingBoxX1"] * scale_x
y1 = face_info["boundingBoxY1"] * scale_y
x2 = face_info["boundingBoxX2"] * scale_x
@@ -105,7 +111,7 @@ def process_face_mode(
face_w, face_h = x2 - x1, y2 - y1
if face_w < min_width or face_h < min_width:
logger.debug("Face too small (%.1fx%.1f)", face_w, face_h)
return None
return f"face too small ({face_w:.0f}x{face_h:.0f}px, min {min_width}px)"
# Re-detect face with InsightFace for landmark-based alignment.
# Immich's /api/faces endpoint does not include landmarks, so the
+26 -7
View File
@@ -39,7 +39,11 @@ def get_immich_version() -> tuple[int, int, int] | None:
)
if resp.ok:
data = resp.json()
return (int(data["major"]), int(data["minor"]), int(data["patch"]))
major, minor, patch = data.get("major"), data.get("minor"), data.get("patch")
if major is None or minor is None or patch is None:
logger.debug("Unexpected Immich version schema: %s", data)
return None
return (int(major), int(minor), int(patch))
return None
except Exception:
return None
@@ -57,8 +61,12 @@ def get_people() -> list[dict]:
logger.error("Immich API key is invalid or expired (401 Unauthorized). Update API_KEY.")
return []
resp.raise_for_status()
return resp.json().get("people", [])
except (requests.RequestException, ValueError) as e:
data = resp.json()
if not isinstance(data, dict):
logger.error("Unexpected response shape from Immich /people: %r", type(data))
return []
return data.get("people") or []
except (requests.RequestException, ValueError, AttributeError) as e:
logger.error("Failed to fetch people from Immich: %s", e)
return []
@@ -118,11 +126,15 @@ def fetch_all_assets(person: dict) -> tuple[list[dict], int]:
logger.error("Error fetching assets for %s (page %s): %s", name, page, resp.status_code)
break
page_assets = resp.json().get("assets", [])
body = resp.json()
if not isinstance(body, dict):
logger.error("Unexpected response shape fetching assets for %s (page %s): %r", name, page, type(body))
break
page_assets = body.get("assets", [])
# Immich ≥2.x returns {"assets": {"items": [...]}};
# earlier versions returned {"assets": [...]} directly.
if isinstance(page_assets, dict):
page_assets = page_assets.get("items", [])
page_assets = page_assets.get("items") or []
page_count = len(page_assets) # raw count for termination check before filtering
@@ -277,10 +289,11 @@ def filter_recent_assets(assets: list[dict], years: int | None = None) -> list[d
logger.debug("Filtering assets older than %s years (%s)", years, cutoff)
recent, skipped = [], 0
recent, skipped, bad_timestamp = [], 0, 0
for asset in assets:
created_at_str = asset.get("fileCreatedAt")
if not isinstance(created_at_str, str) or not created_at_str:
bad_timestamp += 1
continue
try:
@@ -290,9 +303,15 @@ def filter_recent_assets(assets: list[dict], years: int | None = None) -> list[d
recent.append(asset)
else:
skipped += 1
except ValueError:
except (ValueError, TypeError):
bad_timestamp += 1
continue
if bad_timestamp:
logger.warning(
"filter_recent_assets: %s asset(s) had missing or unparseable fileCreatedAt"
" and were excluded from the pool.", bad_timestamp
)
logger.debug("Retained %s assets (filtered %s old assets).", len(recent), skipped)
return recent
+34 -14
View File
@@ -65,12 +65,20 @@ def _get_strategy_choice(has_embedding: bool) -> tuple[int | str, str]:
def _resolve_strategy(strategy: str, has_embedding: bool) -> tuple[int | str, str]:
"""Resolve env var strategy to (limit, selection_mode) without prompts."""
if strategy == "skip":
return 0, "skip"
if not has_embedding:
return _getenv_int("LIMIT", 30), "time"
limit = _getenv_int("LIMIT", 30)
if limit <= 0:
logger.warning("LIMIT=%s is invalid — ignoring and using default 30", limit)
limit = 30
return limit, "time"
custom_limit = _getenv_optional_int("LIMIT")
if custom_limit is not None:
return custom_limit, "smart"
if custom_limit > 0:
return custom_limit, "smart"
logger.warning("LIMIT=%s is invalid — ignoring and using auto strategy", custom_limit)
strategy_map = {
"adaptive": ("auto", "smart"),
@@ -78,7 +86,11 @@ def _resolve_strategy(strategy: str, has_embedding: bool) -> tuple[int | str, st
"standard": (30, "smart"),
"broad": (100, "smart"),
}
return strategy_map.get(strategy, ("auto", "smart"))
result = strategy_map.get(strategy)
if result is None:
logger.warning("Unrecognised STRATEGY=%r — falling back to auto", strategy)
return ("auto", "smart")
return result
def _perform_selection(
@@ -184,13 +196,20 @@ def _configure_person(person: dict, people: list[dict]) -> dict | None:
return job
def _valid_people(people: list[dict]) -> list[dict]:
return sorted(
[p for p in people if (p.get("name") or "").strip() and p.get("id")],
key=lambda x: x["name"],
)
def interactive_configure(people: list[dict]) -> list[dict]:
"""Interactive phase: select person(s), mode, and configure training strategy.
Supports multi-person batch mode — after configuring one person,
prompts to add another.
"""
valid_people = sorted([p for p in people if p.get("name")], key=lambda x: x["name"])
valid_people = _valid_people(people)
if not valid_people:
rprint("[red]No people found with names in Immich.[/red]")
@@ -201,9 +220,9 @@ def interactive_configure(people: list[dict]) -> list[dict]:
while True:
# Select person
console.print("\n[bold cyan]Select Person to Train:[/bold cyan]")
queued_ids = {j["person"]["id"] for j in jobs}
for idx, p in enumerate(valid_people, 1):
# Mark already-queued people
marker = " [dim](queued)[/dim]" if any(j["person"]["id"] == p["id"] for j in jobs) else ""
marker = " [dim](queued)[/dim]" if p.get("id") in queued_ids else ""
console.print(f" [bold]{idx}.[/bold] {p['name']}{marker}")
p_choice = IntPrompt.ask("Enter Number", choices=[str(i) for i in range(1, len(valid_people) + 1)])
@@ -222,20 +241,20 @@ def interactive_configure(people: list[dict]) -> list[dict]:
def auto_configure(people: list[dict]) -> list[dict]:
"""Non-interactive: configure jobs for all named people automatically."""
valid_people = sorted([p for p in people if p.get("name")], key=lambda x: x["name"])
valid_people = _valid_people(people)
if not valid_people:
rprint("[red]No people found with names in Immich.[/red]")
return []
strategy = os.environ.get("STRATEGY", "auto")
skip = [s.strip() for s in os.environ.get("SKIP_PEOPLE", "").split(",") if s.strip()]
only = [s.strip() for s in os.environ.get("ONLY_PEOPLE", "").split(",") if s.strip()]
skip = {s.strip().casefold() for s in os.environ.get("SKIP_PEOPLE", "").split(",") if s.strip()}
only = {s.strip().casefold() for s in os.environ.get("ONLY_PEOPLE", "").split(",") if s.strip()}
if only:
valid_people = [p for p in valid_people if p["name"] in only]
valid_people = [p for p in valid_people if p["name"].casefold() in only]
if skip:
valid_people = [p for p in valid_people if p["name"] not in skip]
valid_people = [p for p in valid_people if p["name"].casefold() not in skip]
min_face_count = Config.MIN_FACE_COUNT
@@ -293,10 +312,11 @@ def auto_configure(people: list[dict]) -> list[dict]:
# decides per-image whether to swap; any candidate could be an improvement).
if not quality_replacement_only:
if limit == "auto":
# Switch from open-ended auto to a fixed budget at remaining capacity
# so the diversity selector itself stops at the right count instead of
# selecting MAX_AUTO_IMAGES and then discarding the excess by position.
if already_uploaded > 0:
# Switch from open-ended auto to a fixed budget at remaining capacity
# so the diversity selector stops at the right count instead of
# selecting more than MAX_AUTO_IMAGES and overflowing the cap.
# First runs keep limit="auto" so FPS adaptive early-stop can fire.
limit = capacity
else:
limit = min(limit, capacity)
+10 -6
View File
@@ -14,6 +14,11 @@ from PIL import Image
logger = logging.getLogger(__name__)
def _laplacian_var(img_np: np.ndarray) -> float:
gray = cv2.cvtColor(img_np, cv2.COLOR_RGB2GRAY) if img_np.ndim == 3 else img_np
return float(cv2.Laplacian(gray, cv2.CV_64F).var())
@dataclass
class QualityResult:
"""Result of quality assessment on a face/image crop."""
@@ -32,8 +37,7 @@ def check_blur(img_np: np.ndarray, threshold: float = 100.0) -> tuple[bool, str]
Lower variance = blurrier image. ArcFace needs clear facial features.
"""
gray = cv2.cvtColor(img_np, cv2.COLOR_RGB2GRAY) if img_np.ndim == 3 else img_np
variance = cv2.Laplacian(gray, cv2.CV_64F).var()
variance = _laplacian_var(img_np)
if variance < threshold:
return False, f"Blurry (laplacian={variance:.1f}, threshold={threshold})"
return True, ""
@@ -115,8 +119,7 @@ def assess_quality(
reasons = []
# Compute laplacian variance once (used by check_blur and stored as blur_score)
gray = cv2.cvtColor(img_np, cv2.COLOR_RGB2GRAY) if img_np.ndim == 3 else img_np
blur_score = float(cv2.Laplacian(gray, cv2.CV_64F).var())
blur_score = _laplacian_var(img_np)
checks = [
(
@@ -152,9 +155,10 @@ def blur_score_from_image(img: Image.Image, max_dim: int = 1440) -> float | None
try:
score_img = img.convert("RGB") if img.mode != "RGB" else img
if score_img.width > max_dim or score_img.height > max_dim:
score_img = score_img.copy()
if score_img is img:
score_img = score_img.copy()
score_img.thumbnail((max_dim, max_dim), Image.LANCZOS)
return float(assess_quality(score_img).blur_score)
return _laplacian_var(np.array(score_img))
except Exception as exc:
logger.debug("blur_score_from_image failed: %s", exc)
return None
+1 -1
View File
@@ -61,7 +61,7 @@ def reconcile_frigate_mappings(
try:
return float(fname.rsplit("_", 1)[-1].rsplit(".", 1)[0])
except (ValueError, IndexError):
return 0.0
return float("inf")
logger.debug(
"%s: mapping %s file(s) by filename timestamp — assumes Frigate processes"
+142 -52
View File
@@ -32,7 +32,7 @@ import logging
import os
from pathlib import Path
from .frigate_api import delete_frigate_person_files
from .frigate_api import _get_frigate_url, delete_frigate_person_files
logger = logging.getLogger(__name__)
@@ -43,6 +43,8 @@ REJECT_TRACKER_FILE = "frigate_rejected_ids.json"
# Reduces per-call JSON reads from O(calls) to O(1) after the first load.
# Keyed by full path so tests with isolated tmp dirs never share entries.
_cache: dict[str, dict] = {}
_deferred: set[str] = set() # paths whose disk writes are batched until flush_batch()
_dirty: set[str] = set() # deferred paths that received at least one _save during the batch
def _tracker_path(filename: str) -> Path:
@@ -69,28 +71,68 @@ def _load(filename: str) -> dict:
return data
def _save(filename: str, data: dict) -> None:
path = _tracker_path(filename)
def _write_to_disk(path: Path, data: dict) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
tmp = path.with_suffix(".tmp")
try:
with open(tmp, "w") as f:
json.dump(data, f, indent=2)
os.replace(tmp, path)
_cache[str(path)] = data # update only after the file is safely on disk
except Exception:
tmp.unlink(missing_ok=True)
raise
def _save(filename: str, data: dict) -> None:
path = _tracker_path(filename)
key = str(path)
if key in _deferred:
_cache[key] = data # accumulate in cache; disk write deferred until flush_batch()
_dirty.add(key)
return
_write_to_disk(path, data)
_cache[key] = data # update cache only after successful write
def begin_batch(filename: str) -> None:
"""Defer tracker disk writes for filename. All _save calls accumulate in the
in-memory cache until flush_batch() is called. Use around per-person upload loops
to reduce N writes to 1.
If a previous batch for this file was interrupted before flush_batch() was called
(e.g. an exception escaped the upload loop), the leftover cache state is flushed
to disk here before starting fresh so that partial progress is not silently lost.
"""
path = _tracker_path(filename)
key = str(path)
if key in _deferred and key in _dirty:
try:
_write_to_disk(path, _cache[key])
except Exception:
logger.warning(
"begin_batch: could not flush leftover deferred state for %s"
" — partial progress may be lost",
path,
)
_deferred.discard(key)
_dirty.discard(key)
_deferred.add(key)
def flush_batch(filename: str) -> None:
"""Write the accumulated cache state for filename to disk."""
path = _tracker_path(filename)
key = str(path)
if key in _dirty and key in _cache:
_write_to_disk(path, _cache[key])
_deferred.discard(key)
_dirty.discard(key)
def _flat_key(filename: str) -> str:
return "uploaded_asset_ids" if filename == UPLOAD_TRACKER_FILE else "rejected_asset_ids"
def _load_flat(filename: str) -> set[str]:
return set(_load(filename).get(_flat_key(filename), []))
def _get_ids(entry: list | dict) -> list[str]:
"""Extract asset_ids from either the old list format or the new dict format."""
if isinstance(entry, list):
@@ -120,35 +162,46 @@ def _mark(
crop_dims: tuple[int, int] | None = None,
frigate_score: float | None = None,
) -> None:
if not person_name:
logger.warning("_mark called with empty person_name for asset %s — asset not recorded", asset_id)
return
data = _load(filename)
flat_key = _flat_key(filename)
flat = set(data.get(flat_key, []))
flat.add(asset_id)
data[flat_key] = sorted(flat)
if person_name:
by_person = data.setdefault("by_person", {})
entry = _migrate_entry(by_person.get(person_name, {}))
ids = set(entry["asset_ids"])
ids.add(asset_id)
entry["asset_ids"] = sorted(ids)
if score is not None:
entry["scores"][asset_id] = round(score, 4)
if crop_dims is not None:
entry["crop_dims"][asset_id] = [crop_dims[0], crop_dims[1]]
if frigate_score is not None:
entry["frigate_scores"][asset_id] = round(frigate_score, 4)
by_person[person_name] = entry
_save(filename, data)
by_person = dict(data.get("by_person", {}))
entry = _migrate_entry(by_person.get(person_name, {}))
ids = set(entry["asset_ids"])
ids.add(asset_id)
entry["asset_ids"] = sorted(ids)
if score is not None:
entry["scores"][asset_id] = round(score, 4)
if crop_dims is not None:
entry["crop_dims"][asset_id] = [crop_dims[0], crop_dims[1]]
if frigate_score is not None:
entry["frigate_scores"][asset_id] = round(frigate_score, 4)
by_person[person_name] = entry
new_data = dict(data)
new_data["by_person"] = by_person
_save(filename, new_data)
logger.debug("Marked %s in %s (%s)", asset_id, filename, person_name)
# ── Public API ────────────────────────────────────────────────────────────────
def load_uploaded_ids() -> set[str]:
return _load_flat(UPLOAD_TRACKER_FILE)
"""Return all asset IDs recorded as uploaded. Derives from by_person (primary)
plus any legacy flat list still present in old tracker files."""
data = _load(UPLOAD_TRACKER_FILE)
ids = {aid for e in data.get("by_person", {}).values() for aid in _get_ids(e)}
ids.update(data.get("uploaded_asset_ids", [])) # backward compat with pre-0.6.1 files
return ids
def load_rejected_ids() -> set[str]:
return _load_flat(REJECT_TRACKER_FILE)
"""Return all asset IDs recorded as rejected. Derives from by_person (primary)
plus any legacy flat list still present in old tracker files."""
data = _load(REJECT_TRACKER_FILE)
ids = {aid for e in data.get("by_person", {}).values() for aid in _get_ids(e)}
ids.update(data.get("rejected_asset_ids", [])) # backward compat with pre-0.6.1 files
return ids
def mark_uploaded(
@@ -159,12 +212,10 @@ def mark_uploaded(
frigate_score: float | None = None,
) -> None:
_mark(UPLOAD_TRACKER_FILE, asset_id, person_name, score=score, crop_dims=crop_dims, frigate_score=frigate_score)
logger.debug(f"Marked {asset_id} as uploaded ({person_name})")
def mark_rejected(asset_id: str, person_name: str | None = None) -> None:
_mark(REJECT_TRACKER_FILE, asset_id, person_name)
logger.debug(f"Marked {asset_id} as rejected ({person_name})")
@@ -178,11 +229,13 @@ def record_frigate_files_batch(person_name: str, mappings: dict[str, str]) -> No
"""Record multiple Frigate filename → asset_id mappings in a single load/save."""
if not mappings:
return
data = _load(UPLOAD_TRACKER_FILE)
by_person = data.setdefault("by_person", {})
src = _load(UPLOAD_TRACKER_FILE)
by_person = dict(src.get("by_person", {}))
entry = _migrate_entry(by_person.get(person_name, {}))
entry["frigate_files"].update(mappings)
by_person[person_name] = entry
data = dict(src)
data["by_person"] = by_person
_save(UPLOAD_TRACKER_FILE, data)
logger.debug(f"Batch-mapped {len(mappings)} Frigate file(s) for {person_name}")
@@ -198,17 +251,19 @@ def remove_frigate_file(person_name: str, frigate_filename: str) -> None:
def remove_frigate_files_batch(person_name: str, frigate_filenames: list[str]) -> None:
"""Remove multiple Frigate filenames in a single load/save."""
data = _load(UPLOAD_TRACKER_FILE)
by_person = data.get("by_person", {})
raw = by_person.get(person_name)
src = _load(UPLOAD_TRACKER_FILE)
raw = src.get("by_person", {}).get(person_name)
if raw is None:
return
entry = _migrate_entry(raw)
for fn in frigate_filenames:
asset_id = entry["frigate_files"].pop(fn, None)
if asset_id:
if asset_id is not None and asset_id not in entry["frigate_files"].values():
entry["frigate_scores"].pop(asset_id, None)
by_person = dict(src.get("by_person", {})) # copy so assignment does not mutate the cache
by_person[person_name] = entry
data = dict(src)
data["by_person"] = by_person
_save(UPLOAD_TRACKER_FILE, data)
logger.debug(f"Removed {len(frigate_filenames)} Frigate file mapping(s) for {person_name}")
@@ -238,9 +293,11 @@ def get_tracked_frigate_filenames(person_name: str) -> set[str]:
def has_frigate_scores(person_name: str) -> bool:
"""Return True if any mapped file for this person has a stored Frigate recognition score."""
data = _load(UPLOAD_TRACKER_FILE)
entry = _migrate_entry(data.get("by_person", {}).get(person_name, {}))
frigate_files = entry.get("frigate_files", {})
frigate_scores = entry.get("frigate_scores", {})
raw = data.get("by_person", {}).get(person_name)
if not raw or isinstance(raw, list):
return False
frigate_files = raw.get("frigate_files", {})
frigate_scores = raw.get("frigate_scores", {})
return any(asset_id in frigate_scores for asset_id in frigate_files.values())
@@ -303,6 +360,8 @@ def find_by_crop_dimension(size: int) -> list[dict]:
asset_to_frigate.setdefault(aid, fn) # first-seen wins; plain inversion silently drops duplicates
frigate_scores = entry.get("frigate_scores", {})
for asset_id, dims in entry.get("crop_dims", {}).items():
if not isinstance(dims, (list, tuple)) or len(dims) < 2:
continue
w, h = dims[0], dims[1]
if w == size or h == size:
results.append({
@@ -320,11 +379,38 @@ def find_by_crop_dimension(size: int) -> list[dict]:
def update_frigate_count(person_name: str, count: int) -> None:
"""Record Frigate's authoritative training image count for a person."""
data = _load(UPLOAD_TRACKER_FILE)
by_person = data.setdefault("by_person", {})
by_person = dict(data.get("by_person", {}))
entry = _migrate_entry(by_person.get(person_name, {}))
entry["frigate_count"] = count
by_person[person_name] = entry
_save(UPLOAD_TRACKER_FILE, data)
new_data = dict(data)
new_data["by_person"] = by_person
_save(UPLOAD_TRACKER_FILE, new_data)
def reset_all_people() -> None:
"""Reset all tracking data in two writes (O(P) Frigate API calls, O(1) disk writes).
Preferred over calling reset_person() in a loop when RESET_PERSON=* — that
approach is O(P²) because each call rebuilds the flat list from all remaining entries.
"""
upload_data = _load(UPLOAD_TRACKER_FILE)
frigate_url = _get_frigate_url()
if not frigate_url:
logger.info("FRIGATE_URL not set — skipping Frigate file deletion")
for person_name, raw_entry in upload_data.get("by_person", {}).items():
entry = _migrate_entry(raw_entry)
frigate_filenames = list(entry.get("frigate_files", {}).keys())
if not frigate_filenames:
continue
if frigate_url:
if delete_frigate_person_files(person_name, frigate_filenames):
logger.info(f"Deleted {len(frigate_filenames)} Frigate file(s) for {person_name}")
else:
logger.warning(f"Could not delete Frigate files for {person_name} — tracker reset proceeding anyway")
_save(UPLOAD_TRACKER_FILE, {})
_save(REJECT_TRACKER_FILE, {})
logger.info("Reset all tracking data")
def reset_person(person_name: str) -> None:
@@ -339,7 +425,7 @@ def reset_person(person_name: str) -> None:
entry = _migrate_entry(upload_data.get("by_person", {}).get(person_name, {}))
frigate_filenames = list(entry.get("frigate_files", {}).keys())
if frigate_filenames:
if not os.environ.get("FRIGATE_URL", "").strip():
if not _get_frigate_url():
logger.info(f"FRIGATE_URL not set — skipping Frigate file deletion for {person_name}")
elif delete_frigate_person_files(person_name, frigate_filenames):
logger.info(f"Deleted {len(frigate_filenames)} Frigate file(s) for {person_name}")
@@ -347,19 +433,23 @@ def reset_person(person_name: str) -> None:
logger.warning(f"Could not delete Frigate files for {person_name} — tracker reset proceeding anyway")
changed = False
tracker_files = ((UPLOAD_TRACKER_FILE, upload_data), (REJECT_TRACKER_FILE, _load(REJECT_TRACKER_FILE)))
for filename, data in tracker_files:
flat_key = _flat_key(filename)
by_person = data.get("by_person", {})
for filename in (UPLOAD_TRACKER_FILE, REJECT_TRACKER_FILE):
src = upload_data if filename == UPLOAD_TRACKER_FILE else _load(REJECT_TRACKER_FILE)
by_person = dict(src.get("by_person", {})) # copy so pop() does not mutate the cache
tracker_entry = by_person.pop(person_name, None)
if tracker_entry is not None:
# Rebuild from remaining entries rather than subtracting, so IDs that
# appear under another person aren't incorrectly removed from the flat list.
remaining_ids: set[str] = set()
for other_entry in by_person.values():
remaining_ids.update(_get_ids(other_entry))
data[flat_key] = sorted(remaining_ids)
data = dict(src)
data["by_person"] = by_person
flat_key = _flat_key(filename)
person_ids = set(_get_ids(tracker_entry))
if person_ids and flat_key in data and not isinstance(data[flat_key], list):
logger.warning(
"reset_person: %s has unexpected type for %s (%s) — skipping flat-list cleanup;"
" all persons' legacy IDs in this field are unaffected but unreadable",
filename, flat_key, type(data[flat_key]).__name__,
)
elif person_ids and flat_key in data:
data[flat_key] = sorted(set(data[flat_key]) - person_ids)
_save(filename, data)
changed = True
if changed: