- _getenv_num: add raw.strip() + empty-string guard so numeric vars set
to "" (common Compose pattern for "use default") return the default
silently instead of warning "not a valid int/float"
- _getenv_bool: same guard so True-defaulted flags set to "" return
the configured default instead of silently returning False
- embeddings.py: replace inline FORCE_CPU bool parse with _getenv_bool
- cache.py: fix np.save extension bug from v0.5.13 — tmp path used
final+".tmp" (abc.npy.tmp) but np.save auto-appends .npy to paths not
ending in .npy, writing to abc.npy.tmp.npy instead; os.replace then
raised FileNotFoundError silently, making every cache write a no-op
and leaking *.npy.tmp.npy files. Fixed by inserting .tmp before .npy:
tmp = final[:-4] + ".tmp.npy"
- config.py: remove str(default) round-trip in _getenv_int/_getenv_float
— use raw = os.getenv(name); return default if raw is None else int(raw)
so a future float default can't cause a spurious "not a valid integer"
warning and return the wrong type
- executor.py: consolidate 4 progress.remove_task calls into one
try/finally around the per-job body; continue inside try/finally
executes the finally before the next iteration, making the invariant
structurally enforced rather than relying on discipline across 4 sites
YEARS_FILTER, MIN_FACE_WIDTH, MIN_FACE_COUNT, MAX_AUTO_IMAGES,
BLUR_THRESHOLD, MIN_CONFIDENCE, and FACE_MARGIN used bare int()/float()
calls with no error handler. A typo (trailing space, non-numeric value)
raised ValueError inside __getattr__, producing a cryptic traceback on
the first config access rather than at the validate() step. Values are
now parsed by _getenv_int/_getenv_float helpers that warn and fall back
to the documented default, matching the existing FRIGATE_SCORE_CEILING
pattern.
- executor.py: wrap shutil.rmtree/os.makedirs in try/except OSError so a
permission failure logs and skips the job rather than aborting the run
- cache.py: write embeddings to a .tmp file and atomically rename into place
via os.replace so a process kill can't leave a corrupted .npy cache slot
- immich_api.py: guard fileCreatedAt with isinstance(str) check before calling
.replace() so a non-string timestamp doesn't raise AttributeError and kill
the entire filter_recent_assets pass
- upload_tracker.py: raise SQLite busy timeout from 5 s to 30 s to handle
concurrent cron+manual run overlap without dropping upload-tracking records
progress.add_task() fires unconditionally at the top of the job loop;
both continue paths (ValueError from _safe_person_dir and the symlink
TOCTOU guard) skipped remove_task(), leaving orphaned 0% rows in the
terminal for the rest of the run.
The v0.5.10 compound guard 'isdir and not islink' silently skipped the
rmtree when person_dir was a symlink-to-directory, then let makedirs
follow the symlink — allowing crop writes outside output_dir with no
diagnostic. Replace with an explicit islink pre-check that logs an error
and continues, matching the ValueError path from _safe_person_dir.
- reconcile.py: re-escalate the < target branch from INFO to WARNING and
add 'permanently unmapped' label. Both post-loop branches produce identical
permanent mapping loss; v0.5.9 incorrectly treated the timeout case as
recoverable.
- executor.py: guard shutil.rmtree with 'not os.path.islink(person_dir)'
so a race-replaced symlink-to-directory is skipped rather than raising
an unhandled OSError that aborts all remaining jobs. Correct comment:
rmtree raises OSError, not NotADirectoryError.
- reconcile.py: swap log levels — external-upload path (permanent mapping
loss) escalated to WARNING; timeout path (transient, retries next cycle)
downgraded to INFO. Also extend the warning message to note the files are
permanently unmapped.
- immich_api.py: extend fetch_all_assets docstring to document that
all-garbage page termination (in addition to network errors) makes
total_raw a lower bound.
- executor.py: add comment above shutil.rmtree noting that POSIX rmtree
raises NotADirectoryError on a top-level symlink, documenting why the
removed islink guard is safe to omit.
- upload_tracker.py: replace setdefault with explicit guard in _entry() —
setdefault evaluates its default-dict argument before checking key
presence, allocating and discarding a dict on every already-present call.
- executor.py: fix _safe_person_dir docstring — realpath+startswith is the
load-bearing traversal guard; islink is a supplementary early-exit for the
symlink sub-case only. The previous comment "checking after realpath would be
too late" implied islink was the primary guard, which is backwards.
- immich_api.py: move total_raw accumulation to after the dead-end-page break
so all-garbage pages don't inflate the count and produce misleading
"N total, 0 recent" output. Mixed pages (some valid, some non-dict) still
count page_count so transient schema issues don't shrink MIN_FACE_COUNT below
threshold. Add warning when a RequestException interrupts pagination mid-way
so operators know total_raw is a lower bound.
- config.py: eliminate residual TOCTOU — change `if config_file.exists():` to
`if _data_cfg_exists or config_file.exists():` so _data_cfg is never
stat'd twice (the v0.5.7 fix cached the first check but not the second).
executor.py:
- Move islink check into _safe_person_dir on the raw path, before realpath
resolves it; the previous check at the rmtree site was unreachable dead code
because realpath already followed any symlink
immich_api.py / jobs.py:
- fetch_all_assets now returns (assets, total_raw) where total_raw is the
item count seen before non-dict filtering; callers use it for MIN_FACE_COUNT
guard and display so transient non-dict API items can't incorrectly skip people
- Add WARNING when pagination stops because a page had items but all were non-dict
config.py:
- Cache _data_cfg.exists() in _data_cfg_exists so the dual-config warning
and config_file selection always read from the same stat() result; previously
two calls created a TOCTOU window where log and code could disagree
Bump version to 0.5.7
immich_api.py:
- Move empty-page break after non-dict filtering — a page of all-null
items no longer loops to MAX_PAGES without terminating
- Single-pass partition replaces two inverse isinstance scans per page
- Upgrade non-dict item log from DEBUG to WARNING (silent asset loss)
reconcile.py:
- Check Frigate before the first sleep so fast responses return
immediately rather than always paying a 1 s delay
- Compute set difference once per poll iteration instead of twice
Bump version to 0.5.6
* fix: quality replacement slot floor uses deleted file's score not failed candidate's
When a blur-score replacement deletes a low-quality Frigate file but the
subsequent upload fails, min_quality_score_for_slot was set to candidate_score
(the good file that failed to upload). This filtered out any subsequent
candidate that didn't beat the failed upload, even if it was better than
the file we just deleted — leaving the freed slot unfilled unnecessarily.
The comment on the guard already documented the correct intent: 'require
the next candidate to beat the deleted file's score'. Fix: use target_score
(the deleted file's blur score) as the floor instead of candidate_score.
* chore: bump version to 0.5.4
* fix: audit hardening — input validation, error handling, and robustness
- immich_api: guard person["id"] with .get() + early return on missing field
- immich_api: include page number in pagination exception log
- immich_api: validate faces response is a list before indexing
- executor: wrap Image.open() in try/except for non-image HTTP responses
- executor: strip leading 'v' from Frigate version before parsing (v0.16.0 was misread)
- config: wrap FRIGATE_SCORE_CEILING float() parse in try/except with warning
- config: warn when both DATA_DIR and legacy CWD config files exist simultaneously
- scheduler: wrap PID file write in try/except so /tmp failures don't crash startup
- scheduler: clamp sleep to 60s max to bound recovery time after NTP clock jumps
- frigate_api: log unexpected non-list type in get_frigate_person_files at DEBUG
* fix: LIMIT env var crash and symlink guard on person output dir
- jobs: wrap int(LIMIT) parse in try/except — bad value (e.g. "30.5", "all")
now logs a warning and falls back to the default instead of crashing
- executor: check for symlink before shutil.rmtree on person_dir — prevents
following a symlink out of OUTPUT_DIR on a shared volume
* chore: bump version to 0.5.3
* refactor: rename CACHE_DIR to DATA_DIR, default path .if_cache → data
CACHE_DIR held both the embedding cache and the SQLite tracker DB, making
the name misleading. DATA_DIR is more accurate.
- Config reads DATA_DIR first; falls back to CACHE_DIR with a deprecation
warning so existing setups don't break on upgrade
- Default local path: data (was .if_cache)
- Docker default path: /app/data (was /app/.if_cache)
- Internal references (embeddings.py, upload_tracker.py) updated to DATA_DIR
- compose.yml, .env.example, README, wiki, and changelog updated
- Version bumped to 0.5.1
* chore: update lockfile
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* refactor: rename CACHE_DIR to DATA_DIR, default path .if_cache → data
CACHE_DIR held both the embedding cache and the SQLite tracker DB, making
the name misleading. DATA_DIR is more accurate.
- Config reads DATA_DIR first; falls back to CACHE_DIR with a deprecation
warning so existing setups don't break on upgrade
- Default local path: data (was .if_cache)
- Docker default path: /app/data (was /app/.if_cache)
- Internal references (embeddings.py, upload_tracker.py) updated to DATA_DIR
- compose.yml, .env.example, README, wiki, and changelog updated
- Version bumped to 0.5.1
* chore: update lockfile
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
- release.yml: replace per-variant lockfile generation loop with a
single 'uv lock'; add idempotent release creation (skip if tag
already has a release so re-triggered runs don't 422)
- docker-publish.yml: remove stale uv-cpu/rocm/intel.lock entries
from paths-ignore (those files no longer exist)
- README: CACHE_DIR description now names winnow_tracker.db; tracker
description mentions SQLite
Two fixes found during post-refactor audit:
1. upload_tracker: remove COUNT(*) guard from _maybe_migrate. The guard
blocked re-migration when a previous run successfully committed both
JSON files but a PermissionError on the second rename() left it on
disk. On the next startup COUNT > 0 → early return → rejected IDs
permanently unimported. INSERT OR IGNORE is idempotent so re-running
migration is always safe; guard not needed.
Also wrap each rename() in its own try/except so a failure on one
file is logged and does not propagate uncaught.
2. reconcile: break early when new_count > target is detected in the
poll loop. Previously the loop ran all four delay intervals (1+2+4+8s)
before the post-loop > target branch fired, wasting up to 15 seconds
when a concurrent external upload was visible on the first poll.
- upload_tracker: partial migration now rolls back atomically on failure;
JSON renamed only after successful commit so failed runs retry cleanly
- upload_tracker: allowlist score_col in _pick_mapped_file to close
latent SQL injection surface
- config: move load_dotenv() from module import into _load() so no I/O
at import time and reset() fully resets env loading
- config: use is None checks for IMMICH_URL/OUTPUT_DIR config-file
fallback so explicitly empty env vars are not overridden by the file
- executor: skip reconcile when Frigate API is unreachable at upload
start — tracker baseline is incomplete and would mis-trigger the
external-upload guard, permanently losing file mappings
- reconcile: change polling break condition from >= to == target so
transient overshoots don't prematurely exit the loop and trigger
the external-upload guard
- jobs: apply capacity cap as the selection limit rather than truncating
post-selection by position, so the diversity algorithm works within
the right budget from the start
- Dockerfile: explicit gpu branch + exit 1 on unknown VARIANT instead
of silent fallback
_build_job no longer calls filter_already_uploaded internally; callers pass
pre-filtered assets so there's no double DB hit and the interactive path
restores the original prompt order (retry_rejected asked before strategy,
so post-dedup count informs the choice). Skip-count rprint restored in
auto_configure. Late 'import time' inside reconcile_frigate_mappings moved
to module level.
- rocm/intel/gpu extras are x86_64-only; aarch64 wheels don't exist so uv
failed to resolve them when required-environments includes aarch64
- test.yml: switch from --all-extras (broken by conflicts + missing wheels)
to --extra cpu which is cross-platform and sufficient for unit tests
- update-lockfile.yml: drop old file-swap loop; single pyproject means a
single uv lock run and a single uv.lock to commit
- Config: remove _ConfigAccessor and ConfigManager; use __getattr__ for lazy
loading on single _Config class; re-register self as _instance in __getattr__
so reset() always clears the correct object (item 1)
- upload_tracker: replace hand-rolled JSON store with sqlite3; auto-migrates
existing JSON on first run; remove dead record_frigate_file function;
connection re-opens when CACHE_DIR changes for test isolation (items 2, 8)
- diversity: move ThreadPoolExecutor import to module level; inject optional
fetch_fn parameter for testability (items 3, 6)
- pyproject: consolidate 4 variant files into extras (gpu/rocm/intel/cpu);
update Dockerfile to use --extra flag; delete variant pyproject/lock files;
uv.lock needs regen with `uv lock` after this change (item 4)
- jobs: extract _build_job helper to separate business logic from terminal I/O;
auto_configure delegates dedup/selection to _build_job (item 5)
- logging: convert f-string log calls to % interpolation throughout all winnow/
modules (item 7)
- reconcile: new module with reconcile_frigate_mappings and
enrich_asset_with_face_data extracted from executor.py (item 9)
- scheduler: print next scheduled run time after startup and after each run;
fix f-string logger.error call (item 10)
22.04 non-GPU amd64 bases were never updated when arm64 moved to 24.04.
Python 3.13 is still pulled from deadsnakes PPA (26.04 ships 3.14 natively).
intel stays on 22.04: the Intel GPU repo URL is pinned to the "jammy"
codename and cannot be bumped until Intel publishes 26.04 packages.
flan
