Commit Graph
15 Commits
Author SHA1 Message Date
flan e2a1924fb0 refactor: collapse Config proxy, migrate tracker to SQLite, split reconcile module
- Config: remove _ConfigAccessor and ConfigManager; use __getattr__ for lazy
  loading on single _Config class; re-register self as _instance in __getattr__
  so reset() always clears the correct object (item 1)
- upload_tracker: replace hand-rolled JSON store with sqlite3; auto-migrates
  existing JSON on first run; remove dead record_frigate_file function;
  connection re-opens when CACHE_DIR changes for test isolation (items 2, 8)
- diversity: move ThreadPoolExecutor import to module level; inject optional
  fetch_fn parameter for testability (items 3, 6)
- pyproject: consolidate 4 variant files into extras (gpu/rocm/intel/cpu);
  update Dockerfile to use --extra flag; delete variant pyproject/lock files;
  uv.lock needs regen with `uv lock` after this change (item 4)
- jobs: extract _build_job helper to separate business logic from terminal I/O;
  auto_configure delegates dedup/selection to _build_job (item 5)
- logging: convert f-string log calls to % interpolation throughout all winnow/
  modules (item 7)
- reconcile: new module with reconcile_frigate_mappings and
  enrich_asset_with_face_data extracted from executor.py (item 9)
- scheduler: print next scheduled run time after startup and after each run;
  fix f-string logger.error call (item 10)
2026-06-14 19:59:17 +00:00
flan 2d7b52470e chore: remove dead object/SigLIP references after pipeline removal
- cache.py: drop siglip MODEL_VERSIONS entry
- log_config.py: drop ultralytics/transformers/torch from noise silencers
- scheduler.py: drop HF_HOME and HuggingFace model check
- scripts/benchmark.py: drop bench_siglip and make_random_image
2026-06-14 17:31:54 +00:00
flanandClaude Sonnet 4.6 85e499a677 fix: audit — score falsy-zero, crop person mismatch, adaptive cap, suppress_output, ldconfig glob, scheduler sleep
- immich_api: `score or confidence` treated 0.0 score as falsy; use explicit None check
- diversity: same falsy-zero fix in _get_face_confidence
- diversity: _crop_face_from_thumbnail scale loop now filters by person_id (was
  using first person's imageWidth/imageHeight regardless of target in group photos)
- jobs: partially-trained auto mode kept limit="auto" for adaptive stopping, then
  caps result to remaining capacity (was converting to int, silently disabling FPS
  adaptive threshold and early-stop)
- embeddings: _suppress_output finally block wraps first dup2 in try/finally so
  stderr is always restored even if stdout restore raises OSError
- Dockerfile: ldconfig find uses python3.* glob instead of hardcoded python3.13
- scheduler: sleep until next_run instead of fixed 60s; eliminates late-fire jitter
  and unnecessary wakeups on long schedules

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 16:48:24 +00:00
flanandClaude Sonnet 4.6 3db52c2141 fix+logging: model load logging, CPU fallback, log level audit
Bugs fixed (from high-effort review):
- InsightFace CPU fallback now passes providers=['CPUExecutionProvider'] and
  wraps in _suppress_output() so broken GPU drivers don't cause fallback to
  try the same broken provider again, and C-extension noise stays suppressed
- scheduler.py: BaseException → Exception (KeyboardInterrupt already re-raised;
  winnow has no sys.exit() calls, so SystemExit would not occur, but Exception
  is the correct scope)
- compose.yml: fix inverted AUTO_MODE comment (docker run -it enables
  interactive mode via TTY, not non-interactive)

Model loading logging (embeddings.py):
- InsightFace: disk cache check, "not cached — downloading now (~300 MB)",
  "loading into memory on GPU/CPU...", "ready on GPU/CPU (Xs)"
- SigLIP: same treatment; cache path derived dynamically from model_name
  via HuggingFace slug convention (models--org--model) so it stays correct
  if the model variant ever changes

Logging level audit (INFO/DEBUG/WARNING/ERROR):
- diversity.py: internal algo steps (clustering, medoids, adaptive threshold,
  auto-stop decision) → DEBUG; final selection summaries stay INFO
- immich_api.py: "Fetching assets" and "Retained N assets" → DEBUG (callers
  already print this to the console via rprint)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 15:23:44 +00:00
flanandClaude Sonnet 4.6 af8bae1c45 fix: embedding cache key mismatch, scheduler path/exception, log handler leak
embeddings.py: face embedding cache used 'immich' for lookup but
'insightface' for storage, so the cache was never hit for locally-
computed embeddings. Unified to 'insightface'/'siglip' throughout.
This affects all users since ENABLE_CACHE now defaults to true.

scheduler.py: INSIGHTFACE_HOME=/models/.insightface was having
'.insightface' appended again, making buffalo_l check always report
'will download'. Also catch BaseException (not just Exception) so a
SystemExit from a library call can't silently kill all future runs.

log_config.py: handlers.clear() abandoned open FileHandler fds on
each scheduled main() call. Close each handler properly before removal.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 15:10:44 +00:00
flanandClaude Sonnet 4.6 125ce54c7f fix: stale __version__, scheduler import order, quality test coverage
- __init__.py: derive __version__ from importlib.metadata instead of
  a hardcoded "0.1.0" that was six releases out of date
- scheduler.py: move winnow.cli import to module top (no more noqa);
  clean up redundant bool variables in check_models
- tests/test_quality.py: 17 tests covering all five quality check
  functions individually plus assess_quality integration cases

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 05:33:38 +00:00
flanandClaude Sonnet 4.6 f50d3fe1ab refactor: rename logging.py, in-process scheduler, trim .gitignore
- winnow/logging.py → winnow/log_config.py: avoids shadowing the stdlib
  logging module; log file renamed from immich_export.log to winnow.log
- scheduler.py: run main() in-process instead of subprocess.run so
  InsightFace and SigLIP models stay resident in memory across scheduled
  runs (hundreds of MB load, previously reloaded every run)
- .gitignore: replaced 200-line boilerplate with ~30 project-relevant
  patterns; removed Django/Flask/Redis/RabbitMQ/Scrapy/etc. noise
- .python-version: untracked (redundant with requires-python in
  pyproject.toml; kept in .gitignore for local pyenv users)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 05:30:37 +00:00
flanandClaude Sonnet 4.6 4b6bce9b74 fix: resolve ruff lint errors in scheduler and tests
- scheduler.py: sort import block (I001)
- tests/test_config.py: remove unused os, pytest imports (F401)
- tests/test_upload_tracker.py: remove unused json, os, tempfile imports (F401)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 02:11:16 +00:00
flanandClaude Sonnet 4.6 eea7baa19f refactor: rename project to winnow
- Rename Python package directory if_curator/ → winnow/
- Update all imports, entry points, and CLI references
- Update pyproject.toml: name, scripts, package list, repository URL
- Update Dockerfile, compose.yml, entrypoint.sh, scheduler.py
- Update GitHub Actions workflow image names
- Update README

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 01:07:46 +00:00
root b9b734507b automated update: 2026-06-10 22:25:47 2026-06-10 22:25:47 -04:00
root 6b35dec201 automated update: 2026-06-10 21:13:39 2026-06-10 21:13:39 -04:00
root ea176c0083 automated update: 2026-06-10 19:47:45 2026-06-10 19:47:45 -04:00
Holden 3759b57b40 models 2026-06-10 19:34:14 -04:00
Holden a2f18484a6 better logging 2026-06-10 18:31:43 -04:00
Holden b8445a1bb3 fix cron 2026-06-10 16:48:10 -04:00