Remove torch/torchvision/transformers/ultralytics from all three variant
pyproject files (rocm/cpu/intel) — no longer needed since SigLIP and YOLO
were removed in v0.4.10. Update version to 0.4.10 and description.
NOTE: uv-rocm.lock, uv-cpu.lock, uv-intel.lock are now stale and must be
regenerated in their respective platform environments before the next Docker
build of those variants.
Also fix fetch_face_data() docstring to not mention embedding (removed field).
- immich_api.py: remove FaceData.embedding field and numpy import — Immich face
embeddings were never consumed after being fetched; bbox/confidence is all that's used
- embeddings.py: remove immich_embedding param from get_embedding() — never passed by
any caller; simplify docstring accordingly
- image_processing.py: wrap insightface_app.get() in warnings filter to suppress the
scikit-image FutureWarning about estimate being deprecated (already suppressed in
embeddings.py for the diversity path, was leaking from the crop-alignment path)
- README.md: remove two stale "object mode" / "object classification" fragments
After removing the object pipeline, several dead 'mode' artifacts remained:
- executor.py: unpack `config` from job even though it was no longer read
- jobs.py: set `"mode": "face"` in both configure paths (key never consumed)
- compose.yml: TRAINING_MODE=face env, OBJECT_CLASS comment, HF_HOME, stale MAX_AUTO_IMAGES default note
- Dockerfile: "and object classification" label, /models/huggingface mkdir, HF_HOME ENV
- cache.py: drop siglip MODEL_VERSIONS entry
- log_config.py: drop ultralytics/transformers/torch from noise silencers
- scheduler.py: drop HF_HOME and HuggingFace model check
- scripts/benchmark.py: drop bench_siglip and make_random_image
winnow is a face recognition training tool. Object mode required manual
file placement with no Frigate API, pulled in torch/torchvision/transformers/
ultralytics (~2 GB), and was architecturally misaligned with the project goal.
Removed:
- process_object_mode (YOLO inference), get_yolo_model
- SigLIP model stack (get_siglip_model, get_object_embedding, batch variant)
- entity_type branching throughout diversity, embeddings, jobs, executor
- TRAINING_MODE and OBJECT_CLASS env vars
- torch, torchvision, transformers, ultralytics dependencies
- pytorch index entries from pyproject.toml
- Object mode from README (Modes section, env var table, How It Works)
- How It Works step 9: "upload freely" → note novelty gate may skip below-cap candidates
- Persistence note: "Frigate rejections" → "rejected assets" (covers confidence skips too)
- RETRY_REJECTED description: explicitly covers all rejection types, not just Frigate
- Image Quality section: split into user-adjustable controls and calibrated image
processing defaults with a support disclaimer to deter blind tuning
Previously, assets that passed embedding-phase selection but failed the
faces API confidence check in execute_jobs were silently skipped with no
tracker entry. They appeared as valid candidates on every future run,
were re-selected, and re-skipped in an endless cycle. Now they are
marked rejected so they are excluded from future runs. RETRY_REJECTED=true
clears them if Immich later re-processes the image.
FRIGATE_SCORE_CEILING now defaults to dynamic mode (unset): below-cap
candidates are skipped if their pre-upload Frigate score exceeds the
most-redundant tracked file's score. This catches conditions already
covered by manually-added Frigate images that winnow cannot track —
the embedding-based diversity selection has no visibility into those.
Set FRIGATE_SCORE_CEILING=0 to disable; a positive value (e.g. 0.85)
still acts as a fixed hard ceiling. First-run safety is unchanged
(pre_run_count==0 prevents recognize_face from being called).
The two quality replacement branches (Frigate-score and blur-score)
shared identical structure and are merged into a single code path
parameterised by score source and comparison direction.
Also raises MIN_FACE_COUNT default from 0 to 3 and updates the
config test to match.
AUTO_MODE (batch/unattended) and STRATEGY=auto (diversity algorithm) shared
the same word for unrelated concepts. Renaming the strategy value to
'adaptive' eliminates the ambiguity. The old value is kept as a silent alias
so existing configs continue to work.
Interactive menu updated from "Auto (Objective Diversity)" to "Adaptive
Diversity". README AUTO_MODE description clarified to emphasise unattended
batch processing, not selection strategy.
- `:latest` image tag listed CUDA 13.3 — actual base is 12.8.1
- MERGE_DUPLICATE_PEOPLE existed in config but was absent from env var table
- TRACE_CROP_SIZE existed in CLI but was absent from env var table
- RESET_PERSON description now mentions `*` wildcard for bulk reset of all people
- Step 5 (near-duplicate removal) was added in v0.4.4 but never
documented; added between embedding and diversity selection
- Auto-stop was described as 'stops when similarity exceeds threshold'
which is backwards; it stops when the next candidate's distance to
already-selected images falls below the threshold (too similar, not
enough new information)
- Renumbered steps 5-8 to 6-9
- Tightened hard-example weighting description to match the code
All _load() calls after the first return the cached dict instead of
re-reading disk. _save() updates both disk and cache atomically.
Drops per-person tracker reads from ~90 to ~1 in the upload loop.
Keyed by resolved file path so test isolation (unique tmp_path dirs)
is preserved with no fixture changes needed.
- _dedup_embeddings: pre-allocated (Q,D) buffer replaces vstack-on-keep,
dropping O(K²×D) copy overhead down to O(K×D) fill work
- _kmedoids: swap cost sum replaced with numpy fancy-index reduction,
~20-50x faster per swap evaluation
- _reconcile_frigate_mappings: O(L) load/save pairs collapsed to one
batch write via record_frigate_files_batch
Fetching up to MAX_PAGES*page_size (1M) assets before the 3000-item
diversity pool cap was applied could exhaust memory on large Immich
libraries. Early-exit once 5000 items are collected — the pool cap
of 3000 makes anything beyond that wasteful. Also filter null/non-dict
items from page responses at fetch time.
- _dedup_embeddings: rebuild kept_stack only on keep (was every iteration → O(N²))
- _dedup_embeddings: fix quality_score sort key to use explicit None check (falsy-zero)
- _select_by_embedding: add post-dedup pool < limit guard with warning
- executor: use full resp.text for 'face' keyword check; only truncate display snippet
- _safe_person_dir: avoid false "//" prefix when output_dir resolves to filesystem root
0.10 only removed burst shots (distance 0.01-0.05). Same-event photos with
similar pose and lighting sit at 0.10-0.20 and were passing through,
producing visually similar training images especially for people with small
datasets. 0.20 removes these while still preserving genuinely different
poses, expressions, and lighting conditions.
os.path.join silently discards the base when the second arg is absolute,
and '../..' sequences escape the output tree. _safe_person_dir() resolves
both paths with realpath and rejects any name that lands outside the output
directory, logging an error and skipping the job rather than touching an
unintended path.
- RESET_PERSON=* resets every tracked person (deletes their Frigate files
and clears the tracker). Any other value resets that specific person by
name, including someone literally named 'all'.
- Near-duplicate removal pass added before diversity clustering: greedily
drops candidates within 0.10 cosine distance of a higher-quality image,
eliminating burst-shot duplicates that FPS would otherwise pass through.
Burst shots produce embeddings that differ slightly (~0.01-0.05 cosine
distance) due to JPEG noise and minor lighting variation, so FPS does not
filter them. Add a greedy dedup pass after embedding collection: sort
candidates by quality score descending, then drop any candidate within
0.10 cosine distance of an already-kept image. The best frame from each
near-identical group survives; the rest are dropped before clustering.
flan
github-actions[bot]