flan/winnow
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: log thumbnail fetch failures and restore get() for duplicate-ID safety

Browse Source 
Two small correctness fixes from post-commit code review:

- except Exception: continue swallowed network/auth errors silently; add
  logger.debug so systematic failures are diagnosable in winnow.log
- batch_images.pop() regressed duplicate-asset-ID handling: if Immich
  returns the same asset ID twice within the same 32-item batch window
  (pagination edge case), the second occurrence got None and its embedding
  was silently dropped. Switching back to .get() matches the old
  thumbnail_map.get() behaviour. Peak memory is still bounded to _BATCH
  images because batch_images goes out of scope between batches.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
flan
2026-06-12 18:51:44 +00:00
co-authored by Claude Sonnet 4.6
parent 86ee9a5ba2
commit 046004a5d0
1 changed files with 5 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
winnow/diversity.py
+5 -3
View File
@@ -227,12 +227,14 @@ def _select_by_embedding(
img = future.result() img = future.result()
if img is not None: if img is not None:
batch_images[asset["id"]] = img batch_images[asset["id"]] = img
except Exception: except Exception as e:
logger.debug(f"Failed to fetch thumbnail for {asset['id']}: {e}")
continue continue
# Process and immediately release each image to cap peak memory # Process each image; batch_images goes out of scope after this loop,
# bounding peak thumbnail memory to _BATCH images per iteration.
for asset in batch: for asset in batch:
img = batch_images.pop(asset["id"], None) img = batch_images.get(asset["id"])
processed += 1 processed += 1
if progress_callback: if progress_callback:
progress_callback(processed, len(candidates)) progress_callback(processed, len(candidates))