The sidecar must carry EVERY pending tree, not just the newest
CI / shell (shellcheck + syntax) (push) Successful in 9s
CI / python 3.12 (push) Successful in 13s
CI / python 3.13 (push) Successful in 17s
CI / python 3.11 (push) Successful in 15s
TrueNAS compatibility / compat (push) Successful in 11s
Release / release (push) Successful in 15s
CI / shell (shellcheck + syntax) (push) Successful in 9s
CI / python 3.12 (push) Successful in 13s
CI / python 3.13 (push) Successful in 17s
CI / python 3.11 (push) Successful in 15s
TrueNAS compatibility / compat (push) Successful in 11s
Release / release (push) Successful in 15s
Found live, in the code written to prevent exactly this. The sidecar held ONE snapshot. So a run that reclaimed an older tree, failed to finish reclaiming it, and then recorded its own snapshot OVERWROTE the only record of the survivor -- orphaning it permanently. Observed: job 24 left one snapshot busy and kept the sidecar (correct). Job 46 reclaimed it, hit ZFS's 300s automount window (the runs were minutes apart), left it behind again, and then wrote its own snapshot over the record. Permanent orphan, created by the safety net. The sidecar is now a list. stage_nested carries forward whatever a reclaim could not delete; cleanup_task sweeps every pending tree and writes back only the survivors. cleanup_all reports them one per line instead of formatting a list into an f-string at the user during uninstall. Job 46 also confirms the automount fix itself: it swept all 256 of its own snapshots with no straggler.
This commit is contained in:
@@ -95,6 +95,25 @@ re-scan each time.
|
||||
|
||||
### Snapshot lifecycle
|
||||
|
||||
> **A snapshot may survive a run, and that is expected.** ZFS **automounts**
|
||||
> `<dataset>/.zfs/snapshot/<snap>` the moment it is read, and holds it for
|
||||
> `zfs_expire_snapshot` seconds (**300** by default) after the last access. So
|
||||
> whatever restic read *last* is still pinned when we try to destroy it, and
|
||||
> `zfs destroy` refuses with `dataset is busy`.
|
||||
>
|
||||
> The patch unmounts those automounts itself and retries, which clears ~255 of 256 on
|
||||
> a real pool. The one that remains is **logged, its sidecar is kept, and the next run
|
||||
> reclaims it before doing anything else** — so the leak is bounded at a single cycle
|
||||
> instead of growing forever. Seeing one `could not delete snapshot … it will be
|
||||
> reclaimed on the next run` in the log is normal. Seeing the count *grow* run over run
|
||||
> is not, and would be a bug.
|
||||
>
|
||||
> This is why the sidecar is removed **only on a confirmed-clean sweep**: it is the
|
||||
> only record those snapshots exist, and a run that dropped it while they were still
|
||||
> around would orphan them permanently. That is precisely what happened before this was
|
||||
> fixed.
|
||||
|
||||
|
||||
`zfs.snapshot.delete` defaults to **`recursive=False`**, and stock
|
||||
`restic_backup()` calls it with no options. Stock is safe only because its
|
||||
validation means a *recursive* snapshot never actually happens in the field.
|
||||
|
||||
Reference in New Issue
Block a user