Collect orphaned snapshots by name: the sidecar lives in tmpfs
CI / shell (shellcheck + syntax) (push) Successful in 10s
CI / python 3.11 (push) Successful in 13s
CI / python 3.12 (push) Successful in 16s
CI / python 3.13 (push) Successful in 16s

A reboot mid-backup orphaned the entire tree, permanently. The sidecar is the record
of which snapshots a run pinned -- and /run is tmpfs. A reboot or crash between the
recursive snapshot and its cleanup destroyed that record, leaving one snapshot per
descendant dataset (250+ on a real pool) with nothing pointing at them. Nothing would
ever have found them.

gc_stale_snapshots() identifies leftovers by NAME, so it works when the record is
gone. It runs after the sidecar reclaim -- the recorded path stays authoritative and
the collector only mops up what the record lost.

It deletes data on a name match, which is a weaker claim than a recorded fact, so the
selection is a pure function with the harshest tests here. A snapshot is collected
only if the name is exactly <dataset>@<task>-<YYYYMMDDHHMMSS>, it is not the current
run's, NOTHING IS MOUNTED FROM IT (this, not the age guard, is what protects a
concurrent backup), and it is over an hour old.

Checked against the real pool: of 4728 snapshots including 2341 periodic ones, it
selects exactly the orphans of the task being run and nothing else.
This commit is contained in:
2026-07-13 19:56:32 +00:00
parent 2b8ef107f7
commit 0d04c2cd1c
4 changed files with 360 additions and 1 deletions
+15
View File
@@ -95,6 +95,21 @@ re-scan each time.
### Snapshot lifecycle
> **Two mechanisms clean up, and the second exists because the first can be destroyed.**
>
> 1. **The sidecar** records exactly which snapshots a run pinned, and is removed only
> on a confirmed-clean sweep. Precise, and it survives a middlewared restart.
> 2. **The garbage collector** finds leftovers by *name*, so it still works when the
> sidecar is gone — and it can be: **the sidecar lives in `/run`, which is tmpfs.** A
> reboot mid-backup takes it, and with it the only record of a 250-snapshot tree.
>
> The collector runs at the start of every backup, after the sidecar reclaim. It will
> only touch a snapshot named `<dataset>@<task>-<timestamp>` that is not the current
> run's, has **nothing mounted from it** (which is what protects a concurrently-running
> backup), and is **over an hour old**. Periodic `auto-*` snapshots, other tasks'
> snapshots, and anything you made by hand are structurally out of reach.
> **A snapshot may survive a run, and that is expected.** ZFS **automounts**
> `<dataset>/.zfs/snapshot/<snap>` the moment it is read, and holds it for
> `zfs_expire_snapshot` seconds (**300** by default) after the last access. So