Commit Graph
4 Commits
Author SHA1 Message Date
flan 0fc994f676 Re-apply and verify the patch before the deferred restart
CI / shell (shellcheck + syntax) (push) Successful in 16s
CI / python 3.12 (push) Successful in 54s
CI / python 3.11 (push) Successful in 55s
CI / python 3.13 (push) Successful in 44s
Patching at PREINIT and restarting minutes later is only sound while the
patched files are still on the live path when middlewared re-imports them,
and PREINIT cannot guarantee that. The overlay sits inside /usr, so anything
that remounts that hierarchy detaches it — a systemd-sysext merge/refresh
from another PREINIT hook, or middlewared's own docker.configure_nvidia at
runtime. Init scripts run sequentially in id order, so a hook registered
after this one always wins, and reordering them would not help because
docker.configure_nvidia fires long after PREINIT is done.

Observed on 25.10.6: the overlay was mounted at 16:41:56, a sysext refresh
unmerged and remerged /usr four seconds later, and the deferred restart at
16:47:24 loaded stock modules. Every B2 cloud_backup task then failed with
NotImplementedError for nineteen hours across four scheduled runs while
apply.log and hook_status.json both reported the patch active.

wait_restart.sh now re-applies immediately before restarting — after boot has
settled, which is also after every sysext merge and docker nvidia
configuration — verifies the marker is on the live path, restarts, and
verifies again, retrying once. It is no longer exec'd, so something can run
after the restart to find out what it loaded. apply.sh records the resolved
middlewared directory in .mw_dir for that check, and honours TRUECLOUD_REAPPLY
so the re-apply pass does not schedule a second restart.

_ensure_writable treated "one of our overlays is listed here" as "already
done", but it only reaches that check when the directory is not writable, and
a live overlay of ours always is — a shadowed overlay was indistinguishable
from a healthy one. It is now detached and re-mounted, reusing the upperdir so
files patched earlier in the boot survive, with a fresh workdir and a retry on
a private one, since overlayfs refuses a workdir a detached mount still holds.

Add a CRITICAL hourly alert for the case none of this can prevent: the patch
being on disk but not in the running process. apply.log can only report the
first. The alert asks the second question from inside middlewared, where the
patch's own stamps make it exact, and checks both halves since either can go
missing alone. It stays quiet when the kill switch is set or the providers
module has been retired as native, and is not muted by update_alerts_disabled.

wait_restart.sh also logs to apply.log now: journald retention on a busy box
is easily shorter than the interval between reboots, and the boot that caused
this had already rotated away by the time it was investigated.
2026-08-26 04:37:04 +00:00
flan ca906f5ee4 docs: TrueNAS 26 is supported; record what has actually been run
CI / python 3.12 (push) Failing after 16s
CI / shell (shellcheck + syntax) (push) Successful in 10s
CI / python 3.11 (push) Failing after 12s
CI / python 3.13 (push) Failing after 16s
TrueNAS compatibility / compat (push) Successful in 9s
The README and how-it-works still said "TrueNAS 26: nested snapshots are not supported
yet" and "the third is not fixed, and is why 26 reports BROKEN". Both shipped in v0.7.0
and are now false — exactly the kind of stale claim that misleads somebody deciding
whether to trust this with their backups.

Adds docs/verification.md: what has ACTUALLY been run, as opposed to what the support
matrix proves. The matrix is static analysis — it shows the patch's assumptions still
hold, which is a strictly weaker claim than "a backup ran and a restore came back". The
new file records the three live tasks exercised on 25.10.4 (nested, nested+zvols,
non-nested), the md5 of the file that came back out of B2, the real orphan the collector
reclaimed from the pool, and what is NOT covered (24.10/25.04 unrun; master broken; no
reboot on v0.7.0). The README now points at it, next to the matrix it qualifies.

The how-it-works TrueNAS 26 section now explains the part that mattered: the public
pool.* queries are not like-for-like replacements for the deleted private zfs.* ones —
they apply a visibility policy hiding 84 of 270 datasets on a real pool, including live
app data — and the rule the module now follows (read the truth from ZFS, make changes
through middleware). Plus the divergence nothing warned about: 26 decides `recursive` by
a different rule than this patch decides `nested`, which orphaned one snapshot per zvol
on every run until ownership of the sweep was made unconditional.
2026-07-14 02:22:36 +00:00
flan 518a22d87e docs: user-facing URLs point at GitHub, the user-facing repo
CI / shell (shellcheck + syntax) (push) Successful in 8s
CI / python 3.11 (push) Successful in 13s
CI / python 3.12 (push) Successful in 14s
CI / python 3.13 (push) Successful in 15s
TrueNAS compatibility / compat (push) Failing after 6s
Release / release (push) Successful in 14s
Gitea is canonical for development; GitHub is where users clone from and where the
box's read-only checkout points. The install instructions, the re-clone hint and the
'file an issue' link are all read by users, so they name GitHub. docs/releasing.md
still names Gitea, because that is a contributor doc about where the code is pushed.
2026-07-13 18:42:44 +00:00
flan 252696f1de docs: split the 969-line README; put Install at the top
CI / shell (shellcheck + syntax) (push) Successful in 8s
CI / python 3.13 (push) Successful in 14s
CI / python 3.12 (push) Successful in 14s
CI / python 3.11 (push) Successful in 13s
Install was at line 517 of 969, under the boot sequence, the snapshot lifecycle and
the release process. Someone deciding whether to trust this with their backups should
not have to scroll past any of that.

README is now 211 lines: what it does, the minimum version, install, the support
matrix, updating, uninstall. Everything else moved to docs/ (nested snapshots, how it
works, recovery, CLI, releasing).

A test enforces it: every internal link resolves, Install stays near the top, and the
README does not grow back. Moving Markdown breaks cross-references -- it broke eight
of them here, including one in a recovery doc, where the person following the link is
by definition already having a bad day.
2026-07-13 18:34:44 +00:00