Run CI as a single job so concurrent jobs cannot race

Two CI failures shared one cause: four jobs starting together on the
self-hosted runner.

act caches each action as one shared clone under /root/.cache/act/<hash> and
re-pulls it per job, so concurrent jobs fight over that directory and the
loser dies with "lstat .../<file>: no such file or directory" before any test
runs — a different victim each push. And the runner force-pulls its base image
per job, so four jobs meant four anonymous Docker Hub pulls per push; that hit
429 Too Many Requests and every job started failing before it began, including
the shell job nothing had touched.

Installing uv without an action only shrank the surface, since every job still
used actions/checkout. Concurrency is the ingredient, so this removes it: one
job cannot race itself whatever actions it uses, and one job is one pull. The
version sweep moves inside the job and still runs every version after one
fails, preserving what fail-fast: false bought.

.github/FUNDING.yml

@@ -0,0 +1,2 @@
+github: sudolulo
+ko_fi: sudolulo

.github/workflows/ci.yml

@@ -9,9 +9,31 @@ on:
 permissions:
   contents: read
 
+# ONE job, deliberately. This was four (shell + a 3-way python matrix) and they
+# started within the same second on the self-hosted Gitea runner, which is what
+# made CI unreliable in two separate ways:
+#
+#   1. The act action-cache race. `act` caches each ACTION as a single shared git
+#      clone under /root/.cache/act/<hash> and re-pulls it per job, so concurrent
+#      jobs using the same action fight over that directory and the loser dies
+#      with `lstat /root/.cache/act/<hash>/<file>: no such file or directory` --
+#      a red `main` with zero suite output, and a different victim each push
+#      (3.12 on one, 3.11 on the next). Dropping one action only shrank the
+#      surface: every job still used actions/checkout. Concurrency is the actual
+#      ingredient, so removing it removes the whole class -- a single job cannot
+#      race itself, no matter which actions it uses.
+#
+#   2. Docker Hub 429s. The runner force-pulls its base image per job, so four
+#      jobs meant four anonymous pulls per push. A few pushes and re-runs in an
+#      afternoon exhausted the anonymous limit and every job failed before it
+#      started -- including the shell job, which nothing had touched. One job is
+#      one pull.
+#
+# The cost is wall-clock parallelism, and this repo does not need it: the suite
+# is ~1.5s, so container start and interpreter downloads dominate either way.
 jobs:
-  shell:
-    name: shell (shellcheck + syntax)
+  ci:
+    name: ci (shell + python 3.11-3.13)
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
@@ -31,32 +53,57 @@ jobs:
         env:
           SHELLCHECK_OPTS: -S warning -e SC1091
 
-  python:
-    name: python ${{ matrix.python }}
-    runs-on: ubuntu-latest
-    strategy:
-      fail-fast: false
-      matrix:
-        # TrueNAS SCALE middleware runs 3.11+; keep the patch importable across
-        # the versions it may be injected into.
-        python: ["3.11", "3.12", "3.13"]
-    steps:
-      - uses: actions/checkout@v4
-
-      - uses: actions/setup-python@v5
-        with:
-          python-version: ${{ matrix.python }}
-
-      - name: install dev deps
-        run: python -m pip install --upgrade pip pytest ruff
+      # uv-managed interpreters instead of actions/setup-python: the prebuilt-CPython
+      # download path setup-python relies on does not work on the self-hosted Gitea
+      # runner (all three matrix jobs failed at setup there while passing on GitHub);
+      # uv works identically on both.
+      #
+      # Installed by a plain `run:` step rather than astral-sh/setup-uv: one fewer
+      # action is one fewer thing to go wrong, and the action was only ever
+      # fetching a binary -- the interpreter is chosen per command by `uvx
+      # --python`, never by the action.
+      #
+      # Pinned for the same reason ruff is pinned below: an unpinned uv means any
+      # upstream release can turn main red with no code change here.
+      - name: install uv
+        env:
+          UV_VERSION: "0.11.21"
+        run: |
+          curl -LsSf "https://astral.sh/uv/${UV_VERSION}/install.sh" | sh
+          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
 
       - name: ruff
-        run: ruff check patch tests tools
+        # Pinned: an unpinned ruff means any upstream release can turn main red
+        # with no code change.
+        run: uvx ruff@0.16.1 check patch tests tools
 
+      # TrueNAS SCALE middleware runs 3.11+; keep the patch importable across the
+      # versions it may be injected into. Every version runs even after one
+      # fails -- that is what `fail-fast: false` bought when this was a matrix,
+      # and losing it would mean a 3.11 break hides whether 3.12 and 3.13 are
+      # fine, which is exactly the information you want at that moment.
       - name: pytest
-        run: pytest tests -v
+        env:
+          PYTHONS: "3.11 3.12 3.13"
+        run: |
+          fail=0
+          for v in $PYTHONS; do
+            echo "::group::pytest on python $v"
+            uvx --python "$v" pytest tests -v \
+              || { echo "::error::suite failed on python $v"; fail=1; }
+            echo "::endgroup::"
+          done
+          exit $fail
 
       - name: verify injected middleware blocks compile
         # Belt-and-braces: the *_BLOCK strings are appended into live middlewared
         # modules. A syntax error there would break the box at boot.
-        run: pytest tests/test_apply_blocks.py -v
+        env:
+          PYTHONS: "3.11 3.12 3.13"
+        run: |
+          fail=0
+          for v in $PYTHONS; do
+            uvx --python "$v" pytest tests/test_apply_blocks.py -v \
+              || { echo "::error::injected blocks failed to compile on python $v"; fail=1; }
+          done
+          exit $fail

.github/workflows/compat.yml

@@ -24,6 +24,11 @@ on:
     paths:
       # The manifest itself changed -- re-check immediately rather than waiting a day.
       - "tools/compat.py"
+      # ...and so did the thing that PUBLISHES the finding. This was missing, and it
+      # showed: the commit teaching the bot to refresh a stale report body touched only
+      # compat_publish.py, so no run fired, and the report stayed stale until the next
+      # scheduled one. A fix nobody runs is a fix nobody has.
+      - "tools/compat_publish.py"
       - ".github/workflows/compat.yml"
 
 permissions:

.github/workflows/release.yml

@@ -124,7 +124,7 @@ jobs:
           echo "--- release body ---"
           cat /tmp/notes.md
 
-      # This repo is canonically hosted on Gitea (git.onetick.ninja) and mirrored to
+      # This repo is canonically hosted on Gitea (git.arch.fyi) and mirrored to
       # GitHub, and BOTH run this workflow -- Gitea reads .github/workflows too. So
       # the publish step has to work on whichever forge it lands on. Everything
       # above is forge-agnostic; only the "create a release" API differs.

.gitignore

@@ -3,6 +3,10 @@
 /apply.log.1
 /apply.log.2
 /hook_status.json
+# The resolved middlewared directory, recorded by apply.sh so wait_restart.sh can
+# check whether the patched modules are still on the live path without
+# re-deriving site-packages.
+/.mw_dir
 /disabled
 /nested_snapshots_enabled
 # Written by apply.sh when a module's assumptions no longer fit the installed

CHANGELOG.md

@@ -6,6 +6,198 @@ is deliberate: see [Releasing](docs/releasing.md). Twelve releases were cut on
 live, every one of those interrupts every user. An alert people learn to ignore is
 worse than no alert, because one day it carries a security fix.
 
+## v0.8.0 — 2026-08-26
+### Changed
+
+- **CI's python matrix is green on the self-hosted Gitea runner again.** The real
+  failure was that the Gitea runner image executes jobs as root, and the two
+  unreadable-sidecar tests build their scenario with `chmod(0)` — which cannot make
+  a file unreadable for root (`CAP_DAC_OVERRIDE`). Those two tests now skip as root
+  with that reason; GitHub's non-root runner still exercises them. The matrix also
+  moved to uv-managed interpreters (one toolchain across both runners) and ruff is
+  pinned to 0.16.1 so an upstream ruff release can't turn `main` red without a code
+  change.
+- **README badges point at the public GitHub mirror** (workflow status and
+  releases) instead of the private forge. The release badge had also been reading
+  the stale Gitea v0.6.1 release instead of the current v0.7.0 on GitHub.
+
+- **`master` is now labelled `27-dev`, because it is not the next release.** iX
+  branches each major onto its own `release/` line and master rolls straight on to the
+  one after — on 2026-07-14 every recent commit on master targeted `27.0.0-BETA.1`
+  while 26 was still in beta. So a **BROKEN** master row, rendered as
+  "master _(unreleased)_", read as *"the version you are about to install is broken"*
+  when the breakage was a major release away on a line nobody can download. In a table
+  whose entire job is helping somebody decide whether to trust this with their backups,
+  that is a false alarm in the worst possible place. The label is derived from the
+  newest major in the matrix plus one, so it rolls over to `28-dev` by itself once 27
+  branches.
+
+  For the record, the breakage is `NAS-141498` (2026-06-24), "Convert cloud_backup
+  plugin to the typesafe pattern": it re-signatures `restic_backup` and
+  `get_restic_config`, splitting `entry`/`credentials` out of the `cloud_backup` dict.
+  It is deliberately not being chased while the 27 line is still churning.
+
+### Fixed
+
+- **CI turned `main` red on two of three pushes without running a single test,
+  then stopped running at all.** Two failures, one cause: four concurrent jobs on
+  one self-hosted runner.
+
+  `act`, the engine behind the Gitea runner, caches each *action* as one shared
+  git clone under `/root/.cache/act/<hash>` and re-pulls it per job. Jobs
+  starting in the same second fight over that directory, and the loser dies with
+  `lstat /root/.cache/act/<hash>/.npmrc: no such file or directory` — before any
+  suite output exists, with a different victim each push (3.12 on one, 3.11 on
+  the next). Separately, the runner force-pulls its base image per job, so four
+  jobs meant four anonymous Docker Hub pulls per push; a few pushes and re-runs
+  in one afternoon hit `429 Too Many Requests` and *every* job began failing
+  before it started — including the shell job, which nothing had touched.
+
+  CI is now a single job. Dropping one action (uv is installed by a `run:` step
+  rather than `astral-sh/setup-uv`, which was only ever fetching a binary) only
+  shrank the surface, because every job still used `actions/checkout`.
+  Concurrency is the actual ingredient, so removing it removes the whole class:
+  one job cannot race itself whatever actions it uses, and one job is one pull.
+  The Python sweep moved inside that job and still runs every version after one
+  fails — that is what `fail-fast: false` bought, and losing it would mean a 3.11
+  break hides whether 3.12 and 3.13 are fine. The cost is wall-clock
+  parallelism, which this repo does not need: the suite is ~1.5s, so container
+  start and interpreter downloads dominate either way.
+
+  A red gate that is usually noise is worse than no gate, because the one time
+  it means something, nobody looks.
+
+- **The patch survived being applied and then silently stopped existing, because
+  something else remounted `/usr` four seconds later.** On a box running
+  TrueNAS 25.10.6 the boot of 2026-08-19 went: 16:41:56 `apply.sh` mounts its
+  overlay on `/usr/lib/python3/dist-packages`, patches `b2.py`/`restic.py`, logs
+  every step `OK`; **16:42:00** a second PREINIT hook runs `systemd-sysext
+  refresh` over `/usr` — `Unmerged '/usr'.` / `Merged extensions into '/usr'.` —
+  and our overlay, which lives *inside* that hierarchy, is torn off with it;
+  16:47:24 our own deferred restart fires exactly as designed and middlewared
+  imports the **stock** modules. Every B2 TrueCloud Backup task then failed with
+  `NotImplementedError` from stock `rclone/base.py` for nineteen hours, across
+  four scheduled runs, while `apply.log` and `hook_status.json` both said the
+  patch was active.
+
+  Nothing in the patch was wrong, which is the point: applying at PREINIT and
+  restarting later is only sound if the patched files are still on the live path
+  when middlewared re-imports them, and **that is not something PREINIT can
+  guarantee**. Init scripts run sequentially in id order, so any hook registered
+  after ours always wins. Worse, hook ordering cannot fix it either —
+  middlewared's own `docker.configure_nvidia` merges a sysext over `/usr` at
+  *runtime*, long after every PREINIT hook is finished.
+
+  So the deferred restart no longer trusts the PREINIT pass. `wait_restart.sh`
+  now re-applies immediately before it restarts middlewared — after boot has
+  settled, which is also after every sysext merge and docker nvidia
+  configuration — and verifies the marker is genuinely on the live path before
+  restarting. It is no longer `exec systemctl try-restart middlewared`, because
+  something has to run afterwards.
+
+  What runs afterwards deliberately does **not** restart again. `try-restart`
+  returns as soon as middlewared is READY, and middlewared then brings docker up
+  — `docker.configure_nvidia` merges the stock nvidia sysext over `/usr` at that
+  point, detaching the overlay *after* the patched modules have already been
+  imported. A disk check there reports "missing" on a perfectly healthy system,
+  and restarting on that signal would restart a correctly-patched middlewared
+  straight back into the same race. So the overlay is re-mounted for the benefit
+  of the next restart, and the question of whether *this* middlewared actually
+  holds the patch is left to the one thing that can answer it exactly — the
+  in-process alert below. That re-mount preserves `hook_status.json`'s
+  `patched_at`: `create_task.py verify` decides "loaded" by comparing
+  middlewared's start time against that stamp, so a re-apply running *after* the
+  restart would have made the stamp newer than the process which correctly
+  imported the patch, and `verify` would have reported FAIL forever on every
+  boot where the sysext merge detaches the overlay. Caught on hardware while
+  validating the candidate — a new lying status introduced by the fix for a
+  lying status.
+
+  Two supporting fixes fell out of the same failure. `_ensure_writable` treated
+  "one of our overlays is listed on this directory" as "already done" — but it
+  only ever reaches that check when the directory is **not** writable, and a live
+  overlay of ours always is. A shadowed overlay was therefore indistinguishable
+  from a healthy one; it is now detached and re-mounted, reusing the same
+  upperdir so everything patched earlier in the boot reappears intact, with a
+  fresh workdir because overlayfs refuses one left behind by a detached mount.
+
+- **middlewared now says so when it is running stock.** The gap that let this
+  cost nineteen hours was not the remount, it was that nothing could tell the
+  difference between "patched on disk" and "patched in the running process".
+  `apply.log` can only ever report the first. A new CRITICAL alert asks the
+  second question from inside middlewared, hourly, where it is exact: the patch
+  stamps the objects it replaces, so a missing stamp means this interpreter
+  imported stock code. It checks both halves — `restic.py`'s `_truecloud_patched`
+  marker and whether `B2RcloneRemote.get_restic_config` is still the base class's
+  — since either can go missing alone. It stays quiet when the kill switch is
+  set or the providers module has been retired as native, and it is deliberately
+  **not** silenced by `update_alerts_disabled`: that mutes release notifications,
+  not a broken backup path.
+
+  Boot-time diagnosis also no longer depends on the journal. `wait_restart.sh`
+  logged only to the journal, and journald retention on a busy box is easily
+  shorter than the interval between reboots — the 2026-08-19 boot had already
+  rotated away by the time it was investigated. It now writes to `apply.log`
+  alongside everything else.
+
+- **The next maintenance release was never checked, and it is the one that reaches
+  users.** Shipped versions were discovered from `TS-*` tags and unreleased ones from
+  `release/*` branches carrying `-BETA`/`-RC`. A branched-but-untagged *maintenance*
+  release is neither: `release/25.10.5` has no tag, and its line has already shipped,
+  so the "a prerelease of a shipped line is history" filter discarded it. It was
+  invisible — and it is precisely what a 25.10.4 box gets on its next update. A break
+  there would have reached real users before the daily check ever looked at it, on the
+  only line anybody is actually running.
+
+  A plain `release/X.Y.Z` branch is now checked when its line **has** shipped and it
+  sorts **newer** than that line's newest tag. Both things that must stay out fall out
+  of the same rule: `release/24.10-RC.2` sorts older than `TS-24.10.2.4` (history, not
+  a warning), and iX's typo branch `release/25.20.2.2` is on a line that has no tag at
+  all, so it is not a release line. This immediately surfaced two refs that had never
+  been checked — `release/25.10.5` and `release/24.10.2.5` — both of which pass.
+
+  `is_unreleased()` now keys off where a ref came from (branch = not yet shipped)
+  rather than looking for `-BETA`/`-RC` in its name. Otherwise `release/25.10.5` would
+  count as shipped and a break in it would fail the build as a live outage — on a
+  version nobody is running yet.
+
+- **An unchanged fingerprint froze the bug report's body, not just its comments.** Two
+  questions were sharing one answer. *Have the findings changed?* gates **comments** —
+  they notify, and a daily "still broken, same as yesterday" is what teaches everyone
+  to ignore the one that finally matters. *Is the body still true?* gates the **body** —
+  and editing an issue body notifies nobody on either forge, so keeping it honest is
+  free. Conflated, the report could never be corrected while the findings held steady,
+  and the fingerprint deliberately ignores everything that moves on its own — healthy
+  rows, the hardware-verified column, point releases, and how a row is labelled. The
+  `master` → `27-dev` relabel above would have reached the README and never the issue
+  anybody actually opens. The body is now rewritten whenever it is out of date (after
+  normalising line endings, so a forge round-tripping `\r\n` does not cause a rewrite
+  every run) and comments remain strictly a changelog of real changes.
+
+- **A change to the publisher did not re-run the check.** `compat.yml`'s `push:` paths
+  listed `tools/compat.py` but not `tools/compat_publish.py` — so the very commit that
+  taught the bot to refresh a stale report body triggered no run, and the report stayed
+  stale until the next scheduled one. A fix nobody runs is a fix nobody has.
+
+- **The compatibility bot filed a new duplicate bug report on every Gitea run.**
+  `find_issue()` skipped pull requests by testing for the *presence* of the
+  `pull_request` key. GitHub omits that key on a plain issue; Gitea sends it as
+  `null`. So on Gitea every issue was discarded as a PR, the lookup always came back
+  empty, and the bot took the "nothing filed yet" branch and opened a fresh report
+  each run — **nine copies on the canonical forge**, four of them filed *after* the
+  commit that was meant to stop precisely this. The mirror was fine, which is why it
+  went unnoticed: GitHub's payload shape is the one the filter was written against.
+
+  It is the same failure the anti-spam fix was written to prevent, moved from
+  comments to issues, and it survived because `find_issue` was the only function in
+  `compat_publish.py` with no test. It now has one, per forge, and the daily cron —
+  which had not yet run once — no longer accumulates a report a day.
+
+  The issue list is also requested with **both** paging parameters (`per_page` for
+  GitHub, `limit` for Gitea). Each forge ignores the other's, and Gitea's default page
+  is 30, so the lookup would have started missing the report again once the pile it
+  was creating grew past one page.
+
 ## v0.7.0 — 2026-07-14
 ### Added
 

NOTICE

@@ -0,0 +1,4 @@
+truenas-truecloud-patch
+
+Parts of this project were written with AI assistance (Claude); all of it is
+reviewed and tested before release.

README.md

@@ -1,5 +1,7 @@
 # truenas-truecloud-patch
 
+[![CI](https://git.arch.fyi/flan/truenas-truecloud-patch/actions/workflows/ci.yml/badge.svg)](https://git.arch.fyi/flan/truenas-truecloud-patch/actions)
+
 Extends TrueNAS SCALE's **TrueCloud Backup** to:
 
 - back up to **Backblaze B2 and any S3-compatible provider**, not just Storj;
@@ -59,9 +61,11 @@ If something is wrong, the reason is in `apply.log` — start at
 | --- | --- | --- | --- |
 | 24.10.2.4 | ok | ok | — |
 | 25.04.2.6 | ok | ok | — |
-| 25.10.4 | ok | ok | nested + providers; 252-snapshot recursive backup of /mnt/Tap, 18m |
+| 25.10.5 | ok | ok | — |
+| 24.10.2.5 _(unreleased)_ | ok | ok | — |
+| 25.10.6 _(unreleased)_ | ok | ok | — |
 | 26.0.0-BETA.3 _(unreleased)_ | ok | ok | — |
-| master _(unreleased)_ | **BROKEN** | **BROKEN** | — |
+| master _(27-dev)_ | **BROKEN** | **BROKEN** | — |
 
 | verdict | meaning |
 | --- | --- |
@@ -72,23 +76,44 @@ If something is wrong, the reason is in `apply.log` — start at
 "ok" means *the patch's assumptions hold*, checked automatically against iX's
 source. It does not mean a human ran a backup on it — that is the
 **Hardware-verified** column, which is filled in by hand and only by doing it.
+
+**`master` is not the next release.** iX branches each major off to its own
+`release/` line and master rolls straight on to the one after — so master is
+`27-dev` while 26 is still in beta. A **BROKEN** master means iX has changed
+something that will reach users *a major release from now*, not in the version you
+are about to install. Read the numbered rows for that.
+
+A row like `25.10.5 _(unreleased)_` is the next maintenance release: branched by iX,
+not tagged yet, and the very next thing a 25.10.4 box gets. It is checked precisely
+because it is the one unshipped ref that reaches real users without warning.
 <!-- END COMPAT MATRIX -->
 
 The table is **regenerated daily by CI** against iXsystems' actual middleware source
 — it is not a claim somebody typed once and forgot.
 
-**On TrueNAS 26:** the patch was run on a real TrueNAS **26.0.0-BETA.1** install — a
-274-snapshot recursive backup of a 292-dataset pool, followed by a byte-identical
-restore of a four-level-deep child dataset. The *Hardware-verified* column tracks the
-newest beta iX has tagged (currently BETA.3), so it does not carry that mark: a build
-nobody has actually run a backup on does not get credit for one.
+It is also **static analysis**: it proves the patch's assumptions still hold, which is
+a weaker claim than "a backup ran and a restore came back". For what has actually been
+run — which tasks, on which hardware, and the md5 of the file that came back — see
+[docs/verification.md](docs/verification.md).
 
-**TrueNAS 26: nested snapshots are not supported yet, and upgrading will not break
-you.** 26 rewrites `cloud_backup` and deletes the ZFS methods this module calls. On
-26 `apply.sh` finds that the assumptions no longer hold and **does not apply the
-module**: TrueNAS is left stock, B2/S3 keeps working, nested datasets are simply not
-covered, and the reason is named in `apply.log`. A broken backup is worse than a
-missing feature. Details: [How it works](docs/how-it-works.md#truenas-26).
+**TrueNAS 26 is supported** as of v0.7.0, and was verified on a real
+**26.0.0-BETA.1** install: a 274-snapshot recursive backup of a 292-dataset pool, and a
+byte-identical restore of a four-level-deep child dataset. The *Hardware-verified*
+column tracks the newest beta iX has tagged (currently BETA.3), so it does not carry
+that mark — a build nobody has actually run a backup on does not get credit for one.
+
+26 rewrites `cloud_backup` from async to synchronous and deletes the private ZFS
+methods this module used to call, so getting there took real work: the patch now
+injects the wrapper flavour that matches the installed middleware, reads dataset and
+snapshot lists **from ZFS rather than middleware** (whose queries hide TrueNAS's own
+datasets — 84 of 270 on a real pool, including live app data), and owns the snapshot
+sweep even when it stages nothing (26 decides `recursive` by a rule this patch does not
+share, and would otherwise orphan one snapshot per zvol on every run).
+
+**And if a future TrueNAS breaks it, you get a missing feature, not a broken backup.**
+`apply.sh` re-checks the patch's assumptions at every boot and **refuses to apply a
+module whose assumptions no longer hold** — TrueNAS is left stock, and the reason is
+named in `apply.log`. Details: [How it works](docs/how-it-works.md#truenas-26).
 
 ---
 
@@ -131,6 +156,17 @@ alert, which both take the newest plain `vX.Y.Z` tag. That is what lets debuggin
 happen in `-rc` tags instead of in your notification bell — see
 [Releasing](docs/releasing.md).
 
+**A second alert reports the patch not being loaded**, and this one you cannot
+turn off with `--no-update-alerts` — it is CRITICAL, hourly, and it means B2/S3
+backup tasks are about to fail. Being patched *on disk* and being patched *in the
+running middlewared* are different facts, and only middlewared can answer the
+second one: the patch stamps the objects it replaces, so a missing stamp means
+the process imported stock code. It fires if something detaches the patch overlay
+(a `systemd-sysext` merge over `/usr`, for instance) and the self-healing re-apply
+in the deferred restart could not put it back. `bash install.sh` clears it. It
+stays quiet when the kill switch is set, or when the providers module has been
+retired because TrueNAS went native.
+
 The changelog is read from whichever forge `origin` points at, derived from the
 remote rather than hard-coded. That is not cosmetic: when the changelog cannot be
 read, the alert deliberately fires **anyway** rather than risk hiding a security
@@ -213,5 +249,7 @@ and restores the original UI bundle from backup.
 - Filing a TrueNAS bug? **Remove the patch first** and reproduce on a stock system.
 - Provided as-is, no warranty. See LICENSE.
 
-Parts of this project were written with AI assistance (Claude); all of it is reviewed
-and tested before release. Bugs are mine.
+## Support
+
+If truenas-truecloud-patch is useful to you, consider supporting development via
+[GitHub Sponsors](https://github.com/sponsors/sudolulo) or [Ko-fi](https://ko-fi.com/sudolulo).

docs/how-it-works.md

@@ -74,22 +74,46 @@ Two different things must survive two different events:
    and creates a transient systemd unit (`truecloud-mw-restart`, via
    `systemd-run --no-block`) running `patch/wait_restart.sh` — detached so it
    cannot disrupt the remainder of the boot sequence.
-5. **Once boot has settled, middlewared restarts once** and imports the
-   patched modules from the overlay. `wait_restart.sh` holds the restart until
-   the systemd boot job queue has drained (so in-flight `ix-*` units like
-   `ix-reporting` finish first) *and* middlewared's docker/apps startup has
-   reached a terminal state — plain unit ordering cannot see either, and
-   restarting middlewared while they run kills apps and dashboard reporting
+5. **Once boot has settled, the patch is re-applied and middlewared restarts
+   once**, importing the patched modules from the overlay. `wait_restart.sh`
+   holds the restart until the systemd boot job queue has drained (so in-flight
+   `ix-*` units like `ix-reporting` finish first) *and* middlewared's docker/apps
+   startup has reached a terminal state — plain unit ordering cannot see either,
+   and restarting middlewared while they run kills apps and dashboard reporting
    for the whole boot. S3/B2 backup support is then active until the next
    reboot, when the cycle repeats.
 
+   The **re-apply** in that sentence is load-bearing, not a safety blanket. The
+   overlay from step 3 sits *inside* `/usr`, so anything that remounts that
+   hierarchy detaches it, and two ordinary things do exactly that after our hook
+   has finished: another PREINIT script running `systemd-sysext merge`/`refresh`
+   over `/usr` (an out-of-tree nvidia driver, say), and middlewared's own
+   `docker.configure_nvidia` when it brings docker up. Init scripts run
+   sequentially in id order, so a hook registered after ours always wins — and
+   ordering them differently would still not help, because `docker.configure_nvidia`
+   fires at runtime. `wait_restart.sh` therefore re-runs `apply.sh` at the point
+   where boot has settled and every such remount is behind it, re-mounting the
+   overlay if it was torn off (same upper layer, so files patched in step 3
+   reappear intact), then verifies the patch is really on the live path, restarts,
+   and verifies again — retrying once if it was lost in between.
+
+   This is the failure that made it necessary: on 2026-08-19 the overlay was
+   mounted at 16:41:56 and a sysext refresh unmerged and remerged `/usr` four
+   seconds later. The restart at 16:47:24 loaded stock modules, and every B2
+   backup failed for nineteen hours while `apply.log` said `OK` — because
+   `apply.log` can only report what was written to disk, never what the restart
+   imported. That second question is now asked from inside middlewared by an
+   hourly CRITICAL alert (see [Update alerts](../README.md#update-alerts)).
+
 What you will observe: one middlewared restart shortly after every boot (a
 brief web UI/API blip; running services are unaffected). Between steps 3
 and 5 there is a short window — typically well under a minute — where the UI
 already shows S3/B2 (the JS bundle is read from disk per request) but the
 backend is still stock. A backup job that fires inside that window fails once
 with `NotImplementedError` and succeeds on its next run; see
-[Troubleshooting](recovery.md) if it persists beyond boot.
+[Troubleshooting](recovery.md) if it persists beyond boot. If the backend is
+still stock an hour after boot, middlewared raises the "installed but NOT
+loaded" alert rather than leaving you to notice via a failed backup.
 
 Manual runs of `bash patch/apply.sh` never trigger the restart — that only
 happens in boot context. `install.sh` and `recover.sh` perform their own
@@ -187,23 +211,71 @@ backup-breaking**, and none of them is visible from the `cloud_backup` files:
 | `get_dataset_recursive()` **deleted** from `plugins/cloud/snapshot.py` | `NameError` — the injected block called it out of the host module's namespace |
 | `plugins/zfs_/dataset.py` and `zfs_/snapshot.py` **deleted** | `zfs.dataset.query`, `zfs.snapshot.query` and `zfs.snapshot.delete` all vanish. 26 uses `filesystem.statfs` and `zfs.resource.*` |
 
-The first two are fixed: the patch reads which flavour of `cloud_backup` your box
-declares and injects the wrapper that matches (one implementation of the real logic,
-two thin wrappers), and it carries its own copy of the deleted helper.
+All three are fixed as of **v0.7.0**, and 26 is supported.
 
-The third is **not** fixed, and is why 26 reports BROKEN. Porting it means rewriting
-the module's ZFS calls onto 26's new API, and no single API spans 24.10 through 26 —
-so it needs a real 26 box to verify against, not a plausible-looking diff. Shipping a
-port nobody has run is exactly the failure this project exists to avoid.
+The first two were straightforward: the patch reads which flavour of `cloud_backup`
+your box declares and injects the wrapper that matches (one implementation of the real
+logic, two thin wrappers), and it carries its own copy of the deleted helper.
 
-It is also the row that would have hurt most. `zfs.snapshot.delete` is what sweeps the
-recursive snapshot; without it, **every run would orphan one snapshot per descendant
-dataset — 250 on a real pool — forever.** The compatibility check caught it only
-because it now asserts the middleware *methods the patch calls*, not just the symbols
-it wraps.
+The third was not, and it is the one that would have hurt most — `zfs.snapshot.delete`
+is what sweeps the recursive snapshot, and without it **every run would orphan one
+snapshot per descendant dataset (250 on a real pool), forever, while reporting
+success.**
 
-`master` (development after 26) reports BROKEN too: iXsystems are still reshaping
+The obvious port is to the public `pool.dataset.query` / `pool.snapshot.query`. Those
+methods exist, are documented, and are covered by iX's deprecation policy — and they
+are **not like-for-like replacements**. They apply a *visibility policy*: they hide the
+datasets TrueNAS considers its own (`ix-apps/*`, `.system/*`, `.ix-virt/*`). On a real
+pool that is **84 of 270 datasets, including live application data.** Staging from that
+view would have omitted every one of them from the backup — and the planner would never
+have seen them, so they would not have appeared in its "skipped" list either. A green
+backup, quietly missing data. The snapshot query lies the same way, so the sweep would
+have orphaned one snapshot per hidden dataset.
+
+No source analysis could have caught that. The methods are all present and correctly
+shaped. Only running it could, which is why it took a real 26 box.
+
+So the module now follows one rule:
+
+> **Read the truth from ZFS. Make changes through middleware.**
+
+Enumeration is `zfs list` — no policy can filter it, and it behaves identically on every
+release, which also means one code path instead of a version conditional. Mutation stays
+a middleware call, so TrueNAS's own bookkeeping stays consistent; an exact-name delete
+works fine even on a dataset the query hides. It is only enumeration that lies.
+
+The snapshot *delete* still needs a namespace, and no single one spans every release —
+24.10 and 25.04 have `zfs.snapshot`, 26 has only `pool.snapshot`, 25.10 has both. So it
+is resolved at runtime, by asking whether the namespace can actually delete. `tools/compat.py`
+asks the identical question against iX's source, and a test binds the two lists together,
+so what CI verifies and what runs cannot drift apart.
+
+### The one 26 changed that nothing warned about
+
+Stock decides whether to take a **recursive** snapshot by its own rule, and on 26 that
+rule stopped being ours:
+
+| | decides `recursive` by |
+| --- | --- |
+| stock ≤ 25.10 | `get_dataset_recursive()` — the same function this patch vendors |
+| **stock 26** | `filesystem.statfs`: `recursive = (path == the dataset's mountpoint)` |
+| this patch | `get_dataset_recursive()` — is a mounted *filesystem* child under the path? |
+
+Up to 25.10 those were the *same question*, so a snapshot the patch declined to stage
+provably had no children and stock's non-recursive delete was correct. On 26 they
+disagree: a dataset whose only descendants are **zvols** or **legacy-mountpoint**
+datasets gets a recursive snapshot, while the patch sees nothing to stage. Stock then
+destroys the parent only — and with no staging tree there was no sidecar, and the
+garbage collector only ever ran from the staging path. Nothing on the box would ever
+have found the children.
+
+It was reproduced on a 26 VM (one orphan per zvol, every run, backup green) and closed:
+**ownership of the sweep is no longer conditional on staging.**
+
+`master` (development after 26) **does** report BROKEN: iXsystems are still reshaping
 these functions there, renaming `middleware` → `context` and `cloud_backup` → `entry`
 and adding a required `credentials` parameter. That is a moving target and is
 deliberately not chased; the check keeps reporting it until it settles into a beta,
-which is when it becomes worth fixing.
+which is when it becomes worth fixing. Until then, a box running master would simply
+not get the modules — `apply.sh` refuses to apply a module whose assumptions no longer
+hold, and says why in `apply.log`.

docs/recovery.md

@@ -120,8 +120,9 @@ If a module shows `[FAIL]`:
 
 The traceback ends in `rclone/base.py` → `raise NotImplementedError` and
 contains no `_tc_` frames: the running middlewared is executing stock code.
-Either the deferred restart never fired, or the patch never landed on disk
-this boot. Diagnose in this order:
+Either the deferred restart never fired, the patch never landed on disk this
+boot, or it landed and was then torn off before the restart. Diagnose in this
+order:
 
 ```bash
 # Did apply.sh run this boot, at which version, and did it schedule the restart?
@@ -130,11 +131,21 @@ tail -40 /mnt/tank/truenas-truecloud-patch/apply.log
 # Full check — compares the running process against the patch timestamp
 python3 /mnt/tank/truenas-truecloud-patch/patch/create_task.py verify
 
-# Did the deferred restart unit run, fail, or never get created?
-systemctl status truecloud-mw-restart.service
+# What the deferred restart did -- re-apply, restart, and what it verified.
+# apply.log is the durable record; journald retention on a busy box is often
+# shorter than the gap between reboots, so the journal may have nothing left.
+grep wait_restart /mnt/tank/truenas-truecloud-patch/apply.log | tail -20
 journalctl -u truecloud-mw-restart.service --no-pager | tail -20
+
+# Did something remount /usr and detach the patch overlay?
+systemd-sysext status
+findmnt -o TARGET,SOURCE /usr/lib/python3/dist-packages
 ```
 
+`systemctl status truecloud-mw-restart.service` reporting *"could not be
+found"* is **normal** — the unit is transient and is collected once it exits.
+It is not evidence that the restart was skipped.
+
 - `verify` reports the process started **before** the patch → the restart
   didn't happen. `systemctl restart middlewared` fixes it immediately; the
   journal output above tells you why it was missed.
@@ -145,6 +156,22 @@ journalctl -u truecloud-mw-restart.service --no-pager | tail -20
 - `apply.log` header shows `[v0.0.3]` or older → update:
   `git pull && bash install.sh` (v0.0.4 fixed patches not loading after
   reboot).
+- `apply.log` says the patch applied, but `findmnt` shows no `truecloud-mw`
+  overlay on the dist-packages path → something remounted `/usr` after our
+  PREINIT hook and detached it. `systemd-sysext status` names the culprit if it
+  is a sysext (the `SINCE` column will sit a few seconds *after* the `apply.log`
+  timestamp). Releases from 2026-08-26 on re-apply and verify immediately before
+  the restart, so this should self-heal; if you are seeing it, update first.
+
+**TrueNAS raises "truecloud-patch is installed but NOT loaded"**
+
+The definitive symptom, and it does not depend on a backup failing first: the
+running middlewared has stock cloud_backup modules even though the patch is
+installed and its providers module is meant to be active. `bash install.sh`
+re-applies and restarts. The alert clears within the hour. It is silent when the
+kill switch is set or the providers module has been retired as native, and it is
+deliberately not muted by `update_alerts_disabled` — that silences release
+notifications, not a broken backup path.
 
 **Apply log** (check after each reboot or install):
 ```bash

docs/releasing.md

@@ -25,7 +25,7 @@ are worth calling out, because nothing else would catch what they catch:
   newest CHANGELOG entry. `VERSION=` had silently drifted to three different
   values across the scripts before anything checked.
 
-The project is hosted on **Gitea** (`git.onetick.ninja/flan/truenas-truecloud-patch`)
+The project is hosted on **Gitea** (`git.arch.fyi/flan/truenas-truecloud-patch`)
 and mirrored to GitHub. Both run the same workflows — Gitea reads
 `.github/workflows/` too — so a change is checked twice, on two independent runners.
 

docs/verification.md

@@ -0,0 +1,77 @@
+# What has actually been run
+
+The support matrix in the README is **static analysis**: it proves the patch's
+assumptions about middlewared still hold. That is a strictly weaker claim than "a
+backup ran and a restore came back". This file is the stronger claim, and it is
+maintained by hand, because the only way to fill it in is to do it.
+
+If you are deciding whether to trust this with your backups, read this file, not the
+matrix.
+
+---
+
+## v0.7.0 — TrueNAS 25.10.4 (production hardware)
+
+Six live TrueCloud tasks, all `snapshot = true`, backing up to Backblaze B2. Three were
+exercised end to end, chosen to cover the three shapes the code handles differently:
+
+| Task | Path | Shape | Result |
+| --- | --- | --- | --- |
+| 5 | `/mnt/Tap` | 191 nested datasets staged, 282-snapshot recursive tree | SUCCESS |
+| 7 | `/mnt/Tank/backups` | 215 filesystems **+ 2 zvols** | SUCCESS |
+| 9 | `/mnt/Tank/flan` | **no** nested filesystem children | SUCCESS |
+
+After every run: **0 orphaned snapshots, 0 leaked bind mounts, 0 stale sidecars.**
+
+**The restore.** `apps/vaultwarden/data/config.json` — a file inside a *child* dataset,
+which is exactly what stock TrueNAS cannot capture — was restored from B2 and compared
+against the live file:
+
+    live      f809df6ba231986b1ba824044228a03a   1808 bytes
+    restored  f809df6ba231986b1ba824044228a03a   1808 bytes
+    => byte-identical
+
+**The collector earned its keep on real data.** The pool was already carrying an orphan:
+`Tap/apps/prometheus@cloud_backup-5-20260713202355`, left behind by an earlier run when
+ZFS's automount held the snapshot busy past all four retries. The first v0.7.0 run found
+it by name, reclaimed it, and the pool's snapshot count went 2148 → 2147. That is the
+garbage collector doing the job it was written for, against a leak that was already
+there and that nothing else would ever have found.
+
+**Boot path.** `apply.sh` is registered as a PREINIT `initshutdownscript`; it was
+re-run against the live middleware and left exactly one `TRUECLOUD_PATCH` marker in
+each patched module (a second copy stacked into a live middlewared module would break
+the box at boot). It correctly detected the box as **async** (`cloud_backup is async
+(TrueNAS <= 25.10)`) and injected the matching wrappers.
+
+**Upgrade path.** `update.sh` was used to move the box from the release candidate to
+the stable tag, in detached HEAD at `v0.7.0`, which is how a user's box actually
+upgrades.
+
+## v0.7.0 — TrueNAS 26.0.0-BETA.1 (VM)
+
+A throwaway VM whose pool reproduces the production pool's *shape* — 292 datasets, 26
+`legacy` mountpoints, nesting five deep — because every bug found on the real box came
+from the shape of the pool, not the bytes in it. MinIO was not used; `rclone serve s3`
+(already on the box) provided the S3 target, so no real B2 credential ever entered the
+VM.
+
+* 274-snapshot recursive backup of the 292-dataset pool. 0 orphans, 0 leaked mounts.
+* Restored `ix-apps/app_mounts/vaultwarden/pgData` — **four levels deep, and a dataset
+  that middleware's own `pool.dataset.query` hides from itself** — byte-identical.
+* The zvol-orphan case was **reproduced with the fix disabled** (one orphan per zvol,
+  every run, backup green), then **closed with it enabled**. See the CHANGELOG entry
+  for why TrueNAS 26 decides `recursive` by a different rule than this patch decides
+  `nested`.
+
+## What is NOT covered
+
+* **24.10 and 25.04** are `ok` in the matrix — the assumptions hold, checked against
+  iX's source — but nobody has run a backup on them. The matrix says so.
+* **master** is BROKEN, and correctly reports so: iX renamed the leading parameters of
+  `get_restic_config` and `restic_backup`. It is not a shipped release; the daily
+  compatibility bot files it, and `apply.sh` would refuse to apply the modules on a box
+  running it.
+* A **reboot** of the production box has not been done on v0.7.0. `apply.sh` was
+  re-executed by hand against the live middleware, which exercises the same code path,
+  but the PREINIT ordering itself has only been proven on earlier versions.

install.sh

@@ -18,7 +18,7 @@
 
 set -euo pipefail
 
-VERSION="0.7.0"
+VERSION="0.8.0"
 
 # The directory containing install.sh is the permanent install location.
 PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"

patch/alert_source.py

@@ -21,6 +21,7 @@ the way a `git fetch` from middlewared (running as root) would.
 
 import datetime
 import importlib.util
+import json
 import logging
 import os
 import re
@@ -47,8 +48,8 @@ _VERSION_RE = re.compile(r'^VERSION="([^"]+)"', re.M)
 #: owner/repo out of any of:
 #:   git@github.com:sudolulo/repo.git
 #:   https://github.com/sudolulo/repo.git
-#:   ssh://git@git.onetick.ninja:55214/flan/repo.git
-#:   https://git.onetick.ninja/flan/repo.git
+#:   ssh://git@git.arch.fyi:55214/flan/repo.git
+#:   https://git.arch.fyi/flan/repo.git
 #: The SSH port is deliberately not captured: it is not the web port.
 _REMOTE_RE = re.compile(
     r"^(?:\w+://)?(?:[^@/]+@)?([^:/]+)(?::\d+)?[:/]([^/]+)/([^/]+?)(?:\.git)?/?$"
@@ -77,6 +78,107 @@ class TrueCloudPatchSecurityUpdateAlertClass(AlertClass):
     )
 
 
+class TrueCloudPatchNotLoadedAlertClass(AlertClass):
+    category = AlertCategory.SYSTEM
+    level = AlertLevel.CRITICAL
+    title = "truecloud-patch is installed but NOT loaded"
+    text = (
+        "truecloud-patch patched middlewared on disk, but this middlewared is "
+        "running the STOCK cloud_backup modules -- B2 and S3 TrueCloud Backup "
+        "tasks will fail with NotImplementedError. Something remounted /usr "
+        "after the patch was applied (a systemd-sysext merge, or "
+        "docker.configure_nvidia), detaching the patch overlay. Re-apply with:  "
+        "bash %(dir)s/install.sh"
+    )
+
+
+class TrueCloudPatchNotLoadedAlertSource(ThreadedAlertSource):
+    """Does the middlewared running this check actually have the patch in it?
+
+    This is the one question apply.log cannot answer. apply.sh reports what it
+    wrote to disk; whether the restart that followed imported those files is a
+    separate fact, and on 2026-08-19 the two disagreed silently for nineteen
+    hours while every B2 backup task failed. Asking from inside the process is
+    exact -- the patch stamps the objects it replaces, so a missing stamp means
+    this interpreter imported stock code.
+
+    Deliberately NOT silenced by the update-alert marker: that mutes release
+    notifications, not a broken backup path. Only the patch's own kill switch
+    (the `disabled` file, meaning the operator turned the patch off) stops it.
+    """
+
+    schedule = IntervalSchedule(datetime.timedelta(hours=1))
+    run_on_backup_node = False
+
+    def check_sync(self):
+        try:
+            return self._check()
+        except Exception:
+            # An alert source must never take middlewared down with it.
+            logger.debug("truecloud-patch loaded check failed", exc_info=True)
+            return None
+
+    # -- internals ------------------------------------------------------------
+
+    def _check(self):
+        if os.path.exists(os.path.join(PATCH_DIR, "disabled")):
+            return None
+
+        # Only the providers module puts B2/S3 on the restic path. If it was
+        # never applied here, or TrueNAS went native and it was retired, then
+        # "not loaded" is the correct state and not a fault.
+        status = self._hook_status()
+        if not status:
+            return None
+        providers = status.get("patches", {}).get("providers", {})
+        if not providers.get("active"):
+            return None
+
+        if self._providers_loaded():
+            return None
+
+        return Alert(
+            TrueCloudPatchNotLoadedAlertClass,
+            {"dir": PATCH_DIR},
+            key=None,
+        )
+
+    def _hook_status(self):
+        try:
+            with open(os.path.join(PATCH_DIR, "hook_status.json")) as f:
+                return json.load(f)
+        except (OSError, ValueError):
+            return None
+
+    def _providers_loaded(self):
+        """True when THIS interpreter holds the patched provider objects.
+
+        Two independent stamps, because the two halves are written separately
+        and either can be missing on its own:
+
+        * restic.py -- apply.sh sets `_truecloud_patched` on the wrapper it
+          installs over `get_restic_config`.
+        * b2.py -- apply.sh binds a B2-specific `get_restic_config` onto
+          `B2RcloneRemote`. Comparing it against the base implementation is
+          exact and survives renames of the patch's own helper.
+        """
+        try:
+            from middlewared.plugins.cloud_backup.restic import get_restic_config
+        except Exception:
+            return False
+        if not getattr(get_restic_config, "_truecloud_patched", False):
+            return False
+
+        try:
+            from middlewared.rclone.base import BaseRcloneRemote
+            from middlewared.rclone.remote.b2 import B2RcloneRemote
+        except Exception:
+            return False
+        base = getattr(BaseRcloneRemote, "get_restic_config", None)
+        b2 = getattr(B2RcloneRemote, "get_restic_config", None)
+        return b2 is not None and b2 is not base
+
+
 class TrueCloudPatchUpdateAlertSource(ThreadedAlertSource):
     schedule = IntervalSchedule(datetime.timedelta(hours=24))
     run_on_backup_node = False

patch/apply.sh

@@ -32,7 +32,7 @@
 # Derive PATCH_DIR from this script's location (parent of the patch/ directory).
 PATCH_DIR="$(cd "$(dirname "$0")/.." && pwd)"
 LOG="$PATCH_DIR/apply.log"
-VERSION="0.7.0"
+VERSION="0.8.0"
 
 # Rotate log at 512 KB to avoid unbounded growth on a system volume.
 # Keep two prior generations (.1 and .2) so the last three boots are always available.
@@ -64,17 +64,45 @@ _ensure_writable() {
         rm -f "$dir/.truecloud-probe"
         return 0
     fi
-    # Already our overlay on this exact directory from an earlier run this boot?
+    # Not writable, so any overlay of ours listed on this directory is a
+    # SHADOWED leftover rather than a working mount: something remounted the
+    # hierarchy above it -- a systemd-sysext merge/refresh over /usr, or
+    # middlewared's own docker.configure_nvidia -- and buried it. A live
+    # overlay of ours is always writable, so this must never be treated as
+    # "already done"; doing so is what let a buried overlay pass for a healthy
+    # one and left the backend patch on disk but never loaded.
     if mount | grep -qF "truecloud-${tag} on ${dir} "; then
-        return 0
+        echo "NOTICE: a previous truecloud-${tag} overlay on $dir is shadowed --"
+        echo "NOTICE: the hierarchy above it was remounted. Detaching and re-mounting."
+        umount -l "$dir" 2>/dev/null
     fi
+    # Keep the SAME upperdir across re-mounts: it holds everything patched
+    # earlier this boot, so re-mounting restores those files intact instead of
+    # re-deriving them. The workdir is scratch and must be empty, so it is
+    # recreated -- a stale one left behind by a detached mount fails the mount.
     local upper="/run/truecloud-${tag}-upper" work="/run/truecloud-${tag}-work"
-    mkdir -p "$upper" "$work"
+    mkdir -p "$upper"
+    rm -rf "$work" 2>/dev/null
+    mkdir -p "$work"
     if mount -t overlay "truecloud-${tag}" \
            -o "lowerdir=$dir,upperdir=$upper,workdir=$work" "$dir" 2>/dev/null; then
         echo "OK: Mounted writable overlay on $dir"
         return 0
     fi
+    # A lazily-detached overlay releases its workdir only once its last user is
+    # gone, and overlayfs refuses a workdir that is still in use. That would turn
+    # the re-mount this function exists to perform into a hard failure, so retry
+    # once on a private workdir. It is scratch in /run (tmpfs) and goes away at
+    # the next boot; the upperdir, which holds the patched files, is unchanged.
+    work="/run/truecloud-${tag}-work.$$"
+    rm -rf "$work" 2>/dev/null
+    mkdir -p "$work"
+    if mount -t overlay "truecloud-${tag}" \
+           -o "lowerdir=$dir,upperdir=$upper,workdir=$work" "$dir" 2>/dev/null; then
+        echo "OK: Mounted writable overlay on $dir (fresh workdir)"
+        return 0
+    fi
+    rmdir "$work" 2>/dev/null
     echo "WARNING: overlay mount failed on $dir — backend patch will be skipped."
     return 1
 }
@@ -196,6 +224,13 @@ _tc_native_nested=$(printf '%s' "$_tc_info" | sed -n '2p')
 SITE_PKG=$(printf '%s' "$_tc_info"          | sed -n '3p')
 _MW_DIR=$(printf '%s' "$_tc_info"           | sed -n '4p')
 
+# Record the resolved middlewared directory so patch/wait_restart.sh can check,
+# without re-deriving any of this, whether the patched modules are still on the
+# live filesystem path at the moment it restarts middlewared.
+if [ -n "$_MW_DIR" ]; then
+    printf '%s\n' "$_MW_DIR" > "$PATCH_DIR/.mw_dir" 2>/dev/null
+fi
+
 # Nested support is opt-in; if it was never enabled, it cannot be the reason to
 # keep the patch alive.
 if [ -f "$PATCH_DIR/nested_snapshots_enabled" ]; then
@@ -999,8 +1034,13 @@ fi
 # runs (install.sh, recovery) never trigger a restart.
 #
 # The unit runs wait_restart.sh, which blocks until boot has actually
-# settled (systemd job queue drained, docker/apps state terminal) before
-# restarting. systemd ordering alone (After=multi-user.target, ≤ v0.0.4)
+# settled (systemd job queue drained, docker/apps state terminal), then
+# RE-APPLIES this script before restarting. The re-apply is not belt-and-
+# braces: our overlay lives inside /usr, and a systemd-sysext merge or
+# middlewared's docker.configure_nvidia remounts /usr *after* PREINIT and
+# detaches it, so what we patch here can be gone by restart time (seen
+# 2026-08-19). wait_restart.sh re-mounts and re-verifies at the moment it
+# matters. systemd ordering alone (After=multi-user.target, ≤ v0.0.4)
 # fired while ix-reporting and the docker/apps startup were still in flight
 # and killed both — apps and dashboard stats stayed down until the next
 # boot. No Type=oneshot: a oneshot's start job would hold the boot queue
@@ -1015,7 +1055,12 @@ echo "--- deferred restart ---"
 #
 # "No module active at all" cannot reach here: that is the kill-switch branch
 # above, which exits.
-if ! grep -aq middlewared "/proc/$PPID/cmdline" 2>/dev/null; then
+if [ "${TRUECLOUD_REAPPLY:-0}" = "1" ]; then
+    # Invoked by patch/wait_restart.sh as its pre-restart re-apply pass. That
+    # unit already exists to do the restart and verifies the result, so
+    # scheduling another one here would be a loop.
+    echo "Re-apply pass from wait_restart.sh — that unit owns the restart."
+elif ! grep -aq middlewared "/proc/$PPID/cmdline" 2>/dev/null; then
     echo "Manual run (parent is not middlewared) — no restart scheduled."
 elif [ "$_backend_ok" != "1" ]; then
     echo "Nothing landed on disk — no restart scheduled (nothing new to load)."

patch/create_task.py

@@ -52,7 +52,7 @@ import subprocess
 import sys
 import time
 
-__version__ = "0.7.0"
+__version__ = "0.8.0"
 
 _PATCH_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 _STATUS_FILE = os.path.join(_PATCH_DIR, "hook_status.json")

patch/wait_restart.sh

@@ -27,6 +27,50 @@
 # --wait` below waits for that same queue to drain — the unit would deadlock
 # on itself until the timeout. apply.sh schedules this with the default
 # service type, whose start job completes at fork.
+#
+# THE RE-APPLY PASS (added 2026-08-26). Applying the patch at PREINIT and
+# restarting later is only sound if the patched files are still on the live
+# path at the moment middlewared re-imports them. They may not be: our patch
+# lives in an overlay mounted *inside* /usr, and anything that remounts the
+# hierarchy above it detaches or buries that overlay. Two things on a normal
+# TrueNAS box do exactly that, both AFTER our PREINIT hook has run:
+#
+#   - `systemd-sysext merge/refresh` over /usr (an nvidia sysext, for
+#     instance) — `Unmerged '/usr'` then `Merged extensions into '/usr'`;
+#   - middlewared's own `docker.configure_nvidia`, which merges the stock
+#     nvidia sysext over /usr when it brings docker up.
+#
+# PREINIT scripts run sequentially in id order, so a hook registered after
+# ours always wins the race, silently. Observed 2026-08-19: our overlay was
+# mounted at 16:41:56 and a sysext refresh tore /usr down four seconds later;
+# the restart at 16:47:24 then loaded stock modules and every B2 cloud_backup
+# job failed for the next nineteen hours while apply.log said "OK".
+#
+# Ordering the hooks cannot fix this — docker.configure_nvidia re-merges at
+# runtime, long after every PREINIT hook is done. So instead of trusting the
+# PREINIT pass, re-apply immediately before the restart (apply.sh is
+# idempotent and re-mounts a lost overlay, keeping the same upperdir so
+# already-patched files survive), verify the marker is really on the live
+# path, and verify again afterwards.
+
+PATCH_DIR="$(cd "$(dirname "$0")/.." && pwd)"
+LOG="$PATCH_DIR/apply.log"
+
+_log() { echo "[wait_restart] $*" >> "$LOG" 2>/dev/null; }
+
+# Is the providers patch visible on the live filesystem path -- i.e. would a
+# middlewared starting right now import it? Reads the marker apply.sh leaves
+# in restic.py. Returns 0 when patched, 1 when stock, 2 when we cannot tell
+# (no recorded middlewared dir yet, or the file is gone).
+_patch_visible() {
+    local mw_dir restic_py
+    mw_dir=$(cat "$PATCH_DIR/.mw_dir" 2>/dev/null)
+    [ -n "$mw_dir" ] || return 2
+    restic_py="$mw_dir/plugins/cloud_backup/restic.py"
+    [ -f "$restic_py" ] || return 2
+    grep -q "TRUECLOUD_PATCH" "$restic_py" 2>/dev/null && return 0
+    return 1
+}
 
 # 1. systemd layer: wait for the boot job queue to drain. This covers every
 #    ix-* oneshot still activating, including ix-reporting's in-flight midclt
@@ -39,6 +83,9 @@ timeout 900 systemctl is-system-running --wait > /dev/null 2>&1
 #    transitional states (PENDING/INITIALIZING/STOPPING/MIGRATING — see
 #    middlewared/plugins/docker/state_utils.py). An empty answer means
 #    midclt could not respond at all; keep waiting. Cap at 10 minutes.
+#    This also covers docker.configure_nvidia, the runtime /usr re-merge:
+#    waiting for docker to reach a terminal state means the merge that would
+#    bury our overlay has already happened by the time we re-apply below.
 for _ in $(seq 1 120); do
     _status=$(midclt call docker.status 2>/dev/null \
                   | grep -oE '"status": "[A-Z_]+"' | cut -d'"' -f4)
@@ -52,4 +99,70 @@ done
 #    queryable state (smb.configure and friends). Bounded insurance.
 sleep 30
 
-exec systemctl try-restart middlewared
+# 4. Re-apply pass. Boot has settled, so every sysext merge and docker nvidia
+#    configuration that could bury our overlay is behind us. Re-running
+#    apply.sh is cheap and idempotent: it re-mounts the overlay if it was
+#    detached (same upperdir, so files patched at PREINIT reappear intact)
+#    and re-patches anything that reverted to stock.
+_patch_visible
+case $? in
+    0) _log "providers patch still visible on the live path before restart" ;;
+    1) _log "PATCH LOST since PREINIT (something remounted /usr) — re-applying" ;;
+    *) _log "cannot confirm patch state before restart — re-applying anyway" ;;
+esac
+
+TRUECLOUD_REAPPLY=1 /bin/bash "$PATCH_DIR/patch/apply.sh"
+
+if ! _patch_visible; then
+    _log "WARNING: patch is STILL not on the live path after the re-apply pass;"
+    _log "WARNING: restarting anyway, but middlewared will load stock modules."
+fi
+
+# 5. The restart itself.
+systemctl try-restart middlewared
+
+# 6. Record what the restart landed on -- but do NOT restart again on a miss.
+#
+# `try-restart` returns as soon as middlewared is READY; it then brings docker
+# up asynchronously, and `docker.configure_nvidia` merges the stock nvidia
+# sysext over /usr at that point. That detaches our overlay AFTER the new
+# middlewared has already imported the patched modules -- so a disk check here
+# can report "missing" on a perfectly healthy system. Restarting on that signal
+# would restart a correctly-patched middlewared and then hit the same race
+# again, so the disk is deliberately not treated as a verdict after the restart.
+#
+# The authoritative answer is whether the running process holds the patch, and
+# only middlewared can answer that. The alert source installed by apply.sh
+# checks exactly that, in-process and hourly, and is what reports a genuine
+# miss. What is still worth doing here is putting the overlay back, so the next
+# middlewared restart -- whenever and whyever it happens -- finds patched files.
+if _patch_visible; then
+    _log "OK: providers patch present on the live path across the restart"
+else
+    _log "overlay detached again after the restart (expected when docker's"
+    _log "nvidia sysext merge follows it) -- re-mounting for the next restart."
+    _log "Whether THIS middlewared loaded the patch is answered in-process by"
+    _log "the 'installed but NOT loaded' alert, not by this check."
+
+    # Preserve hook_status.json's patched_at across this re-mount.
+    #
+    # create_task.py verify decides "loaded" by comparing middlewared's start
+    # time against patched_at. This re-apply restores the SAME patch the boot
+    # pass already applied, but it runs *after* the restart -- so letting it
+    # re-stamp would make patched_at newer than the process that correctly
+    # imported the patch, and verify would report FAIL forever, on every boot
+    # where docker's sysext merge detaches the overlay. That is precisely the
+    # lying-status failure this release exists to remove, so do not introduce a
+    # new one. The snapshot lives in /run, never in the repo: a leftover file
+    # there would leave the tree dirty and update.sh refuses to run over that.
+    _saved_status=/run/truecloud-hook_status.pre
+    cp -p "$PATCH_DIR/hook_status.json" "$_saved_status" 2>/dev/null
+
+    TRUECLOUD_REAPPLY=1 /bin/bash "$PATCH_DIR/patch/apply.sh"
+
+    if [ -f "$_saved_status" ]; then
+        mv -f "$_saved_status" "$PATCH_DIR/hook_status.json" 2>/dev/null
+    fi
+fi
+
+_log "=== deferred restart complete ==="

recover.sh

@@ -17,7 +17,7 @@
 #   bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh
 #   systemctl restart middlewared
 
-VERSION="0.7.0"
+VERSION="0.8.0"
 
 PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
 

tests/test_compat.py

@@ -15,6 +15,7 @@ dangerous thing this file can say -- it means "TrueNAS does this now, retire the
 module" -- and it rests on nothing more than a substring match.
 """
 
+import json
 import os
 import sys
 
@@ -23,6 +24,7 @@ import pytest
 sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "tools"))
 
 import compat  # noqa: E402
+import compat_publish  # noqa: E402
 from compat import (  # noqa: E402
     NESTED,
     PROVIDERS,
@@ -574,3 +576,233 @@ class TestATransientNetworkBlipDoesNotWakeAnybody:
             "def get_restic_config(entry, credentials):\n    pass\n"
         )
         assert compat.fingerprint(self._rows(worse)) != compat.fingerprint(self._rows(broken))
+
+
+class TestTheBotFindsItsOwnIssueOnBOTHForges:
+    """`find_issue` decides "have I already filed this?" -- and it ran on two forges.
+
+    It used to skip pull requests with `"pull_request" not in i`. GitHub omits that key
+    on a plain issue; **Gitea sends it as `null`**. So on Gitea every issue looked like
+    a PR, the match list was always empty, and the bot took the "nothing filed yet"
+    branch on EVERY run: nine duplicate copies of the same report on the canonical
+    forge, four of them filed after the commit that was supposed to stop exactly this.
+
+    It is the same failure the spam fix was written to prevent, moved from comments to
+    issues -- and it survived because `find_issue` was the one function here with no
+    test. So the payload shapes are pinned, per forge, by hand.
+    """
+
+    TITLE = compat_publish.TITLE
+
+    def _find(self, monkeypatch, payload):
+        monkeypatch.setattr(compat_publish, "_call", lambda *a, **k: payload)
+        return compat_publish.find_issue("https://forge/api", "tok", self.TITLE)
+
+    def test_gitea_sends_pull_request_as_null_and_the_issue_is_still_found(self, monkeypatch):
+        found = self._find(monkeypatch, [
+            {"number": 7, "title": self.TITLE, "state": "open", "pull_request": None},
+            {"number": 1, "title": self.TITLE, "state": "open", "pull_request": None},
+        ])
+        assert found is not None, (
+            "find_issue missed a Gitea issue, so the bot files a NEW duplicate report "
+            "every run -- which is how nine of them piled up"
+        )
+        assert found["number"] == 1, "lowest-numbered wins"
+
+    def test_github_omits_the_key_entirely_and_the_issue_is_still_found(self, monkeypatch):
+        found = self._find(monkeypatch, [
+            {"number": 2, "title": self.TITLE, "state": "open"},
+        ])
+        assert found is not None and found["number"] == 2
+
+    def test_a_real_PR_with_the_same_title_is_still_skipped_on_both(self, monkeypatch):
+        # The reason the filter exists at all: both forges list PRs on /issues, and
+        # commenting on a PR instead of the bug report would be worse than useless.
+        assert self._find(monkeypatch, [
+            {"number": 3, "title": self.TITLE, "state": "open",           # Gitea PR
+             "pull_request": {"merged": False}},
+            {"number": 4, "title": self.TITLE, "state": "open",           # GitHub PR
+             "pull_request": {"url": "https://api.github.com/..."}},
+        ]) is None
+
+    def test_an_unrelated_issue_is_not_mistaken_for_the_report(self, monkeypatch):
+        assert self._find(monkeypatch, [
+            {"number": 1, "title": "TypeError when create B2 backup on Electric Eel",
+             "state": "closed", "pull_request": None},
+        ]) is None
+
+
+class TestTheNextMaintenanceReleaseIsChecked:
+    """`release/25.10.5` fell through every sieve, and it is the one that reaches users.
+
+    Shipped versions come from `TS-*` TAGS; unreleased ones come from `release/*`
+    BRANCHES that carry `-BETA`/`-RC`. A branched-but-untagged MAINTENANCE release is
+    neither: no tag, and its line (25.10) has already shipped, so the "prereleases of
+    a shipped line are history" filter threw it out. It was invisible.
+
+    That is backwards. `release/24.10-RC.2` is history -- nobody can install it. But
+    `release/25.10.5` is the FUTURE of a shipped line: it is what a 25.10.4 box gets
+    on its next update. A break there ships to real users before the daily check has
+    ever looked at it.
+    """
+
+    TAGS = ["TS-24.10.2.4", "TS-25.04.2.6", "TS-25.10.4"]
+    HEADS = [
+        "release/25.10.4.1",
+        "release/25.10.5",          # branched, untagged -- the next maintenance release
+        "release/24.10-RC.2",       # history: its line shipped long ago
+        "release/25.20.2.2",        # iX's typo branch: 25.20 is not a TrueNAS version
+        "release/26.0.0-BETA.3",
+        "master",
+    ]
+
+    def _refs(self, monkeypatch):
+        monkeypatch.setattr(
+            compat, "_ls_remote",
+            lambda remote, what: self.TAGS if what == "--tags" else self.HEADS)
+        return compat.discover_refs("origin")
+
+    def test_the_next_maintenance_release_is_checked(self, monkeypatch):
+        assert "release/25.10.5" in self._refs(monkeypatch), (
+            "the next thing a 25.10.4 box updates to is not checked, so a break in it "
+            "reaches users before the bot ever sees it"
+        )
+
+    def test_a_superseded_maintenance_branch_is_not(self, monkeypatch):
+        # 25.10.4.1 sorts OLDER than the newest tag TS-25.10.4? No -- it is NEWER, and
+        # both are on the 25.10 line, so only the newest branch on the line is taken.
+        refs = self._refs(monkeypatch)
+        assert "release/25.10.4.1" not in refs, "only the newest branch per line"
+
+    def test_the_typo_branch_stays_out(self, monkeypatch):
+        # 25.20 has no TS tag, so it is not a release line at all. A typo branch in the
+        # matrix reads as a real supported release we are silently broken on.
+        assert "release/25.20.2.2" not in self._refs(monkeypatch)
+
+    def test_a_prerelease_of_an_already_shipped_line_stays_out(self, monkeypatch):
+        assert "release/24.10-RC.2" not in self._refs(monkeypatch)
+
+    def test_an_untagged_branch_counts_as_UNRELEASED(self, monkeypatch):
+        # The exit code keys off this. Calling 25.10.5 "shipped" would fail the build
+        # as a live outage on a version nobody is running yet.
+        assert compat.is_unreleased("release/25.10.5")
+        assert compat.is_unreleased("master")
+        assert not compat.is_unreleased("TS-25.10.4")
+
+
+class TestMasterIsNotTheNextRelease:
+    """A red `master` row used to read as "the version you are about to install".
+
+    On 2026-07-14 master was 27-dev -- every recent commit targeted 27.0.0-BETA.1 --
+    while 26 was still in beta on its own branches. So `master BROKEN` meant "iX will
+    break us a major release from now", but the matrix said "master _(unreleased)_",
+    which any reader takes as the next thing out the door. For a table whose whole job
+    is helping somebody decide whether to trust this with their backups, that is a
+    false alarm in the worst possible place.
+    """
+
+    def _rows(self, refs):
+        return [{"ref": r, "unreleased": compat.is_unreleased(r), "modules": {}}
+                for r in refs]
+
+    def test_master_is_labelled_with_the_major_AFTER_the_newest_known_one(self):
+        rows = self._rows(["TS-25.10.4", "release/26.0.0-BETA.3", "master"])
+        assert compat.dev_label(rows) == "27-dev"
+
+    def test_it_rolls_over_on_its_own_when_the_next_beta_branches(self):
+        # Derived, not hardcoded: when release/27.0.0-BETA.1 appears, master is 28-dev.
+        rows = self._rows(["TS-26.0.0", "release/27.0.0-BETA.1", "master"])
+        assert compat.dev_label(rows) == "28-dev"
+
+    def test_the_rendered_matrix_says_dev_not_unreleased(self):
+        healthy = check_files(with_())
+        rows = [
+            {"ref": r, "unreleased": compat.is_unreleased(r), "modules": healthy}
+            for r in ("TS-25.10.4", "release/26.0.0-BETA.3", "master")
+        ]
+        md = compat.render_markdown(rows)
+        assert "master _(27-dev)_" in md
+        assert "master _(unreleased)_" not in md
+        # ...and the ordinary rows are untouched.
+        assert "| 25.10.4 |" in md
+        assert "| 26.0.0-BETA.3 _(unreleased)_ |" in md
+
+
+class TestTheBodyIsTruthAndCommentsAreTheChangelog:
+    """An unchanged fingerprint used to freeze the BODY, not just silence the comments.
+
+    Two different questions were sharing one answer. "Have the findings changed?" gates
+    COMMENTS -- they notify, and a daily "still broken, same as yesterday" is what
+    teaches people to ignore the one that finally matters. But "is the body still
+    true?" gates the BODY, and editing a body notifies nobody, so keeping it honest
+    costs nothing.
+
+    Conflated, an unchanged fingerprint meant the report could never be corrected --
+    and the fingerprint deliberately ignores everything that moves on its own, which
+    includes how a row is LABELLED. Relabelling master `27-dev` would have reached the
+    README and never the issue anybody opens.
+    """
+
+    ROWS = [{"ref": "master", "unreleased": True,
+             "modules": check_files(with_(**{
+                 "plugins/cloud_backup/restic.py":
+                     "class ResticConfig:\n    cmd: list\n\n"
+                     "def get_restic_config(entry, credentials):\n    pass\n",
+             }))}]
+
+    def _run(self, monkeypatch, tmp_path, existing_body):
+        calls = []
+
+        def fake(url, token, method="GET", data=None):
+            calls.append((method, url, data))
+            if url.endswith("/issues?state=all&per_page=100&limit=100"):
+                return [{"number": 1, "title": compat_publish.TITLE,
+                         "state": "open", "body": existing_body,
+                         "pull_request": None}]
+            return {"number": 1}
+
+        monkeypatch.setattr(compat_publish, "_call", fake)
+        matrix = tmp_path / "m.json"
+        matrix.write_text(json.dumps(self.ROWS))
+        compat_publish.main([
+            "prog", "--api", "https://forge/api", "--token", "t",
+            "--matrix", str(matrix)])
+        return calls
+
+    def _writes(self, calls):
+        patched = [c for c in calls if c[0] == "PATCH"]
+        commented = [c for c in calls if c[0] == "POST" and c[1].endswith("/comments")]
+        return patched, commented
+
+    def test_identical_body_and_findings_touches_nothing(self, monkeypatch, tmp_path):
+        body = compat.render_issue(self.ROWS)
+        patched, commented = self._writes(self._run(monkeypatch, tmp_path, body))
+        assert not patched and not commented, "a quiet run must be completely silent"
+
+    def test_a_relabel_refreshes_the_body_but_says_NOTHING(self, monkeypatch, tmp_path):
+        # Same findings (same fingerprint), different rendering -- the exact shape of
+        # the master -> 27-dev relabel.
+        stale = compat.render_issue(self.ROWS).replace("27-dev", "unreleased")
+        assert compat.extract_fingerprint(stale) == compat.fingerprint(self.ROWS)
+
+        patched, commented = self._writes(self._run(monkeypatch, tmp_path, stale))
+        assert patched, "the body was left stale, so the issue keeps telling lies"
+        assert "27-dev" in patched[0][2]["body"]
+        assert not commented, (
+            "a rendering change is not news -- commenting on it is how the bot gets "
+            "muted before the next real finding"
+        )
+
+    def test_a_REAL_findings_change_still_comments(self, monkeypatch, tmp_path):
+        # ...and the fix must not have made it mute.
+        stale = compat.render_issue(self.ROWS).replace(
+            compat.fingerprint(self.ROWS), "0" * 16)
+        patched, commented = self._writes(self._run(monkeypatch, tmp_path, stale))
+        assert patched and commented, "a genuine change must still notify"
+
+    def test_a_body_differing_only_by_CRLF_is_not_rewritten(self, monkeypatch, tmp_path):
+        # Forges round-trip line endings. Without normalising, every run would rewrite
+        # the body -- silent, but it churns updated_at and looks freshly touched daily.
+        body = compat.render_issue(self.ROWS).replace("\n", "\r\n")
+        patched, _ = self._writes(self._run(monkeypatch, tmp_path, body))
+        assert not patched

tests/test_not_loaded_alert.py

@@ -0,0 +1,210 @@
+"""Behavioural tests for the "installed but NOT loaded" alert.
+
+apply.log can only report what was written to disk. Whether the middlewared that
+restarted afterwards actually imported those files is a different fact, and when
+the two disagree nothing else notices: on 2026-08-19 every B2 backup failed for
+nineteen hours while the log said OK. This alert is the only thing that closes
+that gap, so it is tested against real objects rather than by reading source.
+
+The middlewared package does not exist off-box, so the modules the alert source
+imports are stubbed here.
+"""
+
+import importlib.util
+import json
+import os
+import sys
+import types
+
+import pytest
+
+ALERT_SRC = os.path.join(os.path.dirname(__file__), "..", "patch", "alert_source.py")
+
+
+class _StubAlertClass:
+    pass
+
+
+class _StubThreadedAlertSource:
+    pass
+
+
+class _StubAlert:
+    def __init__(self, klass, args=None, key=None):
+        self.klass = klass
+        self.args = args
+        self.key = key
+
+
+def _module(name):
+    mod = types.ModuleType(name)
+    sys.modules[name] = mod
+    return mod
+
+
+@pytest.fixture
+def alert_source(monkeypatch, tmp_path):
+    """Load patch/alert_source.py against stubbed middlewared modules."""
+    for name in list(sys.modules):
+        if name == "middlewared" or name.startswith("middlewared."):
+            monkeypatch.delitem(sys.modules, name, raising=False)
+
+    _module("middlewared")
+    _module("middlewared.alert")
+    base = _module("middlewared.alert.base")
+    base.Alert = _StubAlert
+    base.AlertClass = _StubAlertClass
+    base.ThreadedAlertSource = _StubThreadedAlertSource
+    base.AlertCategory = types.SimpleNamespace(SYSTEM="SYSTEM")
+    base.AlertLevel = types.SimpleNamespace(
+        INFO="INFO", WARNING="WARNING", CRITICAL="CRITICAL"
+    )
+    schedule = _module("middlewared.alert.schedule")
+    schedule.IntervalSchedule = lambda delta: ("interval", delta)
+
+    spec = importlib.util.spec_from_file_location("_tc_alert_source", ALERT_SRC)
+    mod = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(mod)
+    mod.PATCH_DIR = str(tmp_path)
+    return mod
+
+
+def _write_status(tmp_path, providers_active=True):
+    payload = {
+        "patched_at": "2026-08-26T00:00:00Z",
+        "patches": {
+            "providers": {"ok": True, "active": providers_active, "detail": "x"},
+            "nested_snapshots": {"ok": True, "active": True, "detail": "x"},
+        },
+    }
+    (tmp_path / "hook_status.json").write_text(json.dumps(payload))
+
+
+def _install_provider_modules(monkeypatch, *, restic_patched, b2_patched):
+    """Stub the two modules the alert inspects, in the requested state."""
+    plugins = _module("middlewared.plugins")
+    _module("middlewared.plugins.cloud_backup")
+    restic = _module("middlewared.plugins.cloud_backup.restic")
+
+    def get_restic_config(task):
+        return None
+
+    if restic_patched:
+        get_restic_config._truecloud_patched = True
+    restic.get_restic_config = get_restic_config
+
+    rclone_base = _module("middlewared.rclone.base")
+    _module("middlewared.rclone")
+    _module("middlewared.rclone.remote")
+    b2_mod = _module("middlewared.rclone.remote.b2")
+
+    class BaseRcloneRemote:
+        def get_restic_config(self, task):
+            raise NotImplementedError
+
+    class B2RcloneRemote(BaseRcloneRemote):
+        pass
+
+    if b2_patched:
+        B2RcloneRemote.get_restic_config = staticmethod(lambda task: ("url", {}))
+
+    rclone_base.BaseRcloneRemote = BaseRcloneRemote
+    b2_mod.B2RcloneRemote = B2RcloneRemote
+    b2_mod.BaseRcloneRemote = BaseRcloneRemote
+    plugins.__path__ = []
+
+    for name in (
+        "middlewared.plugins",
+        "middlewared.plugins.cloud_backup",
+        "middlewared.plugins.cloud_backup.restic",
+        "middlewared.rclone",
+        "middlewared.rclone.base",
+        "middlewared.rclone.remote",
+        "middlewared.rclone.remote.b2",
+    ):
+        monkeypatch.setitem(sys.modules, name, sys.modules[name])
+
+
+def _source(alert_source):
+    cls = alert_source.TrueCloudPatchNotLoadedAlertSource
+    return cls.__new__(cls)
+
+
+def test_no_alert_when_patch_is_loaded(alert_source, monkeypatch, tmp_path):
+    _write_status(tmp_path)
+    _install_provider_modules(monkeypatch, restic_patched=True, b2_patched=True)
+    assert _source(alert_source)._check() is None
+
+
+def test_alert_when_middlewared_loaded_stock_modules(alert_source, monkeypatch, tmp_path):
+    """The exact 2026-08-19 state: patched on disk, stock in the process."""
+    _write_status(tmp_path)
+    _install_provider_modules(monkeypatch, restic_patched=False, b2_patched=False)
+    alert = _source(alert_source)._check()
+    assert alert is not None
+    assert alert.klass is alert_source.TrueCloudPatchNotLoadedAlertClass
+
+
+def test_alert_when_only_b2_half_is_missing(alert_source, monkeypatch, tmp_path):
+    # b2.py is the half that supplies B2's get_restic_config. restic.py alone
+    # being patched still means every B2 task raises NotImplementedError.
+    _write_status(tmp_path)
+    _install_provider_modules(monkeypatch, restic_patched=True, b2_patched=False)
+    assert _source(alert_source)._check() is not None
+
+
+def test_alert_when_only_restic_half_is_missing(alert_source, monkeypatch, tmp_path):
+    _write_status(tmp_path)
+    _install_provider_modules(monkeypatch, restic_patched=False, b2_patched=True)
+    assert _source(alert_source)._check() is not None
+
+
+def test_silent_when_the_kill_switch_is_set(alert_source, monkeypatch, tmp_path):
+    # The operator turned the patch off on purpose; stock is the intended state.
+    _write_status(tmp_path)
+    (tmp_path / "disabled").write_text("")
+    _install_provider_modules(monkeypatch, restic_patched=False, b2_patched=False)
+    assert _source(alert_source)._check() is None
+
+
+def test_silent_when_providers_module_is_retired(alert_source, monkeypatch, tmp_path):
+    # TrueNAS went native for B2: not loading our providers patch is correct.
+    _write_status(tmp_path, providers_active=False)
+    _install_provider_modules(monkeypatch, restic_patched=False, b2_patched=False)
+    assert _source(alert_source)._check() is None
+
+
+def test_silent_when_the_patch_was_never_applied_here(alert_source, monkeypatch, tmp_path):
+    # No hook_status.json at all -- nothing claims a patch, so nothing is broken.
+    _install_provider_modules(monkeypatch, restic_patched=False, b2_patched=False)
+    assert _source(alert_source)._check() is None
+
+
+def test_update_alert_silencer_does_not_mute_a_broken_backup_path(
+    alert_source, monkeypatch, tmp_path
+):
+    # update_alerts_disabled mutes release notifications. It must not hide the
+    # fact that TrueCloud backups are silently running stock.
+    _write_status(tmp_path)
+    (tmp_path / "update_alerts_disabled").write_text("")
+    _install_provider_modules(monkeypatch, restic_patched=False, b2_patched=False)
+    assert _source(alert_source)._check() is not None
+
+
+def test_check_sync_never_raises(alert_source, monkeypatch, tmp_path):
+    """An alert source that raises is polled forever inside middlewared."""
+    _write_status(tmp_path)
+
+    def boom(self):
+        raise RuntimeError("provider import exploded")
+
+    monkeypatch.setattr(
+        alert_source.TrueCloudPatchNotLoadedAlertSource, "_check", boom, raising=True
+    )
+    assert _source(alert_source).check_sync() is None
+
+
+def test_alert_is_critical_and_names_the_recovery_command(alert_source):
+    klass = alert_source.TrueCloudPatchNotLoadedAlertClass
+    assert klass.level == "CRITICAL"
+    assert "install.sh" in klass.text

tests/test_truecloud_nested.py

@@ -1914,6 +1914,9 @@ class TestTheSnapshotRecordIsNeverLostToAnUnreadableFile:
     only record of a tree it had just failed to read.
     """
 
+    @pytest.mark.skipif(os.geteuid() == 0, reason=
+        "chmod(0) cannot make a file unreadable for root (CAP_DAC_OVERRIDE); "
+        "the Gitea runner image executes jobs as root")
     def test_an_unreadable_sidecar_raises_rather_than_reading_as_empty(self, tmp_path):
         sc = tmp_path / "cloud_backup-5.snapshot"
         sc.write_text("Tap@snap\n")
@@ -1927,6 +1930,9 @@ class TestTheSnapshotRecordIsNeverLostToAnUnreadableFile:
     def test_a_missing_sidecar_is_simply_empty(self, tmp_path):
         assert tn._read_sidecar(str(tmp_path / "nope")) == []
 
+    @pytest.mark.skipif(os.geteuid() == 0, reason=
+        "chmod(0) cannot make a file unreadable for root (CAP_DAC_OVERRIDE); "
+        "the Gitea runner image executes jobs as root")
     def test_cleanup_all_still_UNMOUNTS_when_a_sidecar_cannot_be_read(self, tmp_path):
         # cleanup_all is what recover.sh and uninstall.sh call, i.e. it runs precisely
         # when the box is already stuck. Its job is to get the mounts off. Aborting on

tests/test_wait_restart.py

@@ -0,0 +1,199 @@
+"""The deferred restart must re-apply the patch before it restarts middlewared.
+
+Patching at PREINIT and restarting minutes later is only sound while the patched
+files are still on the live path when middlewared re-imports them. They may not
+be: the patch lives in an overlay mounted inside /usr, and anything that
+remounts that hierarchy detaches it. On 2026-08-19 a systemd-sysext refresh over
+/usr ran four seconds after apply.sh mounted its overlay; the deferred restart
+then loaded stock modules and every B2 cloud_backup job failed for nineteen
+hours while apply.log reported "OK".
+
+These tests pin the ordering that makes that non-recoverable failure impossible:
+re-apply, verify, restart, verify again.
+"""
+
+import os
+import re
+import subprocess
+
+import pytest
+
+HERE = os.path.dirname(__file__)
+WAIT_RESTART = os.path.join(HERE, "..", "patch", "wait_restart.sh")
+APPLY_SH = os.path.join(HERE, "..", "patch", "apply.sh")
+
+
+def wait_restart_source():
+    with open(WAIT_RESTART, encoding="utf-8") as fh:
+        return fh.read()
+
+
+def apply_source():
+    with open(APPLY_SH, encoding="utf-8") as fh:
+        return fh.read()
+
+
+def test_wait_restart_is_executable():
+    # apply.sh schedules it as `/bin/bash <script>`, but install.sh ships exec
+    # bits and a mode-only diff once blocked update.sh outright (v0.6.0).
+    assert os.access(WAIT_RESTART, os.X_OK)
+
+
+def test_wait_restart_is_syntactically_valid():
+    subprocess.run(["bash", "-n", WAIT_RESTART], check=True)
+
+
+def test_reapply_runs_before_the_restart():
+    src = wait_restart_source()
+    reapply = src.index("TRUECLOUD_REAPPLY=1")
+    restart = src.index("systemctl try-restart middlewared")
+    assert reapply < restart, "the re-apply pass must precede the restart"
+
+
+def test_restart_is_not_exec_so_verification_can_follow():
+    # Up to v0.7.0 the script ended in `exec systemctl try-restart middlewared`,
+    # which replaces the shell -- nothing could run afterwards. The post-restart
+    # verification only exists if the restart is a plain call.
+    src = wait_restart_source()
+    assert not re.search(r"^\s*exec\s+systemctl", src, re.M)
+
+
+def test_middlewared_is_restarted_exactly_once():
+    """No restart loop.
+
+    `try-restart` returns at READY; middlewared then brings docker up, and
+    docker.configure_nvidia merges the nvidia sysext over /usr right about then
+    -- detaching the overlay AFTER the patched modules are already imported. A
+    disk check after the restart therefore false-negatives on a healthy system,
+    and restarting on that signal would restart a correctly-patched middlewared
+    straight back into the same race.
+    """
+    src = wait_restart_source()
+    assert src.count("systemctl try-restart middlewared") == 1
+
+
+def test_patch_is_verified_after_the_restart():
+    src = wait_restart_source()
+    restart = src.index("systemctl try-restart middlewared")
+    assert "_patch_visible" in src[restart:], (
+        "the script must check what the restart actually loaded"
+    )
+
+
+def test_verification_reads_the_marker_apply_sh_writes():
+    # _patch_visible greps restic.py for TRUECLOUD_PATCH; apply.sh must still be
+    # the thing that puts it there, or the check silently always fails.
+    assert "TRUECLOUD_PATCH" in wait_restart_source()
+    assert "TRUECLOUD_PATCH" in apply_source()
+
+
+def test_verification_uses_the_recorded_middlewared_dir():
+    # wait_restart.sh must not re-derive site-packages; apply.sh records it.
+    assert ".mw_dir" in wait_restart_source()
+    assert ".mw_dir" in apply_source()
+
+
+def test_apply_sh_records_the_middlewared_dir():
+    src = apply_source()
+    assert re.search(r'>\s*"\$PATCH_DIR/\.mw_dir"', src), (
+        "apply.sh must write the resolved middlewared dir for wait_restart.sh"
+    )
+
+
+def test_reapply_pass_does_not_schedule_another_restart():
+    # wait_restart.sh owns the restart. If the re-apply pass scheduled its own
+    # transient unit, each boot would spawn restarts recursively.
+    src = apply_source()
+    guard = src.index('if [ "${TRUECLOUD_REAPPLY:-0}" = "1" ]; then')
+    systemd_run = src.index("systemd-run --no-block")
+    assert guard < systemd_run, (
+        "the TRUECLOUD_REAPPLY branch must short-circuit before systemd-run"
+    )
+
+
+def test_shadowed_overlay_is_remounted_not_accepted():
+    """A buried overlay must never pass for a healthy one.
+
+    _ensure_writable reaches its mount-table check only when the directory is
+    NOT writable -- and a live overlay of ours is always writable. So a
+    truecloud mount listed at that point is shadowed, and returning 0 there is
+    exactly how a detached overlay used to masquerade as applied.
+    """
+    src = apply_source()
+    start = src.index("_ensure_writable()")
+    end = src.index("\n}", start)
+    body = src[start:end]
+
+    check = body.index('mount | grep -qF "truecloud-${tag} on ${dir} "')
+    following = body[check:]
+    # The old code did `return 0` immediately inside this branch.
+    branch_end = following.index("fi")
+    assert "return 0" not in following[:branch_end]
+    assert "umount -l" in following[:branch_end]
+
+
+def test_workdir_is_recreated_before_mounting():
+    # overlayfs refuses a workdir left behind by a detached mount, so a stale
+    # one would turn every re-mount attempt into "overlay mount failed".
+    src = apply_source()
+    start = src.index("_ensure_writable()")
+    end = src.index("\n}", start)
+    body = src[start:end]
+    assert re.search(r'rm -rf "\$work"', body)
+
+
+def test_upperdir_is_preserved_across_remounts():
+    # The upperdir holds everything patched earlier this boot; reusing it is
+    # what lets a re-mount restore those files instead of re-deriving them.
+    src = apply_source()
+    start = src.index("_ensure_writable()")
+    end = src.index("\n}", start)
+    body = src[start:end]
+    assert 'rm -rf "$upper"' not in body
+
+
+@pytest.mark.parametrize("state", ["0", "1", "2"])
+def test_patch_visible_returns_three_distinct_states(state):
+    # patched / stock / cannot-tell must stay distinguishable: "cannot tell"
+    # has to re-apply rather than assume the patch is fine.
+    src = wait_restart_source()
+    assert f"return {state}" in src or f") return {state}" in src
+
+
+def test_mount_retries_on_a_private_workdir():
+    """A lazily-detached overlay can still pin the shared workdir.
+
+    overlayfs refuses a workdir that is in use, so without a retry the re-mount
+    this whole fix depends on would fail exactly when it is most needed.
+    """
+    src = apply_source()
+    start = src.index("_ensure_writable()")
+    end = src.index("\n}", start)
+    body = src[start:end]
+    assert body.count("mount -t overlay") == 2, "expected a retry mount"
+    assert 'work="/run/truecloud-${tag}-work.$$"' in body
+
+
+def test_post_restart_remount_preserves_the_patched_at_stamp():
+    """create_task.py verify compares middlewared's start time to patched_at.
+
+    The post-restart re-mount restores the same patch the boot pass applied, so
+    letting apply.sh re-stamp would make patched_at newer than the process that
+    correctly imported it -- verify would then report FAIL forever on every boot
+    where docker's sysext merge detaches the overlay.
+    """
+    src = wait_restart_source()
+    tail = src[src.index("systemctl try-restart middlewared"):]
+    assert "hook_status.json" in tail
+    save = tail.index("/run/truecloud-hook_status.pre")
+    reapply = tail.index("TRUECLOUD_REAPPLY=1")
+    restore = tail.rindex("hook_status.json")
+    assert save < reapply < restore, "snapshot must bracket the re-apply"
+
+
+def test_status_snapshot_is_not_written_into_the_repo():
+    """A leftover file in the repo dir leaves the tree dirty, and update.sh
+    refuses to run over a dirty tree -- that once made the patch un-updatable."""
+    src = wait_restart_source()
+    assert "/run/truecloud-hook_status.pre" in src
+    assert '"$PATCH_DIR/hook_status.json.pre' not in src

tests/test_workflows.py

@@ -172,3 +172,81 @@ class TestCompatCannotSilentlyPass:
         with open(os.path.join(WORKFLOWS, "compat.yml"), encoding="utf-8") as fh:
             src = fh.read()
         assert "steps.check.outputs.shipped_broken != '0'" in src
+
+
+class TestActionCacheRace:
+    """CI must not run concurrent jobs on the self-hosted runner.
+
+    `act` caches each ACTION as one shared clone under /root/.cache/act/<hash>
+    and re-pulls it per job, so jobs starting together fight over that directory
+    and the loser dies with `lstat .../<file>: no such file or directory` before
+    any test runs -- a red `main` with zero suite output and a different victim
+    each push. The runner also force-pulls its base image per job, so job count
+    is also Docker Hub pull count, and four-per-push exhausted the anonymous
+    limit in an afternoon. Both problems have the same cure: one job.
+    """
+
+    def _ci(self):
+        with open(os.path.join(WORKFLOWS, "ci.yml"), encoding="utf-8") as fh:
+            return fh.read()
+
+    def test_ci_runs_as_exactly_one_job(self):
+        """The fix is the absence of concurrency, not the absence of one action.
+
+        Dropping astral-sh/setup-uv only shrank the surface -- every job still
+        used actions/checkout. A single job cannot race itself whatever actions
+        it uses, which is why this, and not the action count, is the invariant.
+        """
+        ci = self._ci()
+        # Scope to the jobs: block -- `on:` has two-space keys of its own
+        # (push/pull_request/workflow_dispatch) that look identical otherwise.
+        body = ci[ci.index("\njobs:"):]
+        jobs = re.findall(r"^  (\w[\w-]*):$", body, re.M)
+        assert len(jobs) == 1, (
+            f"ci.yml defines {len(jobs)} jobs ({jobs}); concurrent jobs on the "
+            "self-hosted runner race on act's shared action cache and multiply "
+            "Docker Hub pulls. Keep CI to one job."
+        )
+
+    def test_no_matrix_reintroduces_parallel_jobs(self):
+        ci = self._ci()
+        assert "strategy:" not in ci and "matrix:" not in ci, (
+            "a matrix fans out into concurrent jobs again -- sweep versions "
+            "inside one job instead"
+        )
+
+    def test_every_python_version_still_runs_after_one_fails(self):
+        """`fail-fast: false` is what the loop has to preserve.
+
+        A 3.11 break must not hide whether 3.12 and 3.13 are fine; that is
+        precisely the information you want at that moment.
+        """
+        ci = self._ci()
+        assert 'PYTHONS: "3.11 3.12 3.13"' in ci
+        assert ci.count("fail=1") >= 2, "the sweeps must collect failures, not exit early"
+
+    def test_uv_is_installed_without_an_action(self):
+        """Checks `uses:` directives, not prose.
+
+        The comment in ci.yml names the action it deliberately avoids, and that
+        explanation is the most useful thing in the file -- a test that greps the
+        raw text would forbid documenting the very lesson it enforces. Parsed
+        with a regex rather than PyYAML on purpose: CI runs `uvx pytest`, whose
+        environment holds pytest and nothing else, so a third-party import here
+        fails on the runner while passing locally.
+        """
+        ci = self._ci()
+        used = re.findall(r"^\s*-?\s*uses:\s*(\S+)", ci, re.M)
+        assert not [u for u in used if "setup-uv" in u], (
+            "the action was only fetching a binary; a run: step does the same "
+            "with one less moving part"
+        )
+        assert "astral.sh/uv/" in ci
+
+    def test_the_uv_version_is_pinned(self):
+        ci = self._ci()
+        assert re.search(r'UV_VERSION:\s*"\d+\.\d+\.\d+"', ci), (
+            "an unpinned uv lets any upstream release turn main red with no "
+            "code change here -- the same rule ruff is pinned under"
+        )
+        assert "https://astral.sh/uv/${UV_VERSION}/install.sh" in ci

tools/compat.py

@@ -871,6 +871,24 @@ def discover_refs(remote: str = REPO) -> list[str]:
       * UNRELEASED comes from the BRANCHES, because that is where a beta appears
         first: `release/26.0.0-BETA.3` had no tag yet while it was the newest beta.
         Catching breakage here, before it ships, is the whole point of this file.
+
+    THE NEXT MAINTENANCE RELEASE IS ALSO A BRANCH, and it used to fall through both
+    sieves. `release/25.10.5` is branched but not yet tagged, and 25.10 has already
+    shipped -- so it is not in `shipped` (no tag) and it was excluded from `upcoming`
+    (its line is in `shipped_lines`). It was invisible. That is the one ref a 25.10.4
+    user is actually about to be upgraded onto, so a break there reaches people
+    BEFORE the daily check ever looks at it -- the exact hole this file exists to
+    close, on the only line anybody is running.
+
+    The rule that separates it from the two things we must NOT report:
+
+      * `release/24.10-RC.2` -- a prerelease of an already-shipped line. It is
+        history, not a warning; it sorts OLDER than TS-24.10.2.4, so it is dropped.
+      * `release/25.20.2.2` -- iX's typo branch. 25.20 never shipped, so it has no
+        TS tag, so it is not a line at all and is dropped.
+
+    ...which is: a plain `release/X.Y.Z` branch counts only if its line HAS shipped
+    and it sorts NEWER than that line's newest tag. Both exclusions fall out of it.
     """
     tags = _ls_remote(remote, "--tags")
     heads = _ls_remote(remote, "--heads")
@@ -878,26 +896,45 @@ def discover_refs(remote: str = REPO) -> list[str]:
     shipped = _newest_per_line([
         t for t in tags if t.startswith("TS-") and "-BETA" not in t and "-RC" not in t
     ])
+    newest_shipped = {_version_of(t)[0][:2]: _version_of(t) for t in shipped}
+
+    release_heads = [h for h in heads if h.startswith("release/")]
 
     # A prerelease of a line that has ALREADY shipped is history, not a warning:
     # release/24.10-RC.2 still exists, and the nested module does not apply to it,
     # but 24.10 shipped long ago and TS-24.10.2.4 is fine. Reporting it would be a
     # standing red row in the matrix for a version nobody can install.
-    shipped_lines = {_version_of(t)[0][:2] for t in shipped}
     upcoming = [
         h for h in _newest_per_line([
-            h for h in heads
-            if h.startswith("release/") and ("-BETA" in h or "-RC" in h)
+            h for h in release_heads if "-BETA" in h or "-RC" in h
         ])
-        if _version_of(h)[0][:2] not in shipped_lines
+        if _version_of(h)[0][:2] not in newest_shipped
     ]
 
-    return [*shipped, *upcoming, "master"]
+    # The next maintenance release of a line that HAS shipped: branched, untagged,
+    # and the very next thing those users get. See the docstring.
+    pending = []
+    for h in _newest_per_line([
+        h for h in release_heads if "-BETA" not in h and "-RC" not in h
+    ]):
+        v = _version_of(h)
+        tagged = newest_shipped.get(v[0][:2])
+        if tagged and v > tagged:
+            pending.append(h)
+
+    return [*shipped, *pending, *upcoming, "master"]
 
 
 def is_unreleased(ref: str) -> bool:
-    """master and any BETA/RC. Breakage here is early warning, not an outage."""
-    return ref == "master" or "-BETA" in ref or "-RC" in ref
+    """Anything iX has not TAGGED. Breakage here is early warning, not an outage.
+
+    Keyed on where the ref came from, not on its name: `discover_refs` takes shipped
+    releases from `TS-*` TAGS and everything else from BRANCHES, so a `release/*` ref
+    is by construction something iX has not released yet. Testing for `-BETA`/`-RC`
+    instead would call `release/25.10.5` SHIPPED, and a break there would fail the
+    build as a live outage -- on a version nobody is running yet.
+    """
+    return ref == "master" or ref.startswith("release/")
 
 
 def matrix(refs=None, remote: str = REPO) -> list[dict]:
@@ -941,10 +978,15 @@ def is_broken(r: dict) -> bool:
 #: is static analysis of iX's source, which proves the patch's assumptions hold --
 #: a strictly weaker claim than "a restore worked". Add a row only after doing it.
 HARDWARE_VERIFIED = {
-    "25.10.4": "nested + providers; 252-snapshot recursive backup of /mnt/Tap, 18m",
+    "25.10.4": (
+        "v0.7.0: 3 live tasks — 191-dataset nested backup of /mnt/Tap, a "
+        "215-filesystem/2-zvol backup of /mnt/Tank/backups, and a non-nested one; "
+        "0 orphans, 0 leaked mounts, byte-identical restore; the collector also "
+        "reclaimed a real orphan the pool had been carrying"
+    ),
     "26.0.0-BETA.1": (
-        "nested + providers; 274-snapshot recursive backup of a 292-dataset pool, "
-        "restored a 4-deep child dataset byte-identical"
+        "v0.7.0: 274-snapshot recursive backup of a 292-dataset pool; restored a "
+        "4-deep child dataset byte-identical; zvol-orphan case reproduced then closed"
     ),
 }
 
@@ -958,6 +1000,16 @@ _LEGEND = """
 "ok" means *the patch's assumptions hold*, checked automatically against iX's
 source. It does not mean a human ran a backup on it — that is the
 **Hardware-verified** column, which is filled in by hand and only by doing it.
+
+**`master` is not the next release.** iX branches each major off to its own
+`release/` line and master rolls straight on to the one after — so master is
+`27-dev` while 26 is still in beta. A **BROKEN** master means iX has changed
+something that will reach users *a major release from now*, not in the version you
+are about to install. Read the numbered rows for that.
+
+A row like `25.10.5 _(unreleased)_` is the next maintenance release: branched by iX,
+not tagged yet, and the very next thing a 25.10.4 box gets. It is checked precisely
+because it is the one unshipped ref that reaches real users without warning.
 """
 
 
@@ -1005,17 +1057,42 @@ def update_readme(rows: list[dict], path: str = README) -> bool:
     return True
 
 
+def dev_label(rows: list[dict]) -> str:
+    """What `master` is a development line FOR -- e.g. "27-dev".
+
+    `master` is NOT the next release, and labelling it "master _(unreleased)_" said
+    it was. On 2026-07-14 every recent commit on master targeted 27.0.0-BETA.1 while
+    26 was still in beta on its own `release/26.0.0-BETA.*` branches: master had
+    already rolled over to the major AFTER the one that has not shipped yet. So a red
+    `master` row read as "the version you are about to install is broken" when the
+    breakage was a year out, on a line nobody can even download. That is a false alarm
+    aimed squarely at the person deciding whether to trust this with their backups.
+
+    Derived, not hardcoded: the newest major we can see anywhere, plus one. iX branches
+    `release/N.0.0-BETA.1` off master and master immediately becomes N+1 -- so when 27
+    betas appear, this says 28-dev on its own.
+    """
+    majors = [
+        v[0][0] for v in (_version_of(r["ref"]) for r in rows if r["ref"] != "master")
+        if v
+    ]
+    return f"{max(majors) + 1}-dev" if majors else "unreleased"
+
+
 def render_markdown(rows: list[dict]) -> str:
     """The matrix, for the README."""
     out = [
         "| TrueNAS | B2/S3 providers | Nested snapshots | Hardware-verified |",
         "| --- | --- | --- | --- |",
     ]
+    dev = dev_label(rows)
     for row in rows:
         m = row["modules"]
         ref = row["ref"]
         label = ref.removeprefix("TS-").removeprefix("release/")
-        if row["unreleased"]:
+        if ref == "master":
+            label = f"master _({dev})_"
+        elif row["unreleased"]:
             label = f"{label} _(unreleased)_"
 
         cells = []
@@ -1133,7 +1210,13 @@ def render_matrix(rows: list[dict]) -> str:
     hardware-verified column lives in COMPATIBILITY.md and is maintained by hand,
     because nothing else can honestly fill it in.
     """
-    w = max((len(r["ref"]) for r in rows), default=10)
+    dev = dev_label(rows)
+    # Same relabel as the README: a red `master` is a warning about the major AFTER
+    # next, and "master" alone reads as "the release you are about to install".
+    names = {r["ref"]: (f"master ({dev})" if r["ref"] == "master" else r["ref"])
+             for r in rows}
+
+    w = max((len(n) for n in names.values()), default=10)
     lines = [
         f"{'TrueNAS'.ljust(w)}  {'providers':<10}  {'nested':<10}",
         f"{'-' * w}  {'-' * 10}  {'-' * 10}",
@@ -1141,7 +1224,7 @@ def render_matrix(rows: list[dict]) -> str:
     for row in rows:
         m = row["modules"]
         lines.append(
-            f"{row['ref'].ljust(w)}  "
+            f"{names[row['ref']].ljust(w)}  "
             f"{_verdict(m[PROVIDERS]):<10}  {_verdict(m[NESTED]):<10}"
         )
     return "\n".join(lines)

tools/compat_publish.py

@@ -62,6 +62,19 @@ def _call(url, token, method="GET", data=None):
         return json.load(r) if r.length != 0 else {}
 
 
+def _same(a, b):
+    """Is the issue body already what we would write?
+
+    Compared after normalising line endings and trailing space: forges are free to
+    round-trip `\r\n`, and a body that only "differs" by that would be rewritten on
+    every single run -- a silent edit, but a pointless one that churns `updated_at`
+    and makes the issue look freshly touched every morning.
+    """
+    def norm(s):
+        return "\n".join(line.rstrip() for line in (s or "").replace("\r\n", "\n").split("\n")).strip()
+    return norm(a) == norm(b)
+
+
 def find_issue(api, token, title):
     """The LOWEST-numbered issue with this title, open or closed.
 
@@ -69,11 +82,18 @@ def find_issue(api, token, title):
     existed once (an earlier version put the ref list in the title, so the identity
     changed whenever that set changed), and an order-dependent pick would alternate
     between them -- reopening one while commenting on the other.
+
+    Both forges list PRs alongside issues, but they SAY SO DIFFERENTLY: GitHub omits
+    the `pull_request` key on a plain issue, Gitea sends it as `null`. Testing for the
+    KEY therefore discards every Gitea issue as if it were a PR -- so this returned
+    None on every Gitea run, and the bot filed a brand-new duplicate report each time
+    instead of editing the one it already had. Test the VALUE; it is the only form
+    that is true on both.
     """
-    issues = _call(f"{api}/issues?state=all&per_page=100", token)
+    issues = _call(f"{api}/issues?state=all&per_page=100&limit=100", token)
     mine = [
         i for i in issues
-        if i.get("title") == title and "pull_request" not in i   # GitHub lists PRs here
+        if i.get("title") == title and not i.get("pull_request")
     ]
     return min(mine, key=lambda i: i["number"]) if mine else None
 
@@ -117,11 +137,24 @@ def main(argv):
     have = extract_fingerprint(issue.get("body") or "")
     n = issue["number"]
 
-    # ── the findings are UNCHANGED: say nothing ──────────────────────────────
+    # ── two different questions, and they were being answered with one answer ────
     #
-    # This is the whole point. A daily "still broken, same as yesterday" comment is
-    # what taught everyone to ignore the last one.
-    if have == want and issue["state"] == "open":
+    #   * IS THE BODY STILL TRUE?     -> if not, rewrite it. Editing an issue body
+    #     notifies NOBODY on either forge, so keeping it honest is free.
+    #   * HAVE THE FINDINGS CHANGED?  -> only then comment. Comments DO notify, and a
+    #     daily "still broken, same as yesterday" is what teaches everyone to ignore
+    #     the one that finally matters.
+    #
+    # Conflating them meant an unchanged FINGERPRINT froze the BODY. The fingerprint
+    # deliberately ignores everything that moves on its own -- healthy rows, the
+    # hardware-verified column, point releases, how a row is LABELLED -- so none of
+    # that could ever reach the report. Relabelling master `27-dev` (it is not the
+    # next release; a red row there was reading as "the version you are about to
+    # install is broken") would have shipped to the README and never to the issue
+    # anybody actually opens.
+    body_is_current = _same(issue.get("body"), body)
+
+    if have == want and issue["state"] == "open" and body_is_current:
         print(f"#{n} is already current ({want}) — staying quiet")
         return 0
 
@@ -139,8 +172,12 @@ def main(argv):
         )
         _call(f"{args.api}/issues/{n}/comments", args.token, "POST", {"body": note})
         print(f"updated #{n}: {have} -> {want}")
-    else:
+    elif issue["state"] != "open":
         print(f"reopened #{n}")
+    else:
+        # Same findings, new rendering. Silent by design: nothing has changed that
+        # anybody needs waking up for, but the report should not be telling lies.
+        print(f"#{n}: findings unchanged ({want}); body refreshed silently")
     return 0
 
 

uninstall.sh

@@ -3,7 +3,7 @@
 
 set -euo pipefail
 
-VERSION="0.7.0"
+VERSION="0.8.0"
 
 PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
 _HOOK_COMMENT='TrueCloud provider patch (S3/B2)'
@@ -124,6 +124,11 @@ if [ -f "$PATCH_DIR/nested_snapshots_enabled" ]; then
     rm -f "$PATCH_DIR/nested_snapshots_enabled"
     echo "  Removed nested-snapshot opt-in marker."
 fi
+
+# Runtime breadcrumb recorded by apply.sh for wait_restart.sh. Harmless, but a
+# stale path left in an uninstalled tree is exactly the sort of thing that reads
+# as state later.
+rm -f "$PATCH_DIR/.mw_dir"
 echo ""
 
 if [ "$_restore_failed" -eq 1 ]; then

update.sh

@@ -19,7 +19,7 @@
 
 set -euo pipefail
 
-VERSION="0.7.0"
+VERSION="0.8.0"
 
 PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
 _PREV_FILE="$PATCH_DIR/.update_previous"

uv.lock

@@ -0,0 +1,3 @@
+version = 1
+revision = 3
+requires-python = ">=3.12"