• v0.8.0-rc3
    TrueNAS compatibility / compat (push) Successful in 15s
    Release / release (push) Successful in 15s
    CI / python 3.11 (push) Failing after 7s
    CI / python 3.12 (push) Successful in 17s
    CI / python 3.13 (push) Successful in 20s
    CI / shell (shellcheck + syntax) (push) Successful in 9s
    Pre-Release

    flan released this 2026-08-26 01:20:27 -04:00 | 2 commits to main since this release

    Changed

    • CI's python matrix is green on the self-hosted Gitea runner again. The real
      failure was that the Gitea runner image executes jobs as root, and the two
      unreadable-sidecar tests build their scenario with chmod(0) — which cannot make
      a file unreadable for root (CAP_DAC_OVERRIDE). Those two tests now skip as root
      with that reason; GitHub's non-root runner still exercises them. The matrix also
      moved to uv-managed interpreters (one toolchain across both runners) and ruff is
      pinned to 0.16.1 so an upstream ruff release can't turn main red without a code
      change.

    • README badges point at the public GitHub mirror (workflow status and
      releases) instead of the private forge. The release badge had also been reading
      the stale Gitea v0.6.1 release instead of the current v0.7.0 on GitHub.

    • master is now labelled 27-dev, because it is not the next release. iX
      branches each major onto its own release/ line and master rolls straight on to the
      one after — on 2026-07-14 every recent commit on master targeted 27.0.0-BETA.1
      while 26 was still in beta. So a BROKEN master row, rendered as
      "master (unreleased)", read as "the version you are about to install is broken"
      when the breakage was a major release away on a line nobody can download. In a table
      whose entire job is helping somebody decide whether to trust this with their backups,
      that is a false alarm in the worst possible place. The label is derived from the
      newest major in the matrix plus one, so it rolls over to 28-dev by itself once 27
      branches.

      For the record, the breakage is NAS-141498 (2026-06-24), "Convert cloud_backup
      plugin to the typesafe pattern": it re-signatures restic_backup and
      get_restic_config, splitting entry/credentials out of the cloud_backup dict.
      It is deliberately not being chased while the 27 line is still churning.

    Fixed

    • The patch survived being applied and then silently stopped existing, because
      something else remounted /usr four seconds later.
      On a box running
      TrueNAS 25.10.6 the boot of 2026-08-19 went: 16:41:56 apply.sh mounts its
      overlay on /usr/lib/python3/dist-packages, patches b2.py/restic.py, logs
      every step OK; 16:42:00 a second PREINIT hook runs systemd-sysext refresh over /usrUnmerged '/usr'. / Merged extensions into '/usr'.
      and our overlay, which lives inside that hierarchy, is torn off with it;
      16:47:24 our own deferred restart fires exactly as designed and middlewared
      imports the stock modules. Every B2 TrueCloud Backup task then failed with
      NotImplementedError from stock rclone/base.py for nineteen hours, across
      four scheduled runs, while apply.log and hook_status.json both said the
      patch was active.

      Nothing in the patch was wrong, which is the point: applying at PREINIT and
      restarting later is only sound if the patched files are still on the live path
      when middlewared re-imports them, and that is not something PREINIT can
      guarantee
      . Init scripts run sequentially in id order, so any hook registered
      after ours always wins. Worse, hook ordering cannot fix it either —
      middlewared's own docker.configure_nvidia merges a sysext over /usr at
      runtime, long after every PREINIT hook is finished.

      So the deferred restart no longer trusts the PREINIT pass. wait_restart.sh
      now re-applies immediately before it restarts middlewared — after boot has
      settled, which is also after every sysext merge and docker nvidia
      configuration — and verifies the marker is genuinely on the live path before
      restarting. It is no longer exec systemctl try-restart middlewared, because
      something has to run afterwards.

      What runs afterwards deliberately does not restart again. try-restart
      returns as soon as middlewared is READY, and middlewared then brings docker up
      docker.configure_nvidia merges the stock nvidia sysext over /usr at that
      point, detaching the overlay after the patched modules have already been
      imported. A disk check there reports "missing" on a perfectly healthy system,
      and restarting on that signal would restart a correctly-patched middlewared
      straight back into the same race. So the overlay is re-mounted for the benefit
      of the next restart, and the question of whether this middlewared actually
      holds the patch is left to the one thing that can answer it exactly — the
      in-process alert below. That re-mount preserves hook_status.json's
      patched_at: create_task.py verify decides "loaded" by comparing
      middlewared's start time against that stamp, so a re-apply running after the
      restart would have made the stamp newer than the process which correctly
      imported the patch, and verify would have reported FAIL forever on every
      boot where the sysext merge detaches the overlay. Caught on hardware while
      validating the candidate — a new lying status introduced by the fix for a
      lying status.

      Two supporting fixes fell out of the same failure. _ensure_writable treated
      "one of our overlays is listed on this directory" as "already done" — but it
      only ever reaches that check when the directory is not writable, and a live
      overlay of ours always is. A shadowed overlay was therefore indistinguishable
      from a healthy one; it is now detached and re-mounted, reusing the same
      upperdir so everything patched earlier in the boot reappears intact, with a
      fresh workdir because overlayfs refuses one left behind by a detached mount.

    • middlewared now says so when it is running stock. The gap that let this
      cost nineteen hours was not the remount, it was that nothing could tell the
      difference between "patched on disk" and "patched in the running process".
      apply.log can only ever report the first. A new CRITICAL alert asks the
      second question from inside middlewared, hourly, where it is exact: the patch
      stamps the objects it replaces, so a missing stamp means this interpreter
      imported stock code. It checks both halves — restic.py's _truecloud_patched
      marker and whether B2RcloneRemote.get_restic_config is still the base class's
      — since either can go missing alone. It stays quiet when the kill switch is
      set or the providers module has been retired as native, and it is deliberately
      not silenced by update_alerts_disabled: that mutes release notifications,
      not a broken backup path.

      Boot-time diagnosis also no longer depends on the journal. wait_restart.sh
      logged only to the journal, and journald retention on a busy box is easily
      shorter than the interval between reboots — the 2026-08-19 boot had already
      rotated away by the time it was investigated. It now writes to apply.log
      alongside everything else.

    • The next maintenance release was never checked, and it is the one that reaches
      users.
      Shipped versions were discovered from TS-* tags and unreleased ones from
      release/* branches carrying -BETA/-RC. A branched-but-untagged maintenance
      release is neither: release/25.10.5 has no tag, and its line has already shipped,
      so the "a prerelease of a shipped line is history" filter discarded it. It was
      invisible — and it is precisely what a 25.10.4 box gets on its next update. A break
      there would have reached real users before the daily check ever looked at it, on the
      only line anybody is actually running.

      A plain release/X.Y.Z branch is now checked when its line has shipped and it
      sorts newer than that line's newest tag. Both things that must stay out fall out
      of the same rule: release/24.10-RC.2 sorts older than TS-24.10.2.4 (history, not
      a warning), and iX's typo branch release/25.20.2.2 is on a line that has no tag at
      all, so it is not a release line. This immediately surfaced two refs that had never
      been checked — release/25.10.5 and release/24.10.2.5 — both of which pass.

      is_unreleased() now keys off where a ref came from (branch = not yet shipped)
      rather than looking for -BETA/-RC in its name. Otherwise release/25.10.5 would
      count as shipped and a break in it would fail the build as a live outage — on a
      version nobody is running yet.

    • An unchanged fingerprint froze the bug report's body, not just its comments. Two
      questions were sharing one answer. Have the findings changed? gates comments
      they notify, and a daily "still broken, same as yesterday" is what teaches everyone
      to ignore the one that finally matters. Is the body still true? gates the body
      and editing an issue body notifies nobody on either forge, so keeping it honest is
      free. Conflated, the report could never be corrected while the findings held steady,
      and the fingerprint deliberately ignores everything that moves on its own — healthy
      rows, the hardware-verified column, point releases, and how a row is labelled. The
      master27-dev relabel above would have reached the README and never the issue
      anybody actually opens. The body is now rewritten whenever it is out of date (after
      normalising line endings, so a forge round-tripping \r\n does not cause a rewrite
      every run) and comments remain strictly a changelog of real changes.

    • A change to the publisher did not re-run the check. compat.yml's push: paths
      listed tools/compat.py but not tools/compat_publish.py — so the very commit that
      taught the bot to refresh a stale report body triggered no run, and the report stayed
      stale until the next scheduled one. A fix nobody runs is a fix nobody has.

    • The compatibility bot filed a new duplicate bug report on every Gitea run.
      find_issue() skipped pull requests by testing for the presence of the
      pull_request key. GitHub omits that key on a plain issue; Gitea sends it as
      null. So on Gitea every issue was discarded as a PR, the lookup always came back
      empty, and the bot took the "nothing filed yet" branch and opened a fresh report
      each run — nine copies on the canonical forge, four of them filed after the
      commit that was meant to stop precisely this. The mirror was fine, which is why it
      went unnoticed: GitHub's payload shape is the one the filter was written against.

      It is the same failure the anti-spam fix was written to prevent, moved from
      comments to issues, and it survived because find_issue was the only function in
      compat_publish.py with no test. It now has one, per forge, and the daily cron —
      which had not yet run once — no longer accumulates a report a day.

      The issue list is also requested with both paging parameters (per_page for
      GitHub, limit for Gitea). Each forge ignores the other's, and Gitea's default page
      is 30, so the lookup would have started missing the report again once the pile it
      was creating grew past one page.

    Downloads
  • v0.8.0-rc2
    TrueNAS compatibility / compat (push) Successful in 12s
    Release / release (push) Successful in 17s
    CI / shell (shellcheck + syntax) (push) Successful in 8s
    CI / python 3.12 (push) Successful in 21s
    CI / python 3.11 (push) Successful in 21s
    CI / python 3.13 (push) Successful in 24s
    Pre-Release

    flan released this 2026-08-26 00:57:41 -04:00 | 3 commits to main since this release

    Changed

    • CI's python matrix is green on the self-hosted Gitea runner again. The real
      failure was that the Gitea runner image executes jobs as root, and the two
      unreadable-sidecar tests build their scenario with chmod(0) — which cannot make
      a file unreadable for root (CAP_DAC_OVERRIDE). Those two tests now skip as root
      with that reason; GitHub's non-root runner still exercises them. The matrix also
      moved to uv-managed interpreters (one toolchain across both runners) and ruff is
      pinned to 0.16.1 so an upstream ruff release can't turn main red without a code
      change.

    • README badges point at the public GitHub mirror (workflow status and
      releases) instead of the private forge. The release badge had also been reading
      the stale Gitea v0.6.1 release instead of the current v0.7.0 on GitHub.

    • master is now labelled 27-dev, because it is not the next release. iX
      branches each major onto its own release/ line and master rolls straight on to the
      one after — on 2026-07-14 every recent commit on master targeted 27.0.0-BETA.1
      while 26 was still in beta. So a BROKEN master row, rendered as
      "master (unreleased)", read as "the version you are about to install is broken"
      when the breakage was a major release away on a line nobody can download. In a table
      whose entire job is helping somebody decide whether to trust this with their backups,
      that is a false alarm in the worst possible place. The label is derived from the
      newest major in the matrix plus one, so it rolls over to 28-dev by itself once 27
      branches.

      For the record, the breakage is NAS-141498 (2026-06-24), "Convert cloud_backup
      plugin to the typesafe pattern": it re-signatures restic_backup and
      get_restic_config, splitting entry/credentials out of the cloud_backup dict.
      It is deliberately not being chased while the 27 line is still churning.

    Fixed

    • The patch survived being applied and then silently stopped existing, because
      something else remounted /usr four seconds later.
      On a box running
      TrueNAS 25.10.6 the boot of 2026-08-19 went: 16:41:56 apply.sh mounts its
      overlay on /usr/lib/python3/dist-packages, patches b2.py/restic.py, logs
      every step OK; 16:42:00 a second PREINIT hook runs systemd-sysext refresh over /usrUnmerged '/usr'. / Merged extensions into '/usr'.
      and our overlay, which lives inside that hierarchy, is torn off with it;
      16:47:24 our own deferred restart fires exactly as designed and middlewared
      imports the stock modules. Every B2 TrueCloud Backup task then failed with
      NotImplementedError from stock rclone/base.py for nineteen hours, across
      four scheduled runs, while apply.log and hook_status.json both said the
      patch was active.

      Nothing in the patch was wrong, which is the point: applying at PREINIT and
      restarting later is only sound if the patched files are still on the live path
      when middlewared re-imports them, and that is not something PREINIT can
      guarantee
      . Init scripts run sequentially in id order, so any hook registered
      after ours always wins. Worse, hook ordering cannot fix it either —
      middlewared's own docker.configure_nvidia merges a sysext over /usr at
      runtime, long after every PREINIT hook is finished.

      So the deferred restart no longer trusts the PREINIT pass. wait_restart.sh
      now re-applies immediately before it restarts middlewared — after boot has
      settled, which is also after every sysext merge and docker nvidia
      configuration — and verifies the marker is genuinely on the live path before
      restarting. It is no longer exec systemctl try-restart middlewared, because
      something has to run afterwards.

      What runs afterwards deliberately does not restart again. try-restart
      returns as soon as middlewared is READY, and middlewared then brings docker up
      docker.configure_nvidia merges the stock nvidia sysext over /usr at that
      point, detaching the overlay after the patched modules have already been
      imported. A disk check there reports "missing" on a perfectly healthy system,
      and restarting on that signal would restart a correctly-patched middlewared
      straight back into the same race. So the overlay is re-mounted for the benefit
      of the next restart, and the question of whether this middlewared actually
      holds the patch is left to the one thing that can answer it exactly — the
      in-process alert below.

      Two supporting fixes fell out of the same failure. _ensure_writable treated
      "one of our overlays is listed on this directory" as "already done" — but it
      only ever reaches that check when the directory is not writable, and a live
      overlay of ours always is. A shadowed overlay was therefore indistinguishable
      from a healthy one; it is now detached and re-mounted, reusing the same
      upperdir so everything patched earlier in the boot reappears intact, with a
      fresh workdir because overlayfs refuses one left behind by a detached mount.

    • middlewared now says so when it is running stock. The gap that let this
      cost nineteen hours was not the remount, it was that nothing could tell the
      difference between "patched on disk" and "patched in the running process".
      apply.log can only ever report the first. A new CRITICAL alert asks the
      second question from inside middlewared, hourly, where it is exact: the patch
      stamps the objects it replaces, so a missing stamp means this interpreter
      imported stock code. It checks both halves — restic.py's _truecloud_patched
      marker and whether B2RcloneRemote.get_restic_config is still the base class's
      — since either can go missing alone. It stays quiet when the kill switch is
      set or the providers module has been retired as native, and it is deliberately
      not silenced by update_alerts_disabled: that mutes release notifications,
      not a broken backup path.

      Boot-time diagnosis also no longer depends on the journal. wait_restart.sh
      logged only to the journal, and journald retention on a busy box is easily
      shorter than the interval between reboots — the 2026-08-19 boot had already
      rotated away by the time it was investigated. It now writes to apply.log
      alongside everything else.

    • The next maintenance release was never checked, and it is the one that reaches
      users.
      Shipped versions were discovered from TS-* tags and unreleased ones from
      release/* branches carrying -BETA/-RC. A branched-but-untagged maintenance
      release is neither: release/25.10.5 has no tag, and its line has already shipped,
      so the "a prerelease of a shipped line is history" filter discarded it. It was
      invisible — and it is precisely what a 25.10.4 box gets on its next update. A break
      there would have reached real users before the daily check ever looked at it, on the
      only line anybody is actually running.

      A plain release/X.Y.Z branch is now checked when its line has shipped and it
      sorts newer than that line's newest tag. Both things that must stay out fall out
      of the same rule: release/24.10-RC.2 sorts older than TS-24.10.2.4 (history, not
      a warning), and iX's typo branch release/25.20.2.2 is on a line that has no tag at
      all, so it is not a release line. This immediately surfaced two refs that had never
      been checked — release/25.10.5 and release/24.10.2.5 — both of which pass.

      is_unreleased() now keys off where a ref came from (branch = not yet shipped)
      rather than looking for -BETA/-RC in its name. Otherwise release/25.10.5 would
      count as shipped and a break in it would fail the build as a live outage — on a
      version nobody is running yet.

    • An unchanged fingerprint froze the bug report's body, not just its comments. Two
      questions were sharing one answer. Have the findings changed? gates comments
      they notify, and a daily "still broken, same as yesterday" is what teaches everyone
      to ignore the one that finally matters. Is the body still true? gates the body
      and editing an issue body notifies nobody on either forge, so keeping it honest is
      free. Conflated, the report could never be corrected while the findings held steady,
      and the fingerprint deliberately ignores everything that moves on its own — healthy
      rows, the hardware-verified column, point releases, and how a row is labelled. The
      master27-dev relabel above would have reached the README and never the issue
      anybody actually opens. The body is now rewritten whenever it is out of date (after
      normalising line endings, so a forge round-tripping \r\n does not cause a rewrite
      every run) and comments remain strictly a changelog of real changes.

    • A change to the publisher did not re-run the check. compat.yml's push: paths
      listed tools/compat.py but not tools/compat_publish.py — so the very commit that
      taught the bot to refresh a stale report body triggered no run, and the report stayed
      stale until the next scheduled one. A fix nobody runs is a fix nobody has.

    • The compatibility bot filed a new duplicate bug report on every Gitea run.
      find_issue() skipped pull requests by testing for the presence of the
      pull_request key. GitHub omits that key on a plain issue; Gitea sends it as
      null. So on Gitea every issue was discarded as a PR, the lookup always came back
      empty, and the bot took the "nothing filed yet" branch and opened a fresh report
      each run — nine copies on the canonical forge, four of them filed after the
      commit that was meant to stop precisely this. The mirror was fine, which is why it
      went unnoticed: GitHub's payload shape is the one the filter was written against.

      It is the same failure the anti-spam fix was written to prevent, moved from
      comments to issues, and it survived because find_issue was the only function in
      compat_publish.py with no test. It now has one, per forge, and the daily cron —
      which had not yet run once — no longer accumulates a report a day.

      The issue list is also requested with both paging parameters (per_page for
      GitHub, limit for Gitea). Each forge ignores the other's, and Gitea's default page
      is 30, so the lookup would have started missing the report again once the pile it
      was creating grew past one page.

    Downloads
  • v0.8.0-rc1
    CI / shell (shellcheck + syntax) (push) Successful in 12s
    CI / python 3.11 (push) Successful in 20s
    CI / python 3.13 (push) Successful in 21s
    TrueNAS compatibility / compat (push) Successful in 17s
    CI / python 3.12 (push) Successful in 39s
    Release / release (push) Successful in 20s
    Pre-Release

    flan released this 2026-08-26 00:37:50 -04:00 | 4 commits to main since this release

    Changed

    • CI's python matrix is green on the self-hosted Gitea runner again. The real
      failure was that the Gitea runner image executes jobs as root, and the two
      unreadable-sidecar tests build their scenario with chmod(0) — which cannot make
      a file unreadable for root (CAP_DAC_OVERRIDE). Those two tests now skip as root
      with that reason; GitHub's non-root runner still exercises them. The matrix also
      moved to uv-managed interpreters (one toolchain across both runners) and ruff is
      pinned to 0.16.1 so an upstream ruff release can't turn main red without a code
      change.

    • README badges point at the public GitHub mirror (workflow status and
      releases) instead of the private forge. The release badge had also been reading
      the stale Gitea v0.6.1 release instead of the current v0.7.0 on GitHub.

    • master is now labelled 27-dev, because it is not the next release. iX
      branches each major onto its own release/ line and master rolls straight on to the
      one after — on 2026-07-14 every recent commit on master targeted 27.0.0-BETA.1
      while 26 was still in beta. So a BROKEN master row, rendered as
      "master (unreleased)", read as "the version you are about to install is broken"
      when the breakage was a major release away on a line nobody can download. In a table
      whose entire job is helping somebody decide whether to trust this with their backups,
      that is a false alarm in the worst possible place. The label is derived from the
      newest major in the matrix plus one, so it rolls over to 28-dev by itself once 27
      branches.

      For the record, the breakage is NAS-141498 (2026-06-24), "Convert cloud_backup
      plugin to the typesafe pattern": it re-signatures restic_backup and
      get_restic_config, splitting entry/credentials out of the cloud_backup dict.
      It is deliberately not being chased while the 27 line is still churning.

    Fixed

    • The patch survived being applied and then silently stopped existing, because
      something else remounted /usr four seconds later.
      On a box running
      TrueNAS 25.10.6 the boot of 2026-08-19 went: 16:41:56 apply.sh mounts its
      overlay on /usr/lib/python3/dist-packages, patches b2.py/restic.py, logs
      every step OK; 16:42:00 a second PREINIT hook runs systemd-sysext refresh over /usrUnmerged '/usr'. / Merged extensions into '/usr'.
      and our overlay, which lives inside that hierarchy, is torn off with it;
      16:47:24 our own deferred restart fires exactly as designed and middlewared
      imports the stock modules. Every B2 TrueCloud Backup task then failed with
      NotImplementedError from stock rclone/base.py for nineteen hours, across
      four scheduled runs, while apply.log and hook_status.json both said the
      patch was active.

      Nothing in the patch was wrong, which is the point: applying at PREINIT and
      restarting later is only sound if the patched files are still on the live path
      when middlewared re-imports them, and that is not something PREINIT can
      guarantee
      . Init scripts run sequentially in id order, so any hook registered
      after ours always wins. Worse, hook ordering cannot fix it either —
      middlewared's own docker.configure_nvidia merges a sysext over /usr at
      runtime, long after every PREINIT hook is finished.

      So the deferred restart no longer trusts the PREINIT pass. wait_restart.sh
      now re-applies immediately before it restarts middlewared — after boot has
      settled, which is also after every sysext merge and docker nvidia
      configuration — verifies the marker is genuinely on the live path, restarts,
      and verifies again, retrying once if the patch was torn off in between. It is
      no longer exec systemctl try-restart middlewared, because something has to
      run afterwards to find out what that restart actually loaded.

      Two supporting fixes fell out of the same failure. _ensure_writable treated
      "one of our overlays is listed on this directory" as "already done" — but it
      only ever reaches that check when the directory is not writable, and a live
      overlay of ours always is. A shadowed overlay was therefore indistinguishable
      from a healthy one; it is now detached and re-mounted, reusing the same
      upperdir so everything patched earlier in the boot reappears intact, with a
      fresh workdir because overlayfs refuses one left behind by a detached mount.

    • middlewared now says so when it is running stock. The gap that let this
      cost nineteen hours was not the remount, it was that nothing could tell the
      difference between "patched on disk" and "patched in the running process".
      apply.log can only ever report the first. A new CRITICAL alert asks the
      second question from inside middlewared, hourly, where it is exact: the patch
      stamps the objects it replaces, so a missing stamp means this interpreter
      imported stock code. It checks both halves — restic.py's _truecloud_patched
      marker and whether B2RcloneRemote.get_restic_config is still the base class's
      — since either can go missing alone. It stays quiet when the kill switch is
      set or the providers module has been retired as native, and it is deliberately
      not silenced by update_alerts_disabled: that mutes release notifications,
      not a broken backup path.

      Boot-time diagnosis also no longer depends on the journal. wait_restart.sh
      logged only to the journal, and journald retention on a busy box is easily
      shorter than the interval between reboots — the 2026-08-19 boot had already
      rotated away by the time it was investigated. It now writes to apply.log
      alongside everything else.

    • The next maintenance release was never checked, and it is the one that reaches
      users.
      Shipped versions were discovered from TS-* tags and unreleased ones from
      release/* branches carrying -BETA/-RC. A branched-but-untagged maintenance
      release is neither: release/25.10.5 has no tag, and its line has already shipped,
      so the "a prerelease of a shipped line is history" filter discarded it. It was
      invisible — and it is precisely what a 25.10.4 box gets on its next update. A break
      there would have reached real users before the daily check ever looked at it, on the
      only line anybody is actually running.

      A plain release/X.Y.Z branch is now checked when its line has shipped and it
      sorts newer than that line's newest tag. Both things that must stay out fall out
      of the same rule: release/24.10-RC.2 sorts older than TS-24.10.2.4 (history, not
      a warning), and iX's typo branch release/25.20.2.2 is on a line that has no tag at
      all, so it is not a release line. This immediately surfaced two refs that had never
      been checked — release/25.10.5 and release/24.10.2.5 — both of which pass.

      is_unreleased() now keys off where a ref came from (branch = not yet shipped)
      rather than looking for -BETA/-RC in its name. Otherwise release/25.10.5 would
      count as shipped and a break in it would fail the build as a live outage — on a
      version nobody is running yet.

    • An unchanged fingerprint froze the bug report's body, not just its comments. Two
      questions were sharing one answer. Have the findings changed? gates comments
      they notify, and a daily "still broken, same as yesterday" is what teaches everyone
      to ignore the one that finally matters. Is the body still true? gates the body
      and editing an issue body notifies nobody on either forge, so keeping it honest is
      free. Conflated, the report could never be corrected while the findings held steady,
      and the fingerprint deliberately ignores everything that moves on its own — healthy
      rows, the hardware-verified column, point releases, and how a row is labelled. The
      master27-dev relabel above would have reached the README and never the issue
      anybody actually opens. The body is now rewritten whenever it is out of date (after
      normalising line endings, so a forge round-tripping \r\n does not cause a rewrite
      every run) and comments remain strictly a changelog of real changes.

    • A change to the publisher did not re-run the check. compat.yml's push: paths
      listed tools/compat.py but not tools/compat_publish.py — so the very commit that
      taught the bot to refresh a stale report body triggered no run, and the report stayed
      stale until the next scheduled one. A fix nobody runs is a fix nobody has.

    • The compatibility bot filed a new duplicate bug report on every Gitea run.
      find_issue() skipped pull requests by testing for the presence of the
      pull_request key. GitHub omits that key on a plain issue; Gitea sends it as
      null. So on Gitea every issue was discarded as a PR, the lookup always came back
      empty, and the bot took the "nothing filed yet" branch and opened a fresh report
      each run — nine copies on the canonical forge, four of them filed after the
      commit that was meant to stop precisely this. The mirror was fine, which is why it
      went unnoticed: GitHub's payload shape is the one the filter was written against.

      It is the same failure the anti-spam fix was written to prevent, moved from
      comments to issues, and it survived because find_issue was the only function in
      compat_publish.py with no test. It now has one, per forge, and the daily cron —
      which had not yet run once — no longer accumulates a report a day.

      The issue list is also requested with both paging parameters (per_page for
      GitHub, limit for Gitea). Each forge ignores the other's, and Gitea's default page
      is 30, so the lookup would have started missing the report again once the pile it
      was creating grew past one page.

    Downloads
  • v0.6.1 7cc0826c2c

    v0.6.1
    CI / shell (shellcheck + syntax) (push) Successful in 9s
    CI / python 3.11 (push) Successful in 13s
    CI / python 3.13 (push) Successful in 16s
    CI / python 3.12 (push) Successful in 16s
    TrueNAS compatibility / compat (push) Failing after 6s
    Release / release (push) Successful in 16s
    Stable

    flan released this 2026-07-13 16:42:22 -04:00 | 36 commits to main since this release

    Fixed

    • A reboot mid-backup orphaned the entire snapshot tree, permanently. The sidecar
      is the record of which snapshots a run pinned — and it lives in /run, which is
      tmpfs. A reboot (or a crash) between taking the recursive snapshot and cleaning
      it up destroyed that record, leaving one snapshot per descendant dataset — 250+ on
      a real pool
      — with nothing left pointing at them. Nothing would ever have found
      them again.

      gc_stale_snapshots() is the backstop: it identifies leftovers by name, so it
      works when the record is gone. It runs at the start of every backup, after the
      sidecar reclaim — the recorded path stays authoritative, and the collector only ever
      mops up what the record lost.

      Because it deletes data on a name match — a weaker claim than a recorded fact — the
      selection is a pure function with the harshest tests in the suite. A snapshot is
      collected only if all of these hold:

      name is exactly <dataset>@<task>-<YYYYMMDDHHMMSS> so cloud_backup-5 never matches cloud_backup-50, an auto-* periodic snapshot, or anything a human made
      it is not the current run's parent and children are excluded
      nothing is mounted from it an in-flight run pins its own snapshots — this, not the age guard, is what protects a concurrent backup
      it is over an hour old covers the seconds-long window where a live run has snapshotted but not yet mounted

      Verified against the real pool: of 4,728 snapshots — including 2,341 periodic
      ones — it selects exactly the orphans of the task being run, and nothing else.

    Downloads
  • v0.6.1-rc2
    CI / shell (shellcheck + syntax) (push) Successful in 9s
    CI / python 3.11 (push) Successful in 13s
    CI / python 3.13 (push) Successful in 16s
    CI / python 3.12 (push) Successful in 16s
    TrueNAS compatibility / compat (push) Failing after 6s
    Release / release (push) Successful in 16s
    Pre-Release

    flan released this 2026-07-13 16:40:46 -04:00 | 36 commits to main since this release

    Fixed

    • A reboot mid-backup orphaned the entire snapshot tree, permanently. The sidecar
      is the record of which snapshots a run pinned — and it lives in /run, which is
      tmpfs. A reboot (or a crash) between taking the recursive snapshot and cleaning
      it up destroyed that record, leaving one snapshot per descendant dataset — 250+ on
      a real pool
      — with nothing left pointing at them. Nothing would ever have found
      them again.

      gc_stale_snapshots() is the backstop: it identifies leftovers by name, so it
      works when the record is gone. It runs at the start of every backup, after the
      sidecar reclaim — the recorded path stays authoritative, and the collector only ever
      mops up what the record lost.

      Because it deletes data on a name match — a weaker claim than a recorded fact — the
      selection is a pure function with the harshest tests in the suite. A snapshot is
      collected only if all of these hold:

      name is exactly <dataset>@<task>-<YYYYMMDDHHMMSS> so cloud_backup-5 never matches cloud_backup-50, an auto-* periodic snapshot, or anything a human made
      it is not the current run's parent and children are excluded
      nothing is mounted from it an in-flight run pins its own snapshots — this, not the age guard, is what protects a concurrent backup
      it is over an hour old covers the seconds-long window where a live run has snapshotted but not yet mounted

      Verified against the real pool: of 4,728 snapshots — including 2,341 periodic
      ones — it selects exactly the orphans of the task being run, and nothing else.

    Downloads
  • v0.6.1-rc1
    CI / shell (shellcheck + syntax) (push) Successful in 11s
    CI / python 3.11 (push) Successful in 13s
    CI / python 3.12 (push) Successful in 14s
    CI / python 3.13 (push) Successful in 13s
    TrueNAS compatibility / compat (push) Successful in 11s
    Release / release (push) Successful in 14s
    Pre-Release

    flan released this 2026-07-13 15:59:38 -04:00 | 38 commits to main since this release

    Fixed

    • A reboot mid-backup orphaned the entire snapshot tree, permanently. The sidecar
      is the record of which snapshots a run pinned — and it lives in /run, which is
      tmpfs. A reboot (or a crash) between taking the recursive snapshot and cleaning
      it up destroyed that record, leaving one snapshot per descendant dataset — 250+ on
      a real pool
      — with nothing left pointing at them. Nothing would ever have found
      them again.

      gc_stale_snapshots() is the backstop: it identifies leftovers by name, so it
      works when the record is gone. It runs at the start of every backup, after the
      sidecar reclaim — the recorded path stays authoritative, and the collector only ever
      mops up what the record lost.

      Because it deletes data on a name match — a weaker claim than a recorded fact — the
      selection is a pure function with the harshest tests in the suite. A snapshot is
      collected only if all of these hold:

      name is exactly <dataset>@<task>-<YYYYMMDDHHMMSS> so cloud_backup-5 never matches cloud_backup-50, an auto-* periodic snapshot, or anything a human made
      it is not the current run's parent and children are excluded
      nothing is mounted from it an in-flight run pins its own snapshots — this, not the age guard, is what protects a concurrent backup
      it is over an hour old covers the seconds-long window where a live run has snapshotted but not yet mounted

      Verified against the real pool: of 4,728 snapshots — including 2,341 periodic
      ones — it selects exactly the orphans of the task being run, and nothing else.

    Downloads
  • v0.6.0 2b8ef107f7

    v0.6.0
    CI / shell (shellcheck + syntax) (push) Successful in 9s
    CI / python 3.12 (push) Successful in 13s
    CI / python 3.13 (push) Successful in 17s
    CI / python 3.11 (push) Successful in 15s
    TrueNAS compatibility / compat (push) Successful in 11s
    Release / release (push) Successful in 15s
    Stable

    flan released this 2026-07-13 15:46:11 -04:00 | 41 commits to main since this release

    Added

    • release.sh — a two-stage release process, and a barrier that enforces it.
      A stable vX.Y.Z tag is now only publishable if a vX.Y.Z-rcN tag points at the
      same commit, and the release job re-runs the entire suite against that tagged
      commit before publishing. Candidates are invisible to users — update.sh and the
      update alert both take the newest plain vX.Y.Z tag — so debugging happens across
      rc1, rc2, rc3 at nobody's expense, instead of across v0.5.0, v0.5.1, v0.5.2 at
      everybody's.

      bash release.sh 0.6.0 --rc        # candidate. Invisible to users.
      bash release.sh 0.6.0 --promote   # stable. Refused unless an rc passed HERE.
      

      The rule is enforced in tools/release_gate.py, which release.sh runs locally
      (so you fail in 200 ms) and .github/workflows/release.yml runs again where it
      cannot be bypassed (so failing locally is not optional). "The candidate passed,
      then I pushed one more little fix" is refused by name — that is precisely how
      v0.5.1 happened.

    • TrueNAS compatibility is now checked, not hoped for.
      tools/compat.py is a written-down record of everything each
      module assumes about middlewared, checked in two places:

      • CI, daily — against iXsystems' source at every release line including
        master and the current BETA/RC
        . When an unreleased TrueNAS breaks the patch
        it files a bug report automatically, so there is time to fix it before that
        version reaches anyone. It also refreshes the README's support matrix, so the
        table cannot quietly become a false promise.
      • apply.sh, at every boot — against the middleware actually installed on the
        box. A module whose assumptions no longer hold is not applied. Stock TrueNAS
        without a feature beats TrueNAS with a broken backup.

      It immediately found two real breaks: TrueNAS 26 (below), and a nested-snapshot bug
      that had been shipping for two releases (below).

    • The compatibility check now covers the middlewared methods the patch calls,
      not only the symbols it wraps — and that gap was hiding a catastrophe.

      TrueNAS 26 deletes plugins/zfs_/dataset.py and plugins/zfs_/snapshot.py
      outright
      , taking zfs.dataset.query, zfs.snapshot.query and
      zfs.snapshot.delete with them (26 uses filesystem.statfs and zfs.resource.*).
      Nothing about the five cloud_backup files reveals that, so every other check went
      green. The patch would have applied perfectly and then failed on the first
      backup
      — or, far worse, snapshotted successfully and failed to delete,
      orphaning one snapshot per descendant dataset (250 on a real pool) on every
      single run, forever.

      This is now an assumption class of its own, so a method disappearing is a BROKEN
      verdict rather than a silent time bomb.

    • Groundwork for TrueNAS 26 (async→sync and the deleted helper — see below).
      26 is still reported BROKEN and the nested module will not apply there, because
      the ZFS API rewrite above is not yet ported. Porting it needs a real 26 box to
      verify against, and shipping a port nobody has run is exactly the failure this
      project exists to avoid. On 26, TrueNAS is left stock: B2/S3 keeps working, nested
      datasets are simply not covered.

    Fixed

    • A few snapshots leaked on every nested run, forever. Found on real hardware, in
      the one place it could be: a 256-snapshot backup of /mnt/Tap swept 253 cleanly and
      left 3 behind with dataset is busy.

      The cause is ZFS's own automount. Reading anything under
      <dataset>/.zfs/snapshot/<snap>/ makes ZFS automount that snapshot, and it stays
      mounted for zfs_expire_snapshot seconds (300 by default) after the last access.
      teardown() unmounts our bind mounts — but not the automount underneath — so
      zfs destroy refuses for exactly the datasets restic read most recently. Then
      cleanup_task() removed the sidecar anyway, destroying the only record that those
      snapshots existed. Nothing would ever have reclaimed them.

      Three changes, and the third is the one that makes it safe rather than merely
      unlikely:

      • release_snapdirs() unmounts ZFS's own .zfs/snapshot automounts (deepest first)
        before deleting, so the snapshots are not busy in the first place.
      • delete_snapshot_tree() retries the transient busy, and returns the
        snapshots it could not delete
        instead of swallowing them.
      • The sidecar is now removed only on a confirmed-clean sweep — including on the
        staging-failure path, which used to remove it before the caller swept. The
        asymmetry is deliberate: a sidecar left behind when the tree is already gone costs
        one no-op delete on the next run, while a sidecar removed while the tree still
        exists is unrecoverable. Survivors are reclaimed by the next run.

      Expect the occasional straggler, and expect it to clean itself up. On a
      256-snapshot tree this reliably sweeps ~255 immediately and may leave one: it is
      whatever restic read last, so its 300-second window has barely opened. That one is
      logged, its sidecar is kept, and the next run reclaims it before doing anything else.
      The leak is bounded at a single cycle rather than growing without limit — which is
      the property that actually matters. Blocking a backup job for five minutes to chase
      the last snapshot would be a worse trade, so it is not made.

    • Installing the patch permanently blocked updating it. install.sh does
      chmod +x update.sh, and git recorded update.sh as 100644 — so the chmod was a
      tracked modification, and update.sh refuses to run over a dirty tree. Install
      once and you could never update again; the error even told you to run
      git checkout -- ., which just undoes the exec bit so the next install can re-dirty
      it. A real box sat on an old version for exactly this reason.

      Fixed on both sides: the scripts install.sh chmods are now executable in git (so
      the chmod is a no-op), and update.sh's dirty check now looks at content, not
      file mode — git diff --numstat reports 0 0 for a mode-only change. A test
      asserts every script in install.sh's chmod loop is already 100755 in git.

    • Nested snapshots were broken on TrueNAS 24.10 and 25.04, and had been all
      along.
      SYNC_BLOCK's wrapper spelled out the stock signature and forwarded five
      arguments — but those releases declare restic_backup(middleware, job, cloud_backup, dry_run); rate_limit only arrived in 25.10. Every nested backup on
      24.10/25.04 raised TypeError: restic_backup() takes 4 positional arguments but 5 were given. The wrapper now takes *args, **kwargs and forwards whatever it is
      handed, so a trailing parameter appearing or disappearing is a non-event.

      Found by the new compatibility check, not by a user — which is the whole argument
      for having it. The check it replaced only asked whether the parameter names still
      appeared somewhere in the signature, so it happily passed a call that could never
      work.

    • The nested module is now one synchronous implementation behind two thin
      wrappers.
      TrueNAS 26 rewrites cloud_backup from async to synchronous and
      separately deletes get_dataset_recursive(), which an injected block called out
      of the host module's namespace. Either alone is a broken backup found at restore
      time: an async def wrapper hands sync.py a coroutine where it unpacks a tuple,
      and the vanished helper is a straight NameError.

      The module now talks to middlewared through call_sync, and apply.sh reads which
      flavour the installed middleware declares and injects the matching wrapper —
      TrueNAS ≤ 25.10 reaches it via await middleware.run_in_thread(...); a synchronous
      TrueNAS, already in a worker thread, calls it directly. The logic that owns the
      snapshots, the bind mounts and the failure modes exists once; an async twin
      would mean every future fix had to land twice, and the one that got missed would be
      the one that eats a backup. A middleware whose three wrapped functions disagree
      about async-ness is refused outright rather than guessed at, and
      get_dataset_recursive is carried as our own copy — removing the dependency on both
      versions instead of asserting it.

    • The patch no longer reaches into CloudSync tasks it has no business touching.
      create_snapshot is module-global in plugins/cloud/snapshot.py and is imported by
      cloud_sync.py as well as cloud_backup/sync.py — so the wrapper sat in the
      path of every rclone/Storj CloudSync task with snapshot=true, and issued a
      zfs.dataset.query before deciding it had nothing to do. That added a brand-new
      failure mode to jobs that worked fine before this patch was installed, and worse: a
      CloudSync task that ever did get staged would never be torn down, because the
      teardown is wired into cloud_backup's restic_backup and CRUD_BLOCK
      deliberately leaves CloudSync's nesting guard intact — the bind mounts would pin the
      ZFS snapshot forever. The staging path now bails out immediately unless the snapshot
      is named cloud_backup-*, before any middleware call.

    • Teardown warnings are no longer silently swallowed on TrueNAS ≤ 25.10. The async
      wrapper's finally dropped the logger= kwarg that the sync one passes, so a
      cleanup that failed to unmount a bind mount or to delete a snapshot tree logged
      nothing at all — on the only platform anyone actually runs. run_in_thread
      forwards **kwargs via functools.partial; it was a regression, not a limitation.

    • do_delete is recognised as delete. TrueNAS 24.10 and 25.04 declare
      do_delete (the CRUDService convention); 25.10 renamed it to delete. Both
      answer to zfs.snapshot.delete. Accepting only the literal name reported both older
      releases as BROKEN — a false verdict that would have switched nested snapshots off
      on boxes where they work perfectly.

    • An incompatible TrueNAS no longer sets the permanent kill switch. apply.sh
      reused a "nothing left to do" exit that touches disabled, which suppresses
      patching on every future boot and is cleared only by install.sh — never by
      update.sh. On TrueNAS 26 (providers-compatible, nested opt-out by default) that
      branch would have fired, and the very release that fixed 26 could not have
      re-enabled itself: the user would run bash update.sh, exactly as the update alert
      tells them to, and the patch would stay dead with their B2 backups off.
      Incompatibility now means "apply nothing this boot, try again next boot".
      Retirement and incompatibility are opposite situations and no longer share an exit.

    • The compatibility check itself could be fooled, in ways that each had teeth: a
      reordered, keyword-only, or newly-required parameter now reads as broken (the patch
      calls these positionally); a re-exported or conditionally-defined symbol reads
      as unknown rather than broken, so an innocent upstream refactor cannot make a
      working module decline to apply; an unreadable source (rate limit, DNS, timeout)
      is unknown rather than "iXsystems deleted this file", so a network blip cannot
      file a bug report, fail CI, and repaint the published support matrix; and native
      no longer masks BROKEN, which used to render a TrueNAS that both reworded the
      nesting guard and reshaped the functions as good news.

    • compat.py --tree no longer reads the patch's own code as native support.
      B2_BLOCK writes B2RcloneRemote.restic = True into b2.py — exactly the string
      the providers native-probe looks for — so the one command the docs recommend for
      checking a live box said "retire the providers module" on every patched machine.
      It now reads only the part of the file iXsystems wrote.

    • release.sh --promote could never succeed. It refused to run if the stable tag
      existed, and the gate refused if it did not — mutually exclusive, so the only way to
      cut a stable release was to hand-tag and bypass every gate this work exists to
      enforce. The gate now resolves the tag's commit if it exists and HEAD otherwise.
      The tests hid it by always tagging first.

    Changed

    • The minimum supported TrueNAS is stated, and enforced: 24.10. TrueCloud Backup
      does not exist before it — plugins/cloud_backup/ is simply absent — so the patch
      had nothing to attach to and would have done nothing at all, silently, while the
      user believed their backups were configured. install.sh now reads
      system.version and refuses, naming the reason. A version it cannot parse is a
      warning, not a refusal: declining to install over a string we failed to read would
      be a worse failure than the one being prevented.

    • A stable release may not leave work stranded under ## Unreleased. Either it
      is finished and belongs in the release, or the release is premature. Candidates
      are exempt: an rc may legitimately have work queued behind it.

    • release.sh refuses to run on an installed box. The whole repo is cloned onto
      every box, so this file is there too; update.sh pins the checkout to a tag in
      detached HEAD, and release.sh now recognises that and says so, rather than
      emitting a confusing branch error.

    • Gitea (git.onetick.ninja/flan/truenas-truecloud-patch) is now canonical, with
      GitHub as a mirror. Both forges run the same workflows and publish the same
      releases. The update alert now derives the changelog URL from the origin
      remote
      instead of hard-coding GitHub — which matters more than it sounds: when
      the changelog cannot be read, the alert deliberately fires anyway rather than risk
      hiding a security fix, so a stale URL would not have disabled the alert, it would
      have made it nag on every release, including documentation-only ones.

    Security

    • Workflow expressions are no longer interpolated into shell.
      echo "${{ steps.report.outputs.body }}" pasted the compatibility report into the
      script text, and the report is full of backticks — bash ran create-snapshot,
      def and async as commands. Since that report is built from iXsystems' source,
      anything landing in their tree would have executed on the runner. inputs.tag on
      workflow_dispatch had the same shape, and that one is attacker-chosen. Data now
      moves through files and scalars through env:; a test enforces it across every
      workflow.

    Internal

    • Static-analysis annotations in patch/alert_source.py (# noqa placement). No
      runtime change.
    Downloads
  • v0.6.0-rc6
    CI / shell (shellcheck + syntax) (push) Successful in 9s
    CI / python 3.12 (push) Successful in 13s
    CI / python 3.13 (push) Successful in 17s
    CI / python 3.11 (push) Successful in 15s
    TrueNAS compatibility / compat (push) Successful in 11s
    Release / release (push) Successful in 15s
    Pre-Release

    flan released this 2026-07-13 15:35:47 -04:00 | 41 commits to main since this release

    Added

    • release.sh — a two-stage release process, and a barrier that enforces it.
      A stable vX.Y.Z tag is now only publishable if a vX.Y.Z-rcN tag points at the
      same commit, and the release job re-runs the entire suite against that tagged
      commit before publishing. Candidates are invisible to users — update.sh and the
      update alert both take the newest plain vX.Y.Z tag — so debugging happens across
      rc1, rc2, rc3 at nobody's expense, instead of across v0.5.0, v0.5.1, v0.5.2 at
      everybody's.

      bash release.sh 0.6.0 --rc        # candidate. Invisible to users.
      bash release.sh 0.6.0 --promote   # stable. Refused unless an rc passed HERE.
      

      The rule is enforced in tools/release_gate.py, which release.sh runs locally
      (so you fail in 200 ms) and .github/workflows/release.yml runs again where it
      cannot be bypassed (so failing locally is not optional). "The candidate passed,
      then I pushed one more little fix" is refused by name — that is precisely how
      v0.5.1 happened.

    • TrueNAS compatibility is now checked, not hoped for.
      tools/compat.py is a written-down record of everything each
      module assumes about middlewared, checked in two places:

      • CI, daily — against iXsystems' source at every release line including
        master and the current BETA/RC
        . When an unreleased TrueNAS breaks the patch
        it files a bug report automatically, so there is time to fix it before that
        version reaches anyone. It also refreshes the README's support matrix, so the
        table cannot quietly become a false promise.
      • apply.sh, at every boot — against the middleware actually installed on the
        box. A module whose assumptions no longer hold is not applied. Stock TrueNAS
        without a feature beats TrueNAS with a broken backup.

      It immediately found two real breaks: TrueNAS 26 (below), and a nested-snapshot bug
      that had been shipping for two releases (below).

    • The compatibility check now covers the middlewared methods the patch calls,
      not only the symbols it wraps — and that gap was hiding a catastrophe.

      TrueNAS 26 deletes plugins/zfs_/dataset.py and plugins/zfs_/snapshot.py
      outright
      , taking zfs.dataset.query, zfs.snapshot.query and
      zfs.snapshot.delete with them (26 uses filesystem.statfs and zfs.resource.*).
      Nothing about the five cloud_backup files reveals that, so every other check went
      green. The patch would have applied perfectly and then failed on the first
      backup
      — or, far worse, snapshotted successfully and failed to delete,
      orphaning one snapshot per descendant dataset (250 on a real pool) on every
      single run, forever.

      This is now an assumption class of its own, so a method disappearing is a BROKEN
      verdict rather than a silent time bomb.

    • Groundwork for TrueNAS 26 (async→sync and the deleted helper — see below).
      26 is still reported BROKEN and the nested module will not apply there, because
      the ZFS API rewrite above is not yet ported. Porting it needs a real 26 box to
      verify against, and shipping a port nobody has run is exactly the failure this
      project exists to avoid. On 26, TrueNAS is left stock: B2/S3 keeps working, nested
      datasets are simply not covered.

    Fixed

    • A few snapshots leaked on every nested run, forever. Found on real hardware, in
      the one place it could be: a 256-snapshot backup of /mnt/Tap swept 253 cleanly and
      left 3 behind with dataset is busy.

      The cause is ZFS's own automount. Reading anything under
      <dataset>/.zfs/snapshot/<snap>/ makes ZFS automount that snapshot, and it stays
      mounted for zfs_expire_snapshot seconds (300 by default) after the last access.
      teardown() unmounts our bind mounts — but not the automount underneath — so
      zfs destroy refuses for exactly the datasets restic read most recently. Then
      cleanup_task() removed the sidecar anyway, destroying the only record that those
      snapshots existed. Nothing would ever have reclaimed them.

      Three changes, and the third is the one that makes it safe rather than merely
      unlikely:

      • release_snapdirs() unmounts ZFS's own .zfs/snapshot automounts (deepest first)
        before deleting, so the snapshots are not busy in the first place.
      • delete_snapshot_tree() retries the transient busy, and returns the
        snapshots it could not delete
        instead of swallowing them.
      • The sidecar is now removed only on a confirmed-clean sweep — including on the
        staging-failure path, which used to remove it before the caller swept. The
        asymmetry is deliberate: a sidecar left behind when the tree is already gone costs
        one no-op delete on the next run, while a sidecar removed while the tree still
        exists is unrecoverable. Survivors are reclaimed by the next run.

      Expect the occasional straggler, and expect it to clean itself up. On a
      256-snapshot tree this reliably sweeps ~255 immediately and may leave one: it is
      whatever restic read last, so its 300-second window has barely opened. That one is
      logged, its sidecar is kept, and the next run reclaims it before doing anything else.
      The leak is bounded at a single cycle rather than growing without limit — which is
      the property that actually matters. Blocking a backup job for five minutes to chase
      the last snapshot would be a worse trade, so it is not made.

    • Installing the patch permanently blocked updating it. install.sh does
      chmod +x update.sh, and git recorded update.sh as 100644 — so the chmod was a
      tracked modification, and update.sh refuses to run over a dirty tree. Install
      once and you could never update again; the error even told you to run
      git checkout -- ., which just undoes the exec bit so the next install can re-dirty
      it. A real box sat on an old version for exactly this reason.

      Fixed on both sides: the scripts install.sh chmods are now executable in git (so
      the chmod is a no-op), and update.sh's dirty check now looks at content, not
      file mode — git diff --numstat reports 0 0 for a mode-only change. A test
      asserts every script in install.sh's chmod loop is already 100755 in git.

    • Nested snapshots were broken on TrueNAS 24.10 and 25.04, and had been all
      along.
      SYNC_BLOCK's wrapper spelled out the stock signature and forwarded five
      arguments — but those releases declare restic_backup(middleware, job, cloud_backup, dry_run); rate_limit only arrived in 25.10. Every nested backup on
      24.10/25.04 raised TypeError: restic_backup() takes 4 positional arguments but 5 were given. The wrapper now takes *args, **kwargs and forwards whatever it is
      handed, so a trailing parameter appearing or disappearing is a non-event.

      Found by the new compatibility check, not by a user — which is the whole argument
      for having it. The check it replaced only asked whether the parameter names still
      appeared somewhere in the signature, so it happily passed a call that could never
      work.

    • The nested module is now one synchronous implementation behind two thin
      wrappers.
      TrueNAS 26 rewrites cloud_backup from async to synchronous and
      separately deletes get_dataset_recursive(), which an injected block called out
      of the host module's namespace. Either alone is a broken backup found at restore
      time: an async def wrapper hands sync.py a coroutine where it unpacks a tuple,
      and the vanished helper is a straight NameError.

      The module now talks to middlewared through call_sync, and apply.sh reads which
      flavour the installed middleware declares and injects the matching wrapper —
      TrueNAS ≤ 25.10 reaches it via await middleware.run_in_thread(...); a synchronous
      TrueNAS, already in a worker thread, calls it directly. The logic that owns the
      snapshots, the bind mounts and the failure modes exists once; an async twin
      would mean every future fix had to land twice, and the one that got missed would be
      the one that eats a backup. A middleware whose three wrapped functions disagree
      about async-ness is refused outright rather than guessed at, and
      get_dataset_recursive is carried as our own copy — removing the dependency on both
      versions instead of asserting it.

    • The patch no longer reaches into CloudSync tasks it has no business touching.
      create_snapshot is module-global in plugins/cloud/snapshot.py and is imported by
      cloud_sync.py as well as cloud_backup/sync.py — so the wrapper sat in the
      path of every rclone/Storj CloudSync task with snapshot=true, and issued a
      zfs.dataset.query before deciding it had nothing to do. That added a brand-new
      failure mode to jobs that worked fine before this patch was installed, and worse: a
      CloudSync task that ever did get staged would never be torn down, because the
      teardown is wired into cloud_backup's restic_backup and CRUD_BLOCK
      deliberately leaves CloudSync's nesting guard intact — the bind mounts would pin the
      ZFS snapshot forever. The staging path now bails out immediately unless the snapshot
      is named cloud_backup-*, before any middleware call.

    • Teardown warnings are no longer silently swallowed on TrueNAS ≤ 25.10. The async
      wrapper's finally dropped the logger= kwarg that the sync one passes, so a
      cleanup that failed to unmount a bind mount or to delete a snapshot tree logged
      nothing at all — on the only platform anyone actually runs. run_in_thread
      forwards **kwargs via functools.partial; it was a regression, not a limitation.

    • do_delete is recognised as delete. TrueNAS 24.10 and 25.04 declare
      do_delete (the CRUDService convention); 25.10 renamed it to delete. Both
      answer to zfs.snapshot.delete. Accepting only the literal name reported both older
      releases as BROKEN — a false verdict that would have switched nested snapshots off
      on boxes where they work perfectly.

    • An incompatible TrueNAS no longer sets the permanent kill switch. apply.sh
      reused a "nothing left to do" exit that touches disabled, which suppresses
      patching on every future boot and is cleared only by install.sh — never by
      update.sh. On TrueNAS 26 (providers-compatible, nested opt-out by default) that
      branch would have fired, and the very release that fixed 26 could not have
      re-enabled itself: the user would run bash update.sh, exactly as the update alert
      tells them to, and the patch would stay dead with their B2 backups off.
      Incompatibility now means "apply nothing this boot, try again next boot".
      Retirement and incompatibility are opposite situations and no longer share an exit.

    • The compatibility check itself could be fooled, in ways that each had teeth: a
      reordered, keyword-only, or newly-required parameter now reads as broken (the patch
      calls these positionally); a re-exported or conditionally-defined symbol reads
      as unknown rather than broken, so an innocent upstream refactor cannot make a
      working module decline to apply; an unreadable source (rate limit, DNS, timeout)
      is unknown rather than "iXsystems deleted this file", so a network blip cannot
      file a bug report, fail CI, and repaint the published support matrix; and native
      no longer masks BROKEN, which used to render a TrueNAS that both reworded the
      nesting guard and reshaped the functions as good news.

    • compat.py --tree no longer reads the patch's own code as native support.
      B2_BLOCK writes B2RcloneRemote.restic = True into b2.py — exactly the string
      the providers native-probe looks for — so the one command the docs recommend for
      checking a live box said "retire the providers module" on every patched machine.
      It now reads only the part of the file iXsystems wrote.

    • release.sh --promote could never succeed. It refused to run if the stable tag
      existed, and the gate refused if it did not — mutually exclusive, so the only way to
      cut a stable release was to hand-tag and bypass every gate this work exists to
      enforce. The gate now resolves the tag's commit if it exists and HEAD otherwise.
      The tests hid it by always tagging first.

    Changed

    • The minimum supported TrueNAS is stated, and enforced: 24.10. TrueCloud Backup
      does not exist before it — plugins/cloud_backup/ is simply absent — so the patch
      had nothing to attach to and would have done nothing at all, silently, while the
      user believed their backups were configured. install.sh now reads
      system.version and refuses, naming the reason. A version it cannot parse is a
      warning, not a refusal: declining to install over a string we failed to read would
      be a worse failure than the one being prevented.

    • A stable release may not leave work stranded under ## Unreleased. Either it
      is finished and belongs in the release, or the release is premature. Candidates
      are exempt: an rc may legitimately have work queued behind it.

    • release.sh refuses to run on an installed box. The whole repo is cloned onto
      every box, so this file is there too; update.sh pins the checkout to a tag in
      detached HEAD, and release.sh now recognises that and says so, rather than
      emitting a confusing branch error.

    • Gitea (git.onetick.ninja/flan/truenas-truecloud-patch) is now canonical, with
      GitHub as a mirror. Both forges run the same workflows and publish the same
      releases. The update alert now derives the changelog URL from the origin
      remote
      instead of hard-coding GitHub — which matters more than it sounds: when
      the changelog cannot be read, the alert deliberately fires anyway rather than risk
      hiding a security fix, so a stale URL would not have disabled the alert, it would
      have made it nag on every release, including documentation-only ones.

    Security

    • Workflow expressions are no longer interpolated into shell.
      echo "${{ steps.report.outputs.body }}" pasted the compatibility report into the
      script text, and the report is full of backticks — bash ran create-snapshot,
      def and async as commands. Since that report is built from iXsystems' source,
      anything landing in their tree would have executed on the runner. inputs.tag on
      workflow_dispatch had the same shape, and that one is attacker-chosen. Data now
      moves through files and scalars through env:; a test enforces it across every
      workflow.

    Internal

    • Static-analysis annotations in patch/alert_source.py (# noqa placement). No
      runtime change.
    Downloads
  • v0.6.0-rc5
    CI / shell (shellcheck + syntax) (push) Successful in 10s
    CI / python 3.11 (push) Successful in 12s
    CI / python 3.12 (push) Successful in 15s
    CI / python 3.13 (push) Successful in 17s
    TrueNAS compatibility / compat (push) Successful in 13s
    Release / release (push) Successful in 14s
    Pre-Release

    flan released this 2026-07-13 15:18:25 -04:00 | 42 commits to main since this release

    Added

    • release.sh — a two-stage release process, and a barrier that enforces it.
      A stable vX.Y.Z tag is now only publishable if a vX.Y.Z-rcN tag points at the
      same commit, and the release job re-runs the entire suite against that tagged
      commit before publishing. Candidates are invisible to users — update.sh and the
      update alert both take the newest plain vX.Y.Z tag — so debugging happens across
      rc1, rc2, rc3 at nobody's expense, instead of across v0.5.0, v0.5.1, v0.5.2 at
      everybody's.

      bash release.sh 0.6.0 --rc        # candidate. Invisible to users.
      bash release.sh 0.6.0 --promote   # stable. Refused unless an rc passed HERE.
      

      The rule is enforced in tools/release_gate.py, which release.sh runs locally
      (so you fail in 200 ms) and .github/workflows/release.yml runs again where it
      cannot be bypassed (so failing locally is not optional). "The candidate passed,
      then I pushed one more little fix" is refused by name — that is precisely how
      v0.5.1 happened.

    • TrueNAS compatibility is now checked, not hoped for.
      tools/compat.py is a written-down record of everything each
      module assumes about middlewared, checked in two places:

      • CI, daily — against iXsystems' source at every release line including
        master and the current BETA/RC
        . When an unreleased TrueNAS breaks the patch
        it files a bug report automatically, so there is time to fix it before that
        version reaches anyone. It also refreshes the README's support matrix, so the
        table cannot quietly become a false promise.
      • apply.sh, at every boot — against the middleware actually installed on the
        box. A module whose assumptions no longer hold is not applied. Stock TrueNAS
        without a feature beats TrueNAS with a broken backup.

      It immediately found two real breaks: TrueNAS 26 (below), and a nested-snapshot bug
      that had been shipping for two releases (below).

    • The compatibility check now covers the middlewared methods the patch calls,
      not only the symbols it wraps — and that gap was hiding a catastrophe.

      TrueNAS 26 deletes plugins/zfs_/dataset.py and plugins/zfs_/snapshot.py
      outright
      , taking zfs.dataset.query, zfs.snapshot.query and
      zfs.snapshot.delete with them (26 uses filesystem.statfs and zfs.resource.*).
      Nothing about the five cloud_backup files reveals that, so every other check went
      green. The patch would have applied perfectly and then failed on the first
      backup
      — or, far worse, snapshotted successfully and failed to delete,
      orphaning one snapshot per descendant dataset (250 on a real pool) on every
      single run, forever.

      This is now an assumption class of its own, so a method disappearing is a BROKEN
      verdict rather than a silent time bomb.

    • Groundwork for TrueNAS 26 (async→sync and the deleted helper — see below).
      26 is still reported BROKEN and the nested module will not apply there, because
      the ZFS API rewrite above is not yet ported. Porting it needs a real 26 box to
      verify against, and shipping a port nobody has run is exactly the failure this
      project exists to avoid. On 26, TrueNAS is left stock: B2/S3 keeps working, nested
      datasets are simply not covered.

    Fixed

    • A few snapshots leaked on every nested run, forever. Found on real hardware, in
      the one place it could be: a 256-snapshot backup of /mnt/Tap swept 253 cleanly and
      left 3 behind with dataset is busy.

      The cause is ZFS's own automount. Reading anything under
      <dataset>/.zfs/snapshot/<snap>/ makes ZFS automount that snapshot, and it stays
      mounted for zfs_expire_snapshot seconds (300 by default) after the last access.
      teardown() unmounts our bind mounts — but not the automount underneath — so
      zfs destroy refuses for exactly the datasets restic read most recently. Then
      cleanup_task() removed the sidecar anyway, destroying the only record that those
      snapshots existed. Nothing would ever have reclaimed them.

      Three changes, and the third is the one that makes it safe rather than merely
      unlikely:

      • release_snapdirs() unmounts ZFS's own .zfs/snapshot automounts (deepest first)
        before deleting, so the snapshots are not busy in the first place.
      • delete_snapshot_tree() retries the transient busy, and returns the
        snapshots it could not delete
        instead of swallowing them.
      • The sidecar is now removed only on a confirmed-clean sweep — including on the
        staging-failure path, which used to remove it before the caller swept. The
        asymmetry is deliberate: a sidecar left behind when the tree is already gone costs
        one no-op delete on the next run, while a sidecar removed while the tree still
        exists is unrecoverable. Survivors are reclaimed by the next run.
    • Installing the patch permanently blocked updating it. install.sh does
      chmod +x update.sh, and git recorded update.sh as 100644 — so the chmod was a
      tracked modification, and update.sh refuses to run over a dirty tree. Install
      once and you could never update again; the error even told you to run
      git checkout -- ., which just undoes the exec bit so the next install can re-dirty
      it. A real box sat on an old version for exactly this reason.

      Fixed on both sides: the scripts install.sh chmods are now executable in git (so
      the chmod is a no-op), and update.sh's dirty check now looks at content, not
      file mode — git diff --numstat reports 0 0 for a mode-only change. A test
      asserts every script in install.sh's chmod loop is already 100755 in git.

    • Nested snapshots were broken on TrueNAS 24.10 and 25.04, and had been all
      along.
      SYNC_BLOCK's wrapper spelled out the stock signature and forwarded five
      arguments — but those releases declare restic_backup(middleware, job, cloud_backup, dry_run); rate_limit only arrived in 25.10. Every nested backup on
      24.10/25.04 raised TypeError: restic_backup() takes 4 positional arguments but 5 were given. The wrapper now takes *args, **kwargs and forwards whatever it is
      handed, so a trailing parameter appearing or disappearing is a non-event.

      Found by the new compatibility check, not by a user — which is the whole argument
      for having it. The check it replaced only asked whether the parameter names still
      appeared somewhere in the signature, so it happily passed a call that could never
      work.

    • The nested module is now one synchronous implementation behind two thin
      wrappers.
      TrueNAS 26 rewrites cloud_backup from async to synchronous and
      separately deletes get_dataset_recursive(), which an injected block called out
      of the host module's namespace. Either alone is a broken backup found at restore
      time: an async def wrapper hands sync.py a coroutine where it unpacks a tuple,
      and the vanished helper is a straight NameError.

      The module now talks to middlewared through call_sync, and apply.sh reads which
      flavour the installed middleware declares and injects the matching wrapper —
      TrueNAS ≤ 25.10 reaches it via await middleware.run_in_thread(...); a synchronous
      TrueNAS, already in a worker thread, calls it directly. The logic that owns the
      snapshots, the bind mounts and the failure modes exists once; an async twin
      would mean every future fix had to land twice, and the one that got missed would be
      the one that eats a backup. A middleware whose three wrapped functions disagree
      about async-ness is refused outright rather than guessed at, and
      get_dataset_recursive is carried as our own copy — removing the dependency on both
      versions instead of asserting it.

    • The patch no longer reaches into CloudSync tasks it has no business touching.
      create_snapshot is module-global in plugins/cloud/snapshot.py and is imported by
      cloud_sync.py as well as cloud_backup/sync.py — so the wrapper sat in the
      path of every rclone/Storj CloudSync task with snapshot=true, and issued a
      zfs.dataset.query before deciding it had nothing to do. That added a brand-new
      failure mode to jobs that worked fine before this patch was installed, and worse: a
      CloudSync task that ever did get staged would never be torn down, because the
      teardown is wired into cloud_backup's restic_backup and CRUD_BLOCK
      deliberately leaves CloudSync's nesting guard intact — the bind mounts would pin the
      ZFS snapshot forever. The staging path now bails out immediately unless the snapshot
      is named cloud_backup-*, before any middleware call.

    • Teardown warnings are no longer silently swallowed on TrueNAS ≤ 25.10. The async
      wrapper's finally dropped the logger= kwarg that the sync one passes, so a
      cleanup that failed to unmount a bind mount or to delete a snapshot tree logged
      nothing at all — on the only platform anyone actually runs. run_in_thread
      forwards **kwargs via functools.partial; it was a regression, not a limitation.

    • do_delete is recognised as delete. TrueNAS 24.10 and 25.04 declare
      do_delete (the CRUDService convention); 25.10 renamed it to delete. Both
      answer to zfs.snapshot.delete. Accepting only the literal name reported both older
      releases as BROKEN — a false verdict that would have switched nested snapshots off
      on boxes where they work perfectly.

    • An incompatible TrueNAS no longer sets the permanent kill switch. apply.sh
      reused a "nothing left to do" exit that touches disabled, which suppresses
      patching on every future boot and is cleared only by install.sh — never by
      update.sh. On TrueNAS 26 (providers-compatible, nested opt-out by default) that
      branch would have fired, and the very release that fixed 26 could not have
      re-enabled itself: the user would run bash update.sh, exactly as the update alert
      tells them to, and the patch would stay dead with their B2 backups off.
      Incompatibility now means "apply nothing this boot, try again next boot".
      Retirement and incompatibility are opposite situations and no longer share an exit.

    • The compatibility check itself could be fooled, in ways that each had teeth: a
      reordered, keyword-only, or newly-required parameter now reads as broken (the patch
      calls these positionally); a re-exported or conditionally-defined symbol reads
      as unknown rather than broken, so an innocent upstream refactor cannot make a
      working module decline to apply; an unreadable source (rate limit, DNS, timeout)
      is unknown rather than "iXsystems deleted this file", so a network blip cannot
      file a bug report, fail CI, and repaint the published support matrix; and native
      no longer masks BROKEN, which used to render a TrueNAS that both reworded the
      nesting guard and reshaped the functions as good news.

    • compat.py --tree no longer reads the patch's own code as native support.
      B2_BLOCK writes B2RcloneRemote.restic = True into b2.py — exactly the string
      the providers native-probe looks for — so the one command the docs recommend for
      checking a live box said "retire the providers module" on every patched machine.
      It now reads only the part of the file iXsystems wrote.

    • release.sh --promote could never succeed. It refused to run if the stable tag
      existed, and the gate refused if it did not — mutually exclusive, so the only way to
      cut a stable release was to hand-tag and bypass every gate this work exists to
      enforce. The gate now resolves the tag's commit if it exists and HEAD otherwise.
      The tests hid it by always tagging first.

    Changed

    • The minimum supported TrueNAS is stated, and enforced: 24.10. TrueCloud Backup
      does not exist before it — plugins/cloud_backup/ is simply absent — so the patch
      had nothing to attach to and would have done nothing at all, silently, while the
      user believed their backups were configured. install.sh now reads
      system.version and refuses, naming the reason. A version it cannot parse is a
      warning, not a refusal: declining to install over a string we failed to read would
      be a worse failure than the one being prevented.

    • A stable release may not leave work stranded under ## Unreleased. Either it
      is finished and belongs in the release, or the release is premature. Candidates
      are exempt: an rc may legitimately have work queued behind it.

    • release.sh refuses to run on an installed box. The whole repo is cloned onto
      every box, so this file is there too; update.sh pins the checkout to a tag in
      detached HEAD, and release.sh now recognises that and says so, rather than
      emitting a confusing branch error.

    • Gitea (git.onetick.ninja/flan/truenas-truecloud-patch) is now canonical, with
      GitHub as a mirror. Both forges run the same workflows and publish the same
      releases. The update alert now derives the changelog URL from the origin
      remote
      instead of hard-coding GitHub — which matters more than it sounds: when
      the changelog cannot be read, the alert deliberately fires anyway rather than risk
      hiding a security fix, so a stale URL would not have disabled the alert, it would
      have made it nag on every release, including documentation-only ones.

    Security

    • Workflow expressions are no longer interpolated into shell.
      echo "${{ steps.report.outputs.body }}" pasted the compatibility report into the
      script text, and the report is full of backticks — bash ran create-snapshot,
      def and async as commands. Since that report is built from iXsystems' source,
      anything landing in their tree would have executed on the runner. inputs.tag on
      workflow_dispatch had the same shape, and that one is attacker-chosen. Data now
      moves through files and scalars through env:; a test enforces it across every
      workflow.

    Internal

    • Static-analysis annotations in patch/alert_source.py (# noqa placement). No
      runtime change.
    Downloads
  • v0.6.0-rc4
    CI / shell (shellcheck + syntax) (push) Successful in 9s
    CI / python 3.11 (push) Successful in 14s
    CI / python 3.12 (push) Successful in 16s
    CI / python 3.13 (push) Successful in 17s
    TrueNAS compatibility / compat (push) Successful in 11s
    Release / release (push) Successful in 14s
    Pre-Release

    flan released this 2026-07-13 14:58:49 -04:00 | 45 commits to main since this release

    Added

    • release.sh — a two-stage release process, and a barrier that enforces it.
      A stable vX.Y.Z tag is now only publishable if a vX.Y.Z-rcN tag points at the
      same commit, and the release job re-runs the entire suite against that tagged
      commit before publishing. Candidates are invisible to users — update.sh and the
      update alert both take the newest plain vX.Y.Z tag — so debugging happens across
      rc1, rc2, rc3 at nobody's expense, instead of across v0.5.0, v0.5.1, v0.5.2 at
      everybody's.

      bash release.sh 0.6.0 --rc        # candidate. Invisible to users.
      bash release.sh 0.6.0 --promote   # stable. Refused unless an rc passed HERE.
      

      The rule is enforced in tools/release_gate.py, which release.sh runs locally
      (so you fail in 200 ms) and .github/workflows/release.yml runs again where it
      cannot be bypassed (so failing locally is not optional). "The candidate passed,
      then I pushed one more little fix" is refused by name — that is precisely how
      v0.5.1 happened.

    • TrueNAS compatibility is now checked, not hoped for.
      tools/compat.py is a written-down record of everything each
      module assumes about middlewared, checked in two places:

      • CI, daily — against iXsystems' source at every release line including
        master and the current BETA/RC
        . When an unreleased TrueNAS breaks the patch
        it files a bug report automatically, so there is time to fix it before that
        version reaches anyone. It also refreshes the README's support matrix, so the
        table cannot quietly become a false promise.
      • apply.sh, at every boot — against the middleware actually installed on the
        box. A module whose assumptions no longer hold is not applied. Stock TrueNAS
        without a feature beats TrueNAS with a broken backup.

      It immediately found two real breaks: TrueNAS 26 (below), and a nested-snapshot bug
      that had been shipping for two releases (below).

    • The compatibility check now covers the middlewared methods the patch calls,
      not only the symbols it wraps — and that gap was hiding a catastrophe.

      TrueNAS 26 deletes plugins/zfs_/dataset.py and plugins/zfs_/snapshot.py
      outright
      , taking zfs.dataset.query, zfs.snapshot.query and
      zfs.snapshot.delete with them (26 uses filesystem.statfs and zfs.resource.*).
      Nothing about the five cloud_backup files reveals that, so every other check went
      green. The patch would have applied perfectly and then failed on the first
      backup
      — or, far worse, snapshotted successfully and failed to delete,
      orphaning one snapshot per descendant dataset (250 on a real pool) on every
      single run, forever.

      This is now an assumption class of its own, so a method disappearing is a BROKEN
      verdict rather than a silent time bomb.

    • Groundwork for TrueNAS 26 (async→sync and the deleted helper — see below).
      26 is still reported BROKEN and the nested module will not apply there, because
      the ZFS API rewrite above is not yet ported. Porting it needs a real 26 box to
      verify against, and shipping a port nobody has run is exactly the failure this
      project exists to avoid. On 26, TrueNAS is left stock: B2/S3 keeps working, nested
      datasets are simply not covered.

    Fixed

    • Installing the patch permanently blocked updating it. install.sh does
      chmod +x update.sh, and git recorded update.sh as 100644 — so the chmod was a
      tracked modification, and update.sh refuses to run over a dirty tree. Install
      once and you could never update again; the error even told you to run
      git checkout -- ., which just undoes the exec bit so the next install can re-dirty
      it. A real box sat on an old version for exactly this reason.

      Fixed on both sides: the scripts install.sh chmods are now executable in git (so
      the chmod is a no-op), and update.sh's dirty check now looks at content, not
      file mode — git diff --numstat reports 0 0 for a mode-only change. A test
      asserts every script in install.sh's chmod loop is already 100755 in git.

    • Nested snapshots were broken on TrueNAS 24.10 and 25.04, and had been all
      along.
      SYNC_BLOCK's wrapper spelled out the stock signature and forwarded five
      arguments — but those releases declare restic_backup(middleware, job, cloud_backup, dry_run); rate_limit only arrived in 25.10. Every nested backup on
      24.10/25.04 raised TypeError: restic_backup() takes 4 positional arguments but 5 were given. The wrapper now takes *args, **kwargs and forwards whatever it is
      handed, so a trailing parameter appearing or disappearing is a non-event.

      Found by the new compatibility check, not by a user — which is the whole argument
      for having it. The check it replaced only asked whether the parameter names still
      appeared somewhere in the signature, so it happily passed a call that could never
      work.

    • The nested module is now one synchronous implementation behind two thin
      wrappers.
      TrueNAS 26 rewrites cloud_backup from async to synchronous and
      separately deletes get_dataset_recursive(), which an injected block called out
      of the host module's namespace. Either alone is a broken backup found at restore
      time: an async def wrapper hands sync.py a coroutine where it unpacks a tuple,
      and the vanished helper is a straight NameError.

      The module now talks to middlewared through call_sync, and apply.sh reads which
      flavour the installed middleware declares and injects the matching wrapper —
      TrueNAS ≤ 25.10 reaches it via await middleware.run_in_thread(...); a synchronous
      TrueNAS, already in a worker thread, calls it directly. The logic that owns the
      snapshots, the bind mounts and the failure modes exists once; an async twin
      would mean every future fix had to land twice, and the one that got missed would be
      the one that eats a backup. A middleware whose three wrapped functions disagree
      about async-ness is refused outright rather than guessed at, and
      get_dataset_recursive is carried as our own copy — removing the dependency on both
      versions instead of asserting it.

    • The patch no longer reaches into CloudSync tasks it has no business touching.
      create_snapshot is module-global in plugins/cloud/snapshot.py and is imported by
      cloud_sync.py as well as cloud_backup/sync.py — so the wrapper sat in the
      path of every rclone/Storj CloudSync task with snapshot=true, and issued a
      zfs.dataset.query before deciding it had nothing to do. That added a brand-new
      failure mode to jobs that worked fine before this patch was installed, and worse: a
      CloudSync task that ever did get staged would never be torn down, because the
      teardown is wired into cloud_backup's restic_backup and CRUD_BLOCK
      deliberately leaves CloudSync's nesting guard intact — the bind mounts would pin the
      ZFS snapshot forever. The staging path now bails out immediately unless the snapshot
      is named cloud_backup-*, before any middleware call.

    • Teardown warnings are no longer silently swallowed on TrueNAS ≤ 25.10. The async
      wrapper's finally dropped the logger= kwarg that the sync one passes, so a
      cleanup that failed to unmount a bind mount or to delete a snapshot tree logged
      nothing at all — on the only platform anyone actually runs. run_in_thread
      forwards **kwargs via functools.partial; it was a regression, not a limitation.

    • do_delete is recognised as delete. TrueNAS 24.10 and 25.04 declare
      do_delete (the CRUDService convention); 25.10 renamed it to delete. Both
      answer to zfs.snapshot.delete. Accepting only the literal name reported both older
      releases as BROKEN — a false verdict that would have switched nested snapshots off
      on boxes where they work perfectly.

    • An incompatible TrueNAS no longer sets the permanent kill switch. apply.sh
      reused a "nothing left to do" exit that touches disabled, which suppresses
      patching on every future boot and is cleared only by install.sh — never by
      update.sh. On TrueNAS 26 (providers-compatible, nested opt-out by default) that
      branch would have fired, and the very release that fixed 26 could not have
      re-enabled itself: the user would run bash update.sh, exactly as the update alert
      tells them to, and the patch would stay dead with their B2 backups off.
      Incompatibility now means "apply nothing this boot, try again next boot".
      Retirement and incompatibility are opposite situations and no longer share an exit.

    • The compatibility check itself could be fooled, in ways that each had teeth: a
      reordered, keyword-only, or newly-required parameter now reads as broken (the patch
      calls these positionally); a re-exported or conditionally-defined symbol reads
      as unknown rather than broken, so an innocent upstream refactor cannot make a
      working module decline to apply; an unreadable source (rate limit, DNS, timeout)
      is unknown rather than "iXsystems deleted this file", so a network blip cannot
      file a bug report, fail CI, and repaint the published support matrix; and native
      no longer masks BROKEN, which used to render a TrueNAS that both reworded the
      nesting guard and reshaped the functions as good news.

    • compat.py --tree no longer reads the patch's own code as native support.
      B2_BLOCK writes B2RcloneRemote.restic = True into b2.py — exactly the string
      the providers native-probe looks for — so the one command the docs recommend for
      checking a live box said "retire the providers module" on every patched machine.
      It now reads only the part of the file iXsystems wrote.

    • release.sh --promote could never succeed. It refused to run if the stable tag
      existed, and the gate refused if it did not — mutually exclusive, so the only way to
      cut a stable release was to hand-tag and bypass every gate this work exists to
      enforce. The gate now resolves the tag's commit if it exists and HEAD otherwise.
      The tests hid it by always tagging first.

    Changed

    • The minimum supported TrueNAS is stated, and enforced: 24.10. TrueCloud Backup
      does not exist before it — plugins/cloud_backup/ is simply absent — so the patch
      had nothing to attach to and would have done nothing at all, silently, while the
      user believed their backups were configured. install.sh now reads
      system.version and refuses, naming the reason. A version it cannot parse is a
      warning, not a refusal: declining to install over a string we failed to read would
      be a worse failure than the one being prevented.

    • A stable release may not leave work stranded under ## Unreleased. Either it
      is finished and belongs in the release, or the release is premature. Candidates
      are exempt: an rc may legitimately have work queued behind it.

    • release.sh refuses to run on an installed box. The whole repo is cloned onto
      every box, so this file is there too; update.sh pins the checkout to a tag in
      detached HEAD, and release.sh now recognises that and says so, rather than
      emitting a confusing branch error.

    • Gitea (git.onetick.ninja/flan/truenas-truecloud-patch) is now canonical, with
      GitHub as a mirror. Both forges run the same workflows and publish the same
      releases. The update alert now derives the changelog URL from the origin
      remote
      instead of hard-coding GitHub — which matters more than it sounds: when
      the changelog cannot be read, the alert deliberately fires anyway rather than risk
      hiding a security fix, so a stale URL would not have disabled the alert, it would
      have made it nag on every release, including documentation-only ones.

    Security

    • Workflow expressions are no longer interpolated into shell.
      echo "${{ steps.report.outputs.body }}" pasted the compatibility report into the
      script text, and the report is full of backticks — bash ran create-snapshot,
      def and async as commands. Since that report is built from iXsystems' source,
      anything landing in their tree would have executed on the runner. inputs.tag on
      workflow_dispatch had the same shape, and that one is attacker-chosen. Data now
      moves through files and scalars through env:; a test enforces it across every
      workflow.

    Internal

    • Static-analysis annotations in patch/alert_source.py (# noqa placement). No
      runtime change.
    Downloads