Compare commits

..
20 Commits
Author SHA1 Message Date
truecloud-patch bot 1adf8515a6 docs: refresh the TrueNAS compatibility matrix 2026-08-19 06:17:20 +00:00
flan 520b2d3735 Merge pull request 'docs: refresh the TrueNAS compatibility matrix' (#12) from bot/compat-matrix into main
CI / shell (shellcheck + syntax) (push) Successful in 9s
CI / python 3.12 (push) Successful in 2m3s
CI / python 3.13 (push) Successful in 2m4s
CI / python 3.11 (push) Successful in 2m7s
Reviewed-on: #12
2026-08-09 17:41:51 -04:00
truecloud-patch bot 4371797f94 docs: refresh the TrueNAS compatibility matrix 2026-08-09 06:18:05 +00:00
flan 45c89fd001 Add CI status badge to README
CI / shell (shellcheck + syntax) (push) Successful in 8s
CI / python 3.11 (push) Successful in 18s
CI / python 3.12 (push) Successful in 16s
CI / python 3.13 (push) Successful in 18s
2026-08-04 02:45:42 +00:00
flan 670dbd25f6 Remove README badge wall; move development disclosure to NOTICE
CI / python 3.13 (push) Successful in 17s
CI / shell (shellcheck + syntax) (push) Successful in 7s
CI / python 3.11 (push) Successful in 30s
CI / python 3.12 (push) Successful in 15s
2026-08-03 23:43:00 +00:00
flan 827c4358fd Skip the chmod-based unreadable-sidecar tests when running as root
CI / shell (shellcheck + syntax) (push) Successful in 8s
CI / python 3.11 (push) Successful in 20s
CI / python 3.12 (push) Successful in 13s
CI / python 3.13 (push) Successful in 17s
The Gitea runner image executes jobs as root, and chmod(0) cannot make a file
unreadable for root (CAP_DAC_OVERRIDE) — the two tests failed there while
passing on GitHub's non-root runner. Skipping as root keeps the scenario
exercised everywhere it is constructible.
2026-08-03 20:28:37 +00:00
flan ea00bd0685 Repoint forge references from git.onetick.ninja to git.arch.fyi
CI / shell (shellcheck + syntax) (push) Successful in 10s
CI / python 3.11 (push) Failing after 19s
CI / python 3.12 (push) Failing after 25s
CI / python 3.13 (push) Failing after 17s
2026-08-03 20:03:16 +00:00
flan 5a3d4288a2 Fix Gitea-runner python matrix: uv-managed interpreters, pin ruff
CI / shell (shellcheck + syntax) (push) Successful in 7s
CI / python 3.13 (push) Failing after 16s
CI / python 3.11 (push) Failing after 16s
CI / python 3.12 (push) Failing after 19s
Swap README badges to the public GitHub mirror (workflows and releases).
2026-08-03 19:47:29 +00:00
flan b364a17735 Add project badges
CI / python 3.11 (push) Failing after 14s
CI / shell (shellcheck + syntax) (push) Successful in 9s
CI / python 3.13 (push) Failing after 11s
CI / python 3.12 (push) Failing after 16s
2026-08-03 19:28:34 +00:00
flan 3e1de8ffd1 Add sponsor badges to README
CI / python 3.12 (push) Failing after 11s
CI / python 3.13 (push) Failing after 16s
CI / shell (shellcheck + syntax) (push) Successful in 7s
CI / python 3.11 (push) Failing after 10s
2026-08-03 19:09:25 +00:00
flan a7cbb3a994 Add donation links (GitHub Sponsors, Ko-fi)
CI / shell (shellcheck + syntax) (push) Successful in 7s
CI / python 3.11 (push) Failing after 15s
CI / python 3.12 (push) Failing after 25s
CI / python 3.13 (push) Failing after 13s
2026-08-03 17:24:39 +00:00
flan d1216eeb0f merge: re-run the check when the publisher changes
CI / python 3.11 (push) Failing after 13s
CI / python 3.12 (push) Failing after 17s
CI / shell (shellcheck + syntax) (push) Successful in 10s
CI / python 3.13 (push) Failing after 17s
TrueNAS compatibility / compat (push) Successful in 10s
2026-07-14 03:06:44 +00:00
flan 6b7034a8cd A change to the publisher did not re-run the check
CI / shell (shellcheck + syntax) (push) Successful in 9s
CI / python 3.11 (push) Failing after 13s
CI / python 3.12 (push) Failing after 13s
CI / python 3.13 (push) Failing after 11s
TrueNAS compatibility / compat (push) Successful in 9s
compat.yml's push paths listed tools/compat.py but not tools/compat_publish.py.
So the commit that taught the bot to refresh a stale report body fired no run, and
the report stayed stale until the next scheduled one -- caught by watching for the
run that never came. A fix nobody runs is a fix nobody has.
2026-07-14 03:06:44 +00:00
flan 6ce1206f01 merge: the report's body is truth; comments are the changelog
CI / shell (shellcheck + syntax) (push) Successful in 10s
CI / python 3.11 (push) Failing after 12s
CI / python 3.12 (push) Failing after 15s
CI / python 3.13 (push) Failing after 15s
2026-07-14 03:05:01 +00:00
flan e426045255 An unchanged fingerprint froze the report's body, not just its comments
CI / python 3.11 (push) Failing after 13s
CI / shell (shellcheck + syntax) (push) Successful in 8s
CI / python 3.12 (push) Failing after 12s
CI / python 3.13 (push) Failing after 13s
Two questions were sharing one answer.

  "Have the findings changed?" gates COMMENTS. They notify, and a daily "still
  broken, same as yesterday" is what teaches everyone to ignore the one that
  finally matters.

  "Is the body still true?" gates the BODY. Editing an issue body notifies nobody
  on either forge, so keeping it honest is free.

Conflated, an unchanged fingerprint froze the body -- and the fingerprint ignores,
by design, everything that moves on its own: healthy rows, the hardware-verified
column, point releases, and how a row is LABELLED. So the master -> 27-dev relabel
would have shipped to the README and never to the issue anybody actually opens.
The report would have gone on saying "master (unreleased) BROKEN" -- the precise
false alarm the relabel exists to kill -- until iX happened to break something
else.

The body is now rewritten whenever it is stale; comments stay strictly a changelog
of real changes. Bodies are compared after normalising line endings, because a
forge that round-trips \r\n would otherwise trigger a silent rewrite every run and
leave the issue looking freshly touched every morning.
2026-07-14 03:05:01 +00:00
flan 89eb3a16be merge: master is 27-dev; check the next maintenance release
CI / python 3.11 (push) Failing after 12s
CI / shell (shellcheck + syntax) (push) Successful in 10s
CI / python 3.13 (push) Failing after 16s
CI / python 3.12 (push) Failing after 17s
TrueNAS compatibility / compat (push) Successful in 10s
2026-07-14 03:03:21 +00:00
flan 862bdd3399 master is 27-dev, not the next release; and check the next maintenance release
CI / shell (shellcheck + syntax) (push) Successful in 10s
CI / python 3.11 (push) Failing after 13s
CI / python 3.12 (push) Failing after 13s
TrueNAS compatibility / compat (push) Successful in 11s
CI / python 3.13 (push) Failing after 13s
Two ways the matrix misled the person it exists for -- somebody deciding whether
to trust this with their backups.

master is not the next release. iX branches each major onto its own release/ line
and master rolls straight on to the one after: every recent commit on master
targets 27.0.0-BETA.1 while 26 is still in beta. So a BROKEN master row, rendered
"master (unreleased)", read as "the version you are about to install is broken"
when the breakage is a major release away on a line nobody can download. It is now
labelled from the newest major in the matrix plus one, so it becomes 28-dev by
itself once 27 branches.

The break, for the record, is NAS-141498 (2026-06-24), "Convert cloud_backup
plugin to the typesafe pattern": it re-signatures restic_backup and
get_restic_config, splitting entry/credentials out of the cloud_backup dict. Not
being chased while the 27 line churns.

The next maintenance release was never checked -- and it is the one that reaches
users. Shipped came from TS-* tags, unreleased from release/* branches carrying
-BETA/-RC. A branched-but-untagged MAINTENANCE release is neither: release/25.10.5
has no tag, and its line has already shipped, so the "a prerelease of a shipped
line is history" filter threw it out. It was invisible, and it is exactly what a
25.10.4 box gets on its next update; a break in it reaches real users before the
daily check ever looks, on the only line anybody runs.

A plain release/X.Y.Z branch now counts when its line HAS shipped and it sorts
NEWER than that line's newest tag. Both exclusions fall out of the same rule:
release/24.10-RC.2 sorts older than TS-24.10.2.4 (history), and the typo branch
release/25.20.2.2 is on a line with no tag at all (not a release line). This
surfaced two refs never checked before -- release/25.10.5 and release/24.10.2.5 --
both of which pass.

is_unreleased() keys off where a ref came from (a branch is by construction not
shipped) rather than hunting -BETA/-RC in the name. Otherwise release/25.10.5
counts as shipped and a break in it fails the build as a live outage, on a version
nobody is running yet.
2026-07-14 03:03:21 +00:00
flan 753c3f8cad merge: the bug-report bot re-filed itself every run on Gitea
CI / shell (shellcheck + syntax) (push) Successful in 8s
CI / python 3.11 (push) Failing after 12s
CI / python 3.13 (push) Failing after 14s
CI / python 3.12 (push) Failing after 15s
2026-07-14 02:37:32 +00:00
flan 52b11eada2 The bug-report bot re-filed itself every run on Gitea; nine copies
CI / python 3.12 (push) Failing after 13s
CI / python 3.13 (push) Failing after 13s
CI / shell (shellcheck + syntax) (push) Successful in 8s
CI / python 3.11 (push) Failing after 7s
`find_issue()` skipped pull requests by testing for the PRESENCE of the
`pull_request` key. GitHub omits that key on a plain issue. Gitea sends it as
`null`. So on Gitea every issue was thrown away as if it were a PR, the lookup
came back empty on every run, and the bot took the "nothing filed yet" branch and
opened a brand-new report instead of editing the one already open.

Nine of them piled up on the canonical forge. Four were filed AFTER the commit
that was supposed to stop exactly this -- the same failure it was written to
prevent, moved from comments to issues. It survived because `find_issue` was the
one function here with no test; the mirror deduped correctly, and the mirror is
what anybody would have looked at.

Test the value, not the key: it is the only form that is true on both forges.
Pinned by tests that carry each forge's payload shape by hand.

Also send both paging parameters. GitHub reads `per_page`, Gitea reads `limit`,
and each ignores the other's; Gitea's default page is 30, so the lookup would
have begun missing the report once the pile it was creating outgrew one page.
2026-07-14 02:37:32 +00:00
flan ca906f5ee4 docs: TrueNAS 26 is supported; record what has actually been run
CI / python 3.12 (push) Failing after 16s
CI / shell (shellcheck + syntax) (push) Successful in 10s
CI / python 3.11 (push) Failing after 12s
CI / python 3.13 (push) Failing after 16s
TrueNAS compatibility / compat (push) Successful in 9s
The README and how-it-works still said "TrueNAS 26: nested snapshots are not supported
yet" and "the third is not fixed, and is why 26 reports BROKEN". Both shipped in v0.7.0
and are now false — exactly the kind of stale claim that misleads somebody deciding
whether to trust this with their backups.

Adds docs/verification.md: what has ACTUALLY been run, as opposed to what the support
matrix proves. The matrix is static analysis — it shows the patch's assumptions still
hold, which is a strictly weaker claim than "a backup ran and a restore came back". The
new file records the three live tasks exercised on 25.10.4 (nested, nested+zvols,
non-nested), the md5 of the file that came back out of B2, the real orphan the collector
reclaimed from the pool, and what is NOT covered (24.10/25.04 unrun; master broken; no
reboot on v0.7.0). The README now points at it, next to the matrix it qualifies.

The how-it-works TrueNAS 26 section now explains the part that mattered: the public
pool.* queries are not like-for-like replacements for the deleted private zfs.* ones —
they apply a visibility policy hiding 84 of 270 datasets on a real pool, including live
app data — and the rule the module now follows (read the truth from ZFS, make changes
through middleware). Plus the divergence nothing warned about: 26 decides `recursive` by
a different rule than this patch decides `nested`, which orphaned one snapshot per zvol
on every run until ownership of the sweep was made unconditional.
2026-07-14 02:22:36 +00:00
16 changed files with 678 additions and 62 deletions
+2
View File
@@ -0,0 +1,2 @@
github: sudolulo
ko_fi: sudolulo
+10 -9
View File
@@ -43,20 +43,21 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- name: install dev deps
run: python -m pip install --upgrade pip pytest ruff
# uv-managed interpreters instead of actions/setup-python: the prebuilt-CPython
# download path setup-python relies on does not work on the self-hosted Gitea
# runner (all three matrix jobs failed at setup there while passing on GitHub);
# uv works identically on both.
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
- name: ruff
run: ruff check patch tests tools
# Pinned: an unpinned ruff means any upstream release can turn main red
# with no code change.
run: uvx ruff@0.16.1 check patch tests tools
- name: pytest
run: pytest tests -v
run: uvx --python ${{ matrix.python }} pytest tests -v
- name: verify injected middleware blocks compile
# Belt-and-braces: the *_BLOCK strings are appended into live middlewared
# modules. A syntax error there would break the box at boot.
run: pytest tests/test_apply_blocks.py -v
run: uvx --python ${{ matrix.python }} pytest tests/test_apply_blocks.py -v
+5
View File
@@ -24,6 +24,11 @@ on:
paths:
# The manifest itself changed -- re-check immediately rather than waiting a day.
- "tools/compat.py"
# ...and so did the thing that PUBLISHES the finding. This was missing, and it
# showed: the commit teaching the bot to refresh a stale report body touched only
# compat_publish.py, so no run fired, and the report stayed stale until the next
# scheduled one. A fix nobody runs is a fix nobody has.
- "tools/compat_publish.py"
- ".github/workflows/compat.yml"
permissions:
+1 -1
View File
@@ -124,7 +124,7 @@ jobs:
echo "--- release body ---"
cat /tmp/notes.md
# This repo is canonically hosted on Gitea (git.onetick.ninja) and mirrored to
# This repo is canonically hosted on Gitea (git.arch.fyi) and mirrored to
# GitHub, and BOTH run this workflow -- Gitea reads .github/workflows too. So
# the publish step has to work on whichever forge it lands on. Everything
# above is forge-agnostic; only the "create a release" API differs.
+91
View File
@@ -6,6 +6,97 @@ is deliberate: see [Releasing](docs/releasing.md). Twelve releases were cut on
live, every one of those interrupts every user. An alert people learn to ignore is
worse than no alert, because one day it carries a security fix.
## Unreleased
### Changed
- **CI's python matrix is green on the self-hosted Gitea runner again.** The real
failure was that the Gitea runner image executes jobs as root, and the two
unreadable-sidecar tests build their scenario with `chmod(0)` — which cannot make
a file unreadable for root (`CAP_DAC_OVERRIDE`). Those two tests now skip as root
with that reason; GitHub's non-root runner still exercises them. The matrix also
moved to uv-managed interpreters (one toolchain across both runners) and ruff is
pinned to 0.16.1 so an upstream ruff release can't turn `main` red without a code
change.
- **README badges point at the public GitHub mirror** (workflow status and
releases) instead of the private forge. The release badge had also been reading
the stale Gitea v0.6.1 release instead of the current v0.7.0 on GitHub.
- **`master` is now labelled `27-dev`, because it is not the next release.** iX
branches each major onto its own `release/` line and master rolls straight on to the
one after — on 2026-07-14 every recent commit on master targeted `27.0.0-BETA.1`
while 26 was still in beta. So a **BROKEN** master row, rendered as
"master _(unreleased)_", read as *"the version you are about to install is broken"*
when the breakage was a major release away on a line nobody can download. In a table
whose entire job is helping somebody decide whether to trust this with their backups,
that is a false alarm in the worst possible place. The label is derived from the
newest major in the matrix plus one, so it rolls over to `28-dev` by itself once 27
branches.
For the record, the breakage is `NAS-141498` (2026-06-24), "Convert cloud_backup
plugin to the typesafe pattern": it re-signatures `restic_backup` and
`get_restic_config`, splitting `entry`/`credentials` out of the `cloud_backup` dict.
It is deliberately not being chased while the 27 line is still churning.
### Fixed
- **The next maintenance release was never checked, and it is the one that reaches
users.** Shipped versions were discovered from `TS-*` tags and unreleased ones from
`release/*` branches carrying `-BETA`/`-RC`. A branched-but-untagged *maintenance*
release is neither: `release/25.10.5` has no tag, and its line has already shipped,
so the "a prerelease of a shipped line is history" filter discarded it. It was
invisible — and it is precisely what a 25.10.4 box gets on its next update. A break
there would have reached real users before the daily check ever looked at it, on the
only line anybody is actually running.
A plain `release/X.Y.Z` branch is now checked when its line **has** shipped and it
sorts **newer** than that line's newest tag. Both things that must stay out fall out
of the same rule: `release/24.10-RC.2` sorts older than `TS-24.10.2.4` (history, not
a warning), and iX's typo branch `release/25.20.2.2` is on a line that has no tag at
all, so it is not a release line. This immediately surfaced two refs that had never
been checked — `release/25.10.5` and `release/24.10.2.5` — both of which pass.
`is_unreleased()` now keys off where a ref came from (branch = not yet shipped)
rather than looking for `-BETA`/`-RC` in its name. Otherwise `release/25.10.5` would
count as shipped and a break in it would fail the build as a live outage — on a
version nobody is running yet.
- **An unchanged fingerprint froze the bug report's body, not just its comments.** Two
questions were sharing one answer. *Have the findings changed?* gates **comments** —
they notify, and a daily "still broken, same as yesterday" is what teaches everyone
to ignore the one that finally matters. *Is the body still true?* gates the **body** —
and editing an issue body notifies nobody on either forge, so keeping it honest is
free. Conflated, the report could never be corrected while the findings held steady,
and the fingerprint deliberately ignores everything that moves on its own — healthy
rows, the hardware-verified column, point releases, and how a row is labelled. The
`master` → `27-dev` relabel above would have reached the README and never the issue
anybody actually opens. The body is now rewritten whenever it is out of date (after
normalising line endings, so a forge round-tripping `\r\n` does not cause a rewrite
every run) and comments remain strictly a changelog of real changes.
- **A change to the publisher did not re-run the check.** `compat.yml`'s `push:` paths
listed `tools/compat.py` but not `tools/compat_publish.py` — so the very commit that
taught the bot to refresh a stale report body triggered no run, and the report stayed
stale until the next scheduled one. A fix nobody runs is a fix nobody has.
- **The compatibility bot filed a new duplicate bug report on every Gitea run.**
`find_issue()` skipped pull requests by testing for the *presence* of the
`pull_request` key. GitHub omits that key on a plain issue; Gitea sends it as
`null`. So on Gitea every issue was discarded as a PR, the lookup always came back
empty, and the bot took the "nothing filed yet" branch and opened a fresh report
each run — **nine copies on the canonical forge**, four of them filed *after* the
commit that was meant to stop precisely this. The mirror was fine, which is why it
went unnoticed: GitHub's payload shape is the one the filter was written against.
It is the same failure the anti-spam fix was written to prevent, moved from
comments to issues, and it survived because `find_issue` was the only function in
`compat_publish.py` with no test. It now has one, per forge, and the daily cron —
which had not yet run once — no longer accumulates a report a day.
The issue list is also requested with **both** paging parameters (`per_page` for
GitHub, `limit` for Gitea). Each forge ignores the other's, and Gitea's default page
is 30, so the lookup would have started missing the report again once the pile it
was creating grew past one page.
## v0.7.0 — 2026-07-14
### Added
+4
View File
@@ -0,0 +1,4 @@
truenas-truecloud-patch
Parts of this project were written with AI assistance (Claude); all of it is
reviewed and tested before release.
+42 -15
View File
@@ -1,5 +1,7 @@
# truenas-truecloud-patch
[![CI](https://git.arch.fyi/flan/truenas-truecloud-patch/actions/workflows/ci.yml/badge.svg)](https://git.arch.fyi/flan/truenas-truecloud-patch/actions)
Extends TrueNAS SCALE's **TrueCloud Backup** to:
- back up to **Backblaze B2 and any S3-compatible provider**, not just Storj;
@@ -59,9 +61,11 @@ If something is wrong, the reason is in `apply.log` — start at
| --- | --- | --- | --- |
| 24.10.2.4 | ok | ok | — |
| 25.04.2.6 | ok | ok | — |
| 25.10.4 | ok | ok | nested + providers; 252-snapshot recursive backup of /mnt/Tap, 18m |
| 25.10.6 | ok | ok | — |
| 24.10.2.5 _(unreleased)_ | ok | ok | — |
| 25.10.7 _(unreleased)_ | ok | ok | — |
| 26.0.0-BETA.3 _(unreleased)_ | ok | ok | — |
| master _(unreleased)_ | **BROKEN** | **BROKEN** | — |
| master _(27-dev)_ | **BROKEN** | **BROKEN** | — |
| verdict | meaning |
| --- | --- |
@@ -72,23 +76,44 @@ If something is wrong, the reason is in `apply.log` — start at
"ok" means *the patch's assumptions hold*, checked automatically against iX's
source. It does not mean a human ran a backup on it — that is the
**Hardware-verified** column, which is filled in by hand and only by doing it.
**`master` is not the next release.** iX branches each major off to its own
`release/` line and master rolls straight on to the one after — so master is
`27-dev` while 26 is still in beta. A **BROKEN** master means iX has changed
something that will reach users *a major release from now*, not in the version you
are about to install. Read the numbered rows for that.
A row like `25.10.5 _(unreleased)_` is the next maintenance release: branched by iX,
not tagged yet, and the very next thing a 25.10.4 box gets. It is checked precisely
because it is the one unshipped ref that reaches real users without warning.
<!-- END COMPAT MATRIX -->
The table is **regenerated daily by CI** against iXsystems' actual middleware source
— it is not a claim somebody typed once and forgot.
**On TrueNAS 26:** the patch was run on a real TrueNAS **26.0.0-BETA.1** install — a
274-snapshot recursive backup of a 292-dataset pool, followed by a byte-identical
restore of a four-level-deep child dataset. The *Hardware-verified* column tracks the
newest beta iX has tagged (currently BETA.3), so it does not carry that mark: a build
nobody has actually run a backup on does not get credit for one.
It is also **static analysis**: it proves the patch's assumptions still hold, which is
a weaker claim than "a backup ran and a restore came back". For what has actually been
run — which tasks, on which hardware, and the md5 of the file that came back — see
[docs/verification.md](docs/verification.md).
**TrueNAS 26: nested snapshots are not supported yet, and upgrading will not break
you.** 26 rewrites `cloud_backup` and deletes the ZFS methods this module calls. On
26 `apply.sh` finds that the assumptions no longer hold and **does not apply the
module**: TrueNAS is left stock, B2/S3 keeps working, nested datasets are simply not
covered, and the reason is named in `apply.log`. A broken backup is worse than a
missing feature. Details: [How it works](docs/how-it-works.md#truenas-26).
**TrueNAS 26 is supported** as of v0.7.0, and was verified on a real
**26.0.0-BETA.1** install: a 274-snapshot recursive backup of a 292-dataset pool, and a
byte-identical restore of a four-level-deep child dataset. The *Hardware-verified*
column tracks the newest beta iX has tagged (currently BETA.3), so it does not carry
that mark — a build nobody has actually run a backup on does not get credit for one.
26 rewrites `cloud_backup` from async to synchronous and deletes the private ZFS
methods this module used to call, so getting there took real work: the patch now
injects the wrapper flavour that matches the installed middleware, reads dataset and
snapshot lists **from ZFS rather than middleware** (whose queries hide TrueNAS's own
datasets — 84 of 270 on a real pool, including live app data), and owns the snapshot
sweep even when it stages nothing (26 decides `recursive` by a rule this patch does not
share, and would otherwise orphan one snapshot per zvol on every run).
**And if a future TrueNAS breaks it, you get a missing feature, not a broken backup.**
`apply.sh` re-checks the patch's assumptions at every boot and **refuses to apply a
module whose assumptions no longer hold** — TrueNAS is left stock, and the reason is
named in `apply.log`. Details: [How it works](docs/how-it-works.md#truenas-26).
---
@@ -213,5 +238,7 @@ and restores the original UI bundle from backup.
- Filing a TrueNAS bug? **Remove the patch first** and reproduce on a stock system.
- Provided as-is, no warranty. See LICENSE.
Parts of this project were written with AI assistance (Claude); all of it is reviewed
and tested before release. Bugs are mine.
## Support
If truenas-truecloud-patch is useful to you, consider supporting development via
[GitHub Sponsors](https://github.com/sponsors/sudolulo) or [Ko-fi](https://ko-fi.com/sudolulo).
+62 -14
View File
@@ -187,23 +187,71 @@ backup-breaking**, and none of them is visible from the `cloud_backup` files:
| `get_dataset_recursive()` **deleted** from `plugins/cloud/snapshot.py` | `NameError` — the injected block called it out of the host module's namespace |
| `plugins/zfs_/dataset.py` and `zfs_/snapshot.py` **deleted** | `zfs.dataset.query`, `zfs.snapshot.query` and `zfs.snapshot.delete` all vanish. 26 uses `filesystem.statfs` and `zfs.resource.*` |
The first two are fixed: the patch reads which flavour of `cloud_backup` your box
declares and injects the wrapper that matches (one implementation of the real logic,
two thin wrappers), and it carries its own copy of the deleted helper.
All three are fixed as of **v0.7.0**, and 26 is supported.
The third is **not** fixed, and is why 26 reports BROKEN. Porting it means rewriting
the module's ZFS calls onto 26's new API, and no single API spans 24.10 through 26 —
so it needs a real 26 box to verify against, not a plausible-looking diff. Shipping a
port nobody has run is exactly the failure this project exists to avoid.
The first two were straightforward: the patch reads which flavour of `cloud_backup`
your box declares and injects the wrapper that matches (one implementation of the real
logic, two thin wrappers), and it carries its own copy of the deleted helper.
It is also the row that would have hurt most. `zfs.snapshot.delete` is what sweeps the
recursive snapshot; without it, **every run would orphan one snapshot per descendant
dataset — 250 on a real pool — forever.** The compatibility check caught it only
because it now asserts the middleware *methods the patch calls*, not just the symbols
it wraps.
The third was not, and it is the one that would have hurt most — `zfs.snapshot.delete`
is what sweeps the recursive snapshot, and without it **every run would orphan one
snapshot per descendant dataset (250 on a real pool), forever, while reporting
success.**
`master` (development after 26) reports BROKEN too: iXsystems are still reshaping
The obvious port is to the public `pool.dataset.query` / `pool.snapshot.query`. Those
methods exist, are documented, and are covered by iX's deprecation policy — and they
are **not like-for-like replacements**. They apply a *visibility policy*: they hide the
datasets TrueNAS considers its own (`ix-apps/*`, `.system/*`, `.ix-virt/*`). On a real
pool that is **84 of 270 datasets, including live application data.** Staging from that
view would have omitted every one of them from the backup — and the planner would never
have seen them, so they would not have appeared in its "skipped" list either. A green
backup, quietly missing data. The snapshot query lies the same way, so the sweep would
have orphaned one snapshot per hidden dataset.
No source analysis could have caught that. The methods are all present and correctly
shaped. Only running it could, which is why it took a real 26 box.
So the module now follows one rule:
> **Read the truth from ZFS. Make changes through middleware.**
Enumeration is `zfs list` — no policy can filter it, and it behaves identically on every
release, which also means one code path instead of a version conditional. Mutation stays
a middleware call, so TrueNAS's own bookkeeping stays consistent; an exact-name delete
works fine even on a dataset the query hides. It is only enumeration that lies.
The snapshot *delete* still needs a namespace, and no single one spans every release —
24.10 and 25.04 have `zfs.snapshot`, 26 has only `pool.snapshot`, 25.10 has both. So it
is resolved at runtime, by asking whether the namespace can actually delete. `tools/compat.py`
asks the identical question against iX's source, and a test binds the two lists together,
so what CI verifies and what runs cannot drift apart.
### The one 26 changed that nothing warned about
Stock decides whether to take a **recursive** snapshot by its own rule, and on 26 that
rule stopped being ours:
| | decides `recursive` by |
| --- | --- |
| stock ≤ 25.10 | `get_dataset_recursive()` — the same function this patch vendors |
| **stock 26** | `filesystem.statfs`: `recursive = (path == the dataset's mountpoint)` |
| this patch | `get_dataset_recursive()` — is a mounted *filesystem* child under the path? |
Up to 25.10 those were the *same question*, so a snapshot the patch declined to stage
provably had no children and stock's non-recursive delete was correct. On 26 they
disagree: a dataset whose only descendants are **zvols** or **legacy-mountpoint**
datasets gets a recursive snapshot, while the patch sees nothing to stage. Stock then
destroys the parent only — and with no staging tree there was no sidecar, and the
garbage collector only ever ran from the staging path. Nothing on the box would ever
have found the children.
It was reproduced on a 26 VM (one orphan per zvol, every run, backup green) and closed:
**ownership of the sweep is no longer conditional on staging.**
`master` (development after 26) **does** report BROKEN: iXsystems are still reshaping
these functions there, renaming `middleware` → `context` and `cloud_backup` → `entry`
and adding a required `credentials` parameter. That is a moving target and is
deliberately not chased; the check keeps reporting it until it settles into a beta,
which is when it becomes worth fixing.
which is when it becomes worth fixing. Until then, a box running master would simply
not get the modules — `apply.sh` refuses to apply a module whose assumptions no longer
hold, and says why in `apply.log`.
+1 -1
View File
@@ -25,7 +25,7 @@ are worth calling out, because nothing else would catch what they catch:
newest CHANGELOG entry. `VERSION=` had silently drifted to three different
values across the scripts before anything checked.
The project is hosted on **Gitea** (`git.onetick.ninja/flan/truenas-truecloud-patch`)
The project is hosted on **Gitea** (`git.arch.fyi/flan/truenas-truecloud-patch`)
and mirrored to GitHub. Both run the same workflows — Gitea reads
`.github/workflows/` too — so a change is checked twice, on two independent runners.
+77
View File
@@ -0,0 +1,77 @@
# What has actually been run
The support matrix in the README is **static analysis**: it proves the patch's
assumptions about middlewared still hold. That is a strictly weaker claim than "a
backup ran and a restore came back". This file is the stronger claim, and it is
maintained by hand, because the only way to fill it in is to do it.
If you are deciding whether to trust this with your backups, read this file, not the
matrix.
---
## v0.7.0 — TrueNAS 25.10.4 (production hardware)
Six live TrueCloud tasks, all `snapshot = true`, backing up to Backblaze B2. Three were
exercised end to end, chosen to cover the three shapes the code handles differently:
| Task | Path | Shape | Result |
| --- | --- | --- | --- |
| 5 | `/mnt/Tap` | 191 nested datasets staged, 282-snapshot recursive tree | SUCCESS |
| 7 | `/mnt/Tank/backups` | 215 filesystems **+ 2 zvols** | SUCCESS |
| 9 | `/mnt/Tank/flan` | **no** nested filesystem children | SUCCESS |
After every run: **0 orphaned snapshots, 0 leaked bind mounts, 0 stale sidecars.**
**The restore.** `apps/vaultwarden/data/config.json` — a file inside a *child* dataset,
which is exactly what stock TrueNAS cannot capture — was restored from B2 and compared
against the live file:
live f809df6ba231986b1ba824044228a03a 1808 bytes
restored f809df6ba231986b1ba824044228a03a 1808 bytes
=> byte-identical
**The collector earned its keep on real data.** The pool was already carrying an orphan:
`Tap/apps/prometheus@cloud_backup-5-20260713202355`, left behind by an earlier run when
ZFS's automount held the snapshot busy past all four retries. The first v0.7.0 run found
it by name, reclaimed it, and the pool's snapshot count went 2148 → 2147. That is the
garbage collector doing the job it was written for, against a leak that was already
there and that nothing else would ever have found.
**Boot path.** `apply.sh` is registered as a PREINIT `initshutdownscript`; it was
re-run against the live middleware and left exactly one `TRUECLOUD_PATCH` marker in
each patched module (a second copy stacked into a live middlewared module would break
the box at boot). It correctly detected the box as **async** (`cloud_backup is async
(TrueNAS <= 25.10)`) and injected the matching wrappers.
**Upgrade path.** `update.sh` was used to move the box from the release candidate to
the stable tag, in detached HEAD at `v0.7.0`, which is how a user's box actually
upgrades.
## v0.7.0 — TrueNAS 26.0.0-BETA.1 (VM)
A throwaway VM whose pool reproduces the production pool's *shape* — 292 datasets, 26
`legacy` mountpoints, nesting five deep — because every bug found on the real box came
from the shape of the pool, not the bytes in it. MinIO was not used; `rclone serve s3`
(already on the box) provided the S3 target, so no real B2 credential ever entered the
VM.
* 274-snapshot recursive backup of the 292-dataset pool. 0 orphans, 0 leaked mounts.
* Restored `ix-apps/app_mounts/vaultwarden/pgData` — **four levels deep, and a dataset
that middleware's own `pool.dataset.query` hides from itself** — byte-identical.
* The zvol-orphan case was **reproduced with the fix disabled** (one orphan per zvol,
every run, backup green), then **closed with it enabled**. See the CHANGELOG entry
for why TrueNAS 26 decides `recursive` by a different rule than this patch decides
`nested`.
## What is NOT covered
* **24.10 and 25.04** are `ok` in the matrix — the assumptions hold, checked against
iX's source — but nobody has run a backup on them. The matrix says so.
* **master** is BROKEN, and correctly reports so: iX renamed the leading parameters of
`get_restic_config` and `restic_backup`. It is not a shipped release; the daily
compatibility bot files it, and `apply.sh` would refuse to apply the modules on a box
running it.
* A **reboot** of the production box has not been done on v0.7.0. `apply.sh` was
re-executed by hand against the live middleware, which exercises the same code path,
but the PREINIT ordering itself has only been proven on earlier versions.
+2 -2
View File
@@ -47,8 +47,8 @@ _VERSION_RE = re.compile(r'^VERSION="([^"]+)"', re.M)
#: owner/repo out of any of:
#: git@github.com:sudolulo/repo.git
#: https://github.com/sudolulo/repo.git
#: ssh://git@git.onetick.ninja:55214/flan/repo.git
#: https://git.onetick.ninja/flan/repo.git
#: ssh://git@git.arch.fyi:55214/flan/repo.git
#: https://git.arch.fyi/flan/repo.git
#: The SSH port is deliberately not captured: it is not the web port.
_REMOTE_RE = re.compile(
r"^(?:\w+://)?(?:[^@/]+@)?([^:/]+)(?::\d+)?[:/]([^/]+)/([^/]+?)(?:\.git)?/?$"
+232
View File
@@ -15,6 +15,7 @@ dangerous thing this file can say -- it means "TrueNAS does this now, retire the
module" -- and it rests on nothing more than a substring match.
"""
import json
import os
import sys
@@ -23,6 +24,7 @@ import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "tools"))
import compat # noqa: E402
import compat_publish # noqa: E402
from compat import ( # noqa: E402
NESTED,
PROVIDERS,
@@ -574,3 +576,233 @@ class TestATransientNetworkBlipDoesNotWakeAnybody:
"def get_restic_config(entry, credentials):\n pass\n"
)
assert compat.fingerprint(self._rows(worse)) != compat.fingerprint(self._rows(broken))
class TestTheBotFindsItsOwnIssueOnBOTHForges:
"""`find_issue` decides "have I already filed this?" -- and it ran on two forges.
It used to skip pull requests with `"pull_request" not in i`. GitHub omits that key
on a plain issue; **Gitea sends it as `null`**. So on Gitea every issue looked like
a PR, the match list was always empty, and the bot took the "nothing filed yet"
branch on EVERY run: nine duplicate copies of the same report on the canonical
forge, four of them filed after the commit that was supposed to stop exactly this.
It is the same failure the spam fix was written to prevent, moved from comments to
issues -- and it survived because `find_issue` was the one function here with no
test. So the payload shapes are pinned, per forge, by hand.
"""
TITLE = compat_publish.TITLE
def _find(self, monkeypatch, payload):
monkeypatch.setattr(compat_publish, "_call", lambda *a, **k: payload)
return compat_publish.find_issue("https://forge/api", "tok", self.TITLE)
def test_gitea_sends_pull_request_as_null_and_the_issue_is_still_found(self, monkeypatch):
found = self._find(monkeypatch, [
{"number": 7, "title": self.TITLE, "state": "open", "pull_request": None},
{"number": 1, "title": self.TITLE, "state": "open", "pull_request": None},
])
assert found is not None, (
"find_issue missed a Gitea issue, so the bot files a NEW duplicate report "
"every run -- which is how nine of them piled up"
)
assert found["number"] == 1, "lowest-numbered wins"
def test_github_omits_the_key_entirely_and_the_issue_is_still_found(self, monkeypatch):
found = self._find(monkeypatch, [
{"number": 2, "title": self.TITLE, "state": "open"},
])
assert found is not None and found["number"] == 2
def test_a_real_PR_with_the_same_title_is_still_skipped_on_both(self, monkeypatch):
# The reason the filter exists at all: both forges list PRs on /issues, and
# commenting on a PR instead of the bug report would be worse than useless.
assert self._find(monkeypatch, [
{"number": 3, "title": self.TITLE, "state": "open", # Gitea PR
"pull_request": {"merged": False}},
{"number": 4, "title": self.TITLE, "state": "open", # GitHub PR
"pull_request": {"url": "https://api.github.com/..."}},
]) is None
def test_an_unrelated_issue_is_not_mistaken_for_the_report(self, monkeypatch):
assert self._find(monkeypatch, [
{"number": 1, "title": "TypeError when create B2 backup on Electric Eel",
"state": "closed", "pull_request": None},
]) is None
class TestTheNextMaintenanceReleaseIsChecked:
"""`release/25.10.5` fell through every sieve, and it is the one that reaches users.
Shipped versions come from `TS-*` TAGS; unreleased ones come from `release/*`
BRANCHES that carry `-BETA`/`-RC`. A branched-but-untagged MAINTENANCE release is
neither: no tag, and its line (25.10) has already shipped, so the "prereleases of
a shipped line are history" filter threw it out. It was invisible.
That is backwards. `release/24.10-RC.2` is history -- nobody can install it. But
`release/25.10.5` is the FUTURE of a shipped line: it is what a 25.10.4 box gets
on its next update. A break there ships to real users before the daily check has
ever looked at it.
"""
TAGS = ["TS-24.10.2.4", "TS-25.04.2.6", "TS-25.10.4"]
HEADS = [
"release/25.10.4.1",
"release/25.10.5", # branched, untagged -- the next maintenance release
"release/24.10-RC.2", # history: its line shipped long ago
"release/25.20.2.2", # iX's typo branch: 25.20 is not a TrueNAS version
"release/26.0.0-BETA.3",
"master",
]
def _refs(self, monkeypatch):
monkeypatch.setattr(
compat, "_ls_remote",
lambda remote, what: self.TAGS if what == "--tags" else self.HEADS)
return compat.discover_refs("origin")
def test_the_next_maintenance_release_is_checked(self, monkeypatch):
assert "release/25.10.5" in self._refs(monkeypatch), (
"the next thing a 25.10.4 box updates to is not checked, so a break in it "
"reaches users before the bot ever sees it"
)
def test_a_superseded_maintenance_branch_is_not(self, monkeypatch):
# 25.10.4.1 sorts OLDER than the newest tag TS-25.10.4? No -- it is NEWER, and
# both are on the 25.10 line, so only the newest branch on the line is taken.
refs = self._refs(monkeypatch)
assert "release/25.10.4.1" not in refs, "only the newest branch per line"
def test_the_typo_branch_stays_out(self, monkeypatch):
# 25.20 has no TS tag, so it is not a release line at all. A typo branch in the
# matrix reads as a real supported release we are silently broken on.
assert "release/25.20.2.2" not in self._refs(monkeypatch)
def test_a_prerelease_of_an_already_shipped_line_stays_out(self, monkeypatch):
assert "release/24.10-RC.2" not in self._refs(monkeypatch)
def test_an_untagged_branch_counts_as_UNRELEASED(self, monkeypatch):
# The exit code keys off this. Calling 25.10.5 "shipped" would fail the build
# as a live outage on a version nobody is running yet.
assert compat.is_unreleased("release/25.10.5")
assert compat.is_unreleased("master")
assert not compat.is_unreleased("TS-25.10.4")
class TestMasterIsNotTheNextRelease:
"""A red `master` row used to read as "the version you are about to install".
On 2026-07-14 master was 27-dev -- every recent commit targeted 27.0.0-BETA.1 --
while 26 was still in beta on its own branches. So `master BROKEN` meant "iX will
break us a major release from now", but the matrix said "master _(unreleased)_",
which any reader takes as the next thing out the door. For a table whose whole job
is helping somebody decide whether to trust this with their backups, that is a
false alarm in the worst possible place.
"""
def _rows(self, refs):
return [{"ref": r, "unreleased": compat.is_unreleased(r), "modules": {}}
for r in refs]
def test_master_is_labelled_with_the_major_AFTER_the_newest_known_one(self):
rows = self._rows(["TS-25.10.4", "release/26.0.0-BETA.3", "master"])
assert compat.dev_label(rows) == "27-dev"
def test_it_rolls_over_on_its_own_when_the_next_beta_branches(self):
# Derived, not hardcoded: when release/27.0.0-BETA.1 appears, master is 28-dev.
rows = self._rows(["TS-26.0.0", "release/27.0.0-BETA.1", "master"])
assert compat.dev_label(rows) == "28-dev"
def test_the_rendered_matrix_says_dev_not_unreleased(self):
healthy = check_files(with_())
rows = [
{"ref": r, "unreleased": compat.is_unreleased(r), "modules": healthy}
for r in ("TS-25.10.4", "release/26.0.0-BETA.3", "master")
]
md = compat.render_markdown(rows)
assert "master _(27-dev)_" in md
assert "master _(unreleased)_" not in md
# ...and the ordinary rows are untouched.
assert "| 25.10.4 |" in md
assert "| 26.0.0-BETA.3 _(unreleased)_ |" in md
class TestTheBodyIsTruthAndCommentsAreTheChangelog:
"""An unchanged fingerprint used to freeze the BODY, not just silence the comments.
Two different questions were sharing one answer. "Have the findings changed?" gates
COMMENTS -- they notify, and a daily "still broken, same as yesterday" is what
teaches people to ignore the one that finally matters. But "is the body still
true?" gates the BODY, and editing a body notifies nobody, so keeping it honest
costs nothing.
Conflated, an unchanged fingerprint meant the report could never be corrected --
and the fingerprint deliberately ignores everything that moves on its own, which
includes how a row is LABELLED. Relabelling master `27-dev` would have reached the
README and never the issue anybody opens.
"""
ROWS = [{"ref": "master", "unreleased": True,
"modules": check_files(with_(**{
"plugins/cloud_backup/restic.py":
"class ResticConfig:\n cmd: list\n\n"
"def get_restic_config(entry, credentials):\n pass\n",
}))}]
def _run(self, monkeypatch, tmp_path, existing_body):
calls = []
def fake(url, token, method="GET", data=None):
calls.append((method, url, data))
if url.endswith("/issues?state=all&per_page=100&limit=100"):
return [{"number": 1, "title": compat_publish.TITLE,
"state": "open", "body": existing_body,
"pull_request": None}]
return {"number": 1}
monkeypatch.setattr(compat_publish, "_call", fake)
matrix = tmp_path / "m.json"
matrix.write_text(json.dumps(self.ROWS))
compat_publish.main([
"prog", "--api", "https://forge/api", "--token", "t",
"--matrix", str(matrix)])
return calls
def _writes(self, calls):
patched = [c for c in calls if c[0] == "PATCH"]
commented = [c for c in calls if c[0] == "POST" and c[1].endswith("/comments")]
return patched, commented
def test_identical_body_and_findings_touches_nothing(self, monkeypatch, tmp_path):
body = compat.render_issue(self.ROWS)
patched, commented = self._writes(self._run(monkeypatch, tmp_path, body))
assert not patched and not commented, "a quiet run must be completely silent"
def test_a_relabel_refreshes_the_body_but_says_NOTHING(self, monkeypatch, tmp_path):
# Same findings (same fingerprint), different rendering -- the exact shape of
# the master -> 27-dev relabel.
stale = compat.render_issue(self.ROWS).replace("27-dev", "unreleased")
assert compat.extract_fingerprint(stale) == compat.fingerprint(self.ROWS)
patched, commented = self._writes(self._run(monkeypatch, tmp_path, stale))
assert patched, "the body was left stale, so the issue keeps telling lies"
assert "27-dev" in patched[0][2]["body"]
assert not commented, (
"a rendering change is not news -- commenting on it is how the bot gets "
"muted before the next real finding"
)
def test_a_REAL_findings_change_still_comments(self, monkeypatch, tmp_path):
# ...and the fix must not have made it mute.
stale = compat.render_issue(self.ROWS).replace(
compat.fingerprint(self.ROWS), "0" * 16)
patched, commented = self._writes(self._run(monkeypatch, tmp_path, stale))
assert patched and commented, "a genuine change must still notify"
def test_a_body_differing_only_by_CRLF_is_not_rewritten(self, monkeypatch, tmp_path):
# Forges round-trip line endings. Without normalising, every run would rewrite
# the body -- silent, but it churns updated_at and looks freshly touched daily.
body = compat.render_issue(self.ROWS).replace("\n", "\r\n")
patched, _ = self._writes(self._run(monkeypatch, tmp_path, body))
assert not patched
+6
View File
@@ -1914,6 +1914,9 @@ class TestTheSnapshotRecordIsNeverLostToAnUnreadableFile:
only record of a tree it had just failed to read.
"""
@pytest.mark.skipif(os.geteuid() == 0, reason=
"chmod(0) cannot make a file unreadable for root (CAP_DAC_OVERRIDE); "
"the Gitea runner image executes jobs as root")
def test_an_unreadable_sidecar_raises_rather_than_reading_as_empty(self, tmp_path):
sc = tmp_path / "cloud_backup-5.snapshot"
sc.write_text("Tap@snap\n")
@@ -1927,6 +1930,9 @@ class TestTheSnapshotRecordIsNeverLostToAnUnreadableFile:
def test_a_missing_sidecar_is_simply_empty(self, tmp_path):
assert tn._read_sidecar(str(tmp_path / "nope")) == []
@pytest.mark.skipif(os.geteuid() == 0, reason=
"chmod(0) cannot make a file unreadable for root (CAP_DAC_OVERRIDE); "
"the Gitea runner image executes jobs as root")
def test_cleanup_all_still_UNMOUNTS_when_a_sidecar_cannot_be_read(self, tmp_path):
# cleanup_all is what recover.sh and uninstall.sh call, i.e. it runs precisely
# when the box is already stuck. Its job is to get the mounts off. Aborting on
+96 -13
View File
@@ -871,6 +871,24 @@ def discover_refs(remote: str = REPO) -> list[str]:
* UNRELEASED comes from the BRANCHES, because that is where a beta appears
first: `release/26.0.0-BETA.3` had no tag yet while it was the newest beta.
Catching breakage here, before it ships, is the whole point of this file.
THE NEXT MAINTENANCE RELEASE IS ALSO A BRANCH, and it used to fall through both
sieves. `release/25.10.5` is branched but not yet tagged, and 25.10 has already
shipped -- so it is not in `shipped` (no tag) and it was excluded from `upcoming`
(its line is in `shipped_lines`). It was invisible. That is the one ref a 25.10.4
user is actually about to be upgraded onto, so a break there reaches people
BEFORE the daily check ever looks at it -- the exact hole this file exists to
close, on the only line anybody is running.
The rule that separates it from the two things we must NOT report:
* `release/24.10-RC.2` -- a prerelease of an already-shipped line. It is
history, not a warning; it sorts OLDER than TS-24.10.2.4, so it is dropped.
* `release/25.20.2.2` -- iX's typo branch. 25.20 never shipped, so it has no
TS tag, so it is not a line at all and is dropped.
...which is: a plain `release/X.Y.Z` branch counts only if its line HAS shipped
and it sorts NEWER than that line's newest tag. Both exclusions fall out of it.
"""
tags = _ls_remote(remote, "--tags")
heads = _ls_remote(remote, "--heads")
@@ -878,26 +896,45 @@ def discover_refs(remote: str = REPO) -> list[str]:
shipped = _newest_per_line([
t for t in tags if t.startswith("TS-") and "-BETA" not in t and "-RC" not in t
])
newest_shipped = {_version_of(t)[0][:2]: _version_of(t) for t in shipped}
release_heads = [h for h in heads if h.startswith("release/")]
# A prerelease of a line that has ALREADY shipped is history, not a warning:
# release/24.10-RC.2 still exists, and the nested module does not apply to it,
# but 24.10 shipped long ago and TS-24.10.2.4 is fine. Reporting it would be a
# standing red row in the matrix for a version nobody can install.
shipped_lines = {_version_of(t)[0][:2] for t in shipped}
upcoming = [
h for h in _newest_per_line([
h for h in heads
if h.startswith("release/") and ("-BETA" in h or "-RC" in h)
h for h in release_heads if "-BETA" in h or "-RC" in h
])
if _version_of(h)[0][:2] not in shipped_lines
if _version_of(h)[0][:2] not in newest_shipped
]
return [*shipped, *upcoming, "master"]
# The next maintenance release of a line that HAS shipped: branched, untagged,
# and the very next thing those users get. See the docstring.
pending = []
for h in _newest_per_line([
h for h in release_heads if "-BETA" not in h and "-RC" not in h
]):
v = _version_of(h)
tagged = newest_shipped.get(v[0][:2])
if tagged and v > tagged:
pending.append(h)
return [*shipped, *pending, *upcoming, "master"]
def is_unreleased(ref: str) -> bool:
"""master and any BETA/RC. Breakage here is early warning, not an outage."""
return ref == "master" or "-BETA" in ref or "-RC" in ref
"""Anything iX has not TAGGED. Breakage here is early warning, not an outage.
Keyed on where the ref came from, not on its name: `discover_refs` takes shipped
releases from `TS-*` TAGS and everything else from BRANCHES, so a `release/*` ref
is by construction something iX has not released yet. Testing for `-BETA`/`-RC`
instead would call `release/25.10.5` SHIPPED, and a break there would fail the
build as a live outage -- on a version nobody is running yet.
"""
return ref == "master" or ref.startswith("release/")
def matrix(refs=None, remote: str = REPO) -> list[dict]:
@@ -941,10 +978,15 @@ def is_broken(r: dict) -> bool:
#: is static analysis of iX's source, which proves the patch's assumptions hold --
#: a strictly weaker claim than "a restore worked". Add a row only after doing it.
HARDWARE_VERIFIED = {
"25.10.4": "nested + providers; 252-snapshot recursive backup of /mnt/Tap, 18m",
"25.10.4": (
"v0.7.0: 3 live tasks — 191-dataset nested backup of /mnt/Tap, a "
"215-filesystem/2-zvol backup of /mnt/Tank/backups, and a non-nested one; "
"0 orphans, 0 leaked mounts, byte-identical restore; the collector also "
"reclaimed a real orphan the pool had been carrying"
),
"26.0.0-BETA.1": (
"nested + providers; 274-snapshot recursive backup of a 292-dataset pool, "
"restored a 4-deep child dataset byte-identical"
"v0.7.0: 274-snapshot recursive backup of a 292-dataset pool; restored a "
"4-deep child dataset byte-identical; zvol-orphan case reproduced then closed"
),
}
@@ -958,6 +1000,16 @@ _LEGEND = """
"ok" means *the patch's assumptions hold*, checked automatically against iX's
source. It does not mean a human ran a backup on it — that is the
**Hardware-verified** column, which is filled in by hand and only by doing it.
**`master` is not the next release.** iX branches each major off to its own
`release/` line and master rolls straight on to the one after — so master is
`27-dev` while 26 is still in beta. A **BROKEN** master means iX has changed
something that will reach users *a major release from now*, not in the version you
are about to install. Read the numbered rows for that.
A row like `25.10.5 _(unreleased)_` is the next maintenance release: branched by iX,
not tagged yet, and the very next thing a 25.10.4 box gets. It is checked precisely
because it is the one unshipped ref that reaches real users without warning.
"""
@@ -1005,17 +1057,42 @@ def update_readme(rows: list[dict], path: str = README) -> bool:
return True
def dev_label(rows: list[dict]) -> str:
"""What `master` is a development line FOR -- e.g. "27-dev".
`master` is NOT the next release, and labelling it "master _(unreleased)_" said
it was. On 2026-07-14 every recent commit on master targeted 27.0.0-BETA.1 while
26 was still in beta on its own `release/26.0.0-BETA.*` branches: master had
already rolled over to the major AFTER the one that has not shipped yet. So a red
`master` row read as "the version you are about to install is broken" when the
breakage was a year out, on a line nobody can even download. That is a false alarm
aimed squarely at the person deciding whether to trust this with their backups.
Derived, not hardcoded: the newest major we can see anywhere, plus one. iX branches
`release/N.0.0-BETA.1` off master and master immediately becomes N+1 -- so when 27
betas appear, this says 28-dev on its own.
"""
majors = [
v[0][0] for v in (_version_of(r["ref"]) for r in rows if r["ref"] != "master")
if v
]
return f"{max(majors) + 1}-dev" if majors else "unreleased"
def render_markdown(rows: list[dict]) -> str:
"""The matrix, for the README."""
out = [
"| TrueNAS | B2/S3 providers | Nested snapshots | Hardware-verified |",
"| --- | --- | --- | --- |",
]
dev = dev_label(rows)
for row in rows:
m = row["modules"]
ref = row["ref"]
label = ref.removeprefix("TS-").removeprefix("release/")
if row["unreleased"]:
if ref == "master":
label = f"master _({dev})_"
elif row["unreleased"]:
label = f"{label} _(unreleased)_"
cells = []
@@ -1133,7 +1210,13 @@ def render_matrix(rows: list[dict]) -> str:
hardware-verified column lives in COMPATIBILITY.md and is maintained by hand,
because nothing else can honestly fill it in.
"""
w = max((len(r["ref"]) for r in rows), default=10)
dev = dev_label(rows)
# Same relabel as the README: a red `master` is a warning about the major AFTER
# next, and "master" alone reads as "the release you are about to install".
names = {r["ref"]: (f"master ({dev})" if r["ref"] == "master" else r["ref"])
for r in rows}
w = max((len(n) for n in names.values()), default=10)
lines = [
f"{'TrueNAS'.ljust(w)} {'providers':<10} {'nested':<10}",
f"{'-' * w} {'-' * 10} {'-' * 10}",
@@ -1141,7 +1224,7 @@ def render_matrix(rows: list[dict]) -> str:
for row in rows:
m = row["modules"]
lines.append(
f"{row['ref'].ljust(w)} "
f"{names[row['ref']].ljust(w)} "
f"{_verdict(m[PROVIDERS]):<10} {_verdict(m[NESTED]):<10}"
)
return "\n".join(lines)
+44 -7
View File
@@ -62,6 +62,19 @@ def _call(url, token, method="GET", data=None):
return json.load(r) if r.length != 0 else {}
def _same(a, b):
"""Is the issue body already what we would write?
Compared after normalising line endings and trailing space: forges are free to
round-trip `\r\n`, and a body that only "differs" by that would be rewritten on
every single run -- a silent edit, but a pointless one that churns `updated_at`
and makes the issue look freshly touched every morning.
"""
def norm(s):
return "\n".join(line.rstrip() for line in (s or "").replace("\r\n", "\n").split("\n")).strip()
return norm(a) == norm(b)
def find_issue(api, token, title):
"""The LOWEST-numbered issue with this title, open or closed.
@@ -69,11 +82,18 @@ def find_issue(api, token, title):
existed once (an earlier version put the ref list in the title, so the identity
changed whenever that set changed), and an order-dependent pick would alternate
between them -- reopening one while commenting on the other.
Both forges list PRs alongside issues, but they SAY SO DIFFERENTLY: GitHub omits
the `pull_request` key on a plain issue, Gitea sends it as `null`. Testing for the
KEY therefore discards every Gitea issue as if it were a PR -- so this returned
None on every Gitea run, and the bot filed a brand-new duplicate report each time
instead of editing the one it already had. Test the VALUE; it is the only form
that is true on both.
"""
issues = _call(f"{api}/issues?state=all&per_page=100", token)
issues = _call(f"{api}/issues?state=all&per_page=100&limit=100", token)
mine = [
i for i in issues
if i.get("title") == title and "pull_request" not in i # GitHub lists PRs here
if i.get("title") == title and not i.get("pull_request")
]
return min(mine, key=lambda i: i["number"]) if mine else None
@@ -117,11 +137,24 @@ def main(argv):
have = extract_fingerprint(issue.get("body") or "")
n = issue["number"]
# ── the findings are UNCHANGED: say nothing ──────────────────────────────
# ── two different questions, and they were being answered with one answer ────
#
# This is the whole point. A daily "still broken, same as yesterday" comment is
# what taught everyone to ignore the last one.
if have == want and issue["state"] == "open":
# * IS THE BODY STILL TRUE? -> if not, rewrite it. Editing an issue body
# notifies NOBODY on either forge, so keeping it honest is free.
# * HAVE THE FINDINGS CHANGED? -> only then comment. Comments DO notify, and a
# daily "still broken, same as yesterday" is what teaches everyone to ignore
# the one that finally matters.
#
# Conflating them meant an unchanged FINGERPRINT froze the BODY. The fingerprint
# deliberately ignores everything that moves on its own -- healthy rows, the
# hardware-verified column, point releases, how a row is LABELLED -- so none of
# that could ever reach the report. Relabelling master `27-dev` (it is not the
# next release; a red row there was reading as "the version you are about to
# install is broken") would have shipped to the README and never to the issue
# anybody actually opens.
body_is_current = _same(issue.get("body"), body)
if have == want and issue["state"] == "open" and body_is_current:
print(f"#{n} is already current ({want}) — staying quiet")
return 0
@@ -139,8 +172,12 @@ def main(argv):
)
_call(f"{args.api}/issues/{n}/comments", args.token, "POST", {"body": note})
print(f"updated #{n}: {have} -> {want}")
else:
elif issue["state"] != "open":
print(f"reopened #{n}")
else:
# Same findings, new rendering. Silent by design: nothing has changed that
# anybody needs waking up for, but the report should not be telling lies.
print(f"#{n}: findings unchanged ({want}); body refreshed silently")
return 0
Generated
+3
View File
@@ -0,0 +1,3 @@
version = 1
revision = 3
requires-python = ">=3.12"