master is 27-dev, not the next release; and check the next maintenance release
CI / shell (shellcheck + syntax) (push) Successful in 10s
CI / python 3.11 (push) Failing after 13s
CI / python 3.12 (push) Failing after 13s
TrueNAS compatibility / compat (push) Successful in 11s
CI / python 3.13 (push) Failing after 13s

Two ways the matrix misled the person it exists for -- somebody deciding whether
to trust this with their backups.

master is not the next release. iX branches each major onto its own release/ line
and master rolls straight on to the one after: every recent commit on master
targets 27.0.0-BETA.1 while 26 is still in beta. So a BROKEN master row, rendered
"master (unreleased)", read as "the version you are about to install is broken"
when the breakage is a major release away on a line nobody can download. It is now
labelled from the newest major in the matrix plus one, so it becomes 28-dev by
itself once 27 branches.

The break, for the record, is NAS-141498 (2026-06-24), "Convert cloud_backup
plugin to the typesafe pattern": it re-signatures restic_backup and
get_restic_config, splitting entry/credentials out of the cloud_backup dict. Not
being chased while the 27 line churns.

The next maintenance release was never checked -- and it is the one that reaches
users. Shipped came from TS-* tags, unreleased from release/* branches carrying
-BETA/-RC. A branched-but-untagged MAINTENANCE release is neither: release/25.10.5
has no tag, and its line has already shipped, so the "a prerelease of a shipped
line is history" filter threw it out. It was invisible, and it is exactly what a
25.10.4 box gets on its next update; a break in it reaches real users before the
daily check ever looks, on the only line anybody runs.

A plain release/X.Y.Z branch now counts when its line HAS shipped and it sorts
NEWER than that line's newest tag. Both exclusions fall out of the same rule:
release/24.10-RC.2 sorts older than TS-24.10.2.4 (history), and the typo branch
release/25.20.2.2 is on a line with no tag at all (not a release line). This
surfaced two refs never checked before -- release/25.10.5 and release/24.10.2.5 --
both of which pass.

is_unreleased() keys off where a ref came from (a branch is by construction not
shipped) rather than hunting -BETA/-RC in the name. Otherwise release/25.10.5
counts as shipped and a break in it fails the build as a live outage, on a version
nobody is running yet.
This commit is contained in:
2026-07-14 03:03:21 +00:00
parent 753c3f8cad
commit 862bdd3399
4 changed files with 236 additions and 11 deletions
+39
View File
@@ -7,8 +7,47 @@ live, every one of those interrupts every user. An alert people learn to ignore
worse than no alert, because one day it carries a security fix. worse than no alert, because one day it carries a security fix.
## Unreleased ## Unreleased
### Changed
- **`master` is now labelled `27-dev`, because it is not the next release.** iX
branches each major onto its own `release/` line and master rolls straight on to the
one after — on 2026-07-14 every recent commit on master targeted `27.0.0-BETA.1`
while 26 was still in beta. So a **BROKEN** master row, rendered as
"master _(unreleased)_", read as *"the version you are about to install is broken"*
when the breakage was a major release away on a line nobody can download. In a table
whose entire job is helping somebody decide whether to trust this with their backups,
that is a false alarm in the worst possible place. The label is derived from the
newest major in the matrix plus one, so it rolls over to `28-dev` by itself once 27
branches.
For the record, the breakage is `NAS-141498` (2026-06-24), "Convert cloud_backup
plugin to the typesafe pattern": it re-signatures `restic_backup` and
`get_restic_config`, splitting `entry`/`credentials` out of the `cloud_backup` dict.
It is deliberately not being chased while the 27 line is still churning.
### Fixed ### Fixed
- **The next maintenance release was never checked, and it is the one that reaches
users.** Shipped versions were discovered from `TS-*` tags and unreleased ones from
`release/*` branches carrying `-BETA`/`-RC`. A branched-but-untagged *maintenance*
release is neither: `release/25.10.5` has no tag, and its line has already shipped,
so the "a prerelease of a shipped line is history" filter discarded it. It was
invisible — and it is precisely what a 25.10.4 box gets on its next update. A break
there would have reached real users before the daily check ever looked at it, on the
only line anybody is actually running.
A plain `release/X.Y.Z` branch is now checked when its line **has** shipped and it
sorts **newer** than that line's newest tag. Both things that must stay out fall out
of the same rule: `release/24.10-RC.2` sorts older than `TS-24.10.2.4` (history, not
a warning), and iX's typo branch `release/25.20.2.2` is on a line that has no tag at
all, so it is not a release line. This immediately surfaced two refs that had never
been checked — `release/25.10.5` and `release/24.10.2.5` — both of which pass.
`is_unreleased()` now keys off where a ref came from (branch = not yet shipped)
rather than looking for `-BETA`/`-RC` in its name. Otherwise `release/25.10.5` would
count as shipped and a break in it would fail the build as a live outage — on a
version nobody is running yet.
- **The compatibility bot filed a new duplicate bug report on every Gitea run.** - **The compatibility bot filed a new duplicate bug report on every Gitea run.**
`find_issue()` skipped pull requests by testing for the *presence* of the `find_issue()` skipped pull requests by testing for the *presence* of the
`pull_request` key. GitHub omits that key on a plain issue; Gitea sends it as `pull_request` key. GitHub omits that key on a plain issue; Gitea sends it as
+13 -1
View File
@@ -60,8 +60,10 @@ If something is wrong, the reason is in `apply.log` — start at
| 24.10.2.4 | ok | ok | — | | 24.10.2.4 | ok | ok | — |
| 25.04.2.6 | ok | ok | — | | 25.04.2.6 | ok | ok | — |
| 25.10.4 | ok | ok | v0.7.0: 3 live tasks — 191-dataset nested backup of /mnt/Tap, a 215-filesystem/2-zvol backup of /mnt/Tank/backups, and a non-nested one; 0 orphans, 0 leaked mounts, byte-identical restore; the collector also reclaimed a real orphan the pool had been carrying | | 25.10.4 | ok | ok | v0.7.0: 3 live tasks — 191-dataset nested backup of /mnt/Tap, a 215-filesystem/2-zvol backup of /mnt/Tank/backups, and a non-nested one; 0 orphans, 0 leaked mounts, byte-identical restore; the collector also reclaimed a real orphan the pool had been carrying |
| 24.10.2.5 _(unreleased)_ | ok | ok | — |
| 25.10.5 _(unreleased)_ | ok | ok | — |
| 26.0.0-BETA.3 _(unreleased)_ | ok | ok | — | | 26.0.0-BETA.3 _(unreleased)_ | ok | ok | — |
| master _(unreleased)_ | **BROKEN** | **BROKEN** | — | | master _(27-dev)_ | **BROKEN** | **BROKEN** | — |
| verdict | meaning | | verdict | meaning |
| --- | --- | | --- | --- |
@@ -72,6 +74,16 @@ If something is wrong, the reason is in `apply.log` — start at
"ok" means *the patch's assumptions hold*, checked automatically against iX's "ok" means *the patch's assumptions hold*, checked automatically against iX's
source. It does not mean a human ran a backup on it — that is the source. It does not mean a human ran a backup on it — that is the
**Hardware-verified** column, which is filled in by hand and only by doing it. **Hardware-verified** column, which is filled in by hand and only by doing it.
**`master` is not the next release.** iX branches each major off to its own
`release/` line and master rolls straight on to the one after — so master is
`27-dev` while 26 is still in beta. A **BROKEN** master means iX has changed
something that will reach users *a major release from now*, not in the version you
are about to install. Read the numbered rows for that.
A row like `25.10.5 _(unreleased)_` is the next maintenance release: branched by iX,
not tagged yet, and the very next thing a 25.10.4 box gets. It is checked precisely
because it is the one unshipped ref that reaches real users without warning.
<!-- END COMPAT MATRIX --> <!-- END COMPAT MATRIX -->
The table is **regenerated daily by CI** against iXsystems' actual middleware source The table is **regenerated daily by CI** against iXsystems' actual middleware source
+96
View File
@@ -629,3 +629,99 @@ class TestTheBotFindsItsOwnIssueOnBOTHForges:
{"number": 1, "title": "TypeError when create B2 backup on Electric Eel", {"number": 1, "title": "TypeError when create B2 backup on Electric Eel",
"state": "closed", "pull_request": None}, "state": "closed", "pull_request": None},
]) is None ]) is None
class TestTheNextMaintenanceReleaseIsChecked:
"""`release/25.10.5` fell through every sieve, and it is the one that reaches users.
Shipped versions come from `TS-*` TAGS; unreleased ones come from `release/*`
BRANCHES that carry `-BETA`/`-RC`. A branched-but-untagged MAINTENANCE release is
neither: no tag, and its line (25.10) has already shipped, so the "prereleases of
a shipped line are history" filter threw it out. It was invisible.
That is backwards. `release/24.10-RC.2` is history -- nobody can install it. But
`release/25.10.5` is the FUTURE of a shipped line: it is what a 25.10.4 box gets
on its next update. A break there ships to real users before the daily check has
ever looked at it.
"""
TAGS = ["TS-24.10.2.4", "TS-25.04.2.6", "TS-25.10.4"]
HEADS = [
"release/25.10.4.1",
"release/25.10.5", # branched, untagged -- the next maintenance release
"release/24.10-RC.2", # history: its line shipped long ago
"release/25.20.2.2", # iX's typo branch: 25.20 is not a TrueNAS version
"release/26.0.0-BETA.3",
"master",
]
def _refs(self, monkeypatch):
monkeypatch.setattr(
compat, "_ls_remote",
lambda remote, what: self.TAGS if what == "--tags" else self.HEADS)
return compat.discover_refs("origin")
def test_the_next_maintenance_release_is_checked(self, monkeypatch):
assert "release/25.10.5" in self._refs(monkeypatch), (
"the next thing a 25.10.4 box updates to is not checked, so a break in it "
"reaches users before the bot ever sees it"
)
def test_a_superseded_maintenance_branch_is_not(self, monkeypatch):
# 25.10.4.1 sorts OLDER than the newest tag TS-25.10.4? No -- it is NEWER, and
# both are on the 25.10 line, so only the newest branch on the line is taken.
refs = self._refs(monkeypatch)
assert "release/25.10.4.1" not in refs, "only the newest branch per line"
def test_the_typo_branch_stays_out(self, monkeypatch):
# 25.20 has no TS tag, so it is not a release line at all. A typo branch in the
# matrix reads as a real supported release we are silently broken on.
assert "release/25.20.2.2" not in self._refs(monkeypatch)
def test_a_prerelease_of_an_already_shipped_line_stays_out(self, monkeypatch):
assert "release/24.10-RC.2" not in self._refs(monkeypatch)
def test_an_untagged_branch_counts_as_UNRELEASED(self, monkeypatch):
# The exit code keys off this. Calling 25.10.5 "shipped" would fail the build
# as a live outage on a version nobody is running yet.
assert compat.is_unreleased("release/25.10.5")
assert compat.is_unreleased("master")
assert not compat.is_unreleased("TS-25.10.4")
class TestMasterIsNotTheNextRelease:
"""A red `master` row used to read as "the version you are about to install".
On 2026-07-14 master was 27-dev -- every recent commit targeted 27.0.0-BETA.1 --
while 26 was still in beta on its own branches. So `master BROKEN` meant "iX will
break us a major release from now", but the matrix said "master _(unreleased)_",
which any reader takes as the next thing out the door. For a table whose whole job
is helping somebody decide whether to trust this with their backups, that is a
false alarm in the worst possible place.
"""
def _rows(self, refs):
return [{"ref": r, "unreleased": compat.is_unreleased(r), "modules": {}}
for r in refs]
def test_master_is_labelled_with_the_major_AFTER_the_newest_known_one(self):
rows = self._rows(["TS-25.10.4", "release/26.0.0-BETA.3", "master"])
assert compat.dev_label(rows) == "27-dev"
def test_it_rolls_over_on_its_own_when_the_next_beta_branches(self):
# Derived, not hardcoded: when release/27.0.0-BETA.1 appears, master is 28-dev.
rows = self._rows(["TS-26.0.0", "release/27.0.0-BETA.1", "master"])
assert compat.dev_label(rows) == "28-dev"
def test_the_rendered_matrix_says_dev_not_unreleased(self):
healthy = check_files(with_())
rows = [
{"ref": r, "unreleased": compat.is_unreleased(r), "modules": healthy}
for r in ("TS-25.10.4", "release/26.0.0-BETA.3", "master")
]
md = compat.render_markdown(rows)
assert "master _(27-dev)_" in md
assert "master _(unreleased)_" not in md
# ...and the ordinary rows are untouched.
assert "| 25.10.4 |" in md
assert "| 26.0.0-BETA.3 _(unreleased)_ |" in md
+88 -10
View File
@@ -871,6 +871,24 @@ def discover_refs(remote: str = REPO) -> list[str]:
* UNRELEASED comes from the BRANCHES, because that is where a beta appears * UNRELEASED comes from the BRANCHES, because that is where a beta appears
first: `release/26.0.0-BETA.3` had no tag yet while it was the newest beta. first: `release/26.0.0-BETA.3` had no tag yet while it was the newest beta.
Catching breakage here, before it ships, is the whole point of this file. Catching breakage here, before it ships, is the whole point of this file.
THE NEXT MAINTENANCE RELEASE IS ALSO A BRANCH, and it used to fall through both
sieves. `release/25.10.5` is branched but not yet tagged, and 25.10 has already
shipped -- so it is not in `shipped` (no tag) and it was excluded from `upcoming`
(its line is in `shipped_lines`). It was invisible. That is the one ref a 25.10.4
user is actually about to be upgraded onto, so a break there reaches people
BEFORE the daily check ever looks at it -- the exact hole this file exists to
close, on the only line anybody is running.
The rule that separates it from the two things we must NOT report:
* `release/24.10-RC.2` -- a prerelease of an already-shipped line. It is
history, not a warning; it sorts OLDER than TS-24.10.2.4, so it is dropped.
* `release/25.20.2.2` -- iX's typo branch. 25.20 never shipped, so it has no
TS tag, so it is not a line at all and is dropped.
...which is: a plain `release/X.Y.Z` branch counts only if its line HAS shipped
and it sorts NEWER than that line's newest tag. Both exclusions fall out of it.
""" """
tags = _ls_remote(remote, "--tags") tags = _ls_remote(remote, "--tags")
heads = _ls_remote(remote, "--heads") heads = _ls_remote(remote, "--heads")
@@ -878,26 +896,45 @@ def discover_refs(remote: str = REPO) -> list[str]:
shipped = _newest_per_line([ shipped = _newest_per_line([
t for t in tags if t.startswith("TS-") and "-BETA" not in t and "-RC" not in t t for t in tags if t.startswith("TS-") and "-BETA" not in t and "-RC" not in t
]) ])
newest_shipped = {_version_of(t)[0][:2]: _version_of(t) for t in shipped}
release_heads = [h for h in heads if h.startswith("release/")]
# A prerelease of a line that has ALREADY shipped is history, not a warning: # A prerelease of a line that has ALREADY shipped is history, not a warning:
# release/24.10-RC.2 still exists, and the nested module does not apply to it, # release/24.10-RC.2 still exists, and the nested module does not apply to it,
# but 24.10 shipped long ago and TS-24.10.2.4 is fine. Reporting it would be a # but 24.10 shipped long ago and TS-24.10.2.4 is fine. Reporting it would be a
# standing red row in the matrix for a version nobody can install. # standing red row in the matrix for a version nobody can install.
shipped_lines = {_version_of(t)[0][:2] for t in shipped}
upcoming = [ upcoming = [
h for h in _newest_per_line([ h for h in _newest_per_line([
h for h in heads h for h in release_heads if "-BETA" in h or "-RC" in h
if h.startswith("release/") and ("-BETA" in h or "-RC" in h)
]) ])
if _version_of(h)[0][:2] not in shipped_lines if _version_of(h)[0][:2] not in newest_shipped
] ]
return [*shipped, *upcoming, "master"] # The next maintenance release of a line that HAS shipped: branched, untagged,
# and the very next thing those users get. See the docstring.
pending = []
for h in _newest_per_line([
h for h in release_heads if "-BETA" not in h and "-RC" not in h
]):
v = _version_of(h)
tagged = newest_shipped.get(v[0][:2])
if tagged and v > tagged:
pending.append(h)
return [*shipped, *pending, *upcoming, "master"]
def is_unreleased(ref: str) -> bool: def is_unreleased(ref: str) -> bool:
"""master and any BETA/RC. Breakage here is early warning, not an outage.""" """Anything iX has not TAGGED. Breakage here is early warning, not an outage.
return ref == "master" or "-BETA" in ref or "-RC" in ref
Keyed on where the ref came from, not on its name: `discover_refs` takes shipped
releases from `TS-*` TAGS and everything else from BRANCHES, so a `release/*` ref
is by construction something iX has not released yet. Testing for `-BETA`/`-RC`
instead would call `release/25.10.5` SHIPPED, and a break there would fail the
build as a live outage -- on a version nobody is running yet.
"""
return ref == "master" or ref.startswith("release/")
def matrix(refs=None, remote: str = REPO) -> list[dict]: def matrix(refs=None, remote: str = REPO) -> list[dict]:
@@ -963,6 +1000,16 @@ _LEGEND = """
"ok" means *the patch's assumptions hold*, checked automatically against iX's "ok" means *the patch's assumptions hold*, checked automatically against iX's
source. It does not mean a human ran a backup on it — that is the source. It does not mean a human ran a backup on it — that is the
**Hardware-verified** column, which is filled in by hand and only by doing it. **Hardware-verified** column, which is filled in by hand and only by doing it.
**`master` is not the next release.** iX branches each major off to its own
`release/` line and master rolls straight on to the one after — so master is
`27-dev` while 26 is still in beta. A **BROKEN** master means iX has changed
something that will reach users *a major release from now*, not in the version you
are about to install. Read the numbered rows for that.
A row like `25.10.5 _(unreleased)_` is the next maintenance release: branched by iX,
not tagged yet, and the very next thing a 25.10.4 box gets. It is checked precisely
because it is the one unshipped ref that reaches real users without warning.
""" """
@@ -1010,17 +1057,42 @@ def update_readme(rows: list[dict], path: str = README) -> bool:
return True return True
def dev_label(rows: list[dict]) -> str:
"""What `master` is a development line FOR -- e.g. "27-dev".
`master` is NOT the next release, and labelling it "master _(unreleased)_" said
it was. On 2026-07-14 every recent commit on master targeted 27.0.0-BETA.1 while
26 was still in beta on its own `release/26.0.0-BETA.*` branches: master had
already rolled over to the major AFTER the one that has not shipped yet. So a red
`master` row read as "the version you are about to install is broken" when the
breakage was a year out, on a line nobody can even download. That is a false alarm
aimed squarely at the person deciding whether to trust this with their backups.
Derived, not hardcoded: the newest major we can see anywhere, plus one. iX branches
`release/N.0.0-BETA.1` off master and master immediately becomes N+1 -- so when 27
betas appear, this says 28-dev on its own.
"""
majors = [
v[0][0] for v in (_version_of(r["ref"]) for r in rows if r["ref"] != "master")
if v
]
return f"{max(majors) + 1}-dev" if majors else "unreleased"
def render_markdown(rows: list[dict]) -> str: def render_markdown(rows: list[dict]) -> str:
"""The matrix, for the README.""" """The matrix, for the README."""
out = [ out = [
"| TrueNAS | B2/S3 providers | Nested snapshots | Hardware-verified |", "| TrueNAS | B2/S3 providers | Nested snapshots | Hardware-verified |",
"| --- | --- | --- | --- |", "| --- | --- | --- | --- |",
] ]
dev = dev_label(rows)
for row in rows: for row in rows:
m = row["modules"] m = row["modules"]
ref = row["ref"] ref = row["ref"]
label = ref.removeprefix("TS-").removeprefix("release/") label = ref.removeprefix("TS-").removeprefix("release/")
if row["unreleased"]: if ref == "master":
label = f"master _({dev})_"
elif row["unreleased"]:
label = f"{label} _(unreleased)_" label = f"{label} _(unreleased)_"
cells = [] cells = []
@@ -1138,7 +1210,13 @@ def render_matrix(rows: list[dict]) -> str:
hardware-verified column lives in COMPATIBILITY.md and is maintained by hand, hardware-verified column lives in COMPATIBILITY.md and is maintained by hand,
because nothing else can honestly fill it in. because nothing else can honestly fill it in.
""" """
w = max((len(r["ref"]) for r in rows), default=10) dev = dev_label(rows)
# Same relabel as the README: a red `master` is a warning about the major AFTER
# next, and "master" alone reads as "the release you are about to install".
names = {r["ref"]: (f"master ({dev})" if r["ref"] == "master" else r["ref"])
for r in rows}
w = max((len(n) for n in names.values()), default=10)
lines = [ lines = [
f"{'TrueNAS'.ljust(w)} {'providers':<10} {'nested':<10}", f"{'TrueNAS'.ljust(w)} {'providers':<10} {'nested':<10}",
f"{'-' * w} {'-' * 10} {'-' * 10}", f"{'-' * w} {'-' * 10} {'-' * 10}",
@@ -1146,7 +1224,7 @@ def render_matrix(rows: list[dict]) -> str:
for row in rows: for row in rows:
m = row["modules"] m = row["modules"]
lines.append( lines.append(
f"{row['ref'].ljust(w)} " f"{names[row['ref']].ljust(w)} "
f"{_verdict(m[PROVIDERS]):<10} {_verdict(m[NESTED]):<10}" f"{_verdict(m[PROVIDERS]):<10} {_verdict(m[NESTED]):<10}"
) )
return "\n".join(lines) return "\n".join(lines)