master is 27-dev, not the next release; and check the next maintenance release
CI / shell (shellcheck + syntax) (push) Successful in 10s
CI / python 3.11 (push) Failing after 13s
CI / python 3.12 (push) Failing after 13s
TrueNAS compatibility / compat (push) Successful in 11s
CI / python 3.13 (push) Failing after 13s

Two ways the matrix misled the person it exists for -- somebody deciding whether
to trust this with their backups.

master is not the next release. iX branches each major onto its own release/ line
and master rolls straight on to the one after: every recent commit on master
targets 27.0.0-BETA.1 while 26 is still in beta. So a BROKEN master row, rendered
"master (unreleased)", read as "the version you are about to install is broken"
when the breakage is a major release away on a line nobody can download. It is now
labelled from the newest major in the matrix plus one, so it becomes 28-dev by
itself once 27 branches.

The break, for the record, is NAS-141498 (2026-06-24), "Convert cloud_backup
plugin to the typesafe pattern": it re-signatures restic_backup and
get_restic_config, splitting entry/credentials out of the cloud_backup dict. Not
being chased while the 27 line churns.

The next maintenance release was never checked -- and it is the one that reaches
users. Shipped came from TS-* tags, unreleased from release/* branches carrying
-BETA/-RC. A branched-but-untagged MAINTENANCE release is neither: release/25.10.5
has no tag, and its line has already shipped, so the "a prerelease of a shipped
line is history" filter threw it out. It was invisible, and it is exactly what a
25.10.4 box gets on its next update; a break in it reaches real users before the
daily check ever looks, on the only line anybody runs.

A plain release/X.Y.Z branch now counts when its line HAS shipped and it sorts
NEWER than that line's newest tag. Both exclusions fall out of the same rule:
release/24.10-RC.2 sorts older than TS-24.10.2.4 (history), and the typo branch
release/25.20.2.2 is on a line with no tag at all (not a release line). This
surfaced two refs never checked before -- release/25.10.5 and release/24.10.2.5 --
both of which pass.

is_unreleased() keys off where a ref came from (a branch is by construction not
shipped) rather than hunting -BETA/-RC in the name. Otherwise release/25.10.5
counts as shipped and a break in it fails the build as a live outage, on a version
nobody is running yet.
This commit is contained in:
2026-07-14 03:03:21 +00:00
parent 753c3f8cad
commit 862bdd3399
4 changed files with 236 additions and 11 deletions
+96
View File
@@ -629,3 +629,99 @@ class TestTheBotFindsItsOwnIssueOnBOTHForges:
{"number": 1, "title": "TypeError when create B2 backup on Electric Eel",
"state": "closed", "pull_request": None},
]) is None
class TestTheNextMaintenanceReleaseIsChecked:
"""`release/25.10.5` fell through every sieve, and it is the one that reaches users.
Shipped versions come from `TS-*` TAGS; unreleased ones come from `release/*`
BRANCHES that carry `-BETA`/`-RC`. A branched-but-untagged MAINTENANCE release is
neither: no tag, and its line (25.10) has already shipped, so the "prereleases of
a shipped line are history" filter threw it out. It was invisible.
That is backwards. `release/24.10-RC.2` is history -- nobody can install it. But
`release/25.10.5` is the FUTURE of a shipped line: it is what a 25.10.4 box gets
on its next update. A break there ships to real users before the daily check has
ever looked at it.
"""
TAGS = ["TS-24.10.2.4", "TS-25.04.2.6", "TS-25.10.4"]
HEADS = [
"release/25.10.4.1",
"release/25.10.5", # branched, untagged -- the next maintenance release
"release/24.10-RC.2", # history: its line shipped long ago
"release/25.20.2.2", # iX's typo branch: 25.20 is not a TrueNAS version
"release/26.0.0-BETA.3",
"master",
]
def _refs(self, monkeypatch):
monkeypatch.setattr(
compat, "_ls_remote",
lambda remote, what: self.TAGS if what == "--tags" else self.HEADS)
return compat.discover_refs("origin")
def test_the_next_maintenance_release_is_checked(self, monkeypatch):
assert "release/25.10.5" in self._refs(monkeypatch), (
"the next thing a 25.10.4 box updates to is not checked, so a break in it "
"reaches users before the bot ever sees it"
)
def test_a_superseded_maintenance_branch_is_not(self, monkeypatch):
# 25.10.4.1 sorts OLDER than the newest tag TS-25.10.4? No -- it is NEWER, and
# both are on the 25.10 line, so only the newest branch on the line is taken.
refs = self._refs(monkeypatch)
assert "release/25.10.4.1" not in refs, "only the newest branch per line"
def test_the_typo_branch_stays_out(self, monkeypatch):
# 25.20 has no TS tag, so it is not a release line at all. A typo branch in the
# matrix reads as a real supported release we are silently broken on.
assert "release/25.20.2.2" not in self._refs(monkeypatch)
def test_a_prerelease_of_an_already_shipped_line_stays_out(self, monkeypatch):
assert "release/24.10-RC.2" not in self._refs(monkeypatch)
def test_an_untagged_branch_counts_as_UNRELEASED(self, monkeypatch):
# The exit code keys off this. Calling 25.10.5 "shipped" would fail the build
# as a live outage on a version nobody is running yet.
assert compat.is_unreleased("release/25.10.5")
assert compat.is_unreleased("master")
assert not compat.is_unreleased("TS-25.10.4")
class TestMasterIsNotTheNextRelease:
"""A red `master` row used to read as "the version you are about to install".
On 2026-07-14 master was 27-dev -- every recent commit targeted 27.0.0-BETA.1 --
while 26 was still in beta on its own branches. So `master BROKEN` meant "iX will
break us a major release from now", but the matrix said "master _(unreleased)_",
which any reader takes as the next thing out the door. For a table whose whole job
is helping somebody decide whether to trust this with their backups, that is a
false alarm in the worst possible place.
"""
def _rows(self, refs):
return [{"ref": r, "unreleased": compat.is_unreleased(r), "modules": {}}
for r in refs]
def test_master_is_labelled_with_the_major_AFTER_the_newest_known_one(self):
rows = self._rows(["TS-25.10.4", "release/26.0.0-BETA.3", "master"])
assert compat.dev_label(rows) == "27-dev"
def test_it_rolls_over_on_its_own_when_the_next_beta_branches(self):
# Derived, not hardcoded: when release/27.0.0-BETA.1 appears, master is 28-dev.
rows = self._rows(["TS-26.0.0", "release/27.0.0-BETA.1", "master"])
assert compat.dev_label(rows) == "28-dev"
def test_the_rendered_matrix_says_dev_not_unreleased(self):
healthy = check_files(with_())
rows = [
{"ref": r, "unreleased": compat.is_unreleased(r), "modules": healthy}
for r in ("TS-25.10.4", "release/26.0.0-BETA.3", "master")
]
md = compat.render_markdown(rows)
assert "master _(27-dev)_" in md
assert "master _(unreleased)_" not in md
# ...and the ordinary rows are untouched.
assert "| 25.10.4 |" in md
assert "| 26.0.0-BETA.3 _(unreleased)_ |" in md