Compare commits

...
2 Commits
Author SHA1 Message Date
flan 4e0814028c v0.5.0: TrueNAS alert when an update is available
Raises a real alert in the TrueNAS UI bell -- not a log line nobody reads. On by
default, checked once a day. install.sh --no-update-alerts turns it off.

It does not nag
---------------
A release whose CHANGELOG contains only a "### Docs" section changed no code and
raises nothing. Anything else raises INFO; a "### Security" section raises WARNING.

The CHANGELOG's own section headings are the signal, and a security fix anywhere in
the range escalates the whole span -- a docs-only release sitting on top of a
security fix still reports as security rather than hiding it.

Why an AlertSource and not midclt
----------------------------------
TrueNAS cannot raise an alert from the CLI. midclt exposes only alert.dismiss,
alert.list, alert.list_categories, alert.list_policies and alert.restore -- alert
CREATION is internal to middlewared, and none of its ~60 one-shot classes is
generic enough to reuse. Registering an AlertSource is the only way.

It is also the least invasive thing this patch does. The providers and nested
modules both APPEND CODE TO STOCK middleware files; the alert source ADDS ONE FILE
and modifies none. It is the native mechanism -- the same one every built-in
TrueNAS alert uses -- and TrueNAS polls it itself, so there is no cron job and no
systemd timer.

- Fail-safe: every error path returns None; it cannot take middlewared down.
- Read-only: `git ls-remote` plus an HTTPS fetch of the CHANGELOG. It never writes
  to .git, so it cannot leave root-owned objects behind the way a `git fetch` from
  middlewared (running as root) would.
- Removed by uninstall.sh (mw_patch.revert_all).
- It only tells you; it never updates anything.

Verified against the real repo and remote, with middlewared stubbed:
  on v0.4.1, only a README-only v0.4.2 available -> NO ALERT
  on v0.4.0, v0.4.1 fixed real bugs             -> INFO
  on v0.3.2, v0.3.3 was the password fix        -> SECURITY / WARNING

139 tests, ruff and shellcheck -S style clean.
2026-07-13 16:45:26 +00:00
flan 345741e1f1 v0.4.2: README — document updating properly
The Updating section told you to run update.sh, but never said how to GET it. It
ships inside the patch, so any clone older than v0.4.0 does not have it -- the docs
described a script the reader did not possess. There is now an explicit bootstrap
step, including the fix for the "insufficient permission for adding an object to
repository database" failure that past `sudo git pull`s leave behind.

Rewrote "After a TrueNAS update". It never explained that apply.sh re-applies the
patch at every boot (so you never reinstall), and it soft-pedalled what a failure
costs: "fail-safe" means the BOX stays up, not that your backups keep running. A
[FAIL] providers is a broken backup, and the docs now say that plainly instead of
implying everything degrades gracefully.

Added a repo map -- patch/mw_patch.py and tools/release_notes.py were documented
nowhere -- and fixed `ruff check patch tests`, which skips tools/.

Every command and file path in the README was then verified to exist and run:
all five update.sh flags, the ruff/pytest commands, every file in the repo map,
and the truecloud_nested.py cleanup CLI.

132 tests, ruff and shellcheck -S style clean.
2026-07-13 16:36:28 +00:00
13 changed files with 655 additions and 34 deletions
+1
View File
@@ -13,3 +13,4 @@ __pycache__/
.ruff_cache/ .ruff_cache/
.venv/ .venv/
venv/ venv/
/update_alerts_disabled
+55
View File
@@ -1,5 +1,60 @@
# Changelog # Changelog
## v0.5.0 — 2026-07-13
### Added
- **A TrueNAS alert when an update is available** — the bell in the UI, not a log
line nobody reads. On by default, checked once a day.
`install.sh --no-update-alerts` turns it off.
**It does not nag.** A release whose CHANGELOG contains only a `### Docs`
section changed no code and raises nothing. Anything else raises INFO; a
`### Security` section raises WARNING. The CHANGELOG's own section headings are
the signal, and a security fix anywhere in the range escalates the whole span —
so a docs-only release sitting on top of a security fix still reports as
security, rather than hiding it.
**Why an AlertSource and not `midclt`:** TrueNAS cannot raise an alert from the
CLI. `midclt` exposes only `alert.dismiss`, `alert.list`, `alert.list_categories`,
`alert.list_policies` and `alert.restore` — alert *creation* is internal to
middlewared, and none of its ~60 one-shot classes is generic enough to reuse. So
registering an `AlertSource` is the only way, and it is also the least invasive
thing this patch does: it **adds one file and modifies none**, where the
providers and nested modules both append code to stock middleware files. It is
the native mechanism, and TrueNAS polls it itself — no cron, no systemd timer.
- Fail-safe: every error path returns `None`; it cannot take middlewared down.
- Read-only: `git ls-remote` plus an HTTPS fetch of the CHANGELOG. It never
writes to `.git`, so it cannot leave root-owned objects behind the way a
`git fetch` from middlewared (running as root) would.
- Removed by `uninstall.sh`.
- It only *tells* you; it never updates anything.
## v0.4.2 — 2026-07-13
### Docs
- **The Updating section never said how to *get* `update.sh`.** It ships inside the
patch, so a clone older than v0.4.0 doesn't have it — the docs told you to run a
script you didn't have. There is now an explicit bootstrap step (`git pull &&
bash install.sh`, once), including the fix for the *"insufficient permission for
adding an object to repository database"* failure that past `sudo git pull`s
cause.
- **`After a TrueNAS update` rewritten.** It didn't explain that the patch
re-applies itself at every boot (so you never reinstall), and it didn't say what
each failure actually costs you. "Fail-safe" means *the box stays up* — not that
your backups keep running. A `[FAIL] providers` is a **broken backup**, and the
docs now say so rather than implying everything degrades gracefully.
- Added a repo map. `patch/mw_patch.py` and `tools/release_notes.py` were
documented nowhere.
- `Development` told you to run `ruff check patch tests`, which misses `tools/`.
- Every command and file path in the README is now verified to exist and run.
## v0.4.1 — 2026-07-13 ## v0.4.1 — 2026-07-13
### Fixed ### Fixed
+163 -26
View File
@@ -46,6 +46,23 @@ The patch is two independent modules — **providers** (B2/S3) and **nested**
(snapshots on nested datasets) — and each retires on its own once TrueNAS ships (snapshots on nested datasets) — and each retires on its own once TrueNAS ships
that capability natively. See [Native support](#if-truenas-adds-native-support). that capability natively. See [Native support](#if-truenas-adds-native-support).
## What's in the repo
| Path | What it is |
|---|---|
| `install.sh` | Registers the PREINIT boot hook, applies the patch, restarts middlewared. Also `--enable/--disable-nested-snapshots`. |
| `update.sh` | Fetch a newer **release** and apply it. `--check`, `--rollback`, `--to`, `--main`. |
| `uninstall.sh` | Remove everything: boot hook, patched files, UI bundle, staging mounts. |
| `recover.sh` | Emergency: set the kill switch and restart middlewared against stock files. |
| `patch/apply.sh` | The PREINIT script. Runs at **every boot**; re-applies the patch into a fresh overlay. |
| `patch/mw_patch.py` | The one implementation of apply/revert for the `TRUECLOUD_PATCH` blocks. Used by `apply.sh` *and* `uninstall.sh`. |
| `patch/truecloud_nested.py` | Nested-dataset staging: plan, mount, verify, tear down, sweep snapshots. Also `… cleanup` as a CLI. |
| `patch/patch_ui.py` | Widens the Angular credential dropdown. Refuses to write a bundle whose parens it unbalanced. |
| `patch/create_task.py` | Create TrueCloud tasks with S3/B2 credentials; `verify` the patch state. |
| `patch/alert_source.py` | The TrueNAS alert for "an update is available". Installed into `middlewared/alert/source/`. |
| `patch/wait_restart.sh` | Waits for boot to actually settle before restarting middlewared. |
| `tools/release_notes.py` | Extracts a version's CHANGELOG section; enforces version consistency. Used by CI. |
## Development ## Development
Parts of this project were written with AI assistance (Claude). All of it is Parts of this project were written with AI assistance (Claude). All of it is
@@ -54,14 +71,24 @@ make that review meaningful. Bugs are mine.
```bash ```bash
pip install pytest ruff pip install pytest ruff
ruff check patch tests ruff check patch tests tools
pytest tests pytest tests
``` ```
CI runs shellcheck, `bash -n`, ruff, and pytest on Python 3.11–3.13. The tests CI runs shellcheck, `bash -n`, ruff, and pytest on Python 3.11–3.13. Two checks
include a pass that `compile()`s the `*_BLOCK` strings in `patch/apply.sh` — are worth calling out, because nothing else would catch what they catch:
those are Python source appended into live `middlewared` modules, so a syntax
error there would break the box at boot. - The tests **`compile()` the `*_BLOCK` strings** in `patch/apply.sh`. Those are
Python source appended into live `middlewared` modules — a syntax error there
breaks the box at boot, and they're string literals, so nothing else type-checks
them.
- CI asserts **every script declares the same version**, and that it matches the
newest CHANGELOG entry. `VERSION=` had silently drifted to three different
values across the scripts before anything checked.
Releases are automated: push a `vX.Y.Z` tag and the workflow runs the full suite,
verifies the version matches, and cuts a GitHub release whose body **is** the
matching `CHANGELOG.md` section — one source of truth for release notes.
--- ---
@@ -342,27 +369,115 @@ Refresh your browser. S3 and B2 credentials now appear in the
## Updating ## Updating
```bash ```bash
cd /mnt/tank/truenas-truecloud-patch
bash update.sh # to the newest release, with a confirmation bash update.sh # to the newest release, with a confirmation
bash update.sh --check # show what would happen; change nothing
bash update.sh --rollback # undo the last update
``` ```
It preserves your nested-snapshot opt-in setting, shows you the commits and | | |
release notes you don't have yet, and asks before changing anything. It records |---|---|
the previous revision *before* moving, so `--rollback` works even if `install.sh` | `bash update.sh` | Update to the newest release tag. Shows what's coming, asks first. |
dies halfway. | `bash update.sh --check` | Show what *would* happen. Changes nothing. |
| `bash update.sh --rollback` | Undo the last update. |
| `bash update.sh --to v0.3.5` | Go to a specific tag or commit. |
| `bash update.sh --main` | Track **unreleased** `main`. You're on your own. |
| `bash update.sh --yes` | Skip the confirmation (for a scripted, *attended* run). |
**Run it by hand. Never from cron or a systemd timer.** This patch injects Python Run it as **root** — it calls `install.sh`, which needs to reload middlewared.
into `middlewared` and re-applies itself at every boot, so an unattended pull would
let any bad upstream commit reach your box with no human in the loop and take ### First time: bootstrapping `update.sh`
effect on the next reboot. v0.0.4 shipped exactly such a bug and took every app on
the box down. The manual step *is* the safety gate — if you want convenience, watch `update.sh` ships *inside* the patch, so a clone older than v0.4.0 doesn't have it
the [releases](https://github.com/sudolulo/truenas-truecloud-patch/releases) feed, yet. Bootstrap it once with git:
```bash
cd /mnt/tank/truenas-truecloud-patch
git pull # or: git checkout v0.4.1
bash install.sh
```
Every update after that is just `bash update.sh`.
> If a plain `git pull` fails with *"insufficient permission for adding an object
> to repository database"*, past `sudo git pull`s left root-owned objects in
> `.git`. Fix it once as root: `chown -R <you>:<you> .git`. (`update.sh` repairs
> this automatically from then on.)
### What it does for you
- **Preserves your nested-snapshot opt-in setting** — updating never flips it.
- **Shows the commits and release notes you don't have**, then asks before moving.
- **Records the previous revision *before* checking out**, so `--rollback` works
even if `install.sh` dies halfway through.
- **Refuses to run over a dirty working tree**, rather than merging across
hand-edited or scp'd files and losing them.
- **Detects untracked files that would be clobbered** by the checkout and names
them, instead of dying on a raw git error mid-update.
- **Repairs `.git` ownership** left root-owned by past `sudo git pull`s.
It updates to the newest **release tag**, not `main`. `main` can be mid-refactor;
a tag is the tested artifact, and CI gates every release. Pre-release tags
(`-rc`, `-beta`) are skipped — `--to` them explicitly if you want one.
After updating, the checkout is pinned to a release tag (detached HEAD). That is
what you want for a deployment: plain `git pull` no longer applies, and
`update.sh` is the supported path.
### Why there is no auto-update
**Never put this in cron or a systemd timer.** This patch injects Python into
`middlewared` and re-applies itself at *every boot*, so an unattended pull would
let any bad upstream commit reach your box with no human in the loop — and
detonate on the next reboot. That is not hypothetical: **v0.0.4 shipped exactly
such a bug and took all 54 apps on a box down.**
The manual step *is* the safety gate. If you want convenience, watch the
[releases feed](https://github.com/sudolulo/truenas-truecloud-patch/releases);
don't automate the pull. don't automate the pull.
It updates to the newest **release tag**, not `main` — `main` can be mid-refactor, ## Update alerts
and a tag is the tested artifact. `--main` exists if you want unreleased code, and
says so loudly. When a newer release exists, the patch raises a **TrueNAS alert** (the bell in the
UI) telling you so. It's on by default and checks once a day.
```bash
bash install.sh --no-update-alerts # turn it off
bash install.sh --update-alerts # turn it back on
```
**It will not nag you about a README.** A release whose CHANGELOG contains only a
`### Docs` section changed no code, and raises nothing. Anything that touched the
system raises an INFO alert; a release with a `### Security` section raises a
WARNING. The CHANGELOG's own section headings are the signal, and a security fix
anywhere in the range escalates the whole span — a docs-only release on top of a
security fix still reports as security.
### How it works, and why it's built this way
TrueNAS **cannot raise an alert from the CLI** — `midclt` exposes only
`alert.dismiss`, `alert.list`, `alert.list_categories`, `alert.list_policies` and
`alert.restore`. Alert *creation* is internal to middlewared, and none of its ~60
one-shot alert classes is generic enough to reuse. So the only way to get a real
alert is to register an `AlertSource`, which is what `patch/alert_source.py` does.
That is also the **least invasive** thing this patch does:
| | |
|---|---|
| providers module | **modifies** stock files (appends code to `b2.py`, `restic.py`) |
| nested module | **modifies** stock files (3 middleware modules) |
| **update alert** | **adds one file. Modifies nothing.** |
It's the native mechanism — the same one every built-in TrueNAS alert uses — and
TrueNAS polls it itself, so there is no cron job and no systemd timer.
- **Fail-safe.** Every error path returns `None`. It cannot take middlewared down.
- **Read-only.** `git ls-remote` plus an HTTPS fetch of the CHANGELOG. It never
writes to `.git`, so it cannot leave root-owned objects behind the way a
`git fetch` from middlewared (which runs as root) would.
- **Removed by `uninstall.sh`.**
It only *tells* you. It never updates anything — see
[Why there is no auto-update](#why-there-is-no-auto-update).
## Creating a task via CLI ## Creating a task via CLI
@@ -460,12 +575,34 @@ tail -20 /mnt/tank/truenas-truecloud-patch/apply.log
## After a TrueNAS update ## After a TrueNAS update
1. Check the log: `cat /mnt/tank/truenas-truecloud-patch/apply.log | tail -30` A TrueNAS update replaces `/usr/` wholesale, wiping the patch. You do **not** need
2. If you see "WARNING: … pattern not found", the UI patch needs updating. to reinstall: `patch/apply.sh` runs at every boot and re-applies itself from your
[Open an issue](https://github.com/sudolulo/truenas-truecloud-patch/issues) clone. But it targets internal APIs with no stability contract, so an update *can*
with your TrueNAS version number. break it — and the failure is quiet by design (middlewared starts fine; the patch
3. The backend patch (B2 support + URL fix) is more stable — check that a just doesn't).
B2 backup job still completes successfully after any update.
**Check the log after any TrueNAS update:**
```bash
tail -30 /mnt/tank/truenas-truecloud-patch/apply.log
python3 /mnt/tank/truenas-truecloud-patch/patch/create_task.py verify
```
| What you see | What it means |
|---|---|
| `[OK] providers`, `[OK]`/`[SKIP] nested_snapshots` | Fine. Nothing to do. |
| `WARNING: … pattern not found` (UI) | The Angular bundle changed. The UI dropdown reverts to Storj-only, but **backups keep working** — create tasks with `create_task.py` meanwhile, and [open an issue](https://github.com/sudolulo/truenas-truecloud-patch/issues) with your TrueNAS version. |
| `[FAIL] providers` | **Your B2/S3 backups will not run.** middlewared is fine, but the credential/URL handling is gone. Open an issue with your version. |
| `[FAIL] nested_snapshots` | The stock guard is back, so tasks with `snapshot = true` on a nested dataset will fail validation. Turn the option off on those tasks until it's fixed. |
"Fail-safe" means *the box stays up* — not that your backups keep running. A
`[FAIL] providers` is a broken backup, so check the log rather than assume.
Then update the patch itself if a newer release fixes it:
```bash
bash /mnt/tank/truenas-truecloud-patch/update.sh
```
--- ---
+35 -1
View File
@@ -18,7 +18,7 @@
set -euo pipefail set -euo pipefail
VERSION="0.4.1" VERSION="0.5.0"
# The directory containing install.sh is the permanent install location. # The directory containing install.sh is the permanent install location.
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)" PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
@@ -31,6 +31,8 @@ _NESTED_MARKER="$PATCH_DIR/nested_snapshots_enabled"
# `git pull`) must never flip it on or off by itself: with neither flag given, # `git pull`) must never flip it on or off by itself: with neither flag given,
# whatever was chosen previously is preserved. # whatever was chosen previously is preserved.
_nested_choice="" _nested_choice=""
_alert_choice=""
_ALERT_MARKER="$PATCH_DIR/update_alerts_disabled"
usage() { usage() {
cat <<USAGE cat <<USAGE
@@ -42,6 +44,8 @@ Options:
Stock TrueNAS refuses this; see README. Off by Stock TrueNAS refuses this; see README. Off by
default because it changes how backups read data. default because it changes how backups read data.
--disable-nested-snapshots Turn it back off; the stock guard is restored. --disable-nested-snapshots Turn it back off; the stock guard is restored.
--no-update-alerts Do not raise a TrueNAS alert when an update exists.
--update-alerts Re-enable those alerts (they are on by default).
-h, --help Show this help. -h, --help Show this help.
With neither flag, the current setting is left unchanged. With neither flag, the current setting is left unchanged.
@@ -52,6 +56,8 @@ while [ $# -gt 0 ]; do
case "$1" in case "$1" in
--enable-nested-snapshots) _nested_choice="on" ;; --enable-nested-snapshots) _nested_choice="on" ;;
--disable-nested-snapshots) _nested_choice="off" ;; --disable-nested-snapshots) _nested_choice="off" ;;
--no-update-alerts) _alert_choice="off" ;;
--update-alerts) _alert_choice="on" ;;
-h|--help) usage; exit 0 ;; -h|--help) usage; exit 0 ;;
*) *)
echo "ERROR: unknown option: $1" >&2 echo "ERROR: unknown option: $1" >&2
@@ -192,6 +198,34 @@ case "$_nested_choice" in
esac esac
echo "" echo ""
# ── Update alerts (on by default) ─────────────────────────────────────────────
# TrueNAS cannot raise an alert from the CLI (midclt exposes only dismiss/list/
# restore), so this installs an AlertSource into middlewared/alert/source/ — the
# same mechanism every built-in TrueNAS alert uses. It ADDS a file and modifies
# none, which makes it the least invasive thing this patch does.
#
# It only alerts for releases that changed something: a documentation-only release
# raises nothing.
case "$_alert_choice" in
off)
touch "$_ALERT_MARKER"
echo "Update alerts: DISABLED (apply.sh will remove the alert source)."
;;
on)
rm -f "$_ALERT_MARKER"
echo "Update alerts: enabled."
;;
*)
if [ -f "$_ALERT_MARKER" ]; then
echo "Update alerts: disabled (unchanged)."
else
echo "Update alerts: enabled (checks daily; docs-only releases are ignored)."
fi
;;
esac
echo ""
# ── Apply now ───────────────────────────────────────────────────────────────── # ── Apply now ─────────────────────────────────────────────────────────────────
echo "Applying patches ..." echo "Applying patches ..."
+211
View File
@@ -0,0 +1,211 @@
"""TrueNAS alert: a truecloud-patch update is available.
Installed by patch/apply.sh into middlewared/alert/source/, where middlewared
discovers and polls it natively — no cron job, no systemd timer.
@PATCH_DIR@ is substituted at install time.
Two rules govern this file:
1. **It must never break middlewared.** It runs inside the alert framework on a
timer. Every failure path returns None (no alert) rather than raising.
2. **It must not nag.** A release whose CHANGELOG only has a "### Docs" section
changed no code, and nobody wants an alert because a README was reworded. The
CHANGELOG's own section headings are the signal — see tools/release_notes.py.
It also never writes to the repository. `git ls-remote` is read-only and the
CHANGELOG is fetched over HTTPS, so this cannot leave root-owned objects in .git
the way a `git fetch` from middlewared (running as root) would.
"""
import datetime
import logging
import os
import re
import subprocess
import sys
import urllib.request
from middlewared.alert.base import (
Alert,
AlertCategory,
AlertClass,
AlertLevel,
ThreadedAlertSource,
)
from middlewared.alert.schedule import IntervalSchedule
logger = logging.getLogger(__name__)
PATCH_DIR = "@PATCH_DIR@"
DISABLED_MARKER = os.path.join(PATCH_DIR, "update_alerts_disabled")
_TAG_RE = re.compile(r"^v\d+\.\d+\.\d+$")
_VERSION_RE = re.compile(r'^VERSION="([^"]+)"', re.M)
_GITHUB_RE = re.compile(r"github\.com[:/]([^/]+)/([^/.]+)")
_TIMEOUT = 20
class TrueCloudPatchUpdateAlertClass(AlertClass):
category = AlertCategory.SYSTEM
level = AlertLevel.INFO
title = "truecloud-patch update available"
text = (
"truecloud-patch %(current)s is installed; %(latest)s is available.%(summary)s "
"Update with: bash %(dir)s/update.sh"
)
class TrueCloudPatchSecurityUpdateAlertClass(AlertClass):
category = AlertCategory.SYSTEM
level = AlertLevel.WARNING
title = "truecloud-patch security update available"
text = (
"truecloud-patch %(current)s is installed; %(latest)s contains a SECURITY "
"fix.%(summary)s Update with: bash %(dir)s/update.sh"
)
class TrueCloudPatchUpdateAlertSource(ThreadedAlertSource):
schedule = IntervalSchedule(datetime.timedelta(hours=24))
run_on_backup_node = False
def check_sync(self):
try:
return self._check()
except Exception:
# An alert source must never take middlewared down with it.
logger.debug("truecloud-patch update check failed", exc_info=True)
return None
# ── internals ────────────────────────────────────────────────────────────
def _git(self, *args):
return subprocess.run(
["git", "-C", PATCH_DIR, *args],
capture_output=True, text=True, timeout=_TIMEOUT, check=True,
).stdout
def _check(self):
if os.path.exists(DISABLED_MARKER):
return None
if not os.path.isdir(os.path.join(PATCH_DIR, ".git")):
return None
current = self._installed_version()
if not current:
return None
latest = self._latest_release_tag()
if not latest:
return None
sys.path.insert(0, os.path.join(PATCH_DIR, "tools"))
try:
from release_notes import significance, version_tuple
finally:
sys.path.pop(0)
if version_tuple(latest) <= version_tuple(current):
return None
level, versions, summary = self._classify(
current, latest, significance
)
# Documentation-only releases are not worth an alert. This is the whole
# point: nobody should get a notification because a README was reworded.
if level == "docs":
logger.debug(
"truecloud-patch %s -> %s is documentation-only; not alerting",
current, latest,
)
return None
args = {
"current": f"v{current}",
"latest": latest,
"summary": summary,
"dir": PATCH_DIR,
}
klass = (
TrueCloudPatchSecurityUpdateAlertClass if level == "security"
else TrueCloudPatchUpdateAlertClass
)
return Alert(klass, args, key=[current, latest])
def _installed_version(self):
"""The version of the patch actually checked out here."""
try:
with open(os.path.join(PATCH_DIR, "patch", "apply.sh"), encoding="utf-8") as fh:
m = _VERSION_RE.search(fh.read())
except OSError:
return None
return m.group(1) if m else None
def _latest_release_tag(self):
"""Newest plain vX.Y.Z tag on the remote. Read-only: no .git writes.
Pre-release tags (-rc, -beta) are excluded: git's version sort ranks
v0.5.0-rc1 above v0.5.0, so including them would advertise a release
candidate as the latest stable.
"""
try:
out = self._git("ls-remote", "--tags", "--refs", "origin")
except Exception:
return None
tags = []
for line in out.splitlines():
parts = line.split("refs/tags/")
if len(parts) == 2 and _TAG_RE.match(parts[1].strip()):
tags.append(parts[1].strip())
if not tags:
return None
return max(tags, key=lambda t: tuple(int(x) for x in t.lstrip("v").split(".")))
def _classify(self, current, latest, significance):
"""(level, versions, one-line summary). Falls back to alerting."""
text = self._remote_changelog(latest)
if text is None:
# Cannot tell whether it matters. Alert rather than risk hiding a
# security fix -- but say that we could not tell.
return "notable", [], " (could not read the changelog)"
level, versions, headings = significance(text, current, latest)
if level == "docs":
return level, versions, ""
seen, ordered = set(), []
for h in headings:
if h not in seen:
seen.add(h)
ordered.append(h.capitalize())
detail = ", ".join(ordered)
return level, versions, f" Changes: {detail}." if detail else ""
def _remote_changelog(self, tag):
"""CHANGELOG.md at `tag`, over HTTPS. None if it cannot be read."""
try:
remote = self._git("remote", "get-url", "origin").strip()
except Exception:
return None
m = _GITHUB_RE.search(remote)
if not m:
return None # not a GitHub remote; skip classification
url = (
f"https://raw.githubusercontent.com/{m.group(1)}/{m.group(2)}/"
f"{tag}/CHANGELOG.md"
)
try:
with urllib.request.urlopen(url, timeout=_TIMEOUT) as resp: # noqa: S310
if resp.status != 200:
return None
return resp.read().decode("utf-8", "replace")
except Exception:
return None
+40 -1
View File
@@ -32,7 +32,7 @@
# Derive PATCH_DIR from this script's location (parent of the patch/ directory). # Derive PATCH_DIR from this script's location (parent of the patch/ directory).
PATCH_DIR="$(cd "$(dirname "$0")/.." && pwd)" PATCH_DIR="$(cd "$(dirname "$0")/.." && pwd)"
LOG="$PATCH_DIR/apply.log" LOG="$PATCH_DIR/apply.log"
VERSION="0.4.1" VERSION="0.5.0"
# Rotate log at 512 KB to avoid unbounded growth on a system volume. # Rotate log at 512 KB to avoid unbounded growth on a system volume.
# Keep two prior generations (.1 and .2) so the last three boots are always available. # Keep two prior generations (.1 and .2) so the last three boots are always available.
@@ -578,6 +578,40 @@ else:
# that is inactive (superseded, or opt-in and off) is ok -- reporting a disabled # that is inactive (superseded, or opt-in and off) is ok -- reporting a disabled
# opt-in feature as FAIL would make `create_task.py verify` fail on a default # opt-in feature as FAIL would make `create_task.py verify` fail on a default
# install. `active` says whether the module is doing anything. # install. `active` says whether the module is doing anything.
# ── update-available alert ────────────────────────────────────────────────────
# Dropped into middlewared/alert/source/, where middlewared discovers and polls it
# natively — no cron, no timer. It only raises an alert for releases that actually
# changed something: a docs-only release is ignored (see tools/release_notes.py).
alert_ok = False
alert_detail = ''
_patch_dir = os.path.dirname(os.path.dirname(nested_src))
alert_src = os.path.join(os.path.dirname(nested_src), 'alert_source.py')
alert_dst = os.path.join(mw_dir, 'alert', 'source', 'truecloud_patch_update.py')
if os.path.exists(os.path.join(_patch_dir, 'update_alerts_disabled')):
alert_detail = 'disabled (update_alerts_disabled)'
try:
os.unlink(alert_dst)
print('OK: Removed update alert (disabled).')
except OSError:
pass
elif not os.path.exists(alert_src):
alert_detail = 'alert_source.py not found'
print(f'WARNING: {alert_src} missing — no update alert.')
else:
try:
with open(alert_src, encoding='utf-8') as fh:
_body = fh.read()
# PATCH_DIR is baked in: the alert source must find the repo it belongs to.
with open(alert_dst, 'w', encoding='utf-8') as fh:
fh.write(_body.replace('@PATCH_DIR@', _patch_dir))
alert_ok = True
alert_detail = 'update alert installed'
print(f'OK: Installed update alert → {alert_dst}')
except Exception as e:
alert_detail = f'not applied: {e}'
print(f'WARNING: could not install update alert: {e}')
patches = { patches = {
'providers': { 'providers': {
'ok': (not providers_needed) or bool(b2_ok and restic_ok), 'ok': (not providers_needed) or bool(b2_ok and restic_ok),
@@ -589,6 +623,11 @@ patches = {
'active': nested_needed, 'active': nested_needed,
'detail': nested_detail, 'detail': nested_detail,
}, },
'update_alert': {
'ok': True, # never a failure: it is a convenience, not a patch
'active': alert_ok,
'detail': alert_detail,
},
} }
payload = {'patched_at': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), 'patches': patches} payload = {'patched_at': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), 'patches': patches}
tmp = status_path + '.tmp' tmp = status_path + '.tmp'
+1 -1
View File
@@ -52,7 +52,7 @@ import subprocess
import sys import sys
import time import time
__version__ = "0.4.1" __version__ = "0.5.0"
_PATCH_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) _PATCH_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
_STATUS_FILE = os.path.join(_PATCH_DIR, "hook_status.json") _STATUS_FILE = os.path.join(_PATCH_DIR, "hook_status.json")
+12 -2
View File
@@ -37,6 +37,10 @@ NESTED_RELPATHS = [
#: The importable module the nested blocks depend on. #: The importable module the nested blocks depend on.
NESTED_MODULE = ("plugins", "cloud", "_truecloud_nested.py") NESTED_MODULE = ("plugins", "cloud", "_truecloud_nested.py")
#: The update-available alert source. Not a "patch" (it appends nothing to a stock
#: file), but it is a file we install into middlewared and must therefore remove.
ALERT_MODULE = ("alert", "source", "truecloud_patch_update.py")
def patch_file(path, block): def patch_file(path, block):
"""Append `block`, replacing any block we appended before. Idempotent.""" """Append `block`, replacing any block we appended before. Idempotent."""
@@ -101,8 +105,14 @@ def revert_nested(mw_dir):
def revert_all(mw_dir): def revert_all(mw_dir):
"""Undo every patch this project applies.""" """Undo every patch this project applies, and remove every file it installs."""
return revert(mw_dir, NESTED_RELPATHS + PROVIDER_RELPATHS, NESTED_MODULE) reverted = revert(mw_dir, NESTED_RELPATHS + PROVIDER_RELPATHS, NESTED_MODULE)
try:
os.unlink(os.path.join(mw_dir, *ALERT_MODULE))
reverted.append(ALERT_MODULE[-1])
except OSError:
pass
return reverted
def find_middlewared_dir(): def find_middlewared_dir():
+1 -1
View File
@@ -17,7 +17,7 @@
# bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh # bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh
# systemctl restart middlewared # systemctl restart middlewared
VERSION="0.4.1" VERSION="0.5.0"
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)" PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
+80
View File
@@ -21,6 +21,8 @@ from release_notes import ( # noqa: E402
extract_notes, extract_notes,
normalise, normalise,
script_versions, script_versions,
significance,
version_tuple,
) )
REPO = os.path.join(os.path.dirname(__file__), "..") REPO = os.path.join(os.path.dirname(__file__), "..")
@@ -120,3 +122,81 @@ class TestCheckCatchesMistakes:
def test_reports_a_missing_changelog_section(self): def test_reports_a_missing_changelog_section(self):
problems = check("v9.9.9", REPO) problems = check("v9.9.9", REPO)
assert any("no section" in p for p in problems) assert any("no section" in p for p in problems)
class TestSignificance:
"""Drives the TrueNAS update alert: what is worth bothering a human about.
The rule: a release whose CHANGELOG only has a "### Docs" section changed no
code, and nobody should get an alert because a README was reworded.
"""
TEXT = """\
# Changelog
## v0.4.2 — 2026-07-13
### Docs
- reworded the README
## v0.4.1 — 2026-07-13
### Fixed
- a real bug
## v0.4.0 — 2026-07-13
### Added
- a feature
## v0.3.3 — 2026-07-13
### Security
- keep a password out of argv
## v0.3.2 — 2026-07-13
### Fixed
- something
"""
def test_docs_only_release_does_not_alert(self):
level, versions, _ = significance(self.TEXT, "0.4.1", "0.4.2")
assert level == "docs"
assert versions == ["0.4.2"]
def test_a_real_fix_alerts(self):
level, _v, _h = significance(self.TEXT, "0.4.0", "0.4.1")
assert level == "notable"
def test_security_in_range_escalates(self):
level, _v, _h = significance(self.TEXT, "0.3.2", "0.3.3")
assert level == "security"
def test_security_wins_even_when_the_newest_release_is_docs_only(self):
# A docs-only v0.4.2 sitting on top of a security-fixing v0.3.3 must still
# be reported as security — classify the whole span, not just the tip.
level, versions, _ = significance(self.TEXT, "0.3.2", "0.4.2")
assert level == "security"
assert set(versions) == {"0.3.3", "0.4.0", "0.4.1", "0.4.2"}
def test_same_version_is_never_notable(self):
level, versions, _ = significance(self.TEXT, "0.4.2", "0.4.2")
assert level == "docs"
assert versions == []
def test_range_is_exclusive_of_current_inclusive_of_latest(self):
_l, versions, _h = significance(self.TEXT, "0.4.0", "0.4.2")
assert "0.4.0" not in versions
assert "0.4.2" in versions
def test_version_tuple_orders_correctly(self):
assert version_tuple("v0.10.0") > version_tuple("v0.9.9")
assert version_tuple("0.4.2") > version_tuple("0.4.1")
# Pre-release suffixes are dropped, not ranked above the release.
assert version_tuple("v0.5.0-rc1") == version_tuple("v0.5.0")
+54
View File
@@ -88,6 +88,60 @@ def extract_notes(text: str, version: str) -> str:
return "\n".join(lines[start:end]).strip() return "\n".join(lines[start:end]).strip()
# ── significance ──────────────────────────────────────────────────────────────
# Used by the TrueNAS update alert to decide whether a release is worth bothering
# anyone about. The CHANGELOG's own section headings are the signal: a release that
# only has "### Docs" changed no code, and nobody should get an alert for a README.
_SECTION_RE = re.compile(r"^###\s+(.+?)\s*$", re.M)
#: Headings that mean "nothing about the running system changed".
QUIET_SECTIONS = {"docs", "documentation"}
def version_tuple(v: str) -> tuple:
"""Sortable version. Pre-release suffixes are dropped, not ranked."""
return tuple(int(x) for x in normalise(v).split("-")[0].split("."))
def section_headings(body: str) -> list[str]:
"""The `### ...` headings inside one version's body, lowercased."""
return [h.strip().lower() for h in _SECTION_RE.findall(body)]
def significance(text: str, current: str, latest: str):
"""How much does upgrading `current` -> `latest` actually matter?
Returns ``(level, versions, headings)`` where level is one of:
"security" a release in the range has a Security section -> alert loudly
"notable" something about the system changed -> alert quietly
"docs" only documentation changed -> DO NOT alert
Considers every release in the range, not just the newest: a docs-only v0.4.2
on top of a security-fixing v0.4.1 must still be reported as security.
"""
cur, lat = version_tuple(current), version_tuple(latest)
versions = [
v for v in changelog_versions(text)
if cur < version_tuple(v) <= lat
]
headings = []
for v in versions:
try:
headings.extend(section_headings(extract_notes(text, v)))
except KeyError:
continue
if any(h.startswith("security") for h in headings):
return "security", versions, headings
if [h for h in headings if h not in QUIET_SECTIONS]:
return "notable", versions, headings
return "docs", versions, headings
def check(version: str, root: str = ROOT) -> list[str]: def check(version: str, root: str = ROOT) -> list[str]:
"""Every reason this version is not releasable. Empty list means it is.""" """Every reason this version is not releasable. Empty list means it is."""
want = normalise(version) want = normalise(version)
+1 -1
View File
@@ -3,7 +3,7 @@
set -euo pipefail set -euo pipefail
VERSION="0.4.1" VERSION="0.5.0"
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)" PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
_HOOK_COMMENT='TrueCloud provider patch (S3/B2)' _HOOK_COMMENT='TrueCloud provider patch (S3/B2)'
+1 -1
View File
@@ -19,7 +19,7 @@
set -euo pipefail set -euo pipefail
VERSION="0.4.1" VERSION="0.5.0"
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)" PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
_PREV_FILE="$PATCH_DIR/.update_previous" _PREV_FILE="$PATCH_DIR/.update_previous"