diff --git a/.gitignore b/.gitignore index daf6914..4072be9 100644 --- a/.gitignore +++ b/.gitignore @@ -13,3 +13,4 @@ __pycache__/ .ruff_cache/ .venv/ venv/ +/update_alerts_disabled diff --git a/CHANGELOG.md b/CHANGELOG.md index a4e1645..89a8fbe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,36 @@ # Changelog +## v0.5.0 — 2026-07-13 + +### Added + +- **A TrueNAS alert when an update is available** — the bell in the UI, not a log + line nobody reads. On by default, checked once a day. + `install.sh --no-update-alerts` turns it off. + + **It does not nag.** A release whose CHANGELOG contains only a `### Docs` + section changed no code and raises nothing. Anything else raises INFO; a + `### Security` section raises WARNING. The CHANGELOG's own section headings are + the signal, and a security fix anywhere in the range escalates the whole span — + so a docs-only release sitting on top of a security fix still reports as + security, rather than hiding it. + + **Why an AlertSource and not `midclt`:** TrueNAS cannot raise an alert from the + CLI. `midclt` exposes only `alert.dismiss`, `alert.list`, `alert.list_categories`, + `alert.list_policies` and `alert.restore` — alert *creation* is internal to + middlewared, and none of its ~60 one-shot classes is generic enough to reuse. So + registering an `AlertSource` is the only way, and it is also the least invasive + thing this patch does: it **adds one file and modifies none**, where the + providers and nested modules both append code to stock middleware files. It is + the native mechanism, and TrueNAS polls it itself — no cron, no systemd timer. + + - Fail-safe: every error path returns `None`; it cannot take middlewared down. + - Read-only: `git ls-remote` plus an HTTPS fetch of the CHANGELOG. It never + writes to `.git`, so it cannot leave root-owned objects behind the way a + `git fetch` from middlewared (running as root) would. + - Removed by `uninstall.sh`. + - It only *tells* you; it never updates anything. + ## v0.4.2 — 2026-07-13 ### Docs diff --git a/README.md b/README.md index f699388..68613bf 100644 --- a/README.md +++ b/README.md @@ -59,6 +59,7 @@ that capability natively. See [Native support](#if-truenas-adds-native-support). | `patch/truecloud_nested.py` | Nested-dataset staging: plan, mount, verify, tear down, sweep snapshots. Also `… cleanup` as a CLI. | | `patch/patch_ui.py` | Widens the Angular credential dropdown. Refuses to write a bundle whose parens it unbalanced. | | `patch/create_task.py` | Create TrueCloud tasks with S3/B2 credentials; `verify` the patch state. | +| `patch/alert_source.py` | The TrueNAS alert for "an update is available". Installed into `middlewared/alert/source/`. | | `patch/wait_restart.sh` | Waits for boot to actually settle before restarting middlewared. | | `tools/release_notes.py` | Extracts a version's CHANGELOG section; enforces version consistency. Used by CI. | @@ -433,6 +434,51 @@ The manual step *is* the safety gate. If you want convenience, watch the [releases feed](https://github.com/sudolulo/truenas-truecloud-patch/releases); don't automate the pull. +## Update alerts + +When a newer release exists, the patch raises a **TrueNAS alert** (the bell in the +UI) telling you so. It's on by default and checks once a day. + +```bash +bash install.sh --no-update-alerts # turn it off +bash install.sh --update-alerts # turn it back on +``` + +**It will not nag you about a README.** A release whose CHANGELOG contains only a +`### Docs` section changed no code, and raises nothing. Anything that touched the +system raises an INFO alert; a release with a `### Security` section raises a +WARNING. The CHANGELOG's own section headings are the signal, and a security fix +anywhere in the range escalates the whole span — a docs-only release on top of a +security fix still reports as security. + +### How it works, and why it's built this way + +TrueNAS **cannot raise an alert from the CLI** — `midclt` exposes only +`alert.dismiss`, `alert.list`, `alert.list_categories`, `alert.list_policies` and +`alert.restore`. Alert *creation* is internal to middlewared, and none of its ~60 +one-shot alert classes is generic enough to reuse. So the only way to get a real +alert is to register an `AlertSource`, which is what `patch/alert_source.py` does. + +That is also the **least invasive** thing this patch does: + +| | | +|---|---| +| providers module | **modifies** stock files (appends code to `b2.py`, `restic.py`) | +| nested module | **modifies** stock files (3 middleware modules) | +| **update alert** | **adds one file. Modifies nothing.** | + +It's the native mechanism — the same one every built-in TrueNAS alert uses — and +TrueNAS polls it itself, so there is no cron job and no systemd timer. + +- **Fail-safe.** Every error path returns `None`. It cannot take middlewared down. +- **Read-only.** `git ls-remote` plus an HTTPS fetch of the CHANGELOG. It never + writes to `.git`, so it cannot leave root-owned objects behind the way a + `git fetch` from middlewared (which runs as root) would. +- **Removed by `uninstall.sh`.** + +It only *tells* you. It never updates anything — see +[Why there is no auto-update](#why-there-is-no-auto-update). + ## Creating a task via CLI If the UI still shows only Storj after refreshing (e.g. the JS bundle pattern diff --git a/install.sh b/install.sh index 59072ed..391682b 100755 --- a/install.sh +++ b/install.sh @@ -18,7 +18,7 @@ set -euo pipefail -VERSION="0.4.2" +VERSION="0.5.0" # The directory containing install.sh is the permanent install location. PATCH_DIR="$(cd "$(dirname "$0")" && pwd)" @@ -31,6 +31,8 @@ _NESTED_MARKER="$PATCH_DIR/nested_snapshots_enabled" # `git pull`) must never flip it on or off by itself: with neither flag given, # whatever was chosen previously is preserved. _nested_choice="" +_alert_choice="" +_ALERT_MARKER="$PATCH_DIR/update_alerts_disabled" usage() { cat <&2 @@ -192,6 +198,34 @@ case "$_nested_choice" in esac echo "" +# ── Update alerts (on by default) ───────────────────────────────────────────── +# TrueNAS cannot raise an alert from the CLI (midclt exposes only dismiss/list/ +# restore), so this installs an AlertSource into middlewared/alert/source/ — the +# same mechanism every built-in TrueNAS alert uses. It ADDS a file and modifies +# none, which makes it the least invasive thing this patch does. +# +# It only alerts for releases that changed something: a documentation-only release +# raises nothing. + +case "$_alert_choice" in + off) + touch "$_ALERT_MARKER" + echo "Update alerts: DISABLED (apply.sh will remove the alert source)." + ;; + on) + rm -f "$_ALERT_MARKER" + echo "Update alerts: enabled." + ;; + *) + if [ -f "$_ALERT_MARKER" ]; then + echo "Update alerts: disabled (unchanged)." + else + echo "Update alerts: enabled (checks daily; docs-only releases are ignored)." + fi + ;; +esac +echo "" + # ── Apply now ───────────────────────────────────────────────────────────────── echo "Applying patches ..." diff --git a/patch/alert_source.py b/patch/alert_source.py new file mode 100644 index 0000000..ab9b4ed --- /dev/null +++ b/patch/alert_source.py @@ -0,0 +1,211 @@ +"""TrueNAS alert: a truecloud-patch update is available. + +Installed by patch/apply.sh into middlewared/alert/source/, where middlewared +discovers and polls it natively — no cron job, no systemd timer. + +@PATCH_DIR@ is substituted at install time. + +Two rules govern this file: + +1. **It must never break middlewared.** It runs inside the alert framework on a + timer. Every failure path returns None (no alert) rather than raising. + +2. **It must not nag.** A release whose CHANGELOG only has a "### Docs" section + changed no code, and nobody wants an alert because a README was reworded. The + CHANGELOG's own section headings are the signal — see tools/release_notes.py. + +It also never writes to the repository. `git ls-remote` is read-only and the +CHANGELOG is fetched over HTTPS, so this cannot leave root-owned objects in .git +the way a `git fetch` from middlewared (running as root) would. +""" + +import datetime +import logging +import os +import re +import subprocess +import sys +import urllib.request + +from middlewared.alert.base import ( + Alert, + AlertCategory, + AlertClass, + AlertLevel, + ThreadedAlertSource, +) +from middlewared.alert.schedule import IntervalSchedule + +logger = logging.getLogger(__name__) + +PATCH_DIR = "@PATCH_DIR@" +DISABLED_MARKER = os.path.join(PATCH_DIR, "update_alerts_disabled") + +_TAG_RE = re.compile(r"^v\d+\.\d+\.\d+$") +_VERSION_RE = re.compile(r'^VERSION="([^"]+)"', re.M) +_GITHUB_RE = re.compile(r"github\.com[:/]([^/]+)/([^/.]+)") + +_TIMEOUT = 20 + + +class TrueCloudPatchUpdateAlertClass(AlertClass): + category = AlertCategory.SYSTEM + level = AlertLevel.INFO + title = "truecloud-patch update available" + text = ( + "truecloud-patch %(current)s is installed; %(latest)s is available.%(summary)s " + "Update with: bash %(dir)s/update.sh" + ) + + +class TrueCloudPatchSecurityUpdateAlertClass(AlertClass): + category = AlertCategory.SYSTEM + level = AlertLevel.WARNING + title = "truecloud-patch security update available" + text = ( + "truecloud-patch %(current)s is installed; %(latest)s contains a SECURITY " + "fix.%(summary)s Update with: bash %(dir)s/update.sh" + ) + + +class TrueCloudPatchUpdateAlertSource(ThreadedAlertSource): + schedule = IntervalSchedule(datetime.timedelta(hours=24)) + run_on_backup_node = False + + def check_sync(self): + try: + return self._check() + except Exception: + # An alert source must never take middlewared down with it. + logger.debug("truecloud-patch update check failed", exc_info=True) + return None + + # ── internals ──────────────────────────────────────────────────────────── + + def _git(self, *args): + return subprocess.run( + ["git", "-C", PATCH_DIR, *args], + capture_output=True, text=True, timeout=_TIMEOUT, check=True, + ).stdout + + def _check(self): + if os.path.exists(DISABLED_MARKER): + return None + if not os.path.isdir(os.path.join(PATCH_DIR, ".git")): + return None + + current = self._installed_version() + if not current: + return None + + latest = self._latest_release_tag() + if not latest: + return None + + sys.path.insert(0, os.path.join(PATCH_DIR, "tools")) + try: + from release_notes import significance, version_tuple + finally: + sys.path.pop(0) + + if version_tuple(latest) <= version_tuple(current): + return None + + level, versions, summary = self._classify( + current, latest, significance + ) + + # Documentation-only releases are not worth an alert. This is the whole + # point: nobody should get a notification because a README was reworded. + if level == "docs": + logger.debug( + "truecloud-patch %s -> %s is documentation-only; not alerting", + current, latest, + ) + return None + + args = { + "current": f"v{current}", + "latest": latest, + "summary": summary, + "dir": PATCH_DIR, + } + klass = ( + TrueCloudPatchSecurityUpdateAlertClass if level == "security" + else TrueCloudPatchUpdateAlertClass + ) + return Alert(klass, args, key=[current, latest]) + + def _installed_version(self): + """The version of the patch actually checked out here.""" + try: + with open(os.path.join(PATCH_DIR, "patch", "apply.sh"), encoding="utf-8") as fh: + m = _VERSION_RE.search(fh.read()) + except OSError: + return None + return m.group(1) if m else None + + def _latest_release_tag(self): + """Newest plain vX.Y.Z tag on the remote. Read-only: no .git writes. + + Pre-release tags (-rc, -beta) are excluded: git's version sort ranks + v0.5.0-rc1 above v0.5.0, so including them would advertise a release + candidate as the latest stable. + """ + try: + out = self._git("ls-remote", "--tags", "--refs", "origin") + except Exception: + return None + + tags = [] + for line in out.splitlines(): + parts = line.split("refs/tags/") + if len(parts) == 2 and _TAG_RE.match(parts[1].strip()): + tags.append(parts[1].strip()) + if not tags: + return None + + return max(tags, key=lambda t: tuple(int(x) for x in t.lstrip("v").split("."))) + + def _classify(self, current, latest, significance): + """(level, versions, one-line summary). Falls back to alerting.""" + text = self._remote_changelog(latest) + if text is None: + # Cannot tell whether it matters. Alert rather than risk hiding a + # security fix -- but say that we could not tell. + return "notable", [], " (could not read the changelog)" + + level, versions, headings = significance(text, current, latest) + if level == "docs": + return level, versions, "" + + seen, ordered = set(), [] + for h in headings: + if h not in seen: + seen.add(h) + ordered.append(h.capitalize()) + detail = ", ".join(ordered) + return level, versions, f" Changes: {detail}." if detail else "" + + def _remote_changelog(self, tag): + """CHANGELOG.md at `tag`, over HTTPS. None if it cannot be read.""" + try: + remote = self._git("remote", "get-url", "origin").strip() + except Exception: + return None + + m = _GITHUB_RE.search(remote) + if not m: + return None # not a GitHub remote; skip classification + + url = ( + f"https://raw.githubusercontent.com/{m.group(1)}/{m.group(2)}/" + f"{tag}/CHANGELOG.md" + ) + try: + with urllib.request.urlopen(url, timeout=_TIMEOUT) as resp: # noqa: S310 + if resp.status != 200: + return None + return resp.read().decode("utf-8", "replace") + except Exception: + return None diff --git a/patch/apply.sh b/patch/apply.sh index cad048c..a042eec 100755 --- a/patch/apply.sh +++ b/patch/apply.sh @@ -32,7 +32,7 @@ # Derive PATCH_DIR from this script's location (parent of the patch/ directory). PATCH_DIR="$(cd "$(dirname "$0")/.." && pwd)" LOG="$PATCH_DIR/apply.log" -VERSION="0.4.2" +VERSION="0.5.0" # Rotate log at 512 KB to avoid unbounded growth on a system volume. # Keep two prior generations (.1 and .2) so the last three boots are always available. @@ -578,6 +578,40 @@ else: # that is inactive (superseded, or opt-in and off) is ok -- reporting a disabled # opt-in feature as FAIL would make `create_task.py verify` fail on a default # install. `active` says whether the module is doing anything. +# ── update-available alert ──────────────────────────────────────────────────── +# Dropped into middlewared/alert/source/, where middlewared discovers and polls it +# natively — no cron, no timer. It only raises an alert for releases that actually +# changed something: a docs-only release is ignored (see tools/release_notes.py). +alert_ok = False +alert_detail = '' +_patch_dir = os.path.dirname(os.path.dirname(nested_src)) +alert_src = os.path.join(os.path.dirname(nested_src), 'alert_source.py') +alert_dst = os.path.join(mw_dir, 'alert', 'source', 'truecloud_patch_update.py') + +if os.path.exists(os.path.join(_patch_dir, 'update_alerts_disabled')): + alert_detail = 'disabled (update_alerts_disabled)' + try: + os.unlink(alert_dst) + print('OK: Removed update alert (disabled).') + except OSError: + pass +elif not os.path.exists(alert_src): + alert_detail = 'alert_source.py not found' + print(f'WARNING: {alert_src} missing — no update alert.') +else: + try: + with open(alert_src, encoding='utf-8') as fh: + _body = fh.read() + # PATCH_DIR is baked in: the alert source must find the repo it belongs to. + with open(alert_dst, 'w', encoding='utf-8') as fh: + fh.write(_body.replace('@PATCH_DIR@', _patch_dir)) + alert_ok = True + alert_detail = 'update alert installed' + print(f'OK: Installed update alert → {alert_dst}') + except Exception as e: + alert_detail = f'not applied: {e}' + print(f'WARNING: could not install update alert: {e}') + patches = { 'providers': { 'ok': (not providers_needed) or bool(b2_ok and restic_ok), @@ -589,6 +623,11 @@ patches = { 'active': nested_needed, 'detail': nested_detail, }, + 'update_alert': { + 'ok': True, # never a failure: it is a convenience, not a patch + 'active': alert_ok, + 'detail': alert_detail, + }, } payload = {'patched_at': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), 'patches': patches} tmp = status_path + '.tmp' diff --git a/patch/create_task.py b/patch/create_task.py index c839d3c..bc44258 100755 --- a/patch/create_task.py +++ b/patch/create_task.py @@ -52,7 +52,7 @@ import subprocess import sys import time -__version__ = "0.4.2" +__version__ = "0.5.0" _PATCH_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) _STATUS_FILE = os.path.join(_PATCH_DIR, "hook_status.json") diff --git a/patch/mw_patch.py b/patch/mw_patch.py index 27d8f28..3686e89 100644 --- a/patch/mw_patch.py +++ b/patch/mw_patch.py @@ -37,6 +37,10 @@ NESTED_RELPATHS = [ #: The importable module the nested blocks depend on. NESTED_MODULE = ("plugins", "cloud", "_truecloud_nested.py") +#: The update-available alert source. Not a "patch" (it appends nothing to a stock +#: file), but it is a file we install into middlewared and must therefore remove. +ALERT_MODULE = ("alert", "source", "truecloud_patch_update.py") + def patch_file(path, block): """Append `block`, replacing any block we appended before. Idempotent.""" @@ -101,8 +105,14 @@ def revert_nested(mw_dir): def revert_all(mw_dir): - """Undo every patch this project applies.""" - return revert(mw_dir, NESTED_RELPATHS + PROVIDER_RELPATHS, NESTED_MODULE) + """Undo every patch this project applies, and remove every file it installs.""" + reverted = revert(mw_dir, NESTED_RELPATHS + PROVIDER_RELPATHS, NESTED_MODULE) + try: + os.unlink(os.path.join(mw_dir, *ALERT_MODULE)) + reverted.append(ALERT_MODULE[-1]) + except OSError: + pass + return reverted def find_middlewared_dir(): diff --git a/recover.sh b/recover.sh index 0fdab8a..08a6c35 100755 --- a/recover.sh +++ b/recover.sh @@ -17,7 +17,7 @@ # bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh # systemctl restart middlewared -VERSION="0.4.2" +VERSION="0.5.0" PATCH_DIR="$(cd "$(dirname "$0")" && pwd)" diff --git a/tests/test_release_notes.py b/tests/test_release_notes.py index 65113e8..ea0bb8e 100644 --- a/tests/test_release_notes.py +++ b/tests/test_release_notes.py @@ -21,6 +21,8 @@ from release_notes import ( # noqa: E402 extract_notes, normalise, script_versions, + significance, + version_tuple, ) REPO = os.path.join(os.path.dirname(__file__), "..") @@ -120,3 +122,81 @@ class TestCheckCatchesMistakes: def test_reports_a_missing_changelog_section(self): problems = check("v9.9.9", REPO) assert any("no section" in p for p in problems) + + +class TestSignificance: + """Drives the TrueNAS update alert: what is worth bothering a human about. + + The rule: a release whose CHANGELOG only has a "### Docs" section changed no + code, and nobody should get an alert because a README was reworded. + """ + + TEXT = """\ +# Changelog + +## v0.4.2 — 2026-07-13 + +### Docs + +- reworded the README + +## v0.4.1 — 2026-07-13 + +### Fixed + +- a real bug + +## v0.4.0 — 2026-07-13 + +### Added + +- a feature + +## v0.3.3 — 2026-07-13 + +### Security + +- keep a password out of argv + +## v0.3.2 — 2026-07-13 + +### Fixed + +- something +""" + + def test_docs_only_release_does_not_alert(self): + level, versions, _ = significance(self.TEXT, "0.4.1", "0.4.2") + assert level == "docs" + assert versions == ["0.4.2"] + + def test_a_real_fix_alerts(self): + level, _v, _h = significance(self.TEXT, "0.4.0", "0.4.1") + assert level == "notable" + + def test_security_in_range_escalates(self): + level, _v, _h = significance(self.TEXT, "0.3.2", "0.3.3") + assert level == "security" + + def test_security_wins_even_when_the_newest_release_is_docs_only(self): + # A docs-only v0.4.2 sitting on top of a security-fixing v0.3.3 must still + # be reported as security — classify the whole span, not just the tip. + level, versions, _ = significance(self.TEXT, "0.3.2", "0.4.2") + assert level == "security" + assert set(versions) == {"0.3.3", "0.4.0", "0.4.1", "0.4.2"} + + def test_same_version_is_never_notable(self): + level, versions, _ = significance(self.TEXT, "0.4.2", "0.4.2") + assert level == "docs" + assert versions == [] + + def test_range_is_exclusive_of_current_inclusive_of_latest(self): + _l, versions, _h = significance(self.TEXT, "0.4.0", "0.4.2") + assert "0.4.0" not in versions + assert "0.4.2" in versions + + def test_version_tuple_orders_correctly(self): + assert version_tuple("v0.10.0") > version_tuple("v0.9.9") + assert version_tuple("0.4.2") > version_tuple("0.4.1") + # Pre-release suffixes are dropped, not ranked above the release. + assert version_tuple("v0.5.0-rc1") == version_tuple("v0.5.0") diff --git a/tools/release_notes.py b/tools/release_notes.py index 260dd8c..2bc10af 100644 --- a/tools/release_notes.py +++ b/tools/release_notes.py @@ -88,6 +88,60 @@ def extract_notes(text: str, version: str) -> str: return "\n".join(lines[start:end]).strip() +# ── significance ────────────────────────────────────────────────────────────── +# Used by the TrueNAS update alert to decide whether a release is worth bothering +# anyone about. The CHANGELOG's own section headings are the signal: a release that +# only has "### Docs" changed no code, and nobody should get an alert for a README. + +_SECTION_RE = re.compile(r"^###\s+(.+?)\s*$", re.M) + +#: Headings that mean "nothing about the running system changed". +QUIET_SECTIONS = {"docs", "documentation"} + + +def version_tuple(v: str) -> tuple: + """Sortable version. Pre-release suffixes are dropped, not ranked.""" + return tuple(int(x) for x in normalise(v).split("-")[0].split(".")) + + +def section_headings(body: str) -> list[str]: + """The `### ...` headings inside one version's body, lowercased.""" + return [h.strip().lower() for h in _SECTION_RE.findall(body)] + + +def significance(text: str, current: str, latest: str): + """How much does upgrading `current` -> `latest` actually matter? + + Returns ``(level, versions, headings)`` where level is one of: + + "security" a release in the range has a Security section -> alert loudly + "notable" something about the system changed -> alert quietly + "docs" only documentation changed -> DO NOT alert + + Considers every release in the range, not just the newest: a docs-only v0.4.2 + on top of a security-fixing v0.4.1 must still be reported as security. + """ + cur, lat = version_tuple(current), version_tuple(latest) + + versions = [ + v for v in changelog_versions(text) + if cur < version_tuple(v) <= lat + ] + + headings = [] + for v in versions: + try: + headings.extend(section_headings(extract_notes(text, v))) + except KeyError: + continue + + if any(h.startswith("security") for h in headings): + return "security", versions, headings + if [h for h in headings if h not in QUIET_SECTIONS]: + return "notable", versions, headings + return "docs", versions, headings + + def check(version: str, root: str = ROOT) -> list[str]: """Every reason this version is not releasable. Empty list means it is.""" want = normalise(version) diff --git a/uninstall.sh b/uninstall.sh index 205f8f7..827b60b 100755 --- a/uninstall.sh +++ b/uninstall.sh @@ -3,7 +3,7 @@ set -euo pipefail -VERSION="0.4.2" +VERSION="0.5.0" PATCH_DIR="$(cd "$(dirname "$0")" && pwd)" _HOOK_COMMENT='TrueCloud provider patch (S3/B2)' diff --git a/update.sh b/update.sh index 67e130d..e1d2264 100644 --- a/update.sh +++ b/update.sh @@ -19,7 +19,7 @@ set -euo pipefail -VERSION="0.4.2" +VERSION="0.5.0" PATCH_DIR="$(cd "$(dirname "$0")" && pwd)" _PREV_FILE="$PATCH_DIR/.update_previous"