Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 flan
|
4e0814028c | ||
|
flan
|
345741e1f1 |
@@ -13,3 +13,4 @@ __pycache__/
|
||||
.ruff_cache/
|
||||
.venv/
|
||||
venv/
|
||||
/update_alerts_disabled
|
||||
|
||||
@@ -1,5 +1,60 @@
|
||||
# Changelog
|
||||
|
||||
## v0.5.0 — 2026-07-13
|
||||
|
||||
### Added
|
||||
|
||||
- **A TrueNAS alert when an update is available** — the bell in the UI, not a log
|
||||
line nobody reads. On by default, checked once a day.
|
||||
`install.sh --no-update-alerts` turns it off.
|
||||
|
||||
**It does not nag.** A release whose CHANGELOG contains only a `### Docs`
|
||||
section changed no code and raises nothing. Anything else raises INFO; a
|
||||
`### Security` section raises WARNING. The CHANGELOG's own section headings are
|
||||
the signal, and a security fix anywhere in the range escalates the whole span —
|
||||
so a docs-only release sitting on top of a security fix still reports as
|
||||
security, rather than hiding it.
|
||||
|
||||
**Why an AlertSource and not `midclt`:** TrueNAS cannot raise an alert from the
|
||||
CLI. `midclt` exposes only `alert.dismiss`, `alert.list`, `alert.list_categories`,
|
||||
`alert.list_policies` and `alert.restore` — alert *creation* is internal to
|
||||
middlewared, and none of its ~60 one-shot classes is generic enough to reuse. So
|
||||
registering an `AlertSource` is the only way, and it is also the least invasive
|
||||
thing this patch does: it **adds one file and modifies none**, where the
|
||||
providers and nested modules both append code to stock middleware files. It is
|
||||
the native mechanism, and TrueNAS polls it itself — no cron, no systemd timer.
|
||||
|
||||
- Fail-safe: every error path returns `None`; it cannot take middlewared down.
|
||||
- Read-only: `git ls-remote` plus an HTTPS fetch of the CHANGELOG. It never
|
||||
writes to `.git`, so it cannot leave root-owned objects behind the way a
|
||||
`git fetch` from middlewared (running as root) would.
|
||||
- Removed by `uninstall.sh`.
|
||||
- It only *tells* you; it never updates anything.
|
||||
|
||||
## v0.4.2 — 2026-07-13
|
||||
|
||||
### Docs
|
||||
|
||||
- **The Updating section never said how to *get* `update.sh`.** It ships inside the
|
||||
patch, so a clone older than v0.4.0 doesn't have it — the docs told you to run a
|
||||
script you didn't have. There is now an explicit bootstrap step (`git pull &&
|
||||
bash install.sh`, once), including the fix for the *"insufficient permission for
|
||||
adding an object to repository database"* failure that past `sudo git pull`s
|
||||
cause.
|
||||
|
||||
- **`After a TrueNAS update` rewritten.** It didn't explain that the patch
|
||||
re-applies itself at every boot (so you never reinstall), and it didn't say what
|
||||
each failure actually costs you. "Fail-safe" means *the box stays up* — not that
|
||||
your backups keep running. A `[FAIL] providers` is a **broken backup**, and the
|
||||
docs now say so rather than implying everything degrades gracefully.
|
||||
|
||||
- Added a repo map. `patch/mw_patch.py` and `tools/release_notes.py` were
|
||||
documented nowhere.
|
||||
|
||||
- `Development` told you to run `ruff check patch tests`, which misses `tools/`.
|
||||
|
||||
- Every command and file path in the README is now verified to exist and run.
|
||||
|
||||
## v0.4.1 — 2026-07-13
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -46,6 +46,23 @@ The patch is two independent modules — **providers** (B2/S3) and **nested**
|
||||
(snapshots on nested datasets) — and each retires on its own once TrueNAS ships
|
||||
that capability natively. See [Native support](#if-truenas-adds-native-support).
|
||||
|
||||
## What's in the repo
|
||||
|
||||
| Path | What it is |
|
||||
|---|---|
|
||||
| `install.sh` | Registers the PREINIT boot hook, applies the patch, restarts middlewared. Also `--enable/--disable-nested-snapshots`. |
|
||||
| `update.sh` | Fetch a newer **release** and apply it. `--check`, `--rollback`, `--to`, `--main`. |
|
||||
| `uninstall.sh` | Remove everything: boot hook, patched files, UI bundle, staging mounts. |
|
||||
| `recover.sh` | Emergency: set the kill switch and restart middlewared against stock files. |
|
||||
| `patch/apply.sh` | The PREINIT script. Runs at **every boot**; re-applies the patch into a fresh overlay. |
|
||||
| `patch/mw_patch.py` | The one implementation of apply/revert for the `TRUECLOUD_PATCH` blocks. Used by `apply.sh` *and* `uninstall.sh`. |
|
||||
| `patch/truecloud_nested.py` | Nested-dataset staging: plan, mount, verify, tear down, sweep snapshots. Also `… cleanup` as a CLI. |
|
||||
| `patch/patch_ui.py` | Widens the Angular credential dropdown. Refuses to write a bundle whose parens it unbalanced. |
|
||||
| `patch/create_task.py` | Create TrueCloud tasks with S3/B2 credentials; `verify` the patch state. |
|
||||
| `patch/alert_source.py` | The TrueNAS alert for "an update is available". Installed into `middlewared/alert/source/`. |
|
||||
| `patch/wait_restart.sh` | Waits for boot to actually settle before restarting middlewared. |
|
||||
| `tools/release_notes.py` | Extracts a version's CHANGELOG section; enforces version consistency. Used by CI. |
|
||||
|
||||
## Development
|
||||
|
||||
Parts of this project were written with AI assistance (Claude). All of it is
|
||||
@@ -54,14 +71,24 @@ make that review meaningful. Bugs are mine.
|
||||
|
||||
```bash
|
||||
pip install pytest ruff
|
||||
ruff check patch tests
|
||||
ruff check patch tests tools
|
||||
pytest tests
|
||||
```
|
||||
|
||||
CI runs shellcheck, `bash -n`, ruff, and pytest on Python 3.11–3.13. The tests
|
||||
include a pass that `compile()`s the `*_BLOCK` strings in `patch/apply.sh` —
|
||||
those are Python source appended into live `middlewared` modules, so a syntax
|
||||
error there would break the box at boot.
|
||||
CI runs shellcheck, `bash -n`, ruff, and pytest on Python 3.11–3.13. Two checks
|
||||
are worth calling out, because nothing else would catch what they catch:
|
||||
|
||||
- The tests **`compile()` the `*_BLOCK` strings** in `patch/apply.sh`. Those are
|
||||
Python source appended into live `middlewared` modules — a syntax error there
|
||||
breaks the box at boot, and they're string literals, so nothing else type-checks
|
||||
them.
|
||||
- CI asserts **every script declares the same version**, and that it matches the
|
||||
newest CHANGELOG entry. `VERSION=` had silently drifted to three different
|
||||
values across the scripts before anything checked.
|
||||
|
||||
Releases are automated: push a `vX.Y.Z` tag and the workflow runs the full suite,
|
||||
verifies the version matches, and cuts a GitHub release whose body **is** the
|
||||
matching `CHANGELOG.md` section — one source of truth for release notes.
|
||||
|
||||
---
|
||||
|
||||
@@ -342,27 +369,115 @@ Refresh your browser. S3 and B2 credentials now appear in the
|
||||
## Updating
|
||||
|
||||
```bash
|
||||
cd /mnt/tank/truenas-truecloud-patch
|
||||
bash update.sh # to the newest release, with a confirmation
|
||||
bash update.sh --check # show what would happen; change nothing
|
||||
bash update.sh --rollback # undo the last update
|
||||
```
|
||||
|
||||
It preserves your nested-snapshot opt-in setting, shows you the commits and
|
||||
release notes you don't have yet, and asks before changing anything. It records
|
||||
the previous revision *before* moving, so `--rollback` works even if `install.sh`
|
||||
dies halfway.
|
||||
| | |
|
||||
|---|---|
|
||||
| `bash update.sh` | Update to the newest release tag. Shows what's coming, asks first. |
|
||||
| `bash update.sh --check` | Show what *would* happen. Changes nothing. |
|
||||
| `bash update.sh --rollback` | Undo the last update. |
|
||||
| `bash update.sh --to v0.3.5` | Go to a specific tag or commit. |
|
||||
| `bash update.sh --main` | Track **unreleased** `main`. You're on your own. |
|
||||
| `bash update.sh --yes` | Skip the confirmation (for a scripted, *attended* run). |
|
||||
|
||||
**Run it by hand. Never from cron or a systemd timer.** This patch injects Python
|
||||
into `middlewared` and re-applies itself at every boot, so an unattended pull would
|
||||
let any bad upstream commit reach your box with no human in the loop and take
|
||||
effect on the next reboot. v0.0.4 shipped exactly such a bug and took every app on
|
||||
the box down. The manual step *is* the safety gate — if you want convenience, watch
|
||||
the [releases](https://github.com/sudolulo/truenas-truecloud-patch/releases) feed,
|
||||
Run it as **root** — it calls `install.sh`, which needs to reload middlewared.
|
||||
|
||||
### First time: bootstrapping `update.sh`
|
||||
|
||||
`update.sh` ships *inside* the patch, so a clone older than v0.4.0 doesn't have it
|
||||
yet. Bootstrap it once with git:
|
||||
|
||||
```bash
|
||||
cd /mnt/tank/truenas-truecloud-patch
|
||||
git pull # or: git checkout v0.4.1
|
||||
bash install.sh
|
||||
```
|
||||
|
||||
Every update after that is just `bash update.sh`.
|
||||
|
||||
> If a plain `git pull` fails with *"insufficient permission for adding an object
|
||||
> to repository database"*, past `sudo git pull`s left root-owned objects in
|
||||
> `.git`. Fix it once as root: `chown -R <you>:<you> .git`. (`update.sh` repairs
|
||||
> this automatically from then on.)
|
||||
|
||||
### What it does for you
|
||||
|
||||
- **Preserves your nested-snapshot opt-in setting** — updating never flips it.
|
||||
- **Shows the commits and release notes you don't have**, then asks before moving.
|
||||
- **Records the previous revision *before* checking out**, so `--rollback` works
|
||||
even if `install.sh` dies halfway through.
|
||||
- **Refuses to run over a dirty working tree**, rather than merging across
|
||||
hand-edited or scp'd files and losing them.
|
||||
- **Detects untracked files that would be clobbered** by the checkout and names
|
||||
them, instead of dying on a raw git error mid-update.
|
||||
- **Repairs `.git` ownership** left root-owned by past `sudo git pull`s.
|
||||
|
||||
It updates to the newest **release tag**, not `main`. `main` can be mid-refactor;
|
||||
a tag is the tested artifact, and CI gates every release. Pre-release tags
|
||||
(`-rc`, `-beta`) are skipped — `--to` them explicitly if you want one.
|
||||
|
||||
After updating, the checkout is pinned to a release tag (detached HEAD). That is
|
||||
what you want for a deployment: plain `git pull` no longer applies, and
|
||||
`update.sh` is the supported path.
|
||||
|
||||
### Why there is no auto-update
|
||||
|
||||
**Never put this in cron or a systemd timer.** This patch injects Python into
|
||||
`middlewared` and re-applies itself at *every boot*, so an unattended pull would
|
||||
let any bad upstream commit reach your box with no human in the loop — and
|
||||
detonate on the next reboot. That is not hypothetical: **v0.0.4 shipped exactly
|
||||
such a bug and took all 54 apps on a box down.**
|
||||
|
||||
The manual step *is* the safety gate. If you want convenience, watch the
|
||||
[releases feed](https://github.com/sudolulo/truenas-truecloud-patch/releases);
|
||||
don't automate the pull.
|
||||
|
||||
It updates to the newest **release tag**, not `main` — `main` can be mid-refactor,
|
||||
and a tag is the tested artifact. `--main` exists if you want unreleased code, and
|
||||
says so loudly.
|
||||
## Update alerts
|
||||
|
||||
When a newer release exists, the patch raises a **TrueNAS alert** (the bell in the
|
||||
UI) telling you so. It's on by default and checks once a day.
|
||||
|
||||
```bash
|
||||
bash install.sh --no-update-alerts # turn it off
|
||||
bash install.sh --update-alerts # turn it back on
|
||||
```
|
||||
|
||||
**It will not nag you about a README.** A release whose CHANGELOG contains only a
|
||||
`### Docs` section changed no code, and raises nothing. Anything that touched the
|
||||
system raises an INFO alert; a release with a `### Security` section raises a
|
||||
WARNING. The CHANGELOG's own section headings are the signal, and a security fix
|
||||
anywhere in the range escalates the whole span — a docs-only release on top of a
|
||||
security fix still reports as security.
|
||||
|
||||
### How it works, and why it's built this way
|
||||
|
||||
TrueNAS **cannot raise an alert from the CLI** — `midclt` exposes only
|
||||
`alert.dismiss`, `alert.list`, `alert.list_categories`, `alert.list_policies` and
|
||||
`alert.restore`. Alert *creation* is internal to middlewared, and none of its ~60
|
||||
one-shot alert classes is generic enough to reuse. So the only way to get a real
|
||||
alert is to register an `AlertSource`, which is what `patch/alert_source.py` does.
|
||||
|
||||
That is also the **least invasive** thing this patch does:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| providers module | **modifies** stock files (appends code to `b2.py`, `restic.py`) |
|
||||
| nested module | **modifies** stock files (3 middleware modules) |
|
||||
| **update alert** | **adds one file. Modifies nothing.** |
|
||||
|
||||
It's the native mechanism — the same one every built-in TrueNAS alert uses — and
|
||||
TrueNAS polls it itself, so there is no cron job and no systemd timer.
|
||||
|
||||
- **Fail-safe.** Every error path returns `None`. It cannot take middlewared down.
|
||||
- **Read-only.** `git ls-remote` plus an HTTPS fetch of the CHANGELOG. It never
|
||||
writes to `.git`, so it cannot leave root-owned objects behind the way a
|
||||
`git fetch` from middlewared (which runs as root) would.
|
||||
- **Removed by `uninstall.sh`.**
|
||||
|
||||
It only *tells* you. It never updates anything — see
|
||||
[Why there is no auto-update](#why-there-is-no-auto-update).
|
||||
|
||||
## Creating a task via CLI
|
||||
|
||||
@@ -460,12 +575,34 @@ tail -20 /mnt/tank/truenas-truecloud-patch/apply.log
|
||||
|
||||
## After a TrueNAS update
|
||||
|
||||
1. Check the log: `cat /mnt/tank/truenas-truecloud-patch/apply.log | tail -30`
|
||||
2. If you see "WARNING: … pattern not found", the UI patch needs updating.
|
||||
[Open an issue](https://github.com/sudolulo/truenas-truecloud-patch/issues)
|
||||
with your TrueNAS version number.
|
||||
3. The backend patch (B2 support + URL fix) is more stable — check that a
|
||||
B2 backup job still completes successfully after any update.
|
||||
A TrueNAS update replaces `/usr/` wholesale, wiping the patch. You do **not** need
|
||||
to reinstall: `patch/apply.sh` runs at every boot and re-applies itself from your
|
||||
clone. But it targets internal APIs with no stability contract, so an update *can*
|
||||
break it — and the failure is quiet by design (middlewared starts fine; the patch
|
||||
just doesn't).
|
||||
|
||||
**Check the log after any TrueNAS update:**
|
||||
|
||||
```bash
|
||||
tail -30 /mnt/tank/truenas-truecloud-patch/apply.log
|
||||
python3 /mnt/tank/truenas-truecloud-patch/patch/create_task.py verify
|
||||
```
|
||||
|
||||
| What you see | What it means |
|
||||
|---|---|
|
||||
| `[OK] providers`, `[OK]`/`[SKIP] nested_snapshots` | Fine. Nothing to do. |
|
||||
| `WARNING: … pattern not found` (UI) | The Angular bundle changed. The UI dropdown reverts to Storj-only, but **backups keep working** — create tasks with `create_task.py` meanwhile, and [open an issue](https://github.com/sudolulo/truenas-truecloud-patch/issues) with your TrueNAS version. |
|
||||
| `[FAIL] providers` | **Your B2/S3 backups will not run.** middlewared is fine, but the credential/URL handling is gone. Open an issue with your version. |
|
||||
| `[FAIL] nested_snapshots` | The stock guard is back, so tasks with `snapshot = true` on a nested dataset will fail validation. Turn the option off on those tasks until it's fixed. |
|
||||
|
||||
"Fail-safe" means *the box stays up* — not that your backups keep running. A
|
||||
`[FAIL] providers` is a broken backup, so check the log rather than assume.
|
||||
|
||||
Then update the patch itself if a newer release fixes it:
|
||||
|
||||
```bash
|
||||
bash /mnt/tank/truenas-truecloud-patch/update.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
|
||||
+35
-1
@@ -18,7 +18,7 @@
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
VERSION="0.4.1"
|
||||
VERSION="0.5.0"
|
||||
|
||||
# The directory containing install.sh is the permanent install location.
|
||||
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
@@ -31,6 +31,8 @@ _NESTED_MARKER="$PATCH_DIR/nested_snapshots_enabled"
|
||||
# `git pull`) must never flip it on or off by itself: with neither flag given,
|
||||
# whatever was chosen previously is preserved.
|
||||
_nested_choice=""
|
||||
_alert_choice=""
|
||||
_ALERT_MARKER="$PATCH_DIR/update_alerts_disabled"
|
||||
|
||||
usage() {
|
||||
cat <<USAGE
|
||||
@@ -42,6 +44,8 @@ Options:
|
||||
Stock TrueNAS refuses this; see README. Off by
|
||||
default because it changes how backups read data.
|
||||
--disable-nested-snapshots Turn it back off; the stock guard is restored.
|
||||
--no-update-alerts Do not raise a TrueNAS alert when an update exists.
|
||||
--update-alerts Re-enable those alerts (they are on by default).
|
||||
-h, --help Show this help.
|
||||
|
||||
With neither flag, the current setting is left unchanged.
|
||||
@@ -52,6 +56,8 @@ while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--enable-nested-snapshots) _nested_choice="on" ;;
|
||||
--disable-nested-snapshots) _nested_choice="off" ;;
|
||||
--no-update-alerts) _alert_choice="off" ;;
|
||||
--update-alerts) _alert_choice="on" ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*)
|
||||
echo "ERROR: unknown option: $1" >&2
|
||||
@@ -192,6 +198,34 @@ case "$_nested_choice" in
|
||||
esac
|
||||
echo ""
|
||||
|
||||
# ── Update alerts (on by default) ─────────────────────────────────────────────
|
||||
# TrueNAS cannot raise an alert from the CLI (midclt exposes only dismiss/list/
|
||||
# restore), so this installs an AlertSource into middlewared/alert/source/ — the
|
||||
# same mechanism every built-in TrueNAS alert uses. It ADDS a file and modifies
|
||||
# none, which makes it the least invasive thing this patch does.
|
||||
#
|
||||
# It only alerts for releases that changed something: a documentation-only release
|
||||
# raises nothing.
|
||||
|
||||
case "$_alert_choice" in
|
||||
off)
|
||||
touch "$_ALERT_MARKER"
|
||||
echo "Update alerts: DISABLED (apply.sh will remove the alert source)."
|
||||
;;
|
||||
on)
|
||||
rm -f "$_ALERT_MARKER"
|
||||
echo "Update alerts: enabled."
|
||||
;;
|
||||
*)
|
||||
if [ -f "$_ALERT_MARKER" ]; then
|
||||
echo "Update alerts: disabled (unchanged)."
|
||||
else
|
||||
echo "Update alerts: enabled (checks daily; docs-only releases are ignored)."
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
echo ""
|
||||
|
||||
# ── Apply now ─────────────────────────────────────────────────────────────────
|
||||
|
||||
echo "Applying patches ..."
|
||||
|
||||
@@ -0,0 +1,211 @@
|
||||
"""TrueNAS alert: a truecloud-patch update is available.
|
||||
|
||||
Installed by patch/apply.sh into middlewared/alert/source/, where middlewared
|
||||
discovers and polls it natively — no cron job, no systemd timer.
|
||||
|
||||
@PATCH_DIR@ is substituted at install time.
|
||||
|
||||
Two rules govern this file:
|
||||
|
||||
1. **It must never break middlewared.** It runs inside the alert framework on a
|
||||
timer. Every failure path returns None (no alert) rather than raising.
|
||||
|
||||
2. **It must not nag.** A release whose CHANGELOG only has a "### Docs" section
|
||||
changed no code, and nobody wants an alert because a README was reworded. The
|
||||
CHANGELOG's own section headings are the signal — see tools/release_notes.py.
|
||||
|
||||
It also never writes to the repository. `git ls-remote` is read-only and the
|
||||
CHANGELOG is fetched over HTTPS, so this cannot leave root-owned objects in .git
|
||||
the way a `git fetch` from middlewared (running as root) would.
|
||||
"""
|
||||
|
||||
import datetime
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
from middlewared.alert.base import (
|
||||
Alert,
|
||||
AlertCategory,
|
||||
AlertClass,
|
||||
AlertLevel,
|
||||
ThreadedAlertSource,
|
||||
)
|
||||
from middlewared.alert.schedule import IntervalSchedule
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
PATCH_DIR = "@PATCH_DIR@"
|
||||
DISABLED_MARKER = os.path.join(PATCH_DIR, "update_alerts_disabled")
|
||||
|
||||
_TAG_RE = re.compile(r"^v\d+\.\d+\.\d+$")
|
||||
_VERSION_RE = re.compile(r'^VERSION="([^"]+)"', re.M)
|
||||
_GITHUB_RE = re.compile(r"github\.com[:/]([^/]+)/([^/.]+)")
|
||||
|
||||
_TIMEOUT = 20
|
||||
|
||||
|
||||
class TrueCloudPatchUpdateAlertClass(AlertClass):
|
||||
category = AlertCategory.SYSTEM
|
||||
level = AlertLevel.INFO
|
||||
title = "truecloud-patch update available"
|
||||
text = (
|
||||
"truecloud-patch %(current)s is installed; %(latest)s is available.%(summary)s "
|
||||
"Update with: bash %(dir)s/update.sh"
|
||||
)
|
||||
|
||||
|
||||
class TrueCloudPatchSecurityUpdateAlertClass(AlertClass):
|
||||
category = AlertCategory.SYSTEM
|
||||
level = AlertLevel.WARNING
|
||||
title = "truecloud-patch security update available"
|
||||
text = (
|
||||
"truecloud-patch %(current)s is installed; %(latest)s contains a SECURITY "
|
||||
"fix.%(summary)s Update with: bash %(dir)s/update.sh"
|
||||
)
|
||||
|
||||
|
||||
class TrueCloudPatchUpdateAlertSource(ThreadedAlertSource):
|
||||
schedule = IntervalSchedule(datetime.timedelta(hours=24))
|
||||
run_on_backup_node = False
|
||||
|
||||
def check_sync(self):
|
||||
try:
|
||||
return self._check()
|
||||
except Exception:
|
||||
# An alert source must never take middlewared down with it.
|
||||
logger.debug("truecloud-patch update check failed", exc_info=True)
|
||||
return None
|
||||
|
||||
# ── internals ────────────────────────────────────────────────────────────
|
||||
|
||||
def _git(self, *args):
|
||||
return subprocess.run(
|
||||
["git", "-C", PATCH_DIR, *args],
|
||||
capture_output=True, text=True, timeout=_TIMEOUT, check=True,
|
||||
).stdout
|
||||
|
||||
def _check(self):
|
||||
if os.path.exists(DISABLED_MARKER):
|
||||
return None
|
||||
if not os.path.isdir(os.path.join(PATCH_DIR, ".git")):
|
||||
return None
|
||||
|
||||
current = self._installed_version()
|
||||
if not current:
|
||||
return None
|
||||
|
||||
latest = self._latest_release_tag()
|
||||
if not latest:
|
||||
return None
|
||||
|
||||
sys.path.insert(0, os.path.join(PATCH_DIR, "tools"))
|
||||
try:
|
||||
from release_notes import significance, version_tuple
|
||||
finally:
|
||||
sys.path.pop(0)
|
||||
|
||||
if version_tuple(latest) <= version_tuple(current):
|
||||
return None
|
||||
|
||||
level, versions, summary = self._classify(
|
||||
current, latest, significance
|
||||
)
|
||||
|
||||
# Documentation-only releases are not worth an alert. This is the whole
|
||||
# point: nobody should get a notification because a README was reworded.
|
||||
if level == "docs":
|
||||
logger.debug(
|
||||
"truecloud-patch %s -> %s is documentation-only; not alerting",
|
||||
current, latest,
|
||||
)
|
||||
return None
|
||||
|
||||
args = {
|
||||
"current": f"v{current}",
|
||||
"latest": latest,
|
||||
"summary": summary,
|
||||
"dir": PATCH_DIR,
|
||||
}
|
||||
klass = (
|
||||
TrueCloudPatchSecurityUpdateAlertClass if level == "security"
|
||||
else TrueCloudPatchUpdateAlertClass
|
||||
)
|
||||
return Alert(klass, args, key=[current, latest])
|
||||
|
||||
def _installed_version(self):
|
||||
"""The version of the patch actually checked out here."""
|
||||
try:
|
||||
with open(os.path.join(PATCH_DIR, "patch", "apply.sh"), encoding="utf-8") as fh:
|
||||
m = _VERSION_RE.search(fh.read())
|
||||
except OSError:
|
||||
return None
|
||||
return m.group(1) if m else None
|
||||
|
||||
def _latest_release_tag(self):
|
||||
"""Newest plain vX.Y.Z tag on the remote. Read-only: no .git writes.
|
||||
|
||||
Pre-release tags (-rc, -beta) are excluded: git's version sort ranks
|
||||
v0.5.0-rc1 above v0.5.0, so including them would advertise a release
|
||||
candidate as the latest stable.
|
||||
"""
|
||||
try:
|
||||
out = self._git("ls-remote", "--tags", "--refs", "origin")
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
tags = []
|
||||
for line in out.splitlines():
|
||||
parts = line.split("refs/tags/")
|
||||
if len(parts) == 2 and _TAG_RE.match(parts[1].strip()):
|
||||
tags.append(parts[1].strip())
|
||||
if not tags:
|
||||
return None
|
||||
|
||||
return max(tags, key=lambda t: tuple(int(x) for x in t.lstrip("v").split(".")))
|
||||
|
||||
def _classify(self, current, latest, significance):
|
||||
"""(level, versions, one-line summary). Falls back to alerting."""
|
||||
text = self._remote_changelog(latest)
|
||||
if text is None:
|
||||
# Cannot tell whether it matters. Alert rather than risk hiding a
|
||||
# security fix -- but say that we could not tell.
|
||||
return "notable", [], " (could not read the changelog)"
|
||||
|
||||
level, versions, headings = significance(text, current, latest)
|
||||
if level == "docs":
|
||||
return level, versions, ""
|
||||
|
||||
seen, ordered = set(), []
|
||||
for h in headings:
|
||||
if h not in seen:
|
||||
seen.add(h)
|
||||
ordered.append(h.capitalize())
|
||||
detail = ", ".join(ordered)
|
||||
return level, versions, f" Changes: {detail}." if detail else ""
|
||||
|
||||
def _remote_changelog(self, tag):
|
||||
"""CHANGELOG.md at `tag`, over HTTPS. None if it cannot be read."""
|
||||
try:
|
||||
remote = self._git("remote", "get-url", "origin").strip()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
m = _GITHUB_RE.search(remote)
|
||||
if not m:
|
||||
return None # not a GitHub remote; skip classification
|
||||
|
||||
url = (
|
||||
f"https://raw.githubusercontent.com/{m.group(1)}/{m.group(2)}/"
|
||||
f"{tag}/CHANGELOG.md"
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(url, timeout=_TIMEOUT) as resp: # noqa: S310
|
||||
if resp.status != 200:
|
||||
return None
|
||||
return resp.read().decode("utf-8", "replace")
|
||||
except Exception:
|
||||
return None
|
||||
+40
-1
@@ -32,7 +32,7 @@
|
||||
# Derive PATCH_DIR from this script's location (parent of the patch/ directory).
|
||||
PATCH_DIR="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
LOG="$PATCH_DIR/apply.log"
|
||||
VERSION="0.4.1"
|
||||
VERSION="0.5.0"
|
||||
|
||||
# Rotate log at 512 KB to avoid unbounded growth on a system volume.
|
||||
# Keep two prior generations (.1 and .2) so the last three boots are always available.
|
||||
@@ -578,6 +578,40 @@ else:
|
||||
# that is inactive (superseded, or opt-in and off) is ok -- reporting a disabled
|
||||
# opt-in feature as FAIL would make `create_task.py verify` fail on a default
|
||||
# install. `active` says whether the module is doing anything.
|
||||
# ── update-available alert ────────────────────────────────────────────────────
|
||||
# Dropped into middlewared/alert/source/, where middlewared discovers and polls it
|
||||
# natively — no cron, no timer. It only raises an alert for releases that actually
|
||||
# changed something: a docs-only release is ignored (see tools/release_notes.py).
|
||||
alert_ok = False
|
||||
alert_detail = ''
|
||||
_patch_dir = os.path.dirname(os.path.dirname(nested_src))
|
||||
alert_src = os.path.join(os.path.dirname(nested_src), 'alert_source.py')
|
||||
alert_dst = os.path.join(mw_dir, 'alert', 'source', 'truecloud_patch_update.py')
|
||||
|
||||
if os.path.exists(os.path.join(_patch_dir, 'update_alerts_disabled')):
|
||||
alert_detail = 'disabled (update_alerts_disabled)'
|
||||
try:
|
||||
os.unlink(alert_dst)
|
||||
print('OK: Removed update alert (disabled).')
|
||||
except OSError:
|
||||
pass
|
||||
elif not os.path.exists(alert_src):
|
||||
alert_detail = 'alert_source.py not found'
|
||||
print(f'WARNING: {alert_src} missing — no update alert.')
|
||||
else:
|
||||
try:
|
||||
with open(alert_src, encoding='utf-8') as fh:
|
||||
_body = fh.read()
|
||||
# PATCH_DIR is baked in: the alert source must find the repo it belongs to.
|
||||
with open(alert_dst, 'w', encoding='utf-8') as fh:
|
||||
fh.write(_body.replace('@PATCH_DIR@', _patch_dir))
|
||||
alert_ok = True
|
||||
alert_detail = 'update alert installed'
|
||||
print(f'OK: Installed update alert → {alert_dst}')
|
||||
except Exception as e:
|
||||
alert_detail = f'not applied: {e}'
|
||||
print(f'WARNING: could not install update alert: {e}')
|
||||
|
||||
patches = {
|
||||
'providers': {
|
||||
'ok': (not providers_needed) or bool(b2_ok and restic_ok),
|
||||
@@ -589,6 +623,11 @@ patches = {
|
||||
'active': nested_needed,
|
||||
'detail': nested_detail,
|
||||
},
|
||||
'update_alert': {
|
||||
'ok': True, # never a failure: it is a convenience, not a patch
|
||||
'active': alert_ok,
|
||||
'detail': alert_detail,
|
||||
},
|
||||
}
|
||||
payload = {'patched_at': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), 'patches': patches}
|
||||
tmp = status_path + '.tmp'
|
||||
|
||||
@@ -52,7 +52,7 @@ import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
__version__ = "0.4.1"
|
||||
__version__ = "0.5.0"
|
||||
|
||||
_PATCH_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
_STATUS_FILE = os.path.join(_PATCH_DIR, "hook_status.json")
|
||||
|
||||
+12
-2
@@ -37,6 +37,10 @@ NESTED_RELPATHS = [
|
||||
#: The importable module the nested blocks depend on.
|
||||
NESTED_MODULE = ("plugins", "cloud", "_truecloud_nested.py")
|
||||
|
||||
#: The update-available alert source. Not a "patch" (it appends nothing to a stock
|
||||
#: file), but it is a file we install into middlewared and must therefore remove.
|
||||
ALERT_MODULE = ("alert", "source", "truecloud_patch_update.py")
|
||||
|
||||
|
||||
def patch_file(path, block):
|
||||
"""Append `block`, replacing any block we appended before. Idempotent."""
|
||||
@@ -101,8 +105,14 @@ def revert_nested(mw_dir):
|
||||
|
||||
|
||||
def revert_all(mw_dir):
|
||||
"""Undo every patch this project applies."""
|
||||
return revert(mw_dir, NESTED_RELPATHS + PROVIDER_RELPATHS, NESTED_MODULE)
|
||||
"""Undo every patch this project applies, and remove every file it installs."""
|
||||
reverted = revert(mw_dir, NESTED_RELPATHS + PROVIDER_RELPATHS, NESTED_MODULE)
|
||||
try:
|
||||
os.unlink(os.path.join(mw_dir, *ALERT_MODULE))
|
||||
reverted.append(ALERT_MODULE[-1])
|
||||
except OSError:
|
||||
pass
|
||||
return reverted
|
||||
|
||||
|
||||
def find_middlewared_dir():
|
||||
|
||||
+1
-1
@@ -17,7 +17,7 @@
|
||||
# bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh
|
||||
# systemctl restart middlewared
|
||||
|
||||
VERSION="0.4.1"
|
||||
VERSION="0.5.0"
|
||||
|
||||
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
|
||||
|
||||
@@ -21,6 +21,8 @@ from release_notes import ( # noqa: E402
|
||||
extract_notes,
|
||||
normalise,
|
||||
script_versions,
|
||||
significance,
|
||||
version_tuple,
|
||||
)
|
||||
|
||||
REPO = os.path.join(os.path.dirname(__file__), "..")
|
||||
@@ -120,3 +122,81 @@ class TestCheckCatchesMistakes:
|
||||
def test_reports_a_missing_changelog_section(self):
|
||||
problems = check("v9.9.9", REPO)
|
||||
assert any("no section" in p for p in problems)
|
||||
|
||||
|
||||
class TestSignificance:
|
||||
"""Drives the TrueNAS update alert: what is worth bothering a human about.
|
||||
|
||||
The rule: a release whose CHANGELOG only has a "### Docs" section changed no
|
||||
code, and nobody should get an alert because a README was reworded.
|
||||
"""
|
||||
|
||||
TEXT = """\
|
||||
# Changelog
|
||||
|
||||
## v0.4.2 — 2026-07-13
|
||||
|
||||
### Docs
|
||||
|
||||
- reworded the README
|
||||
|
||||
## v0.4.1 — 2026-07-13
|
||||
|
||||
### Fixed
|
||||
|
||||
- a real bug
|
||||
|
||||
## v0.4.0 — 2026-07-13
|
||||
|
||||
### Added
|
||||
|
||||
- a feature
|
||||
|
||||
## v0.3.3 — 2026-07-13
|
||||
|
||||
### Security
|
||||
|
||||
- keep a password out of argv
|
||||
|
||||
## v0.3.2 — 2026-07-13
|
||||
|
||||
### Fixed
|
||||
|
||||
- something
|
||||
"""
|
||||
|
||||
def test_docs_only_release_does_not_alert(self):
|
||||
level, versions, _ = significance(self.TEXT, "0.4.1", "0.4.2")
|
||||
assert level == "docs"
|
||||
assert versions == ["0.4.2"]
|
||||
|
||||
def test_a_real_fix_alerts(self):
|
||||
level, _v, _h = significance(self.TEXT, "0.4.0", "0.4.1")
|
||||
assert level == "notable"
|
||||
|
||||
def test_security_in_range_escalates(self):
|
||||
level, _v, _h = significance(self.TEXT, "0.3.2", "0.3.3")
|
||||
assert level == "security"
|
||||
|
||||
def test_security_wins_even_when_the_newest_release_is_docs_only(self):
|
||||
# A docs-only v0.4.2 sitting on top of a security-fixing v0.3.3 must still
|
||||
# be reported as security — classify the whole span, not just the tip.
|
||||
level, versions, _ = significance(self.TEXT, "0.3.2", "0.4.2")
|
||||
assert level == "security"
|
||||
assert set(versions) == {"0.3.3", "0.4.0", "0.4.1", "0.4.2"}
|
||||
|
||||
def test_same_version_is_never_notable(self):
|
||||
level, versions, _ = significance(self.TEXT, "0.4.2", "0.4.2")
|
||||
assert level == "docs"
|
||||
assert versions == []
|
||||
|
||||
def test_range_is_exclusive_of_current_inclusive_of_latest(self):
|
||||
_l, versions, _h = significance(self.TEXT, "0.4.0", "0.4.2")
|
||||
assert "0.4.0" not in versions
|
||||
assert "0.4.2" in versions
|
||||
|
||||
def test_version_tuple_orders_correctly(self):
|
||||
assert version_tuple("v0.10.0") > version_tuple("v0.9.9")
|
||||
assert version_tuple("0.4.2") > version_tuple("0.4.1")
|
||||
# Pre-release suffixes are dropped, not ranked above the release.
|
||||
assert version_tuple("v0.5.0-rc1") == version_tuple("v0.5.0")
|
||||
|
||||
@@ -88,6 +88,60 @@ def extract_notes(text: str, version: str) -> str:
|
||||
return "\n".join(lines[start:end]).strip()
|
||||
|
||||
|
||||
# ── significance ──────────────────────────────────────────────────────────────
|
||||
# Used by the TrueNAS update alert to decide whether a release is worth bothering
|
||||
# anyone about. The CHANGELOG's own section headings are the signal: a release that
|
||||
# only has "### Docs" changed no code, and nobody should get an alert for a README.
|
||||
|
||||
_SECTION_RE = re.compile(r"^###\s+(.+?)\s*$", re.M)
|
||||
|
||||
#: Headings that mean "nothing about the running system changed".
|
||||
QUIET_SECTIONS = {"docs", "documentation"}
|
||||
|
||||
|
||||
def version_tuple(v: str) -> tuple:
|
||||
"""Sortable version. Pre-release suffixes are dropped, not ranked."""
|
||||
return tuple(int(x) for x in normalise(v).split("-")[0].split("."))
|
||||
|
||||
|
||||
def section_headings(body: str) -> list[str]:
|
||||
"""The `### ...` headings inside one version's body, lowercased."""
|
||||
return [h.strip().lower() for h in _SECTION_RE.findall(body)]
|
||||
|
||||
|
||||
def significance(text: str, current: str, latest: str):
|
||||
"""How much does upgrading `current` -> `latest` actually matter?
|
||||
|
||||
Returns ``(level, versions, headings)`` where level is one of:
|
||||
|
||||
"security" a release in the range has a Security section -> alert loudly
|
||||
"notable" something about the system changed -> alert quietly
|
||||
"docs" only documentation changed -> DO NOT alert
|
||||
|
||||
Considers every release in the range, not just the newest: a docs-only v0.4.2
|
||||
on top of a security-fixing v0.4.1 must still be reported as security.
|
||||
"""
|
||||
cur, lat = version_tuple(current), version_tuple(latest)
|
||||
|
||||
versions = [
|
||||
v for v in changelog_versions(text)
|
||||
if cur < version_tuple(v) <= lat
|
||||
]
|
||||
|
||||
headings = []
|
||||
for v in versions:
|
||||
try:
|
||||
headings.extend(section_headings(extract_notes(text, v)))
|
||||
except KeyError:
|
||||
continue
|
||||
|
||||
if any(h.startswith("security") for h in headings):
|
||||
return "security", versions, headings
|
||||
if [h for h in headings if h not in QUIET_SECTIONS]:
|
||||
return "notable", versions, headings
|
||||
return "docs", versions, headings
|
||||
|
||||
|
||||
def check(version: str, root: str = ROOT) -> list[str]:
|
||||
"""Every reason this version is not releasable. Empty list means it is."""
|
||||
want = normalise(version)
|
||||
|
||||
+1
-1
@@ -3,7 +3,7 @@
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
VERSION="0.4.1"
|
||||
VERSION="0.5.0"
|
||||
|
||||
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
_HOOK_COMMENT='TrueCloud provider patch (S3/B2)'
|
||||
|
||||
Reference in New Issue
Block a user