Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
60b3ac4557 | ||
|
|
8aa9038226 | ||
|
|
ba533dc8ae | ||
|
|
eb91a337cd |
@@ -51,7 +51,7 @@ jobs:
|
|||||||
run: python -m pip install --upgrade pip pytest ruff
|
run: python -m pip install --upgrade pip pytest ruff
|
||||||
|
|
||||||
- name: ruff
|
- name: ruff
|
||||||
run: ruff check patch tests
|
run: ruff check patch tests tools
|
||||||
|
|
||||||
- name: pytest
|
- name: pytest
|
||||||
run: pytest tests -v
|
run: pytest tests -v
|
||||||
|
|||||||
@@ -0,0 +1,102 @@
|
|||||||
|
name: Release
|
||||||
|
|
||||||
|
# Push a tag, get a release. The body always comes from CHANGELOG.md, so there is
|
||||||
|
# no second place to write release notes and therefore no second place for them to
|
||||||
|
# go stale.
|
||||||
|
#
|
||||||
|
# git tag -a v0.4.0 -m "v0.4.0" && git push origin v0.4.0
|
||||||
|
#
|
||||||
|
# workflow_dispatch re-cuts (or updates) the release for a tag that already
|
||||||
|
# exists, since re-pushing an existing tag triggers nothing.
|
||||||
|
#
|
||||||
|
# It checks out the TAG, because the tagged code is what people install and it has
|
||||||
|
# to pass its own tests. That means it only works for tags that actually contain
|
||||||
|
# this tooling (>= v0.3.0). Tags older than that were backfilled by hand.
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
tags: ["v*"]
|
||||||
|
workflow_dispatch:
|
||||||
|
inputs:
|
||||||
|
tag:
|
||||||
|
description: "Existing tag to create a release for (e.g. v0.2.1)"
|
||||||
|
required: true
|
||||||
|
type: string
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: write
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
release:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- name: Resolve tag
|
||||||
|
id: tag
|
||||||
|
run: |
|
||||||
|
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
|
||||||
|
echo "tag=${{ inputs.tag }}" >> "$GITHUB_OUTPUT"
|
||||||
|
else
|
||||||
|
echo "tag=${GITHUB_REF#refs/tags/}" >> "$GITHUB_OUTPUT"
|
||||||
|
fi
|
||||||
|
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
ref: ${{ steps.tag.outputs.tag }}
|
||||||
|
fetch-depth: 0
|
||||||
|
|
||||||
|
- uses: actions/setup-python@v5
|
||||||
|
with:
|
||||||
|
python-version: "3.13"
|
||||||
|
|
||||||
|
# Never publish a release for code that does not pass its own tests. A
|
||||||
|
# tagged commit is what people install; it has to be at least as good as
|
||||||
|
# main.
|
||||||
|
- name: install dev deps
|
||||||
|
run: python -m pip install --upgrade pip pytest ruff
|
||||||
|
|
||||||
|
- name: ruff
|
||||||
|
run: ruff check patch tests tools
|
||||||
|
|
||||||
|
- name: pytest
|
||||||
|
run: pytest tests -q
|
||||||
|
|
||||||
|
- name: shell syntax
|
||||||
|
run: |
|
||||||
|
fail=0
|
||||||
|
while IFS= read -r f; do
|
||||||
|
bash -n "$f" || { echo "::error file=$f::bash syntax error"; fail=1; }
|
||||||
|
done < <(find . -name '*.sh' -not -path './.git/*')
|
||||||
|
exit $fail
|
||||||
|
|
||||||
|
# Catches the failure mode this repo actually had: VERSION= drifted to
|
||||||
|
# three different values across the scripts, and nothing noticed.
|
||||||
|
- name: version matches tag and CHANGELOG has a section
|
||||||
|
run: python3 tools/release_notes.py check "${{ steps.tag.outputs.tag }}"
|
||||||
|
|
||||||
|
- name: extract release notes from CHANGELOG
|
||||||
|
run: |
|
||||||
|
python3 tools/release_notes.py notes "${{ steps.tag.outputs.tag }}" > /tmp/notes.md
|
||||||
|
echo "--- release body ---"
|
||||||
|
cat /tmp/notes.md
|
||||||
|
|
||||||
|
- name: create or update the release
|
||||||
|
env:
|
||||||
|
GH_TOKEN: ${{ github.token }}
|
||||||
|
TAG: ${{ steps.tag.outputs.tag }}
|
||||||
|
run: |
|
||||||
|
# Pre-1.0 and any -rc/-beta suffix ship as prereleases, not "Latest".
|
||||||
|
prerelease=""
|
||||||
|
case "$TAG" in
|
||||||
|
*-rc*|*-beta*|*-alpha*) prerelease="--prerelease" ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
if gh release view "$TAG" >/dev/null 2>&1; then
|
||||||
|
echo "Release $TAG exists — updating notes."
|
||||||
|
gh release edit "$TAG" --notes-file /tmp/notes.md
|
||||||
|
else
|
||||||
|
# shellcheck disable=SC2086
|
||||||
|
gh release create "$TAG" \
|
||||||
|
--title "$TAG" \
|
||||||
|
--notes-file /tmp/notes.md \
|
||||||
|
$prerelease
|
||||||
|
fi
|
||||||
@@ -1,5 +1,85 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## v0.3.3 — 2026-07-13
|
||||||
|
|
||||||
|
### Security
|
||||||
|
|
||||||
|
- **The restic repository password no longer passes through a process's argv.**
|
||||||
|
`create_task.py` shelled out to `midclt call cloud_backup.create '<json>'`, and
|
||||||
|
that JSON contains the repo password — so it appeared in the process's argv,
|
||||||
|
which is world-readable via `ps`, for the duration of the call. That password is
|
||||||
|
the encryption key for the entire cloud backup repository.
|
||||||
|
|
||||||
|
It now talks to the middleware through `truenas_api_client` (the library that
|
||||||
|
backs `midclt` itself), so the password never leaves the process's memory.
|
||||||
|
|
||||||
|
- **`--password` no longer required.** Passing a secret as a CLI argument writes it
|
||||||
|
to shell history permanently. `--password-stdin` reads it from stdin, and with
|
||||||
|
neither flag the tool prompts via `getpass`. `--password` still works but now
|
||||||
|
warns.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **`uninstall.sh` could leave every patch installed.** It reverted by unmounting
|
||||||
|
the overlay — but `apply.sh` only mounts one when the target directory is
|
||||||
|
read-only. On a writable `/usr` it patches the real files in place, and uninstall
|
||||||
|
would remove the boot hook, report success, and leave the patch applied. It now
|
||||||
|
strips the appended blocks from the middleware files explicitly.
|
||||||
|
|
||||||
|
- **`create_task.py.__version__` had been stuck at `0.2.0`** for three releases.
|
||||||
|
The version-drift check added in v0.3.1 only looked at `VERSION=` in shell
|
||||||
|
scripts, so it missed the one file that actually shows a version to users
|
||||||
|
(`--version`). The check now covers `__version__` too — and caught this
|
||||||
|
immediately.
|
||||||
|
|
||||||
|
## v0.3.2 — 2026-07-13
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **`install.sh --disable-nested-snapshots` did not actually disable anything
|
||||||
|
until the next reboot.** `apply.sh` only ever *added* patches — there was no
|
||||||
|
revert path. Disabling removed the opt-in marker and then merely *skipped*
|
||||||
|
re-applying, but the overlay persists for the whole boot, so the previously
|
||||||
|
patched `plugins/cloud/{snapshot,crud}.py`, `plugins/cloud_backup/sync.py` and
|
||||||
|
`_truecloud_nested.py` were all still sitting there — and middlewared
|
||||||
|
re-imported them on the restart `install.sh` performs.
|
||||||
|
|
||||||
|
It printed *"DISABLED (stock guard restored)"* while the feature kept running.
|
||||||
|
Someone turning it off *because they were worried about it* would have believed
|
||||||
|
it was off.
|
||||||
|
|
||||||
|
`apply.sh` now actively reverts: it removes the module first (every injected
|
||||||
|
block is guarded by `if _tc_nested is not None`, so the stock guard is restored
|
||||||
|
even if a later step fails), then strips its appended blocks from the three
|
||||||
|
patched files. `restic.py` also carries a `TRUECLOUD_PATCH` block but belongs to
|
||||||
|
the *providers* module and is deliberately left alone — reverting it would break
|
||||||
|
B2 backups. `install.sh --disable` also tears down any staging tree first, since
|
||||||
|
those bind mounts pin ZFS snapshots that could otherwise never be destroyed.
|
||||||
|
|
||||||
|
Updating **without** the flag was always correct and is unchanged: the nested
|
||||||
|
module is never installed into middleware unless it is explicitly enabled.
|
||||||
|
|
||||||
|
## v0.3.1 — 2026-07-13
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **Automated releases.** Pushing a `v*` tag runs the full test suite and then
|
||||||
|
cuts a GitHub release whose body is the matching `CHANGELOG.md` section — so
|
||||||
|
release notes have exactly one source of truth, and no second place to go stale.
|
||||||
|
The workflow refuses to publish if the tests fail, if the tag does not match the
|
||||||
|
`VERSION=` declared by every script, or if the CHANGELOG has no section for it.
|
||||||
|
|
||||||
|
- **Version-drift check.** `VERSION=` had silently diverged to three different
|
||||||
|
values across `install.sh`, `uninstall.sh`, `recover.sh`, and `patch/apply.sh`,
|
||||||
|
and nothing noticed. CI now asserts every script agrees with the others and with
|
||||||
|
the newest CHANGELOG entry.
|
||||||
|
|
||||||
|
### Note
|
||||||
|
|
||||||
|
- Releases for `v0.2.0` and `v0.2.1` were backfilled — they had been tagged but
|
||||||
|
never released, so the releases page jumped v0.1.0 → v0.3.0 and hid the fix for
|
||||||
|
the boot race that took every app down.
|
||||||
|
|
||||||
## v0.3.0 — 2026-07-13
|
## v0.3.0 — 2026-07-13
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|||||||
+10
-3
@@ -18,7 +18,7 @@
|
|||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
VERSION="0.3.0"
|
VERSION="0.3.3"
|
||||||
|
|
||||||
# The directory containing install.sh is the permanent install location.
|
# The directory containing install.sh is the permanent install location.
|
||||||
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||||
@@ -160,10 +160,17 @@ case "$_nested_choice" in
|
|||||||
off)
|
off)
|
||||||
if [ -f "$_NESTED_MARKER" ]; then
|
if [ -f "$_NESTED_MARKER" ]; then
|
||||||
rm -f "$_NESTED_MARKER"
|
rm -f "$_NESTED_MARKER"
|
||||||
echo "Nested-dataset snapshots: DISABLED (stock guard restored)."
|
# Tear down any staging tree first: those bind mounts PIN their ZFS
|
||||||
|
# snapshots, so leaving them would block those snapshots from ever
|
||||||
|
# being destroyed. apply.sh (below) then reverts the patched files.
|
||||||
|
python3 "$PATCH_DIR/patch/truecloud_nested.py" cleanup || \
|
||||||
|
echo " WARNING: staging mounts remain; unmount them manually."
|
||||||
|
echo "Nested-dataset snapshots: DISABLED."
|
||||||
|
echo " apply.sh will revert the patched middleware files and the stock"
|
||||||
|
echo " guard is restored when middlewared restarts (this script does that)."
|
||||||
echo " Any task that already has snapshot=true on a nested dataset will"
|
echo " Any task that already has snapshot=true on a nested dataset will"
|
||||||
echo " fail validation on its next edit. Turn the option off on those"
|
echo " fail validation on its next edit. Turn the option off on those"
|
||||||
echo " tasks, or re-run with --enable-nested-snapshots."
|
echo " tasks first, or re-run with --enable-nested-snapshots."
|
||||||
else
|
else
|
||||||
echo "Nested-dataset snapshots: already disabled."
|
echo "Nested-dataset snapshots: already disabled."
|
||||||
fi
|
fi
|
||||||
|
|||||||
+68
-5
@@ -32,7 +32,7 @@
|
|||||||
# Derive PATCH_DIR from this script's location (parent of the patch/ directory).
|
# Derive PATCH_DIR from this script's location (parent of the patch/ directory).
|
||||||
PATCH_DIR="$(cd "$(dirname "$0")/.." && pwd)"
|
PATCH_DIR="$(cd "$(dirname "$0")/.." && pwd)"
|
||||||
LOG="$PATCH_DIR/apply.log"
|
LOG="$PATCH_DIR/apply.log"
|
||||||
VERSION="0.3.0"
|
VERSION="0.3.3"
|
||||||
|
|
||||||
# Rotate log at 512 KB to avoid unbounded growth on a system volume.
|
# Rotate log at 512 KB to avoid unbounded growth on a system volume.
|
||||||
# Keep two prior generations (.1 and .2) so the last three boots are always available.
|
# Keep two prior generations (.1 and .2) so the last three boots are always available.
|
||||||
@@ -477,6 +477,57 @@ def patch_file(path, block):
|
|||||||
with open(path, "w", encoding="utf-8") as fh:
|
with open(path, "w", encoding="utf-8") as fh:
|
||||||
fh.write(base.rstrip("\n") + "\n" + block)
|
fh.write(base.rstrip("\n") + "\n" + block)
|
||||||
|
|
||||||
|
|
||||||
|
def unpatch_file(path):
|
||||||
|
"""Strip our appended block, restoring the stock file. True if it was patched."""
|
||||||
|
try:
|
||||||
|
with open(path, encoding="utf-8") as fh:
|
||||||
|
content = fh.read()
|
||||||
|
except OSError:
|
||||||
|
return False
|
||||||
|
idx = content.find("\n# TRUECLOUD_PATCH")
|
||||||
|
if idx == -1:
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
with open(path, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(content[:idx].rstrip("\n") + "\n")
|
||||||
|
except OSError:
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def revert_nested(cloud_dir, sync_path):
|
||||||
|
"""Undo the nested patch. Returns the names of what was actually reverted.
|
||||||
|
|
||||||
|
Skipping the patch is NOT enough to disable the feature. The overlay persists
|
||||||
|
for the whole boot, so an earlier run this boot may already have written the
|
||||||
|
patched files -- and middlewared re-imports them on the restart that
|
||||||
|
install.sh performs. Without this, `install.sh --disable-nested-snapshots`
|
||||||
|
would report "disabled" while the feature kept running until the next reboot.
|
||||||
|
"""
|
||||||
|
reverted = []
|
||||||
|
|
||||||
|
# Remove the module FIRST. Every injected block is guarded by
|
||||||
|
# `if _tc_nested is not None`, so once it is gone they all no-op even if a
|
||||||
|
# later step here fails -- the guard is restored no matter what.
|
||||||
|
try:
|
||||||
|
os.unlink(os.path.join(cloud_dir, '_truecloud_nested.py'))
|
||||||
|
reverted.append('_truecloud_nested.py')
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# NB: restic.py also carries a TRUECLOUD_PATCH block, but that belongs to the
|
||||||
|
# providers module. Only these three are ours to revert.
|
||||||
|
for name, path in (
|
||||||
|
('crud.py', os.path.join(cloud_dir, 'crud.py')),
|
||||||
|
('sync.py', sync_path),
|
||||||
|
('snapshot.py', os.path.join(cloud_dir, 'snapshot.py')),
|
||||||
|
):
|
||||||
|
if unpatch_file(path):
|
||||||
|
reverted.append(name)
|
||||||
|
|
||||||
|
return reverted
|
||||||
|
|
||||||
b2_ok = restic_ok = False
|
b2_ok = restic_ok = False
|
||||||
nested_ok = False
|
nested_ok = False
|
||||||
|
|
||||||
@@ -517,16 +568,28 @@ else:
|
|||||||
# guard LAST. If anything fails partway, the guard is still in place and the
|
# guard LAST. If anything fails partway, the guard is still in place and the
|
||||||
# option stays unavailable -- we never expose "guard removed, traversal missing".
|
# option stays unavailable -- we never expose "guard removed, traversal missing".
|
||||||
nested_detail = ''
|
nested_detail = ''
|
||||||
if not nested_enabled:
|
if not nested_needed:
|
||||||
|
# Not just "skip": actively revert. The overlay lives for the whole boot, so a
|
||||||
|
# previously-applied patch is still sitting there and middlewared would
|
||||||
|
# re-import it on restart. See revert_nested().
|
||||||
|
if not nested_enabled:
|
||||||
nested_detail = 'disabled (opt-in; enable with: install.sh --enable-nested-snapshots)'
|
nested_detail = 'disabled (opt-in; enable with: install.sh --enable-nested-snapshots)'
|
||||||
print('INFO: Nested-dataset snapshot support is disabled (opt-in feature).')
|
print('INFO: Nested-dataset snapshot support is disabled (opt-in feature).')
|
||||||
print('INFO: Enable with: bash install.sh --enable-nested-snapshots')
|
elif nested_native:
|
||||||
elif nested_native:
|
|
||||||
nested_detail = 'superseded: TrueNAS handles nested-dataset snapshots natively'
|
nested_detail = 'superseded: TrueNAS handles nested-dataset snapshots natively'
|
||||||
print('INFO: Nested module skipped — TrueNAS now handles nesting natively.')
|
print('INFO: Nested module skipped — TrueNAS now handles nesting natively.')
|
||||||
elif not nested_needed:
|
else:
|
||||||
nested_detail = 'not needed'
|
nested_detail = 'not needed'
|
||||||
print('INFO: Nested module skipped.')
|
print('INFO: Nested module skipped.')
|
||||||
|
|
||||||
|
reverted = revert_nested(cloud_dir, sync_path)
|
||||||
|
if reverted:
|
||||||
|
print('OK: Reverted a previously-applied nested patch (' + ', '.join(reverted) + ').')
|
||||||
|
print(' The stock nesting guard is restored once middlewared restarts.')
|
||||||
|
nested_detail += ' — previous patch reverted'
|
||||||
|
|
||||||
|
if not nested_enabled:
|
||||||
|
print('INFO: Enable with: bash install.sh --enable-nested-snapshots')
|
||||||
else:
|
else:
|
||||||
try:
|
try:
|
||||||
snapshot_py = os.path.join(cloud_dir, 'snapshot.py')
|
snapshot_py = os.path.join(cloud_dir, 'snapshot.py')
|
||||||
|
|||||||
+70
-23
@@ -26,8 +26,9 @@ Create a task backed by a B2 credential (id=3):
|
|||||||
--credential 3 \\
|
--credential 3 \\
|
||||||
--bucket my-bucket \\
|
--bucket my-bucket \\
|
||||||
--folder backups/tank \\
|
--folder backups/tank \\
|
||||||
--password "restic-repo-password" \\
|
--password-stdin \\
|
||||||
--keep-last 14
|
--keep-last 14
|
||||||
|
(pipe the password in: echo -n "s3cret" | python3 create_task.py create ... )
|
||||||
|
|
||||||
Create a task using an S3-compatible credential (Wasabi, R2, etc.):
|
Create a task using an S3-compatible credential (Wasabi, R2, etc.):
|
||||||
python3 create_task.py create \\
|
python3 create_task.py create \\
|
||||||
@@ -36,7 +37,7 @@ Create a task using an S3-compatible credential (Wasabi, R2, etc.):
|
|||||||
--credential 5 \\
|
--credential 5 \\
|
||||||
--bucket my-bucket \\
|
--bucket my-bucket \\
|
||||||
--folder backups \\
|
--folder backups \\
|
||||||
--password "restic-repo-password"
|
--password-stdin
|
||||||
|
|
||||||
List existing TrueCloud Backup tasks:
|
List existing TrueCloud Backup tasks:
|
||||||
python3 create_task.py list-tasks
|
python3 create_task.py list-tasks
|
||||||
@@ -44,39 +45,49 @@ List existing TrueCloud Backup tasks:
|
|||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
import calendar
|
import calendar
|
||||||
|
import getpass
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
import time
|
import time
|
||||||
|
|
||||||
__version__ = "0.2.0"
|
__version__ = "0.3.3"
|
||||||
|
|
||||||
_PATCH_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
_PATCH_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
_STATUS_FILE = os.path.join(_PATCH_DIR, "hook_status.json")
|
_STATUS_FILE = os.path.join(_PATCH_DIR, "hook_status.json")
|
||||||
|
|
||||||
|
|
||||||
def midclt_call(method, *args):
|
def midclt_call(method, *args):
|
||||||
"""Call a middleware method locally via `midclt`, the supported JSON-RPC transport
|
"""Call a middleware method on the local host.
|
||||||
that replaces the deprecated /api/v2.0 REST API (removed in TrueNAS 26.04). Must run
|
|
||||||
on the TrueNAS host. Each arg is JSON-encoded (a dict for create; none for queries).
|
Uses `truenas_api_client` -- the library that backs `midclt` itself -- rather
|
||||||
Exits with a clear message on failure."""
|
than shelling out to `midclt`.
|
||||||
cmd = ["midclt", "call", method] + [json.dumps(a) for a in args]
|
|
||||||
|
This is a SECURITY requirement, not a style choice. `midclt call <method>
|
||||||
|
<json>` puts its arguments in the process's **argv**, and `cloud_backup.create`
|
||||||
|
carries the restic repository password. argv is world-readable via `ps`, so
|
||||||
|
shelling out would expose the key to the entire backup repo to every local
|
||||||
|
user for the duration of the call. Going through the client library keeps it
|
||||||
|
in this process's memory.
|
||||||
|
"""
|
||||||
try:
|
try:
|
||||||
proc = subprocess.run(cmd, capture_output=True, text=True, timeout=120)
|
from truenas_api_client import Client
|
||||||
except FileNotFoundError:
|
except ImportError:
|
||||||
print("ERROR: `midclt` not found — run this script ON the TrueNAS host.",
|
print(
|
||||||
file=sys.stderr)
|
"ERROR: `truenas_api_client` not importable — run this script ON the\n"
|
||||||
|
" TrueNAS host. (It ships with midclt.)",
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
except subprocess.SubprocessError as exc:
|
|
||||||
print(f"ERROR: midclt call failed: {exc}", file=sys.stderr)
|
try:
|
||||||
|
with Client() as client:
|
||||||
|
return client.call(method, *args)
|
||||||
|
except Exception as exc: # noqa: BLE001 - surface any middleware error verbatim
|
||||||
|
# Never echo `args` here: for cloud_backup.create it contains the password.
|
||||||
|
print(f"ERROR: {method}: {exc}", file=sys.stderr)
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
if proc.returncode != 0:
|
|
||||||
print(f"ERROR: midclt {method}: {(proc.stderr or proc.stdout).strip()}",
|
|
||||||
file=sys.stderr)
|
|
||||||
sys.exit(1)
|
|
||||||
out = proc.stdout.strip()
|
|
||||||
return json.loads(out) if out else None
|
|
||||||
|
|
||||||
|
|
||||||
# ── Sub-commands ──────────────────────────────────────────────────────────────
|
# ── Sub-commands ──────────────────────────────────────────────────────────────
|
||||||
@@ -213,6 +224,37 @@ def cmd_list_tasks(_args):
|
|||||||
print(f"{t['id']:>4} {enabled:<8} {ptype:<14} {t.get('description', '')}")
|
print(f"{t['id']:>4} {enabled:<8} {ptype:<14} {t.get('description', '')}")
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_password(args):
|
||||||
|
"""Get the restic repo password without writing it to the user's shell history.
|
||||||
|
|
||||||
|
That password is the key to the whole backup repository. `--password <secret>`
|
||||||
|
persists it in ~/.bash_history and exposes it in `ps` for the lifetime of the
|
||||||
|
shell command, so it is accepted but warned about; stdin and an interactive
|
||||||
|
prompt are the safe paths.
|
||||||
|
"""
|
||||||
|
if args.password_stdin:
|
||||||
|
if args.password:
|
||||||
|
print("ERROR: use either --password or --password-stdin, not both.",
|
||||||
|
file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
password = sys.stdin.readline().rstrip("\n")
|
||||||
|
elif args.password:
|
||||||
|
print(
|
||||||
|
"WARNING: --password puts the restic repository password in your shell\n"
|
||||||
|
" history. Prefer: echo -n 'pw' | ... --password-stdin",
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
password = args.password
|
||||||
|
else:
|
||||||
|
password = getpass.getpass("Restic repository password: ")
|
||||||
|
|
||||||
|
if not password:
|
||||||
|
print("ERROR: the restic repository password must not be empty.",
|
||||||
|
file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
return password
|
||||||
|
|
||||||
|
|
||||||
def cmd_create(args):
|
def cmd_create(args):
|
||||||
parts = args.schedule.split()
|
parts = args.schedule.split()
|
||||||
if len(parts) != 5:
|
if len(parts) != 5:
|
||||||
@@ -223,6 +265,8 @@ def cmd_create(args):
|
|||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
minute, hour, dom, month, dow = parts
|
minute, hour, dom, month, dow = parts
|
||||||
|
|
||||||
|
password = _resolve_password(args)
|
||||||
|
|
||||||
body = {
|
body = {
|
||||||
"description": args.name,
|
"description": args.name,
|
||||||
"path": args.path,
|
"path": args.path,
|
||||||
@@ -231,7 +275,7 @@ def cmd_create(args):
|
|||||||
"bucket": args.bucket,
|
"bucket": args.bucket,
|
||||||
"folder": args.folder,
|
"folder": args.folder,
|
||||||
},
|
},
|
||||||
"password": args.password,
|
"password": password,
|
||||||
"keep_last": args.keep_last,
|
"keep_last": args.keep_last,
|
||||||
"transfer_setting": args.transfer_setting,
|
"transfer_setting": args.transfer_setting,
|
||||||
"schedule": {
|
"schedule": {
|
||||||
@@ -297,8 +341,11 @@ def main():
|
|||||||
help="Bucket (S3) or container (B2) name")
|
help="Bucket (S3) or container (B2) name")
|
||||||
c.add_argument("--folder", default="",
|
c.add_argument("--folder", default="",
|
||||||
help="Path within the bucket (default: root)")
|
help="Path within the bucket (default: root)")
|
||||||
c.add_argument("--password", required=True,
|
c.add_argument("--password", default=None,
|
||||||
help="Restic repository encryption password (choose a strong one)")
|
help="Restic repository password. UNSAFE: it lands in your shell "
|
||||||
|
"history. Prefer --password-stdin, or omit both and be prompted.")
|
||||||
|
c.add_argument("--password-stdin", action="store_true",
|
||||||
|
help="Read the restic repository password from stdin (recommended)")
|
||||||
c.add_argument("--keep-last", type=int, default=14, metavar="N",
|
c.add_argument("--keep-last", type=int, default=14, metavar="N",
|
||||||
help="Snapshots to retain after each run (default: 14)")
|
help="Snapshots to retain after each run (default: 14)")
|
||||||
c.add_argument("--schedule", default="0 2 * * *",
|
c.add_argument("--schedule", default="0 2 * * *",
|
||||||
|
|||||||
+1
-1
@@ -17,7 +17,7 @@
|
|||||||
# bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh
|
# bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh
|
||||||
# systemctl restart middlewared
|
# systemctl restart middlewared
|
||||||
|
|
||||||
VERSION="0.3.0"
|
VERSION="0.3.3"
|
||||||
|
|
||||||
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||||
|
|
||||||
|
|||||||
@@ -263,10 +263,57 @@ class TestOptIn:
|
|||||||
def test_patching_is_skipped_entirely_when_disabled(self):
|
def test_patching_is_skipped_entirely_when_disabled(self):
|
||||||
# The guard-relaxing crud.py patch must be inside the enabled branch.
|
# The guard-relaxing crud.py patch must be inside the enabled branch.
|
||||||
src = heredoc_source()
|
src = heredoc_source()
|
||||||
gate = src.index("if not nested_enabled:")
|
gate = src.index("if not nested_needed:")
|
||||||
crud = src.index("patch_file(crud_py, CRUD_BLOCK)")
|
crud = src.index("patch_file(crud_py, CRUD_BLOCK)")
|
||||||
assert gate < crud, "crud.py patch must sit inside the opt-in branch"
|
assert gate < crud, "crud.py patch must sit inside the opt-in branch"
|
||||||
|
|
||||||
|
def test_disabling_REVERTS_the_patch_rather_than_merely_skipping_it(self):
|
||||||
|
"""Skipping is not disabling.
|
||||||
|
|
||||||
|
The overlay persists for the whole boot, so a patch applied by an earlier
|
||||||
|
run this boot is still on disk — and middlewared re-imports it on the
|
||||||
|
restart install.sh performs. Without an active revert,
|
||||||
|
`--disable-nested-snapshots` reports "disabled" while the feature keeps
|
||||||
|
running until the next reboot.
|
||||||
|
"""
|
||||||
|
src = heredoc_source()
|
||||||
|
assert "def unpatch_file(" in src
|
||||||
|
assert "def revert_nested(" in src
|
||||||
|
# The revert must run on every not-needed path (opt-out, superseded).
|
||||||
|
gate = src.index("if not nested_needed:")
|
||||||
|
revert = src.index("reverted = revert_nested(")
|
||||||
|
patch = src.index("patch_file(crud_py, CRUD_BLOCK)")
|
||||||
|
assert gate < revert < patch, "revert belongs in the not-needed branch"
|
||||||
|
|
||||||
|
def test_revert_removes_the_module_before_unpatching_files(self):
|
||||||
|
# Every injected block is guarded by `if _tc_nested is not None`, so
|
||||||
|
# deleting the module first means the guard is restored even if a later
|
||||||
|
# unpatch step fails.
|
||||||
|
src = heredoc_source()
|
||||||
|
body = src[src.index("def revert_nested("):src.index("def patch_file(") if
|
||||||
|
src.index("def patch_file(") > src.index("def revert_nested(") else len(src)]
|
||||||
|
body = src[src.index("def revert_nested("):]
|
||||||
|
body = body[:body.index("\n\n\n")] if "\n\n\n" in body else body
|
||||||
|
assert body.index("_truecloud_nested.py") < body.index("crud.py")
|
||||||
|
|
||||||
|
def test_revert_never_touches_the_providers_patch(self):
|
||||||
|
# restic.py also carries a TRUECLOUD_PATCH block, but it belongs to the
|
||||||
|
# providers module. Reverting it would silently break B2 backups.
|
||||||
|
src = heredoc_source()
|
||||||
|
body = src[src.index("def revert_nested("):]
|
||||||
|
body = body[:body.index("return reverted")]
|
||||||
|
# Comments legitimately *mention* restic.py to explain why it is excluded;
|
||||||
|
# what matters is that no code line touches it.
|
||||||
|
code = "\n".join(
|
||||||
|
ln for ln in body.splitlines() if not ln.lstrip().startswith("#")
|
||||||
|
)
|
||||||
|
assert "restic" not in code
|
||||||
|
assert "b2.py" not in code
|
||||||
|
# It must only ever revert these three, plus the module itself.
|
||||||
|
assert "crud.py" in code
|
||||||
|
assert "sync_path" in code
|
||||||
|
assert "snapshot.py" in code
|
||||||
|
|
||||||
|
|
||||||
def test_guard_is_relaxed_only_after_traversal_is_installed():
|
def test_guard_is_relaxed_only_after_traversal_is_installed():
|
||||||
# Ordering in apply.sh is a safety property: copy module -> patch snapshot.py
|
# Ordering in apply.sh is a safety property: copy module -> patch snapshot.py
|
||||||
|
|||||||
@@ -0,0 +1,107 @@
|
|||||||
|
"""Tests for create_task.py, focused on the restic repository password.
|
||||||
|
|
||||||
|
That password is the encryption key for the entire cloud backup repository. It
|
||||||
|
used to travel through `midclt call cloud_backup.create '<json>'` -- i.e. through
|
||||||
|
the subprocess's **argv**, which is world-readable via `ps` -- and `--password`
|
||||||
|
wrote it into the user's shell history forever.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
|
import io
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import types
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
SPEC = importlib.util.spec_from_file_location(
|
||||||
|
"create_task",
|
||||||
|
os.path.join(os.path.dirname(__file__), "..", "patch", "create_task.py"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def load():
|
||||||
|
mod = importlib.util.module_from_spec(SPEC)
|
||||||
|
SPEC.loader.exec_module(mod)
|
||||||
|
return mod
|
||||||
|
|
||||||
|
|
||||||
|
class Args:
|
||||||
|
def __init__(self, password=None, password_stdin=False):
|
||||||
|
self.password = password
|
||||||
|
self.password_stdin = password_stdin
|
||||||
|
|
||||||
|
|
||||||
|
class TestPasswordNeverReachesArgv:
|
||||||
|
"""The whole reason this module talks to the client library."""
|
||||||
|
|
||||||
|
def test_midclt_call_spawns_no_subprocess(self):
|
||||||
|
import inspect
|
||||||
|
|
||||||
|
src = inspect.getsource(load().midclt_call)
|
||||||
|
assert "subprocess" not in src, (
|
||||||
|
"shelling out to `midclt` puts cloud_backup.create's JSON -- including "
|
||||||
|
"the restic repo password -- into argv, which any local user can read "
|
||||||
|
"with ps"
|
||||||
|
)
|
||||||
|
assert "truenas_api_client" in src
|
||||||
|
|
||||||
|
def test_errors_never_echo_the_call_arguments(self):
|
||||||
|
# A failed cloud_backup.create must not print the body back at the user;
|
||||||
|
# it contains the password.
|
||||||
|
import inspect
|
||||||
|
|
||||||
|
src = inspect.getsource(load().midclt_call)
|
||||||
|
assert "{args}" not in src
|
||||||
|
assert "args!r" not in src
|
||||||
|
|
||||||
|
|
||||||
|
class TestResolvePassword:
|
||||||
|
def test_reads_from_stdin(self, monkeypatch):
|
||||||
|
mod = load()
|
||||||
|
monkeypatch.setattr(sys, "stdin", io.StringIO("s3cret\n"))
|
||||||
|
assert mod._resolve_password(Args(password_stdin=True)) == "s3cret"
|
||||||
|
|
||||||
|
def test_strips_only_the_trailing_newline(self, monkeypatch):
|
||||||
|
# A password may legitimately contain spaces; only the line ending goes.
|
||||||
|
mod = load()
|
||||||
|
monkeypatch.setattr(sys, "stdin", io.StringIO(" pass word \n"))
|
||||||
|
assert mod._resolve_password(Args(password_stdin=True)) == " pass word "
|
||||||
|
|
||||||
|
def test_cli_password_still_works_but_warns(self, monkeypatch, capsys):
|
||||||
|
mod = load()
|
||||||
|
pw = mod._resolve_password(Args(password="cli-secret"))
|
||||||
|
assert pw == "cli-secret"
|
||||||
|
assert "shell" in capsys.readouterr().err.lower(), "must warn about history"
|
||||||
|
|
||||||
|
def test_prompts_when_neither_flag_given(self, monkeypatch):
|
||||||
|
mod = load()
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mod, "getpass", types.SimpleNamespace(getpass=lambda _p: "prompted")
|
||||||
|
)
|
||||||
|
assert mod._resolve_password(Args()) == "prompted"
|
||||||
|
|
||||||
|
def test_rejects_both_flags(self, monkeypatch):
|
||||||
|
mod = load()
|
||||||
|
monkeypatch.setattr(sys, "stdin", io.StringIO("x\n"))
|
||||||
|
with pytest.raises(SystemExit):
|
||||||
|
mod._resolve_password(Args(password="a", password_stdin=True))
|
||||||
|
|
||||||
|
def test_rejects_an_empty_password(self, monkeypatch):
|
||||||
|
# An empty restic password would silently create an unencrypted-ish repo.
|
||||||
|
mod = load()
|
||||||
|
monkeypatch.setattr(sys, "stdin", io.StringIO("\n"))
|
||||||
|
with pytest.raises(SystemExit):
|
||||||
|
mod._resolve_password(Args(password_stdin=True))
|
||||||
|
|
||||||
|
|
||||||
|
class TestVersion:
|
||||||
|
def test_version_is_not_stale(self):
|
||||||
|
# __version__ sat at 0.2.0 through three releases because the drift check
|
||||||
|
# only looked at VERSION= in shell scripts. It covers this file now.
|
||||||
|
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "tools"))
|
||||||
|
from release_notes import normalise, script_versions
|
||||||
|
|
||||||
|
repo = os.path.join(os.path.dirname(__file__), "..")
|
||||||
|
versions = {normalise(v) for v in script_versions(repo).values()}
|
||||||
|
assert len(versions) == 1, f"version drift: {sorted(versions)}"
|
||||||
@@ -0,0 +1,122 @@
|
|||||||
|
"""Tests for the release automation.
|
||||||
|
|
||||||
|
The release workflow refuses to publish unless these hold, so a bad tag fails
|
||||||
|
loudly in CI instead of shipping a release whose notes are empty, wrong, or whose
|
||||||
|
scripts announce a different version than the tag.
|
||||||
|
|
||||||
|
That last one is not hypothetical: VERSION= drifted to three different values
|
||||||
|
across install.sh / uninstall.sh / recover.sh / apply.sh and nothing noticed.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "tools"))
|
||||||
|
|
||||||
|
from release_notes import ( # noqa: E402
|
||||||
|
changelog_versions,
|
||||||
|
check,
|
||||||
|
extract_notes,
|
||||||
|
normalise,
|
||||||
|
script_versions,
|
||||||
|
)
|
||||||
|
|
||||||
|
REPO = os.path.join(os.path.dirname(__file__), "..")
|
||||||
|
|
||||||
|
SAMPLE = """\
|
||||||
|
# Changelog
|
||||||
|
|
||||||
|
## v0.3.0 — 2026-07-13
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- the new thing
|
||||||
|
|
||||||
|
## v0.2.1 — 2026-07-09
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- the old thing
|
||||||
|
|
||||||
|
## v0.2.0 — 2026-07-08
|
||||||
|
|
||||||
|
- first
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
class TestExtractNotes:
|
||||||
|
def test_returns_only_that_versions_body(self):
|
||||||
|
body = extract_notes(SAMPLE, "v0.3.0")
|
||||||
|
assert "the new thing" in body
|
||||||
|
assert "the old thing" not in body
|
||||||
|
# The version heading itself is dropped (GitHub renders its own title),
|
||||||
|
# but sub-headings like "### Added" must survive.
|
||||||
|
assert not body.startswith("## v")
|
||||||
|
assert body.startswith("### Added")
|
||||||
|
|
||||||
|
def test_stops_at_the_next_version_heading(self):
|
||||||
|
body = extract_notes(SAMPLE, "v0.2.1")
|
||||||
|
assert "the old thing" in body
|
||||||
|
assert "first" not in body
|
||||||
|
|
||||||
|
def test_last_section_runs_to_end_of_file(self):
|
||||||
|
assert "first" in extract_notes(SAMPLE, "v0.2.0")
|
||||||
|
|
||||||
|
def test_accepts_the_tag_with_or_without_the_v(self):
|
||||||
|
assert extract_notes(SAMPLE, "0.3.0") == extract_notes(SAMPLE, "v0.3.0")
|
||||||
|
|
||||||
|
def test_unknown_version_raises_rather_than_returning_empty(self):
|
||||||
|
# An empty release body is worse than a failed release.
|
||||||
|
with pytest.raises(KeyError, match="no section"):
|
||||||
|
extract_notes(SAMPLE, "v9.9.9")
|
||||||
|
|
||||||
|
|
||||||
|
class TestChangelogVersions:
|
||||||
|
def test_lists_versions_newest_first(self):
|
||||||
|
assert changelog_versions(SAMPLE) == ["0.3.0", "0.2.1", "0.2.0"]
|
||||||
|
|
||||||
|
|
||||||
|
class TestAgainstTheRealRepo:
|
||||||
|
"""These run against the actual files, so drift breaks the build."""
|
||||||
|
|
||||||
|
def test_every_script_declares_a_version(self):
|
||||||
|
from release_notes import VERSIONED_FILES
|
||||||
|
|
||||||
|
found = script_versions(REPO)
|
||||||
|
missing = [f for f in VERSIONED_FILES if f not in found]
|
||||||
|
assert not missing, f"no VERSION= in: {missing}"
|
||||||
|
|
||||||
|
def test_all_scripts_agree_on_the_version(self):
|
||||||
|
versions = {normalise(v) for v in script_versions(REPO).values()}
|
||||||
|
assert len(versions) == 1, f"scripts disagree on version: {sorted(versions)}"
|
||||||
|
|
||||||
|
def test_the_current_version_has_a_changelog_section(self):
|
||||||
|
version = next(iter({normalise(v) for v in script_versions(REPO).values()}))
|
||||||
|
with open(os.path.join(REPO, "CHANGELOG.md"), encoding="utf-8") as fh:
|
||||||
|
body = extract_notes(fh.read(), version)
|
||||||
|
assert body, f"CHANGELOG.md has no content for v{version}"
|
||||||
|
|
||||||
|
def test_the_current_version_is_the_newest_changelog_entry(self):
|
||||||
|
version = next(iter({normalise(v) for v in script_versions(REPO).values()}))
|
||||||
|
with open(os.path.join(REPO, "CHANGELOG.md"), encoding="utf-8") as fh:
|
||||||
|
newest = changelog_versions(fh.read())[0]
|
||||||
|
assert newest == version, (
|
||||||
|
f"scripts say v{version} but the newest CHANGELOG entry is v{newest}"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_check_passes_for_the_current_version(self):
|
||||||
|
version = next(iter({normalise(v) for v in script_versions(REPO).values()}))
|
||||||
|
assert check(version, REPO) == []
|
||||||
|
|
||||||
|
|
||||||
|
class TestCheckCatchesMistakes:
|
||||||
|
def test_reports_a_tag_that_no_script_matches(self):
|
||||||
|
problems = check("v9.9.9", REPO)
|
||||||
|
assert problems
|
||||||
|
assert any("declares VERSION" in p for p in problems)
|
||||||
|
|
||||||
|
def test_reports_a_missing_changelog_section(self):
|
||||||
|
problems = check("v9.9.9", REPO)
|
||||||
|
assert any("no section" in p for p in problems)
|
||||||
@@ -0,0 +1,160 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Extract one version's section from CHANGELOG.md, and check version consistency.
|
||||||
|
|
||||||
|
Used by .github/workflows/release.yml so a release's body is always the changelog
|
||||||
|
entry -- there is no second place to write release notes, and therefore no second
|
||||||
|
place for them to be wrong.
|
||||||
|
|
||||||
|
python3 tools/release_notes.py notes v0.3.0 # -> the section body
|
||||||
|
python3 tools/release_notes.py version # -> version per the scripts
|
||||||
|
python3 tools/release_notes.py check v0.3.0 # -> exit 1 on any mismatch
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
|
||||||
|
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
CHANGELOG = os.path.join(ROOT, "CHANGELOG.md")
|
||||||
|
|
||||||
|
# Everything that announces a version must agree with everything else. They drifted
|
||||||
|
# to three different values once (0.0.4 / 0.2.1) before anything checked them --
|
||||||
|
# and create_task.py's __version__ then sat at 0.2.0 through three more releases,
|
||||||
|
# because the first version of this check only looked at VERSION= in shell scripts.
|
||||||
|
VERSIONED_FILES = [
|
||||||
|
"install.sh",
|
||||||
|
"uninstall.sh",
|
||||||
|
"recover.sh",
|
||||||
|
os.path.join("patch", "apply.sh"),
|
||||||
|
os.path.join("patch", "create_task.py"), # exposes `--version` to users
|
||||||
|
]
|
||||||
|
|
||||||
|
# `VERSION="x"` (shell) or `__version__ = "x"` (python).
|
||||||
|
_VERSION_RE = re.compile(r'^(?:VERSION=|__version__\s*=\s*)"([^"]+)"', re.M)
|
||||||
|
_HEADING_RE = re.compile(r"^##\s+v?(\d+\.\d+\.\d+[^\s]*)", re.M)
|
||||||
|
|
||||||
|
|
||||||
|
def normalise(v: str) -> str:
|
||||||
|
return v.strip().lstrip("v")
|
||||||
|
|
||||||
|
|
||||||
|
def script_versions(root: str = ROOT) -> dict[str, str]:
|
||||||
|
"""VERSION= as declared by each script."""
|
||||||
|
found = {}
|
||||||
|
for rel in VERSIONED_FILES:
|
||||||
|
path = os.path.join(root, rel)
|
||||||
|
try:
|
||||||
|
with open(path, encoding="utf-8") as fh:
|
||||||
|
m = _VERSION_RE.search(fh.read())
|
||||||
|
except OSError:
|
||||||
|
continue
|
||||||
|
if m:
|
||||||
|
found[rel] = m.group(1)
|
||||||
|
return found
|
||||||
|
|
||||||
|
|
||||||
|
def changelog_versions(text: str) -> list[str]:
|
||||||
|
"""Versions with a section in the changelog, newest first."""
|
||||||
|
return [normalise(v) for v in _HEADING_RE.findall(text)]
|
||||||
|
|
||||||
|
|
||||||
|
def extract_notes(text: str, version: str) -> str:
|
||||||
|
"""The body of one version's section, without its heading.
|
||||||
|
|
||||||
|
Raises KeyError if the version has no section -- a release with an empty or
|
||||||
|
wrong body is worse than a failed release.
|
||||||
|
"""
|
||||||
|
want = normalise(version)
|
||||||
|
lines = text.splitlines()
|
||||||
|
|
||||||
|
start = None
|
||||||
|
for i, line in enumerate(lines):
|
||||||
|
m = _HEADING_RE.match(line)
|
||||||
|
if m and normalise(m.group(1)) == want:
|
||||||
|
start = i + 1
|
||||||
|
break
|
||||||
|
if start is None:
|
||||||
|
raise KeyError(f"CHANGELOG.md has no section for v{want}")
|
||||||
|
|
||||||
|
end = len(lines)
|
||||||
|
for i in range(start, len(lines)):
|
||||||
|
if _HEADING_RE.match(lines[i]):
|
||||||
|
end = i
|
||||||
|
break
|
||||||
|
|
||||||
|
return "\n".join(lines[start:end]).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def check(version: str, root: str = ROOT) -> list[str]:
|
||||||
|
"""Every reason this version is not releasable. Empty list means it is."""
|
||||||
|
want = normalise(version)
|
||||||
|
problems = []
|
||||||
|
|
||||||
|
versions = script_versions(root)
|
||||||
|
for rel, got in sorted(versions.items()):
|
||||||
|
if normalise(got) != want:
|
||||||
|
problems.append(f"{rel} declares VERSION={got!r}, tag is v{want}")
|
||||||
|
missing = [r for r in VERSIONED_FILES if r not in versions]
|
||||||
|
for rel in missing:
|
||||||
|
problems.append(f"{rel} has no VERSION= line")
|
||||||
|
|
||||||
|
try:
|
||||||
|
with open(os.path.join(root, "CHANGELOG.md"), encoding="utf-8") as fh:
|
||||||
|
text = fh.read()
|
||||||
|
except OSError as e:
|
||||||
|
problems.append(f"cannot read CHANGELOG.md: {e}")
|
||||||
|
return problems
|
||||||
|
|
||||||
|
try:
|
||||||
|
body = extract_notes(text, want)
|
||||||
|
except KeyError as e:
|
||||||
|
problems.append(str(e))
|
||||||
|
else:
|
||||||
|
if not body:
|
||||||
|
problems.append(f"CHANGELOG.md section for v{want} is empty")
|
||||||
|
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv):
|
||||||
|
if len(argv) < 2:
|
||||||
|
print(__doc__, file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
cmd = argv[1]
|
||||||
|
|
||||||
|
if cmd == "version":
|
||||||
|
versions = set(map(normalise, script_versions().values()))
|
||||||
|
if len(versions) != 1:
|
||||||
|
print(f"scripts disagree on version: {sorted(versions)}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
print(versions.pop())
|
||||||
|
return 0
|
||||||
|
|
||||||
|
if len(argv) < 3:
|
||||||
|
print(f"usage: {argv[0]} {cmd} <version>", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
version = argv[2]
|
||||||
|
|
||||||
|
if cmd == "notes":
|
||||||
|
with open(CHANGELOG, encoding="utf-8") as fh:
|
||||||
|
print(extract_notes(fh.read(), version))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
if cmd == "check":
|
||||||
|
problems = check(version)
|
||||||
|
for p in problems:
|
||||||
|
print(f"::error::{p}")
|
||||||
|
if problems:
|
||||||
|
return 1
|
||||||
|
print(f"v{normalise(version)} is consistent across scripts and CHANGELOG")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
print(f"unknown command: {cmd}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main(sys.argv))
|
||||||
+58
-1
@@ -3,7 +3,7 @@
|
|||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
VERSION="0.3.0"
|
VERSION="0.3.3"
|
||||||
|
|
||||||
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||||
_HOOK_COMMENT='TrueCloud provider patch (S3/B2)'
|
_HOOK_COMMENT='TrueCloud provider patch (S3/B2)'
|
||||||
@@ -91,6 +91,63 @@ if [ "$_ov_found" -eq 0 ]; then
|
|||||||
fi
|
fi
|
||||||
echo ""
|
echo ""
|
||||||
|
|
||||||
|
# ── Revert file-level patches ─────────────────────────────────────────────────
|
||||||
|
# Unmounting the overlay is what normally reverts everything — the lower layer is
|
||||||
|
# the untouched /usr. But apply.sh only mounts an overlay when the directory is
|
||||||
|
# read-only; on a writable /usr it patches the real files in place. Uninstall
|
||||||
|
# would then remove the boot hook and report success while leaving every patch
|
||||||
|
# applied. Strip our appended blocks explicitly.
|
||||||
|
|
||||||
|
echo "Reverting any file-level patches ..."
|
||||||
|
python3 - <<'PYEOF'
|
||||||
|
import os
|
||||||
|
|
||||||
|
try:
|
||||||
|
import middlewared
|
||||||
|
except ImportError:
|
||||||
|
print(" middlewared not importable — nothing to revert.")
|
||||||
|
raise SystemExit(0)
|
||||||
|
|
||||||
|
mw = os.path.dirname(os.path.abspath(middlewared.__file__))
|
||||||
|
targets = [
|
||||||
|
os.path.join(mw, "rclone", "remote", "b2.py"),
|
||||||
|
os.path.join(mw, "plugins", "cloud_backup", "restic.py"),
|
||||||
|
os.path.join(mw, "plugins", "cloud_backup", "sync.py"),
|
||||||
|
os.path.join(mw, "plugins", "cloud", "crud.py"),
|
||||||
|
os.path.join(mw, "plugins", "cloud", "snapshot.py"),
|
||||||
|
]
|
||||||
|
|
||||||
|
reverted = []
|
||||||
|
for path in targets:
|
||||||
|
try:
|
||||||
|
with open(path, encoding="utf-8") as fh:
|
||||||
|
content = fh.read()
|
||||||
|
except OSError:
|
||||||
|
continue
|
||||||
|
idx = content.find("\n# TRUECLOUD_PATCH")
|
||||||
|
if idx == -1:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
with open(path, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(content[:idx].rstrip("\n") + "\n")
|
||||||
|
reverted.append(os.path.basename(path))
|
||||||
|
except OSError as e:
|
||||||
|
print(f" WARNING: could not revert {path}: {e}")
|
||||||
|
|
||||||
|
nested = os.path.join(mw, "plugins", "cloud", "_truecloud_nested.py")
|
||||||
|
try:
|
||||||
|
os.unlink(nested)
|
||||||
|
reverted.append("_truecloud_nested.py")
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
if reverted:
|
||||||
|
print(" Reverted: " + ", ".join(reverted))
|
||||||
|
else:
|
||||||
|
print(" Nothing to revert (overlay already removed, or never patched).")
|
||||||
|
PYEOF
|
||||||
|
echo ""
|
||||||
|
|
||||||
# ── Unmount nested-snapshot staging trees ─────────────────────────────────────
|
# ── Unmount nested-snapshot staging trees ─────────────────────────────────────
|
||||||
# These bind mounts pin their ZFS snapshots, so they must go before anything
|
# These bind mounts pin their ZFS snapshots, so they must go before anything
|
||||||
# tries to destroy those snapshots. Deepest first.
|
# tries to destroy those snapshots. Deepest first.
|
||||||
|
|||||||
Reference in New Issue
Block a user