Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 flan
|
345741e1f1 | ||
|
flan
|
092bdeae29 |
@@ -1,5 +1,61 @@
|
||||
# Changelog
|
||||
|
||||
## v0.4.2 — 2026-07-13
|
||||
|
||||
### Docs
|
||||
|
||||
- **The Updating section never said how to *get* `update.sh`.** It ships inside the
|
||||
patch, so a clone older than v0.4.0 doesn't have it — the docs told you to run a
|
||||
script you didn't have. There is now an explicit bootstrap step (`git pull &&
|
||||
bash install.sh`, once), including the fix for the *"insufficient permission for
|
||||
adding an object to repository database"* failure that past `sudo git pull`s
|
||||
cause.
|
||||
|
||||
- **`After a TrueNAS update` rewritten.** It didn't explain that the patch
|
||||
re-applies itself at every boot (so you never reinstall), and it didn't say what
|
||||
each failure actually costs you. "Fail-safe" means *the box stays up* — not that
|
||||
your backups keep running. A `[FAIL] providers` is a **broken backup**, and the
|
||||
docs now say so rather than implying everything degrades gracefully.
|
||||
|
||||
- Added a repo map. `patch/mw_patch.py` and `tools/release_notes.py` were
|
||||
documented nowhere.
|
||||
|
||||
- `Development` told you to run `ruff check patch tests`, which misses `tools/`.
|
||||
|
||||
- Every command and file path in the README is now verified to exist and run.
|
||||
|
||||
## v0.4.1 — 2026-07-13
|
||||
|
||||
### Fixed
|
||||
|
||||
- **`update.sh` would have picked a release candidate as "the newest release".**
|
||||
Git's version sort ranks `v0.5.0-rc1` *above* `v0.5.0` (verified), and the
|
||||
release workflow deliberately supports rc/beta tags — so an RC would have been
|
||||
installed as though it were the latest stable. Tag selection is now filtered to
|
||||
plain `vX.Y.Z`.
|
||||
|
||||
- **`update.sh` would have died mid-update on an untracked file.** The dirty-tree
|
||||
guard uses `--untracked-files=no`, so an untracked file that the *target* tracks
|
||||
slipped past it — and `git checkout` then aborts. Under `set -e` the script died
|
||||
with a raw git error, *after* recording the rollback point. This is exactly what
|
||||
blocked a pull on a real box (a hand-copied `patch/wait_restart.sh`). It now
|
||||
detects the collision up front and names the files. Gitignored files are
|
||||
correctly *not* treated as blockers — git overwrites those silently.
|
||||
|
||||
Special case: if `update.sh` *itself* is the blocker, you hand-copied it in to
|
||||
bootstrap — and "delete `update.sh`, then re-run `update.sh`" is impossible. It
|
||||
now says so and prints the git commands that bootstrap it properly.
|
||||
|
||||
- **`--rollback` skipped that check entirely**, so it would have hit the identical
|
||||
failure. The check is now a shared function used by both paths, and rollback also
|
||||
validates that the recorded revision still exists (history can be rewritten).
|
||||
|
||||
- `install.sh`'s `chmod` aborted under `set -e` if any listed file was missing. The
|
||||
file set changes between versions, so `update.sh --rollback` to an older revision
|
||||
must not be killed by a filename this version happens to know about.
|
||||
|
||||
- `--to` with no value was silently ignored and fell back to the default target.
|
||||
|
||||
## v0.4.0 — 2026-07-13
|
||||
|
||||
### Added
|
||||
|
||||
@@ -46,6 +46,22 @@ The patch is two independent modules — **providers** (B2/S3) and **nested**
|
||||
(snapshots on nested datasets) — and each retires on its own once TrueNAS ships
|
||||
that capability natively. See [Native support](#if-truenas-adds-native-support).
|
||||
|
||||
## What's in the repo
|
||||
|
||||
| Path | What it is |
|
||||
|---|---|
|
||||
| `install.sh` | Registers the PREINIT boot hook, applies the patch, restarts middlewared. Also `--enable/--disable-nested-snapshots`. |
|
||||
| `update.sh` | Fetch a newer **release** and apply it. `--check`, `--rollback`, `--to`, `--main`. |
|
||||
| `uninstall.sh` | Remove everything: boot hook, patched files, UI bundle, staging mounts. |
|
||||
| `recover.sh` | Emergency: set the kill switch and restart middlewared against stock files. |
|
||||
| `patch/apply.sh` | The PREINIT script. Runs at **every boot**; re-applies the patch into a fresh overlay. |
|
||||
| `patch/mw_patch.py` | The one implementation of apply/revert for the `TRUECLOUD_PATCH` blocks. Used by `apply.sh` *and* `uninstall.sh`. |
|
||||
| `patch/truecloud_nested.py` | Nested-dataset staging: plan, mount, verify, tear down, sweep snapshots. Also `… cleanup` as a CLI. |
|
||||
| `patch/patch_ui.py` | Widens the Angular credential dropdown. Refuses to write a bundle whose parens it unbalanced. |
|
||||
| `patch/create_task.py` | Create TrueCloud tasks with S3/B2 credentials; `verify` the patch state. |
|
||||
| `patch/wait_restart.sh` | Waits for boot to actually settle before restarting middlewared. |
|
||||
| `tools/release_notes.py` | Extracts a version's CHANGELOG section; enforces version consistency. Used by CI. |
|
||||
|
||||
## Development
|
||||
|
||||
Parts of this project were written with AI assistance (Claude). All of it is
|
||||
@@ -54,14 +70,24 @@ make that review meaningful. Bugs are mine.
|
||||
|
||||
```bash
|
||||
pip install pytest ruff
|
||||
ruff check patch tests
|
||||
ruff check patch tests tools
|
||||
pytest tests
|
||||
```
|
||||
|
||||
CI runs shellcheck, `bash -n`, ruff, and pytest on Python 3.11–3.13. The tests
|
||||
include a pass that `compile()`s the `*_BLOCK` strings in `patch/apply.sh` —
|
||||
those are Python source appended into live `middlewared` modules, so a syntax
|
||||
error there would break the box at boot.
|
||||
CI runs shellcheck, `bash -n`, ruff, and pytest on Python 3.11–3.13. Two checks
|
||||
are worth calling out, because nothing else would catch what they catch:
|
||||
|
||||
- The tests **`compile()` the `*_BLOCK` strings** in `patch/apply.sh`. Those are
|
||||
Python source appended into live `middlewared` modules — a syntax error there
|
||||
breaks the box at boot, and they're string literals, so nothing else type-checks
|
||||
them.
|
||||
- CI asserts **every script declares the same version**, and that it matches the
|
||||
newest CHANGELOG entry. `VERSION=` had silently drifted to three different
|
||||
values across the scripts before anything checked.
|
||||
|
||||
Releases are automated: push a `vX.Y.Z` tag and the workflow runs the full suite,
|
||||
verifies the version matches, and cuts a GitHub release whose body **is** the
|
||||
matching `CHANGELOG.md` section — one source of truth for release notes.
|
||||
|
||||
---
|
||||
|
||||
@@ -342,28 +368,71 @@ Refresh your browser. S3 and B2 credentials now appear in the
|
||||
## Updating
|
||||
|
||||
```bash
|
||||
cd /mnt/tank/truenas-truecloud-patch
|
||||
bash update.sh # to the newest release, with a confirmation
|
||||
bash update.sh --check # show what would happen; change nothing
|
||||
bash update.sh --rollback # undo the last update
|
||||
```
|
||||
|
||||
It preserves your nested-snapshot opt-in setting, shows you the commits and
|
||||
release notes you don't have yet, and asks before changing anything. It records
|
||||
the previous revision *before* moving, so `--rollback` works even if `install.sh`
|
||||
dies halfway.
|
||||
| | |
|
||||
|---|---|
|
||||
| `bash update.sh` | Update to the newest release tag. Shows what's coming, asks first. |
|
||||
| `bash update.sh --check` | Show what *would* happen. Changes nothing. |
|
||||
| `bash update.sh --rollback` | Undo the last update. |
|
||||
| `bash update.sh --to v0.3.5` | Go to a specific tag or commit. |
|
||||
| `bash update.sh --main` | Track **unreleased** `main`. You're on your own. |
|
||||
| `bash update.sh --yes` | Skip the confirmation (for a scripted, *attended* run). |
|
||||
|
||||
**Run it by hand. Never from cron or a systemd timer.** This patch injects Python
|
||||
into `middlewared` and re-applies itself at every boot, so an unattended pull would
|
||||
let any bad upstream commit reach your box with no human in the loop and take
|
||||
effect on the next reboot. v0.0.4 shipped exactly such a bug and took every app on
|
||||
the box down. The manual step *is* the safety gate — if you want convenience, watch
|
||||
the [releases](https://github.com/sudolulo/truenas-truecloud-patch/releases) feed,
|
||||
Run it as **root** — it calls `install.sh`, which needs to reload middlewared.
|
||||
|
||||
### First time: bootstrapping `update.sh`
|
||||
|
||||
`update.sh` ships *inside* the patch, so a clone older than v0.4.0 doesn't have it
|
||||
yet. Bootstrap it once with git:
|
||||
|
||||
```bash
|
||||
cd /mnt/tank/truenas-truecloud-patch
|
||||
git pull # or: git checkout v0.4.1
|
||||
bash install.sh
|
||||
```
|
||||
|
||||
Every update after that is just `bash update.sh`.
|
||||
|
||||
> If a plain `git pull` fails with *"insufficient permission for adding an object
|
||||
> to repository database"*, past `sudo git pull`s left root-owned objects in
|
||||
> `.git`. Fix it once as root: `chown -R <you>:<you> .git`. (`update.sh` repairs
|
||||
> this automatically from then on.)
|
||||
|
||||
### What it does for you
|
||||
|
||||
- **Preserves your nested-snapshot opt-in setting** — updating never flips it.
|
||||
- **Shows the commits and release notes you don't have**, then asks before moving.
|
||||
- **Records the previous revision *before* checking out**, so `--rollback` works
|
||||
even if `install.sh` dies halfway through.
|
||||
- **Refuses to run over a dirty working tree**, rather than merging across
|
||||
hand-edited or scp'd files and losing them.
|
||||
- **Detects untracked files that would be clobbered** by the checkout and names
|
||||
them, instead of dying on a raw git error mid-update.
|
||||
- **Repairs `.git` ownership** left root-owned by past `sudo git pull`s.
|
||||
|
||||
It updates to the newest **release tag**, not `main`. `main` can be mid-refactor;
|
||||
a tag is the tested artifact, and CI gates every release. Pre-release tags
|
||||
(`-rc`, `-beta`) are skipped — `--to` them explicitly if you want one.
|
||||
|
||||
After updating, the checkout is pinned to a release tag (detached HEAD). That is
|
||||
what you want for a deployment: plain `git pull` no longer applies, and
|
||||
`update.sh` is the supported path.
|
||||
|
||||
### Why there is no auto-update
|
||||
|
||||
**Never put this in cron or a systemd timer.** This patch injects Python into
|
||||
`middlewared` and re-applies itself at *every boot*, so an unattended pull would
|
||||
let any bad upstream commit reach your box with no human in the loop — and
|
||||
detonate on the next reboot. That is not hypothetical: **v0.0.4 shipped exactly
|
||||
such a bug and took all 54 apps on a box down.**
|
||||
|
||||
The manual step *is* the safety gate. If you want convenience, watch the
|
||||
[releases feed](https://github.com/sudolulo/truenas-truecloud-patch/releases);
|
||||
don't automate the pull.
|
||||
|
||||
It updates to the newest **release tag**, not `main` — `main` can be mid-refactor,
|
||||
and a tag is the tested artifact. `--main` exists if you want unreleased code, and
|
||||
says so loudly.
|
||||
|
||||
## Creating a task via CLI
|
||||
|
||||
If the UI still shows only Storj after refreshing (e.g. the JS bundle pattern
|
||||
@@ -460,12 +529,34 @@ tail -20 /mnt/tank/truenas-truecloud-patch/apply.log
|
||||
|
||||
## After a TrueNAS update
|
||||
|
||||
1. Check the log: `cat /mnt/tank/truenas-truecloud-patch/apply.log | tail -30`
|
||||
2. If you see "WARNING: … pattern not found", the UI patch needs updating.
|
||||
[Open an issue](https://github.com/sudolulo/truenas-truecloud-patch/issues)
|
||||
with your TrueNAS version number.
|
||||
3. The backend patch (B2 support + URL fix) is more stable — check that a
|
||||
B2 backup job still completes successfully after any update.
|
||||
A TrueNAS update replaces `/usr/` wholesale, wiping the patch. You do **not** need
|
||||
to reinstall: `patch/apply.sh` runs at every boot and re-applies itself from your
|
||||
clone. But it targets internal APIs with no stability contract, so an update *can*
|
||||
break it — and the failure is quiet by design (middlewared starts fine; the patch
|
||||
just doesn't).
|
||||
|
||||
**Check the log after any TrueNAS update:**
|
||||
|
||||
```bash
|
||||
tail -30 /mnt/tank/truenas-truecloud-patch/apply.log
|
||||
python3 /mnt/tank/truenas-truecloud-patch/patch/create_task.py verify
|
||||
```
|
||||
|
||||
| What you see | What it means |
|
||||
|---|---|
|
||||
| `[OK] providers`, `[OK]`/`[SKIP] nested_snapshots` | Fine. Nothing to do. |
|
||||
| `WARNING: … pattern not found` (UI) | The Angular bundle changed. The UI dropdown reverts to Storj-only, but **backups keep working** — create tasks with `create_task.py` meanwhile, and [open an issue](https://github.com/sudolulo/truenas-truecloud-patch/issues) with your TrueNAS version. |
|
||||
| `[FAIL] providers` | **Your B2/S3 backups will not run.** middlewared is fine, but the credential/URL handling is gone. Open an issue with your version. |
|
||||
| `[FAIL] nested_snapshots` | The stock guard is back, so tasks with `snapshot = true` on a nested dataset will fail validation. Turn the option off on those tasks until it's fixed. |
|
||||
|
||||
"Fail-safe" means *the box stays up* — not that your backups keep running. A
|
||||
`[FAIL] providers` is a broken backup, so check the log rather than assume.
|
||||
|
||||
Then update the patch itself if a newer release fixes it:
|
||||
|
||||
```bash
|
||||
bash /mnt/tank/truenas-truecloud-patch/update.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
|
||||
+9
-3
@@ -18,7 +18,7 @@
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
VERSION="0.4.0"
|
||||
VERSION="0.4.2"
|
||||
|
||||
# The directory containing install.sh is the permanent install location.
|
||||
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
@@ -95,8 +95,14 @@ fi
|
||||
# ── Set permissions ───────────────────────────────────────────────────────────
|
||||
|
||||
echo "Setting permissions ..."
|
||||
chmod +x "$PATCH_DIR/patch/apply.sh" "$PATCH_DIR/patch/create_task.py" \
|
||||
"$PATCH_DIR/recover.sh" "$PATCH_DIR/uninstall.sh" "$PATCH_DIR/update.sh"
|
||||
# Guard each path: under `set -e` a chmod on a missing file aborts the install.
|
||||
# The file set changes between versions, so `update.sh --rollback` to an older
|
||||
# revision must not be killed by a name this version happens to know about.
|
||||
for _exe in patch/apply.sh patch/create_task.py recover.sh uninstall.sh update.sh; do
|
||||
if [ -f "$PATCH_DIR/$_exe" ]; then
|
||||
chmod +x "$PATCH_DIR/$_exe"
|
||||
fi
|
||||
done
|
||||
echo "Done."
|
||||
echo ""
|
||||
|
||||
|
||||
+1
-1
@@ -32,7 +32,7 @@
|
||||
# Derive PATCH_DIR from this script's location (parent of the patch/ directory).
|
||||
PATCH_DIR="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
LOG="$PATCH_DIR/apply.log"
|
||||
VERSION="0.4.0"
|
||||
VERSION="0.4.2"
|
||||
|
||||
# Rotate log at 512 KB to avoid unbounded growth on a system volume.
|
||||
# Keep two prior generations (.1 and .2) so the last three boots are always available.
|
||||
|
||||
@@ -52,7 +52,7 @@ import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
__version__ = "0.4.0"
|
||||
__version__ = "0.4.2"
|
||||
|
||||
_PATCH_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
_STATUS_FILE = os.path.join(_PATCH_DIR, "hook_status.json")
|
||||
|
||||
+1
-1
@@ -17,7 +17,7 @@
|
||||
# bash /mnt/tank/truenas-truecloud-patch/patch/apply.sh
|
||||
# systemctl restart middlewared
|
||||
|
||||
VERSION="0.4.0"
|
||||
VERSION="0.4.2"
|
||||
|
||||
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
|
||||
|
||||
+1
-1
@@ -3,7 +3,7 @@
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
VERSION="0.4.0"
|
||||
VERSION="0.4.2"
|
||||
|
||||
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
_HOOK_COMMENT='TrueCloud provider patch (S3/B2)'
|
||||
|
||||
@@ -19,7 +19,7 @@
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
VERSION="0.4.0"
|
||||
VERSION="0.4.2"
|
||||
|
||||
PATCH_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
_PREV_FILE="$PATCH_DIR/.update_previous"
|
||||
@@ -46,9 +46,59 @@ Updating preserves your nested-snapshot opt-in setting either way.
|
||||
USAGE
|
||||
}
|
||||
|
||||
# An UNTRACKED file that the target tracks makes `git checkout` abort. The dirty-
|
||||
# tree check deliberately ignores untracked files, so this slips past it and the
|
||||
# checkout then dies mid-operation. Not hypothetical: a hand-copied
|
||||
# patch/wait_restart.sh blocked a pull on a real box exactly this way.
|
||||
#
|
||||
# Used by BOTH the update and the rollback path -- rolling back moves the tree too,
|
||||
# and would hit the identical failure.
|
||||
_abort_if_untracked_blockers() {
|
||||
local ref="$1" blocking
|
||||
|
||||
# Set intersection of {untracked, not ignored} and {tracked by the target}. Two
|
||||
# git calls, not one `ls-files --error-unmatch` per file in the target tree.
|
||||
# --exclude-standard is deliberate: git silently overwrites *ignored* files on
|
||||
# checkout, so those are not blockers — only untracked-and-not-ignored ones are.
|
||||
blocking="$(comm -12 \
|
||||
<(git ls-files --others --exclude-standard | sort) \
|
||||
<(git ls-tree -r --name-only "$ref" | sort) \
|
||||
| sed 's/^/ /')"
|
||||
|
||||
[ -n "$blocking" ] || return 0
|
||||
|
||||
echo "ERROR: these untracked files would be overwritten:" >&2
|
||||
printf '%s\n\n' "$blocking" >&2
|
||||
echo " They exist here but git does not track them — most likely hand-copied" >&2
|
||||
echo " or scp'd in. Move or delete them, then re-run." >&2
|
||||
|
||||
# "Delete update.sh, then re-run update.sh" is impossible. If the script itself
|
||||
# is a blocker, it was hand-copied in to bootstrap; the honest answer is to
|
||||
# bootstrap with git instead, which installs it properly.
|
||||
case "$blocking" in
|
||||
*update.sh*)
|
||||
echo "" >&2
|
||||
echo " update.sh itself is untracked here — you copied it in to bootstrap." >&2
|
||||
echo " Do that with git instead, once; it installs update.sh properly:" >&2
|
||||
echo "" >&2
|
||||
echo " rm -f $PATCH_DIR/update.sh" >&2
|
||||
echo " git -C $PATCH_DIR checkout $ref" >&2
|
||||
echo " bash $PATCH_DIR/install.sh" >&2
|
||||
echo "" >&2
|
||||
echo " Every later update is then just: bash update.sh" >&2
|
||||
;;
|
||||
esac
|
||||
exit 1
|
||||
}
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--to) _target="${2:-}"; shift ;;
|
||||
--to)
|
||||
if [ -z "${2:-}" ]; then
|
||||
echo "ERROR: --to needs a tag, branch, or commit." >&2
|
||||
exit 1
|
||||
fi
|
||||
_target="$2"; shift ;;
|
||||
--main) _use_main=1 ;;
|
||||
--check) _check_only=1 ;;
|
||||
--rollback) _rollback=1 ;;
|
||||
@@ -103,8 +153,15 @@ if [ "$_rollback" -eq 1 ]; then
|
||||
exit 1
|
||||
fi
|
||||
_prev="$(cat "$_PREV_FILE")"
|
||||
if ! git rev-parse --verify --quiet "${_prev}^{commit}" >/dev/null; then
|
||||
echo "ERROR: recorded revision '$_prev' is not a valid commit." >&2
|
||||
echo " The history may have been rewritten. Pick a target explicitly:" >&2
|
||||
echo " bash update.sh --to <tag>" >&2
|
||||
exit 1
|
||||
fi
|
||||
_abort_if_untracked_blockers "$_prev"
|
||||
echo "Rolling back to $_prev ..."
|
||||
git checkout -q "$_prev"
|
||||
git checkout -q --detach "$_prev"
|
||||
echo "Reverted. Re-applying ..."
|
||||
echo ""
|
||||
bash "$PATCH_DIR/install.sh"
|
||||
@@ -128,7 +185,13 @@ else
|
||||
# correct while tags are created in ascending version order; it breaks the
|
||||
# moment a hotfix is tagged out of band (a v0.3.6 released after v0.4.0 would
|
||||
# sort as "newest" by date and silently downgrade the box).
|
||||
_target="$(git tag -l 'v*' --sort=-version:refname | head -1)"
|
||||
#
|
||||
# Filter to PLAIN vX.Y.Z: git's version sort ranks `v0.5.0-rc1` ABOVE `v0.5.0`
|
||||
# (verified), so without this a release candidate would be installed as though
|
||||
# it were the newest release. The release workflow deliberately supports
|
||||
# rc/beta/alpha tags, so they will exist.
|
||||
_target="$(git tag -l 'v*' --sort=-version:refname \
|
||||
| grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | head -1)"
|
||||
if [ -z "$_target" ]; then
|
||||
echo "ERROR: no release tags found; use --main to track unreleased code." >&2
|
||||
exit 1
|
||||
@@ -149,6 +212,8 @@ if [ "$_current" = "$_target_sha" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
_abort_if_untracked_blockers "$_target_sha"
|
||||
|
||||
# ── Show what is coming ───────────────────────────────────────────────────────
|
||||
|
||||
echo "Commits you do not have yet:"
|
||||
@@ -217,6 +282,12 @@ echo ""
|
||||
echo "=== Update complete ==="
|
||||
echo " $_current_desc -> $(git describe --tags --always)"
|
||||
echo ""
|
||||
if ! git symbolic-ref -q HEAD >/dev/null; then
|
||||
echo "NOTE: the checkout is now pinned to a release tag (detached HEAD), which is"
|
||||
echo " what you want for a deployment. Plain \`git pull\` will not work here —"
|
||||
echo " use \`bash update.sh\` from now on."
|
||||
echo ""
|
||||
fi
|
||||
echo "If anything looks wrong:"
|
||||
echo " bash $PATCH_DIR/update.sh --rollback # back to $_current_desc"
|
||||
echo " bash $PATCH_DIR/recover.sh # kill switch + restart"
|
||||
|
||||
Reference in New Issue
Block a user